Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function

@inproceedings{Leshno1993MultilayerFN,
  title={Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function},
  author={M. Leshno and Vladimir Ya. Lin and A. Pinkus and S. Schocken},
  booktitle={Neural Networks},
  year={1993}
}
  • M. Leshno, Vladimir Ya. Lin, +1 author S. Schocken
  • Published in Neural Networks 1993
  • Mathematics, Computer Science
  • Several researchers characterized the activation functions under which multilayer feedforwardnetworks can act as universal approximators. We show that all the characterizationsthat were reported thus far in the literature ark special cases of the following general result:a standard multilayer feedforward network can approximate any continuous functionto any degree of accuracy if and only if the network's activation functions are not polynomial.We also emphasize the important role of the… CONTINUE READING
    1,304 Citations
    NEURAL NETWORKS FOR OPTIMAL APPROXIMATION OF SMOOTH
    • 7
    • Highly Influenced
    Neural Networks for Optimal Approximation of Smooth and Analytic Functions
    • H. Mhaskar
    • Mathematics, Computer Science
    • Neural Computation
    • 1996
    • 255
    • Highly Influenced
    • PDF
    Three-Layer Feedforward Structures Smoothly Approximating Polynomial Functions
    Some new results on neural network approximation
    • K. Hornik
    • Mathematics, Computer Science
    • Neural Networks
    • 1993
    • 465
    Approximation rates for neural networks with general activation functions
    • 15
    • PDF
    On smooth activation functions
    • 6
    On the Approximation Properties of Neural Networks
    • 8
    • Highly Influenced

    References

    SHOWING 1-10 OF 53 REFERENCES
    Approximation capabilities of multilayer feedforward networks
    • K. Hornik
    • Mathematics, Computer Science
    • Neural Networks
    • 1991
    • 3,872
    • PDF
    Approximating and learning unknown mappings using multilayer feedforward networks with bounded weights
    • M. Stinchcombe, H. White
    • Mathematics, Computer Science
    • 1990 IJCNN International Joint Conference on Neural Networks
    • 1990
    • 117
    Multilayer feedforward networks are universal approximators
    • 15,300
    • PDF
    Approximation by superpositions of a sigmoidal function
    • G. Cybenko
    • Mathematics, Computer Science
    • Math. Control. Signals Syst.
    • 1989
    • 4,049
    • Highly Influential
    • PDF
    On the approximate realization of continuous mappings by neural networks
    • 4,037
    • PDF
    There exists a neural network that does not make avoidable mistakes
    • A. Gallant, H. White
    • Mathematics, Computer Science
    • IEEE 1988 International Conference on Neural Networks
    • 1988
    • 194
    Capabilities of three-layered perceptrons
    • B. Irie, S. Miyake
    • Computer Science
    • IEEE 1988 International Conference on Neural Networks
    • 1988
    • 398
    Theory of the backpropagation neural network
    • 956
    Connectionist Learning Procedures
    • 1,447
    • PDF