Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function
@article{Leshno1993MultilayerFN,
title={Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function},
author={Moshe Leshno and Vladimir Ya. Lin and Allan Pinkus and Shimon Schocken},
journal={New York University Stern School of Business Research Paper Series},
year={1993}
}1,536 Citations
NEURAL NETWORKS FOR OPTIMAL APPROXIMATION OF SMOOTH
- Mathematics, Computer Science
- 1996
We prove that neural networks with a single hidden layer are capable of providing an optimal order of approximation for functions assumed to possess a given number of derivatives, if the activation…
Neural Networks for Optimal Approximation of Smooth and Analytic Functions
- Mathematics, Computer ScienceNeural Computation
- 1996
We prove that neural networks with a single hidden layer are capable of providing an optimal order of approximation for functions assumed to possess a given number of derivatives, if the activation…
Three-Layer Feedforward Structures Smoothly Approximating Polynomial Functions
- Computer ScienceICANN
- 2010
This paper considers a structure of three-layer feedforward networks that approximate polynomial functions and shows that the obtained feedforward network smoothly approximates the polynometric function.
On smooth activation functions
- Mathematics
- 1997
We had earlier constructed neural networks which are capable of providing optimal approximation rates for smooth target functions. The activation functions evaluated by the principal elements of…
A Single Hidden Layer Feedforward Network with Only One Neuron in the Hidden Layer Can Approximate Any Univariate Function
- Computer ScienceNeural Computation
- 2016
This work constructs algorithmically a smooth, sigmoidal, almost monotone activation function providing approximation to an arbitrary continuous function within any degree of accuracy.
On the Approximation Properties of Neural Networks
- Computer ScienceArXiv
- 2019
This work improves upon existing results in the literature by significantly relaxing the required assumptions on the activation function and by providing a better rate of approximation of a two layer neural network as the number of neurons increases.
Simultaneous approximations of multivariate functions and their derivatives by neural networks with one hidden layer
- Mathematics, Computer ScienceNeurocomputing
- 1996
Approximation to continuous functionals and operators using adaptive higher-order feedforward neural networks
- Computer ScienceIJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339)
- 1999
Universal approximation theorems of AHFNN to continuous functionals and continuous operators are given, and learning algorithms based on the steepest descent rule are derived to tune the free parameters in NAF as well as connection weights between neurons.
Some negative results for single layer and multilayer feedforward neural networks
- Computer Science
- 2018
This work proves a negative result for approximation of functions defined con compact subsets of $\mathbb{R}^d$ with single layer feedforward neural networks with arbitrary activation functions, and claims the existence of learning functions f(x) which are as difficult to approximate with these neural networks as one may want.
References
SHOWING 1-10 OF 53 REFERENCES
Approximation capabilities of multilayer feedforward networks
- Computer ScienceNeural Networks
- 1991
Approximating and learning unknown mappings using multilayer feedforward networks with bounded weights
- Computer Science1990 IJCNN International Joint Conference on Neural Networks
- 1990
It is shown that feedforward networks having bounded weights are not undesirable restricted, but are in fact universal approximators, provided that the hidden-layer activation function belongs to one…
Multilayer feedforward networks are universal approximators
- Computer Science, MathematicsNeural Networks
- 1989
Approximation by superpositions of a sigmoidal function
- Computer ScienceMath. Control. Signals Syst.
- 1989
In this paper we demonstrate that finite linear combinations of compositions of a fixed, univariate function and a set of affine functionals can uniformly approximate any continuous function ofn real…
On the approximate realization of continuous mappings by neural networks
- Computer ScienceNeural Networks
- 1989
There exists a neural network that does not make avoidable mistakes
- Computer ScienceIEEE 1988 International Conference on Neural Networks
- 1988
The authors show that a multiple-input, single-output, single-hidden-layer feedforward network with (known) hardwired connections from input to hidden layer, monotone squashing at the hidden layer…
Theory of the backpropagation neural network
- Computer Science
- 1989
A speculative neurophysiological model illustrating how the backpropagation neural network architecture might plausibly be implemented in the mammalian brain for corticocortical learning between nearby regions of the cerebral cortex is presented.
Approximation by superpositions of a sigmoidal function
- MathematicsMath. Control. Signals Syst.
- 1992
The reduction of multidimensional density to one-dimensional density as in the proof of Lemma 1 had previously been obtained by Dahmen and Micchelli, using the same techniques, in work on ridge regression.