Corpus ID: 145050106

Approximation spaces of deep neural networks

@article{Gribonval2019ApproximationSO,
  title={Approximation spaces of deep neural networks},
  author={R. Gribonval and G. Kutyniok and M. Nielsen and Felix Voigtl{\"a}nder},
  journal={ArXiv},
  year={2019},
  volume={abs/1905.01208}
}
We study the expressivity of deep neural networks. Measuring a network's complexity by its number of connections or by its number of neurons, we consider the class of functions for which the error of best approximation with networks of a given complexity decays at a certain rate when increasing the complexity budget. Using results from classical approximation theory, we show that this class can be endowed with a (quasi)-norm that makes it a linear function space, called approximation space. We… Expand
31 Citations
Approximation with Tensor Networks. Part III: Multivariate Approximation
  • PDF
How degenerate is the parametrization of neural networks with the ReLU activation function?
  • 11
  • PDF
Approximation of Smoothness Classes by Deep ReLU Networks
  • 3
  • Highly Influenced
  • PDF
Quantitative Rates and Fundamental Obstructions to Non-Euclidean Universal Approximation with Deep Narrow Feed-Forward Networks
  • Highly Influenced
  • PDF
Proof of the Theory-to-Practice Gap in Deep Learning via Sampling Complexity bounds for Neural Network Approximation Spaces
  • 1
  • Highly Influenced
  • PDF
Neural Network Approximation
  • 5
  • PDF
Deep Network with Approximation Error Being Reciprocal of Width to Power of Square Root of Depth
  • 8
  • PDF
Stochastic Feedforward Neural Networks: Universal Approximation
  • PDF
Learning Activation Functions in Deep (Spline) Neural Networks
  • 1
  • PDF
Overcoming The Limitations of Neural Networks in Composite-Pattern Learning with Architopes
  • PDF
...
1
2
3
4
...

References

SHOWING 1-10 OF 74 REFERENCES
Optimal approximation of piecewise smooth functions using deep ReLU neural networks
  • 184
  • PDF
Optimal Approximation with Sparsely Connected Deep Neural Networks
  • 140
  • PDF
Provable approximation properties for deep neural networks
  • 130
  • PDF
Optimal approximation of continuous functions by very deep ReLU networks
  • 101
  • Highly Influential
  • PDF
On the Expressive Power of Deep Learning: A Tensor Analysis
  • 259
Neural Networks for Optimal Approximation of Smooth and Analytic Functions
  • H. Mhaskar
  • Mathematics, Computer Science
  • Neural Computation
  • 1996
  • 262
  • PDF
Deep vs. shallow networks : An approximation theory perspective
  • 222
  • PDF
Error bounds for approximations with deep ReLU networks
  • 391
  • PDF
Convolutional Rectifier Networks as Generalized Tensor Decompositions
  • 119
  • PDF
Nearly-tight VC-dimension and Pseudodimension Bounds for Piecewise Linear Neural Networks
  • 123
  • PDF
...
1
2
3
4
5
...