## 187 Citations

Convergence of Deep Convolutional Neural Networks

- Computer ScienceArXiv
- 2021

It turns out the convergence of deep neural networks as the depth of the networks tends to infinity reduces to convergence of infinite products of matrices with increasing sizes, which has not been considered in the literature.

Optimal Learning Rates of Deep Convolutional Neural Networks: Additive Ridge Functions

- Computer ScienceArXiv
- 2022

It is shown that, for additive ridge functions, convolutional neural networks followed by one fully connected layer with ReLU activation functions can reach optimal mini-max rates (up to a log factor) and the convergence rates are dimension independent.

Theory of Deep Convolutional Neural Networks II: Spherical Analysis

- Computer ScienceNeural Networks
- 2020

Convergence of Deep Neural Networks with General Activation Functions and Pooling

- Computer ScienceArXiv
- 2022

This work studies the convergence of deep neural networks as the depth tends to inﬁnity for two other important activation functions: the leaky ReLU and the sigmoid function, and establishes a weaker suﬃcient condition for uniform convergence ofDeep neural networks.

Butterfly-Net: Optimal Function Representation Based on Convolutional Neural Networks

- Computer ScienceCommunications in Computational Physics
- 2020

Butterfly-net, a low-complexity CNN with structured and sparse across-channel connections, which aims at an optimal hierarchical function representation of the input signal, outperforms the hard-coded Butterfly-net and achieves similar accuracy as the trained CNN but with much less parameters.

Universal Consistency of Deep Convolutional Neural Networks

- Computer ScienceIEEE Transactions on Information Theory
- 2022

It is proved that implementing empirical risk minimization on DCNNs with expansive convolution (with zero-padding) is strongly universally consistent.

Theory of Deep Convolutional Neural Networks III: Approximating Radial Functions

- Computer ScienceNeural Networks
- 2021

Convergence Analysis of Deep Residual Networks

- Computer ScienceArXiv
- 2022

A matrix-vector description of general deep neural networks with shortcut connections is given and an explicit expression for the networks is formulated by using the notions of activation domains and activation matrices to characterize the convergence of deep Residual Networks.

A Sparse Coding Interpretation of Neural Networks and Theoretical Implications

- Computer ScienceArXiv
- 2021

A sparse coding interpretation of neural networks that have ReLU activation and of convolutional neural networks in particular is proposed and potentially more robust forward transformations are motivated by maintaining sparse priors in convolutionAL neural networks as well performing a stronger nonlinear transformation.

## References

SHOWING 1-10 OF 33 REFERENCES

Deep distributed convolutional neural networks: Universality

- Computer ScienceAnalysis and Applications
- 2018

It is shown that these deep neural networks have the same order of computational complexity as the deep convolutional neural networks, and it is proved their universality of approximation.

ImageNet classification with deep convolutional neural networks

- Computer ScienceCommun. ACM
- 2012

A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.

Deep vs. shallow networks : An approximation theory perspective

- Computer ScienceArXiv
- 2016

A new definition of relative dimension is proposed to encapsulate different notions of sparsity of a function class that can possibly be exploited by deep networks but not by shallow ones to drastically reduce the complexity required for approximation and learning.

Optimal Approximation with Sparsely Connected Deep Neural Networks

- Computer ScienceSIAM J. Math. Data Sci.
- 2019

All function classes that are optimally approximated by a general class of representation systems---so-called affine systems---can be approximating by deep neural networks with minimal connectivity and memory requirements, and it is proved that the lower bounds are achievable for a broad family of function classes.

Optimal approximation of piecewise smooth functions using deep ReLU neural networks

- Computer ScienceNeural Networks
- 2018

A Fast Learning Algorithm for Deep Belief Nets

- Computer ScienceNeural Computation
- 2006

A fast, greedy algorithm is derived that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory.

The Power of Depth for Feedforward Neural Networks

- Computer ScienceCOLT
- 2016

It is shown that there is a simple (approximately radial) function on $\reals^d$, expressible by a small 3-layer feedforward neural networks, which cannot be approximated by any 2-layer network, unless its width is exponential in the dimension.

Deep Learning

- Computer ScienceNature
- 2015

Deep learning is making major advances in solving problems that have resisted the best attempts of the artificial intelligence community for many years, and will have many more successes in the near future because it requires very little engineering by hand and can easily take advantage of increases in the amount of available computation and data.

Learning with Hierarchical Gaussian Kernels

- Computer ScienceArXiv
- 2016

It is shown that Gaussian kernels are universal and that SVMs using these kernels are universally consistent, and a parameter optimization method for the kernel parameters is described that is empirically compared to SVMs, random forests, a multiple kernel learning approach, and to some deep neural networks.