Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy?
@article{Giryes2016DeepNN, title={Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy?}, author={Raja Giryes and G. Sapiro and A. Bronstein}, journal={IEEE Transactions on Signal Processing}, year={2016}, volume={64}, pages={3444-3457} }
Three important properties of a classification machinery are i) the system preserves the core information of the input data; ii) the training examples convey information about unseen data; and iii) the system is able to treat differently points from different classes. In this paper, we show that these fundamental properties are satisfied by the architecture of deep neural networks. We formally prove that these networks with random Gaussian weights perform a distance-preserving embedding of the… CONTINUE READING
Paper Mentions
132 Citations
Comments on “Deep Neural Networks With Random Gaussian Weights: A Universal Classification Strategy?”
- Computer Science, Mathematics
- IEEE Transactions on Signal Processing
- 2020
- 1
- Highly Influenced
- PDF
Building a Regular Decision Boundary with Deep Networks
- Computer Science
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- 2017
- 14
- PDF
Generalization error of deep neural networks: Role of classification margin and data structure
- Mathematics
- 2017 International Conference on Sampling Theory and Applications (SampTA)
- 2017
- 5
Deep Stochastic Configuration Networks: Universal Approximation and Learning Representation
- Computer Science, Mathematics
- ArXiv
- 2017
- 3
- PDF
Deep Stochastic Configuration Networks with Universal Approximation Property
- Computer Science, Mathematics
- 2018 International Joint Conference on Neural Networks (IJCNN)
- 2018
- 21
- PDF
Robust Large Margin Deep Neural Networks
- Computer Science, Mathematics
- IEEE Transactions on Signal Processing
- 2017
- 166
- PDF
Entropy and mutual information in models of deep neural networks
- Computer Science, Mathematics
- NeurIPS
- 2018
- 73
- PDF
Intriguing Properties of Randomly Weighted Networks: Generalizing While Learning Next to Nothing
- Computer Science
- 2019 16th Conference on Computer and Robot Vision (CRV)
- 2019
- 23
- PDF
References
SHOWING 1-10 OF 80 REFERENCES
Representation Learning: A Review and New Perspectives
- Computer Science, Mathematics
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- 2013
- 6,761
- PDF
Geometry-Aware Deep Transform
- Computer Science
- 2015 IEEE International Conference on Computer Vision (ICCV)
- 2015
- 8
- PDF
ImageNet classification with deep convolutional neural networks
- Computer Science
- Commun. ACM
- 2012
- 58,440
- PDF
Unsupervised learning of invariant representations
- Mathematics, Computer Science
- Theor. Comput. Sci.
- 2016
- 68
- PDF
How to Scale Up Kernel Methods to Be As Good As Deep Neural Nets
- Computer Science, Mathematics
- ArXiv
- 2014
- 58
- PDF