Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy?

@article{Giryes2016DeepNN,
  title={Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy?},
  author={Raja Giryes and G. Sapiro and A. Bronstein},
  journal={IEEE Transactions on Signal Processing},
  year={2016},
  volume={64},
  pages={3444-3457}
}
  • Raja Giryes, G. Sapiro, A. Bronstein
  • Published 2016
  • Computer Science, Mathematics
  • IEEE Transactions on Signal Processing
  • Three important properties of a classification machinery are i) the system preserves the core information of the input data; ii) the training examples convey information about unseen data; and iii) the system is able to treat differently points from different classes. In this paper, we show that these fundamental properties are satisfied by the architecture of deep neural networks. We formally prove that these networks with random Gaussian weights perform a distance-preserving embedding of the… CONTINUE READING
    126 Citations

    Figures and Topics from this paper.

    Comments on “Deep Neural Networks With Random Gaussian Weights: A Universal Classification Strategy?”
    • 1
    • Highly Influenced
    • PDF
    Building a Regular Decision Boundary with Deep Networks
    • Edouard Oyallon
    • Computer Science
    • 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
    • 2017
    • 13
    • PDF
    Generalization error of deep neural networks: Role of classification margin and data structure
    • 5
    Deep Stochastic Configuration Networks with Universal Approximation Property
    • D. Wang, Ming Li
    • Computer Science, Mathematics
    • 2018 International Joint Conference on Neural Networks (IJCNN)
    • 2018
    • 20
    • PDF
    Robust Large Margin Deep Neural Networks
    • 154
    • PDF
    Entropy and mutual information in models of deep neural networks
    • 67
    • PDF
    Intriguing Properties of Randomly Weighted Networks: Generalizing While Learning Next to Nothing
    • 20
    • PDF
    Net-Trim: Convex Pruning of Deep Neural Networks with Performance Guarantee
    • 79
    • PDF
    Deep Randomized Neural Networks
    • 7
    • Highly Influenced
    • PDF

    References

    SHOWING 1-10 OF 79 REFERENCES
    On Random Weights and Unsupervised Feature Learning
    • 338
    • PDF
    Convolutional Kernel Networks
    • 266
    • PDF
    Provable Bounds for Learning Some Deep Representations
    • 252
    • PDF
    Representation Learning: A Review and New Perspectives
    • 6,509
    • PDF
    Geometry-Aware Deep Transform
    • 8
    • PDF
    ImageNet classification with deep convolutional neural networks
    • 55,155
    • PDF
    Unsupervised learning of invariant representations
    • 66
    • PDF
    The Loss Surfaces of Multilayer Networks
    • 745
    • PDF
    On the Number of Linear Regions of Deep Neural Networks
    • 661
    • PDF
    How to Scale Up Kernel Methods to Be As Good As Deep Neural Nets
    • 54
    • PDF