Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes

@article{Baldassi2016UnreasonableEO,
  title={Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes},
  author={Carlo Baldassi and C. Borgs and J. Chayes and A. Ingrosso and Carlo Lucibello and Luca Saglietti and R. Zecchina},
  journal={Proceedings of the National Academy of Sciences},
  year={2016},
  volume={113},
  pages={E7655 - E7662}
}
  • Carlo Baldassi, C. Borgs, +4 authors R. Zecchina
  • Published 2016
  • Computer Science, Medicine, Mathematics, Physics
  • Proceedings of the National Academy of Sciences
  • Significance Artificial neural networks are some of the most widely used tools in data science. Learning is, in principle, a hard problem in these systems, but in practice heuristic algorithms often find solutions with good generalization properties. We propose an explanation of this good performance in terms of a nonequilibrium statistical physics framework: We show that there are regions of the optimization landscape that are both robust and accessible and that their existence is crucial to… CONTINUE READING
    Mean-field inference methods for neural networks
    2
    Shaping the learning landscape in neural networks around wide flat minima
    6
    Reinforced backpropagation for deep neural network learning
    Statistical physics of neural systems
    A Picture of the Energy Landscape of Deep Neural Networks
    Deep learning systems as complex networks
    4

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 56 REFERENCES
    On optimization methods for deep learning
    730
    Distributed stochastic optimization for deep learning (thesis)
    2
    Deep learning with Elastic Averaging SGD
    345
    Bounds on the learning capacity of some multi-layer networks
    31