The Loss Surfaces of Multilayer Networks

@inproceedings{Choromanska2015TheLS,
  title={The Loss Surfaces of Multilayer Networks},
  author={Anna Choromanska and Mikael Henaff and Micha{\"e}l Mathieu and G{\'e}rard Ben Arous and Yann LeCun},
  booktitle={AISTATS},
  year={2015}
}
We study the connection between the highly non-convex loss function of a simple model of the fully-connected feed-forward neural network and the Hamiltonian of the spherical spin-glass model under the assumptions of: i) variable independence, ii) redundancy in network parametrization, and iii) uniformity. These assumptions enable us to explain the complexity of the fully decoupled neural network through the prism of the results from random matrix theory. We show that for large-size decoupled… CONTINUE READING
Highly Influential
This paper has highly influenced 25 other papers. REVIEW HIGHLY INFLUENTIAL CITATIONS
Highly Cited
This paper has 424 citations. REVIEW CITATIONS

Citations

Publications citing this paper.

424 Citations

010020020152016201720182019
Citations per Year
Semantic Scholar estimates that this publication has 424 citations based on the available data.

See our FAQ for additional information.

References

Publications referenced by this paper.

Similar Papers

Loading similar papers…