Exact solutions to the nonlinear dynamics of learning in deep linear neural networks

@article{Saxe2013ExactST,
  title={Exact solutions to the nonlinear dynamics of learning in deep linear neural networks},
  author={Andrew M. Saxe and James L. McClelland and Surya Ganguli},
  journal={CoRR},
  year={2013},
  volume={abs/1312.6120}
}
Despite the widespread practical success of deep learning methods, our theoretical understanding of the dynamics of learning in deep neural networks remains quite sparse. We attempt to bridge the gap between the theory and practice of deep learning by systematically analyzing learning dynamics for the restricted case of deep linear neural networks. Despite the linearity of their input-output map, such networks have nonlinear gradient descent dynamics on weights that change with the addition of… CONTINUE READING
Highly Influential
This paper has highly influenced 31 other papers. REVIEW HIGHLY INFLUENTIAL CITATIONS
Highly Cited
This paper has 570 citations. REVIEW CITATIONS

Citations

Publications citing this paper.
Showing 1-10 of 367 extracted citations

571 Citations

010020020142015201620172018
Citations per Year
Semantic Scholar estimates that this publication has 571 citations based on the available data.

See our FAQ for additional information.

Similar Papers

Loading similar papers…