Gradient Descent Can Take Exponential Time to Escape Saddle Points

@inproceedings{Du2017GradientDC,
  title={Gradient Descent Can Take Exponential Time to Escape Saddle Points},
  author={Simon S. Du and Chi Jin and Jason D. Lee and Michael I. Jordan and Barnab{\'a}s P{\'o}czos and Aarti Singh},
  booktitle={NIPS},
  year={2017}
}
Although gradient descent (GD) almost always escapes saddle points asymptotically [Lee et al., 2016], this paper shows that even with fairly natural random initialization schemes and non-pathological functions, GD can be significantly slowed down by saddle points, taking exponential time to escape. On the other hand, gradient descent with perturbations [Ge et al., 2015, Jin et al., 2017] is not slowed down by saddle points—it can find an approximate local minimizer in polynomial time. This… CONTINUE READING
Highly Cited
This paper has 26 citations. REVIEW CITATIONS
Recent Discussions
This paper has been referenced on Twitter 46 times over the past 90 days. VIEW TWEETS

Topics

Statistics

0204020172018
Citations per Year

Citation Velocity: 17

Averaging 17 citations per year over the last 2 years.

Learn more about how we calculate this metric in our FAQ.