Gradient Descent Can Take Exponential Time to Escape Saddle Points

@inproceedings{Du2017GradientDC,
  title={Gradient Descent Can Take Exponential Time to Escape Saddle Points},
  author={Simon S. Du and Chi Jin and Jason D. Lee and Michael I. Jordan and Barnab{\'a}s P{\'o}czos and Aarti Singh},
  booktitle={NIPS},
  year={2017}
}
Although gradient descent (GD) almost always escapes saddle points asymptotically [Lee et al., 2016], this paper shows that even with fairly natural random initialization schemes and non-pathological functions, GD can be significantly slowed down by saddle points, taking exponential time to escape. On the other hand, gradient descent with perturbations [Ge et al., 2015, Jin et al., 2017] is not slowed down by saddle points—it can find an approximate local minimizer in polynomial time. This… CONTINUE READING
Highly Cited
This paper has 29 citations. REVIEW CITATIONS

References

Publications referenced by this paper.
Showing 1-10 of 32 references

Introductory Lectures on Convex Optimization: A Basic Course, volume 87

  • Yurii Nesterov
  • Springer Science & Business Media,
  • 2013
Highly Influential
5 Excerpts

Similar Papers

Loading similar papers…