Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions

@inproceedings{Panageas2017GradientDO,
  title={Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions},
  author={Ioannis Panageas and Georgios Piliouras},
  booktitle={ITCS},
  year={2017}
}
Given a twice continuously differentiable cost function f , we prove that the set of initial conditions so that gradient descent converges to saddle points where ∇2f has at least one strictly negative eigenvalue, has (Lebesgue) measure zero, even for cost functions f with non-isolated critical points, answering an open question in [12]. Moreover, this result extends to forward-invariant convex subspaces, allowing for weak (non-globally Lipschitz) smoothness assumptions. Finally, we produce an… CONTINUE READING
Highly Cited
This paper has 21 citations. REVIEW CITATIONS

From This Paper

Figures, tables, and topics from this paper.

Explore Further: Topics Discussed in This Paper

Citations

Publications citing this paper.
Showing 1-10 of 16 extracted citations

References

Publications referenced by this paper.
Showing 1-10 of 27 references

Similar Papers

Loading similar papers…