A Lyapunov Analysis of Momentum Methods in Optimization

@article{Wilson2016ALA,
  title={A Lyapunov Analysis of Momentum Methods in Optimization},
  author={Ashia C. Wilson and Benjamin Recht and Michael I. Jordan},
  journal={CoRR},
  year={2016},
  volume={abs/1611.02635}
}
Momentum methods play a central role in optimization. Several momentum methods are provably optimal, and all use a technique called estimate sequences to analyze their convergence properties. The technique of estimate sequences has long been considered difficult to understand, leading many researchers to generate alternative, “more intuitive” methods and analyses. In this paper we show there is an equivalence between the technique of estimate sequences and a family of Lyapunov functions in both… CONTINUE READING
Highly Cited
This paper has 56 citations. REVIEW CITATIONS

From This Paper

Figures, tables, and topics from this paper.

Citations

Publications citing this paper.
Showing 1-10 of 34 citations

57 Citations

020402016201720182019
Citations per Year
Semantic Scholar estimates that this publication has 57 citations based on the available data.

See our FAQ for additional information.

References

Publications referenced by this paper.
Showing 1-10 of 26 references

Introductory Lectures on Convex Optimization: A Basic Course

  • Yurii Nesterov
  • Applied Optimization. Kluwer,
  • 2004
Highly Influential
17 Excerpts

Some methods of speeding up the convergence of iteration methods

  • Boris T. Polyak
  • USSR Computational Mathematics and Mathematical…
  • 1964
Highly Influential
11 Excerpts