Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems

@article{Nesterov2012EfficiencyOC,
  title={Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems},
  author={Yurii Nesterov},
  journal={SIAM Journal on Optimization},
  year={2012},
  volume={22},
  pages={341-362}
}
In this paper we propose new methods for solving huge-scale optimization problems. For problems of this size, even the simplest full-dimensional vector operations are very expensive. Hence, we propose to apply an optimization technique based on random partial update of decision variables. For these methods, we prove the global estimates for the rate of convergence. Surprisingly enough, for certain classes of objective functions, our results are better than the standard worst-case bounds for… CONTINUE READING

Citations

Publications citing this paper.
SHOWING 1-10 OF 644 CITATIONS

Explorer On the Complexity of Parallel Coordinate Descent

Rachael Tappendena, Martin Takáčb, Peter Richtárikc
  • 2017
VIEW 8 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

On the optimal order of worst case complexity of direct search

  • Optimization Letters
  • 2016
VIEW 4 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

Random permutations fix a worst case for cyclic coordinate descent

VIEW 6 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

Guaranteed Matrix Completion via Non-Convex Factorization

  • IEEE Transactions on Information Theory
  • 2015
VIEW 7 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

An Accelerated Proximal Coordinate Gradient Method

VIEW 6 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

Randomized coordinate descent methods for big data optimization

VIEW 17 EXCERPTS
CITES BACKGROUND, METHODS & RESULTS
HIGHLY INFLUENCED

FILTER CITATIONS BY YEAR

2011
2019

CITATION STATISTICS

  • 132 Highly Influenced Citations

  • Averaged 92 Citations per year from 2017 through 2019

References

Publications referenced by this paper.
SHOWING 1-7 OF 7 REFERENCES

Convergence of a block coordinate descent methods for nondifferentiable minimization. JOTA

P. Tseng
  • 2001

A method for unconstrained convex minimization problem with the rate of convergence O

Yu. Nesterov
  • Doklady AN SSSR (translated as Soviet Math. Docl.),
  • 1983
VIEW 2 EXCERPTS