Corpus ID: 14366590

Beneath the valley of the noncommutative arithmetic-geometric mean inequality: conjectures, case-studies, and consequences

@article{Recht2012BeneathTV,
  title={Beneath the valley of the noncommutative arithmetic-geometric mean inequality: conjectures, case-studies, and consequences},
  author={B. Recht and C. R{\'e}},
  journal={arXiv: Optimization and Control},
  year={2012}
}
  • B. Recht, C. Ré
  • Published 2012
  • Mathematics
  • arXiv: Optimization and Control
Randomized algorithms that base iteration-level decisions on samples from some pool are ubiquitous in machine learning and optimization. Examples include stochastic gradient descent and randomized coordinate descent. This paper makes progress at theoretically evaluating the difference in performance between sampling with- and without-replacement in such algorithms. Focusing on least means squares optimization, we formulate a noncommutative arithmetic-geometric mean inequality that would prove… Expand
45 Citations

Figures from this paper

How Good is SGD with Random Shuffling?
  • 17
  • PDF
Dimension-Free Iteration Complexity of Finite Sum Optimization Problems
  • 24
  • PDF
Noncommutative versions of the arithmetic-geometric mean inequality
  • 3
  • Highly Influenced
  • PDF
Mini-Workshop: Mathematical Physics meets Sparse Recovery
  • PDF
Random Shuffling Beats SGD after Finite Epochs
  • 33
  • PDF
Permutation-Based SGD: Is Random Optimal?
  • PDF
Without-Replacement Sampling for Stochastic Gradient Methods: Convergence Results and Application to Distributed Optimization
  • O. Shamir
  • Computer Science, Mathematics
  • ArXiv
  • 2016
  • 24
  • PDF
Without-Replacement Sampling for Stochastic Gradient Methods
  • O. Shamir
  • Computer Science, Mathematics
  • NIPS
  • 2016
  • 71
  • Highly Influenced
  • PDF
On the Efficiency of Random Permutation for ADMM and Coordinate Descent
  • 10
  • PDF
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 42 REFERENCES
Randomized Methods for Linear Constraints: Convergence Rates and Conditioning
  • 196
  • PDF
User-Friendly Tail Bounds for Sums of Random Matrices
  • J. Tropp
  • Mathematics, Computer Science
  • Found. Comput. Math.
  • 2012
  • 1,249
  • PDF
Acceleration of randomized Kaczmarz method via the Johnson–Lindenstrauss Lemma
  • 91
  • PDF
A Randomized Kaczmarz Algorithm with Exponential Convergence
  • 531
  • Highly Influential
  • PDF
Robust Stochastic Approximation Approach to Stochastic Programming
  • 1,572
  • Highly Influential
  • PDF
Parallel stochastic gradient algorithms for large-scale matrix completion
  • B. Recht, C. Ré
  • Mathematics, Computer Science
  • Math. Program. Comput.
  • 2013
  • 368
  • PDF
Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems
  • Y. Nesterov
  • Mathematics, Computer Science
  • SIAM J. Optim.
  • 2012
  • 1,078
  • PDF
Curiously Fast Convergence of some Stochastic Gradient Descent Algorithms
  • 54
  • PDF
An Incremental Gradient(-Projection) Method with Momentum Term and Adaptive Stepsize Rule
  • P. Tseng
  • Mathematics, Computer Science
  • SIAM J. Optim.
  • 1998
  • 143
Convergence Rate of Incremental Subgradient Algorithms
  • 250
...
1
2
3
4
5
...