Accelerated Coordinate Descent with Arbitrary Sampling and Best Rates for Minibatches

@inproceedings{Hanzely2018AcceleratedCD,
title={Accelerated Coordinate Descent with Arbitrary Sampling and Best Rates for Minibatches},
author={Filip Hanzely and Peter Richt{\'a}rik},
booktitle={AISTATS},
year={2018}
}
• Published in AISTATS 2018
Accelerated coordinate descent is a widely popular optimization algorithm due to its efficiency on large-dimensional problems. It achieves state-of-the-art complexity on an important class of empirical risk minimization problems. In this paper we design and analyze an accelerated coordinate descent (\texttt{ACD}) method which in each iteration updates a random subset of coordinates according to an arbitrary but fixed probability law, which is a parameter of the method. While mini-batch… CONTINUE READING
1

Citations

Publications citing this paper.
SHOWING 1-9 OF 9 CITATIONS

One Method to Rule Them All: Variance Reduction for Data, Parameters and Many New Methods

• ArXiv
• 2019
VIEW 4 EXCERPTS
CITES BACKGROUND & METHODS

Nonconvex Variance Reduced Optimization with Arbitrary Sampling

VIEW 4 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

• ArXiv
• 2019
VIEW 1 EXCERPT
CITES METHODS

• ArXiv
• 2019
VIEW 2 EXCERPTS
CITES BACKGROUND

L-SVRG and L-Katyusha with Arbitrary Sampling

• 2019
VIEW 2 EXCERPTS
CITES METHODS & BACKGROUND

• ArXiv
• 2019
VIEW 2 EXCERPTS
CITES BACKGROUND

SAGA with Arbitrary Sampling

• ICML
• 2019
VIEW 2 EXCERPTS
CITES BACKGROUND & METHODS

SGD: General Analysis and Improved Rates

• ICML 2019
• 2019
VIEW 2 EXCERPTS
CITES BACKGROUND & METHODS

SEGA: Variance Reduction via Gradient Sketching

• NeurIPS
• 2018
VIEW 3 EXCERPTS
CITES METHODS & BACKGROUND

References

Publications referenced by this paper.
SHOWING 1-10 OF 34 REFERENCES

Even Faster Accelerated Coordinate Descent Using Non-Uniform Sampling

• ICML
• 2015
VIEW 10 EXCERPTS
HIGHLY INFLUENTIAL

Accelerated, Parallel and Proximal Coordinate Descent

• SIAM Journal on Optimization
• 2013
VIEW 8 EXCERPTS

Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems

• SIAM Journal on Optimization
• 2012
VIEW 6 EXCERPTS
HIGHLY INFLUENTIAL

A method for solving the convex programming problem with convergence rate O(1/k^2)

VIEW 11 EXCERPTS
HIGHLY INFLUENTIAL

Distributed Coordinate Descent Method for Learning with Big Data

VIEW 14 EXCERPTS
HIGHLY INFLUENTIAL

Coordinate descent with arbitrary sampling I: algorithms and complexity

• Optimization Methods and Software
• 2014
VIEW 10 EXCERPTS

Coordinate descent with arbitrary sampling II: expected separable overapproximation

• Optimization Methods and Software
• 2014
VIEW 7 EXCERPTS

Fast distributed coordinate descent for non-strongly convex losses

• 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP)
• 2014
VIEW 4 EXCERPTS

On optimal probabilities in stochastic coordinate descent methods

• Optimization Letters
• 2013
VIEW 11 EXCERPTS

Parallel coordinate descent methods for big data optimization

• Math. Program.
• 2012
VIEW 11 EXCERPTS