## A Comprehensive Linear Speedup Analysis for Asynchronous Stochastic Parallel Optimization from Zeroth-Order to First-Order

- Xiangru Lian, Huan Zhang, Cho-Jui Hsieh, Yijun Huang, Ji Liu
- NIPS
- 2016

Highly Influenced

12 Excerpts

- Published 2014 in Math. Program.

In this paper we develop a randomized block-coordinate descent method for minimizing the sum of a smooth and a simple nonsmooth block-separable convex function and prove that it obtains an ε-accurate solution with probability at least 1− ρ in at most O((n/ε) log(1/ρ)) iterations, where n is the number of blocks. This extends recent results of Nesterov [Efficiency of coordinate descent methods on huge-scale optimization problems, SIAM J Optimization 22(2), pp. 341–362, 2012], which cover the smooth case, to composite minimization, while at the same time improving the complexity by the factor of 4 and removing ε from the logarithmic term. More importantly, in contrast with the aforementioned work in which the author achieves the results by applying the method to a regularized version of the objective function with an unknown scaling factor, we show that this is not necessary, thus achieving first true iteration complexity bounds. For strongly convex functions the method converges linearly. In the smooth case we also allow for arbitrary probability vectors and non-Euclidean norms. Finally, we demonstrate numerically that the algorithm is able to solve huge-scale l1-regularized least squares with a billion variables.

Citations per Year

Semantic Scholar estimates that this publication has **355** citations based on the available data.

See our **FAQ** for additional information.

Showing 1-10 of 224 extracted citations

Highly Influenced

12 Excerpts

Highly Influenced

20 Excerpts

Highly Influenced

8 Excerpts

Highly Influenced

9 Excerpts

Highly Influenced

9 Excerpts

Highly Influenced

4 Excerpts

Highly Influenced

10 Excerpts

Highly Influenced

6 Excerpts

Highly Influenced

5 Excerpts

Highly Influenced

4 Excerpts

Showing 1-10 of 29 references

Highly Influential

16 Excerpts

Highly Influential

4 Excerpts

Highly Influential

6 Excerpts

Highly Influential

13 Excerpts

Highly Influential

4 Excerpts

Highly Influential

6 Excerpts

Highly Influential

4 Excerpts

Highly Influential

8 Excerpts

Highly Influential

4 Excerpts

Highly Influential

4 Excerpts

@article{Richtrik2014IterationCO,
title={Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function},
author={Peter Richt{\'a}rik and Martin Tak{\'a}c},
journal={Math. Program.},
year={2014},
volume={144},
pages={1-38}
}