Parallel coordinate descent methods for big data optimization

@article{Richtrik2012ParallelCD,
  title={Parallel coordinate descent methods for big data optimization},
  author={Peter Richt{\'a}rik and Martin Tak{\'a}c},
  journal={Mathematical Programming},
  year={2012},
  volume={156},
  pages={433-484}
}
In this work we show that randomized (block) coordinate descent methods can be accelerated by parallelization when applied to the problem of minimizing the sum of a partially separable smooth convex function and a simple separable convex function. The theoretical speedup, as compared to the serial method, and referring to the number of iterations needed to approximately solve the problem with high probability, is a simple expression depending on the number of parallel processors and a natural… CONTINUE READING

Citations

Publications citing this paper.
SHOWING 1-10 OF 288 CITATIONS

L-SVRG and L-Katyusha with Arbitrary Sampling

VIEW 4 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

Data sampling strategies in stochastic algorithms for empirical risk minimization

VIEW 16 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

Parallel Stochastic Newton Method

VIEW 12 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

A flexible coordinate descent method

  • Comp. Opt. and Appl.
  • 2015
VIEW 17 EXCERPTS
CITES BACKGROUND, RESULTS & METHODS
HIGHLY INFLUENCED

Dual Free Adaptive Minibatch SDCA for Empirical Risk Minimization

  • Front. Appl. Math. Stat.
  • 2015
VIEW 7 EXCERPTS
CITES BACKGROUND & METHODS

Fast distributed coordinate descent for non-strongly convex losses

  • 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP)
  • 2014
VIEW 7 EXCERPTS
CITES METHODS & BACKGROUND

Separable Approximations and Decomposition Methods for the Augmented Lagrangian

  • Optimization Methods and Software
  • 2013
VIEW 28 EXCERPTS
CITES METHODS & BACKGROUND

FILTER CITATIONS BY YEAR

2012
2019

CITATION STATISTICS

  • 51 Highly Influenced Citations

  • Averaged 41 Citations per year from 2017 through 2019

References

Publications referenced by this paper.
SHOWING 1-10 OF 17 REFERENCES

Taká£. E cient serial and parallel coordinate descent method for huge-scale truss topology design

Peter Richtárik, Martin
  • In Operations Research Proceedings,
  • 2012
VIEW 3 EXCERPTS
HIGHLY INFLUENTIAL

Clipici. E cient parallel coordinate descent algorithm for convex optimization problems with separable constraints: application to distributed MPC

Ion Necoara, Dragos
  • Technical report, Politehnica University of Bucharest,
  • 2012
VIEW 3 EXCERPTS

E ciency of randomized coordinate descent methods on optimization problems with linearly coupled constraints

Ion Necoara, Yurii Nesterov, Francois Glineur
  • Technical report, Politehnica University of Bucharest,
  • 2012
VIEW 1 EXCERPT

Inexact block coordinate descent method: complexity and preconditioning

Peter Richtárik, Jacek Gondzio, Rachael Tappenden
  • Technical report, University of Edinburgh,
  • 2012
VIEW 2 EXCERPTS