Stochastic Dual Coordinate Ascent with Adaptive Probabilities

Abstract

This paper introduces AdaSDCA: an adaptive variant of stochastic dual coordinate ascent (SDCA) for solving the regularized empirical risk minimization problems. Our modification consists in allowing the method adaptively change the probability distribution over the dual variables throughout the iterative process. AdaSDCA achieves provably better complexity bound than SDCA with the best fixed probability distribution, known as importance sampling. However, it is of a theoretical character as it is expensive to implement. We also propose AdaSDCA+: a practical variant which in our experiments outperforms existing non-adaptive methods.

View Slides

Extracted Key Phrases

35 Figures and Tables

01020201520162017
Citations per Year

Citation Velocity: 13

Averaging 13 citations per year over the last 3 years.

Learn more about how we calculate this metric in our FAQ.

Cite this paper

@inproceedings{Csiba2015StochasticDC, title={Stochastic Dual Coordinate Ascent with Adaptive Probabilities}, author={Dominik Csiba and Zheng Qu and Peter Richt{\'a}rik}, booktitle={ICML}, year={2015} }