Corpus ID: 59316619

Surrogate Losses for Online Learning of Stepsizes in Stochastic Non-Convex Optimization

@inproceedings{Zhuang2019SurrogateLF,
  title={Surrogate Losses for Online Learning of Stepsizes in Stochastic Non-Convex Optimization},
  author={Zhenxun Zhuang and Ashok Cutkosky and F. Orabona},
  booktitle={ICML},
  year={2019}
}

Zhenxun Zhuang, Ashok Cutkosky, F. Orabona

Published in ICML 2019

Computer Science, Mathematics

Stochastic Gradient Descent (SGD) has played a central role in machine learning. However, it requires a carefully hand-picked stepsize for fast convergence, which is notoriously tedious and time-consuming to tune. Over the last several years, a plethora of adaptive gradient-based algorithms have emerged to ameliorate this problem. They have proved efficient in reducing the labor of tuning in practice, but many of them lack theoretic guarantees even in the convex setting. In this paper, we… CONTINUE READING

View PDF on arXiv

Save to Library

Create Alert

Cite

Launch Research Feed