A survey of Algorithms and Analysis for Adaptive Online Learning

  title={A survey of Algorithms and Analysis for Adaptive Online Learning},
  author={H. Brendan McMahan},
  journal={Journal of Machine Learning Research},
We present tools for the analysis of Follow-The-Regularized-Leader (FTRL), Dual Averaging, and Mirror Descent algorithms when the regularizer (equivalently, proxfunction or learning rate schedule) is chosen adaptively based on the data. Adaptivity can be used to prove regret bounds that hold on every round, and also allows for data-dependent regret bounds as in AdaGrad-style algorithms (e.g., Online Gradient Descent with adaptive per-coordinate learning rates). We present results from a large… CONTINUE READING
Highly Cited
This paper has 18 citations. REVIEW CITATIONS

6 Figures & Tables



Citations per Year

Citation Velocity: 7

Averaging 7 citations per year over the last 3 years.

Learn more about how we calculate this metric in our FAQ.