• Publications
  • Influence
On the Convergence of Adam and Beyond
TLDR
We provide an explicit example of a simple convex optimization setting where Adam does not converge to the optimal solution, and describe the precise problems with the previous analysis of Adam algorithm. Expand
  • 1,021
  • 220
  • PDF
Logarithmic regret algorithms for online convex optimization
TLDR
We propose several algorithms that achieve regret O(log (T) for an arbitrary sequence of strictly convex functions (with bounded first and second derivatives). Expand
  • 804
  • 182
  • PDF
The Multiplicative Weights Update Method: a Meta-Algorithm and Applications
TLDR
This project was supported by David and Lucile Packard Fellowship and NSF grants MSPA-MCS 0528414 and CCR0205594. Expand
  • 698
  • 88
  • PDF
Beyond the regret minimization barrier: an optimal algorithm for stochastic strongly-convex optimization
TLDR
We consider stochastic convex optimization with a strongly convex (but not necessarily smooth) objective. Expand
  • 310
  • 53
  • PDF
Privacy, accuracy, and consistency too: a holistic solution to contingency table release
TLDR
The contingency table is a work horse of official statistics, the format of reported data for the US Census, Bureau of Labor Statistics, and the Internal Revenue Service. Expand
  • 427
  • 42
  • PDF
Projection-free Online Learning
TLDR
We present efficient online learning algorithms that eschew projections in favor of much more efficient linear optimization steps using the Frank-Wolfe technique and show the theoretical improvements to be clearly visible on standard datasets. Expand
  • 153
  • 30
  • PDF
Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits
TLDR
We present a new algorithm for the contextual bandit learning problem, where the learner repeatedly takes one of K actions in response to the observed context, and observes the reward only for that action. Expand
  • 293
  • 28
  • PDF
A combinatorial, primal-dual approach to semidefinite programs
TLDR
We develop a general primal-dual approach to solve SDPs using a generalization of the well-known multiplicative weights update rule to symmetricmatrices. Expand
  • 174
  • 23
  • PDF
Algorithms for portfolio management based on the Newton method
TLDR
We experimentally study on-line investment algorithms first proposed by Agarwal and Hazan and extended by Hazan et al. which achieve almost the same wealth as the best constant-rebalanced portfolio determined in hindsight. Expand
  • 142
  • 23
  • PDF
Adaptive Methods for Nonconvex Optimization
TLDR
Adaptive gradient methods that rely on scaling gradients down by the square root of exponential moving averages of squared gradients, such RMSProp, Adam, Adadelta have found wide application in optimizing the nonconvex problems that arise in deep learning. Expand
  • 94
  • 22
  • PDF