Share This Author
Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization
This paper investigates the optimality of SGD in a stochastic setting, and shows that for smooth problems, the algorithm attains the optimal O(1/T) rate, however, for non-smooth problems the convergence rate with averaging might really be Ω(log(T)/T), and this is not just an artifact of the analysis.
Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis
The present work provides a nonasymptotic analysis in the context of non-convex learning problems, giving finite-time guarantees for SGLD to find approximate minimizers of both empirical and population risks.
Optimization, Learning, and Games with Predictable Sequences
It is proved that a version of Optimistic Mirror Descent can be used by two strongly-uncoupled players in a finite zero-sum matrix game to converge to the minimax equilibrium at the rate of O((log T)/T).
Competing in the Dark: An Efficient Algorithm for Bandit Linear Optimization
This work introduces an efficient algorithm for the problem of online linear optimization in the bandit setting which achieves the optimal O∗( √ T ) regret and presents a novel connection between online learning and interior point methods.
Online Learning with Predictable Sequences
Methods for online linear optimization that take advantage of benign (as opposed to worst-case) sequences and compete with a set of possible predictable processes concurrently with using it to obtain better regret guarantees are presented.
Size-Independent Sample Complexity of Neural Networks
The sample complexity of learning neural networks is studied by providing new bounds on their Rademacher complexity, assuming norm constraints on the parameter matrix of each layer, and under some additional assumptions, these bounds are fully independent of the network size.
Online Optimization : Competing with Dynamic Comparators
- A. Jadbabaie, A. Rakhlin, Shahin Shahrampour, Karthik Sridharan
- Computer ScienceAISTATS
- 25 January 2015
A fully adaptive method is presented that competes with dynamic benchmarks in which regret guarantee scales with regularity of the sequence of cost functions and comparators, and adapts to the smaller complexity measure in the problem environment.
Stochastic Convex Optimization with Bandit Feedback
- Alekh Agarwal, Dean P. Foster, Daniel J. Hsu, S. Kakade, A. Rakhlin
- Computer Science, MathematicsSIAM J. Optim.
- 8 July 2011
This paper addresses the problem of minimizing a convex, Lipschitz function f over a conveX, compact set χ under a stochastic bandit feedback model and demonstrates a generalization of the ellipsoid algorithm that incurs O(poly (d) √T) regret.
Adaptive Online Gradient Descent
An algorithm is provided, Adaptive Online Gradient Descent, which interpolates between the results of Zinkevich for linear functions and of Hazan et al for strongly convex functions, achieving intermediate rates between √T and log T and shows strong optimality of the algorithm.
Optimal Stragies and Minimax Lower Bounds for Online Convex Games
This work analyzes Online Convex Game settings from a minimax perspective, proving minimax strategies and lower bounds in each case and proving that the existing algorithms are essentially optimal.