Corpus ID: 211011003

Last Iterate is Slower than Averaged Iterate in Smooth Convex-Concave Saddle Point Problems

@article{Golowich2020LastII,
  title={Last Iterate is Slower than Averaged Iterate in Smooth Convex-Concave Saddle Point Problems},
  author={Noah Golowich and S. Pattathil and C. Daskalakis and A. Ozdaglar},
  journal={ArXiv},
  year={2020},
  volume={abs/2002.00057}
}
In this paper we study the smooth convex-concave saddle point problem. Specifically, we analyze the last iterate convergence properties of the Extragradient (EG) algorithm. It is well known that the ergodic (averaged) iterates of EG converge at a rate of $O(1/T)$ (Nemirovski, 2004). In this paper, we show that the last iterate of EG converges at a rate of $O(1/\sqrt{T})$. To the best of our knowledge, this is the first paper to provide a convergence rate guarantee for the last iterate of EG for… Expand
Linear Last-iterate Convergence for Matrix Games and Stochastic Games
Complexity Lower Bounds for Nonconvex-Strongly-Concave Min-Max Optimization
A Parameter-free Algorithm for Convex-concave Min-max Problems
Potential Function-based Framework for Making the Gradients Small in Convex and Min-Max Optimization
Independent Policy Gradient Methods for Competitive Reinforcement Learning
Optimistic Dual Extrapolation for Coherent Non-monotone Variational Inequalities
...
1
2
...

References

SHOWING 1-10 OF 59 REFERENCES
Convergence Rate of 풪(1/k) for Optimistic Gradient and Extragradient Methods in Smooth Convex-Concave Saddle Point Problems
On Lower Iteration Complexity Bounds for the Saddle Point Problems
Last-iterate convergence rates for min-max optimization
Lower complexity bounds of first-order methods for convex-concave bilinear saddle-point problems
Stochastic Gradient Descent for Non-smooth Optimization: Convergence Results and Optimal Averaging Schemes
Last-Iterate Convergence: Zero-Sum Games and Constrained Min-Max Optimization
On the Complexity of the Hybrid Proximal Extragradient Method for the Iterates and the Ergodic Mean
Making the Last Iterate of SGD Information Theoretically Optimal
Monotone Operators and the Proximal Point Algorithm
...
1
2
3
4
5
...