# Generalization Error Bounds for Optimization Algorithms via Stability

@inproceedings{Meng2017GeneralizationEB, title={Generalization Error Bounds for Optimization Algorithms via Stability}, author={Qi Meng and Y. Wang and Wei Chen and Taifeng Wang and Z. Ma and Tie-Yan Liu}, booktitle={AAAI}, year={2017} }

Many machine learning tasks can be formulated as Regularized Empirical Risk Minimization (R-ERM), and solved by optimization algorithms such as gradient descent (GD), stochastic gradient descent (SGD), and stochastic variance reduction (SVRG). Conventional analysis on these optimization algorithms focuses on their convergence rates during the training process, however, people in the machine learning community may care more about the generalization performance of the learned model on unseen test… Expand

#### 5 Citations

Exploring the Generalization Performance of Neural Networks via Diversity

- Computer Science
- Aust. J. Intell. Inf. Process. Syst.
- 2019

Why ResNet Works? Residuals Generalize

- Mathematics, Computer Science
- IEEE Transactions on Neural Networks and Learning Systems
- 2020

A Neural Network Model of Incubation through Mind Wandering

- Computer Science
- Aust. J. Intell. Inf. Process. Syst.
- 2019

#### References

SHOWING 1-10 OF 39 REFERENCES

Train faster, generalize better: Stability of stochastic gradient descent

- Computer Science, Mathematics
- ICML
- 2016

Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization

- Mathematics, Computer Science
- ICML
- 2012

On the Generalization Ability of Online Strongly Convex Programming Algorithms

- Computer Science
- NIPS
- 2008

Learning theory: stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization

- Mathematics, Computer Science
- Adv. Comput. Math.
- 2006

On the generalization ability of on-line learning algorithms

- Computer Science, Mathematics
- IEEE Transactions on Information Theory
- 2004

Learnability, Stability and Uniform Convergence

- Mathematics, Computer Science
- J. Mach. Learn. Res.
- 2010

Competing with the Empirical Risk Minimizer in a Single Pass

- Mathematics, Computer Science
- COLT
- 2015

Stability and Generalization

- Mathematics, Computer Science
- J. Mach. Learn. Res.
- 2002

Accelerating Stochastic Gradient Descent using Predictive Variance Reduction

- Computer Science, Mathematics
- NIPS
- 2013

Stochastic Variance Reduction for Nonconvex Optimization

- Mathematics, Computer Science
- ICML
- 2016