Data-Dependent Stability of Stochastic Gradient Descent

  title={Data-Dependent Stability of Stochastic Gradient Descent},
  author={Ilja Kuzborskij and Christoph H. Lampert},
We establish a data-dependent notion of algorithmic stability for Stochastic Gradient Descent (SGD) and employ it to develop novel generalization bounds. This is in contrast to previous distribution-free algorithmic stability results for SGD which depend on the worstcase constants. By virtue of the data-dependent argument, our bounds provide new insights into learning with SGD on convex and non-convex problems. In the convex case, we show that the bound on the generalization error is… CONTINUE READING