SGD: General Analysis and Improved Rates

@article{Gower2019SGDGA,
  title={SGD: General Analysis and Improved Rates},
  author={Robert Mansel Gower and Nicolas Loizou and Xun Qian and Alibek Sailanbayev and Egor Shulgin and Peter Richt{\'a}rik},
  journal={ArXiv},
  year={2019},
  volume={abs/1901.09401}
}
We propose a general yet simple theorem describing the convergence of SGD under the arbitrary sampling paradigm. Our theorem describes the convergence of an infinite array of variants of SGD, each of which is associated with a specific probability law governing the data selection rule used to form mini-batches. This is the first time such an analysis is performed, and most of our variants of SGD were never explicitly considered in the literature before. Our analysis relies on the recently… CONTINUE READING

Figures and Topics from this paper.

Citations

Publications citing this paper.
SHOWING 1-10 OF 10 CITATIONS

Unified Optimal Analysis of the (Stochastic) Gradient Method

VIEW 4 EXCERPTS
CITES BACKGROUND & RESULTS
HIGHLY INFLUENCED

References

Publications referenced by this paper.
SHOWING 1-10 OF 45 REFERENCES