SEGA: Variance Reduction via Gradient Sketching

@inproceedings{Hanzely2018SEGAVR,
  title={SEGA: Variance Reduction via Gradient Sketching},
  author={Filip Hanzely and Konstantin Mishchenko and Peter Richt{\'a}rik},
  booktitle={NeurIPS},
  year={2018}
}
We propose a novel randomized first order optimization method---SEGA (SkEtched GrAdient method)---which progressively throughout its iterations builds a variance-reduced estimate of the gradient from random linear measurements (sketches) of the gradient provided at each iteration by an oracle. In each iteration, SEGA updates the current estimate of the gradient through a sketch-and-project operation using the information provided by the latest sketch, and this is subsequently used to compute an… CONTINUE READING
3
Twitter Mentions

References

Publications referenced by this paper.
SHOWING 1-10 OF 47 REFERENCES

Random direct search method for minimizing nonconvex, convex and strongly convex functions

El Houcine Bergou, Peter Richtárik, Eduard Gorbunov
  • Manuscript
  • 2018
VIEW 1 EXCERPT