SEGA: Variance Reduction via Gradient Sketching

@inproceedings{Hanzely2018SEGAVR,
  title={SEGA: Variance Reduction via Gradient Sketching},
  author={Filip Hanzely and Konstantin Mishchenko and Peter Richt{\'a}rik},
  booktitle={NeurIPS},
  year={2018}
}

• Filip Hanzely, Konstantin Mishchenko, Peter Richtárik
• Published in NeurIPS 2018

We propose a novel randomized first order optimization method---SEGA (SkEtched GrAdient method)---which progressively throughout its iterations builds a variance-reduced estimate of the gradient from random linear measurements (sketches) of the gradient provided at each iteration by an oracle. In each iteration, SEGA updates the current estimate of the gradient through a sketch-and-project operation using the information provided by the latest sketch, and this is subsequently used to compute an… CONTINUE READING

View PDF

Save to Library

Create Alert

Cite