Riemannian stochastic variance reduced gradient

@article{Sato2019RiemannianSV,
  title={Riemannian stochastic variance reduced gradient},
  author={Hiroyuki Sato and H. Kasai and Bamdev Mishra},
  journal={SIAM J. Optim.},
  year={2019},
  volume={29},
  pages={1444-1472}
}
  • Hiroyuki Sato, H. Kasai, Bamdev Mishra
  • Published 2019
  • Computer Science, Mathematics
  • SIAM J. Optim.
  • In recent years, stochastic variance reduction algorithms have attracted considerable attention for minimizing the average of a large but finite number of loss functions. This paper proposes a novel Riemannian extension of the Euclidean stochastic variance reduced gradient (R-SVRG) algorithm to a manifold search space. The key challenges of averaging, adding, and subtracting multiple gradients are addressed with retraction and vector transport. For the proposed algorithm, we present a global… CONTINUE READING
    36 Citations
    Riemannian stochastic quasi-Newton algorithm with variance reduction and its convergence analysis
    • 12
    • PDF
    Riemannian Stochastic Recursive Gradient Algorithm
    • 16
    • PDF
    Riemannian Stochastic Recursive Gradient Algorithm with Retraction and Vector Transport and Its Convergence Analysis
    • 5
    • PDF
    Adaptive stochastic gradient algorithms on Riemannian manifolds
    • 3
    • PDF
    MASAGA: A Stochastic Algorithm for Manifold Optimization
    • Highly Influenced
    • PDF
    Projection-free nonconvex stochastic optimization on Riemannian manifolds
    • 5
    • PDF
    Averaging Stochastic Gradient Descent on Riemannian Manifolds
    • 38
    • PDF
    Riemannian adaptive stochastic gradient algorithms on matrix manifolds
    • 13
    • PDF
    MASAGA: A Linearly-Convergent Stochastic First-Order Method for Optimization on Manifolds
    • 4
    • PDF
    Nonconvex stochastic optimization on manifolds via Riemannian Frank-Wolfe methods
    • 6
    • Highly Influenced

    References

    SHOWING 1-10 OF 47 REFERENCES
    Riemannian stochastic variance reduced gradient on Grassmann manifold
    • 18
    • PDF
    Riemannian stochastic quasi-Newton algorithm with variance reduction and its convergence analysis
    • 12
    • PDF
    Riemannian Stochastic Recursive Gradient Algorithm
    • 16
    • PDF
    Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifolds
    • 111
    • Highly Influential
    • PDF
    Fast stochastic optimization on Riemannian manifolds
    • 24
    • Highly Influential
    • PDF
    Stochastic Gradient Descent on Riemannian Manifolds
    • S. Bonnabel
    • Mathematics, Computer Science
    • IEEE Transactions on Automatic Control
    • 2013
    • 269
    • PDF
    Accelerating Stochastic Gradient Descent using Predictive Variance Reduction
    • 1,644
    • Highly Influential
    • PDF
    Linear Regression under Fixed-Rank Constraints: A Riemannian Approach
    • 58
    • PDF
    A Riemannian Framework for Tensor Computing
    • 1,280
    • PDF