Stochastic Gradient Descent on Riemannian Manifolds

@article{Bonnabel2013StochasticGD,
  title={Stochastic Gradient Descent on Riemannian Manifolds},
  author={S. Bonnabel},
  journal={IEEE Transactions on Automatic Control},
  year={2013},
  volume={58},
  pages={2217-2229}
}
  • S. Bonnabel
  • Published 2013
  • Mathematics, Computer Science
  • IEEE Transactions on Automatic Control
  • Stochastic gradient descent is a simple approach to find the local minima of a cost function whose evaluations are corrupted by noise. In this paper, we develop a procedure extending stochastic gradient descent algorithms to the case where the function is defined on a Riemannian manifold. We prove that, as in the Euclidian case, the gradient descent algorithm converges to a critical point of the cost function. The algorithm has numerous potential applications, and is illustrated here by four… CONTINUE READING
    279 Citations
    Practical Accelerated Optimization on Riemannian Manifolds
    • 5
    Averaging Stochastic Gradient Descent on Riemannian Manifolds
    • 41
    • Highly Influenced
    • PDF
    Riemannian Stochastic Recursive Gradient Algorithm
    • 16
    • PDF
    Escaping from saddle points on Riemannian manifolds
    • PDF
    Gradient descent in hyperbolic space
    • 18
    • PDF
    MASAGA: A Linearly-Convergent Stochastic First-Order Method for Optimization on Manifolds
    • 4
    • PDF
    Escaping from saddle points on Riemannian manifolds
    • 20
    • PDF
    Riemannian stochastic variance reduced gradient
    • 36
    • PDF
    Riemannian stochastic variance reduced gradient on Grassmann manifold
    • 18
    • PDF

    References

    SHOWING 1-10 OF 91 REFERENCES
    Prior Knowledge and Preferential Structures in Gradient Descent Learning Algorithms
    • 23
    • PDF
    A Riemannian Framework for Tensor Computing
    • 1,287
    • PDF
    Stochastic algorithms for computing means of probability measures
    • 54
    • PDF
    Linear Regression under Fixed-Rank Constraints: A Riemannian Approach
    • 58
    • PDF
    Average consensus on Riemannian manifolds with bounded curvature
    • R. Tron, B. Afsari, R. Vidal
    • Computer Science, Mathematics
    • IEEE Conference on Decision and Control and European Control Conference
    • 2011
    • 19
    • PDF
    Adaptive filtering for estimation of a low-rank positive semidefinite matrix
    • 7
    • PDF
    The Geometry of Algorithms with Orthogonality Constraints
    • 2,259
    • PDF
    An intrinsic observer for a class of Lagrangian systems
    • 144
    • PDF