Linear convergence of cyclic SAGA

@article{Park2020LinearCO,
  title={Linear convergence of cyclic SAGA},
  author={Youngsuk Park and E. K. Ryu},
  journal={Optimization Letters},
  year={2020},
  volume={14},
  pages={1583-1598}
}
  • Youngsuk Park, E. K. Ryu
  • Published 2020
  • Mathematics, Computer Science
  • Optimization Letters
  • In this work, we present and analyze C-SAGA, a (deterministic) cyclic variant of SAGA. C-SAGA is an incremental gradient method that minimizes a sum of differentiable convex functions by cyclically accessing their gradients. Even though the theory of stochastic algorithms is more mature than that of cyclic counterparts in general, practitioners often prefer cyclic algorithms. We prove C-SAGA converges linearly under the standard assumptions. Then, we compare the rate of convergence with the… CONTINUE READING
    Structured Policy Iteration for Linear Quadratic Regulator

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 38 REFERENCES
    On the Convergence Rate of Incremental Aggregated Gradient Algorithms
    93
    SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
    914
    Global Convergence Rate of Proximal Incremental Aggregated Gradient Methods
    28
    Minimizing finite sums with the stochastic average gradient
    723
    A Convergent Incremental Gradient Method with a Constant Step Size
    232
    A Simple Proof for the Iteration Complexity of the Proximal Gradient Algorithm
    1
    A Proximal Stochastic Gradient Method with Progressive Variance Reduction
    518