# Linear convergence of cyclic SAGA

@article{Park2020LinearCO, title={Linear convergence of cyclic SAGA}, author={Youngsuk Park and E. K. Ryu}, journal={Optimization Letters}, year={2020}, volume={14}, pages={1583-1598} }

In this work, we present and analyze C-SAGA, a (deterministic) cyclic variant of SAGA. C-SAGA is an incremental gradient method that minimizes a sum of differentiable convex functions by cyclically accessing their gradients. Even though the theory of stochastic algorithms is more mature than that of cyclic counterparts in general, practitioners often prefer cyclic algorithms. We prove C-SAGA converges linearly under the standard assumptions. Then, we compare the rate of convergence with the… CONTINUE READING

One Citation

#### References

##### Publications referenced by this paper.

SHOWING 1-10 OF 38 REFERENCES

On the Convergence Rate of Incremental Aggregated Gradient Algorithms

- Mathematics, Computer Science
- 2017

93

A Stronger Convergence Result on the Proximal Incremental Aggregated Gradient Method

- Mathematics
- 2016

13

Surpassing Gradient Descent Provably: A Cyclic Incremental Method with Linear Convergence Rate

- Computer Science, Mathematics
- 2018

19

SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives

- Mathematics, Computer Science
- 2014

914

Global Convergence Rate of Proximal Incremental Aggregated Gradient Methods

- Mathematics, Computer Science
- 2018

28

A Convergent Incremental Gradient Method with a Constant Step Size

- Mathematics, Computer Science
- 2007

232

A Proximal Stochastic Gradient Method with Progressive Variance Reduction

- Computer Science, Mathematics
- 2014

518