Corpus ID: 208910534

Variance-Reduced Decentralized Stochastic Optimization with Accelerated Convergence

@article{Xin2019VarianceReducedDS,
  title={Variance-Reduced Decentralized Stochastic Optimization with Accelerated Convergence},
  author={Ran Xin and Usman A. Khan and Soummya Kar},
  journal={ArXiv},
  year={2019},
  volume={abs/1912.04230}
}
  • Ran Xin, Usman A. Khan, Soummya Kar
  • Published 2019
  • Computer Science, Mathematics, Engineering
  • ArXiv
  • This paper studies decentralized optimization to minimize a finite-sum of functions available over a network of~$n$ nodes. We propose two efficient algorithms, namely~\textbf{\texttt{GT-SAGA}} and~\textbf{\texttt{GT-SVRG}}, that leverage decentralized stochastic gradient{}-tracking methods and variance-reduction techniques. We show that both algorithms achieve accelerated linear convergence for smooth and strongly-convex functions. We further describe the regimes in which~\textbf{\texttt{GT… CONTINUE READING

    Citations

    Publications citing this paper.
    SHOWING 1-4 OF 4 CITATIONS

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 70 REFERENCES

    A Distributed Stochastic Gradient Tracking Method

    • Shi Pu, Angelia Nedic
    • Computer Science, Mathematics
    • 2018 IEEE Conference on Decision and Control (CDC)
    • 2018
    VIEW 7 EXCERPTS
    HIGHLY INFLUENTIAL

    Augmented distributed gradient methods for multi-agent optimization under uncoordinated constant stepsizes

    VIEW 6 EXCERPTS
    HIGHLY INFLUENTIAL

    Convergence of Variance-Reduced Learning Under Random Reshuffling

    VIEW 6 EXCERPTS
    HIGHLY INFLUENTIAL

    DSA: Decentralized Double Stochastic Averaging Gradient Algorithm

    VIEW 8 EXCERPTS
    HIGHLY INFLUENTIAL

    Distributed Subgradient Methods for Multi-Agent Optimization

    VIEW 8 EXCERPTS
    HIGHLY INFLUENTIAL

    EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization

    VIEW 5 EXCERPTS
    HIGHLY INFLUENTIAL

    Exact Diffusion for Distributed Optimization and Learning—Part I: Algorithm Development

    VIEW 5 EXCERPTS
    HIGHLY INFLUENTIAL

    Minimizing finite sums with the stochastic average gradient

    VIEW 6 EXCERPTS
    HIGHLY INFLUENTIAL

    Optimization Methods for Large-Scale Machine Learning

    VIEW 8 EXCERPTS
    HIGHLY INFLUENTIAL