A Distributed Optimization Algorithm over Time-Varying Graphs with Efficient Gradient Evaluations

@article{VanScoy2019ADO,
  title={A Distributed Optimization Algorithm over Time-Varying Graphs with Efficient Gradient Evaluations},
  author={Bryan Van Scoy and Laurent Lessard},
  journal={IFAC-PapersOnLine},
  year={2019}
}

Figures from this paper

Projected Gradient Method for Decentralized Optimization over Time-Varying Networks
TLDR
This paper improves the rate of geometric convergence in the latter paper for the considered class of problems, using an original penalty method trick and robustness of projected gradient descent.
A Unified Algorithmic Framework for Distributed Composite Optimization
TLDR
A by-product of the analysis is a tuning recommendation for several existing (non-accelerated) distributed algorithms yielding the fastest provable (worst-case) convergence rate.

References

SHOWING 1-10 OF 13 REFERENCES
Analysis and Design of First-Order Distributed Optimization Algorithms Over Time-Varying Graphs
TLDR
A unified analysis is provided of distributed first-order optimization algorithms over time-varying graphs that yields the worst-case linear convergence rate as a function of the condition number of the local functions, the spectral gap of the graph, and the parameters of the algorithm.
Optimal Algorithms for Smooth and Strongly Convex Distributed Optimization in Networks
TLDR
The efficiency of MSDA against state-of-the-art methods for two problems: least-squares regression and classification by logistic regression is verified.
Achieving Geometric Convergence for Distributed Optimization Over Time-Varying Graphs
TLDR
This paper introduces a distributed algorithm, referred to as DIGing, based on a combination of a distributed inexact gradient method and a gradient tracking technique that converges to a global and consensual minimizer over time-varying graphs.
Distributed Subgradient Methods for Multi-Agent Optimization
TLDR
The authors' convergence rate results explicitly characterize the tradeoff between a desired accuracy of the generated approximate optimal solutions and the number of iterations needed to achieve the accuracy.
Harnessing Smoothness to Accelerate Distributed Optimization
  • Guannan Qu, Na Li
  • Computer Science
    IEEE Transactions on Control of Network Systems
  • 2018
TLDR
It is shown that it is impossible for a class of distributed algorithms like DGD to achieve a linear convergence rate without using history information even if the objective function is strongly convex and smooth, and a novel gradient estimation scheme is proposed that uses history information to achieve fast and accurate estimation of the average gradient.
Augmented distributed gradient methods for multi-agent optimization under uncoordinated constant stepsizes
TLDR
A new augmented distributed gradient method (termed Aug-DGM) based on consensus theory is developed that will be able to seek the exact optimum even with constant stepsizes assuming that the global objective function has Lipschitz gradient.
Exact Diffusion for Distributed Optimization and Learning—Part I: Algorithm Development
TLDR
The exact diffusion method is applicable to locally balanced left-stochastic combination matrices which, compared to the conventional doubly stochastic matrix, are more general and able to endow the algorithm with faster convergence rates, more flexible step-size choices, and improved privacy-preserving properties.
A Decentralized Proximal-Gradient Method With Network Independent Step-Sizes and Separated Convergence Rates
TLDR
This paper proposes a novel proximal-gradient algorithm for a decentralized optimization problem with a composite objective containing smooth and nonsmooth terms that is as good as one of the two convergence rates that match the typical rates for the general gradient descent and the consensus averaging.
EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization
TLDR
A novel decentralized exact first-order algorithm (abbreviated as EXTRA) to solve the consensus optimization problem and uses a fixed, large step size, which can be determined independently of the network size or topology.
...
...