# Optimal Gradient Tracking for Decentralized Optimization

@inproceedings{Song2021OptimalGT, title={Optimal Gradient Tracking for Decentralized Optimization}, author={Zhuoqing Song and Lei Shi and Shi Pu and Ming Yan}, year={2021} }

In this paper, we focus on solving the decentralized optimization problem of minimizing the sum of n objective functions over a multi-agent network. The agents are embedded in an undirected graph where they can only send/receive information directly to/from their immediate neighbors. Assuming smooth and strongly convex objective functions, we propose an Optimal Gradient Tracking (OGT) method that achieves the optimal gradient computation complexity O (√ κ log 1 ǫ ) and the optimal communication…

## One Citation

Recent theoretical advances in decentralized distributed convex optimization.

- Mathematics, Computer Science
- 2020

This paper focuses on how the results of decentralized distributed convex optimization can be explained based on optimal algorithms for the non-distributed setup, and provides recent results that have not been published yet.

## References

SHOWING 1-10 OF 68 REFERENCES

Optimal Algorithms for Smooth and Strongly Convex Distributed Optimization in Networks

- Computer Science, MathematicsICML
- 2017

The efficiency of MSDA against state-of-the-art methods for two problems: least-squares regression and classification by logistic regression is verified.

Push–Pull Gradient Methods for Distributed Optimization in Networks

- Computer ScienceIEEE Transactions on Automatic Control
- 2021

“push–pull” is the first class of algorithms for distributed optimization over directed graphs for strongly convex and smooth objective functions over a network and outperform other existing linearly convergent schemes, especially for ill-conditioned problems and networks that are not well balanced.

Achieving Linear Convergence in Distributed Asynchronous Multiagent Optimization

- Computer Science, MathematicsIEEE Transactions on Automatic Control
- 2020

This article proposes a general distributed asynchronous algorithmic framework whereby agents can update their local variables as well as communicate with their neighbors at any time, without any form of coordination, and proves that this is the first distributed algorithm with provable geometric convergence rate in such a general asynchronous setting.

Multi-consensus Decentralized Accelerated Gradient Descent

- Computer Science, MathematicsArXiv
- 2020

A novel algorithm is proposed that can achieve near optimal communication complexity, matching the known lower bound up to a logarithmic factor of the condition number of the problem.

Fast Distributed Gradient Methods

- Mathematics, Computer ScienceIEEE Transactions on Automatic Control
- 2014

This work proposes two fast distributed gradient algorithms based on the centralized Nesterov gradient algorithm and establishes their convergence rates in terms of the per-node communications K and theper-node gradient evaluations k.

Optimal Algorithms for Non-Smooth Distributed Optimization in Networks

- Mathematics, Computer ScienceNeurIPS
- 2018

The error due to limits in communication resources decreases at a fast rate even in the case of non-strongly-convex objective functions, and the first optimal first-order decentralized algorithm called multi-step primal-dual (MSPD) and its corresponding optimal convergence rate are provided.

Convergence of Asynchronous Distributed Gradient Methods Over Stochastic Networks

- Mathematics, Computer ScienceIEEE Transactions on Automatic Control
- 2018

A new asynchronous distributed gradient method (AsynDGM) based on consensus theory is developed that not only allows for asynchronous implementation in a completely distributed manner but also is able to seek the exact optimum even with constant stepsizes.

Harnessing smoothness to accelerate distributed optimization

- Mathematics, Computer Science2016 IEEE 55th Conference on Decision and Control (CDC)
- 2016

This paper proposes a distributed algorithm that, despite using the same amount of communication per iteration as DGD, can effectively harnesses the function smoothness and converge to the optimum with a rate of O(1/t) if the objective function is strongly convex and smooth.

Augmented distributed gradient methods for multi-agent optimization under uncoordinated constant stepsizes

- Mathematics, Computer Science2015 54th IEEE Conference on Decision and Control (CDC)
- 2015

A new augmented distributed gradient method (termed Aug-DGM) based on consensus theory is developed that will be able to seek the exact optimum even with constant stepsizes assuming that the global objective function has Lipschitz gradient.

Distributed Algorithms for Composite Optimization: Unified Framework and Convergence Analysis

- Computer Science, MathematicsIEEE Transactions on Signal Processing
- 2021

A by-product of this analysis is a tuning recommendation for several existing (non-accelerated) distributed algorithms yielding provably faster (worst-case) convergence rate for the class of problems under consideration.