Distributed Optimization, Averaging via ADMM, and Network Topology
@article{Frana2020DistributedOA, title={Distributed Optimization, Averaging via ADMM, and Network Topology}, author={Guilherme França and Jos{\'e} Bento}, journal={Proceedings of the IEEE}, year={2020}, volume={108}, pages={1939-1952} }
There has been an increasing necessity for scalable optimization methods, especially due to the explosion in the size of data sets and model complexity in modern machine learning applications. Scalable solvers often distribute the computation over a network of processing units. For simple algorithms, such as gradient descent, the dependence of the convergence time with the topology of this network is well known. However, for more involved algorithms, such as the alternating direction method of…
References
SHOWING 1-10 OF 56 REFERENCES
How is Distributed ADMM Affected by Network Topology
- Mathematics, Computer Science
- 2017
A full characterization of the convergence of distributed over-relaxed ADMM for the same type of consensus problem in terms of the topology of the underlying graph is provided and a proof of the aforementioned conjecture is shown it is valid for any graph, even the ones whose random walks cannot be accelerated via Markov chain lifting.
Convergence Rate of Distributed ADMM Over Networks
- Computer ScienceIEEE Transactions on Automatic Control
- 2017
A new distributed algorithm based on alternating direction method of multipliers (ADMM) to minimize sum of locally known convex functions using communication over a network and highlights the effect of network and communication weights on the convergence rate through degrees of the nodes, the smallest nonzero eigenvalue, and operator norm of the communication matrix.
Distributed Averaging Via Lifted Markov Chains
- Mathematics, Computer ScienceIEEE Transactions on Information Theory
- 2010
This paper designs an algorithm with the fastest possible rate of convergence using a nonreversible Markov chain on the given network graph using the Metropolis-Hastings method, and provides the fastest mixingMarkov chain given the network topological constraints.
Polynomial Filtering for Fast Convergence in Distributed Consensus
- Computer Science, MathematicsIEEE Transactions on Signal Processing
- 2009
This paper proposes to accelerate the convergence rate for given network matrices by the use of polynomial filtering algorithms, and forms the computation of the coefficients of the optimal polynometric as a semidefinite program that can be efficiently and globally solved for both static and dynamic network topologies.
Optimal Algorithms for Smooth and Strongly Convex Distributed Optimization in Networks
- Computer ScienceICML
- 2017
The efficiency of MSDA against state-of-the-art methods for two problems: least-squares regression and classification by logistic regression is verified.
Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization
- Computer ScienceProceedings of the IEEE
- 2018
This paper presents an overview of recent work in decentralized optimization and surveys the state-of-theart algorithms and their analyses tailored to these different scenarios, highlighting the role of the network topology.
Optimal Algorithms for Non-Smooth Distributed Optimization in Networks
- Mathematics, Computer ScienceNeurIPS
- 2018
The error due to limits in communication resources decreases at a fast rate even in the case of non-strongly-convex objective functions, and the first optimal first-order decentralized algorithm called multi-step primal-dual (MSPD) and its corresponding optimal convergence rate are provided.
Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers
- Computer ScienceFound. Trends Mach. Learn.
- 2011
It is argued that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas.
The ADMM algorithm for distributed averaging: Convergence rates and optimal parameter selection
- Computer Science2014 48th Asilomar Conference on Signals, Systems and Computers
- 2014
This study derives the optimal step-size and over-relaxation parameter that minimizes the convergence time of two ADMM-based algorithms for distributed averaging and optimize the edge-weights of the communication graph to improve the convergence speed.
Distributed Estimation and Control of Algebraic Connectivity Over Random Graphs
- Computer Science, MathematicsIEEE Transactions on Signal Processing
- 2014
Using results from stochastic approximation theory, it is proved that the proposed method converges almost surely (a.s.) to the desired value of connectivity even in the presence of imperfect communication scenarios.