# Distributed Stochastic Gradient Descent: Nonconvexity, Nonsmoothness, and Convergence to Local Minima

@inproceedings{Swenson2020DistributedSG, title={Distributed Stochastic Gradient Descent: Nonconvexity, Nonsmoothness, and Convergence to Local Minima}, author={B. Swenson and R. Murray and S. Kar and H. Poor}, year={2020} }

In centralized settings, it is well known that stochastic gradient descent (SGD) avoids saddle points and converges to local minima in nonconvex problems. However, similar guarantees are lacking for distributed first-order algorithms. The paper studies distributed stochastic gradient descent (D-SGD)—a simple network-based implementation of SGD. Conditions under which D-SGD avoids saddle points and converges to local minima are studied. First, we consider the problem of computing critical points… CONTINUE READING

3 Citations

Distributed Gradient Flow: Nonsmoothness, Nonconvexity, and Saddle Point Evasion

- Mathematics, Computer Science
- 2020

2

#### References

##### Publications referenced by this paper.

SHOWING 1-10 OF 88 REFERENCES

Distributed Gradient Flow: Nonsmoothness, Nonconvexity, and Saddle Point Evasion

- Mathematics, Computer Science
- 2020

2

Distributed Gradient Descent: Nonconvergence to Saddle Points and the Stable-Manifold Theorem

- Computer Science, Mathematics
- 2019

9

Escaping From Saddle Points - Online Stochastic Gradient for Tensor Decomposition

- Computer Science, Mathematics
- 2015

637

On Nonconvex Optimization for Machine Learning: Gradients, Stochasticity, and Saddle Points

- Mathematics
- 2019

26

On Distributed Stochastic Gradient Algorithms for Global Optimization

- Mathematics, Computer Science
- 2020

5

Distributed Learning in Non-Convex Environments - Part II: Polynomial Escape from Saddle-Points

- Computer Science, Engineering
- 2019

22

Second-Order Guarantees of Stochastic Gradient Descent in Non-Convex Optimization

- Mathematics, Computer Science
- 2019

7

DSA: Decentralized Double Stochastic Averaging Gradient Algorithm

- Mathematics, Computer Science
- 2016

97