Corpus ID: 221112331

Distributed Stochastic Gradient Descent: Nonconvexity, Nonsmoothness, and Convergence to Local Minima

@inproceedings{Swenson2020DistributedSG,
  title={Distributed Stochastic Gradient Descent: Nonconvexity, Nonsmoothness, and Convergence to Local Minima},
  author={B. Swenson and R. Murray and S. Kar and H. Poor},
  year={2020}
}
  • B. Swenson, R. Murray, +1 author H. Poor
  • Published 2020
  • Mathematics
  • In centralized settings, it is well known that stochastic gradient descent (SGD) avoids saddle points and converges to local minima in nonconvex problems. However, similar guarantees are lacking for distributed first-order algorithms. The paper studies distributed stochastic gradient descent (D-SGD)—a simple network-based implementation of SGD. Conditions under which D-SGD avoids saddle points and converges to local minima are studied. First, we consider the problem of computing critical points… CONTINUE READING

    Figures from this paper.

    Distributed Gradient Flow: Nonsmoothness, Nonconvexity, and Saddle Point Evasion
    2
    Convergence of a Distributed Kiefer-Wolfowitz Algorithm

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 88 REFERENCES
    Distributed Gradient Flow: Nonsmoothness, Nonconvexity, and Saddle Point Evasion
    2
    Distributed Gradient Descent: Nonconvergence to Saddle Points and the Stable-Manifold Theorem
    9
    Escaping From Saddle Points - Online Stochastic Gradient for Tensor Decomposition
    637
    On Distributed Stochastic Gradient Algorithms for Global Optimization
    5
    Second-order Guarantees of Distributed Gradient Algorithms
    28
    NEXT: In-Network Nonconvex Optimization
    219
    DSA: Decentralized Double Stochastic Averaging Gradient Algorithm
    97