• DOI:10.1109/INFOCOM.2019.8737367
Corpus ID: 53732628

MG-WFBP: Efficient Data Communication for Distributed Synchronous SGD Algorithms

@article{Shi2019MGWFBPED,
  title={MG-WFBP: Efficient Data Communication for Distributed Synchronous SGD Algorithms},
  author={Shaohuai Shi and Xiaowen Chu},
  journal={IEEE INFOCOM 2019 - IEEE Conference on Computer Communications},
  year={2019},
  pages={172-180}
}

Shaohuai Shi, Xiaowen Chu

Published 2019

Computer Science

IEEE INFOCOM 2019 - IEEE Conference on Computer Communications

Distributed synchronous stochastic gradient descent has been widely used to train deep neural networks on computer clusters. With the increase of computational power, network communications have become one limiting factor on the system scalability. In this paper, we observe that many deep neural networks have a large number of layers with only a small amount of data to be communicated. Based on the fact that merging some short communication tasks into a single one may reduce the overall… CONTINUE READING

View on IEEE

arxiv.org

Save to Library

Create Alert

Cite

Launch Research Feed