Distributed Training Large-Scale Deep Architectures

  title={Distributed Training Large-Scale Deep Architectures},
  author={Shang-Xuan Zou and Chun-Yen Chen and Jui-Lin Wu and Chun-Nan Chou and Chia-Chin Tsao and Kuan-Chieh Tung and Ting-Wei Lin and Cheng-Lung Sung and Edward Y. Chang},
Scale of data and scale of computation infrastructures together enable the current deep learning renaissance. However, training large-scale deep architectures demands both algorithmic improvement and careful system configuration. In this paper, we focus on employing the system approach to speed up large-scale training. Taking both the algorithmic and system aspects into consideration, we develop a procedure for setting mini-batch size and choosing computation algorithms. We also derive lemmas… CONTINUE READING
Highly Cited
This paper has 26 citations. REVIEW CITATIONS
21 Citations
42 References
Similar Papers


Publications referenced by this paper.
Showing 1-10 of 42 references

Distributed training large-scale deep architectures

  • Zou, S.-X
  • HTC technical report
  • 2017
1 Excerpt

Similar Papers

Loading similar papers…