Coded distributed computing: Fundamental limits and practical challenges

@article{Li2016CodedDC,
  title={Coded distributed computing: Fundamental limits and practical challenges},
  author={Songze Li and Qian Yu and Mohammad Ali Maddah-Ali and Amir Salman Avestimehr},
  journal={2016 50th Asilomar Conference on Signals, Systems and Computers},
  year={2016},
  pages={509-513}
}
In this paper, we demonstrate a coded computing framework, named Coded Distributed Computing (CDC), which optimally trades extra computation resources for communication bandwidth in a MapReduce-type distributed computing environment. We also empirically illustrate the practical impact of CDC by analyzing the performance of a distributed sorting algorithm, named CodedTeraSort, which was developed by integrating the coding principle of CDC into the Hadoop benchmark TeraSort. Experiment results… CONTINUE READING

Citations

Publications citing this paper.

References

Publications referenced by this paper.
Showing 1-10 of 19 references

Coded TeraSort

S. Supittayapornpong, S. Li, A. S. Avestimehr
submitted to IEEE IPDPS 2017. • 2017
View 2 Excerpts

Coded Distributed Computing: Straggling Servers and Multistage Dataflows

2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton) • 2016
View 1 Excerpt

Fundamental Limits of Caching in Wireless D2D Networks

IEEE Transactions on Information Theory • 2016
View 1 Excerpt

Coded MapReduce

2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton) • 2015
View 3 Excerpts

Similar Papers

Loading similar papers…