Sparse Allreduce: Efficient Scalable Communication for Power-Law Data

@article{Zhao2013SparseAE,
  title={Sparse Allreduce: Efficient Scalable Communication for Power-Law Data},
  author={Huasha Zhao and John F. Canny},
  journal={ArXiv},
  year={2013},
  volume={abs/1312.3020}
}
Many large datasets exhibit power-law statistics: The web graph, social networks, text data, click through data etc. Their adjacency graphs are termed natural graphs, and are known to be difficult to partition. As a consequence most distributed algorithms on these graphs are communication intensive. Many algorithms on natural graphs involve an Allreduce: a sum or average of partitioned data which is then shared back to the cluster nodes. Examples include PageRank, spectral partitioning, and… CONTINUE READING
2
Twitter Mentions

Similar Papers

Citations

Publications citing this paper.
SHOWING 1-2 OF 2 CITATIONS

Machine learning at the limit

  • 2015 IEEE International Conference on Big Data (Big Data)
  • 2015
VIEW 1 EXCERPT
CITES METHODS