Communication efficient algorithms for fundamental big data problems


Big Data applications often store or obtain their data distributed over many computers connected by a network. Since the network is usually slower than the local memory of the machines, it is crucial to process the data in such a way that not too much communication takes place. Indeed, only communication volume sublinear in the input size may be affordable. We believe that this direction of research deserves more intensive study. We give examples for several fundamental algorithmic problems where nontrivial algorithms with sublinear communication volume are possible. Our main technical contribution are several related results on distributed Bloom filter replacements, duplicate detection, and data base join. As an example of a very different family of techniques, we discuss linear programming in low dimensions.

DOI: 10.1109/BigData.2013.6691549

1 Figure or Table

Citations per Year

Citation Velocity: 7

Averaging 7 citations per year over the last 3 years.

Learn more about how we calculate this metric in our FAQ.

Cite this paper

@article{Sanders2013CommunicationEA, title={Communication efficient algorithms for fundamental big data problems}, author={Peter Sanders and Sebastian Schlag and Ingo M{\"{u}ller}, journal={2013 IEEE International Conference on Big Data}, year={2013}, pages={15-23} }