• DOI:10.1137/S0097539797326502
• Corpus ID: 7100729

Automatic Methods for Hiding Latency in Parallel and Distributed Computation

@article{Andrews1999AutomaticMF,
  title={Automatic Methods for Hiding Latency in Parallel and Distributed Computation},
  author={Matthew Andrews and Frank Thomson Leighton and Panagiotis Takis Metaxas and Lisa Zhang},
  journal={SIAM J. Comput.},
  year={1999},
  volume={29},
  pages={615-647}
}

• M. Andrews, F. Leighton, +1 author Lisa Zhang
• Published 1 October 1999
• Computer Science
• SIAM J. Comput.

In this paper we describe methods for mitigating the degradation in performance caused by high latencies in parallel and distributed networks. For example, given any "dataflow" type of algorithm that runs in T steps on an n-node ring with unit link delays, we show how to run the algorithm in O(T) steps on any n-node bounded-degree connected network with average link delay O(1). This is a significant improvement over prior approaches to latency hiding, which require slowdowns proportional to the maximum link delay. In the case when the network has average link delay $\dave$, our simulation runs in $O(\sqrt{\dave} T)$ steps using $n/\sqrt{\dave}$ processors, thereby preserving efficiency. We also show how to efficiently simulate an n X n array with unit link delays using slowdown $\tilde O(\dave^{2/3})$ on a two-dimensional array with average link delay $\dave$. Last, we present results for the case in which large local databases are involved in the computation.