Design and implementation tradeoffs for wide-area resource discovery

Abstract

We describe the design and implementation of SWORD, a scalable resource discovery service for wide-area distributed systems. In contrast to previous systems, SWORD allows users to describe desired resources as a topology of interconnected groups with required intragroup, intergroup, and per-node characteristics, along with the utility that the application derives from specified ranges of metric values. This design gives users the flexibility to find geographically distributed resources for applications that are sensitive to both node and network characteristics, and allows the system to rank acceptable configurations based on their quality for that application. Rather than evaluating a single implementation of SWORD, we explore a variety of architectural designs that deliver the required functionality in a scalable and highly available manner. We discuss the trade-offs of using a centralized architecture as compared to a fully decentralized design to perform wide-area resource discovery. To summarize our results, we found that a centralized architecture based on 4-node server cluster sites at network-peering facilities outperforms a decentralized DHT-based resource discovery infrastructure with respect to query latency for all but the smallest number of sites. However, although a centralized architecture shows significant promise in stable environments, we find that our decentralized implementation has acceptable performance and also benefits from the DHT's self-healing properties in more volatile environments. We evaluate the advantages and disadvantages of centralized and distributed resource discovery architectures on 1000 hosts in emulation and on approximately 200 PlanetLab nodes spread across the Internet.

DOI: 10.1145/1391949.1391952
View Slides

Extracted Key Phrases

8 Figures and Tables

Statistics

0102030'05'06'07'08'09'10'11'12'13'14'15'16'17
Citations per Year

129 Citations

Semantic Scholar estimates that this publication has 129 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@article{Oppenheimer2005DesignAI, title={Design and implementation tradeoffs for wide-area resource discovery}, author={David L. Oppenheimer and Jeannie R. Albrecht and David A. Patterson and Amin Vahdat}, journal={HPDC-14. Proceedings. 14th IEEE International Symposium on High Performance Distributed Computing, 2005.}, year={2005}, pages={113-124} }