Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining
@article{Renesse2003AstrolabeAR, title={Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining}, author={Robbert van Renesse and Kenneth P. Birman and Werner Vogels}, journal={ACM Trans. Comput. Syst.}, year={2003}, volume={21}, pages={164-206} }
Scalable management and self-organizational capabilities areemerging as central requirements for a generation of large-scale,highly dynamic, distributed applications. We have developed anentirely new distributed information management system calledAstrolabe. Astrolabe collects large-scale system state, permittingrapid updates and providing on-the-fly attribute aggregation. Thislatter capability permits an application to locate a resource, andalso offers a scalable way to track system state as…
Figures and Tables from this paper
821 Citations
A scalable information management middleware for large distributed systems
- Computer Science
- 2005
This dissertation presents a Scalable Distributed Information Management System (SDIMS) that aggregates information about large-scale networked systems and that can serve as a basic building block for a broad range of large- scale distributed applications by providing detailed views of nearby information and summary views of global information.
MON: Design and Implementation of Management Overlay Networks for Distributed Systems
- Computer Science
- 2005
The concept of MONs addresses concerns and opinions raised by both peer-to-peer networking and Grid computing communities in the recent past, and their simplicity, scalability, lightweight-ness and fault-tolerance enable them to run side-by-side with existing distributed applications.
DObjects : A Metacomputing Framework with Dynamic Query Processing for Distributed Data Networks
- Computer Science
- 2007
DObjects, a general-purpose distributed data objects framework that provides an easy and scalable way of querying and operating data from heterogeneous data sources, and the details of the dynamic query execution engine within the metacomputing framework that dynamically adapts to network and node conditions are presented.
The ganglia distributed monitoring system: design, implementation, and experience
- Computer ScienceParallel Comput.
- 2004
The One Minute Manager: Lightweight On-Demand Overlays for Distributed Application Management
- Computer Science
- 2006
The utility of MON is demonstrated by showing how it can be used to query the aggregate state of a real application deployed in a real world environment by way of an end-to-end response time of just a couple of seconds.
Scalable Management and Data Mining Using Astrolabe
- Computer ScienceIPTPS
- 2002
This paper focuses on wide- area implementation challenges of Astrolabe, a new kind of peer-to-peer system implementing a hierarchical distributed database abstraction that can also support wide-area multicast and offers powerful aggregation mechanisms.
DMake : A Tool for Monitoring and Configuring Distributed Systems
- Computer Science
- 2014
DMake is unusual in offering a very flexible range of management options through a familiar and widely popular model: that of the Unix make utility, and its extended “makefile” format can express complex policies.
Design and Implementation of a Scalable Network Monitoring System
- Computer Science
- 2008
This work presents a monitoring system that scale to over 100000 nodes, has minimal local and global overhead, and maintains integrity in the face of transient network failure, and includes a web service interface, which allows access to the system via HTTP.
A scalable distributed information management system
- Computer ScienceSIGCOMM
- 2004
This work designs, implements and evaluates a Scalable Distributed Information Management System (SDIMS) that leverages Distributed Hash Tables (DHT) to create scalable aggregation trees, achieves isolation properties at the cost of modestly increased read latency in comparison to flat DHTs, and gracefully handles failures.
Moara: Flexible and Scalable Group-Based Querying System
- Computer ScienceMiddleware
- 2008
Moara is presented, a new querying system that makes two novel contributions: first, Moara builds aggregation trees for different groups and adaptively maintains the trees to optimize the total message cost, and secondly Moara supports a query language allowing groups to be specified implicitly via predicates consisting of arbitrarily nested unions and intersections.
References
SHOWING 1-10 OF 52 REFERENCES
INS/Twine: A Scalable Peer-to-Peer Architecture for Intentional Resource Discovery
- Computer SciencePervasive
- 2002
The design, implementation, and evaluation of INS/Twine is described, an approach to scalable intentional resource discovery, where resolvers collaborate as peers to distribute resource information and to resolve queries.
The Information Bus: an architecture for extensible distributed systems
- Computer ScienceSOSP '93
- 1993
The Information Bus, the solution, is a novel synthesis of four design principles: core communication protocols have minimal semantics, objects are self-describing, types can be dynamically defined, and communication is anonymous.
Pastry: Scalable, distributed object location and routing for large-scale peer-to-
- Computer Science
- 2001
Experimental results obtained with a prototype implementa tion on a simulated network of up to 100,000 nodes confirm Pastry’s scalability, its ability to selfconfigure and adapt to node failures, and its good network loc ality properties.
Exploiting virtual synchrony in distributed systems
- Computer ScienceSOSP '87
- 1987
It is argued that this approach to building distributed and fault-tolerant software is more straightforward, more flexible, and more likely to yield correct solutions than alternative approaches.
A Weak-Consistency Architecture for Distributed Information Services
- Computer ScienceComput. Syst.
- 1992
This architecture implements the replicas as a weak-consistency process group, which provides good scalability and availability, handles portable computer systems, and minimizes the effect of users on each other.
The design and implementation of an intentional naming system
- Computer ScienceSOSP
- 1999
The design and implementation of the Intentional Naming System (INS), a resource discovery and service location system for dynamic and mobile networks of devices and computers, is presented and three applications are described that demonstrate the feasibility and utility of INS.
Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems
- Computer ScienceMiddleware
- 2001
Experimental results obtained with a prototype implementation on an emulated network of up to 100,000 nodes confirm Pastry's scalability and efficiency, its ability to self-organize and adapt to node failures, and its good network locality properties.
Designing a global name service
- Computer SciencePODC '86
- 1986
The global name service described here is meant to do this for billions of names distributed throughout the world and addresses the problems of high availability, large size, continuing evolution, fault isolation and lack of global trust.
Design and evaluation of a wide-area event notification service
- Computer ScienceTOCS
- 2001
SIENA, an event notification service that is designed and implemented to exhibit both expressiveness and scalability, is presented and the service's interface to applications, the algorithms used by networks of servers to select and deliver event notifications, and the strategies used to optimize performance are described.
Matching events in a content-based subscription system
- Computer SciencePODC '99
- 1999
It is proved that for predicates reducible to conjunctions of elementary tests, the expected time to match a random event is no greater than O(N 1 ) where N is the number of subscriptions, and is a closed-form expression that depends on the number and type of attributes.