Corpus ID: 61506904

Robust and efficient data management for a distributed hash table

@inproceedings{Cates2003RobustAE,
  title={Robust and efficient data management for a distributed hash table},
  author={Josh Cates},
  year={2003}
}
This thesis presents a new design and implementation of the DHash distributed hash table based on erasure encoding. This design is both more robust and more efficient than the previous replication-based implementation [15]. DHash uses erasure coding to store each block as a set of fragments. Erasure coding increases availability while saving storage and communication costs compared to a replication based design. DHash combines Chord's synthetic coordinates with the the set of fragments to… Expand
A distributed hash table
TLDR
Sostenuto, a data maintenance algorithm, lets DHash maintain data durability while minimizing the number of copies of data that the system sends over limited-capacity links. Expand
Storing and managing data in a distributed hash table
TLDR
This thesis describes a new algorithm, Passing Tone, that maintains durability efficiently, in a completely decentralized manner, despite transient and permanent failures, and is the first public DHT to store terabytes of data. Expand
Designing a DHT for Low Latency and High Throughput
TLDR
New techniques that resulted from this exploration include use of latency predictions based on synthetic co-ordinates, efficient integration of lookup routing and data fetching, and a congestion control mechanism suitable for fetching data striped over large numbers of servers. Expand
A DHT-based Backup System
Distributed hash tables have been proposed as a way to simplify the construction of large-scale distributed applications (e.g. [1, 6]). DHTs are completely decentralized systems that provide blockExpand
Resource Reclamation in Distributed Hash Tables
Distributed hash tables (DHTs) are increasingly being proposed as the core communication and storage layer for distributed services in the Internet. This trend spans many traditional distributedExpand
A Comparison of Replication Strategies for Reliable Decentralised Storage
TLDR
This paper presents a comparative analysis of replication algorithms for architectures based upon a specific design of DHT, and presents also a novel maintenance algorithm for dynamic replica placement, and considers the reliability of the resulting designs at the system level. Expand
Reliable Data Storage in Distributed Hash Tables
TLDR
A new dynamic replication algorithm which can operate in unstable environments and compare all replication algorithms through simulation, showing quantitatively the difference between their bandwidth use, fault tolerance and performance. Expand
Replication strategies for reliable decentralised storage
TLDR
This paper presents a comparative analysis of replication algorithms for architectures based upon a specific design of DHT, and presents also a novel maintenance algorithm for dynamic replica placement, and considers the reliability of the resulting designs at the system level. Expand
Approximate Hash-Based Set Reconciliation for Distributed Replica Repair
TLDR
This work develops models to deduce parameters for equally accurate set reconciliation algorithms for replica repair in a distributed system and compares equally accurate instances of two trivial hash-based algorithms, an algorithm using Bloom filters and a Merkle tree based algorithm. Expand
Distributed k-ary System: Algorithms for Distributed Hash Tables
TLDR
This dissertation presents algorithms for data structures called distributed hash tables (DHT) or structured overlay networks, which are used to build scalable self-managing distributed systems, and describes a novel way to place replicas in a DHT, called symmetric replication, that enables parallel recursive lookups. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 42 REFERENCES
Scalable, Distributed Data Structures for Internet Service Construction
TLDR
The distributed hash table simplifies Internet service construction by decoupling service-specific logic from the complexities of persistent, consistent state management, and by allowing services to inherit the necessary service properties from the DDS rather than having to implement the properties themselves. Expand
Wide-area cooperative storage with CFS
The Cooperative File System (CFS) is a new peer-to-peer read-only storage system that provides provable guarantees for the efficiency, robustness, and load-balance of file storage and retrieval. CFSExpand
Pond: The OceanStore Prototype
TLDR
Pond is the OceanStore prototype; it contains many of the features of a complete system including location-independent routing, Byzantine update commitment, push-based update of cached copies through an overlay multicast network, and continuous archiving to erasure-coded form. Expand
Farsite: federated, available, and reliable storage for an incompletely trusted environment
TLDR
The design of Farsite is reported on and the lessons learned by implementing much of that design are reported, including how to locally caching file data, lazily propagating file updates, and varying the duration and granularity of content leases. Expand
Fast and secure distributed read-only file system
TLDR
The self-certifying read- only file system is presented, a content distribution system providing secure, scalable access to public, read-only data, and the overhead of cryptography low on clients, allowing servers to scale to a large number of clients. Expand
Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility
TLDR
The storage management and caching in PAST, a large-scale peer-to-peer persistent storage utility based on a self-organizing, Internet-based overlay network of storage nodes that cooperatively route file queries, store multiple replicas of files, and cache additional copies of popular files, is evaluated. Expand
Sloppy Hashing and Self-Organizing Clusters
TLDR
A new abstraction called a distributed sloppy hash table (DSHT) lets nodes locate nearby copies of a file, regardless of its popularity, without causing hot spots in the indexing infrastructure. Expand
Chord: a scalable peer-to-peer lookup protocol for internet applications
TLDR
Results from theoretical analysis and simulations show that Chord is scalable: Communication cost and the state maintained by each node scale logarithmically with the number of Chord nodes. Expand
Chord: A scalable peer-to-peer lookup service for internet applications
TLDR
Results from theoretical analysis, simulations, and experiments show that Chord is scalable, with communication cost and the state maintained by each node scaling logarithmically with the number of Chord nodes. Expand
Chord: A scalable peer-to-peer lookup service for internet applications
TLDR
Results from theoretical analysis, simulations, and experiments show that Chord is scalable, with communication cost and the state maintained by each node scaling logarithmically with the number of Chord nodes. Expand
...
1
2
3
4
5
...