ZHT: A Light-Weight Reliable Persistent Dynamic Scalable Zero-Hop Distributed Hash Table

@article{Li2013ZHTAL,
  title={ZHT: A Light-Weight Reliable Persistent Dynamic Scalable Zero-Hop Distributed Hash Table},
  author={Tonglin Li and Xiaobing Zhou and Kevin Brandstatter and Dongfang Zhao and Ke Wang and Anupam Rajendran and Zhao Zhang and Ioan Raicu},
  journal={2013 IEEE 27th International Symposium on Parallel and Distributed Processing},
  year={2013},
  pages={775-787}
}
  • Tonglin Li, Xiaobing Zhou, I. Raicu
  • Published 20 May 2013
  • Computer Science
  • 2013 IEEE 27th International Symposium on Parallel and Distributed Processing
This paper presents ZHT, a zero-hop distributed hash table, which has been tuned for the requirements of high-end computing systems. ZHT aims to be a building block for future distributed systems, such as parallel and distributed file systems, distributed job management systems, and parallel programming systems. The goals of ZHT are delivering high availability, good fault tolerance, high throughput, and low latencies, at extreme scales of millions of nodes. ZHT has some important properties… 
A convergence of key‐value storage systems from clouds to supercomputers
TLDR
This paper presents ZHT, a zero‐hop distributed key‐value store system, which has been tuned for the requirements of high‐end computing systems and aims to be a building block for future distributed systems, such as parallel and distributed file systems, distributed job management systems, and parallel programming systems.
A CONVERGENCE OF NOSQL STORAGE SYSTEMS FROM CLOUDS TO SUPERCOMPUTERS
TLDR
This work presents ZHT, a zero-hop distributed keyvalue store system, which has been tuned for the requirements of high-end computing systems, and compared it against other key/value stores and found it offers superior performance for the features and portability it supports.
HyCache+: Towards Scalable High-Performance Caching Middleware for Parallel File Systems
TLDR
A distributed storage middleware right on the compute nodes, which allows I/O to effectively leverage the high bi-section bandwidth of the high-speed interconnect of massively parallel high-end computing systems, and a 2-phase mechanism to cache the hot data for parallel applications, called 2-Layer Scheduling (2LS).
FaBRiQ: Leveraging Distributed Hash Tables towards Distributed Publish-Subscribe Message Queues
TLDR
The design goal of Fabriq is to achieve lower latency and higher efficiency while being able to handle large scales, and the framework outperforms other state of the art systems including Kafka and SQS in throughput and latency.
Towards cost-effective and high-performance caching middleware for distributed systems
TLDR
This paper designs and implements a user-level caching system that offers SSD-like performance at a cost similar to a HDD, and demonstrates how such a middleware improves the performance of distributed file systems, such as the HDFS.
OHT : Hierarchical Distributed Hash Tables
TLDR
OHT, a hierarchical distributed hash table, which improves the performance of practical ZHT, is presented, to give a problem statement, describe related work, and illustrate the design idea and implementation details of OHT.
Achieving Efficient Distributed Scheduling with Message Queues in the Cloud for Many-Task Computing and High-Performance Computing
TLDR
The results show that Cloud Kon delivers better scalability compared to other state-of-the-art systems for some metrics - all with a significantly smaller code-base (5%).
Storage Support for Data-Intensive Applications on Large Scale High-Performance Computing Systems
TLDR
This work proposes a new architecture with nodelocal persistent storage, called FusionFS, with two major design principles: maximal metadata concurrency and optimal file write, both of which are crucial to HPC applications.
MATRIX : MAny-Task computing execution fabRIc at eXascale
TLDR
This paper presents the design, analysis and implementation of a distributed execution fabric called MATRIX (MAny-Task computing execution fabRIc at eXascale), which utilizes an adaptive work stealing algorithm for distributed load balancing, and distributed hash tables for managing task metadata.
MetaKV: A Key-Value Store for Metadata Management of Distributed Burst Buffers
  • Teng Wang, A. Moody, Weikuan Yu
  • Computer Science
    2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
  • 2017
TLDR
MetaKV is proposed: a key-value store that provides fast and scalable metadata management for HPC metadata workloads on distributed burst buffers that complements the functionality of an existing key- Value store with specialized metadata services that efficiently handle bursty and concurrent metadata workloading.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 71 REFERENCES
NoVoHT : a Lightweight Dynamic Persistent NoSQL Key / Value Store
TLDR
The goal was to create a fast persistent key/value store that could be easily integrated and operated in lightweight Linux OS typically found on today's supercomputers, and NoVoHT was designed from the ground up to be lightweight, fast, and dependency-free.
C-MPI : A DHT Implementation for Grid and HPC Environments
TLDR
The implementation described here, called Content-MPI (C- MPI), employs a layered software design built on MPI functionality, and offers a scalable data store that is fault tolerant to the extent of the capability of the facility.
Pastry: Scalable, distributed object location and routing for large-scale peer-to-
TLDR
Experimental results obtained with a prototype implementa tion on a simulated network of up to 100,000 nodes confirm Pastry’s scalability, its ability to selfconfigure and adapt to node failures, and its good network loc ality properties.
PVFS: A Parallel File System for Linux Clusters
TLDR
The design and implementation of PVFS are described and performance results on the Chiba City cluster at Argonne are presented, both for a concurrent read/write workload and for the BTIO benchmark.
Making a case for distributed file systems at Exascale
TLDR
It is proposed that future high-end computing systems be designed with non-volatile memory on every compute nodes, allowing every compute node to actively participate in the metadata and data management and leveraging many-core processors high bisection bandwidth in torus networks.
A scalable content-addressable network
TLDR
The concept of a Content-Addressable Network (CAN) as a distributed infrastructure that provides hash table-like functionality on Internet-like scales is introduced and its scalability, robustness and low-latency properties are demonstrated through simulation.
IStore : Towards High Efficiency , Performance , and Reliability in Distributed Data Storage with Information Dispersal Algorithms
TLDR
The possibility of utilizing erasure coding (a form of information dispersal algorithms) for data redundancy while accelerating its operation with GPUs is explored, showing a 10X higher throughput for the GPU.
Cycloid: a constant-degree and lookup-efficient P2P overlay network
BMI: a network abstraction layer for parallel I/O
TLDR
The buffered message interface (BMI) has low processor overhead, minimal impact on latency, and can improve throughput for parallel file system workloads by as much as 40% compared to other more generic network abstractions.
The case for a versatile storage system
TLDR
This work proposes versatile storage, an application-optimized and highly configurable storage system that harnesses node-local resources, is configured and deployed at application deployment time, and has a lifetime dependent on the application lifetime.
...
1
2
3
4
5
...