Toward Scalable and Asynchronous Object-Centric Data Management for HPC

@article{Tang2018TowardSA,
  title={Toward Scalable and Asynchronous Object-Centric Data Management for HPC},
  author={Houjun Tang and Surendra Byna and François Tessier and Teng Wang and Bin Dong and Jingqing Mu and Quincey Koziol and J{\'e}rome Soumagne and Venkatram Vishwanath and Jialin Liu and Richard Warren},
  journal={2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)},
  year={2018},
  pages={113-122}
}
  • Houjun Tang, S. Byna, R. Warren
  • Published 1 May 2018
  • Computer Science
  • 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)
Emerging high performance computing (HPC) systems are expected to be deployed with an unprecedented level of complexity due to a deep system memory and storage hierarchy. Efficient and scalable methods of data management and movement through this hierarchy is critical for scientific applications using exascale systems. Moving toward new paradigms for scalable I/O in the extreme-scale era, we introduce novel object-centric data abstractions and storage mechanisms that take advantage of the deep… 
Analysis in the Data Path of an Object-Centric Data Management System
TLDR
This paper proposes in locus analysis that allows registering user-defined functions (UDFs) and running those functions automatically while the data is moving between levels of a storage hierarchy, and implements this analysis in the data path approach in the authors' object-centric data management system, called Proactive Data Containers (PDC).
Tuning Object-Centric Data Management Systems for Large Scale Scientific Applications
  • Houjun Tang, S. Byna, Bin Dong
  • Computer Science
    2019 IEEE 26th International Conference on High Performance Computing, Data, and Analytics (HiPC)
  • 2019
TLDR
This paper explores and evaluates various well-known I/O tuning techniques on a new ODMS called Proactive Data Containers (PDC), and demonstrates that the benefits of these tuning methods with up to 9X I/o performance speedup over the previous version of PDC, and 47X over a highly optimized HDF5 implementation.
A Transparent Server-Managed Object Storage System for HPC
TLDR
A novel object-based data abstraction that takes advantage of deep memory hierarchies by providing a simplified programming interface that enables autonomous, asynchronous, and transparent data movement with a server-driven architecture is described.
UniviStor: Integrated Hierarchical and Distributed Storage for HPC
TLDR
UniviStor is introduced, a data management service offering a unified view of storage layers that provides performance optimizations and data structures tailored for distributed and hierarchical data placement, interferenceaware data movement scheduling, adaptive data striping, and lightweight workflow management.
Interfacing HDF5 with a scalable object‐centric storage system on hierarchical storage
TLDR
An HDF5 VOL connector that interfaces with PDC is developed and its performance is evaluated on Cori, a Cray XC40 supercomputer located at the National Energy Research Scientific Computing Center (NERSC).
HCompress: Hierarchical Data Compression for Multi-Tiered Storage Environments
TLDR
HCompress is a hierarchical data compression library that can improve the application’s performance by harmoniously leveraging both multi-tiered storage and data compression, and has been developed a novel compression selection algorithm that facilitates the optimal matching of compression libraries to the tiered storage.
Parallel Query Service for Object-centric Data Management Systems
TLDR
This paper introduces a parallel query service, called PDC-Query, for an object data management systems (ODMS) on HPC systems, which operates on partitioned objects in parallel, and provides several optimization strategies for fast query evaluation.
Enabling Transparent Asynchronous I/O using Background Threads
TLDR
The prototype asynchronous I/O implementation as an HDF5 VOL connector demonstrates the effectiveness of hiding theI/O cost from the application with low overhead and easy-to-use programming interface.
Demystifying asynchronous I/O Interference in HPC applications
TLDR
The causes and consequences of interference due to asynchronous I/O on HPC systems are investigated, and an in-depth study is performed to explain the interplay and contention in a variety of resource sharing scenarios such as sendfile, read/write, mmap/write underlining trade-offs.
Profiling the Usage of an Extreme-Scale Archival Storage System
  • Hyogi Sim, Sudharshan S. Vazhkudai
  • Computer Science
    2019 IEEE 27th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)
  • 2019
TLDR
This paper analyzes eight years worth of data transfer logs for accessing the archival file system (HPSS) in the Oak Ridge Leadership Computing Facility (OLCF) to derive insights into the future evolution of the archive in terms of provisioning, desired features and functionality from the archive software.
...
1
2
...

References

SHOWING 1-10 OF 34 REFERENCES
SoMeta: Scalable Object-Centric Metadata Management for High Performance Computing
TLDR
SoMeta is presented, a scalable and decentralized metadata management approach for object-centric storage in HPC systems that provides a flat namespace that is dynamically partitioned, a tagging approach to manage metadata that can be efficiently searched and updated, and a light-weight and fault tolerant management strategy.
Data Elevator: Low-Contention Data Movement in Hierarchical Storage System
  • Bin Dong, S. Byna, N. Keen
  • Computer Science
    2016 IEEE 23rd International Conference on High Performance Computing (HiPC)
  • 2016
TLDR
This paper proposes a new system, named Data Elevator, for transparently and efficiently moving data in hierarchical storage, which reduces the resource contention on BB servers via offloading the data movement from a fixed number of BB server nodes to compute nodes.
Exploring memory hierarchy and network topology for runtime AMR data sharing across scientific applications
TLDR
Results show the proposed framework's spatial access pattern detection and prefetching methods demonstrate about 26% performance improvement for client analytical processes and the framework's topology-aware data placement can improve overall data access performance by up to 18%.
Rebasing I/O for Scientific Computing: Leveraging Storage Class Memory in an IBM BlueGene/Q Supercomputer
TLDR
The novel DSA strategy enables a low-overhead, byte addressable, asynchronous, kernel by-pass access method for very high user space IOPs in multithreaded application environments and is exposed through HDF5 using a custom file driver.
TAPIOCA: An I/O Library for Optimized Topology-Aware Data Aggregation on Large-Scale Supercomputers
TLDR
This paper shows how TAPIOCA can take advantage of double-buffering and one-sided communication to reduce as much as possible the idle time during data aggregation, and introduces the cost model leading to a topology-aware aggregator placement optimizing the movements of data.
DAOS and Friends: A Proposal for an Exascale Storage System
TLDR
Each layer of the proposed stack of the DOE Extreme-Scale Technology Acceleration Fast Forward Storage and IO Stack is examined in some detail along with cross-cutting topics, such as transactions and metadata management.
Triple-A: a Non-SSD based autonomic all-flash array for high performance storage systems
TLDR
Triple-A, a non-SSD based Autonomic All-Flash Array, which is a self-optimizing, from-scratch NAND flash cluster, which can detect two different types of resource contentions and autonomically alleviate them by reshaping the physical data-layout on its flash array network.
Ceph: a scalable, high-performance distributed file system
TLDR
Performance measurements under a variety of workloads show that Ceph has excellent I/O performance and scalable metadata management, supporting more than 250,000 metadata operations per second.
Exploring Memory Hierarchy to Improve Scientific Data Read Performance
TLDR
This paper proposes a framework that exploits the memory hierarchy resource to address the read contention issues involved with SSDs and achieves up to 50% read performance improvement when tested on datasets from real-world scientific simulations.
Parallel netCDF: A High-Performance Scientific I/O Interface
TLDR
This work presents a new parallel interface for writing and reading netCDF datasets that defines semantics for parallel access and is tailored for high performance, and compares the implementation strategies and performance with HDF5.
...
1
2
3
4
...