SDF: software-defined flash for web-scale internet storage systems
@article{Ouyang2014SDFSF, title={SDF: software-defined flash for web-scale internet storage systems}, author={Jian Ouyang and Shiding Lin and Jiang Song and Zhenyu Hou and Yong Wang and Yuanzhen Wang}, journal={Proceedings of the 19th international conference on Architectural support for programming languages and operating systems}, year={2014} }
In the last several years hundreds of thousands of SSDs have been deployed in the data centers of Baidu, China's largest Internet search company. Currently only 40\% or less of the raw bandwidth of the flash memory in the SSDs is delivered by the storage system to the applications. Moreover, because of space over-provisioning in the SSD to accommodate non-sequential or random writes, and additionally, parity coding across flash channels, typically only 50-70\% of the raw capacity of a commodity…
Figures and Tables from this paper
197 Citations
Application-Aware and Software-Defined SSD Scheme for Tencent Large-Scale Storage System
- Computer Science2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS)
- 2016
The results demonstrate that TSSD significantly improves the storage system performance and meets the needs of the Tencent's large-scale business applications.
Simulations of the Software-Defined Flash
- Computer Science
- 2017
This chapter describes a simulation model that can be used to simulate SDF architectures exploiting the Open-Channel standard proposed by theopen-Power initiative.
Gemini: A Novel Hardware and Software Implementation of High-performance PCIe SSD
- Computer ScienceInternational Journal of Parallel Programming
- 2016
This paper proposes a high-performance PCIe SSD, Gemini, and depicts its principles in hardware and software implementation, and an FPGA-based prototype of Gemini with 2 TB storage capacity is implemented for verification.
minFlash: A minimalistic clustered flash array
- Computer Science2016 Design, Automation & Test in Europe Conference & Exhibition (DATE)
- 2016
This paper proposes a minimalistic clustered flash array (minFlash), which exposes a simple, stable, error-free, shared-memory flash interface that enables the host to perform cross-layer flash management optimizations in file systems, databases and other user applications.
LSM-tree based Database System Optimization using Application-Driven Flash Management
- Computer Science
- 2019
IsoKV is presented, an isolation scheme for key-value stores by exploiting internal parallelism in SSDs that reduces interference between I/O requests and improves overall throughput and response time.
A user-visible solid-state storage system with software-defined fusion methods for PCM and NAND flash
- Computer ScienceJ. Syst. Archit.
- 2016
Exploiting Write Heterogeneity of Morphable MLC/SLC SSDs in Datacenters with Service-Level Objectives
- Computer ScienceIEEE Transactions on Computers
- 2017
The proposed SLO-aware morphable SSD design seeks to meet the SLO requirement by deciding the write mode of each write request while minimizing the number of SLC writes, and Experimental results show that the proposed design meets theSLO requirement for all of the tested I/O traces.
Bridging the Gap between Application and Solid-State-Drives
- Computer Science
- 2018
This dissertation will improve the data storage performance from both top-down and bottomup approach, and investigate exposing the storage level parallelism so that applications can avoid I/O contentions and workloads skew when scheduling the jobs.
Design of a Host Interface Logic for GC-Free SSDs
- Computer ScienceIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
- 2020
A novel host interface I/O scheduler that is both GC aware and QoS aware is proposed that reduces the standard deviation for latency and the worst-case latency and avoids channel resource conflicts and reduces latency in comparison to the state-of-the-art I/o schedulers.
Flashield: a Hybrid Key-value Cache that Controls Flash Write Amplification
- Computer ScienceNSDI
- 2019
This work presents Flashield, a hybrid key-value cache that uses DRAM as a “filter” to control and limit writes to SSD, and designs a novel in-memory index for the variable-sized objects stored on flash that requires only 4 bytes per object in DRAM.
References
SHOWING 1-10 OF 19 REFERENCES
SFS: random write considered harmful in solid state drives
- Computer ScienceFAST
- 2012
A new file system for SSDs, SFS, which exploits the maximum write bandwidth of SSD by taking a log-structured approach and transforms all random writes at file system level to sequential ones at SSD level.
Towards SSD-Ready Enterprise Platforms
- Computer ScienceADMS@VLDB
- 2010
It is found that the majority of platform I/O latency still lies in the SSD and not in system software, and data copies, uncacheable MMIO reads, interrupt processing, and context switches to be the primary contributors ofI/O processing cost.
Essential roles of exploiting internal parallelism of flash memory based solid state drives in high-speed data processing
- Computer Science2011 IEEE 17th International Symposium on High Performance Computer Architecture
- 2011
This work shows that by exploiting internal parallelism, SSD performance is no longer highly sensitive to access patterns, but rather to other factors, such as data access interferences and physical data layout, which allows for significantly increasing data processing throughput.
LAST: locality-aware sector translation for NAND flash memory-based storage systems
- Computer ScienceOPSR
- 2008
A new flash management technique called LAST is proposed which is optimized for access characteristics of general-purpose computing systems and reduces the garbage collection overhead significantly, thus increasing the I/O performance of flash-based storage devices.
CAFTL: A Content-Aware Flash Translation Layer Enhancing the Lifespan of Flash Memory based Solid State Drives
- Computer ScienceFAST
- 2011
A Content-Aware Flash Translation Layer (CAFTL) is proposed to enhance the endurance of SSDs at the device level to reduce write traffic to flash memory by removing unnecessary duplicate writes and extend available free flash memory space by coalescing redundant data in SSDs.
A Flash-Memory Based File System
- Computer ScienceUSENIX
- 1995
A flash memory device driver that supports a conventional UNIX file system transparently writes data to the flash memory system sequentially as a Log-structured File System does and uses a cleaner to collect valid data blocks and reclaim invalid ones by erasing the corresponding flash memory regions.
Active SSD design for energy-efficiency improvement of web-scale data analysis
- Computer ScienceInternational Symposium on Low Power Electronics and Design (ISLPED)
- 2013
The Active SSD design is presented, considering unique features of computation tasks in web-scale data analysis, and an important issue about interference between normal data processing and local computation in Active SSDs is addressed.
SOS: Software-based out-of-order scheduling for high-performance NAND flash-based SSDs
- Computer Science2013 IEEE 29th Symposium on Mass Storage Systems and Technologies (MSST)
- 2013
Experimental results on a prototype SSD show that SOS is effective in improving the overall SSD performance, lowering the average I/O response time by up to 42% over a hardware-based out-of-order flash controller.
JFFS : The Journalling Flash File System
- Computer Science
- 2001
An overview of the restrictions imposed by flash technology and hence the design aims of JFFS, and the implementation of both J FFS and the improvements made in version 2, including compression and more efficient garbage collection are given.
High performance solid state storage under Linux
- Computer Science2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST)
- 2010
It is demonstrated that an SSD can perform with high throughput, high operation rates, and low latency under the most difficult conditions, suggesting that high-performance SSDs can dramatically improve parallel I/O performance for future high performance computing (HPC) systems.