On the Delay-Storage Trade-Off in Content Download from Coded Distributed Storage Systems

@article{Joshi2013OnTD,
  title={On the Delay-Storage Trade-Off in Content Download from Coded Distributed Storage Systems},
  author={Gauri Joshi and Yanpei Liu and Emina Soljanin},
  journal={IEEE Journal on Selected Areas in Communications},
  year={2013},
  volume={32},
  pages={989-997}
}
We study how coding in distributed storage reduces expected download time, in addition to providing reliability against disk failures. The expected download time is reduced because when a content file is encoded with redundancy and distributed across multiple disks, reading only a subset of the disks is sufficient for content reconstruction. For the same total storage used, coding exploits the diversity in storage better than simple replication, and hence gives faster download. We use a novel… 
View on IEEE
arxiv.org
182 Citations
Highly Influential Citations
23
Background Citations
106
Methods Citations
48
Results Citations
10

182 Citations

Download Time Analysis for Distributed Storage Codes With Locality and Availability

Techniques for analyzing the expected download time in distributed storage systems that employ systematic availability codes are presented and an M/G/1 queue approximation has a high degree of accuracy.

Reducing Delay of Flexible Download in Coded Distributed Storage System

  • Q. ShuaiV. Li
  • Computer Science
    2016 IEEE Global Communications Conference (GLOBECOM)
  • 2016
This paper proposes the compound read method, characterize its mean download delay in low arrival rate scenario and derive upper and lower bounds on its mean downloading delay in high arrival rates scenario, and proposes a scheme C & K to dynamically take advantage of them according to users' required size of files to reduce themean download delay.

Mean-Field Analysis of Coding Versus Replication in Large Data Storage Systems

A mean-field limit is obtained for the steady-state distribution of the number of file access requests waiting at each server to show that, for a given storage capacity per file, coding strictly outperforms replication at all traffic loads while improving reliability.

Mean-field-analysis of coding versus replication in cloud storage systems

A mean-field limit is obtained for the steady-state distribution of the number of file access requests waiting at each server to show that, for a given storage capacity per file, coding strictly outperforms replication at all traffic loads while improving reliability.

When do the availability codes make the stored data more available?

Although, in principle, higher availability should help in reducing download delays, this is not the case in this scenario and suggests the necessity to further investigate the impact of availability on download latency by exploring different arrival, service, and content access models.

Title Reducing delay of flexible download in coded distributedstorage system

This paper proposes the compound read method, characterize its mean download delay in low arrival rate scenario and derive upper and lower bounds on its mean downloads delay in high arrival rate scenarios, and proposes a scheme C & K to dynamically take advantage of them according to users’ required size of files to reduce themean download delay.

Analyzing the Download Time of Availability Codes

The goal of this paper is to analyze, using a queuing theoretical approach, the download time in storage systems that employ availability codes, and indicates that availability codes can reduce downloads in some settings, but are not always optimal.

Latency Analysis for Distributed Coded Storage Systems

New analytical bounds and approximation techniques for the latency-redundancy tradeoff for a range of system loads and a class of symmetric redundancy schemes, under the assumption of Poisson arrivals, exponential service-rates, and fork-join scheduling policy are introduced.

Analyzing the download time of availability codes

This paper analyzes the average time necessary to download a block of data under the Poisson request arrival model in two service/scheduling scenarios and indicates that availability codes can minimize the download time in some settings, but are not always optimal.

Latency Optimal Storage and Scheduling of Replicated Fragments for Memory Constrained Servers

This work model the fragment download time as an exponential random variable independent and identically distributed for all fragments across all servers, and shows that the mean file download time can be lower bounded in terms of the expected number of useful servers summed over all distinct fragment downloads.
...

References

SHOWING 1-10 OF 43 REFERENCES

Coding for fast content download

It is shown that the download time can be significantly reduced by dividing the content into chunks, encoding it to add redundancy and then distributing it across multiple disks, even for the total storage used.

Network Coding for Distributed Storage Systems

This paper shows how to optimally generate MDS fragments directly from existing fragments in the system, and introduces a new scheme called regenerating codes which use slightly larger fragments than MDS but have lower overall bandwidth use.

Codes can reduce queueing delay in data centers

Although in this work this work focuses on a simplified setting where the storage system stores a single content, the methodology developed can be applied to more general settings with multiple contents and offer insightful guidance to the design of storage systems in data centers and content distribution networks.

Highly Available Distributed Storage Systems

Reliability is achieved by computationally efficient MDS array codes that eliminate single points of failure in the systems, thus providing more reliability and flexibility to the systems.

Update efficient codes for distributed storage

This paper determines mechanisms for distributed storage that are simultaneously repair and update efficient, and establishes the existence of the codes that meet both requirements: require only logarithmic updates when data changes, while simultaneously minimizing repair bandwidth for exact reconstruction.

Explicit construction of optimal exact regenerating codes for distributed storage

Exact Regenerating Codes are introduced, which are regenerating codes possessing the additional property of being able to duplicate the data stored at a failed node, relevant to storage in peer-to-peer systems.

The MDS queue: Analysing the latency performance of erasure codes

This paper studies data storage systems based on MDS codes through the lens of queueing theory, and term the queueing system arising under codes as an “MDS queue,” and employs the framework of the MDS queue to analyse different methods of performing so-called degraded reads in distributed data storage.

Interference Alignment in Regenerating Codes for Distributed Storage: Necessity and Code Constructions

The constructions presented in this paper are the first explicit constructions of regenerating codes that achieve the cut-set bound, and Interference alignment is a theme that runs throughout the paper.

Toward sustainable networking: Storage area networks with network coding

It is illustrated that for enterprise-level SANs energy savings of 20-50% can be realized and it is shown that coding operates differently than the amalgamation of file chunks and energy savings are shown to scale well with striping number.

Power-reduction techniques for data-center storage systems

This survey summarizes, organizes, and integrates a decade of research on power-aware enterprise storage systems, and intends to stimulate integration of different power-reduction techniques in new energy-efficient file and storage systems.