Scheduling with multi-level data locality: Throughput and heavy-traffic optimality

@article{Xie2016SchedulingWM,
  title={Scheduling with multi-level data locality: Throughput and heavy-traffic optimality},
  author={Qiaomin Xie and Ali Yekkehkhany and Yi Lu},
  journal={IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications},
  year={2016},
  pages={1-9}
}
A fundamental problem to all data-parallel applications is data locality. An example is map task scheduling in the MapReduce framework. Existing theoretical work analyzes systems with only two levels of locality, despite the existence of multiple locality levels within and across data centers. We found that going from two to three levels of locality changes the problem drastically, as a tradeoff between performance and throughput emerges. The recently proposed priority algorithm, which isโ€ฆย CONTINUE READING

Citations

Publications citing this paper.
SHOWING 1-10 OF 20 CITATIONS

Scheduling for data centers with multi-level data locality

  • 2017 Iranian Conference on Electrical Engineering (ICEE)
  • 2017
VIEW 10 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

Optimal physiology-aware scheduling of clinical states in rural ambulance transport

  • 2017 International Conference on Inventive Computing and Informatics (ICICI)
  • 2017
VIEW 4 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

A Dynamic Resource Allocation Method for Load-Balance Scheduling Over Big Data Platforms

  • 2018 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData)
  • 2018
VIEW 1 EXCERPT
CITES BACKGROUND

Analysis of a Parallel/Distributed Application Using a Cycle-Accurate Parallel/Distributed Simulator

  • Electrical Engineering (ICEE), Iranian Conference on
  • 2018
VIEW 1 EXCERPT
CITES BACKGROUND

References

Publications referenced by this paper.
SHOWING 1-10 OF 18 REFERENCES

Priority algorithm for near-data scheduling: Throughput and heavy-traffic optimality

  • 2015 IEEE Conference on Computer Communications (INFOCOM)
  • 2015
VIEW 9 EXCERPTS

Scheduling with multi-level data locality: Throughput and heavy-traffic optimality

Q. Xie, Y. Lu
  • Technical Report,
  • 2015
VIEW 5 EXCERPTS
HIGHLY INFLUENTIAL

Maestro: Replica-Aware Map Scheduling for MapReduce

  • 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
  • 2012
VIEW 1 EXCERPT

BAR: An Efficient Data Locality Driven Task Scheduling Algorithm for Cloud Computing

  • 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing
  • 2011
VIEW 1 EXCERPT

Matchmaking: A New MapReduce Scheduling Technique

  • 2011 IEEE Third International Conference on Cloud Computing Technology and Science
  • 2011
VIEW 1 EXCERPT