Joint optimization of overlapping phases in MapReduce

@article{Lin2013JointOO,
  title={Joint optimization of overlapping phases in MapReduce},
  author={Minghong Lin and Li Zhang and Adam Wierman and Jian Tan},
  journal={SIGMETRICS Performance Evaluation Review},
  year={2013},
  volume={41},
  pages={16-18}
}
MapReduce is a scalable parallel computing framework for big data processing. It exhibits multiple processing phases, and thus an efficient job scheduling mechanism is crucial for ensuring efficient resource utilization. This work studies the scheduling challenge that results from the overlapping of the "map" and "shuffle" phases in MapReduce. We propose a new, general model for this scheduling problem. Further, we prove that scheduling to minimize average response time in this model is… CONTINUE READING

Citations

Publications citing this paper.
SHOWING 1-10 OF 62 CITATIONS

Optimizing MapReduce Framework through Joint Scheduling of Overlapping Phases

VIEW 4 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

Speculation-aware Resource Allocation for Cluster Schedulers

VIEW 6 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

A comprehensive view of Hadoop research - A systematic literature review

VIEW 7 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

Optimizing Network Usage in MapReduce Scheduling ∗

VIEW 10 EXCERPTS
CITES METHODS, BACKGROUND & RESULTS
HIGHLY INFLUENCED

Addressing job processing variability through redundant execution and opportunistic checkpointing: A competitive analysis

VIEW 5 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

Speculation-aware Cluster Scheduling

VIEW 3 EXCERPTS
CITES BACKGROUND & METHODS

Energy- and locality-efficient multi-job scheduling based on MapReduce for heterogeneous datacenter

VIEW 1 EXCERPT
CITES METHODS

FILTER CITATIONS BY YEAR

2013
2019

CITATION STATISTICS

  • 6 Highly Influenced Citations

  • Averaged 9 Citations per year from 2017 through 2019

References

Publications referenced by this paper.