A two-phase execution engine of reduce tasks in Hadoop MapReduce

@article{Zhang2012ATE,
  title={A two-phase execution engine of reduce tasks in Hadoop MapReduce},
  author={Xiaohong Zhang and Guowei Wang and Zijing Yang and Yang Ding},
  journal={2012 International Conference on Systems and Informatics (ICSAI2012)},
  year={2012},
  pages={858-864}
}
In Hadoop MapReduce, reduce tasks issue massive remote I/O operations to copy the intermediate results of map tasks. The operations cause massive remote data access delays which degrade the system performance. To Handle this problem, we propose an execution engine of reduce tasks. The engine partitions the execution of reduce tasks into two phases. In the first phase, the engine selects the nodes to run reduce tasks, and then order the nodes to prefetch intermediate results for the reduce tasks… CONTINUE READING

References

Publications referenced by this paper.
SHOWING 1-10 OF 11 REFERENCES

LEEN: Locality/Fairness-Aware Key Partitioning for MapReduce in the Cloud

  • 2010 IEEE Second International Conference on Cloud Computing Technology and Science
  • 2010
VIEW 2 EXCERPTS

Locality / fairnessaware key partitioning for mapreduce in the cloud

H. Jin, S. Wu
  • Cloud  Computing  Tech nology and Science , IEEE International Conference on
  • 2010

MapReduce Online

VIEW 1 EXCERPT

Yahoo! launches worlds largest hadoop production

J. Zawodny
  • 2010
VIEW 1 EXCERPT

HPMR: Prefetching and pre-shuffling in shared MapReduce computation environment

  • 2009 IEEE International Conference on Cluster Computing and Workshops
  • 2009
VIEW 2 EXCERPTS

Mars: A MapReduce Framework on graphics processors

  • 2008 International Conference on Parallel Architectures and Compilation Techniques (PACT)
  • 2008
VIEW 1 EXCERPT