Characterization and Optimization of Memory-Resident MapReduce on HPC Systems

@article{Wang2014CharacterizationAO,
  title={Characterization and Optimization of Memory-Resident MapReduce on HPC Systems},
  author={Y. Wang and R. Goldstone and Weikuan Yu and T. Wang},
  journal={2014 IEEE 28th International Parallel and Distributed Processing Symposium},
  year={2014},
  pages={799-808}
}
  • Y. Wang, R. Goldstone, +1 author T. Wang
  • Published 2014
  • Computer Science
  • 2014 IEEE 28th International Parallel and Distributed Processing Symposium
MapReduce is a widely accepted framework for addressing big data challenges. Recently, it has also gained broad attention from scientists at the U.S. leadership computing facilities as a promising solution to process gigantic simulation results. However, conventional high-end computing systems are constructed based on the compute-centric paradigm while big data analytics applications prefer a data-centric paradigm such as MapReduce. This work characterizes the performance impact of key… Expand
57 Citations
Can Non-volatile Memory Benefit MapReduce Applications on HPC Clusters?
  • 4
  • PDF
Memory-Efficient and Skew-Tolerant MapReduce Over MPI for Supercomputing Systems
  • 2
  • PDF
Accelerating big data analytics on HPC clusters using two-level storage
  • 16
Horme: Random Access Big Data Analytics
  • 1
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 27 REFERENCES
Cloud Analytics: Do We Really Need to Reinvent the Storage Stack?
  • 88
  • PDF
MapReduce: Simplified Data Processing on Large Clusters
  • 21,228
Evaluation of HPC Applications on Cloud
  • 132
  • PDF
Shark: SQL and rich analytics at scale
  • 437
  • PDF
CooMR: Cross-task coordination for efficient data management in MapReduce programs
  • 16
  • PDF
Hadoop acceleration through network levitated merge
  • 120
  • PDF
A comparative study of high-performance computing on the cloud
  • 60
  • PDF
...
1
2
3
...