R-Storm: Resource-Aware Scheduling in Storm

@article{Peng2015RStormRS,
  title={R-Storm: Resource-Aware Scheduling in Storm},
  author={Boyang Peng and Mohammad Hosseini and Zhihao Hong and Reza Farivar and Roy H. Campbell},
  journal={ArXiv},
  year={2015},
  volume={abs/1904.05456}
}
The era of big data has led to the emergence of new systems for real-time distributed stream processing, e.g., Apache Storm is one of the most popular stream processing systems in industry today. However, Storm, like many other stream processing systems lacks an intelligent scheduling mechanism. The default round-robin scheduling currently deployed in Storm disregards resource demands and availability, and can therefore be inefficient at times. We present R-Storm (Resource-Aware Storm), a… CONTINUE READING

Figures, Results, and Topics from this paper.

Key Quantitative Results

  • From our experimental results we conclude that R-Storm achieves 30-47% higher throughput and 69-350% better CPU utilization than default Storm for the micro-benchmarks.
  • R-Storm outperforms default Storm by around 50% based on overall throughput.
  • From our experimental results we conclude that R-Storm achieves 30-47% higher throughput and 69-350% better CPU utiliza­tion than default Storm for the micro-benchmarks.
  • On average, the Page Load and Processing Topologies have 50% and 47% better overall throughput, respectively, when scheduled by R-Storm as compared to Storm s default scheduler.
  • From our experimental results we conclude that R-Storm achieves 30-47% higher throughput and 69-350% better CPU utilization than de­fault Storm for the micro-benchmarks topologies.
  • Storm topologies, R-Storm outperforms default Storm by around 50% based on overall throughput.

Citations

Publications citing this paper.
SHOWING 1-10 OF 82 CITATIONS, ESTIMATED 92% COVERAGE

Performance-Oriented Deployment of Streaming Applications on Cloud

  • IEEE Transactions on Big Data
  • 2019
VIEW 15 EXCERPTS
CITES BACKGROUND, RESULTS & METHODS
HIGHLY INFLUENCED

Task Scheduling in Data Stream Processing Systems

VIEW 10 EXCERPTS
CITES BACKGROUND, METHODS & RESULTS
HIGHLY INFLUENCED

Into the Storm: Descrying Optimal Configurations Using Genetic Algorithms and Bayesian Optimization

  • 2017 IEEE 2nd International Workshops on Foundations and Applications of Self* Systems (FAS*W)
  • 2017
VIEW 4 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

An Adaptive Online Scheme for Scheduling and Resource Enforcement in Storm

  • IEEE/ACM Transactions on Networking
  • 2019
VIEW 5 EXCERPTS
CITES BACKGROUND & RESULTS
HIGHLY INFLUENCED

DABS-Storm: A Data-Aware Approach for Elastic Stream Processing

  • T. Large-Scale Data- and Knowledge-Centered Systems
  • 2019
VIEW 7 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

On the Cost of Acking in Data Stream Processing Systems

  • 2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)
  • 2019
VIEW 7 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

A Preventive Auto-Parallelization Approach for Elastic Stream Processing

  • 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS)
  • 2017
VIEW 5 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

D-Storm: Dynamic Resource-Efficient Scheduling of Stream Processing Applications

  • 2017 IEEE 23rd International Conference on Parallel and Distributed Systems (ICPADS)
  • 2017
VIEW 4 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

FILTER CITATIONS BY YEAR

2015
2019

CITATION STATISTICS

  • 23 Highly Influenced Citations

  • Averaged 22 Citations per year from 2017 through 2019

  • 12% Increase in citations per year in 2019 over 2018