Apache Spark Streaming, Kafka and HarmonicIO: A Performance Benchmark and Architecture Comparison for Enterprise and Scientific Computing

@inproceedings{Blamey2019ApacheSS,
  title={Apache Spark Streaming, Kafka and HarmonicIO: A Performance Benchmark and Architecture Comparison for Enterprise and Scientific Computing},
  author={Ben Blamey and Andreas Hellander and Salman Toor},
  booktitle={Bench},
  year={2019}
}
  • Ben Blamey, Andreas Hellander, Salman Toor
  • Published in Bench 2019
  • Computer Science
  • This paper presents a benchmark of stream processing throughput comparing Apache Spark Streaming (under file-, TCP socket- and Kafka-based stream integration), with a prototype P2P stream processing framework, HarmonicIO. [...] Key Result Based on these results, we suggest which frameworks and streaming sources are likely to offer good performance for a given load.Expand Abstract

    Citations

    Publications citing this paper.
    SHOWING 1-3 OF 3 CITATIONS

    Adapting the Secretary Hiring Problem for Optimal Hot-Cold Tier Placement Under Top-K Workloads

    Design and Implementation of High Performance Advertising Deduction System

    • Yunjie Qiu
    • Computer Science
    • 2019 International Conference on Computer Network, Electronic and Automation (ICCNEA)
    • 2019

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 18 REFERENCES

    Benchmarking modern distributed streaming platforms

    VIEW 2 EXCERPTS

    Benchmarking Streaming Computation Engines: Storm, Flink and Spark Streaming

    VIEW 1 EXCERPT

    HarmonicIO: Scalable Data Stream Processing for Scientific Datasets

    VIEW 4 EXCERPTS

    Apache Flink™: Stream and Batch Processing in a Single Engine

    VIEW 8 EXCERPTS
    HIGHLY INFLUENTIAL

    SNIC Science Cloud (SSC): A National-Scale Cloud Infrastructure for Swedish Academia

    Apache Spark

    VIEW 2 EXCERPTS