Skip to search formSkip to main contentSkip to account menu

Comparative analysis of large data processing in Apache Spark using Java, Python and Scala

It was found that the programming language significantly affects the efficiency of data processing by the Apache Spark algorithm, with Scala and Java being more productive for processing large amounts of data and complex operations, while Python demonstrates an advantage in working with small amounts of data.
arXiv (opens in a new tab)

Performance Benchmarking of Continuous Processing and Micro-Batch Modes in Spark Structured Streaming

The findings indicate that while the Continuous Processing mode offers significantly lower latencies while using Rate source, its performance in high-throughput scenarios using Kafka source may be less consistent, and have practical implications for optimizing text data flow strategies in big data analytics.
View Paper (opens in a new tab)
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy (opens in a new tab), Terms of Service (opens in a new tab), and Dataset License (opens in a new tab)