Preserving Relations in Parallel Flow Data Processing

  title={Preserving Relations in Parallel Flow Data Processing},
  author={Tom{\'a}{\vs} {\vC}ejka and Martin Z{\'a}dn{\'i}k},
  booktitle={Autonomous Infrastructure, Management and Security},
Network monitoring produces high volume of data that must be analyzed ideally in near real-time to support network security operations. It is possible to process the data using Big Data frameworks, however, such approach requires adaptation or complete redesign of processing tools to get the same results. This paper elaborates on a parallel processing based on splitting a stream of flow records. The goal is to create subsets of traffic that contain enough information for parallel anomaly… 

The Hierarchial Data Aggregation Method in Backbone Traffic Streaming Analyzing to Ensure Digital Systems Information Security

A new hierarchical method of data aggregation is proposed, which allows to more effectively reducing the data size and speed up the processing of each separate new fragment.



Distributed network traffic feature extraction for a real-time IDS

This work implements one of the challenging modules of an IDS, i.e., feature extraction, and presents the computational results of the same for TCP-based traffic and exploits the Apache Spark framework and Netmap to achieve this.

Scalable hybrid stream and hadoop network analysis system

The deployment of a system designed and implemented at the University of Kentucky that supports analysis of network traffic across the enterprise that addresses problems of scale in existing systems, by using distributed computing methodologies, and is based on a combination of stream and batch processing techniques.

A Spark-Based DDoS Attack Detection Model in Cloud Services

A Spark based analysis model to identify abnormal packets and compute statistics for the detection model on the number of abnormal packets to detect DDoS attack efficiently is proposed.

A study on improvement of internet traffic measurement and analysis using Hadoop system

A Hadoop based traffic monitoring system which performs analysis of multi-terabytes of the internet traffic system in a scalable manner and for in-depth analysis of the problem, the paper shows the different features.

Hashdoop: A MapReduce framework for network anomaly detection

Hashdoop is proposed, a MapReduce framework that splits traffic with a hash function to preserve traffic structures and, hence, profits of distributed computing infrastructures to detect network anomalies.

Toward scalable internet traffic measurement and analysis with Hadoop

This paper presents a Hadoop-based traffic monitoring system that performs IP, TCP, HTTP, and NetFlow analysis of multi-terabytes of Internet traffic in a scalable manner and explains the performance issues related with traffic analysis MapReduce jobs.

NEMEA: A framework for network traffic analysis