Detecting Flow Anomalies in Distributed Systems

  title={Detecting Flow Anomalies in Distributed Systems},
  author={Freddy Chongtat Chua and Ee-Peng Lim and Bernardo A. Huberman},
  journal={2014 IEEE International Conference on Data Mining},
Deep within the networks of distributed systems, one often finds anomalies that affect their efficiency and performance. These anomalies are difficult to detect because the distributed systems may not have sufficient sensors to monitor the flow of traffic within the interconnected nodes of the networks. Without early detection and making corrections, these anomalies may aggravate over time and could possibly cause disastrous outcomes in the system in the unforeseeable future. Using only coarse… 

Where are the passengers?: a grid-based gaussian mixture model for taxi bookings

This paper proposes a Grid-based Gaussian Mixture Model (GGMM) with spatio-temporal dimensions that groups booking data into a number of spatio/temporal clusters by observing the bookings occurring at different time of the day in each spatial grid cell, and shows that GGMM outperforms two strong baselines.

Passenger Travel Patterns and Behavior Analysis of Long-Term Staying in Subway System by Massive Smart Card Data

A method for identifying the Long-term Staying in Subway System (LSSS) in the subway based on the shortest path and analyze its travel mode is proposed and a SAE-DNN algorithm is proposed to identify suspected anomalies.

Inferring the Root Cause in Road Traffic Anomalies

A novel two-step mining and optimization framework for inferring the root cause of anomalies that appear in road traffic data and can discover routes which can clearly explain the appearance of link anomalies is proposed.

Crowd sensing of traffic anomalies based on human mobility and social media

This paper addresses the problem of detecting and describing traffic anomalies using crowd sensing with two forms of data, human mobility and social media, and identifies anomalies according to drivers' routing behavior on an urban road network.

Online detection of network traffic anomalies using behavioral distance

A behavioral distance based anomaly detection mechanism with the capability of performing on-line traffic analysis and validate the efficacy of the detection system by using network traffic traces collected at Abilene and MAWI high-speed links.

Histogram-based traffic anomaly detection

This work describes a new approach to feature-based anomaly detection that constructs histograms of different traffic features, models histogram patterns, and identifies deviations from the created models.

Anomaly detection on ITS data via view association

An anomaly detection method for transportation systems where a police report is created automatically after detecting anomalies is proposed, and two well-known ITS datasets are studied which include the data from Mobile Century project and the PeMS dataset.

Execution Anomaly Detection in Distributed Systems through Unstructured Log Analysis

This paper proposes an unstructured log analysis technique for anomalies detection and proposes a novel algorithm to convert free form text messages in log files to log keys without heavily relying on application specific knowledge.

Fast Distributed Outlier Detection in Mixed-Attribute Data Sets

A tunable algorithm for distributed outlier detection in dynamic mixed-attribute data sets that are prone to concept drift and models of the data must be dynamic as well is presented.

Discovering spatio-temporal causal interactions in traffic data streams

Algorithms which construct outlier causality trees based on temporal and spatial properties of detected outliers reveal not only recurring interactions among spatio-temporal outliers, but potential flaws in the design of existing traffic networks.

Detecting Novel Network Intrusions Using Bayes Estimators

This work has been funded by AFRL Rome Labs under the contract F 30602-00-2-0512 and aims to detect well-known attacks as well as slight variations of them, by characterizing the rules that govern these attacks.

iBAT: detecting anomalous taxi trajectories from GPS traces

An Isolation-Based Anomalous Trajectory (iBAT) detection method is proposed and the potential of iBAT in enabling innovative applications is demonstrated by using it for taxi driving fraud detection and road network change detection.