DenseAlert: Incremental Dense-Subtensor Detection in Tensor Streams

@article{Shin2017DenseAlertID,
  title={DenseAlert: Incremental Dense-Subtensor Detection in Tensor Streams},
  author={Kijung Shin and Bryan Hooi and Jisu Kim and Christos Faloutsos},
  journal={Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining},
  year={2017}
}
  • Kijung ShinBryan Hooi C. Faloutsos
  • Published 11 June 2017
  • Computer Science
  • Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
Consider a stream of retweet events - how can we spot fraudulent lock-step behavior in such multi-aspect data (i.e., tensors) evolving over time? Can we detect it in real time, with an accuracy guarantee? Past studies have shown that dense subtensors tend to indicate anomalous or even fraudulent behavior in many tensor data, including social media, Wikipedia, and TCP dumps. Thus, several algorithms have been proposed for detecting dense subtensors rapidly and accurately. However, existing… 

Figures and Tables from this paper

Detecting Group Anomalies in Tera-Scale Multi-Aspect Data via Dense-Subtensor Mining

D-Cube is proposed, a disk-based dense-subtensor detection method, which also can run in a distributed manner across multiple machines and is Provably Accurate, which gives a guarantee on the densities of the detected subtensors.

Hierarchical Dense Pattern Detection in Tensors

CatchCore outperforms the top competitors in accuracy for detecting dense subtensors and anomaly patterns, like network attacks; it can also capture core collaboration and multi-hop relations around some query objects and scales linearly with all aspects of tensors.

AugSplicing: Synchronized Behavior Detection in Streaming Tensors

A fast streaming algorithm, AUGSPLICING, which can detect the top dense blocks by incrementally splicing the previous detection with the incoming ones in new tuples, avoiding re-runs over all the history data at every tracking time step.

CatchCore: Catching Hierarchical Dense Subtensor

This paper proposes CatchCore, a novel framework to effectively find the hierarchical dense subtensors, and designs a unified metric for dense subtensor detection, which can be optimized with gradient-based methods.

Fast, Accurate, and Flexible Algorithms for Dense Subtensor Mining

This work proposes two algorithms, called M-Zoom and M-Biz, for fast and accurate dense-subtensor detection with various density measures, which give a lower bound on the density and a guarantee on the local optimality of detected subtensors.

A DenseAlert : Incremental Dense-Subtensor Detection in Tensor Streams-Supplementary Document

In this section, we show that the density measure (Definition 1) used in the main paper satisfies properties that a reasonable “anomalousness” measure should meet. These properties were proposed in

MStream: Fast Anomaly Detection in Multi-Aspect Streams

MStream is a streaming multi-aspect data anomaly detection framework which can detect unusual group anomalies as they occur, in a dynamic manner, and outperforms state-of-the-art baselines.

Sketch-Based Streaming Anomaly Detection in Dynamic Graphs

This paper first extends the count-min sketch data structure to a higher-order sketch, which has the useful property of preserving the dense subgraph structure, and proposes four online algorithms that utilize this enhanced data structure that detect both edge and graph anomalies.

MStream: Fast Streaming Multi-Aspect Group Anomaly Detection

This work aims to define a streaming multi-aspect data anomaly detection framework, termed MStream, which can detect unusual group anomalies as they occur, in a dynamic manner, and outperforms state-of-the-art baselines.

Fast and Multi-aspect Mining of Complex Time-stamped Event Streams

The proposed method, CubeScope, captures dynamical multi-aspect patterns, i.e., regimes and components, and statistically summarizes all the events, and is practical for successful application to data compression, pattern discovery, and anomaly detection on various types of tensor streams.
...

References

SHOWING 1-10 OF 35 REFERENCES

D-Cube: Dense-Block Detection in Terabyte-Scale Tensors

D-Cube is proposed, a disk-based dense-block detection method, which also can be run in a distributed manner across multiple machines, and successfully spotted network attacks from TCP dumps and synchronized behavior in rating data with the highest accuracy.

M-Zoom: Fast Dense-Block Detection in Tensors with Quality Guarantees

This work proposes M-Zoom, a flexible framework for finding dense blocks in tensors, which works with a broad class of density measures and provides a guarantee on the lowest density of the blocks it finds.

Efficient Densest Subgraph Computation in Evolving Graphs

This work studies the densest subgraph problem in the the dynamic graph model, for which it is presented the first scalable algorithm with provable guarantees, and shows that (approximate) densmost subgraphs can be maintained efficiently within hundred of microseconds per update.

A General Suspiciousness Metric for Dense Blocks in Multimodal Data

This work gives a list of axioms that any metric of suspicousness should satisfy, proposes CROSSSPOT, an algorithm to spot dense regions, and sort them in importance ("suspiciousness") order, and applies it to real data, where it improves the F1 score over previous techniques by 68% and finds retweet-boosting in a real social dataset spanning 0.3 billion posts.

Accelerating Online CP Decompositions for Higher Order Tensors

This work proposes an efficient online algorithm that can incrementally track the CP decompositions of dynamic tensors with an arbitrary number of dimensions and shows not only significantly better decomposition quality, but also better performance in terms of stability, efficiency and scalability.

Beyond streams and graphs: dynamic tensor analysis

The dynamic tensor analysis (DTA) method, and its variants are introduced, which provides a compact summary for high-order and high-dimensional data, and it also reveals the hidden correlations.

MultiAspectForensics: Pattern Mining on Large-Scale Heterogeneous Networks with Tensor Analysis

This work introduces MultiAspectForensics, a handy tool to automatically detect and visualize novel sub graph patterns within a local community of nodes in a heterogenous network, such as a set of vertices that form a dense bipartite graph whose edges share exactly the same set of attributes.

Densest Subgraph in Streaming and MapReduce

New algorithms for finding the densest subgraph in the streaming model by making O(log1+e n) passes over the input and finding a subgraph whose density is guaranteed to be within a factor 2(1 + e) of the optimum are presented.

CopyCatch: stopping group attacks by spotting lockstep behavior in social networks

The method, which is referred to as CopyCatch, detects lockstep Page Like patterns on Facebook by analyzing only the social graph between users and Pages and the times at which the edges in the graph were created.

Space- and Time-Efficient Algorithm for Maintaining Dense Subgraphs on One-Pass Dynamic Streams

This paper develops an algorithm that is the first streaming algorithm that can maintain the densest subgraph in one pass and can be extended to a (2+ε)-approximation sublinear-time algorithm and a distributed-streaming algorithm.