• Corpus ID: 239010038

Robust Change Detection for Large-Scale Data Streams

  title={Robust Change Detection for Large-Scale Data Streams},
  author={Ruizhi Zhang and Yajun Mei and Jianjun Shi},
Robust change-point detection for large-scale data streams has many real-world applications in industrial quality control, signal detection, biosurveillance. Unfortunately, it is highly non-trivial to develop efficient schemes due to three challenges: (1) the unknown sparse subset of affected data streams, (2) the unexpected outliers, and (3) computational scalability for real-time monitoring and detection. In this article, we develop a family of efficient real-time robust detection schemes for… 

Figures and Tables from this paper


Scalable SUM-Shrinkage Schemes for Distributed Monitoring Large-Scale Data Streams
This article proposes to develop scalable global monitoring schemes by parallel running local detection procedures and by combining these local procedures together to make a global decision based on SUM-shrinkage techniques.
Optimal sequential detection in multi-stream data
Consider a large number of detectors each generating a data stream. The task is to detect online, distribution changes in a small fraction of the data streams. Previous approaches to this problem
Sequential multi-sensor change-point detection
  • Yao Xie, D. Siegmund
  • Mathematics, Computer Science
    2013 Information Theory and Applications Workshop (ITA)
  • 2013
We develop a mixture procedure to monitor parallel streams of data for a change-point that affects only a subset of them, without assuming a spatial structure relating the data streams to one
Efficient scalable schemes for monitoring a large number of data streams
The sequential changepoint detection problem is studied in the context of global online monitoring of a large number of independent data streams. We are interested in detecting an occurring event as
Asymptotic Statistical Properties of Communication-Efficient Quickest Detection Schemes in Sensor Networks
The quickest change detection problem is studied in a general context of monitoring a large number K of data streams in sensor networks when the “trigger event” may affect different sensors
An online kernel change detection algorithm
This work considers the case where no accurate nor tractable model can be found, using a model-free approach, called Kernel change detection (KCD), and builds a dissimilarity measure in feature space between two sets of descriptors, shown to be asymptotically equivalent to the Fisher ratio in the Gaussian case.
Efficient Computer Network Anomaly Detection by Changepoint Detection Methods
A novel score-based multi-cyclic detection algorithm based on the Shiryaev-Roberts procedure, which is as easy to employ in practice and as computationally inexpensive as the popular Cumulative Sum chart and the Exponentially Weighted Moving Average scheme is proposed.
Minimax Robust Quickest Change Detection
A robust version of these quickest change detection problems is considered when the pre-change and post-change distributions are not known exactly but belong to known uncertainty classes of distributions, such that the detection rule designed for the LFDs is optimal for the robust problem in a minimax sense.
Statistical Challenges Facing Early Outbreak Detection in Biosurveillance
This work focuses mainly on the monitoring of time series to provide early alerts of anomalies to stimulate investigation of potential outbreaks, with a brief summary of methods to detect significant spatial and spatiotemporal case clusters.
Asymptotically Optimal Quickest Change Detection in Distributed Sensor Systems
Abstract In the standard formulation of the quickest change-point detection problem, a sequence of observations, whose distribution changes at some unknown point in time, is available to a decision