Time-Series Anomaly Detection Service at Microsoft

@article{Ren2019TimeSeriesAD,
  title={Time-Series Anomaly Detection Service at Microsoft},
  author={Hansheng Ren and Bixiong Xu and Yujing Wang and Chao Yi and Congrui Huang and Xiaoyu Kou and Tony Xing and Mao Yang and Jie Tong and Qiang Zhang},
  journal={Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery \& Data Mining},
  year={2019}
}
  • Hansheng Ren, Bixiong Xu, +7 authors Q. Zhang
  • Published 10 June 2019
  • Computer Science, Mathematics
  • Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
Large companies need to monitor various metrics (for example, Page Views and Revenue) of their applications and services in real time. [...] Key Method The pipeline consists of three major modules, including data ingestion, experimentation platform and online compute. To tackle the problem of time-series anomaly detection, we propose a novel algorithm based on Spectral Residual (SR) and Convolutional Neural Network (CNN).Expand
RobustTAD: Robust Time Series Anomaly Detection via Decomposition and Convolutional Neural Networks
TLDR
This paper proposes RobustTAD, a Robust Time series Anomaly Detection framework by integrating robust seasonal-trend decomposition and convolutional neural network for time series data and introduces label-based weight and value- based weight in the loss function by utilizing the unbalanced nature of the time series anomaly detection problem. Expand
NVAE-GAN Based Approach for Unsupervised Time Series Anomaly Detection
TLDR
This work proposes T2IVAE, an unsupervised model based on NVAE for univariate series, transforming 1D time series to 2D image as input, and adopting the reconstruction error to detect anomalies. Expand
An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series
TLDR
A systematic and comprehensive evaluation of unsupervised and semisupervised deep-learning-based methods for anomaly detection and diagnosis on multivariate time series data from cyberphysical systems finds that a simple, channel-wise model--the univariate fully connected auto-encoder, with the dynamic Gaussian scoring function emerges as a winning candidate for both anomaly Detection and diagnosis, beating state-of-the-art algorithms. Expand
One-step Predictive Encoder - Gaussian Segment Model for Time Series Anomaly Detection
TLDR
A novel TSAD method which consists of a bidirectional LSTM (BiLSTM) autoencoder and a subsequent Gaussian segmentation model that can find all three kinds of anomaly points is proposed. Expand
Sequential VAE-LSTM for Anomaly Detection on Time Series
TLDR
This work is the first attempt to integrate unsupervised anomaly detection and trend prediction under one framework and achieves competitive experimental results compared with other state-of-the-art methods on public datasets. Expand
Multivariate Time-series Anomaly Detection via Graph Attention Network
TLDR
This paper proposes a novel self-supervised framework for multivariate time-series anomaly detection that outperforms other state-of-the-art models on three real-world datasets and has good interpretability and is useful for anomaly diagnosis. Expand
A joint model for IT operation series prediction and anomaly detection
TLDR
A joint model Predictor & Anomaly Detector (PAD) is proposed to address these two issues under one framework that leads to the better performance of VAE in anomaly detection than it is trained alone. Expand
RLAD: Time Series Anomaly Detection through Reinforcement Learning and Active Learning
TLDR
A new semi-supervised, time series anomaly detection algorithm that uses deep reinforcement learning (DRL) and active learning to efficiently learn and adapt to anomalies in real-world time series data that outperforms all state-of-art methods. Expand
Unsupervised anomaly detection with LSTM autoencoders using statistical data-filtering
TLDR
An enhanced training algorithm for anomaly detection in unlabelled sequential data such as time-series is developed and a probability criterion based on the classical central limit theorem is introduced that allows evaluation of the likelihood that a data-point is drawn from U . Expand
Automated Model Selection for Time-Series Anomaly Detection
TLDR
This paper presents an automated model selection framework to automatically find the most suitable detection model with proper parameters for the incoming data and incorporates a customized tuning algorithm to flexibly filter anomalies to meet customers' criteria. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 31 REFERENCES
Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications
TLDR
Donut is proposed, an unsupervised anomaly detection algorithm based on VAE that greatly outperforms a state-of-arts supervised ensemble approach and a baseline VAE approach, and its best F-scores range from 0.75 to 0.9 for the studied KPIs from a top global Internet company. Expand
Generic and Scalable Framework for Automated Time-series Anomaly Detection
TLDR
A generic and scalable framework for automated anomaly detection on large scale time-series data and the open-sourcing of the data represents the first of its kind effort to establish the standard benchmark for anomaly detection. Expand
Time Series Anomaly Detection; Detection of anomalous drops with limited features and sparse examples in noisy highly periodic data
TLDR
The goal is to utilize Machine Learning and statistical approaches to classify anomalous drops in periodic, but noisy, traffic patterns and found that using the intersection of the two anomaly detection methods proved to be an effective method of detecting anomalies on almost all of the models. Expand
Assumption-Free Anomaly Detection in Time Series
TLDR
This demonstration will show an online anomaly detection system that does not need to be customized for individual domains, yet performs with exceptionally high precision/recall, based on the recently introduced idea of time series bitmaps. Expand
Opprentice: Towards Practical and Automatic Anomaly Detection Through Machine Learning
TLDR
The proposed system, Opprentice (Operators' apprentice), allows operators to label data in only tens of minutes, while operators traditionally have to spend more than ten days selecting and tuning detectors, which may still turn out not to work in the end. Expand
Anomaly Detection in Streams with Extreme Value Theory
TLDR
This work proposes a new approach to detect outliers in streaming univariate time series based on Extreme Value Theory that does not require to hand-set thresholds and makes no assumption on the distribution: the main parameter is only the risk, controlling the number of false positives. Expand
Network Anomaly Detection Based on Wavelet Analysis
  • Wei Lu, A. Ghorbani
  • Computer Science
  • EURASIP J. Adv. Signal Process.
  • 2009
TLDR
A new network signal modelling technique for detecting network anomalies, combining the wavelet approximation and system identification theory is proposed, which achieves high-detection rates in terms of both attack instances and attack types. Expand
A Novel Technique for Long-Term Anomaly Detection in the Cloud
TLDR
A novel statistical technique to automatically detect long-term anomalies in cloud data that employs statistical learning to detect anomalies in both application as well as system metrics is developed. Expand
Saliency detection by multi-context deep learning
TLDR
This paper proposes a multi-context deep learning framework for salient object detection that employs deep Convolutional Neural Networks to model saliency of objects in images and investigates different pre-training strategies to provide a better initialization for training the deep neural networks. Expand
Spatio-temporal Saliency detection using phase spectrum of quaternion fourier transform
TLDR
A simple and fast approach based on Fourier transform called spectral residual (SR) was proposed, which used SR of the amplitude spectrum to obtain the saliency map, and the results are good, but the reason is questionable. Expand
...
1
2
3
4
...