skip to main content
research-article
Free access

Imputation-based Time-Series Anomaly Detection with Conditional Weight-Incremental Diffusion Models

Published: 04 August 2023 Publication History

Abstract

Existing anomaly detection models for time series are primarily trained with normal-point-dominant data and would become ineffective when anomalous points intensively occur in certain episodes. To solve this problem, we propose a new approach, called DiffAD, from the perspective of time series imputation. Unlike previous prediction- and reconstruction-based methods that adopt either partial or complete data as observed values for estimation, DiffAD uses a density ratio-based strategy to select normal observations flexibly that can easily adapt to the anomaly concentration scenarios. To alleviate the model bias problem in the presence of anomaly concentration, we design a new denoising diffusion-based imputation method to enhance the imputation performance of missing values with conditional weight-incremental diffusion, which can preserve the information of observed values and substantially improves data generation quality for stable anomaly detection. Besides, we customize a multi-scale state space model to capture the long-term dependencies across episodes with different anomaly patterns. Extensive experimental results on real-world datasets show that DiffAD performs better than state-of-the-art benchmarks.

Supplementary Material

MP4 File (rtfp0647-2min-promo.mp4)
The video provides a brief explanation of the background and method in the paper.

References

[1]
Ahmed Abdulaal, Zhuanghua Liu, and Tomer Lancewicki. 2021. Practical approach to asynchronous multivariate time series anomaly detection and localization. In KDD. 2485--2494.
[2]
Samaneh Aminikhanghahi, Tinghui Wang, and Diane J. Cook. 2019. Real-Time Change Point Detection with Application to Smart Home Time Series Data. IEEE Transactions on Knowledge and Data Engineering, Vol. 31, 5 (2019), 1010--1023.
[3]
Julien Audibert, Pietro Michiardi, Frédéric Guyard, Sébastien Marti, and Maria A Zuluaga. 2020. Usad: Unsupervised anomaly detection on multivariate time series. In KDD. 3395--3404.
[4]
Ane Blázquez-Garc'ia, Angel Conde, Usue Mori, and Jose A Lozano. 2021. A review on outlier/anomaly detection in time series data. Comput. Surveys, Vol. 54, 3 (2021), 1--33.
[5]
Mohammad Braei and Sebastian Wagner. 2020. Anomaly detection in univariate time-series: A survey on the state-of-the-art. arXiv (2020).
[6]
Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander. 2000. LOF: identifying density-based local outliers. In SIGMOD. 93--104.
[7]
Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, and Yitan Li. 2018. Brits: Bidirectional recurrent imputation for time series. In NIPS.
[8]
Ashutosh Chandra and Rahul Kala. 2019. Regularised encoder-decoder architecture for anomaly detection in ECG time signals. In ICTC. 1--6.
[9]
Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. 2018. Recurrent neural networks for multivariate time series with missing values. Scientific reports, Vol. 8, 1 (2018), 1--12.
[10]
Andrea Cini, Ivan Marisca, and Cesare Alippi. 2022. Filling the G_ap_s: Multivariate Time Series Imputation by Graph Neural Networks. In ICLR.
[11]
Andrew A Cook, Göksel Misirli, and Zhong Fan. 2019. Anomaly detection for IoT time-series data: A survey. IEEE Internet of Things Journal, Vol. 7, 7 (2019), 6481--6494.
[12]
Ailin Deng and Bryan Hooi. 2021. Graph neural network-based anomaly detection in multivariate time series. In AAAI, Vol. 35. 4027--4035.
[13]
Chenguang Fang and Chen Wang. 2020. Time series data imputation: A survey on deep learning approaches. arXiv (2020).
[14]
Vincent Fortuin, Dmitry Baranchuk, Gunnar R"atsch, and Stephan Mandt. 2020. Gp-vae: Deep probabilistic time series imputation. In AISTATS. 1651--1661.
[15]
Alexander Geiger, Dongyu Liu, Sarah Alnegheimish, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. 2020. TadGAN: Time series anomaly detection using generative adversarial networks. In Big Data. 33--43.
[16]
Karan Goel, Albert Gu, Chris Donahue, and Christopher Re. 2022. Its Raw! Audio Generation with State-Space Models. In ICML. 7616--7633.
[17]
Albert Gu, Karan Goel, and Christopher Re. 2022a. Efficiently Modeling Long Sequences with Structured State Spaces. In ICLR.
[18]
Albert Gu, Ankit Gupta, Karan Goel, and Christopher Ré. 2022b. On the Parameterization and Initialization of Diagonal State Space Models. In NIPS.
[19]
Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. In NIPS. 6840--6851.
[20]
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Soderstrom. 2018. Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In KDD. 387--395.
[21]
Md Mohaiminul Islam and Gedas Bertasius. 2022. Long movie clip classification with state-space video models. In ECCV.
[22]
Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR.
[23]
Dan Li, Dacheng Chen, Baihong Jin, Lei Shi, Jonathan Goh, and See-Kiong Ng. 2019. MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks. In ICANN. 703--716.
[24]
Zhihan Li, Youjian Zhao, Jiaqi Han, Ya Su, Rui Jiao, Xidao Wen, and Dan Pei. 2021. Multivariate time series anomaly detection and interpretation using hierarchical inter-metric and temporal embedding. In KDD. 3220--3230.
[25]
Yukai Liu, Rose Yu, Stephan Zheng, Eric Zhan, and Yisong Yue. 2019. Naomi: Non-autoregressive multiresolution sequence imputation. In NIPS.
[26]
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. 2022. Repaint: Inpainting using denoising diffusion probabilistic models. In CVPR. 11461--11471.
[27]
Yonghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu, et al. 2018. Multivariate time series imputation with generative adversarial networks. In NIPS.
[28]
Yonghong Luo, Ying Zhang, Xiangrui Cai, and Xiaojie Yuan. 2019. E2gan: End-to-end generative adversarial network for multivariate time series imputation. In IJCAI. 3094--3100.
[29]
Aditya P Mathur and Nils Ole Tippenhauer. 2016. SWaT: A water treatment testbed for research and training on ICS security. In CySWater. 31--36.
[30]
Gyoung S Na, Donghyun Kim, and Hwanjo Yu. 2018. Dilof: Effective and memory efficient local outlier detection in data streams. In KDD. 1993--2002.
[31]
Eunkyu Oh, Taehun Kim, Yunhu Ji, and Sushil Khyalia. 2021. STING: Self-attention based Time-series Imputation Networks using GAN. In ICDM.
[32]
Daehyung Park, Yuuna Hoshi, and Charles C Kemp. 2018. A multimodal anomaly detector for robot-assisted feeding using an lstm-based variational autoencoder. IEEE Robotics and Automation Letters, Vol. 3, 3 (2018), 1544--1551.
[33]
Tal Peer, Simon Welker, and Timo Gerkmann. 2023. DiffPhase: Generative Diffusion-based STFT Phase Retrieval. In ICASSP. 7402--7406.
[34]
Siddharth Ramchandran, Gleb Tikhonov, Kalle Kujanp"a"a, Miika Koskinen, and Harri L"ahdesm"aki. 2021. Longitudinal variational autoencoder. In AISTATS. 3898--3906.
[35]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In MICCAI. 234--241.
[36]
Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Emmanuel Müller, and Marius Kloft. 2018. Deep one-class classification. In ICML. 4393--4402.
[37]
Sebastian Schmidl, Phillip Wenig, and Thorsten Papenbrock. 2022. Anomaly detection in time series: a comprehensive evaluation. VLDB, Vol. 15, 9 (2022), 1779--1797.
[38]
Bernhard Schölkopf, John C Platt, John Shawe-Taylor, Alex J Smola, and Robert C Williamson. 2001. Estimating the support of a high-dimensional distribution. Neural computation, Vol. 13, 7 (2001), 1443--1471.
[39]
Lifeng Shen, Zhuocong Li, and James Kwok. 2020. Timeseries anomaly detection using temporal hierarchical one-class network. In NIPS, Vol. 33. 13016--13026.
[40]
Youjin Shin, Sangyup Lee, Shahroz Tariq, Myeong Shin Lee, Okchul Jung, Daewon Chung, and Simon S Woo. 2020. Itad: integrative tensor-based anomaly detection system for reducing false positives of satellite systems. In CIKM. 2733--2740.
[41]
Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. 2019. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In KDD. 2828--2837.
[42]
Shahroz Tariq, Sangyup Lee, Youjin Shin, Myeong Shin Lee, Okchul Jung, Daewon Chung, and Simon S Woo. 2019. Detecting anomalies in space using multivariate convolutional LSTM with mixtures of probabilistic PCA. In KDD. 2123--2133.
[43]
Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. 2021. CSDI: Conditional score-based diffusion models for probabilistic time series imputation. In NIPS.
[44]
Luan Tran, Min Y Mun, and Cyrus Shahabi. 2020. Real-time distance-based outlier detection in data streams. VLDB, Vol. 14, 2 (2020), 141--153.
[45]
Charles Truong, Laurent Oudre, and Nicolas Vayatis. 2020. Selective review of offline change point detection methods. Signal Processing, Vol. 167 (2020), 107299.
[46]
Shreshth Tuli, Giuliano Casale, and Nicholas R Jennings. 2022. TranAD: deep transformer networks for anomaly detection in multivariate time series data. VLDB, Vol. 15, 6 (2022), 1201--1214.
[47]
Chunjing Xiao, Shiming Chen, Fan Zhou, and Jie Wu. 2022. Self-Supervised Few-Shot Time-series Segmentation for Activity Recognition. IEEE Transactions on Mobile Computing (2022).
[48]
Haowen Xu, Wenxiao Chen, Nengwen Zhao, Zeyan Li, Jiahao Bu, Zhihan Li, Ying Liu, Youjian Zhao, Dan Pei, Yang Feng, et al. 2018. Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications. In WWW. 187--196.
[49]
Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. 2022. Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy. In ICLR.
[50]
Takehisa Yairi, Naoya Takeishi, Tetsuo Oda, Yuta Nakajima, Naoki Nishimura, and Noboru Takata. 2017. A data-driven health monitoring method for satellite housekeeping data based on probabilistic clustering and dimensionality reduction. IEEE Trans. Aerospace Electron. Systems, Vol. 53, 3 (2017), 1384--1401.
[51]
Yueyue Yao, Jianghong Ma, and Yunming Ye. 2022. KfreqGAN: Unsupervised detection of sequence anomaly with adversarial learning and frequency domain information. Knowledge-Based Systems, Vol. 236 (2022), 107757.
[52]
Jinsung Yoon, William R Zame, and Mihaela van der Schaar. 2018. Estimating missing data in temporal data streams using multi-directional recurrent neural networks. IEEE Transactions on Biomedical Engineering, Vol. 66, 5 (2018), 1477--1490.
[53]
Susik Yoon, Jae-Gil Lee, and Byung Suk Lee. 2020. Ultrafast local outlier detection from a data stream with stationary region skipping. In KDD. 1181--1191.
[54]
Sergey Zagoruyko and Nikos Komodakis. 2016. Wide Residual Networks. In British Machine Vision Conference.
[55]
Aoqian Zhang, Shaoxu Song, and Jianmin Wang. 2016. Sequential data cleaning: A statistical approach. In SIGMOD. 909--924.
[56]
Weiqi Zhang, Chen Zhang, and Fugee Tsung. 2022. GRELEN: Multivariate Time Series Anomaly Detection from the Perspective of Graph Relational Learning. In IJCAI. 2390--2397.
[57]
Yuxin Zhang, Yiqiang Chen, Jindong Wang, and Zhiwen Pan. 2023. Unsupervised Deep Anomaly Detection for Multi-Sensor Time-Series Signals. IEEE Transactions on Knowledge and Data Engineering, Vol. 35, 2 (2023), 2118--2132.
[58]
Hang Zhao, Yujing Wang, Juanyong Duan, Congrui Huang, Defu Cao, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, and Qi Zhang. 2020. Multivariate time-series anomaly detection via graph attention network. In ICDM. 841--850.
[59]
Bin Zhou, Shenghua Liu, Bryan Hooi, Xueqi Cheng, and Jing Ye. 2019. BeatGAN: Anomalous Rhythm Detection using Adversarially Generated Time Series. In IJCAI. 4433--4439.
[60]
Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, and Haifeng Chen. 2018. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In ICLR. io

Cited By

View all
  • (2024)A Study on Network Anomaly Detection Using Fast Persistent Contrastive DivergenceSymmetry10.3390/sym1609122016:9(1220)Online publication date: 17-Sep-2024
  • (2024)Leveraging Large Language Models and BERT for Log Parsing and Anomaly DetectionMathematics10.3390/math1217275812:17(2758)Online publication date: 5-Sep-2024
  • (2024)Multi-Behavior Collaborative Filtering with Partial Order Graph Convolutional NetworksProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671569(6257-6268)Online publication date: 25-Aug-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
August 2023
5996 pages
ISBN:9798400701030
DOI:10.1145/3580305
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 August 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. data imputation
  2. diffusion models
  3. state space model
  4. time series

Qualifiers

  • Research-article

Funding Sources

Conference

KDD '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1,851
  • Downloads (Last 6 weeks)187
Reflects downloads up to 23 Oct 2024

Other Metrics

Citations

Cited By

View all
  • (2024)A Study on Network Anomaly Detection Using Fast Persistent Contrastive DivergenceSymmetry10.3390/sym1609122016:9(1220)Online publication date: 17-Sep-2024
  • (2024)Leveraging Large Language Models and BERT for Log Parsing and Anomaly DetectionMathematics10.3390/math1217275812:17(2758)Online publication date: 5-Sep-2024
  • (2024)Multi-Behavior Collaborative Filtering with Partial Order Graph Convolutional NetworksProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671569(6257-6268)Online publication date: 25-Aug-2024
  • (2024)Diffusion-Model-Based Contrastive Learning for Human Activity RecognitionIEEE Internet of Things Journal10.1109/JIOT.2024.342924511:20(33525-33536)Online publication date: 15-Oct-2024
  • (2024)ProDiffAD: Progressively Distilled Diffusion Models for Multivariate Time Series Anomaly Detection in JointCloud Environment2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650364(1-8)Online publication date: 30-Jun-2024
  • (2024)A robust multi-scale feature extraction framework with dual memory module for multivariate time series anomaly detectionNeural Networks10.1016/j.neunet.2024.106395177:COnline publication date: 1-Sep-2024
  • (2024)Controlled graph neural networks with denoising diffusion for anomaly detectionExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.121533237:PBOnline publication date: 1-Feb-2024
  • (2024)Unsupervised diffusion based anomaly detection for time seriesApplied Intelligence10.1007/s10489-024-05341-054:19(8968-8981)Online publication date: 8-Jul-2024
  • (2024)Dynamic Splitting of Diffusion Models for Multivariate Time Series Anomaly Detection in a JointCloud EnvironmentKnowledge Science, Engineering and Management10.1007/978-981-97-5498-4_3(28-40)Online publication date: 27-Jul-2024
  • (2024)Diffusion Model in Normal Gathering Latent Space for Time Series Anomaly DetectionMachine Learning and Knowledge Discovery in Databases. Research Track10.1007/978-3-031-70352-2_17(284-300)Online publication date: 22-Aug-2024

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media

Access Granted

The conference sponsors are committed to making content openly accessible in a timely manner.
This article is provided by ACM and the conference, through the ACM OpenTOC service.