Cheng Feng
ABSTRACT
Recent advances in AIoT technologies have led to an increasing popularity of utilizing machine learning algorithms to detect operational failures for cyber-physical systems (CPS). In its basic form, an anomaly detection module monitors the sensor measurements and actuator states from the physical plant, and detects anomalies in these measurements to identify abnormal operation status. Nevertheless, building effective anomaly detection models for CPS is rather challenging as the model has to accurately detect anomalies in presence of highly complicated system dynamics and unknown amount of sensor noise. In this work, we propose a novel time series anomaly detection method called Neural System Identification and Bayesian Filtering (NSIBF) in which a specially crafted neural network architecture is posed for system identification, i.e., capturing the dynamics of CPS in a dynamical state-space model; then a Bayesian filtering algorithm is naturally applied on top of the "identified" state-space model for robust anomaly detection by tracking the uncertainty of the hidden state of the system recursively over time. We provide qualitative as well as quantitative experiments with the proposed method on a synthetic and three real-world CPS datasets, showing that NSIBF compares favorably to the state-of-the-art methods with considerable improvements on anomaly detection in CPS.
Supplemental Material
- Mart'in Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. Tensorflow: A system for large-scale machine learning. In 12th USENIX symposium on operating systems design and implementation (OSDI 16). 265--283.Google Scholar
Digital Library
- Chuadhry Mujeeb Ahmed, Venkata Reddy Palleti, and Aditya P Mathur. 2017. WADI: a water distribution testbed for research in the design of secure cyber physical systems. In Proceedings of the 3rd International Workshop on Cyber-Physical Systems for Smart Water Networks. 25--28.Google ScholarDigital Library
- Chuadhry Mujeeb Ahmed, Jianying Zhou, and Aditya P Mathur. 2018. Noise matters: Using sensor and process noise fingerprint to detect stealthy cyber attacks and authenticate sensors in cps. In Proceedings of the 34th Annual Computer Security Applications Conference. 566--581.Google ScholarDigital Library
- Abdulrahman Al-Abassi, Jacob Sakhnini, and Hadis Karimipour. 2020. Unsupervised Stacked Autoencoders for Anomaly Detection on Smart Cyber-physical Grids. In 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 3123--3129.Google Scholar
- Jinwon An and Sungzoon Cho. 2015. Variational autoencoder based anomaly detection using reconstruction probability. Special Lecture on IE, Vol. 2, 1 (2015), 1--18.Google Scholar
- M Sanjeev Arulampalam, Simon Maskell, Neil Gordon, and Tim Clapp. 2002. A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Transactions on signal processing, Vol. 50, 2 (2002), 174--188.Google ScholarDigital Library
- Julien Audibert, Pietro Michiardi, Frédéric Guyard, Sébastien Marti, and Maria A Zuluaga. 2020. USAD: UnSupervised Anomaly Detection on Multivariate Time Series. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3395--3404.Google ScholarDigital Library
- Mikel Canizo, Isaac Triguero, Angel Conde, and Enrique Onieva. 2019. Multi-head CNN--RNN for multi-time series anomaly detection: An industrial case study. Neurocomputing, Vol. 363 (2019), 246--260.Google ScholarDigital Library
- Raghavendra Chalapathy and Sanjay Chawla. 2019. Deep learning for anomaly detection: A survey. arXiv preprint arXiv:1901.03407 (2019).Google Scholar
- Varun Chandola, Arindam Banerjee, and Vipin Kumar. 2009. Anomaly detection: A survey. ACM computing surveys (CSUR), Vol. 41, 3 (2009), 1--58.Google ScholarDigital Library
- Jinghui Chen, Saket Sathe, Charu Aggarwal, and Deepak Turaga. 2017. Outlier detection with autoencoder ensembles. In Proceedings of the 2017 SIAM international conference on data mining. SIAM, 90--98.Google ScholarCross Ref
- Yuqi Chen, Christopher M Poskitt, and Jun Sun. 2018. Learning from mutants: Using code mutation to learn and monitor invariants of a cyber-physical system. In 2018 IEEE Symposium on Security and Privacy (SP). IEEE, 648--660.Google ScholarCross Ref
- Charles K Chui, Guanrong Chen, et al. 2017. Kalman filtering .Springer.Google Scholar
- Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Bengio. 2015. A recurrent latent variable model for sequential data. Advances in neural information processing systems, Vol. 28 (2015), 2980--2988.Google Scholar
- Derui Ding, Qing-Long Han, Xiaohua Ge, and Jun Wang. 2020. Secure State Estimation and Control of Cyber-Physical Systems: A Survey. IEEE Transactions on Systems, Man, and Cybernetics: Systems (2020).Google Scholar
- Cheng Feng, Tingting Li, and Deeph Chana. 2017. Multi-level anomaly detection in industrial control systems via package signatures and LSTM networks. In 2017 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). IEEE, 261--272.Google ScholarCross Ref
- Cheng Feng, Venkata Reddy Palleti, Aditya Mathur, and Deeph Chana. 2019. A Systematic Framework to Generate Invariants for Anomaly Detection in Industrial Control Systems. In 26th Annual Network and Distributed System Security Symposium, NDSS 2019, San Diego, California, USA, February 24--27, 2019. The Internet Society.Google ScholarCross Ref
- Marco Fraccaro, Simon Kamronn, Ulrich Paquet, and Ole Winther. 2017. A disentangled recognition and nonlinear dynamics model for unsupervised learning. In Advances in Neural Information Processing Systems. 3601--3610.Google Scholar
- Jairo Giraldo, David Urbina, Alvaro Cardenas, Junia Valente, Mustafa Faisal, Justin Ruths, Nils Ole Tippenhauer, Henrik Sandberg, and Richard Candell. 2018. A survey of physics-based attack detection in cyber-physical systems. ACM Computing Surveys (CSUR), Vol. 51, 4 (2018), 1--36.Google ScholarDigital Library
- Jonathan Goh, Sridhar Adepu, Khurum Nazir Junejo, and Aditya Mathur. 2016. A dataset to support research in the design of secure water treatment systems. In International Conference on Critical Information Infrastructures Security. Springer, 88--99.Google Scholar
- Jonathan Goh, Sridhar Adepu, Marcus Tan, and Zi Shan Lee. 2017. Anomaly detection in cyber physical systems using recurrent neural networks. In 2017 IEEE 18th International Symposium on High Assurance Systems Engineering (HASE). IEEE, 140--145.Google ScholarCross Ref
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.Google ScholarDigital Library
- Abdulmalik Humayed, Jingqiang Lin, Fengjun Li, and Bo Luo. 2017. Cyber-physical systems security-A survey. IEEE Internet of Things Journal, Vol. 4, 6 (2017), 1802--1831.Google ScholarCross Ref
- Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Soderstrom. 2018. Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 387--395.Google ScholarDigital Library
- Simon J Julier. 2002. The scaled unscented transformation. In Proceedings of the 2002 American Control Conference (IEEE Cat. No. CH37301), Vol. 6. IEEE, 4555--4559.Google ScholarCross Ref
- Simon J Julier and Jeffrey K Uhlmann. 2004. Unscented filtering and nonlinear estimation. Proc. IEEE, Vol. 92, 3 (2004), 401--422.Google ScholarCross Ref
- Rudolph Emil Kalman. 1960. A new approach to linear filtering and prediction problems. (1960).Google Scholar
- Maximilian Karl, Maximilian Soelch, Justin Bayer, and Patrick Van der Smagt. 2016. Deep variational bayes filters: Unsupervised learning of state space models from raw data. arXiv preprint arXiv:1605.06432 (2016).Google Scholar
- Tung Kieu, Bin Yang, Chenjuan Guo, and Christian S Jensen. 2019. Outlier Detection for Time Series with Recurrent Autoencoder Ensembles.. In IJCAI. 2725--2732.Google Scholar
- Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- Genshiro Kitagawa. 1996. Monte Carlo filter and smoother for non-Gaussian nonlinear state space models. Journal of computational and graphical statistics, Vol. 5, 1 (1996), 1--25.Google ScholarCross Ref
- Rahul Krishnan, Uri Shalit, and David Sontag. 2017. Structured inference networks for nonlinear state space models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31.Google ScholarCross Ref
- Rahul G Krishnan, Uri Shalit, and David Sontag. 2015. Deep kalman filters. arXiv preprint arXiv:1511.05121 (2015).Google Scholar
- Sungmoon Kwon, Hyunguk Yoo, and Taeshik Shon. 2019. RNN-based anomaly detection in DNP3 transport layer. In 2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm). IEEE, 1--7.Google ScholarCross Ref
- Roger R Labbe. 2018. FilterPy Documentation. (2018).Google Scholar
- Roger R Labbe. 2019. Kalman and Bayesian Filters in Python.Google Scholar
- Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature, Vol. 521, 7553 (2015), 436--444.Google Scholar
- Edward Ashford Lee and Sanjit A Seshia. 2017. Introduction to embedded systems: A cyber-physical systems approach .Mit Press.Google ScholarDigital Library
- Bryan Lim, Stefan Zohren, and Stephen Roberts. 2019. Recurrent Neural Filters: Learning Independent Bayesian Filtering Steps for Time Series Prediction. arXiv preprint arXiv:1901.08096 (2019).Google Scholar
- Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. 2008. Isolation forest. In 2008 Eighth IEEE International Conference on Data Mining. IEEE, 413--422.Google ScholarDigital Library
- Lennart Ljung. 1999. System identification. Wiley encyclopedia of electrical and electronics engineering (1999), 1--19.Google Scholar
- Yuan Luo, Ya Xiao, Long Cheng, Guojun Peng, and Danfeng Daphne Yao. 2020. Deep Learning-Based Anomaly Detection in Cyber-Physical Systems: Progress and Opportunities. arXiv preprint arXiv:2003.13213 (2020).Google Scholar
- Pankaj Malhotra, Anusha Ramakrishnan, Gaurangi Anand, Lovekesh Vig, Puneet Agarwal, and Gautam Shroff. 2016. LSTM-based encoder-decoder for multi-sensor anomaly detection. arXiv preprint arXiv:1607.00148 (2016).Google Scholar
- Larry M Manevitz and Malik Yousef. 2001. One-class SVMs for document classification. Journal of machine Learning research, Vol. 2, Dec (2001), 139--154.Google ScholarDigital Library
- Goeffrey J McLachlan. 1999. Mahalanobis distance. Resonance, Vol. 4, 6 (1999), 20--26.Google ScholarCross Ref
- Andrew Ng et al. 2011. Sparse autoencoder. CS294A Lecture notes, Vol. 72, 2011 (2011), 1--19.Google Scholar
- Daehyung Park, Yuuna Hoshi, and Charles C Kemp. 2018. A multimodal anomaly detector for robot-assisted feeding using an lstm-based variational autoencoder. IEEE Robotics and Automation Letters, Vol. 3, 3 (2018), 1544--1551.Google ScholarCross Ref
- Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. 2018. Deep state space models for time series forecasting. Advances in neural information processing systems, Vol. 31 (2018), 7785--7794.Google Scholar
- Maria Isabel Ribeiro. 2004. Kalman and extended kalman filters: Concept, derivation and properties. Institute for Systems and Robotics, Vol. 43 (2004), 46.Google Scholar
- David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. 2020. DeepAR: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, Vol. 36, 3 (2020), 1181--1191.Google Scholar
- Stanley F Schmidt. 1966. Application of state-space methods to navigation problems. In Advances in control systems. Vol. 3. Elsevier, 293--340.Google Scholar
- John Sipple. 2020. Interpretable, multidimensional, multimodal anomaly detection with negative sampling for detection of device failure. In International Conference on Machine Learning. PMLR, 9016--9025.Google Scholar
- Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. 2019. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2828--2837.Google ScholarDigital Library
- Shahroz Tariq, Sangyup Lee, Youjin Shin, Myeong Shin Lee, Okchul Jung, Daewon Chung, and Simon S Woo. 2019. Detecting anomalies in space using multivariate convolutional LSTM with mixtures of probabilistic PCA. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2123--2133.Google ScholarDigital Library
- Haowen Xu, Wenxiao Chen, Nengwen Zhao, Zeyan Li, Jiahao Bu, Zhihan Li, Ying Liu, Youjian Zhao, Dan Pei, Yang Feng, et al. 2018. Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications. In Proceedings of the 2018 World Wide Web Conference. 187--196.Google ScholarDigital Library
- Chuxu Zhang, Dongjin Song, Yuncong Chen, Xinyang Feng, Cristian Lumezanu, Wei Cheng, Jingchao Ni, Bo Zong, Haifeng Chen, and Nitesh V Chawla. 2019. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 1409--1416.Google ScholarDigital Library
- Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, and Haifeng Chen. 2018. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In International Conference on Learning Representations.Google Scholar
Index Terms
- Time Series Anomaly Detection for Cyber-physical Systems via Neural System Identification and Bayesian Filtering
Recommendations
A hybrid behavior- and Bayesian network-based framework for cyber–physical anomaly detection
AbstractIn recent years, the increasing Internet connectivity and heterogeneity of industrial protocols have been raising the number and nature of cyber-attacks against Industrial Control Systems (ICS). Such cyber-attacks may lead to cyber anomalies and ...
Highlights- Hybrid behavior- and Bayesian network-based cyber–physical anomaly detection.
- Hybrid anomaly detection framework based on both cyber and physical data from ICS.
- Identification of cyber, physical and cyber–physical anomalies in ICS.
Anomaly and change point detection for time series with concept drift
AbstractAnomaly detection is one of the most important research contents in time series data analysis, which is widely used in many fields. In real world, the environment is usually dynamically changing, and the distribution of data changes over time, ...
Anomaly detection method based on penalty least squares algorithm and time window entropy for Cyber–Physical Systems
AbstractReal-time system status detection must be accurate and reliable due to the close coupling of Cyber–Physical Systems (CPS) components. In order to improve the effectiveness of the CPS anomaly detection method, this paper proposes a real-time ...
Comments