Skip to main content
Top
Published in: Neural Processing Letters 8/2023

07-07-2023

Normal Spatio-Temporal Information Enhance for Unsupervised Video Anomaly Detection

Authors: Jun Wang, Di Jia, Ziqing Huang, Miaohui Zhang, Xing Ren

Published in: Neural Processing Letters | Issue 8/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Video anomaly detection is the study of detecting low probability anomalies from high probability normal behaviour. The imbalance of data raises the need to effectively distinguish between normal and abnormal behaviour in foreground targets. In the current state of research, the focus of many researchers has been on predictions closer to acquisition, but this need to effectively distinguish between normal and abnormal has received little attention. To address this problem, we propose a spatio-temporal enhancement network, a prediction-based unsupervised learning approach to video anomaly detection. In order to capture similar properties between foreground targets, we design the correlation enhancement module, which aims to quantify the spatial information of normal foreground targets. The temporal information enhancement module is also designed to stabilise the temporal information of normal foreground targets to reduce the effect of background variation and improve the robustness of the network structure. To clarify the boundary between normal and abnormal targets, a poorer substitution is represents the abnormal target feature vector, a superior substitution is represents the normal target feature vector. In this way, to obtain a clear and accurate locus of the abnormal occurrence time. We have extensively test on the UCSD Ped1, UCSD Ped2, and Avenue datasets and obtained desirable average AUC metrics of 82.7, 97.6, and 89.6, respectively.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
3.
go back to reference Asad M, Jiang H, Yang J, Tu E, Malik AA (2022) Multi-stream 3D latent feature clustering for abnormality detection in videos. Appl Intell 52(1):1126–1143CrossRef Asad M, Jiang H, Yang J, Tu E, Malik AA (2022) Multi-stream 3D latent feature clustering for abnormality detection in videos. Appl Intell 52(1):1126–1143CrossRef
4.
go back to reference Liu ZY et al (2022) Correlation-based feature partition regression method for unsupervised anomaly detection. Appl Intell 52(13):15074–15090CrossRef Liu ZY et al (2022) Correlation-based feature partition regression method for unsupervised anomaly detection. Appl Intell 52(13):15074–15090CrossRef
5.
go back to reference Wang S, Hu J, Yu G, Zhu E, Cai Z (2020) A survey of video abnormal event detection. Comput Eng Sci Rev 42(8):1393–1405 (Art no. 1007-130x) Wang S, Hu J, Yu G, Zhu E, Cai Z (2020) A survey of video abnormal event detection. Comput Eng Sci Rev 42(8):1393–1405 (Art no. 1007-130x)
6.
go back to reference He P, Li G, Li H (2022) A survey on deep learning based video anomaly detection. Comput Eng Sci Rev 44(9):1620–1629 (Art no. 1007-130x) He P, Li G, Li H (2022) A survey on deep learning based video anomaly detection. Comput Eng Sci Rev 44(9):1620–1629 (Art no. 1007-130x)
7.
go back to reference Ren J, Xia F, Liu YM, Lee I (2021) Deep video anomaly detection: opportunities and challenges. In: 21st IEEE international conference on data mining (IEEE ICDM), electr network, pp 959–966 Ren J, Xia F, Liu YM, Lee I (2021) Deep video anomaly detection: opportunities and challenges. In: 21st IEEE international conference on data mining (IEEE ICDM), electr network, pp 959–966
8.
go back to reference Abbas ZK, Al-Ani AA, IEEE (2022) A comprehensive review for video anomaly detection on videos. In: 2nd international conference on computer science and software engineering (CSASE), Univ Duhok, Duhok, IRAQ, pp 30–35 Abbas ZK, Al-Ani AA, IEEE (2022) A comprehensive review for video anomaly detection on videos. In: 2nd international conference on computer science and software engineering (CSASE), Univ Duhok, Duhok, IRAQ, pp 30–35
9.
go back to reference Adam A, Rivlin E, Shimshoni I, Reinitz D (2008) Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Trans Pattern Anal Mach Intell 30(3):555–560CrossRef Adam A, Rivlin E, Shimshoni I, Reinitz D (2008) Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Trans Pattern Anal Mach Intell 30(3):555–560CrossRef
10.
go back to reference Benezeth Y, Jodoin P-M, Saligrama V, Rosenberger C, IEEE (2009) Abnormal events detection based on spatio-temporal co-occurences. In: 2009 IEEE conference on computer vision and pattern recognition, pp 2458–2465 Benezeth Y, Jodoin P-M, Saligrama V, Rosenberger C, IEEE (2009) Abnormal events detection based on spatio-temporal co-occurences. In: 2009 IEEE conference on computer vision and pattern recognition, pp 2458–2465
11.
go back to reference Kim J, Grauman K, IEEE (2009) Observe locally, infer globally: a space-time mrf for detecting abnormal activities with incremental updates. In: 2009 IEEE conference on computer vision and pattern recognition, pp 2921–2928 Kim J, Grauman K, IEEE (2009) Observe locally, infer globally: a space-time mrf for detecting abnormal activities with incremental updates. In: 2009 IEEE conference on computer vision and pattern recognition, pp 2921–2928
16.
go back to reference Szegedy C et al (2015) Going deeper with convolutions. In: IEEE conference on computer vision and pattern recognition (CVPR), Boston, MA, pp 1–9 Szegedy C et al (2015) Going deeper with convolutions. In: IEEE conference on computer vision and pattern recognition (CVPR), Boston, MA, pp 1–9
17.
go back to reference He K, Zhang X, Ren S, Sun J, IEEE (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), Seattle, WA, pp 770–778 He K, Zhang X, Ren S, Sun J, IEEE (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), Seattle, WA, pp 770–778
18.
go back to reference Wang L, Tan HS, Zhou FQ, Zuo WX, Sun PF (2022) Unsupervised anomaly video detection via a double-flow ConvLSTM variational autoencoder. IEEE Access 10:44278–44289CrossRef Wang L, Tan HS, Zhou FQ, Zuo WX, Sun PF (2022) Unsupervised anomaly video detection via a double-flow ConvLSTM variational autoencoder. IEEE Access 10:44278–44289CrossRef
19.
go back to reference Wang X et al (2022) Robust unsupervised video anomaly detection by multipath frame prediction. IEEE Trans Neural Netw Learn Syst 33(6):2301–2312MathSciNetCrossRef Wang X et al (2022) Robust unsupervised video anomaly detection by multipath frame prediction. IEEE Trans Neural Netw Learn Syst 33(6):2301–2312MathSciNetCrossRef
20.
go back to reference Su CY, Ding SF, Jia WK, Wang X, Xu XZ (2008) Some progress of supervised learning. In: 4th international conference on intelligent computing, Shanghai, China, vol 5227, pp 661 Su CY, Ding SF, Jia WK, Wang X, Xu XZ (2008) Some progress of supervised learning. In: 4th international conference on intelligent computing, Shanghai, China, vol 5227, pp 661
21.
go back to reference Li N, Wu X, Guo H, Xu D, Ou Y, Chen Y-L (2015) Anomaly detection in video surveillance via Gaussian process. Int J Pattern Recognit Artif Intell 29(6):Art. no. 1555011CrossRef Li N, Wu X, Guo H, Xu D, Ou Y, Chen Y-L (2015) Anomaly detection in video surveillance via Gaussian process. Int J Pattern Recognit Artif Intell 29(6):Art. no. 1555011CrossRef
22.
go back to reference Iqbal MS (2015) Unsupervised multi-modal learning. In: 28th Canadian conference on artificial intelligence (Canadian AI), Halifax, CANADA, vol 9091, pp 343–346 Iqbal MS (2015) Unsupervised multi-modal learning. In: 28th Canadian conference on artificial intelligence (Canadian AI), Halifax, CANADA, vol 9091, pp 343–346
23.
go back to reference Hasan M, Choi J, Neumann J, Roy-Chowdhury AK, Davis LS, IEEE (2016) Learning temporal regularity in video sequences. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), Seattle, WA, pp 733–742 Hasan M, Choi J, Neumann J, Roy-Chowdhury AK, Davis LS, IEEE (2016) Learning temporal regularity in video sequences. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), Seattle, WA, pp 733–742
24.
go back to reference Wang X et al (2022) Robust unsupervised video anomaly detection by multipath frame prediction. IEEE Trans Neural Netw Learn Syst 33(6):2301–2312MathSciNetCrossRef Wang X et al (2022) Robust unsupervised video anomaly detection by multipath frame prediction. IEEE Trans Neural Netw Learn Syst 33(6):2301–2312MathSciNetCrossRef
25.
go back to reference Ravanbakhsh M et al (2017) Abnormal event detection in videos using generative adversarial nets. In: 24th IEEE international conference on image processing (ICIP), Beijing, China, pp 1577–1581 Ravanbakhsh M et al (2017) Abnormal event detection in videos using generative adversarial nets. In: 24th IEEE international conference on image processing (ICIP), Beijing, China, pp 1577–1581
26.
go back to reference Chong YS, Tay YH (2017) Abnormal event detection in videos using spatiotemporal autoencoder. In: 14th international symposium on neural networks (ISNN). Japan, vol 10262, pp 189–196 Chong YS, Tay YH (2017) Abnormal event detection in videos using spatiotemporal autoencoder. In: 14th international symposium on neural networks (ISNN). Japan, vol 10262, pp 189–196
27.
go back to reference Sabokrou M, Fathy M, Hoseini M (2016) Video anomaly detection and localisation based on the sparsity and reconstruction error of auto-encoder. Electron Lett 52(13):1122–1123CrossRef Sabokrou M, Fathy M, Hoseini M (2016) Video anomaly detection and localisation based on the sparsity and reconstruction error of auto-encoder. Electron Lett 52(13):1122–1123CrossRef
28.
go back to reference Li N, Wu X, Xu D, Guo H, Feng W (2015) Spatio-temporal context analysis within video volumes for anomalous-event detection and localization. Neurocomputing 155:309–319CrossRef Li N, Wu X, Xu D, Guo H, Feng W (2015) Spatio-temporal context analysis within video volumes for anomalous-event detection and localization. Neurocomputing 155:309–319CrossRef
29.
go back to reference Cong Y, Yuan J, Liu J, IEEE (2011) Sparse reconstruction cost for abnormal event detection. In: IEEE conference on computer vision and pattern recognition (CVPR), Colorado Springs, CO, p 1807 Cong Y, Yuan J, Liu J, IEEE (2011) Sparse reconstruction cost for abnormal event detection. In: IEEE conference on computer vision and pattern recognition (CVPR), Colorado Springs, CO, p 1807
30.
go back to reference Liu W, Luo W, Lian D, Gao S, IEEE (2018) Future frame prediction for anomaly detection - a new baseline. In: 31st IEEE/CVF conference on computer vision and pattern recognition (CVPR), Salt Lake City, UT, 2018, pp 6536–6545 Liu W, Luo W, Lian D, Gao S, IEEE (2018) Future frame prediction for anomaly detection - a new baseline. In: 31st IEEE/CVF conference on computer vision and pattern recognition (CVPR), Salt Lake City, UT, 2018, pp 6536–6545
31.
go back to reference Wang W, Chang F, Mi H (2021) Intermediate fused network with multiple timescales for anomaly detection. Neurocomputing 433:37–49CrossRef Wang W, Chang F, Mi H (2021) Intermediate fused network with multiple timescales for anomaly detection. Neurocomputing 433:37–49CrossRef
32.
go back to reference Smeureanu S, Ionescu RT, Popescu M, Alexe B (2017) Deep appearance features for abnormal behavior detection in video. In: 19th international conference on image analysis and processing (ICIAP), Catania, Italy, vol 10485, pp 779–789 Smeureanu S, Ionescu RT, Popescu M, Alexe B (2017) Deep appearance features for abnormal behavior detection in video. In: 19th international conference on image analysis and processing (ICIAP), Catania, Italy, vol 10485, pp 779–789
33.
go back to reference Ionescu RT, Smeureanu S, Popescu M, Alexe B, IEEE (2019) Detecting abnormal events in video using narrowed normality clusters. In: 19th IEEE winter conference on applications of computer vision (WACV), Waikoloa Village, HI, pp 1951–1960 Ionescu RT, Smeureanu S, Popescu M, Alexe B, IEEE (2019) Detecting abnormal events in video using narrowed normality clusters. In: 19th IEEE winter conference on applications of computer vision (WACV), Waikoloa Village, HI, pp 1951–1960
34.
go back to reference Sabokrou M, Fayyaz M, Fathy M, Moayed Z, Klette R (2018) Deep-anomaly: fully convolutional neural network for fast anomaly detection in crowded scenes. Comput Vis Image Underst 172:88–97CrossRefMATH Sabokrou M, Fayyaz M, Fathy M, Moayed Z, Klette R (2018) Deep-anomaly: fully convolutional neural network for fast anomaly detection in crowded scenes. Comput Vis Image Underst 172:88–97CrossRefMATH
35.
go back to reference Sun Q, Liu H, Harada T (2017) Online growing neural gas for anomaly detection in changing surveillance scenes. Pattern Recogn 64:187–201CrossRef Sun Q, Liu H, Harada T (2017) Online growing neural gas for anomaly detection in changing surveillance scenes. Pattern Recogn 64:187–201CrossRef
36.
go back to reference Wang S, Miao Z (2010) Anomaly detection in crowd scene. In: IEEE 10th international conference on signal processing, Beijing, China, pp 1220–1223 Wang S, Miao Z (2010) Anomaly detection in crowd scene. In: IEEE 10th international conference on signal processing, Beijing, China, pp 1220–1223
37.
go back to reference Gong D et al (2019) Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. In: IEEE/CVF international conference on computer vision (ICCV), Seoul, pp 1705–1714 Gong D et al (2019) Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. In: IEEE/CVF international conference on computer vision (ICCV), Seoul, pp 1705–1714
38.
go back to reference Hao Y, Li J, Wang N, Wang X, Gao X (2022) Spatiotemporal consistency-enhanced network for video anomaly detection. Pattern Recogn 121:Art no. 108232CrossRef Hao Y, Li J, Wang N, Wang X, Gao X (2022) Spatiotemporal consistency-enhanced network for video anomaly detection. Pattern Recogn 121:Art no. 108232CrossRef
39.
go back to reference Paszke A et al (2019) PyTorch: an imperative style, high-performance deep learning library. In: 33rd conference on neural information processing systems (NeurIPS), Vancouver, vol 32 Paszke A et al (2019) PyTorch: an imperative style, high-performance deep learning library. In: 33rd conference on neural information processing systems (NeurIPS), Vancouver, vol 32
41.
go back to reference Luo W, Liu W, Gao S, IEEE (2017) Remembering history with convolutional lSTM for anomaly detection. In: IEEE international conference on multimedia and expo (ICME), Hong Kong, pp 439–444 Luo W, Liu W, Gao S, IEEE (2017) Remembering history with convolutional lSTM for anomaly detection. In: IEEE international conference on multimedia and expo (ICME), Hong Kong, pp 439–444
42.
go back to reference Luo W, Liu W, Gao S, IEEE (2017) A revisit of sparse coding based anomaly detection in stacked RNN framework. In: 16th IEEE international conference on computer vision (ICCV), Venice, vol 2017, pp 341–349 Luo W, Liu W, Gao S, IEEE (2017) A revisit of sparse coding based anomaly detection in stacked RNN framework. In: 16th IEEE international conference on computer vision (ICCV), Venice, vol 2017, pp 341–349
43.
go back to reference Wang S et al (2018) Detecting abnormality without knowing normality: a two-stage approach for unsupervised video abnormal event detection. In: 26th ACM multimedia conference (MM). Seoul, vol 2018, pp 636–644 Wang S et al (2018) Detecting abnormality without knowing normality: a two-stage approach for unsupervised video abnormal event detection. In: 26th ACM multimedia conference (MM). Seoul, vol 2018, pp 636–644
44.
go back to reference Wu P, Liu J, Shen F (2020) A deep one-class neural network for anomalous event detection in complex scenes. IEEE Trans Neural Netw Learn Syst 31(7):2609–2622 Wu P, Liu J, Shen F (2020) A deep one-class neural network for anomalous event detection in complex scenes. IEEE Trans Neural Netw Learn Syst 31(7):2609–2622
45.
go back to reference Ionescu RT, Smeureanu S, Alexe B, Popescu M, IEEE (2017) Unmasking the abnormal events in video. In: 16th IEEE international conference on computer vision (ICCV), Venice, Italy, pp 2914–2922 Ionescu RT, Smeureanu S, Alexe B, Popescu M, IEEE (2017) Unmasking the abnormal events in video. In: 16th IEEE international conference on computer vision (ICCV), Venice, Italy, pp 2914–2922
46.
go back to reference Mu HY, Sun RZ, Chen ZQ, Qin J (2023) Intelligent abnormal behavior detection using double sparseness method. Appl Intell 53(7):7728–7740CrossRef Mu HY, Sun RZ, Chen ZQ, Qin J (2023) Intelligent abnormal behavior detection using double sparseness method. Appl Intell 53(7):7728–7740CrossRef
47.
go back to reference Le VT, Kim YG (2023) Attention-based residual autoencoder for video anomaly detection. Appl Intell 53(3):3240–3254CrossRef Le VT, Kim YG (2023) Attention-based residual autoencoder for video anomaly detection. Appl Intell 53(3):3240–3254CrossRef
48.
go back to reference Zhou JT, Zhang L, Fang Z, Du J, Peng X, Xiao Y (2020) Attention-driven loss for anomaly detection in video surveillance. IEEE Trans Circuits Syst Video Technol 30(12):4639–4647CrossRef Zhou JT, Zhang L, Fang Z, Du J, Peng X, Xiao Y (2020) Attention-driven loss for anomaly detection in video surveillance. IEEE Trans Circuits Syst Video Technol 30(12):4639–4647CrossRef
Metadata
Title
Normal Spatio-Temporal Information Enhance for Unsupervised Video Anomaly Detection
Authors
Jun Wang
Di Jia
Ziqing Huang
Miaohui Zhang
Xing Ren
Publication date
07-07-2023
Publisher
Springer US
Published in
Neural Processing Letters / Issue 8/2023
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-023-11347-5

Other articles of this Issue 8/2023

Neural Processing Letters 8/2023 Go to the issue