Video anomaly detection based on multi-scale optical flow spatio-temporal enhancement and normality mining

Video anomaly detection aims to detect anomaly scores in video frames, and it is a challenging research area since the types of anomalies are limitless. In response to the fact that abnormal behavior is likely to be misidentified as normal and anomalies are typically generated by the fast motion of foreground objects, this paper proposes a novel model called the Multi-scale Optical Flow Spatio-Temporal Enhancement and Normality Mining Network (MOFSTE-NM). It contains the Spatio-temporal Information Attention Enhancement Module (SIAEM) that incorporates reconstructed optical flows at multiple scales and considers spatial and temporal aspects. This strategy reduces the influence of the background and normal objects, enhancing the model’s ability to focus on anomalous fast moving objects in the foreground. Additionally, we propose a Normality Mining Convolution (NMC) module embedded in the decoder to refine the boundary between normality and abnormality. The NMC uses a multihead attention mechanism for dynamic weight adjustment, enabling the precise extraction of normal information. We compute the final anomaly score by fusing two components: (1) the reconstruction error of the optical flows and (2) the peak signal-to-noise ratio between the predicted frame and its ground truth. We evaluate our model on three well-established video anomaly detection datasets. A comparison of different models indicates that the proposed model achieves superior performance compared to state-of-the-art approaches, with area under the receiver operating characteristic curve (AUROC) values of 99.23\(\%\) on UCSD Ped2, 88.84\(\%\) on CUHK Avenue, and 74.80\(\%\) on Shanghaitech.