Towards Facial Expression Analysis for Enhanced Threat Detection in Surveillance

As cyber-physical systems become increasingly integrated into critical infrastructure such as energy distribution, transportation, healthcare, and public services, they are also becoming exposed to complex cyber threats. These threats range from traditional cyber intrusions to physical breaches and insider threats aimed at disrupting real-time operations. Enhancing situational awareness in such environments requires the development of proactive surveillance mechanisms that can detect early behavioural cues associated with potential threats. This paper presents a deep learning-based surveillance framework that incorporates facial expression analysis as a behavioral indicator to support the detection of anomalous. The framework takes into consideration that emotional states such as sustained anger, fear, and disgust can precede hostile actions. To operationalise these, we employed a convolutional neural network (CNN) and a recurrent neural network architecture trained in two benchmark datasets, the Amsterdam Dynamic Facial Expression Set (ADFES), and the Chinese Face Dataset with Dynamic Expressions to classify seven basic emotions (anger, disgust, fear, happiness, sadness, surprise, neutrality) from video streams. Based on a system throughput of 43.09 frames per second, a macro-averaged F1-score of 95%, and a per-frame inference time of 0.0232 s, preliminary results show that using facial expression analysis for real-time threat detection is feasible. These results underscore its potential to augment surveillance capabilities within cyberphysical systems, contributing to more proactive surveillance.