Micro-expression recognition based on SqueezeNet and C3D

Micro-expression recognition has attracted extensive attention from psychological and computer vision communities due to its multiple real-life applications. Compared with macro-expression, the change of micro-expression is subtle and difficult for detection and capture. Hence, the recognition is challenging. This paper proposes a micro-expression recognition model SQU-C3D which combines SqueezeNet and C3D methods. The features of micro-expression are mainly reflected in the apex frame, therefore a lightweight network called SqueezeNet is adopted to implement a reliable apex frame spotting method for dataset without apex frame labels. The position of the apex frame is detected by comparing the feature difference between the current and onset frames. In addition, the Convolutional 3D (C3D) network is utilized for micro-expression recognition due to its strong capability of extracting spatial–temporal features. The apex frame determined by SqueezeNet is fed into the C3D network along with the onset and offset frames, and the micro-expression is recognized by learning the features of these three key frames. Extensive experiments are conducted on three spontaneous micro-expression databases, namely, CASME II, SAMM, and SMIC-HS, where CASME II and SAMM include apex frame labels, whereas SMIC-HS does not. SQU-C3D achieves accuracy of 80.29% with 7 classes, 81.33% with 5 classes and 79.12% with 3 classes on the micro-expression benchmark dataset of CASME II, SAMM and SMIC-HS, respectively. Experimental results reveal that the proposed framework performs better than the state-of-the-art methods in the comparison.