Classification Mechanism of Convolutional Neural Network for Facial Expression Recognition

With the development of deep learning, the structures of convolutional neural networks (CNNs) are becoming more complex and the performance of expression recognition is getting better. However, the classification mechanism of CNN is still a black box. The main problem is that CNNs have a great number of parameters, which makes it difficult to analyze them clearly. In this paper, we explain the essence of deep learning from the perspective of manifold geometry. The main purpose of deep learning especially CNN is to learn the probability distributions on manifolds. And we design a neural network based on the facial expression recognition to explore the classification mechanism of CNN. By using the deconvolution visualization method, we qualitatively verify that the trained CNN forms a detector for specific facial action unit (FAU) and each neuron of CNN is a specific manifold feature extractor for facial images. Moreover, we design a distance function to measure the differences of activation value distributions on the same feature map of FAU. The greater the distance, the more sensitive the feature map is to the FAU. The results show that the mapping relationship between FAUs and feature maps of CNN is determined, the trained CNN has generated an internal detector for each FAU to extract the facial manifold feature.