Leveraging Transfer Learning for Effective Recognition of Emotions from Images: A Review

Emotions constitute  an integral part of interpersonal communication and comprehending human behavior. Reliable analysis and interpretation of facial expressions are essential to gain a deeper insight into human behavior. Even though facial emotion recognition (FER) is extensively studied to improve human–computer interaction, it is yet elusive to human interpretation. Albeit humans have the innate capability to identify emotions through facial expressions, it is a challenging task to be accomplished by computer systems due to intra-class variations. While most of the recent works have performed well on datasets with images captured under controlled conditions, they fail to perform well on datasets that consist of variations in image lighting, shadows, facial orientation, noise, and partial faces. For all the tremendous performances of the existing works, there appears to be significant room for researchers. This paper emphasizes automatic FER on a single image for real-time emotion recognition using transfer learning. Since natural images suffer from problems of resolution, pose, and noise, this study proposes a deep learning approach based on transfer learning from a pre-trained VGG-16 network to significantly reduce training time and effort while achieving commendable improvement over previously proposed techniques and models on the FER-2013 dataset. The main contribution of this paper is to study and demonstrate the efficacy of multiple state-of-the-art models using transfer learning to conclude which is better to classify an input image as having one of the seven basic emotions: happy, sad, surprise, angry, disgust, fear, and neutral. The analysis shows that the VGG-16 model outperforms ResNet-50, DenseNet-121, EfficientNet-B2, and others while attaining a training accuracy of about 85% and validation accuracy as high as 67% in just 15 epochs with significantly lower training time.