Hybrid CNN-SVM Classifier for Human Emotion Recognition Using ROI Extraction and Feature Fusion

Emotions expressed on a human face have a significant impact on decisions and arguments on a variety of topics. According to psychological theory, a person’s emotional states can be categorized as the afraid, disgusted, angry, sad, happy, neutral face and surprised. The automatic extraction of these emotions from images of human faces can help in human–computer interaction, among other things. Convolution Neural Network (CNN), Deep Belief Network (DBN), Bi-directional Long Short Term Memory (Bi-LSTM) are some of the existing techniques used to recognize the emotions of a human. This technique has some impacts like low accuracy and high error. To achieve better accuracy, hybrid CNN-SVM (Support Vector Machine) model is designed for classifying emotional state of humans. Initially, preprocessing is used to remove unwanted things from the image dataset. Resizing, Gaussian filter, Median filter, Histogram Equalization and Wiener filters are used in the preprocessing stage. After that, Region of Interest of the preprocessed image is extracted. Then features of the images are extracted based on Local Binary Pattern and Gabor feature technique. These obtained features are fused using the feature fusion process. The fused image data is fed to a hybrid CNN-SVM classifier. The hybrid CNN-SVM classifies the different emotional states of humans. The proposed method achieves an accuracy of 94% for CK_Plus, 86% FER_2013, 78% for KDEF, 96% for KMU_FED and 85% for the TFEID dataset. Thus the proposed human emotion recognition using the CNN-SVM approach produced optimal solutions compared to the existing systems.