Abstract
As a high-level function of the human brain, emotion is the external manifestation of people’s psychological characteristics. The emotion has a great impact on people’s personality and mental health. At the same time, emotion classification from electroencephalogram (EEG) signals have attracted much attention. To improve the precision of EEG-based emotion recognition, we proposed a fused feature extraction method to complete the classification of three emotions (neutral, happiness, and sadness). The standardized movie clips were selected to induce the corresponding emotion and the EEG response of 10 participants is collected by Emotiv EPOC. This paper systematically compared two kinds of EEG features (power spectrum and wavelet energy entropy) and their fusion for emotion classification. To reduce the dimension of fused features, we used principal component analysis (PCA) for dimensionality reduction and feature selection. The support vector machine (SVM) classifier and the relevance vector machines (RVM) classifier were utilized for emotion recognition respectively. From experimental results, we found that the fusion of two kinds of features outperformed a single feature for emotion classification by both the SVM classifier and the RVM classifier, and the averaged classification accuracy was 89.17% and 91.18%, respectively.
Similar content being viewed by others
References
Adeel A, Khan MA, Akram T, Sharif A, Yasmin M, Saba T, Javed K (2020) Entropy controlled deep features selection framework for grape leaf diseases recognition. Expert Syst
Ahern GL, Schwartz GE (1985) Differential lateralization for positive and negative emotion in the human brain: EEG spectral analysis. Neuropsychologia 23:745–755
Alarcao SM, Fonseca MJ (2017) Emotions recognition using EEG signals: a survey. IEEE Trans Affect Comput
Alm C, Roth D, Sproat R (2005) Emotions from text: machine learning for text-based emotion prediction. In: Proceedings of the conference on human language technology and empirical methods in natural language processing, pp 579–586
Arshad H, Khan MA, Sharif MI, Yasmin M, Tavares JMRS, Zhang Y, Satapathy SC (2020) A multilevel paradigm for deep convolutional neural network features selection with an application to human gait recognition. Expert Syst
Candra H, Yuwono M, Chai R et al (2015) Investigation of window size in classification of EEG-emotion signal with wavelet entropy and support vector machine. In: The 37th annual international conference of the IEEE engineering in medicine and biology society
Chang Chih-Chung, Lin Chih-Jen (2011) LIBSVM: a library for support vector machines
de Cheveigné A, Di Liberto GM, Arzounian D et al (2018) Multiway canonical correlation analysis of brain data. NeuroImage
Douglas-Cowie E, Cowie R, Schroder M (2000) A new emotion database: considerations, sources and scope. In: Proceedings of the ISCA workshop on Speech and emotion, pp 39–44
Ekman P (1970) Universal facial expressions of emotion. California Mental Health Research Digest 8:151–158
Fernando L, Alonso N, Gomez-Gil J (2012) Brain computer interfaces, a review. Sensors 12:1211–1279
Forster J, Strack F (1996) Influence of overt head movements on memory for valenced words: a case of conceptual–motor compatibility. J Pers Soc Psychol 71:421–430
Geng X-S, She Q-S (2017) Classification of motion imagination EEG signals based on artificial bee Colony optimized gauss process. Chinese Journal of Sensors and Actuators 30:378–384
Hillman C, Rosengren K, Smith D (2004) Emotion and motivated behavior: postural adjustments to affective picture viewing. Biol Psychol 66:51–62
Hubert W, de Jong-Meyer R (1991) Autonomic, neuroendocrine, and subjective responses to emotion-inducing film stimuli. Int J Psychophysiol 11:131–140
Hussain N, Khan MA, Sharif M et al (2020) A deep neural network and classical features based scheme for objects recognition: an application for machine inspection. Multimedia Tools and Applications:1–23
Hyvärinen A, Ramkumar P, Parkkonen L, Hari R (2010) Independent component analysis of short-time Fourier transforms for spontaneous EEG/MEG analysis. Neuroimage 49:257–271
Iacoviello D, Petracca A, Spezialetti M, Placidi G (2015) A real-time classification algorithm for EEG-based BCI driven by self-induced emotions. Comput Methods Prog Biomed 122:293–303
Ji N, Ma L, Dong H, Zhang X (2019) EEG signals feature extraction based on DWT and EMD combined with approximate entropy. Brain Sciences 9:201
Jung TP, Makeig S, Stensmo M et al (1997) Estimating alertness from the EEG power spectrum. IEEE Trans Biomed Eng 44:60–69
Khan M, Khan MA, Ahmed F, Mittal M, Goyal LM, Hemanth DJ, Satapathy SC (2019) Gastrointestinal diseases segmentation and classification based on duo-deep architectures. Pattern Recogn Lett
Khosrowabadi R, Quek C, Ang KK et al (2014) ERNN: a biologically inspired feedforward neural network to discriminate emotion from EEG signal. IEEE Trans Affect Comput 25:609–620
Koelstra S, Muhl C, Soleymani M, Jong-Seok Lee, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2012) DEAP: a database for emotion analysis using physiological signals. IEEE Trans Affect Comput 3:18–31
Lang PJ (1995) The emotion probe. Studies of motivation and attention. AmPsychol 50:372–385
Li D, Wang Z, GAO Q, Song Y, Yu X, Wang C (2019) Facial expression recognition based on electroencephalogram and facial landmark localization. Technol Health Care 27:373–387
Liu YJ, Yu M, Ge Y et al (2018) Real-time movie-induced discrete emotion recognition from EEG signals. IEEE Trans Affect Comput 9:550–562
Majid A, Khan MA, Yasmin M, Rehman A, Yousafzai A, Tariq U (2020) Classification of stomach infections: a paradigm of convolutional neural network along with classical features fusion and selection. Microsc Res Tech 83:562–576
Martis RJ, Acharya UR, Min LC (2013) ECG beat classification using PCA, LDA, ICA and discrete wavelet transform. Biomedical Signal Processing and Control 8:437–448
Mehmood A, Khan MA, Sharif M, Khan SA, Shaheen M, Saba T, Ashraf I (2020) Prosperous human gait recognition: an end-to-end system based on pre-trained CNN features selection. Multimed Tools Appl
Momennezhad A (2018) EEG-based emotion recognition utilizing wavelet coefficients. Multimed Tools Appl 77:27089–27106
Morris JD (1995) Observations: SAM: the self-assessment manikin; an efficient cross-cultural measurement of emotional response
Newson JJ, Thiagarajan TC (2019) EEG frequency bands in psychiatric disorders: a review of resting state studies. Front Hum Neurosci 12
Pane ES, Wibawa AD, Purnomo MH (2019) Improving the accuracy of EEG emotion recognition by combining valence lateralization and ensemble learning with tuning parameters. Cogn Process 20:405–417
Peng Y, Wang S, Long XZ et al (2015) Discriminative graph regularized extreme learning machine and its application to face recognition. Neurocomputing 49:340–353
Perales FJ, Ramis S, Guerrero A et al (2019) Evaluation of a VR system for pain management using binaural acoustic stimulation. Multimed Tools Appl 78:32869–32890
Rusell JA (1980) A circumplex model of affect. J Pers SocPsychol 39:116–121 178
Scherer K (2004) Which emotions can be induced by music? what are the underlying mechanisms? and how can we measure them? J New Music Res 33:239–251
Tao R, Li YL, Wang Y (2010) Short-time fractional Fourier transform and its applications. IEEE Trans Signal Process 58:2568–2580
Wang XW, Nie D, Lu BL (2014) Emotional state classification from EEG data using machine learning approach. Neurocomputing. 129:94–106
Xue W, Xie Z, Liu C (2009) A soft sensor for the surface cleanliness level of ultra-smooth optical component based on the Genetic-algorithm and the Related Vector Machine. In: 2009 8th IEEE International Conference on Cognitive Informatics
Zheng WL, Lu BL (2015) Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans Auton Ment Dev 7:162–175
Acknowledgments
This work was supported by the Fundamental Research on Advanced Technology and Engineering Application Team, Tianjin, China (grant No.20160524) and Natural Science Foundation of Tianjin (No.18JCYBJC87700).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Gao, Q., Wang, Ch., Wang, Z. et al. EEG based emotion recognition using fusion feature extraction method. Multimed Tools Appl 79, 27057–27074 (2020). https://doi.org/10.1007/s11042-020-09354-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-09354-y