Skip to main content
Erschienen in: International Journal of Machine Learning and Cybernetics 9/2019

19.10.2018 | Original Article

Music-evoked emotion recognition based on cognitive principles inspired EEG temporal and spectral features

verfasst von: Hongjian Bo, Lin Ma, Quansheng Liu, Ruifeng Xu, Haifeng Li

Erschienen in: International Journal of Machine Learning and Cybernetics | Ausgabe 9/2019

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Electroencephalographic (EEG) based emotion recognition has attracted increasing attention in the field of human-computer interaction (HCI). But, how to use the cognitive principles to enhance the emotion recognition model is still a challenge. The purpose of this research paper is to investigate the emotion cognitive process and its application. Firstly, to evoke the response emotions, a three-stage experimental paradigm of long-time music stimuli was designed. The EEG signals were recorded in 15 healthy adults during the listening of 16 music clips respectively. Then, the time course analysis method of music-evoked emotions was proposed to examine the differences of brain activities. There was great increase of the spectral power in alpha band and a slight decrease in high-frequency beta and gamma bands during music listening. Through time analysis, the characteristics of the inspiring–keeping–fading were also found in different emotional states. After that, the most relevant EEG features were selected based on the time correlation analysis between EEG and music features. Finally, based on the cognitive principles inspired EEG features, an emotional prediction system was built. From the results, the accuracies of binary classification were 66.8% for valence and 59.5% for arousal. The accuracies of 3-classes classification performed as 45.9% for valence and 45.1% for arousal. These results suggest that with the help of cognitive principles, a better emotional recognition system could be built. Understanding the cognitive process could promote the development of artificial intelligence.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Weitere Produktempfehlungen anzeigen
Literatur
1.
Zurück zum Zitat Luo J (2012) Affective computing and intelligent interaction, vol 137. Springer, New YorkCrossRef Luo J (2012) Affective computing and intelligent interaction, vol 137. Springer, New YorkCrossRef
2.
Zurück zum Zitat Basu A, Das S, Horain P (2017) Intelligent human computer interaction. Springer International Publishing, New YorkCrossRef Basu A, Das S, Horain P (2017) Intelligent human computer interaction. Springer International Publishing, New YorkCrossRef
3.
Zurück zum Zitat Yang YH, Chen HH (2012) Machine recognition of music emotion: a review. ACM Trans Intell Syst Technol 3(3):40CrossRef Yang YH, Chen HH (2012) Machine recognition of music emotion: a review. ACM Trans Intell Syst Technol 3(3):40CrossRef
4.
Zurück zum Zitat Bradley MM, Lang PJ (1999) Affective norms for English words (ANEW): instruction manual and affective ratings, vol. 30, No. 1. Technical report C-1, the center for research in psychophysiology, University of Florida, pp 25–36 Bradley MM, Lang PJ (1999) Affective norms for English words (ANEW): instruction manual and affective ratings, vol. 30, No. 1. Technical report C-1, the center for research in psychophysiology, University of Florida, pp 25–36
5.
Zurück zum Zitat Lang PJ, Bradley MM, Cuthbert BN (2008) International affective picture system (IAPS): affective ratings of pictures and instruction manual. In: Tech Rep A-8. The center for research in psychophysiology, University of Florida Lang PJ, Bradley MM, Cuthbert BN (2008) International affective picture system (IAPS): affective ratings of pictures and instruction manual. In: Tech Rep A-8. The center for research in psychophysiology, University of Florida
6.
Zurück zum Zitat Bradley MM, Lang PJ (2007) The international affective digitized sounds (IADS-2): affective ratings of sounds and instruction manual. In: Tech Rep B-3. The center for research in psychophysiology, University of Florida Bradley MM, Lang PJ (2007) The international affective digitized sounds (IADS-2): affective ratings of sounds and instruction manual. In: Tech Rep B-3. The center for research in psychophysiology, University of Florida
7.
Zurück zum Zitat Brady K, Gwon Y, Khorrami P, et al (2016) Multi-modal audio, video and physiological sensor learning for continuous emotion prediction. International Workshop on Audio/visual Emotion Challenge, pp 97–104 Brady K, Gwon Y, Khorrami P, et al (2016) Multi-modal audio, video and physiological sensor learning for continuous emotion prediction. International Workshop on Audio/visual Emotion Challenge, pp 97–104
8.
Zurück zum Zitat Wang XW, Nie D, Lu BL (2014) Emotional state classification from EEG data using machine learning approach. Neurocomputing 129(4):94–106CrossRef Wang XW, Nie D, Lu BL (2014) Emotional state classification from EEG data using machine learning approach. Neurocomputing 129(4):94–106CrossRef
9.
Zurück zum Zitat Peng Y, Lu BL (2016) Discriminative manifold extreme learning machine and applications to image and EEG signal classification. Neurocomputing 174:265–277CrossRef Peng Y, Lu BL (2016) Discriminative manifold extreme learning machine and applications to image and EEG signal classification. Neurocomputing 174:265–277CrossRef
10.
Zurück zum Zitat Krumhansl CL (1997) An exploratory study of musical emotions and psychophysiology. Can J Exp Psychol 51(4):336–353CrossRef Krumhansl CL (1997) An exploratory study of musical emotions and psychophysiology. Can J Exp Psychol 51(4):336–353CrossRef
11.
Zurück zum Zitat Bo H, Li H, Ma L, Yu B (2016) Time-course eeg spectrum evidence for music key perception and emotional effects. In: International conference on brain inspired cognitive systems (BICS), pp 184–195 Bo H, Li H, Ma L, Yu B (2016) Time-course eeg spectrum evidence for music key perception and emotional effects. In: International conference on brain inspired cognitive systems (BICS), pp 184–195
12.
Zurück zum Zitat Hevner K (1935) The affective character of the major and minor modes in music. Am J Psychol 47(1):103–118CrossRef Hevner K (1935) The affective character of the major and minor modes in music. Am J Psychol 47(1):103–118CrossRef
13.
Zurück zum Zitat Koelsch S (2010) Towards a neural basis of music-evoked emotions. Trends Cogn Sci 14(3):131–137CrossRef Koelsch S (2010) Towards a neural basis of music-evoked emotions. Trends Cogn Sci 14(3):131–137CrossRef
14.
Zurück zum Zitat Blum K, Chen TJ, Chen AL, Madigan M, Downs BW, Waite RL, Braverman ER, Kerner M, Bowirrat A, Giordano J, Henshaw H (2010) Do dopaminergic gene polymorphisms affect mesolimbic reward activation of music listening response? therapeutic impact on reward deficiency syndrome (RDS). Med hypotheses 74(3):513–520CrossRef Blum K, Chen TJ, Chen AL, Madigan M, Downs BW, Waite RL, Braverman ER, Kerner M, Bowirrat A, Giordano J, Henshaw H (2010) Do dopaminergic gene polymorphisms affect mesolimbic reward activation of music listening response? therapeutic impact on reward deficiency syndrome (RDS). Med hypotheses 74(3):513–520CrossRef
15.
Zurück zum Zitat Zatorre RJ, Salimpoor VN (2013) From perception to pleasure: music and its neural substrates. Proc Natl Acad Sci 110(Supplement 2):10430–10437CrossRef Zatorre RJ, Salimpoor VN (2013) From perception to pleasure: music and its neural substrates. Proc Natl Acad Sci 110(Supplement 2):10430–10437CrossRef
16.
Zurück zum Zitat Li Y, Dan C, Ling W (2015) Abnormal functional connectivity of EEG gamma band in patients with depression during emotional face processing. Clin Neurophysiol 126(11):2078–2089CrossRef Li Y, Dan C, Ling W (2015) Abnormal functional connectivity of EEG gamma band in patients with depression during emotional face processing. Clin Neurophysiol 126(11):2078–2089CrossRef
17.
Zurück zum Zitat Partanen E, Kujala T, Tervaniemi M, Huotilainen M (2013) Prenatal music exposure induces long-term neural effects. PLoS One 8(10):e78946CrossRef Partanen E, Kujala T, Tervaniemi M, Huotilainen M (2013) Prenatal music exposure induces long-term neural effects. PLoS One 8(10):e78946CrossRef
18.
Zurück zum Zitat Mao M, Rau PL (2014) EEG-based measurement of emotion induced by mode, rhythm, and mv of chinese pop music. In: International Conference on Cross-Cultural Design, pp 89–100 Mao M, Rau PL (2014) EEG-based measurement of emotion induced by mode, rhythm, and mv of chinese pop music. In: International Conference on Cross-Cultural Design, pp 89–100
19.
Zurück zum Zitat Akar SA, Kara S, Agambayev S (2015) Nonlinear analysis of EEGs of patients with major depression during different emotional states. Comput Biol Med 67:49–60CrossRef Akar SA, Kara S, Agambayev S (2015) Nonlinear analysis of EEGs of patients with major depression during different emotional states. Comput Biol Med 67:49–60CrossRef
20.
Zurück zum Zitat Lin YP, Wang CH, Jung TP, Wu TL, Jeng SK, Duann JR, Chen JH (2010) EEG-based emotion recognition in music listening. IEEE Trans Biomed Eng 57(7):1798–1806CrossRef Lin YP, Wang CH, Jung TP, Wu TL, Jeng SK, Duann JR, Chen JH (2010) EEG-based emotion recognition in music listening. IEEE Trans Biomed Eng 57(7):1798–1806CrossRef
21.
Zurück zum Zitat Wang XW, Nie D, Lu BL (2011) EEG-based emotion recognition using frequency domain features and support vector machines. In: International Conference on Neural Information Processing, pp 734–743 Wang XW, Nie D, Lu BL (2011) EEG-based emotion recognition using frequency domain features and support vector machines. In: International Conference on Neural Information Processing, pp 734–743
22.
Zurück zum Zitat Koelstra S, Muhl C, Soleymani M (2012) Deap: a database for emotion analysis; using physiological signals. IEEE Trans Affect Comput 3(1):18–31CrossRef Koelstra S, Muhl C, Soleymani M (2012) Deap: a database for emotion analysis; using physiological signals. IEEE Trans Affect Comput 3(1):18–31CrossRef
23.
Zurück zum Zitat Tandle A, Jog N, Dharmadhikari A, et al (2016) Estimation of valence of emotion from musically stimulated EEG using frontal theta asymmetry. International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery, pp 63–68 Tandle A, Jog N, Dharmadhikari A, et al (2016) Estimation of valence of emotion from musically stimulated EEG using frontal theta asymmetry. International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery, pp 63–68
24.
Zurück zum Zitat Morris JD (1995) Observations: SAM: the Self-Assessment Manikin; an efficient cross-cultural measurement of emotional response. J Advert Res 35(6):63–68 Morris JD (1995) Observations: SAM: the Self-Assessment Manikin; an efficient cross-cultural measurement of emotional response. J Advert Res 35(6):63–68
25.
Zurück zum Zitat Jung TP, Humphries C, Lee TW, Makeig S, McKeown MJ, Iragui V, Sejnowski TJ (1998) Removing electroencephalographic artifacts: comparison between ica and pca. In: Neural Networks for Signal Processing VIII, pp 63–72 Jung TP, Humphries C, Lee TW, Makeig S, McKeown MJ, Iragui V, Sejnowski TJ (1998) Removing electroencephalographic artifacts: comparison between ica and pca. In: Neural Networks for Signal Processing VIII, pp 63–72
26.
Zurück zum Zitat Gao C, Ma L, Li H (2015) An ICA/HHT hybrid approach for automatic ocular artifact correction. Int J Pattern Recogn 29(2):1558001MathSciNetCrossRef Gao C, Ma L, Li H (2015) An ICA/HHT hybrid approach for automatic ocular artifact correction. Int J Pattern Recogn 29(2):1558001MathSciNetCrossRef
27.
Zurück zum Zitat Scherer KR (1986) Vocal affect expression: A review and a model for future research. Psychol Bull 99(2):143–165CrossRef Scherer KR (1986) Vocal affect expression: A review and a model for future research. Psychol Bull 99(2):143–165CrossRef
28.
Zurück zum Zitat Murray IR, Arnott JL (1993) Toward the simulation of emotion in synthetic speech: a review of the literature on human vocal emotion. J Acoust Soc Am 93(2):1097–1108CrossRef Murray IR, Arnott JL (1993) Toward the simulation of emotion in synthetic speech: a review of the literature on human vocal emotion. J Acoust Soc Am 93(2):1097–1108CrossRef
29.
Zurück zum Zitat Johnstone T, Scherer KR (2000) Vocal communication of emotion. In: Handbook of Emotions, 2nd edition. The Guilford Press, New York, pp 220–235 Johnstone T, Scherer KR (2000) Vocal communication of emotion. In: Handbook of Emotions, 2nd edition. The Guilford Press, New York, pp 220–235
30.
Zurück zum Zitat Juslin PN, Laukka P (2003) Communication of emotions in vocal expression and music performance: different channels, same code? Psychol Bull 129(5):770–814CrossRef Juslin PN, Laukka P (2003) Communication of emotions in vocal expression and music performance: different channels, same code? Psychol Bull 129(5):770–814CrossRef
31.
Zurück zum Zitat Lartillot O, Toiviainen P, Eerola T (2008) A matlab toolbox for music information retrieval. Data analysis, machine learning and applications, pp 261–268 Lartillot O, Toiviainen P, Eerola T (2008) A matlab toolbox for music information retrieval. Data analysis, machine learning and applications, pp 261–268
32.
Zurück zum Zitat Glasberg BR, Moore BC (2002) A model of loudness applicable to Time-Varying Sounds. J Audio Eng Soc 50(5):331–342 Glasberg BR, Moore BC (2002) A model of loudness applicable to Time-Varying Sounds. J Audio Eng Soc 50(5):331–342
33.
Zurück zum Zitat Skinner ER (1935) A calibrated recording and analysis of the pitch, force and quality of vocal tones expressing happiness and sadness; and a determination of the pitch and force of the subjective concepts of ordinary, soft, and loud tones. Commun Monogr 2(1):81–137 Skinner ER (1935) A calibrated recording and analysis of the pitch, force and quality of vocal tones expressing happiness and sadness; and a determination of the pitch and force of the subjective concepts of ordinary, soft, and loud tones. Commun Monogr 2(1):81–137
34.
Zurück zum Zitat Williams CE, Stevens KN (1972) Emotions and Speech: Some Acoustical Correlates. J Acoust Soc Am 52(4B):1238–1250CrossRef Williams CE, Stevens KN (1972) Emotions and Speech: Some Acoustical Correlates. J Acoust Soc Am 52(4B):1238–1250CrossRef
35.
Zurück zum Zitat Gobl C, Chasaide AN (2003) The role of voice quality in communicating emotion, mood and attitude. Speech Commun 40(12):189–212CrossRefMATH Gobl C, Chasaide AN (2003) The role of voice quality in communicating emotion, mood and attitude. Speech Commun 40(12):189–212CrossRefMATH
36.
Zurück zum Zitat Huang X, Acero A, Hon HW, Reddy R (2001) Spoken language processing: a guide to theory, algorithm, and system development. Prentice hall PTR, Upper Saddle River Huang X, Acero A, Hon HW, Reddy R (2001) Spoken language processing: a guide to theory, algorithm, and system development. Prentice hall PTR, Upper Saddle River
37.
Zurück zum Zitat Kasi K, Zahorian SA (2002) Yet another algorithm for pitch tracking. In: International conference on acoustics, speech, and signal processing (ICASSP), pp 361–364 Kasi K, Zahorian SA (2002) Yet another algorithm for pitch tracking. In: International conference on acoustics, speech, and signal processing (ICASSP), pp 361–364
38.
Zurück zum Zitat Myers JL, Well AD (2013) Research design and statistical analysis, 2nd edn. Routledge, Abingdon Myers JL, Well AD (2013) Research design and statistical analysis, 2nd edn. Routledge, Abingdon
39.
Zurück zum Zitat Klimesch W (1999) EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Res Rev 29(2):169–195CrossRef Klimesch W (1999) EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Res Rev 29(2):169–195CrossRef
40.
Zurück zum Zitat Welch P (1967) The use of fast fourier transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms. IEEE Trans Audio Electr 15(2):70–73CrossRef Welch P (1967) The use of fast fourier transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms. IEEE Trans Audio Electr 15(2):70–73CrossRef
41.
Zurück zum Zitat Vapnik V (2013) The nature of statistical learning theory. Springer, New YorkMATH Vapnik V (2013) The nature of statistical learning theory. Springer, New YorkMATH
Metadaten
Titel
Music-evoked emotion recognition based on cognitive principles inspired EEG temporal and spectral features
verfasst von
Hongjian Bo
Lin Ma
Quansheng Liu
Ruifeng Xu
Haifeng Li
Publikationsdatum
19.10.2018
Verlag
Springer Berlin Heidelberg
Erschienen in
International Journal of Machine Learning and Cybernetics / Ausgabe 9/2019
Print ISSN: 1868-8071
Elektronische ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-018-0880-z

Weitere Artikel der Ausgabe 9/2019

International Journal of Machine Learning and Cybernetics 9/2019 Zur Ausgabe

Neuer Inhalt