Skip to main content
Erschienen in:

21.02.2023 | Original Article

Negative emotion recognition using multimodal physiological signals for advanced driver assistance systems

verfasst von: Chie Hieida, Tomoaki Yamamoto, Takatomi Kubo, Junichiro Yoshimoto, Kazushi Ikeda

Erschienen in: Artificial Life and Robotics | Ausgabe 2/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Recent advanced driver assistance systems’ (ADASs) control cars to avoid accidents, but few of them consider driver’s comfort. To realize comfortable driving, an ADAS must sense the driver’s emotions, especially when they are negative. Since emotions are reflected in a person’s physiological signals, they are informative for sensing emotions. However, it is unclear which signals are most useful for detecting a driver’s negative emotions. To examine the usefulness of each physiological signal, we implemented an emotion classifier (negative or non-negative) using sparse logistic regression for multimodal signals. This classifier was trained using a multimodal physiological signal dataset with negative emotion labels collected, while subjects were driving a vehicle. The resulting classifier successfully classifies emotions with an area under the curve of 0.74 and identifies the physiological signals that are useful for detecting negative emotions.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Alotaiby T, Abd El-Samie FE, Alshebeili SA, Ahmad I (2015) A review of channel selection algorithms for EEG signal processing. EURASIP J Adv Signal Process 2015(1):1–21CrossRef Alotaiby T, Abd El-Samie FE, Alshebeili SA, Ahmad I (2015) A review of channel selection algorithms for EEG signal processing. EURASIP J Adv Signal Process 2015(1):1–21CrossRef
2.
Zurück zum Zitat Alvarsson JJ, Wiens S, Nilsson ME (2010) Stress recovery during exposure to nature sound and environmental noise. Int J Environ Res Public Health 7(3):1036–1046CrossRef Alvarsson JJ, Wiens S, Nilsson ME (2010) Stress recovery during exposure to nature sound and environmental noise. Int J Environ Res Public Health 7(3):1036–1046CrossRef
3.
Zurück zum Zitat Annerstedt M, Jönsson P, Wallergård M, Johansson G, Karlson B, Grahn P, Hansen ÅM, Währborg P (2013) Inducing physiological stress recovery with sounds of nature in a virtual reality forest-results from a pilot study. Physiol Behav 118:240–250CrossRef Annerstedt M, Jönsson P, Wallergård M, Johansson G, Karlson B, Grahn P, Hansen ÅM, Währborg P (2013) Inducing physiological stress recovery with sounds of nature in a virtual reality forest-results from a pilot study. Physiol Behav 118:240–250CrossRef
4.
Zurück zum Zitat Benedek M, Kaernbach C (2010) Decomposition of skin conductance data by means of nonnegative deconvolution. Psychophysiology 47(4):647–658 Benedek M, Kaernbach C (2010) Decomposition of skin conductance data by means of nonnegative deconvolution. Psychophysiology 47(4):647–658
5.
Zurück zum Zitat Busso C, Deng Z, Yildirim S, Bulut M, Lee CM, Kazemzadeh A, Lee S, Neumann U, Narayanan S (2004) Analysis of emotion recognition using facial expressions, speech and multimodal information. In: Proceedings of the 6th international conference on multimodal interfaces, pp 205–211 Busso C, Deng Z, Yildirim S, Bulut M, Lee CM, Kazemzadeh A, Lee S, Neumann U, Narayanan S (2004) Analysis of emotion recognition using facial expressions, speech and multimodal information. In: Proceedings of the 6th international conference on multimodal interfaces, pp 205–211
6.
Zurück zum Zitat Davidson RJ, Ekman P, Saron CD, Senulis JA, Friesen WV (1990) Approach-withdrawal and cerebral asymmetry: emotional expression and brain physiology: I. J Pers Soc Psychol 58(2):330CrossRef Davidson RJ, Ekman P, Saron CD, Senulis JA, Friesen WV (1990) Approach-withdrawal and cerebral asymmetry: emotional expression and brain physiology: I. J Pers Soc Psychol 58(2):330CrossRef
7.
Zurück zum Zitat Grimm M, Kroschel K, Harris H, Nass C, Schuller B, Rigoll G, Moosmayr T (2007) On the necessity and feasibility of detecting a driver’s emotional state while driving. In: International conference on affective computing and intelligent interaction. Springer, pp 126–138 Grimm M, Kroschel K, Harris H, Nass C, Schuller B, Rigoll G, Moosmayr T (2007) On the necessity and feasibility of detecting a driver’s emotional state while driving. In: International conference on affective computing and intelligent interaction. Springer, pp 126–138
8.
Zurück zum Zitat Healey JA, Picard RW (2005) Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans Intell Transp Syst 6(2):156–166CrossRef Healey JA, Picard RW (2005) Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans Intell Transp Syst 6(2):156–166CrossRef
9.
Zurück zum Zitat Jia X, Li K, Li X, Zhang A (2014) A novel semi-supervised deep learning framework for affective state recognition on EEG signals. In: 2014 IEEE international conference on bioinformatics and bioengineering. IEEE, pp 30–37 Jia X, Li K, Li X, Zhang A (2014) A novel semi-supervised deep learning framework for affective state recognition on EEG signals. In: 2014 IEEE international conference on bioinformatics and bioengineering. IEEE, pp 30–37
10.
Zurück zum Zitat Leiner D, Fahr A, Früh H (2012) EDA positive change: a simple algorithm for electrodermal activity to measure general audience arousal during media exposure. Commun Methods Meas 6(4):237–250CrossRef Leiner D, Fahr A, Früh H (2012) EDA positive change: a simple algorithm for electrodermal activity to measure general audience arousal during media exposure. Commun Methods Meas 6(4):237–250CrossRef
11.
Zurück zum Zitat Li K, Li X, Zhang Y, Zhang A (2013) Affective state recognition from EEG with deep belief networks. In: 2013 IEEE international conference on bioinformatics and biomedicine. IEEE, pp 305–310 Li K, Li X, Zhang Y, Zhang A (2013) Affective state recognition from EEG with deep belief networks. In: 2013 IEEE international conference on bioinformatics and biomedicine. IEEE, pp 305–310
12.
Zurück zum Zitat Lisetti CL, Nasoz F (2005) Affective intelligent car interfaces with emotion recognition. In: Proceedings of 11th international conference on human computer interaction. Citeseer, Las Vegas Lisetti CL, Nasoz F (2005) Affective intelligent car interfaces with emotion recognition. In: Proceedings of 11th international conference on human computer interaction. Citeseer, Las Vegas
13.
Zurück zum Zitat Michels N, Sioen I, Clays E, De Buyzere M, Ahrens W, Huybrechts I, Vanaelst B, De Henauw S (2013) Children’s heart rate variability as stress indicator: association with reported stress and cortisol. Biol Psychol 94(2):433–440CrossRef Michels N, Sioen I, Clays E, De Buyzere M, Ahrens W, Huybrechts I, Vanaelst B, De Henauw S (2013) Children’s heart rate variability as stress indicator: association with reported stress and cortisol. Biol Psychol 94(2):433–440CrossRef
14.
Zurück zum Zitat Shekhar S, Maria A, Kotilahti K, Huotilainen M, Heiskala J, Tuulari JJ, Hirvi P, Karlsson L, Karlsson H, Nissilä I (2019) Hemodynamic responses to emotional speech in two-month-old infants imaged using diffuse optical tomography. Sci Rep 9(1):1–15CrossRef Shekhar S, Maria A, Kotilahti K, Huotilainen M, Heiskala J, Tuulari JJ, Hirvi P, Karlsson L, Karlsson H, Nissilä I (2019) Hemodynamic responses to emotional speech in two-month-old infants imaged using diffuse optical tomography. Sci Rep 9(1):1–15CrossRef
15.
Zurück zum Zitat Shu L, Xie J, Yang M, Li Z, Li Z, Liao D, Xu X, Yang X (2018) A review of emotion recognition using physiological signals. Sensors 18(7):2074CrossRef Shu L, Xie J, Yang M, Li Z, Li Z, Liao D, Xu X, Yang X (2018) A review of emotion recognition using physiological signals. Sensors 18(7):2074CrossRef
16.
Zurück zum Zitat Stone M (1974) Cross-validatory choice and assessment of statistical predictions. J Roy Stat Soc Ser B (Methodol) 36(2):111–133MathSciNetMATH Stone M (1974) Cross-validatory choice and assessment of statistical predictions. J Roy Stat Soc Ser B (Methodol) 36(2):111–133MathSciNetMATH
17.
Zurück zum Zitat Tibshirani R (1996) Regression shrinkage and selection via the lasso. J Roy Stat Soc Ser B (Methodol) 58(1):267–288MathSciNetMATH Tibshirani R (1996) Regression shrinkage and selection via the lasso. J Roy Stat Soc Ser B (Methodol) 58(1):267–288MathSciNetMATH
18.
Zurück zum Zitat Umezawa A (1994) Comparison of skin conductance and skin potential as an index in electrodermal biofeedback studies. Jpn Soc Biofeedback Res 21:29–36 (in Japanese) Umezawa A (1994) Comparison of skin conductance and skin potential as an index in electrodermal biofeedback studies. Jpn Soc Biofeedback Res 21:29–36 (in Japanese)
19.
Zurück zum Zitat Villarejo MV, Zapirain BG, Zorrilla AM (2013) Algorithms based on CWT and classifiers to control cardiac alterations and stress using an ECG and a SCR. Sensors 13(5):6141–6170CrossRef Villarejo MV, Zapirain BG, Zorrilla AM (2013) Algorithms based on CWT and classifiers to control cardiac alterations and stress using an ECG and a SCR. Sensors 13(5):6141–6170CrossRef
20.
Zurück zum Zitat Yang G, Lin Y, Bhattacharya P (2010) A driver fatigue recognition model based on information fusion and dynamic Bayesian network. Inf Sci 180(10):1942–1954CrossRef Yang G, Lin Y, Bhattacharya P (2010) A driver fatigue recognition model based on information fusion and dynamic Bayesian network. Inf Sci 180(10):1942–1954CrossRef
21.
Zurück zum Zitat Yin Z, Zhao M, Wang Y, Yang J, Zhang J (2017) Recognition of emotions using multimodal physiological signals and an ensemble deep learning model. Comput Methods Progr Biomed 140:93–110CrossRef Yin Z, Zhao M, Wang Y, Yang J, Zhang J (2017) Recognition of emotions using multimodal physiological signals and an ensemble deep learning model. Comput Methods Progr Biomed 140:93–110CrossRef
22.
Zurück zum Zitat Yuki M, Maddux WW, Masuda T (2007) Are the windows to the soul the same in the east and west? Cultural differences in using the eyes and mouth as cues to recognize emotions in Japan and the United States. J Exp Soc Psychol 43(2):303–311CrossRef Yuki M, Maddux WW, Masuda T (2007) Are the windows to the soul the same in the east and west? Cultural differences in using the eyes and mouth as cues to recognize emotions in Japan and the United States. J Exp Soc Psychol 43(2):303–311CrossRef
Metadaten
Titel
Negative emotion recognition using multimodal physiological signals for advanced driver assistance systems
verfasst von
Chie Hieida
Tomoaki Yamamoto
Takatomi Kubo
Junichiro Yoshimoto
Kazushi Ikeda
Publikationsdatum
21.02.2023
Verlag
Springer Japan
Erschienen in
Artificial Life and Robotics / Ausgabe 2/2023
Print ISSN: 1433-5298
Elektronische ISSN: 1614-7456
DOI
https://doi.org/10.1007/s10015-023-00858-y

Weitere Artikel der Ausgabe 2/2023

Artificial Life and Robotics 2/2023 Zur Ausgabe