Skip to main content

2017 | OriginalPaper | Buchkapitel

Bimodal Recognition of Cognitive Load Based on Speech and Physiological Changes

verfasst von : Dennis Held, Sascha Meudt, Friedhelm Schwenker

Erschienen in: Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

An essential component of the interaction between humans is the reaction through their emotional intelligence to emotional states of the counterpart and respond appropriately. This kind of action results in a successful interpersonal communication. The first step to achieve this goal within HCI is the identification of these emotional states.
This paper deals with the development of procedures and an automated classification system for recognition of mental overload and mental underload utilizing speech an physiological signals. Mental load states are induced through easy and tedious tasks for mental underload and complex and hard tasks for mental overload. It will be shown, how to select suitable features, build uni modal classifiers which then are combined to a bimodal mental load estimation by the use of early and late fusion. Additionally the impact of speech artifacts on physiological data is investigated.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Ekman, P.: An argument for basic emotions. Cogn. Emot. 6(3–4), 169–200 (1992)CrossRef Ekman, P.: An argument for basic emotions. Cogn. Emot. 6(3–4), 169–200 (1992)CrossRef
2.
Zurück zum Zitat Gevins, A., Smith, M.E., Leong, H., McEvoy, L., Whitfield, S., Du, R., Rush, G.: Monitoring working memory load during computer-based tasks with EEG pattern recognition methods. Hum. Factors J. Hum. Factors Ergon. Soc. 40(1), 79–91 (1998)CrossRef Gevins, A., Smith, M.E., Leong, H., McEvoy, L., Whitfield, S., Du, R., Rush, G.: Monitoring working memory load during computer-based tasks with EEG pattern recognition methods. Hum. Factors J. Hum. Factors Ergon. Soc. 40(1), 79–91 (1998)CrossRef
3.
Zurück zum Zitat Haapalainen, E., Kim, S., Forlizzi, J.F., Dey, A.K.: Psycho-physiological measures for assessing cognitive load. In: Proceedings of the 12th ACM international conference on Ubiquitous computing, pp. 301–310. ACM (2010) Haapalainen, E., Kim, S., Forlizzi, J.F., Dey, A.K.: Psycho-physiological measures for assessing cognitive load. In: Proceedings of the 12th ACM international conference on Ubiquitous computing, pp. 301–310. ACM (2010)
4.
Zurück zum Zitat Kim, J.: Bimodal emotion recognition using speech and physiological changes. Citeseer (2007) Kim, J.: Bimodal emotion recognition using speech and physiological changes. Citeseer (2007)
5.
Zurück zum Zitat Kim, J., André, E.: Emotion recognition based on physiological changes in music listening. IEEE Trans. Pattern Anal. Mach. Intell. 30(12), 2067–2083 (2008)CrossRef Kim, J., André, E.: Emotion recognition based on physiological changes in music listening. IEEE Trans. Pattern Anal. Mach. Intell. 30(12), 2067–2083 (2008)CrossRef
6.
Zurück zum Zitat Mayer, J.D., Geher, G.: Emotional intelligence and the identification of emotion. Intelligence 22(2), 89–113 (1996)CrossRef Mayer, J.D., Geher, G.: Emotional intelligence and the identification of emotion. Intelligence 22(2), 89–113 (1996)CrossRef
7.
Zurück zum Zitat Meudt, S., Zharkov, D., Kächele, M., Schwenker, F.: Multi classifier systems and forward backward feature selection algorithms to classify emotional coloured speech. In: Proceedings of the 15th ACM on International conference on multimodal interaction, pp. 551–556. ACM (2013) Meudt, S., Zharkov, D., Kächele, M., Schwenker, F.: Multi classifier systems and forward backward feature selection algorithms to classify emotional coloured speech. In: Proceedings of the 15th ACM on International conference on multimodal interaction, pp. 551–556. ACM (2013)
8.
Zurück zum Zitat Pantic, M., Rothkrantz, L.J.: Toward an affect-sensitive multimodal human-computer interaction. Proc. IEEE 91(9), 1370–1390 (2003)CrossRef Pantic, M., Rothkrantz, L.J.: Toward an affect-sensitive multimodal human-computer interaction. Proc. IEEE 91(9), 1370–1390 (2003)CrossRef
9.
Zurück zum Zitat Phinyomark, A., Limsakul, C., Phukpattaranont, P.: A novel feature extraction for robust emg pattern recognition. arXiv preprint arXiv:0912.3973 (2009) Phinyomark, A., Limsakul, C., Phukpattaranont, P.: A novel feature extraction for robust emg pattern recognition. arXiv preprint arXiv:​0912.​3973 (2009)
10.
Zurück zum Zitat Picard, R.W.: Affective Computing, vol. 252. MIT Press, Cambridge (1997) Picard, R.W.: Affective Computing, vol. 252. MIT Press, Cambridge (1997)
11.
Zurück zum Zitat Scherer, K.R.: Speech and emotional states. In: Speech Evaluation in Psychiatry, pp. 189–220 (1981) Scherer, K.R.: Speech and emotional states. In: Speech Evaluation in Psychiatry, pp. 189–220 (1981)
12.
Zurück zum Zitat Schüssel, F., Honold, F., Bubalo, N., Huckauf, A., Traue, H., Hazer-Rau, D.: In-depth analysis of multimodal interaction: an explorative paradigm. In: Kurosu, M. (ed.) HCI 2016. LNCS, vol. 9732, pp. 233–240. Springer, Cham (2016). doi:10.1007/978-3-319-39516-6_22 CrossRef Schüssel, F., Honold, F., Bubalo, N., Huckauf, A., Traue, H., Hazer-Rau, D.: In-depth analysis of multimodal interaction: an explorative paradigm. In: Kurosu, M. (ed.) HCI 2016. LNCS, vol. 9732, pp. 233–240. Springer, Cham (2016). doi:10.​1007/​978-3-319-39516-6_​22 CrossRef
13.
Zurück zum Zitat Tang, X., Shu, L.: Classification of electrocardiogram signals with rs and quantum neural networks. Int. J. Multimedia Ubiquit. Eng. 9(2), 363–372 (2014)CrossRef Tang, X., Shu, L.: Classification of electrocardiogram signals with rs and quantum neural networks. Int. J. Multimedia Ubiquit. Eng. 9(2), 363–372 (2014)CrossRef
14.
Zurück zum Zitat Tkach, D., Huang, H., Kuiken, T.A.: Study of stability of time-domain features for electromyographic pattern recognition. J. Neuroengineering Rehabil. 7(1), 1 (2010)CrossRef Tkach, D., Huang, H., Kuiken, T.A.: Study of stability of time-domain features for electromyographic pattern recognition. J. Neuroengineering Rehabil. 7(1), 1 (2010)CrossRef
15.
Zurück zum Zitat Zhao, Q., Zhang, L.: ECG feature extraction and classification using wavelet transform and support vector machines. In: International Conference on Neural Networks and Brain, ICNN&B 2005, vol. 2, pp. 1089–1092. IEEE (2005) Zhao, Q., Zhang, L.: ECG feature extraction and classification using wavelet transform and support vector machines. In: International Conference on Neural Networks and Brain, ICNN&B 2005, vol. 2, pp. 1089–1092. IEEE (2005)
Metadaten
Titel
Bimodal Recognition of Cognitive Load Based on Speech and Physiological Changes
verfasst von
Dennis Held
Sascha Meudt
Friedhelm Schwenker
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-59259-6_2