Skip to main content

2018 | OriginalPaper | Buchkapitel

An Exploratory Study of Multimodal Perception for Affective Computing System Design

verfasst von : Chih-Hung Wu, Bor-Chen Kuo

Erschienen in: Frontier Computing

Verlag: Springer Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Affective computing (AC) is an emerging research direction to deal with the great challenge of creating emotional intelligence for a machine. Affective computing is a cross-disciplinary research knowledge that integrates recognition, interpretation, and simulation of human emotion into a system. This article describes the design of multimodal perception of affective computing system. Our multimodal physiological channels include facial expression recognition, heart rate monitoring, blood oxygen level (SpO2), skin conductance response (SCR), and electroencephalogram (EEG) signals for building our affective computing system. To solve the various data sampling problem, we developed a concurrent control integration mechanism to automatically average all the sensors’ data into the same data sampling rate (one record per second) and rearrange all the data into the same time. We believed the proposed system design is benefit for helping researchers in collecting and integrating experiment data in affective computing area.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Wu, C.-H., Huang, Y.-M., Hwang, J.-P.: Review of affective computing in education/learning: trends and challenges. Br. J. Educ. Technol. 47, 1304–1323 (2016)CrossRef Wu, C.-H., Huang, Y.-M., Hwang, J.-P.: Review of affective computing in education/learning: trends and challenges. Br. J. Educ. Technol. 47, 1304–1323 (2016)CrossRef
2.
Zurück zum Zitat Piaget, J.: Les Relations entre l’affectivité et l’intelligence dans le développement mental de l’enfant. Centre de documentation universitaire, Paris (1964) Piaget, J.: Les Relations entre l’affectivité et l’intelligence dans le développement mental de l’enfant. Centre de documentation universitaire, Paris (1964)
3.
Zurück zum Zitat Ben Ammar, M., Neji, M., Alimi, A.M., Gouardères, G.: The affective tutoring system. Expert Syst. Appl. 37, 3013–3023 (2010)CrossRef Ben Ammar, M., Neji, M., Alimi, A.M., Gouardères, G.: The affective tutoring system. Expert Syst. Appl. 37, 3013–3023 (2010)CrossRef
4.
Zurück zum Zitat Dimigen, O., Sommer, W., Hohlfeld, A., Jacobs, A.M., Kliegl, R.: Coregistration of eye movements and EEG in natural reading: analyses and review. J. Exp. Psychol. Gen. 140, 552–572 (2011)CrossRef Dimigen, O., Sommer, W., Hohlfeld, A., Jacobs, A.M., Kliegl, R.: Coregistration of eye movements and EEG in natural reading: analyses and review. J. Exp. Psychol. Gen. 140, 552–572 (2011)CrossRef
5.
Zurück zum Zitat Latanov, A.V., Konovalova, N.S., Yermachenko, A.A.: EEG and EYE tracking for visual search task investigation in humans. Int. J. Psychophysiol. 69, 140 (2008)CrossRef Latanov, A.V., Konovalova, N.S., Yermachenko, A.A.: EEG and EYE tracking for visual search task investigation in humans. Int. J. Psychophysiol. 69, 140 (2008)CrossRef
6.
Zurück zum Zitat Lin, T., Imamiya, A., Mao, X.: Using multiple data sources to get closer insights into user cost and task performance. Interact. Comput. 20, 364–374 (2008)CrossRef Lin, T., Imamiya, A., Mao, X.: Using multiple data sources to get closer insights into user cost and task performance. Interact. Comput. 20, 364–374 (2008)CrossRef
7.
Zurück zum Zitat Schmid, P.C., Schmid Mast, M., Bombari, D., Mast, F.W., Lobmaier, J.S.: How mood states affect information processing during facial emotion recognition: an eye tracking study. Swiss J. Psychol. 70, 223–231 (2011)CrossRef Schmid, P.C., Schmid Mast, M., Bombari, D., Mast, F.W., Lobmaier, J.S.: How mood states affect information processing during facial emotion recognition: an eye tracking study. Swiss J. Psychol. 70, 223–231 (2011)CrossRef
8.
Zurück zum Zitat Chen, C.-M., Wang, H.-P.: Using emotion recognition technology to assess the effects of different multimedia materials on learning emotion and performance. Libr. Inf. Sci. Res. 33, 244–255 (2011)CrossRef Chen, C.-M., Wang, H.-P.: Using emotion recognition technology to assess the effects of different multimedia materials on learning emotion and performance. Libr. Inf. Sci. Res. 33, 244–255 (2011)CrossRef
9.
Zurück zum Zitat Zhang, C., Zheng, C.-X., Yu, X.-L.: Automatic recognition of cognitive fatigue from physiological indices by using wavelet packet transform and kernel learning algorithms. Exp. Syst. Appl. 36, 4664–4671 (2009)CrossRef Zhang, C., Zheng, C.-X., Yu, X.-L.: Automatic recognition of cognitive fatigue from physiological indices by using wavelet packet transform and kernel learning algorithms. Exp. Syst. Appl. 36, 4664–4671 (2009)CrossRef
10.
Zurück zum Zitat Patel, M., Lal, S.K.L., Kavanagh, D., Rossiter, P.: Applying neural network analysis on heart rate variability data to assess driver fatigue. Exp. Syst. Appl. 38, 7235–7242 (2011)CrossRef Patel, M., Lal, S.K.L., Kavanagh, D., Rossiter, P.: Applying neural network analysis on heart rate variability data to assess driver fatigue. Exp. Syst. Appl. 38, 7235–7242 (2011)CrossRef
11.
Zurück zum Zitat Zhao, C., Zhao, M., Liu, J., Zheng, C.: Electroencephalogram and electrocardiograph assessment of mental fatigue in a driving simulator. Accid. Anal. Prev. 45, 83–90 (2012)CrossRef Zhao, C., Zhao, M., Liu, J., Zheng, C.: Electroencephalogram and electrocardiograph assessment of mental fatigue in a driving simulator. Accid. Anal. Prev. 45, 83–90 (2012)CrossRef
12.
Zurück zum Zitat Bahreini, K., Nadolski, R., Westera, W.: Towards multimodal emotion recognition in e-learning environments. Interact. Learn. Env. 24, 590–605 (2016)CrossRef Bahreini, K., Nadolski, R., Westera, W.: Towards multimodal emotion recognition in e-learning environments. Interact. Learn. Env. 24, 590–605 (2016)CrossRef
13.
Zurück zum Zitat Wu, C.H.: New technology for developing facial expression recognition in e-learning. In: 2016 Portland International Conference on Management of Engineering and Technology (PICMET), pp. 1719–1722 (2016) Wu, C.H.: New technology for developing facial expression recognition in e-learning. In: 2016 Portland International Conference on Management of Engineering and Technology (PICMET), pp. 1719–1722 (2016)
14.
Zurück zum Zitat Chanel, G., Ansari-Asl, K., Pun, T.: Valence-arousal evaluation using physiological signals in an emotion recall paradigm. Lecturer Notes in Computer Science, vol. 1, pp. 530–537(2007) Chanel, G., Ansari-Asl, K., Pun, T.: Valence-arousal evaluation using physiological signals in an emotion recall paradigm. Lecturer Notes in Computer Science, vol. 1, pp. 530–537(2007)
15.
Zurück zum Zitat Soleymani, M., Lichtenauer, J., Pun, T., Pantic, M.: A multimodal database for affect recognition and implicit tagging. IEEE Trans. Affect. Comput. 3, 42–55 (2012)CrossRef Soleymani, M., Lichtenauer, J., Pun, T., Pantic, M.: A multimodal database for affect recognition and implicit tagging. IEEE Trans. Affect. Comput. 3, 42–55 (2012)CrossRef
16.
Zurück zum Zitat Chen, L., Zhou, C., Shen, L.: Facial expression recognition based on SVM in E-learning. IERI Procedia 2, 781–787 (2012)CrossRef Chen, L., Zhou, C., Shen, L.: Facial expression recognition based on SVM in E-learning. IERI Procedia 2, 781–787 (2012)CrossRef
17.
Zurück zum Zitat Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C.M., Kazemzadeh, A., Lee, S., Neumann, U., Narayanan, S.: Analysis of emotion recognition using facial expressions, speech and multimodal information. In: Sixth International Conference on Multimodal Interfaces ICMI 2004 (2004) Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C.M., Kazemzadeh, A., Lee, S., Neumann, U., Narayanan, S.: Analysis of emotion recognition using facial expressions, speech and multimodal information. In: Sixth International Conference on Multimodal Interfaces ICMI 2004 (2004)
18.
Zurück zum Zitat Lin, H.-C.K., Wu, C.-H., Hsueh, Y.-P.: The influence of using affective tutoring system in accounting remedial instruction on learning performance and usability. Comput. Hum. Behav. 41, 514–522 (2014)CrossRef Lin, H.-C.K., Wu, C.-H., Hsueh, Y.-P.: The influence of using affective tutoring system in accounting remedial instruction on learning performance and usability. Comput. Hum. Behav. 41, 514–522 (2014)CrossRef
19.
Zurück zum Zitat Thompson, N., McGill, T.J.: Genetics with jean: the design, development and evaluation of an affective tutoring system. Educ. Tech. Res. Dev. 65, 279–299 (2017)CrossRef Thompson, N., McGill, T.J.: Genetics with jean: the design, development and evaluation of an affective tutoring system. Educ. Tech. Res. Dev. 65, 279–299 (2017)CrossRef
20.
Zurück zum Zitat Gonzalez-Sanchez, J., Chavez-Echeagaray, M.E., Atkinson, R., Burleson, W.: ABE: an agent-based software architecture for a multimodal emotion recognition framework. In: 2011 9th Working IEEE/IFIP Conference on Software Architecture (WICSA), pp. 187–193 (2011) Gonzalez-Sanchez, J., Chavez-Echeagaray, M.E., Atkinson, R., Burleson, W.: ABE: an agent-based software architecture for a multimodal emotion recognition framework. In: 2011 9th Working IEEE/IFIP Conference on Software Architecture (WICSA), pp. 187–193 (2011)
Metadaten
Titel
An Exploratory Study of Multimodal Perception for Affective Computing System Design
verfasst von
Chih-Hung Wu
Bor-Chen Kuo
Copyright-Jahr
2018
Verlag
Springer Singapore
DOI
https://doi.org/10.1007/978-981-10-7398-4_20

Premium Partner