Skip to main content
Top

2015 | OriginalPaper | Chapter

Modeling Users Emotional State for an Enhanced Human-Machine Interaction

Authors : David Griol, José Manuel Molina

Published in: Hybrid Artificial Intelligent Systems

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Spoken conversational agents have been proposed to enable a more natural and intuitive interaction with the environment and human-computer interfaces. In this paper, we propose a framework to model the user’s emotional state during the dialog and adapt the dialog model dynamically, thus developing more efficient, adapted, and usable conversational agents. We have evaluated our proposal developing a user-adapted agent that facilitates touristic information, and provide a detailed discussion of the positive influence of our proposal in the success of the interaction, the information and services provided, as well as the perceived quality.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Pieraccini, R., Rabiner, L.: The Voice in the Machine: Building Computers That Understand Speech. MIT Press, Cambridge (2012) Pieraccini, R., Rabiner, L.: The Voice in the Machine: Building Computers That Understand Speech. MIT Press, Cambridge (2012)
2.
go back to reference Osland, P., Viken, B., Solsvik, F., Nygreen, G., Wedvik, J., Myklbust, S.: Enabling context-aware applications. In: Proceedings of the International Conference on Convergence in Services, Media and Networks (ICIN 2006), pp. 1–6 (2006) Osland, P., Viken, B., Solsvik, F., Nygreen, G., Wedvik, J., Myklbust, S.: Enabling context-aware applications. In: Proceedings of the International Conference on Convergence in Services, Media and Networks (ICIN 2006), pp. 1–6 (2006)
3.
go back to reference Strauss, P., Minker, W.: Proactive Spoken Dialogue Interaction in Multi-Party Environments. Springer, US (2010)CrossRefMATH Strauss, P., Minker, W.: Proactive Spoken Dialogue Interaction in Multi-Party Environments. Springer, US (2010)CrossRefMATH
4.
go back to reference Kartakis, S.: A design-and-play approach to accesible user interface development in ambient intelligence environments. J. Comput. Ind. 61(4), 318–328 (2010). Elsevier Kartakis, S.: A design-and-play approach to accesible user interface development in ambient intelligence environments. J. Comput. Ind. 61(4), 318–328 (2010). Elsevier
5.
go back to reference Schuller, B., Batliner, A., Steidl, S., Seppi, D.: Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge. Speech Commun. 53(9–10), 1062–1087 (2011). Elsevier Schuller, B., Batliner, A., Steidl, S., Seppi, D.: Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge. Speech Commun. 53(9–10), 1062–1087 (2011). Elsevier
6.
go back to reference Batliner, A., Burkhardt, F., van Ballegooy, M., Nöth, E.: A taxonomy of applications that utilize emotional awareness. In: Proceedings of the 1st International Language Technologies Conference (IS-LTC 2006), pp. 246–250 (2006) Batliner, A., Burkhardt, F., van Ballegooy, M., Nöth, E.: A taxonomy of applications that utilize emotional awareness. In: Proceedings of the 1st International Language Technologies Conference (IS-LTC 2006), pp. 246–250 (2006)
7.
go back to reference Bickmore, T., Giorgino, T.: Some novel aspects of health communication from a dialogue systems perspective. In: Proceedings of AAAI Fall Symposium on Dialogue Systems for Health Communication, pp. 275–291 (2004) Bickmore, T., Giorgino, T.: Some novel aspects of health communication from a dialogue systems perspective. In: Proceedings of AAAI Fall Symposium on Dialogue Systems for Health Communication, pp. 275–291 (2004)
8.
go back to reference Litman, D., Forbes-Riley, K.: Recognizing student emotions and attitudes on the basis of utterances in spoken tutoring dialogues with both human and computer tutors. Speech Commun. 48(5), 559–590 (2006). Elsevier Litman, D., Forbes-Riley, K.: Recognizing student emotions and attitudes on the basis of utterances in spoken tutoring dialogues with both human and computer tutors. Speech Commun. 48(5), 559–590 (2006). Elsevier
9.
go back to reference Khalifa, O., Ahmad, Z., Gunawan, T.: SMaTTS: standard malay text to speech system. Int. J. Comput. Sci. 2(4), 285–293 (2007). JARCS Khalifa, O., Ahmad, Z., Gunawan, T.: SMaTTS: standard malay text to speech system. Int. J. Comput. Sci. 2(4), 285–293 (2007). JARCS
10.
go back to reference Acosta, J., Ward, N.: Responding to user emotional state by adding emotional coloring to utterances. In: Proceeding of 10th Annual Conference of the International Speech Communication Association (Interspeech 2009), pp. 1587–1590 (2009) Acosta, J., Ward, N.: Responding to user emotional state by adding emotional coloring to utterances. In: Proceeding of 10th Annual Conference of the International Speech Communication Association (Interspeech 2009), pp. 1587–1590 (2009)
11.
go back to reference Boril, H., Hansen, J.: Unsupervised equalization of Lombard effect for speech recognition in noisy adverse environments. IEEE Trans. Audio Speech Lang. Process. 28(6), 1379–1393 (2010). IEEE Boril, H., Hansen, J.: Unsupervised equalization of Lombard effect for speech recognition in noisy adverse environments. IEEE Trans. Audio Speech Lang. Process. 28(6), 1379–1393 (2010). IEEE
12.
go back to reference Bosma, W., Andre, E.: Exploiting emotions to disambiguate dialogue acts. In: Proceedings of 9th International Conference on Intelligent User Interface, pp. 85–92 (2004) Bosma, W., Andre, E.: Exploiting emotions to disambiguate dialogue acts. In: Proceedings of 9th International Conference on Intelligent User Interface, pp. 85–92 (2004)
13.
go back to reference Wilks, Y., Catizone, R., Worgan, S., Turunen, M.: Some background on dialogue management and conversational speech for dialogue systems. Comput. Speech Lang. 25(2), 128–139 (2011). Elsevier Wilks, Y., Catizone, R., Worgan, S., Turunen, M.: Some background on dialogue management and conversational speech for dialogue systems. Comput. Speech Lang. 25(2), 128–139 (2011). Elsevier
14.
go back to reference Riccardi, G., Hakkani-Tür, D.: Grounding emotions in human-machine conversational systems. In: Maybury, M., Stock, O., Wahlster, W. (eds.) INTETAIN 2005. LNCS (LNAI), vol. 3814, pp. 144–154. Springer, Heidelberg (2005) CrossRef Riccardi, G., Hakkani-Tür, D.: Grounding emotions in human-machine conversational systems. In: Maybury, M., Stock, O., Wahlster, W. (eds.) INTETAIN 2005. LNCS (LNAI), vol. 3814, pp. 144–154. Springer, Heidelberg (2005) CrossRef
15.
go back to reference Marreiros, G., Santos, R., Ramos, C., Neves, J.: Context-aware emotion-based model for group decision making. IEEE Intell. Syst. 25(2), 31–39 (2010). IEEE Marreiros, G., Santos, R., Ramos, C., Neves, J.: Context-aware emotion-based model for group decision making. IEEE Intell. Syst. 25(2), 31–39 (2010). IEEE
16.
go back to reference Santos, R., Marreiros, G., Ramos, C., Neves, J., Bulas-Cruz, J.: Personality, emotion, and mood in agent-based group decision making. IEEE Intell. Syst. 26(6), 58–66 (2011). IEEE Santos, R., Marreiros, G., Ramos, C., Neves, J., Bulas-Cruz, J.: Personality, emotion, and mood in agent-based group decision making. IEEE Intell. Syst. 26(6), 58–66 (2011). IEEE
17.
go back to reference Pittermann, J., Pittermann, A., Minker, W.: Emotion recognition and adaptation in spoken dialogue systems. Int. J. Speech Technol. 13, 49–60 (2010). Springer Pittermann, J., Pittermann, A., Minker, W.: Emotion recognition and adaptation in spoken dialogue systems. Int. J. Speech Technol. 13, 49–60 (2010). Springer
18.
go back to reference Bui, T., Poel, M., Nijholt, A., Zwiers, J.: A tractable hybrid DDN-POMDP approach to affective dialogue modeling for probabilistic frame-based dialogue systems. Nat. Lang. Eng. 15(2), 273–307 (2009). Cambridge University Press Bui, T., Poel, M., Nijholt, A., Zwiers, J.: A tractable hybrid DDN-POMDP approach to affective dialogue modeling for probabilistic frame-based dialogue systems. Nat. Lang. Eng. 15(2), 273–307 (2009). Cambridge University Press
19.
go back to reference Williams, J., Young, S.: Partially observable Markov decision processes for spoken dialogue systems. Comput. Speech Lang. 21, 393–422 (2007). Elsevier Williams, J., Young, S.: Partially observable Markov decision processes for spoken dialogue systems. Comput. Speech Lang. 21, 393–422 (2007). Elsevier
20.
go back to reference Callejas, Z., López-Cózar, R.: Influence of contextual information in emotion annotation for spoken dialogue systems. Speech Commun. 50(5), 416–433 (2008). Elsevier Callejas, Z., López-Cózar, R.: Influence of contextual information in emotion annotation for spoken dialogue systems. Speech Commun. 50(5), 416–433 (2008). Elsevier
21.
go back to reference Witten, I., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, San Francisco (2005) Witten, I., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, San Francisco (2005)
22.
go back to reference Hansen, J.: Analysis and compensation of speech under stress and noise for environmental robustness in speech recognition. Speech Commun. 20(2), 151–170 (1996). Elsevier Hansen, J.: Analysis and compensation of speech under stress and noise for environmental robustness in speech recognition. Speech Commun. 20(2), 151–170 (1996). Elsevier
23.
go back to reference Ververidis, D., Kotropoulos, C.: Emotional speech recognition: resources, features and methods. Speech Commun. 48, 1162–1181 (2006). Elsevier Ververidis, D., Kotropoulos, C.: Emotional speech recognition: resources, features and methods. Speech Commun. 48, 1162–1181 (2006). Elsevier
24.
go back to reference Morrison, D., Wang, R., Silva, L.: Ensemble methods for spoken emotion recognition in call-centers. Speech Commun. 49(2), 98–112 (2007). Elsevier Morrison, D., Wang, R., Silva, L.: Ensemble methods for spoken emotion recognition in call-centers. Speech Commun. 49(2), 98–112 (2007). Elsevier
25.
go back to reference Batliner, A., Steidl, S., Schuller, B., Seppi, D., Vogt, T., Wagner, J., Devillers, L., Vidrascu, L., Aharonson, V., Kessous, L., Amir, N.: Whodunnit - searching for the most important feature types signalling emotion-related user states in speech. Comput. Speech Lang. 25(1), 4–28 (2011). Elsevier Batliner, A., Steidl, S., Schuller, B., Seppi, D., Vogt, T., Wagner, J., Devillers, L., Vidrascu, L., Aharonson, V., Kessous, L., Amir, N.: Whodunnit - searching for the most important feature types signalling emotion-related user states in speech. Comput. Speech Lang. 25(1), 4–28 (2011). Elsevier
26.
go back to reference Burkhardt, F., van Ballegooy, M., Engelbrecht, K., Polzehl, T., Stegmann, J.: Emotion detection in dialog systems - usecases, strategies and challenges. In: Proceedings of International Conference on Affective Computing and Intelligent Interaction (ACII 2009), pp. 1–6 (2009) Burkhardt, F., van Ballegooy, M., Engelbrecht, K., Polzehl, T., Stegmann, J.: Emotion detection in dialog systems - usecases, strategies and challenges. In: Proceedings of International Conference on Affective Computing and Intelligent Interaction (ACII 2009), pp. 1–6 (2009)
27.
go back to reference Griol, D., García-Jiménez, M.: Development of interactive virtual voice portals to provide municipal information. Adv. Intell. Soft Comput. 151, 161–172 (2012). Springer Griol, D., García-Jiménez, M.: Development of interactive virtual voice portals to provide municipal information. Adv. Intell. Soft Comput. 151, 161–172 (2012). Springer
28.
go back to reference Will, T.: A Simple Guide to IBM SPSS: For Version 20.0. Akademiker Verlag (2012) Will, T.: A Simple Guide to IBM SPSS: For Version 20.0. Akademiker Verlag (2012)
29.
go back to reference Ai, H., Raux, A., Bohus, D., Eskenazi, M., Litman, D.: Comparing spoken dialog corpora collected with recruited subjects versus real users. In: Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue, pp. 124–131 (2007) Ai, H., Raux, A., Bohus, D., Eskenazi, M., Litman, D.: Comparing spoken dialog corpora collected with recruited subjects versus real users. In: Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue, pp. 124–131 (2007)
30.
go back to reference Schatzmann, J., Georgila, K., Young, S.: Quantitative evaluation of user simulation techniques for spoken dialogue systems. In: Proceedings of the 6th SIGdial Workshop on Discourse and Dialogue, pp. 45–54 (2005) Schatzmann, J., Georgila, K., Young, S.: Quantitative evaluation of user simulation techniques for spoken dialogue systems. In: Proceedings of the 6th SIGdial Workshop on Discourse and Dialogue, pp. 45–54 (2005)
Metadata
Title
Modeling Users Emotional State for an Enhanced Human-Machine Interaction
Authors
David Griol
José Manuel Molina
Copyright Year
2015
DOI
https://doi.org/10.1007/978-3-319-19644-2_30

Premium Partner