Skip to main content

2017 | OriginalPaper | Buchkapitel

Boredom Recognition Based on Users’ Spontaneous Behaviors in Multiparty Human-Robot Interactions

verfasst von : Yasuhiro Shibasaki, Kotaro Funakoshi, Koichi Shinoda

Erschienen in: MultiMedia Modeling

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

To recognize boredom in users interacting with machines is valuable to improve user experiences in human-machine long term interactions, especially for intelligent tutoring systems, health-care systems, and social assistants. This paper proposes a two-staged framework and feature design for boredom recognition in multiparty human-robot interactions. At the first stage the proposed framework detects boredom-indicating user behaviors based on skeletal data obtained by motion capture, and then it recognizes boredom in combination with detection results and two types of multiparty information, i.e., gaze direction to other participants and incoming-and-outgoing of participants. We experimentally confirmed the effectiveness of both the proposed framework and the multiparty information. In comparison with a simple baseline method, the proposed framework gained 35% points in the F1 score.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
This is intended to make an annotator’s judgement on an interval independent of the neighboring intervals as much as possible in a reasonable cost of annotation. In forward annotation, the influence of the judgement of an interval on the judgement of the next interval could be larger. The playback of each interval was of course forwarded.
 
2
Although it is known that there are several types of boredom [6], we employ a naive notion of boredom for the sake of simplicity.
 
3
The meanings of strong/weak are different between the boredom state and the other boredom behavior categories (BB-*). For the boredom state, ‘strong’, ‘weak’, and ‘none’ respectively mean the samples with boredom state labels ‘BORED’, ‘MAYBE’, and ‘NOT BORED’. For the other behavior categories, ‘strong’ means the samples that contain relevant behaviors more than 2.5 s. ‘weak’ means those less than 2.5 s, and ‘none’ means those with no relevant behaviors.
 
4
‘#targets’ means the numbers of samples in column ‘strong’ shown in Table 2. ‘#residuals’ means the sums of the numbers of the ‘weak’ and ‘none’ samples.
 
Literatur
1.
Zurück zum Zitat Bixler, R., D’Mello, S.: Detecting boredom and engagement during writing with keystroke analysis, task appraisals, and stable traits. In: 2013 International Conference on Intelligent User Interfaces, pp. 225–234 (2013) Bixler, R., D’Mello, S.: Detecting boredom and engagement during writing with keystroke analysis, task appraisals, and stable traits. In: 2013 International Conference on Intelligent User Interfaces, pp. 225–234 (2013)
2.
Zurück zum Zitat Brand, M., Oliver, N., Pentland, A.: Coupled hidden markov models for complex action recognition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 994–999 (1997) Brand, M., Oliver, N., Pentland, A.: Coupled hidden markov models for complex action recognition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 994–999 (1997)
3.
Zurück zum Zitat Breazeal, C., Aryananda, L.: Recognition of affective communicative intent in robot-directed speech. Auton. Robots 12, 83–104 (2002)CrossRefMATH Breazeal, C., Aryananda, L.: Recognition of affective communicative intent in robot-directed speech. Auton. Robots 12, 83–104 (2002)CrossRefMATH
4.
Zurück zum Zitat Castellano, G., Pereira, A., Leite, I., Paiva, A., McOwan, P.W.: Detecting user engagement with a robot companion using task and social interaction-based features. In: 2009 International Conference on Multimodal Interfaces, pp. 119–126 (2009) Castellano, G., Pereira, A., Leite, I., Paiva, A., McOwan, P.W.: Detecting user engagement with a robot companion using task and social interaction-based features. In: 2009 International Conference on Multimodal Interfaces, pp. 119–126 (2009)
5.
Zurück zum Zitat D’Mello, S., Chipman, P., Graesser, A.: Posture as a predictor of learner’s affective engagement. In: 29th Annual Conference of the Cognitive Science Society, pp. 905–910 (2007) D’Mello, S., Chipman, P., Graesser, A.: Posture as a predictor of learner’s affective engagement. In: 29th Annual Conference of the Cognitive Science Society, pp. 905–910 (2007)
6.
Zurück zum Zitat Goetz, T., Frenzel, A.C., Hall, N.C., Nett, U.E., Pekrun, R., Lipnevich, A.A.: Types of boredom: an experience sampling approach. Motiv. Emot. 38(3), 401–419 (2014)CrossRef Goetz, T., Frenzel, A.C., Hall, N.C., Nett, U.E., Pekrun, R., Lipnevich, A.A.: Types of boredom: an experience sampling approach. Motiv. Emot. 38(3), 401–419 (2014)CrossRef
7.
Zurück zum Zitat Gratch, J., Wang, N., Okhmatovskaia, A., Lamothe, F., Morales, M., Werf, R.J., Morency, L.-P.: Can virtual humans be more engaging than real ones? In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4552, pp. 286–297. Springer, Heidelberg (2007)CrossRef Gratch, J., Wang, N., Okhmatovskaia, A., Lamothe, F., Morales, M., Werf, R.J., Morency, L.-P.: Can virtual humans be more engaging than real ones? In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4552, pp. 286–297. Springer, Heidelberg (2007)CrossRef
8.
Zurück zum Zitat Hudlicka, E.: To feel or not to feel: the role of affect in human-computer interaction. Int. J. Hum. Comput. Stud. 59(1), 1–32 (2003)CrossRef Hudlicka, E.: To feel or not to feel: the role of affect in human-computer interaction. Int. J. Hum. Comput. Stud. 59(1), 1–32 (2003)CrossRef
9.
Zurück zum Zitat Jacko, J.A.: Human Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications. CRC Press, Boca Raton (2012)CrossRef Jacko, J.A.: Human Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications. CRC Press, Boca Raton (2012)CrossRef
10.
Zurück zum Zitat Jacobs, A.M., Fransen, B., McCurry, J.M., Heckel, F.W., Wagner, A.R., Trafton, J.G.: A preliminary system for recognizing boredom. In: 4th ACM/IEEE International Conference on Human-Robot Interaction, pp. 299–300 (2009) Jacobs, A.M., Fransen, B., McCurry, J.M., Heckel, F.W., Wagner, A.R., Trafton, J.G.: A preliminary system for recognizing boredom. In: 4th ACM/IEEE International Conference on Human-Robot Interaction, pp. 299–300 (2009)
11.
Zurück zum Zitat Kennedy, L.S., Ellis, D.P.: Pitch-based emphasis detection for characterization of meeting recordings. In: 2003 IEEE Workshop on Automatic Speech Recognition and Understanding, pp. 243–248 (2003) Kennedy, L.S., Ellis, D.P.: Pitch-based emphasis detection for characterization of meeting recordings. In: 2003 IEEE Workshop on Automatic Speech Recognition and Understanding, pp. 243–248 (2003)
12.
Zurück zum Zitat Liu, Y., Chee, Y.S.: Intelligent pedagogical agents with multiparty interaction support. In: IEEE/WIC/ACM International Conference on Intelligent Agent Technology, pp. 134–140 (2004) Liu, Y., Chee, Y.S.: Intelligent pedagogical agents with multiparty interaction support. In: IEEE/WIC/ACM International Conference on Intelligent Agent Technology, pp. 134–140 (2004)
13.
Zurück zum Zitat Mehrabian, A.: Silent messages. Wadsworth, Belmont (1971) Mehrabian, A.: Silent messages. Wadsworth, Belmont (1971)
14.
Zurück zum Zitat Mota, S., Picard, R.W.: Automated posture analysis for detecting learner’s interest level. In: Computer Vision and Pattern Recognition Workshop, pp. 49–49 (2003) Mota, S., Picard, R.W.: Automated posture analysis for detecting learner’s interest level. In: Computer Vision and Pattern Recognition Workshop, pp. 49–49 (2003)
15.
Zurück zum Zitat Pantic, M., Caridakis, G., André, E., Kim, J., Karpouzis, K., Kollias, S.: Multimodal emotion recognition from low-level cues. In: Petta, P., Pelachaud, C., Cowie, R. (eds.) Emotion-Oriented Systems, pp. 115–132. Springer, Heidelberg (2011)CrossRef Pantic, M., Caridakis, G., André, E., Kim, J., Karpouzis, K., Kollias, S.: Multimodal emotion recognition from low-level cues. In: Petta, P., Pelachaud, C., Cowie, R. (eds.) Emotion-Oriented Systems, pp. 115–132. Springer, Heidelberg (2011)CrossRef
16.
Zurück zum Zitat Qvarfordt, P., Beymer, D., Zhai, S.: RealTourist – a study of augmenting human-human and human-computer dialogue with eye-gaze overlay. In: Costabile, M.F., Paternò, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 767–780. Springer, Heidelberg (2005). doi:10.1007/11555261_61 CrossRef Qvarfordt, P., Beymer, D., Zhai, S.: RealTourist – a study of augmenting human-human and human-computer dialogue with eye-gaze overlay. In: Costabile, M.F., Paternò, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 767–780. Springer, Heidelberg (2005). doi:10.​1007/​11555261_​61 CrossRef
17.
Zurück zum Zitat Sugiyama, T., Funakoshi, K., Nakano, M., Komatani, K.: Estimating response obligation in multi-party human-robot dialogues. In: 2015 IEEE-RAS 15th International Conference on Humanoid Robots, pp. 166–172 (2015) Sugiyama, T., Funakoshi, K., Nakano, M., Komatani, K.: Estimating response obligation in multi-party human-robot dialogues. In: 2015 IEEE-RAS 15th International Conference on Humanoid Robots, pp. 166–172 (2015)
Metadaten
Titel
Boredom Recognition Based on Users’ Spontaneous Behaviors in Multiparty Human-Robot Interactions
verfasst von
Yasuhiro Shibasaki
Kotaro Funakoshi
Koichi Shinoda
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-51811-4_55

Neuer Inhalt