Top

Published in:

2016 | OriginalPaper | Chapter

Multimodal Detection of Engagement in Groups of Children Using Rank Learning

Authors : Jaebok Kim, Khiet P. Truong, Vicky Charisi, Cristina Zaga, Vanessa Evers, Mohamed Chetouani

Published in: Human Behavior Understanding

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

In collaborative play, children exhibit different levels of engagement. Some children are engaged with other children while some play alone. In this study, we investigated multimodal detection of individual levels of engagement using a ranking method and non-verbal features: turn-taking and body movement. Firstly, we automatically extracted turn-taking and body movement features in naturalistic and challenging settings. Secondly, we used an ordinal annotation scheme and employed a ranking method considering the great heterogeneity and temporal dynamics of engagement that exist in interactions. We showed that levels of engagement can be characterised by relative levels between children. In particular, a ranking method, Ranking SVM, outperformed a conventional method, SVM classification. While either turn-taking or body movement features alone did not achieve promising results, combining the two features yielded significant error reduction, showing their complementary power.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Assessing Affective Dimensions of Play in Psychodynamic Child Psychotherapy via Text Analysis

next chapter Anomaly Detection in Elderly Daily Behavior in Ambient Sensing Environments

Aggarwal, J.K., Park, S.: Human motion: modeling and recognition of actions and interactions. In: Proceedings of 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004, pp. 640–647. IEEE (2004)

Al Moubayed, S., Lehman, J.: Toward better understanding of engagement in multiparty spoken interaction with children. In: Proceedings of the International Conference on Multimodal Interaction, pp. 211–218. ACM (2015)

Antić, B., Letić, D., Crnojević, V., et al.: K-means based segmentation for real-time zenithal people counting. In: Proceedings of International Conference on Image Processing (ICIP), pp. 2565–2568. IEEE (2009)

Anzalone, S.M., Boucenna, S., Ivaldi, S., Chetouani, M.: Evaluating the engagement with social robots. Int. J. Soc. Robot. 7(4), 465–478 (2015)CrossRef

Argyle, M.: Social Interaction, vol. 103. Transaction Publishers (1973)

Bianchi-Berthouze, N., Kim, W.W., Patel, D.: Does body movement engage you more in digital game play? and why? In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 102–113. Springer, Heidelberg (2007). doi:10.1007/978-3-540-74889-2_10 CrossRef

Oertel gen bierbach, C.: On the use of multimodal cues for the prediction of involvement in spontaneous conversation. In: Proceedings of the INTERSPEECH, pp. 1541–1544 (2011)

Bobick, A.F.: Movement, activity and action: the role of knowledge in the perception of motion. Philos. Trans. R. Soc. Lond. B Biol. Sci. 352(1358), 1257–1265 (1997)CrossRef

Bouckaert, R.R., Frank, E.: Evaluating the replicability of significance tests for comparing learning algorithms. In: Dai, H., Srikant, R., Zhang, C. (eds.) PAKDD 2004. LNCS (LNAI), vol. 3056, pp. 3–12. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24775-3_3 CrossRef

10.

Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision with the OpenCV Library. O’Reilly Media Inc., Sebastopol (2008)

11.

Bradski, G.R., Davis, J.W.: Motion segmentation and pose recognition with motion history gradients. Mach. Vis. Appl. 13(3), 174–184 (2002)CrossRef

12.

Busso, C., Georgiou, G., P., Narayanan, S.S.: Real-time monitoring of participant’s interaction in a meeting using audio-visual sensors. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE (2007)

13.

Campbell, N., Scherer, S.: Comparing measures of synchrony and alignment in dialogue speech timing with respect to turn-taking activity. In: Proceedings of the INTERSPEECH, pp. 2546–2549 (2010)

14.

Cao, H., Verma, R., Nenkova, A.: Speaker-sensitive emotion recognition via ranking: studies on acted and spontaneous speech. Comput. Speech Lang. 29(1), 186–202 (2015)CrossRef

15.

Cao, Z., Qin, T., Liu, T.Y., Tsai, M.F., Li, H.: Learning to rank: from pairwise approach to listwise approach. In: Proceedings of International Conference on Machine Learning, pp. 129–136. ACM (2007)

16.

Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2(3), 27 (2011)

17.

Dittmann, A.T., Llewellyn, L.G.: Body movement and speech rhythm in social conversation. J. Personal. Soc. Psychol. 11(2), 98 (1969)CrossRef

18.

Fagin, R., Kumar, R., Sivakumar, D.: Comparing top k lists. SIAM J. Discrete Math. 17(1), 134–160 (2003)MathSciNetCrossRefMATH

19.

Gatica-Perez, D., McCowan, I.A., Zhang, D., Bengio, S.: Detecting group interest-level in meetings. Technical report, IDIAP (2004)

20.

Geng, X., Liu, T.Y., Qin, T., Li, H.: Feature selection for ranking. In: Proceedings of the International Conference on Research and Development in Information Retrieval, pp. 407–414. ACM (2007)

21.

Gupta, R., Lee, C.c., Lee, S., Narayanan, S.: Assessment of a child’s engagement using sequence model based features. In: Workshop on Affective Social Speech Signals (2013)

22.

Hall, J.A., Coats, E.J., LeBeau, L.S.: Nonverbal behavior and the vertical dimension of social relations: a meta-analysis. Psychol. Bull. 131(6), 898 (2005)CrossRef

23.

Hang, L.: A short introduction to learning to rank. IEICE Trans. Inf. Syst. 94(10), 1854–1862 (2011)

24.

Heldner, M., Edlund, J.: Pauses, gaps and overlaps in conversations. J. Phon. 38(4), 555–568 (2010)CrossRef

25.

Jayagopi, D.B., Ba, S., Odobez, J.M., Gatica-Perez, D.: Predicting two facets of social verticality in meetings from five-minute time slices and nonverbal cues. In: Proceedings of International Conference on Multimodal Interfaces, pp. 45–52. ACM (2008)

26.

Joachims, T.: Optimizing search engines using clickthrough data. In: Proceedings of the International Conference on Knowledge Discovery and Data Mining, pp. 133–142. ACM (2002)

27.

Kim, J., Truong, K.P., Charisi, V., Zaga, C., Lohse, M., Heylen, D., Evers, V.: Vocal turn-taking patterns in groups of children performing collaborative tasks: an exploratory study. In: Proceedings of the INTERSPEECH, pp. 1645–1649 (2015)

28.

Kim, J., Truong, K.P., Evers, V.: Automatic detection of children’s engagement using non-verbal features and ordinal learning. In: Workshop on Child Computer Interaction (2016)

29.

Leite, I., McCoy, M., Ullman, D., Salomons, N., Scassellati, B.: Comparing models of disengagement in individual and group interactions. In: Proceedings of Annual ACM/IEEE International Conference on Human-Robot Interaction, pp. 99–105. ACM (2015)

30.

Li, L., Lin, H.T.: Ordinal regression by extended binary classification. In: Advances in Neural Information Processing Systems, pp. 865–872 (2006)

31.

Parten, M.B.: Social participation among pre-school children. J. Abnorm. Soc. Psychol. 27(3), 243 (1932)CrossRef

32.

Piaget, J.: The Psychology of the Child. Basic Books, New York (1972)

33.

Pianesi, F., Zancanaro, M., Lepri, B., Cappelletti, A.: A multimodal annotated corpus of consensus decision making meetings. Lang. Resour. Eval. 41(3–4), 409–429 (2007)CrossRef

34.

Robins, B., Dautenhahn, K., Te Boekhorst, R., Billard, A.: Robotic assistants in therapy and education of children with autism: can a small humanoid robot help encourage social interaction skills? Univers. Access Inf. Soc. 4(2), 105–120 (2005)CrossRef

35.

Sidner, C.L., Lee, C., Kidd, C.D., Lesh, N., Rich, C.: Explorations in engagement for humans and robots. Artif. Intell. 166(1), 140–164 (2005)CrossRef

36.

Siegel, S.: Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill, New York (1956)MATH

37.

Stangor, C.: Social Groups in Action and Interaction. Psychology Press, New York (2004)

38.

Vinciarelli, A., Pantic, M., Heylen, D., Pelachaud, C., Poggi, I., D’Errico, F., Schröder, M.: Bridging the gap between social animal and unsocial machine: a survey of social signal processing. IEEE Trans. Affect. Comput. 3(1), 69–87 (2012)CrossRef

39.

Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., Sloetjes, H.: ELAN: a professional framework for multimodality research. In: Proceedings of LREC, pp. 5–8 (2006)

Title: Multimodal Detection of Engagement in Groups of Children Using Rank Learning
Authors: Jaebok Kim
Khiet P. Truong
Vicky Charisi
Cristina Zaga
Vanessa Evers
Mohamed Chetouani
Publisher: Springer International Publishing
Book: Human Behavior Understanding
Print ISBN: 978-3-319-46842-6

Electronic ISBN: 978-3-319-46843-3

Copyright Year: 2016
DOI: https://doi.org/10.1007/978-3-319-46843-3_3

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner