Liyuan Li
ABSTRACT
For robots to perform interaction with multiple persons, they have to be able to identify the addressees to interact with. We classify the methods of addressee detection and selection into two categories, namely, passive and active approaches. For passive approaches, the robot is programmed to detect a predefined signal, e.g., a voice command or a specific gesture, from a person who is supposed to be the addressee. In contrast, for active approaches, the robot is able to select a person as an addressee based on subtle cues that are inferred from the human pose, gaze, and facial expression. We present two new approaches for attention-based addressee selection, one is a passive method and the other is an active method. The passive method is designed for the robot to recognize common hand-waving gesture, where a Bayessian ensemble approach is proposed to fuse hand detections from depth segmentation, palm shape, skin color, and body pose. The active method is developed for the robot to perform natural interaction with multiple persons. It employs a novel human attention estimation algorithm based on human detection, tracking, upper body pose recogni-tion, face detection, gaze detection, lip motion analysis, and facial expression recognition. Extensive experiments have been conducted and the effectiveness of the proposed approaches is reported.
- Bohus, D., and Horvitz, E. 2009. Dialog in the open world: Platform and applications. In Proceedings of ICM '09, 31--38. Google Scholar
Digital Library
- Bohus, D., and Horvitz, E. 2009. Models for multiparty engagement in open world dialog. In Proceedings of SIGDIAL'09, 225--234. Google ScholarDigital Library
- Feng, X., Pietikinen, M., and Hadid, A. 2004. Facial expression recognition with local binary patterns and linear programming. In Proceedings of PRIA-7.Google Scholar
- Katzenmaier, M., Stiefelhagen, R., and Schultz, T. 2004. Identifying the addressee in human-human-robot interactions based on head pose and speech. In Proceedings of ICMI'04, 144--151. Google ScholarDigital Library
- Lee, M., Kiesler, S., and Forlizzi, J. 2010. Receptionist or information kiosk: How do people talk with a robot? In Proceedings of CSCW 2010, 31--40. Google ScholarDigital Library
- Li, L., Hoe, K., Yan, S., and Yu, X. 2009. Ml-fusion based multi-model human detection and tracking for robust human-robot interfaces. In Proceedings of WACV'09.Google Scholar
- Li, L., Hoe, K., Yu, X., Dong, L., and Chu, X. 2010. Human upper body pose recognition using adaboost template for natural human robot interaction. In Proceedings of CRV'10. Google ScholarDigital Library
- Li, L., Yu, X., Li, J., Wang, G., Shi, J., Tan, Y., and Li, H. 2012. Vision-based attention estimation and selection for social robot to perform natural interaction in the open world. In Proceedings of HRI'12. Google ScholarDigital Library
- Yamazaki, A., Yamazaki, K., Ohyama, T., Kobayashi, Y., and Kuno, Y. 2012. A techno-sociological solution for designing a museum guide robot: Regarding choosing an appropriate visitor. In Proceedings of HRI'12, 309--316. Google ScholarDigital Library
- Yu, X., Han, W., Li, L., Shi, J., and Wang, G. 2011. An eye detection and localization system for natural human and robot interaction without face detection. In Proceedings of TAROS'11. Google ScholarDigital Library
Index Terms
- Attention-based addressee selection for service and social robots to interact with multiple persons
Recommendations
Context aware addressee estimation for human robot interaction
GazeIn '13: Proceedings of the 6th workshop on Eye gaze in intelligent human machine interaction: gaze in multimodal interactionThe paper investigates the problem of addressee recognition -to whom a speaker's utterance is intended- in a setting involving a humanoid robot interacting with multiple persons. More specifically, as it is well known that addressee can primarily be ...
Towards a Robust Interactive and Learning Social Robot
AAMAS '18: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent SystemsPepper is a humanoid robot, specifically designed for social interaction, that has been deployed in a variety of public environments. A programmable version of Pepper is also available, enabling our focused research on perception and behavior robustness ...
Vision-based attention estimation and selection for social robot to perform natural interaction in the open world
HRI '12: Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot InteractionIn this paper, a novel vision system is proposed to estimate attention of people from rich visual clues for social robot to perform natural interactions with multiple participants in public environments. The vision detection and recognition modules ...
Comments