Roel Vertegaal
Robert Slagter
Anton Nijholt
ABSTRACT
In multi-agent, multi-user environments, users as well as agents should have a means of establishing who is talking to whom. In this paper, we present an experiment aimed at evaluating whether gaze directional cues of users could be used for this purpose. Using an eye tracker, we measured subject gaze at the faces of conversational partners during four-person conversations. Results indicate that when someone is listening or speaking to individuals, there is indeed a high probability that the person looked at is the person listened (p=88%) or spoken to (p=77%). We conclude that gaze is an excellent predictor of conversational attention in multiparty conversations. As such, it may form a reliable source of input for conversational systems that need to establish whom the user is speaking or listening to. We implemented our findings in FRED, a multi-agent conversational system that uses eye input to gauge which agent the user is listening or speaking to.
- 1.Argyle,M.The Psychology of Interpersonal Behaviour .London:Penguin Books,1967.Google Scholar
- 2.Argyle,M.Social Interaction .London:Tavistock Publications,1969.Google Scholar
- 3.Argyle,M.and Cook,M.Ga e and Mutual Gaze . London:Cambridge University Press,1976.Google Scholar
- 4.Argyle,M.and Dean,J.Eye-contact,Distance and Affiliation .Sociometry 28,1965,pp.289-304.Google ScholarCross Ref
- 5.Argyle,M.and Graham,J.The Central Europe Experiment -Looking at Persons and Looking at Things .Journal of Environmental Psychology and Nonverbal Behaviour 1,1977,pp.6-16.Google ScholarCross Ref
- 6.Argyle,M.,Lalljee,M.,and Cook,M.The Effects of Visibility on Interaction in a Dyad .Human Relations 21,1968,pp.3-17.Google ScholarCross Ref
- 7.Brady,P.T.A Statistical Analysis of On-off Patterns in 16 Conversations .The Bell System Technical Journal (Jan.)1968,pp.73-91.Google Scholar
- 8.Cassell,J.,Bickmore,T.,et al.Embodiment in Conversational Interfaces:Rea.In Proceedings of CHI '99.Pittsburgh,PA:ACM,1999. Google ScholarDigital Library
- 9.Jaffe J.and Feldstein S.Rhythms of Dialogue .New York:Academic Press,1970.Google Scholar
- 10.Kendon,A.Some Functions of Gaze Direction in Social Interaction .Acta Psychologica 32,1967,pp.1- 25.Google Scholar
- 11.LC Technologies,Inc.The Eyegaze Communication System .Fairfax,VA:http://www.eyegaze.com ,1997.Google Scholar
- 12.Mobbs,N.A.Eye-contact in Relation to Social Introversion-Extraversion .British Journal of Social Clinical Psychology 7(305-306),1968.Google Scholar
- 13.Nijholt,A.and Hulstijn,J.Multimodal Interactions with Agents in Virtual Worlds.In Future Directions for Intelligent Information Systems and Information Science N.Kasabov (ed.),Physica-Verlag:Studies in Fuzziness and Soft Computing 45,2000,pp.148-173.Google ScholarCross Ref
- 14.Sellen A.J.Speech Patterns in Video-Mediated Conversations.In Proceedings of CHI '92.Monterey, CA:ACM,1992,pp.49-59. Google ScholarDigital Library
- 15.Velichkovsky,B.and Hansen,J.P.New Technological Windows to Mind:There is More in Eyes and Brains for Human Computer Interaction.In Proceedings of CHI '96.Vancouver,Canada:ACM,1996,pp.496- 503. Google ScholarDigital Library
- 16.Vertegaal,R.The GAZE Groupware System: Mediating Joint Attention in Multiparty Communication and Collaboration.In Proceedings of CHI '99.Pittsburg,PA:ACM,1999,pp.294-301. Google ScholarDigital Library
- 17.Vertegaal,R.Look Who's Talking to Whom .PhD Thesis.Enschede,The Netherlands:Cognitive Ergonomics Department,Twente University,1998.Google Scholar
- 18.Vertegaal,R.Slagter R.Van der Veer,G.C.and Nijholt,A.Why Conversational Agents Should Catch the Eye.In Extended Abstracts of CHI '2000.The Hague,The Netherlands:ACM,2000,pp.257-258. Google ScholarDigital Library
- 19.Vertegaal,R.Van der Veer,G.C.and Vons,H. Effects of Gaze on Multiparty Mediated Communication.In Proceedings of Graphics Interface 2000.Montreal,Canada:Morgan Kaufmann Publishers,2000,pp.95-102.Google Scholar
- 20.Waters,K.and Frisbee,J.A Coordinated Muscle Model for Speech Animation.In Proceedings of Graphics Interface '95.Canada,1995,pp.163-170.Google Scholar
- 21.Weisbrod,R.M.Looking Behavior in a Discussion Group.Unpublished paper,Department of Psychology, Cornell University,1965.Google Scholar
Index Terms
- Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes
Recommendations
Looking for Laughs: Gaze Interaction with Laughter Pragmatics and Coordination
ICMI '21: Proceedings of the 2021 International Conference on Multimodal InteractionLaughter and gaze have an important role in managing and coordi-nating social interactions. In the current work, using a multimodal corpus of dyadic taste-testing interactions, we explore whether laughs performing different pragmatic functions are ...
Facilitating multiparty dialog with gaze, gesture, and speech
ICMI-MLMI '10: International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal InteractionWe study how synchronized gaze, gesture and speech rendered by an embodied conversational agent can influence the flow of conversations in multiparty settings. We begin by reviewing a computational framework for turn-taking that provides the foundation ...
Conversational gaze mechanisms for humanlike robots
During conversations, speakers employ a number of verbal and nonverbal mechanisms to establish who participates in the conversation, when, and in what capacity. Gaze cues and mechanisms are particularly instrumental in establishing the participant roles ...
Comments