Justine Cassell
Timothy W. Bickmore
ABSTRACT
This paper addresses the issue of designing embodied conversational agents that exhibit appropriate posture shifts during dialogues with human users. Previous research has noted the importance of hand gestures, eye gaze and head nods in conversations between embodied agents and humans. We present an analysis of human monologues and dialogues that suggests that postural shifts can be predicted as a function of discourse state in monologues, and discourse and conversation state in dialogues. On the basis of these findings, we have implemented an embodied conversational agent that uses Collagen in such a way as to generate postural shifts.
- Andre, E., Rist, T., & Muller, J., Employing Al methods to control the behavior of animated interface agents, Applied Artificial Intelligence, vol. 13, pp. 415--448, 1999.Google Scholar
Cross Ref
- Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L., Chang, K., Vilhjalmsson, H., & Yan, H., Embodiment in Conversational Interfaces: Rea, Proc. of CHI 99, Pittsburgh, PA, ACM, 1999. Google ScholarDigital Library
- Cassell, J., Stone, M., & Yan, H., Coordination and context-dependence in the generation of embodied conversation, Proc. INLG 2000, Mitzpe Ramon, Israel, 2000. Google ScholarDigital Library
- Cassell, J. and Thorisson, K. R., The Power of a Nod and a Glance: Envelope vs. Emotional Feedback in Animated Conversational Agents, Applied Art. Intell., vol. 13, pp. 519--538, 1999.Google ScholarCross Ref
- Cassell, J., Torres, O., & Prevost, S., Turn Taking vs. Discourse Structure: How Best to Model Multimodal Conversation., in Machine Conversations, Y. Wilks, Ed. The Hague: Kluwer, 1999, pp. 143--154.Google Scholar
- Cassell, J., Vilhjálmasson, H., & Bickmore, T., BEAT: The Behavior Expression Animation Toolkit, Proc. of SIGGRAPH, ACM Press, 2001. Google ScholarDigital Library
- Chovil, N., Discourse-Oriented Facial Displays in Conversation, Research on Language and Social Interaction, vol. 25, pp. 163--194, 1992.Google ScholarCross Ref
- Chu-Carroll, J. & Brown, M., Initiative in Collaborative Interactions - Its Cues and Effects, Proc. of AAAI Spring 1997 Symp. on Computational Models of Mixed Initiative, 1997.Google Scholar
- Condon, W. S. & Osgton, W. D., Speech and body motion synchrony of the speaker-hearer, in The perception of language, D. Horton & J. Jenkins, Eds. NY: Academic Press, 1971, pp. 150--184.Google Scholar
- Duncan, S., On the structure of speaker-auditor interaction during speaking turns, Language in Society, vol. 3, pp. 161--180, 1974.Google ScholarCross Ref
- Green, N., Carenini, G., Kerpedjiev, S., & Roth, S, A Media-Independent Content Language for Integrated Text and Graphics Generation, Proc. of Workshop on Content Visualization and Intermedia Representations at COLING and ACL '98, 1998.Google Scholar
- Grosz, B. & Sidner, C., Attention, Intentions, and the Structure of Discourse, Computational Linguistics, vol. 12, pp. 175--204, 1986. Google ScholarDigital Library
- Kendon, A., Some Relationships between Body Motion and Speech, in Studies in Dyadic Communication, A. W. Siegman and B. Pope, Eds. Elmsford, NY: Pergamon Press, 1972, pp. 177--210.Google ScholarCross Ref
- Lesh, N., Rich, C., & Sidner, C., Using Plan Recognition in Human-Computer Collaboration, Proc. of the Conference on User Modelling, Banff, Canada, NY: Springer Wien, 1999. Google ScholarDigital Library
- Lester, J., Towns, S., Callaway, C., Voerman, J., & FitzGerald, P., Deictic and Emotive Communication in Animated Pedagogical Agents, in Embodied Conversational Agents, J. Cassell, J. Sullivan, et. al, Eds. Cambridge: MIT Press, 2000. Google ScholarDigital Library
- Lochbaum, K., A Collaborative Planning Model of Intentional Structure, Computational Linguistics, vol. 24, pp. 525--572, 1998. Google ScholarDigital Library
- McNeill, D., Hand and Mind: What Gestures Reveal about Thought. Chicago, IL/London, UK: The University of Chicago Press, 1992.Google Scholar
- Rich, C. & Sidner, C. L., COLLAGEN: A Collaboration Manager for Software Interface Agents, User Modeling and User-Adapted Interaction, vol. 8, pp. 315--350, 1998. Google ScholarDigital Library
- Rickel, J. & Johnson, W. L., Task-Oriented Collaboration with Embodied Agents in Virtual Worlds, in Embodied Conversational Agents, J. Cassell, Ed. Cambridge, MA: MIT Press, 2000. Google ScholarDigital Library
- Sidner, C., An Artificial Discourse Language for Collaborative Negotiation, Proc. of 12th Intnl. Conf. on Artificial Intelligence (AAAI), Seattle, WA, MIT Press, 1994. Google ScholarDigital Library
- Takeuchi, A. & Nagao, K., Communicative facial displays as a new conversational modality, Proc. of InterCHI '93, Amsterdam, NL, ACM, 1993. Google ScholarDigital Library
- Thompson, L. and Massaro, D., Evaluation and Integration of Speech and Pointing Gestures during Referential Understanding, Journal of Experimental Child Psychology, vol. 42, pp. 144--168, 1986.Google ScholarCross Ref
Recommendations
Multimodal Collaboration in Expository Discourse: Verbal and Nonverbal Moves Alignment
Speech and ComputerAbstractThe paper explores multimodal collaboration in expository discourse considering verbal and nonverbal moves used by the participants in turn-taking. It reports the results of an experiment which tested speech, gesture and gaze alignment as affected ...
Graph representations of discourse structure
The aim of this paper is to introduce a new method to represent discourse structures. This approach is based on three discourse theories in order to highlight three discourse features: cohesion, coherence and intentionality. The graph representations of ...
Attention, intentions, and the structure of discourse
In this paper we explore a new theory of discourse structure that stresses the role of purpose and processing in discourse. In this theory, discourse structure is composed of three separate but interrelated components: the structure of the sequence of ...
Comments