Multimodal Collaboration in Expository Discourse: Verbal and Nonverbal Moves Alignment

The paper explores multimodal collaboration in expository discourse considering verbal and nonverbal moves used by the participants in turn-taking. It reports the results of an experiment which tested speech, gesture and gaze alignment as affected by intentional and spontaneous communication. The results based on the 46-min long pilot subcorpus (out of 640-min long collected database) reveal that collaborative expository discourse is vastly multimodal where nonverbal moves in contact-establishing gesture and face-oriented gaze modulate each type of verbal move. Despite the fact that verbal moves do not manifest specific alignment with nonverbal moves, there are several tendencies in their distribution, for instance, verbal request move and elaboration moves are more frequently accompanied by nonverbal moves than other verbal moves. The study also shows that verbal and nonverbal moves alignment is not modulated by either intentional or spontaneous collaboration since there are no distinctions observed in the nonverbal move use with either common or new topic elaboration move, which proves that more intricate patterns of moves are exploited to provide intentionality and spontaneity in expository discourse.