Extending Empirical Analysis of Usability and Playability to Multimodal Computer Games

The published research examining usability and playability of games is largely theoretical. A prior empirical study of a game with an embodied conversational agent found that most frustration episodes could be understood in terms of both usability and playability, but this study was based on a game in which the interaction by both player and agent were limited to verbal communication. To explore whether these results would hold for a game in which the player and agent communicated with both speech and gesture, we conducted an empirical formative user-experience evaluation of a multimodal game. Our findings strongly confirmed that frustration episodes can be understood as issues of both usability and playability. However, the relative frequencies of the categories of usability and playability issues differed between the speech-only and the speech-and-gesture games. Much of this difference likely arose because higher levels of engagement and rapport between player and agent in the speech-and-gesture game led to the players having greater, and in many cases unfulfilled, expectations for the capabilities of the agent.