Analysis of conversational listening skills toward agent-based social skills training

Listening skills are critical for human communication. Social skills training (SST), performed by human trainers, is a well-established method for obtaining appropriate skills in social interaction. Previous work automated the process of social skills training by developing a dialogue system that teaches speaking skills through interaction with a computer agent. Even though previous work that simulated social skills training considered speaking skills, the SST framework incorporates other skills, such as listening, asking questions, and expressing discomfort. In this paper, we extend our automated social skills training by considering user listening skills during conversations with computer agents. We prepared two scenarios: Listening 1 and Listening 2, which respectively assume small talk and job training. A female agent spoke to the participants about a recent story and how to make a telephone call, and the participants listened. We recorded the data of 27 Japanese graduate students who interacted with the agent. Two expert external raters assessed the participants’ listening skills. We manually extracted features that might be related to the eye fixation and behavioral cues of the participants and confirmed that a simple linear regression with selected features correctly predicted listening skills with a correlation coefficient above 0.50 in both scenarios. The number of noddings and backchannels within the utterances contributes to the predictions because we found that just using these two features predicted listening skills with a correlation coefficient above 0.43. Since these two features are easier to understand for users, we plan to integrate them into the framework of automated social skills training.