Understanding visually impaired people’s experiences of social signal perception in face-to-face communication

Social signals (e.g., facial expression, gestures) are important in social interactions. Most of them are visual cues, which are hardly accessible for visually impaired people, causing difficulties in their daily living. In human–computer interaction (HCI), assistive systems for social interactions are getting increasing attention due to related technological advancements. Yet, there is still lack of a comprehensive and vivid understanding of visually impaired people’s social signal perception to broadly identify their needs in face-to-face communication. To fill this gap, we conducted in-depth interviews to study the lived experiences of 20 visually impaired participants. We analyzed a rich set of qualitative empirical data based on a comprehensive taxonomy of social signals, using a standard qualitative content analysis method. Our results revealed a set of vivid examples and an overview of visually impaired people’s lived experiences regarding social signals, including both their capabilities and limitations. As reported, the participants perceived social signals through their compensatory modalities such as hearing, touch, smell, or obstacle sense. However, their perception of social signals is generally with low resolution and limited by certain environmental factors (e.g., crowdedness, or noise level of the surrounding). Interestingly, sight was still importantly relied on by low-vision participants in social signal perception (e.g., rough postures and gestures). Besides, the participants experienced difficulties in sensing others’ subtle emotional states which are often revealed by nuanced behaviors (e.g., a smile). Based on rich empirical findings, we propose a set of design implications to inform future-related HCI works aimed at supporting visually impaired users’ social signal perception.