Matthew Seita
Matt Huenerfauth
ABSTRACT
As automatic speech recognition (ASR) becomes more accurate, many deaf and hard-of-hearing (DHH) individuals are interested in ASR-based mobile applications to facilitate in-person communication with hearing peers. We investigate DHH users' preferences regarding the behaviors of the hearing person in this context. Using an ASR-based captioning app, eight Deaf/deaf participants held short conversations, with a hearing actor who exhibited certain behaviors, e.g. speaking quietly/loudly or slowly/quickly. Participants indicated some of the hearing individual's behaviors were more influential as to their subjective impression of the communication efficacy. We also found that these behaviors differed in how noticeable they were to the Deaf participants. This study provides guidance, from a Deaf perspective, about the types of behaviors hearing users should ideally exhibit in this context, motivating a focus on such behaviors in future design or evaluation of ASR-based communication apps.
- Jon P. Barker, Ricard Marxer, Emmanuel Vincent, Shinji Watanabe. 2017. The CHiME challenges: Robust speech recognition in everyday environments. In: Watanabe S., Delcroix M., Metze F., Hershey J. (eds.), New Era for Robust Speech Recognition. Springer, Cham, 327--344. https://doi.org/10.1007/978--3--319--64680-0_14Google Scholar
Cross Ref
- Larwan Berke, Christopher Caulfield, and Matt Huenerfauth. 2017. Deaf and Hard-of-Hearing Perspectives on Imperfect Automatic Speech Recognition for Captioning One-on-One Meetings. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17). ACM, NY, NY, USA, 155--164. https://doi.org/10.1145/3132525.3132541Google ScholarDigital Library
- Lisa B. Elliot, Michael Stinson, Syed Ahmed, and Donna Easton. 2017. User Experiences When Testing a Messaging App for Communication Between Individuals Who Are Hearing and Deaf or Hard of Hearing. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17). ACM, NY, NY, USA, 405--406. https://doi.org/10.1145/3132525.3134798Google ScholarDigital Library
- Google Live Transcribe. 2019. Introducing Live Transcribe. Retrieved January 6, 2019 from https://www.android.com/accessibility/livetranscribe/Google Scholar
- Hearing Loss Association of America. 2017. Basic Facts About Hearing Loss. Retrieved December 17, 2017 from http://www.hearingloss.org/content/basic-facts-about-hearing-lossGoogle Scholar
- Saba Kawas, George Karalis, Tzu Wen, and Richard E. Ladner. 2016. Improving Real-Time Captioning Experiences for Deaf and Hard of Hearing Students. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16). ACM, NY, NY, USA, 15--23. https://doi.org/10.1145/2982142.2982164Google ScholarDigital Library
- Ronald R. Kelly. 2015. The Employment and Career Growth of Deaf and Hard of Hearing Individuals. Raising and Educating Deaf Children: Foundations for Policy, Practice, and Outcomes. Retrieved from http://www.raisingandeducatingdeafchildren.org/2 015 /01/12/the-employment-and-career-growthof-deafand-hard-of-hearing-individuals/Google Scholar
- James R. Mallory, Michael Stinson, Lisa Elliot, and Donna Easton. 2017. Personal Perspectives on Using Automatic Speech Recognition to Facilitate Communication Between Deaf Students and Hearing Customers. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17). ACM, NY, NY, USA, 419--421. https://doi.org/10.1145/3132525.3134779Google ScholarDigital Library
- Sharon Oviatt, Gina-Anne Levow, Elliott Moreton, and Margaret MacEachern. 1998. Modeling Global and Focal Hyperarticulation during Human-- Computer Error Resolution. J. Acoust. Soc. Amer. 104, 3080-- 3098. https://doi.org/10.1121/1.423888Google ScholarCross Ref
- Matthew Seita, Khaled Albusays, Sushant Kafle, Michael Stinson, and Matt Huenerfauth. 2018. Behavioral Changes in Speakers who are Automatically Captioned in Meetings with Deaf or Hard-of-Hearing Peers. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '18). Association for Computing Machinery, New York, NY, USA, 68--80. DOI:https://doiorg.ezproxy.rit.edu/10.1145/3234695.3236355Google ScholarDigital Library
- Rein Ove Sikveland. 2006. How do We Speak to Foreigners? - Phonetic Analyses of Speech Communication between L1 and L2 Speakers of Norwegian. Working Papers 52, 109--112. Centre for Language and Literature, Lund University, Sweden.Google Scholar
- Amanda J. Stent, Marie K. Huffman, and Susan E. Brennan. 2008. Adapting Speaking After Evidence of Misrecognition: Local and Global Hyperarticulation. Speech Commun. 50, 3 (March 2008), 163--178. https://doi.org/10.1016/j.specom.2007.07.005Google ScholarDigital Library
Index Terms
- Deaf Individuals' Views on Speaking Behaviors of Hearing Peers when Using an Automatic Captioning App
Recommendations
Deaf and hard-of-hearing users' preferences for hearing speakers' behavior during technology-mediated in-person and remote conversations
W4A '21: Proceedings of the 18th International Web for All ConferenceVarious technologies mediate synchronous audio-visual one-on-one communication (SAVOC) between Deaf and Hard-of-Hearing (DHH) and hearing colleagues, including automatic-captioning smartphone apps for in-person settings, or text-chat features of ...
Behavioral Changes in Speakers who are Automatically Captioned in Meetings with Deaf or Hard-of-Hearing Peers
ASSETS '18: Proceedings of the 20th International ACM SIGACCESS Conference on Computers and AccessibilityDeaf and hard of hearing (DHH) individuals face barriers to communication in small-group meetings with hearing peers; we examine generation of captions on mobile devices by automatic speech recognition (ASR). While ASR output displays errors, we study ...
Designing Automatic Speech Recognition Technologies to Improve Accessibility for Deaf and Hard-of-Hearing People in Small Group Meetings
CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing SystemsDeaf and hard of hearing (DHH) individuals face several barriers to communication in the workplace, particularly in small-group meetings with their hearing peers. The impromptu nature of these meetings makes scheduling sign-language interpreting or ...
Comments