Matthew Seita
Matt Huenerfauth
ABSTRACT
Various technologies mediate synchronous audio-visual one-on-one communication (SAVOC) between Deaf and Hard-of-Hearing (DHH) and hearing colleagues, including automatic-captioning smartphone apps for in-person settings, or text-chat features of videoconferencing software in remote settings. Speech and non-verbal behaviors of hearing speakers, e.g. speaking too quietly, can make SAVOC difficult for DHH users, but prior work had not examined technology-mediated contexts. In an in-person study (N=20) with an automatic captioning smartphone app, variations in a hearing actor's enunciation and intonation dynamics affected DHH users' satisfaction. In a remote study (N=23) using a videoconferencing platform with text chat, variations in speech rate, voice intensity, enunciation, intonation dynamics, and eye contact led to such differences. This work contributes empirical evidence that specific behaviors of hearing speakers affect the accessibility of technology-mediated SAVOC for DHH users, providing motivation for future work on detecting or encouraging useful communication behaviors among hearing individuals.
- Altieri NA, Pisoni DB, Townsend JT. Some normative data on lip-reading skills (L). J Acoust Soc Am. 2011;130(1):1--4. Google Scholar
Cross Ref
- S. Antia, D. Sabers, M. Stinson. 2007. Reliability and validity of the classroom communication questionnaire. Journal of Deaf Studies and Deaf Education. 12, 2, (March 2007), 158--171. Google ScholarCross Ref
- Jon P. Barker, Ricard Marxer, Emmanuel Vincent, Shinji Watanabe. 2017. The CHiME challenges: Robust speech recognition in everyday environments. In: Watanabe S., Delcroix M., Metze F., Hershey J. (eds.), New Era for Robust Speech Recognition. Springer, Cham, 327--344. Google ScholarCross Ref
- Larwan Berke, Christopher Caulfield, and Matt Huenerfauth. 2017. Deaf and Hard-of-Hearing Perspectives on Imperfect Automatic Speech Recognition for Captioning One-on-One Meetings. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17). ACM, NY, NY, USA, 155--164. Google ScholarDigital Library
- Debra L Blackwell, Jacqueline W Lucas, and Tainya C Clarke. 2014. Summary Health Statistics for U.S. Adults: National Health Interview Survey, 2012. National Center for Health Statistics. https://www.ncbi.nlm.nih.gov/pubmed/24819891.Google Scholar
- Denis Burnham, Sebastian Joeffry, Lauren Rice. 2010. Computer- and Human-Directed Speech Before and After Correction. In Proceedings of the 9th Speech Science and Technology Conference 2010, Melbourne, Australia. Australian Speech Science and Technology Association.Google Scholar
- Herbert H. Clark, Meredyth A. Krych, Speaking while monitoring addressees for understanding, Journal of Memory and Language, Volume 50, Issue 1,2004, Pages 62--81, ISSN 0749-596X Google ScholarCross Ref
- Lisa B. Elliot, Michael Stinson, Syed Ahmed, and Donna Easton. 2017. User Experiences When Testing a Messaging App for Communication Between Individuals Who Are Hearing and Deaf or Hard of Hearing. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17). ACM, NY, NY, USA, 405--406. Google ScholarDigital Library
- Lisa B. Elliot, Michael Stinson, James Mallory, Donna Easton, and Matt Huenerfauth. 2016. Deaf and Hard of Hearing Individuals' Perceptions of Communication with Hearing Colleagues in Small Groups. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16). ACM, NY, NY, USA, 271--272. Google ScholarDigital Library
- C. Garberoglio, S. Cawthon, M. Bond. 2016. Deaf People and Employment in the United States. Retrieved August 4, 2017 from https://www.nationaldeafcenter.org/sites/default/files/resources/Deaf%20Employment%20Report_final.pdf.Google Scholar
- Carrie Lou Garberoglio, Stephanie Cawthon, and Adam Sales. 2017. Deaf People and Educational Attainment in the United States: 2017. Technical Report. Washington, DC.Google Scholar
- Google Live Transcribe. 2019. Introducing Live Transcribe. Retrieved May 7, 2020 from https://www.android.com/accessibility/live-transcribe/Google Scholar
- Google Meet. Retrieved May 7, 2020. https://meet.google.com/Google Scholar
- Benjamin M. Gorman and David R. Flatla. 2018. MirrorMirror: A Mobile Application to Improve Speechreading Acquisition. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, NY, USA, Paper 26, 1--12. Google ScholarDigital Library
- Gournaris, M. J., & Leigh, I. W. (2019). Comparison of Face-to-Face and Video-Mediated Communication with Deaf Individuals: Implications for Telepsychotherapy. JADARA, 37(2). Retrieved from https://repository.wcsu.edu/jadara/vol37/iss2/5Google Scholar
- Jennifer Harris & Claire Bamford (2001) The Uphill Struggle: Services for Deaf and hard of hearing people - issues of equality, participation and access, Disability & Society, 16:7, 969--979 Google ScholarCross Ref
- Hearing Loss Association of America. 2017. Basic Facts About Hearing Loss. Retrieved December 17, 2017 from http://www.hearingloss.org/content/basic-facts-about-hearing-loss.Google Scholar
- Hnath-Chisolm T, Boothroyd A. Speechreading enhancement by voice fundamental frequency: the effects of Fo contour distortions. J Speech Hear Res. 1992;35(5):1160--1168. Google ScholarCross Ref
- IJsseldijk, F. J. (1992). Speechreading performance under different conditions of video image, repetition, and speech rate. Journal of Speech & Hearing Research, 35(2), 466--471. Google ScholarCross Ref
- Dhruv Jain, Audrey Desjardins, Leah Findlater, and Jon E. Froehlich. 2019. Autoethnography of a Hard of Hearing Traveler. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '19). Association for Computing Machinery, New York, NY, USA, 236--248. Google ScholarDigital Library
- Saba Kawas, George Karalis, Tzu Wen, and Richard E. Ladner. 2016. Improving Real-Time Captioning Experiences for Deaf and Hard of Hearing Students. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16). ACM, NY, NY, USA, 15--23. Google ScholarDigital Library
- Konstantinidis, B. and Fels, D. (2006). Hand waving apparatus for effective turn-taking (HWAET) using video conferencing for deaf people, in the Proceedings of the 3rd Cambridge Workshop on Universal Access and Assistive Technology (CWUAAT 2006), 10--12 April 2006, Cambridge, U.K., pp. 77--82. Cambridge, U.K.: Cambridge University Press.Google Scholar
- Raja Kushalnagar, Walter Lasecki, Jeffrey Bigham. 2014. Accessibility evaluation of classroom captions. ACM Trans. Access. Comput. 5, 3, Article 7 (Jan. 2014), 24 pp. Google ScholarDigital Library
- William Lewis. 2015. Skype Translator: Breaking Down Language and Hearing Barriers. Proceedings of the 37th Conference Translating and the Computer, pages 58--65, London, UK, November 26--27, 2015Google Scholar
- Lezzoni LI, O'Day BL, Killeen M, et al. Communicating about Health Care: Observations from Persons Who Are Deaf or Hard of Hearing. Ann Intern Med. 2004;140:356--362. Google ScholarCross Ref
- J. Mallory, M. Stinson, L. Elliot, D. Easton. 2017. Personal perspectives on using automatic speech recognition to facilitate communication between deaf students and hearing customers. To appear in Proceedings of the 19th Annual International Conference on Computers and Accessibility (ASSETS '17), ACM, New York, NY.Google Scholar
- A. O'Donnell. 2006. The role of peers and group learning. In P. Alexander & P. Winne (Eds.), Handbook of educational psychology. 781--802. Mahwah, NJ: Lawrence Erlbaum Assoc.Google Scholar
- Sharon Oviatt, Gina-Anne Levow, Elliott Moreton, and Margaret MacEachern. 1998. Modeling Global and Focal Hyperarticulation during Human-Computer Error Resolution. J. Acoust. Soc. Amer. 104, 3080--3098. Google ScholarCross Ref
- Carol Padden, Tom Humphries, and Carol Padden. 2009. Inside deaf culture. Harvard University Press.Google Scholar
- Picheny MA, Durlach NI, Braida LD. Speaking clearly for the hard of hearing I: Intelligibility differences between clear and conversational speech. J Speech Hear Res. 1985 Mar; 28(1):96--103Google Scholar
- Picheny MA, Durlach NI, Braida LD. Speaking clearly for the hard of hearing. II: Acoustic characteristics of clear and conversational speech. J Speech Hear Res. 1986 Dec; 29(4):434--4.Google Scholar
- Picheny MA, Durlach NI, Braida LD. Speaking clearly for the hard of hearing. III: An attempt to determine the contribution of speaking rate to differences in intelligibility between clear and conversational speech. J Speech Hear Res. 1989 Sep; 32(3):600--3Google Scholar
- Anne Marie Piper and James D. Hollan. 2008. Supporting medical conversations between deaf and hearing individuals with tabletop displays. In Proceedings of the 2008 ACM conference on Computer supported cooperative work (CSCW '08). Association for Computing Machinery, New York, NY, USA, 147--156. Google ScholarDigital Library
- Matthew Seita, Khaled Albusays, Sushant Kafle, Michael Stinson, and Matt Huenerfauth. 2018. Behavioral Changes in Speakers who are Automatically Captioned in Meetings with Deaf or Hard-of-Hearing Peers. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '18). Association for Computing Machinery, New York, NY, USA, 68--80. Google ScholarDigital Library
- Matthew Seita and Matt Huenerfauth. 2020. Deaf Individuals' Views on Speaking Behaviors of Hearing Peers when Using an Automatic Captioning App. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI EA '20). Association for Computing Machinery, New York, NY, USA, 1--8. Google ScholarDigital Library
- Rein Ove Sikveland. 2006. How do We Speak to Foreigners? --- Phonetic Analyses of Speech Communication between L1 and L2 Speakers of Norwegian. Working Papers 52, 109--112. Centre for Language and Literature, Lund University, Sweden.Google Scholar
- Shen J, Souza PE. On Dynamic Pitch Benefit for Speech Recognition in Speech Masker. Front Psychol. 2018;9:1967. Published 2018 Oct 22. Google ScholarCross Ref
- Amanda J. Stent, Marie K. Huffman, and Susan E. Brennan. 2008. Adapting Speaking After Evidence of Misrecognition: Local and Global Hyperarticulation. Speech Commun. 50, 3 (March 2008), 163--178. Google ScholarDigital Library
- Allan Sutherland & Tessa Padden (1999) Videoconferencing for Deaf people: a case study of on-line education for Deaf people, Deafness & Education International, 1:2, 114--120 Google ScholarCross Ref
- Tye-Murray, N., & Clark, W. (1998). Foundations of aural rehabilitation: Children, adults, and their family members. San Diego: Singular Pub. Group.Google Scholar
- M. Uther, M.A. Knoll, D. Burnham, Do you speak E-NG-L-I-SH? A comparison of foreigner- and infant-directed speech, Speech Communication, Volume 49, Issue 1, 2007, Pages 2--7, ISSN 0167-6393 Google ScholarDigital Library
- Vogler, C., Tucker, P., & Williams, N. (2013). Mixed local and remote participation in teleconferences from a deaf and hard of hearing perspective. ASSETS '13. Google ScholarCross Ref
- Gerard G Walter. 2010. Deaf and Hard-of-Hearing Students in Transition: Demographics with an Emphasis on STEM Education. National Technical Institute for the Deaf, Rochester, NY.Google Scholar
- Emily Q. Wang and Anne Marie Piper. 2018. Accessibility in Action: Co-Located Collaboration among Deaf and Hearing Professionals. Proc. ACM Hum.-Comput Interact. 2, CSCW, Article 180 (November 2018), 25 pages. Google ScholarDigital Library
- Puisan Wong and Kelly Wing Sum Ng. Testing the Hyperarticulation and Prosodic Hypotheses of Child-Directed Speech: Insights From the Perceptual and Acoustic Characteristics of Child-Directed Cantonese Tones. Journal of Speech Language and Hearing Research. Volume 61, Issue 8. August 2018. Pages: 1907--1925. Google ScholarCross Ref
- Zoom. Retrieved May 7, 2020. https://zoom.us/Google Scholar
Index Terms
- Deaf and hard-of-hearing users' preferences for hearing speakers' behavior during technology-mediated in-person and remote conversations
Recommendations
Behavioral Changes in Speakers who are Automatically Captioned in Meetings with Deaf or Hard-of-Hearing Peers
ASSETS '18: Proceedings of the 20th International ACM SIGACCESS Conference on Computers and AccessibilityDeaf and hard of hearing (DHH) individuals face barriers to communication in small-group meetings with hearing peers; we examine generation of captions on mobile devices by automatic speech recognition (ASR). While ASR output displays errors, we study ...
Deaf, Hard of Hearing, and Hearing Perspectives on Using Automatic Speech Recognition in Conversation
ASSETS '17: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and AccessibilityThis experience report describes the accessibility challenges in using the top seven most popular Automatic Speech Recognition (ASR) applications on personal devices for commands and group conversation, by five deaf, hard of hearing and hearing ...
How people who are deaf, Deaf, and hard of hearing use technology in creative sound activities
ASSETS '22: Proceedings of the 24th International ACM SIGACCESS Conference on Computers and AccessibilityCreative sound activities, such as music playing and audio engineering, are said to have been democratized with the development of technology. Yet, the use of technology in creative sound activities by people who are deaf, Deaf, and hard of hearing (DHH)...
Comments