Abraham Glasser
Vaishnavi Mande
Matt Huenerfauth
ABSTRACT
Proliferation of voice-controlled personal-assistant devices poses accessibility barriers for Deaf and Hard of Hearing (DHH) users. In this mixed interview (N=21) and survey (N=86) study, DHH American Sign Language (ASL) signers reported little experience with such devices yet strong interest if these devices could understand ASL commands. Beyond traditional uses, e.g. weather information, participants were interested in uses specific to the DHH experience, e.g. receiving alerts about sounds, triggering video calls, or requesting ASL interpretation. Participants gave examples of commands they would issue in these categories, where they would use these devices in their home, and how this interaction could be structured. This study contributes empirical knowledge of DHH signers' interest in ASL interaction with these devices, providing motivation and foundation for future research, including what to include in future datasets of videos of users giving commands in ASL, needed to create future sign-recognition technology.
- Ali Abdolrahmani, Ravi Kuber, and Stacy M. Branham. 2018. "Siri Talks at You": An Empirical Investigation of Voice-Activated Personal Assistant (VAPA) Usage by Individuals Who Are Blind. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (Galway, Ireland) (ASSETS '18). Association for Computing Machinery, New York, NY, USA, 249--258. Google Scholar
Digital Library
- BBC. 2018. Sign-language hack lets Amazon Alexa respond to gestures. https://www.bbc.com/news/technology-44891054Google Scholar
- Jeffrey P. Bigham, Raja Kushalnagar, Ting-Hao Kenneth Huang, Juan Pablo Flores, and Saiph Savage. 2017. On How Deaf People Might Use Speech to Control Devices. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (Baltimore, Maryland, USA) (ASSETS '17). Association for Computing Machinery, New York, NY, USA, 383--384. Google ScholarDigital Library
- Danielle Bragg, Oscar Koller, Mary Bellard, Larwan Berke, Patrick Boudreault, Annelies Braffort, Naomi Caselli, Matt Huenerfauth, Hernisa Kacorri, Tessa Verhoef, Christian Vogler, and Meredith Ringel Morris. 2019. Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility (Pittsburgh, PA, USA) (ASSETS '19). Association for Computing Machinery, New York, NY, USA, 16--31. Google ScholarDigital Library
- Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77--101. Google ScholarCross Ref
- Devin Coldewey. 2018. SignAll is slowly but surely building a sign language translation platform. https://techcrunch.com/2018/02/14/signall-is-slowly-but-surely-building-a-sign-language-translation-platform/Google Scholar
- Julie Doyle, Emma Murphy, Janneke Kuiper, Suzanne Smith, Caoimhe Hannigan, An Jacobs, and John Dinsmore. 2019. Managing Multimorbidity: Identifying Design Requirements for a Digital Self-Management Tool to Support Older Adults with Multiple Chronic Conditions. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--14. Google ScholarDigital Library
- Eleni Efthimiou, Stavroula-Evita Fotinea, Thomas Hanke, Julie A. Hochgesang, Jette Kristoffersen, and Johanna Mesch. 2020. Proceedings of the 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives. (2020). https://lrec2020.lrec-conf.org/media/proceedings/Workshops/Books/SIGN2020book.pdfGoogle Scholar
- Leah Findlater, Bonnie Chinh, Dhruv Jain, Jon Froehlich, Raja Kushalnagar, and Angela Carey Lin. 2019. Deaf and Hard-of-Hearing Individuals' Preferences for Wearable and Mobile Sound Awareness Technologies. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--13. Google ScholarDigital Library
- Abraham Glasser. 2019. Automatic Speech Recognition Services: Deaf and Hard-of-Hearing Usability. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA '19). Association for Computing Machinery, New York, NY, USA, 1--6. Google ScholarDigital Library
- Abraham T. Glasser, Kesavan R. Kushalnagar, and Raja S. Kushalnagar. 2017. Feasibility of Using Automatic Speech Recognition with Voices of Deaf and Hard-of-Hearing Individuals. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (Baltimore, Maryland, USA) (ASSETS '17). Association for Computing Machinery, New York, NY, USA, 373--374. Google ScholarDigital Library
- Matthew B. Hoy. 2018. Alexa, Siri, Cortana, and More: An Introduction to Voice Assistants. Medical Reference Services Quarterly 37, 1 (2018), 81--88. Google ScholarCross Ref
- David Isbitski. 2017. How to Build Alexa Skills for Echo Show. https://developer.amazon.com/blogs/alexa/post/12826e9e-e06a-4ab4-a583-8e074709a9f3/how-to-build-alexa-skills-for-echo-showGoogle Scholar
- Dhruv Jain, Angela Lin, Rose Guttman, Marcus Amalachandran, Aileen Zeng, Leah Findlater, and Jon Froehlich. 2019. Exploring Sound Awareness in the Home for People Who Are Deaf or Hard of Hearing. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--13. Google ScholarDigital Library
- Jiepu Jiang, Ahmed Hassan Awadallah, Rosie Jones, Umut Ozertem, Imed Zitouni, Ranjitha Gurunath Kulkarni, and Omar Zia Khan. 2015. Automatic Online Evaluation of Intelligent Assistants. In Proceedings of the 24th International Conference on World Wide Web (Florence, Italy) (WWW '15). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 506--516. Google ScholarDigital Library
- Irene Lopatovska, Katrina Rink, Ian Knight, Kieran Raines, Kevin Cosenza, Harriet Williams, Perachya Sorsche, David Hirsch, Qi Li, Adrianna Martinez, and et al. 2018. Talk to me: Exploring user interactions with the Amazon Alexa. Journal of Librarianship and Information Science 51, 4 (2018), 984--997. Google Scholar
- Tara Matthews, Janette Fong, F. Wai-Ling Ho-Ching, and Jennifer Mankoff. 2006. Evaluating non-speech sound visualizations for the deaf. Behaviour & Information Technology 25, 4 (2006), 333--351. Google Scholar
- Microsoft. 2019. AI for Accessibility Hackathon 2019. https://blogs.partner.microsoft.com/mpn-apac/ai-for-accessibility-hackathon-2019/Google Scholar
- Christi Olson and Kelli Kemery. 2019. 2019 Microsoft Voice report. https://about.ads.microsoft.com/en-us/insights/2019-voice-reportGoogle Scholar
- Alisha Pradhan, Kanika Mehta, and Leah Findlater. 2018. "Accessibility Came by Accident": Use of Voice-Controlled Intelligent Personal Assistants by People with Disabilities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--13. Google ScholarDigital Library
- Jason Rodolitz, Evan Gambill, Brittany Willis, Christian Vogler, and Raja Kushalnagar. 2019. Accessibility of Voice-Activated Agents for People who are Deaf or Hard of Hearing. Journal on Technology and Persons with Disabilities 7 (2019), 144--156. http://hdl.handle.net/10211.3/210397Google Scholar
- Kaleigh Rogers. 2018. Augmented Reality App Can Translate Sign Language Into Spoken English, and Vice Versa. https://www.vice.com/en_us/article/zmgnd9/app-to-translate-sign-languageGoogle Scholar
- Alex Sciuto, Arnita Saini, Jodi Forlizzi, and Jason I. Hong. 2018. "Hey Alexa, What's Up?": A Mixed-Methods Studies of In-Home Conversational Agent Usage. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS '18). Association for Computing Machinery, New York, NY, USA, 857--868. Google ScholarDigital Library
- SignGenius. 2020. Do's & Don'ts - Getting Attention in the Deaf Community. https://www.signgenius.com/info-do's&don'ts.shtmlGoogle Scholar
- Sorenson. 2020. Sorenson VRS. https://www.sorensonvrs.com/Google Scholar
- Jason Ward. 2018. Microsoft's Windows and Cortana absolutely must learn sign language. https://www.windowscentral.com/microsoft-must-bring-sign-language-recognition-windows-and-cortanaGoogle Scholar
Index Terms
- Understanding deaf and hard-of-hearing users' interest in sign-language interaction with personal-assistant devices
Recommendations
Analyzing Deaf and Hard-of-Hearing Users’ Behavior, Usage, and Interaction with a Personal Assistant Device that Understands Sign-Language Input
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing SystemsAs voice-based personal assistant technologies proliferate, e.g., smart speakers in homes, and more generally as voice-control of technology becomes increasingly ubiquitous, new accessibility barriers are emerging for many Deaf and Hard of Hearing (DHH) ...
Deaf Users’ Preferences Among Wake-Up Approaches during Sign-Language Interaction with Personal Assistant Devices
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing SystemsPersonal-assistant devices like Amazon Alexa and Google Assistant are increasingly popular among consumers. Users activate these systems through some type of wake-up approach, e.g. using a wake-word “Alexa” or “Ok, Google.” Voice-based interaction poses ...
Accessibility for Deaf and Hard of Hearing Users: Sign Language Conversational User Interfaces
CUI '20: Proceedings of the 2nd Conference on Conversational User InterfacesWith the proliferation of voice-based conversational user interfaces (CUIs) comes accessibility barriers for Deaf and Hard of Hearing (DHH) users. There has not been significant prior research on sign-language conversational interactions with ...
Comments