Saba Kawas
Richard E. Ladner
ABSTRACT
We take a qualitative approach to understanding deaf and hard of hearing (DHH) students' experiences with real-time captioning as an access technology in mainstream university classrooms. We consider both existing human-based captioning as well as new machine-based solutions that use automatic speech recognition (ASR). We employed a variety of qualitative research methods to gather data about students' captioning experiences including in-class observations, interviews, diary studies, and usability evaluations. We also conducted a co-design workshop with 8 stakeholders after our initial research findings. Our results show that accuracy and reliability of the technology are still the most important issues across captioning solutions. However, we additionally found that current captioning solutions tend to limit students' autonomy in the classroom and present a variety of user experience shortcomings, such as complex setups, poor feedback and limited control over caption presentation. Based on these findings, we propose design requirements and recommend features for real-time captioning in mainstream classrooms.
- Araujo, G. F., & Macedo, H. T. (2014, April). Context formalization and its use on dynamic adaptation of language model in ASR systems. In Proceedings of the 7th Euro American Conference on Telematics and Information Systems(p.4).ACM.DOI:http://dx.doi.org/10.1145/2590651.2590655 Google Scholar
Digital Library
- Bauman, H. D., & Murray, J. (2009). Reframing: From hearing loss to deaf gain. Deaf Studies Digital Journal, 1(1), 1--10.Google Scholar
- Benzeghiba, M., De Mori, R., Deroo, O., Dupont, S., Erbes, T., Jouvet, D., ... & Rose, R. (2007). Automatic speech recognition and speech variability: A review. Speech Communication, 49(10), 763--786. Google ScholarDigital Library
- Bumbalek, Z., Zelenka, J., & Kencl, L. (2010). E-Scribe: ubiquitous real-time speech transcription for the hearing-impaired. In Computers Helping People with Special Needs (pp. 160--168). Springer Berlin Heidelberg. Google ScholarDigital Library
- Cavender, A. C., Bigham, J. P., & Ladner, R. E. (2009, October). ClassInFocus: enabling improved visual attention strategies for deaf and hard of hearing students. In Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility (pp. 67--74). ACM. DOI: http://dx.doi.org/10.1145/1639642.1639656 Google ScholarDigital Library
- Elliot, L. B., Stinson, M. S., McKee, B. G., Everhart, V. S., & Francis, P. J. (2001). College students' perceptions of the C-Print speech-to-text transcription system. Journal of deaf studies and deaf education, 6(4), 285--298.Google ScholarCross Ref
- Gaur, Y. The Effects of Automatic Speech Recognition Quality on Human Transcription Latency. 2015. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility (pp. 367--368). ACM. DOI: http://dx.doi.org/10.1145/2700648.2811331 Google ScholarDigital Library
- Iglesias, A., Ruiz-Mezcua, B., López, J. F., & Figueroa, D. C. (2013). New communication technologies for inclusive education in and outside the classroom.Google Scholar
- Keith Bain, Sara H. Basson, and Mike Wald. 2002. Speech recognition in university classrooms. In Proceedings of the 5th International ACM Conference on Assistive Technologies (Assets'02). ACM Press, New York, 192. DOI:http://dx.doi.org/10.1145/638249.638284. Google ScholarDigital Library
- Kheir, R., & Way, T. (2007, June). Inclusion of deaf students in computer science classes using real-time speech transcription. In ACM SIGCSE Bulletin (Vol. 39, No. 3, pp. 261--265). ACM.DOI:http://dx.doi.org/10.1145/1269900.1268860 Google ScholarDigital Library
- Kushalnagar, R. S., Behm, G. W., Kelstone, A. W., & Ali, S. (2015, October). Tracked Speech-To-Text Display: Enhancing Accessibility and Readability of Real-Time Speech-To-Text. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility (pp. 223--230). ACM. DOI: http://dx.doi.org/10.1145/2700648.2809843 Google ScholarDigital Library
- Kushalnagar, R. S., Lasecki, W. S., & Bigham, J. P. (2013, May). Captions versus transcripts for online video content. In Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility (p. 32). ACM DOI: http://dx.doi.org/ 10.1145/2461121.2461142 Google ScholarDigital Library
- Kushalnagar, R. S., Lasecki, W. S., & Bigham, J. P. (2014). Accessibility evaluation of classroom captions. ACM Transactions on Accessible Computing (TACCESS), 5(3), 7. DOI: http://dx.doi.org/10.1145/2543578 Google ScholarDigital Library
- Lartz & Stout, "Perspectives of Assistive Technology from Deaf Students at a Hearing University," Assistive Technology Outcomes & Benefits, Fall 2008, Vol.5, Num. 1Google Scholar
- Lasecki, W. S., Kushalnagar, R., & Bigham, J. P. (2014, April). Helping students keep up with real-time captions by pausing and highlighting. InProceedings of the 11th Web for All Conference (p. 39). ACM. DOI: http://dx.doi.org/10.1145/2596695.2596701 Google ScholarDigital Library
- Lasecki, W. S., Kushalnagar, R., & Bigham, J. P. (2014, October). Legion scribe: real-time captioning by non-experts. In Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility(pp. 303--304). ACM. DOI: http://dx.doi.org/10.1145/2661334.2661352 Google ScholarDigital Library
- Lasecki, Walter S.; Miller, Christopher D.; Sadilek, Adam; Abumoussa, Andrew; Borrello, Donato; Kushalnagar, Raja; Bigham, Jeffrey P. 2012b. Real-Time Captioning by Groups of Non-Experts. UIST'12, October 7-10, Cambridge, Massachusetts, USA. DOI: http://dx.doi.org/10.1145/2380116.2380122 Google ScholarDigital Library
- Miltiades Papadopoulos and Elaine Pearson. 2008. Accessible lectures: moving towards automatic speech recognition models based on human methods. In Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility (Assets '08). ACM, New York, NY, USA, 273--274. DOI: http://dx.doi.org/10.1145/1414471.1414534. Google ScholarDigital Library
- O'Shaughnessy, D. (2008). Invited paper: Automatic speech recognition: History, methods and challenges. Pattern Recognition, 41(10), 2965--2979. Google ScholarDigital Library
- Richard Kheir and Thomas Way. 2007. Inclusion of deaf students in computer science classes using real- time speech transcription. In Proceedings of the 12th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education (ITiCSE'07). ACM, New York, 261--265. DOI: http://dx.doi.org/10.1145/1268784.1268860. Google ScholarDigital Library
- Seago, K. (Director), & Ladner, R., Burgstahler, S., & Roth, R. (Producers). (2014). Communication Access Realtime Translation: CART Services for Deaf and Hard-of-Hearing People {Video file}. Retrieved 2016, from http://www.washington.edu/doit/videos/index.php?vid=57Google Scholar
- Shiver, B. N., & Wolfe, R. J. (2015, October). Evaluating Alternatives for Better Deaf Accessibility to Selected Web-Based Multimedia. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility (pp. 231--238). ACM. DOI: http://dx.doi.org/10.1145/2700648.2809857 Google ScholarDigital Library
- Steinfeld, A. (1998). The Benefit of Real-Time Captioning in a Mainstream Classroom as Measured by Working Memory. Volta Review, 100(1), 29--44.Google Scholar
- Stinson, M. S., Elliot, L. B., Kelly, R. R., & Liu, Y. (2009). Deaf and hard-of-hearing students' memory of lectures with speech-to-text and interpreting/note taking services. The Journal of Special Education, 43(1), 52--64.Google ScholarCross Ref
- Van Gelder, Joris; Van Peer, Irene; Aliakseyeu, Dzmitry. 2005. Transcription Table: Text Support During Meetings. M.F. Costabile and F. Paternò (Eds.): INTERACT 2005, LNCS 3585, pp. 1002--1005. Google ScholarDigital Library
- Wald, M. (2006). Captioning for deaf and hard of hearing people by editing automatic speech recognition in real time. In Computers Helping People with Special Needs (pp. 683--690). Springer Berlin Heidelberg Google ScholarDigital Library
Index Terms
- Improving Real-Time Captioning Experiences for Deaf and Hard of Hearing Students
Recommendations
Exploration of Automatic Speech Recognition for Deaf and Hard of Hearing Students in Higher Education Classes
ASSETS '19: Proceedings of the 21st International ACM SIGACCESS Conference on Computers and AccessibilityAutomatic speech recognition (ASR) programs that generate real-time speech-to-text captions can be provided as supplemental access technologies for deaf and hard of hearing (DHH) students in higher education classes. As part of a pilot program, we ...
Deaf and Hard-of-Hearing Perspectives on Imperfect Automatic Speech Recognition for Captioning One-on-One Meetings
ASSETS '17: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and AccessibilityRecent advances in Automatic Speech Recognition (ASR) have made this technology a potential solution for transcribing audio input in real-time for people who are Deaf or Hard of Hearing (DHH). However, ASR is imperfect; users must cope with errors in ...
Evaluating the Usability of Automatically Generated Captions for People who are Deaf or Hard of Hearing
ASSETS '17: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and AccessibilityThe accuracy of Automated Speech Recognition (ASR) technology has improved, but it is still imperfect in many settings. Researchers who evaluate ASR performance often focus on improving the Word Error Rate (WER) metric, but WER has been found to have ...
Comments