Amal Nanavati
Aaron Steinfeld
ABSTRACT
The ability to communicate is crucial to leading an independent life. Unfortunately, individuals from developing communities who are deaf and hard of hearing tend to encounter difficulty communicating, due to a lack of educational resources. We present findings from a two-year deployment of Speak Up, a suite of voice-powered games to motivate speech therapy, at a school for the deaf in India. Using ethnographic methods, we investigated the interplay between Speak Up and local educational practices. We found that teachers' speech therapy goals had evolved to differ from those encoded in the games, that the games influenced classroom dynamics, and that teachers had improved their computer literacy and developed creative uses for the games. We used these insights to further enhance Speak Up by creating an explicit teacher role in the games, making changes that encouraged teachers to build their computer literacy, and adding an embodied agent.
Supplemental Material
- Rashid Al-Abri, Mustafa Al-Balushi, Arif Kolethekkat, Deepa Bhargava, Amna Al-Alwi, Hana Al-Bahlani, and Manal Al-Garadi. 2016. The accuracy of IOS device-based uHear as a screening tool for hearing loss: a preliminary study from the Middle East. Oman medical journal 31, 2 (2016), 142.Google Scholar
- Mona A Azim, Nagwa L Badr, and Mohamed F Tolba. 2016. An Enhanced Arabic Phonemes Classification Approach. In Proceedings of the 10th International Conference on Informatics and Systems. ACM, 210--214. Google ScholarDigital Library
- L Bernstein, J Ferguson, and M Goldstein. 1986. Speech training devices for profoundly deaf children. In Acoustics, Speech, and Signal Processing, IEEE International Conference on ICASSP'86., Vol. 11. IEEE, 633--636.Google Scholar
- David Calder. 2008. SpeechKit: a multimedia speech tool. In Proceedings of the 10th International Conference on Information Integration and Web-based Applications & Services. ACM, 647--650. Google ScholarDigital Library
- Madeleine Clute, Madelyn Gioffre, Poornima Kaniarasu, Aditya Kodkany, Vivek Nair, Shree Lakshmi Rao, Aveed Sheikh, Avia Weinstein, Ermine A. Teves, Mary Beatrice Dias, and M Bernardine Dias. 2016. iSTEP 2013: Development and Assessment of Assistive Technology for a School for the Blind in India. Technical Report Carnegie Mellon University-RI-TR-16--30. Pittsburgh, PA.Google Scholar
- Andy Dearden. 2012. See no evil?: ethics in an interventionist ICTD. In Proceedings of the Fifth International Conference on Information and Communication Technologies and Development. ACM, 46--55. Google ScholarDigital Library
- Compassionate Engineering. 2016a. The Braille Writing Tutor. (2016). https://compassionateengineering. wordpress.com/2016/02/26/the-braille-writing-tutor/Google Scholar
- Compassionate Engineering. 2016b. Meet Ms. Muktha. (2016). https://compassionateengineering.wordpress. com/2016/03/03/meet-ms-muktha/Google Scholar
- Santiago Fernández, Alex Graves, and Jürgen Schmidhuber. 2008. Phoneme recognition in TIMIT with BLSTM-CTC. arXiv preprint arXiv:0804.3269 (2008).Google Scholar
- Voula C Georgopoulos. 1999. An investigation of audio-visual speech recognition as applied to multimedia speech therapy applications. In Multimedia Computing and Systems, 1999. IEEE International Conference on, Vol. 1. IEEE, 481--486. Google ScholarDigital Library
- André Grossinho, Sofia Cavaco, and João Magalhães. 2014. An interactive toolset for speech therapy. In Proceedings of the 11th Conference on Advances in Computer Entertainment Technology. ACM, 36. Google ScholarDigital Library
- Thilina Halloluwa, Dhaval Vyas, Hakim Usoof, and KP Hewagamage. 2017. Gamification for development: a case of collaborative learning in Sri Lankan primary schools. Personal and Ubiquitous Computing (2017), 1--17. Google ScholarDigital Library
- iSTEPtbw. 2015a. VLOG 1: Welcome To Mathru! (2015). https://www.youtube.com/watch?v=ASWics-VrJkGoogle Scholar
- iSTEPtbw. 2015b. VLOG 2: Tour of the Mathru Center. (2015). https://www.youtube.com/watch?v=7PIStgUCqWYGoogle Scholar
- Nidhi Kalra, Tom Lauwers, and M Bernardine Dias. 2007. A Braille Writing Tutor to Combat Illiteracy in Developing Communities. In AI in ICT for Development Workshop, International Joint Conference on Artificial Intelligence. Pittsburgh, PA.Google Scholar
- Matthew Kam, Akhil Mathur, Anuj Kumar, and John Canny. 2009. Designing digital games for rural children: a study of traditional village games in India. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 31--40. Google ScholarDigital Library
- Neha Kumar, Trevor Perrier, Michelle Desmond, Kiersten Israel-Ballard, Vikrant Kumar, Sudip Mahapatra, Anil Mishra, Shreya Agarwal, Rikin Gandhi, Pallavi Lal, and Richard Anderson. 2015. Projecting Health: Community-led Video Education for Maternal Health. In Proceedings of the Seventh International Conference on Information and Communication Technologies and Development (ICTD '15). ACM, New York, NY, USA, Article 17, 10 pages. Google ScholarDigital Library
- Maya Lassiter, Amal Nanavati, Erik Pintar, Minnar Xie, Ermine A. Teves, and M Bernardine Dias. 2016. iSTEP 2015: Cross-Cultural Technology Development Toward Language Access for the Deaf and Hard of Hearing. Technical Report Carnegie Mellon University-RI-TR-16--32. Pittsburgh, PA.Google Scholar
- Shaimaa Lazem and Hussein Aly Jad. 2017. We Play We Learn: Exploring the Value of Digital Educational Games in Rural Egypt. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 2782--2791. Google ScholarDigital Library
- Daniel Ling. 2002. Speech and the hearing-impaired child: Theory and practice (2nd ed.). Alex Graham Bell Association for the Deaf and Hard of Hearing.Google Scholar
- Faheema Mahomed-Asmail, Robert H Eikelboom, Hermanus C Myburgh, James Hall III, and others. 2016. Clinical validity of hearScreenTM smartphone hearing screening for school children. Ear and hearing 37, 1 (2016), e11--e17.Google Scholar
- James J Mahshie, Dianne Vari-Alquist, Betty Waddy-Smith, and Lynne E. Bernstein. 1988. Speech Training aids for hearing-impaired individuals: III. Preliminary observations in the clinic and children's homes. Journal of rehabilitation research and development 25, 4 (1988), 69--82.Google Scholar
- Anne-Marie Mann, Uta Hinrichs, Janet C. Read, and Aaron Quigley. 2016. Facilitator, Functionary, Friend or Foe?: Studying the Role of iPads Within Learning Activities Across a School Year. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 1833--1845. Google ScholarDigital Library
- B McPherson, MMS Law, and MSM Wong. 2010. Hearing screening for school children: comparison of low-cost, computer-based and conventional audiometry. Child: care, health and development 36, 3 (2010), 323--331.Google Scholar
- Jean S Moog and Karen K Stein. 2008. Teaching deaf children to talk. Contemporary Issues in Communication Science and Disorders 35 (2008), 133--142.Google ScholarCross Ref
- Hafeni Mthoko and Caroline Khene. 2015. Assessing Outcome and Impact: Towards a Comprehensive Evaluation Approach in ICT4D. In Proceedings of the Seventh International Conference on Information and Communication Technologies and Development (ICTD '15). ACM, New York, NY, USA, Article 50, 4 pages. Google ScholarDigital Library
- Raymond S. Nickerson and Kenneth N. Stevens. 1973. Teaching speech to the deaf: Can a computer help? IEEE Transactions on audio and electroacoustics 21, 5 (1973), 445--455.Google ScholarCross Ref
- Republic of India Parliament. 1995. The Persons with Disabilities (Equal Opportunities, Protection of Rights and Full Participation) Act. (1995).Google Scholar
- World Health Organization. 2017. Deafness and hearing loss. (2017). http://www.who.int/mediacentre/factsheets/fs300/en/Google Scholar
- Becky Sue Parton, Robert Hancock, Jeff Oescher, and others. 2009. Interactive media to support language acquisition for deaf students. i-Manager's Journal on School Educational Technology 5, 1 (2009), 17.Google Scholar
- United Nations Development Programme. 2009. Capacity Development: A UNDP Primer. (13 Oct 2009).Google Scholar
- Shree Lakshmi Rao, Aveed Sheikh, Avia Weinstein, Aditya Kodkany, Madeleine Clute, Madelyn Gioffre, Poornima Kaniarasu, Vivek Nair, Ermine A. Teves, Mary Beatrice Dias, and M Bernardine Dias. 2016. iSTEP 2013: Exploring the Feasibility and Suitability of Assistive Technology at the Mathru Center for Differently-Abled. Technical Report Carnegie Mellon University-RI-TR-16--31. Pittsburgh, PA.Google Scholar
- Sadaf Abdul Rauf, Aatka Javed Butt, Aliza Zahid, Ayesha Jabeen, Abdul Rauf Siddiqi, and Hina Shafique. 2016. Urdu Language Learning Aid based on Lip Syncing and Sign Language for Hearing Impaired Children. International Journal of Computer Science & Information Security 14, 12 (2016), 478.Google Scholar
- Ovidiu Andrei Schipor, Stefan Gheorghe Pentiuc, and Maria Doina Schipor. 2012. Improving computer based speech therapy using a fuzzy expert system. Computing and Informatics 29, 2 (2012), 303--318.Google Scholar
- Carnegie Mellon University Sphinx. 2017. Phoneme Recognition (caveat emptor). (2017). https://cmusphinx.github.io/wiki/phonemerecognition/Google Scholar
- Fiona Ssozi-Mugarura, Ulrike Rivett, and Edwin Blake. 2016. Using Activity Theory to Understand Technology Use and Perception Among Rural Users in Uganda. In Proceedings of the Eighth International Conference on Information and Communication Technologies and Development (ICTD '16). ACM, New York, NY, USA, Article 13, 10 pages. Google ScholarDigital Library
- Rachel E Stark. 1972. Teaching/ba/and/pa/to deaf children using real-time spectral displays. Language and speech 15, 1 (1972), 14--29.Google Scholar
- Chek Tien Tan, Andrew Johnston, Kirrie Ballard, Samuel Ferguson, and Dharani Perera-Schulz. 2013. sPeAK-MAN: towards popular gameplay for speech therapy. In Proceedings of The 9th Australasian Conference on Interactive Entertainment: Matters of Life and Death. ACM, 28. Google ScholarDigital Library
- Chek Tien Tan, Andrew Johnston, Andrew Bluff, Samuel Ferguson, and Kirrie J Ballard. 2014. Retrogaming as visual feedback for speech therapy. In SIGGRAPH Asia 2014 Mobile Graphics and Interactive Applications. ACM, 4. Google ScholarDigital Library
- Saurabh Varshney. 2016. Deafness in India. Indian Journal of Otology 22, 2 (2016), 73.Google ScholarCross Ref
- Madan Vasishta, James Woodward, and Kirk Wilson. 1978. Sign language in India: regional variation within the deaf population. Indian Journal of Applied Linguistics 4, 2 (1978), 66--74.Google Scholar
- Madan M Vasishta. 2011. Deaf around the World, The Impact of Language. Oxford University Press, New York, US, Chapter Social Situations and the Education of Deaf Children in India, 352--358.Google Scholar
- Video Voice. 2017. Entertaining Games and Displays for Speech Therapy. (2017). http://www.videovoice.com/vv_fandg.htmGoogle Scholar
- Akira Watanabe, Yuichi Ueda, and Akiyoshi Shigenaga. 1985. Color display system for connected speech to be used for the hearing impaired. IEEE transactions on acoustics, speech, and signal processing 33, 1 (1985), 164--173.Google ScholarCross Ref
Index Terms
- Speak Up: A Multi-Year Deployment of Games to Motivate Speech Therapy in India
Recommendations
sPeAK-MAN: towards popular gameplay for speech therapy
IE '13: Proceedings of The 9th Australasian Conference on Interactive Entertainment: Matters of Life and DeathCurrent speech therapy treatments are not easily accessible to the general public due to cost and demand. Therapy sessions are also laborious and maintaining motivation of patients is hard. We propose using popular games and speech recognition ...
Towards Establishing a Training Program to Support Future CS Teaching-focused Faculty
SIGCSE 2024: Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1Computer Science programs have seen high enrollments in recent years, which contributed to widening the capacity gap. One way to address this problem is to hire more teaching-focused faculty at both research and non-doctoral granting institutions. ...
Motivating Individuals with Spastic Cerebral Palsy to Speak Using Mobile Speech Recognition
ASSETS '16: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and AccessibilityIndividuals with cerebral palsy (CP) struggle with conditions such as dysarthria, dysphagia, and dyspraxia as they speak. While speech therapy is successful in practice, outside practice requires increased commitment and effort from caregivers. ...
Comments