Skip to main content
Log in

Improving multimodal web accessibility for deaf people: sign language interpreter module

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The World Wide Web is becoming increasingly necessary for everybody regardless of age, gender, culture, health and individual disabilities. Unfortunately, there are evidently still problems for some deaf and hard of hearing people trying to use certain web pages. These people require the translation of existing written information into their first language, which can be one of many sign languages. In previous technological solutions, the video window dominates the screen, interfering with the presentation and thereby distracting the general public, who have no need of a bilingual web site. One solution to this problem is the development of transparent sign language videos which appear on the screen on request. Therefore, we have designed and developed a system to enable the embedding of selective interactive elements into the original text in appropriate locations, which act as triggers for the video translation into sign language. When the short video clip terminates, the video window is automatically closed and the original web page is shown. In this way, the system significantly simplifies the expansion and availability of additional accessibility functions to web developers, as it preserves the original web page with the addition of a web layer of sign language video. Quantitative and qualitative evaluation has demonstrated that information presented through a transparent sign language video increases the users’ interest in the content of the material by interpreting terms, phrases or sentences, and therefore facilitates the understanding of the material and increases its usefulness for deaf people.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Bangham J, Cox SJ, Lincoln M, Marshall I, Tutt M, Wells M (2000) Signing for the deaf using virtual humans. In: Proceedings of the IEE Seminar on Speech and Language Processing for Disabled and Elderly People (Ref. No. 2000/025), pp 4/1–4/5

  2. Beskow J, Engwall O, Granström B, Nordqvist P, Wik P (2008) Visualization of speech and audio for hearing impaired persons. Technol Disabil 20(2):97–107

    Google Scholar 

  3. Brophy P, Craven J (2007) Web accessibility. Libr Trends 55(4):950–972

    Google Scholar 

  4. Cavender A, Rahul V, Barney DK, Ladner RE, Riskin EA (2007) MobileASL: intelligibility of sign language video over mobile phones. Disabil Rehabil Assist Technol 3(1):93–105

    Article  Google Scholar 

  5. Davis CD (1999) Visual enhancements: improving deaf students’ transition skills using multimedia technology. Career Dev Except Individ 22(2):267–281

    Article  Google Scholar 

  6. Debevc M, Peljhan Z (2004) The role of video technology in on-line lectures for the deaf. Disabil Rehabil 26(17):1048–1059

    Article  Google Scholar 

  7. Debevc M, Stjepanovič Z, Povalej P, Verlič M, Kokol P (2007) Accessible and adaptive e-learning materials: considerations for design and development. In: Universal access in human-computer interaction. Applications and Services, Lecture Notes in Computer Sciences (LNCS 4556). Springer, Berlin, Heidelberg, pp 549–558

  8. EBU Technical-Information I44-2004, EBU report on Access Services—includes recommendations, http://www.ebu.ch/CMSimages/en/tec_text_i44-2004_tcm6-14894.pdf. Accessed 15 Sep 2009

  9. Elliott R, Glauert J, Kennaway J, Marshall I, Safr E (2004) Linguistic modelling and language processing technologies for Avatar-based sign language presentation. Univ Access Inf Soc 6(4):375–391

    Article  Google Scholar 

  10. Fajardo I, Cañas JJ, Salmerón L, Abascal J (2006) Improving deaf users’ accessibility in hypertext information retrieval: are graphical interfaces useful for them? Behav Inf Technol 25(6):455–467

    Article  Google Scholar 

  11. Farwell RM (1976) Speechreading: a research review. Am Ann Deaf 121(1):18–22

    Google Scholar 

  12. Fels DI, Richards J, Hardman JL, Daniel G (2006) Sign language web pages. Am Ann Deaf 151(4):423–433

    Article  Google Scholar 

  13. Hanson VL (2009) Computing technologies for deaf and hard of hearing users. In: Sears A, Jacko JA (eds) Human-computer interaction handbook: fundamentals, evolving technologies and emerging applications, 2nd edn. Erlbaum, Mahwah, pp 885–893

    Google Scholar 

  14. Hermans D, Knoors H, Ormel E, Verhoeven L (2008) The relationship between the reading and signing skills of deaf children in bilingual education programs. J Deaf Stud Deaf Educ 13(4):518–530

    Article  Google Scholar 

  15. Hellström G (1997) Quality measurement on video communication for sign language. In: Proceedings of the 16th International Symposium on Human Factors in Telecommunications, Oslo, Norway

  16. Hilzensauer M (2006) Information technology for deaf people. In: Kacprzyk J (ed) Studies in computational intelligence. Springer, Berlin/Heidelberg, pp 183–206

    Google Scholar 

  17. Holley MC (2005) Keynote review: the auditory system, hearing loss and potential targets for drug development. Drug Discov Today 10(19):1269–1282

    Article  Google Scholar 

  18. Holt J (1995) Efficiency of screening procedures for assigning levels of the stanford achievement test (Eighth Edition) to students who are deaf or hard of hearing. Am Ann Deaf 140(1):23–27

    Google Scholar 

  19. Holzinger A (2005) Usability engineering for software developers. Commun ACM 48(1):71–74

    Article  Google Scholar 

  20. Holzinger A, Searle G, Nischelwitzer A (2007) On some aspects of improving mobile applications for the elderly. In: Constantine S (ed) Coping with diversity in universal access, Lecture Notes in Computer Science (LNCS 4554). Springer, Berlin, pp 923–932

    Chapter  Google Scholar 

  21. Huenerfauth M (2008) Generating American Sign Language animation: overcoming misconceptions and technical challenges. Univ Access Inf Soc 6(4):419–434

    Article  Google Scholar 

  22. Ivory M, Hearst M (2001) State of the art in automating usability evaluation of user interfaces. ACM Comput Surv 33(4):1–47

    Article  Google Scholar 

  23. Kennaway J, Glauert J, Zwitserlood I (2007) Providing signed content on the Internet by synthesized animation. ACM Transactions on Computer-Human Interaction 14(3)

  24. Kronreif G, Dotter F, Bergmeister E, Krammer K, Hilzensauer M, Okorn I, Skant A, Orter R, Rezzonico S, Barreto B (2000) SMILE: demonstration of a cognitively oriented solution to the improvement of written language competence of deaf people. In: Proceedings of the 7th International Conference on Computers Helping People with Special Needs—ICCHP. Karlsruhe, Germany

  25. Marschark M, Green V, Hindmarsh G, Walker S (2000) Understanding theory of mind in children who are deaf. J Child Psychol Psychiatry Allied Discipl 41(8):1067–1073

    Article  Google Scholar 

  26. Martini A, Mazzoli M (1999) Achievements of the European Working Group on genetics of hearing impairment. Int J Pediatr Otorhinolaryngol 49(1):155–158

    Article  Google Scholar 

  27. Muir L, Richardson I (2005) Perception of sign language and its application to visual communications for deaf people. J Deaf Stud Deaf Educ 10(4):390–401

    Article  Google Scholar 

  28. Nigay L, Coutaz J (1993) A design space for multimodal systems: concurrent processing and data fusion. In: Proceedings of the INTERACT ’93 and CHI ’93 conference on Human factors in computing systems table of contents, Amsterdam, The Netherlands, pp.172–178

  29. Olivrin GJL (2007) Is video on the web for sign languages. In: Proceedings of the W3C Video on the Web Workshop. San Jose, California and Brussels, Belgium

  30. Roberts VL, Fels DI (2006) Methods for inclusion: employing think aloud protocols in software usability studies with individuals who are deaf. Int J Hum Comput Stud 64(6):489–501

    Article  Google Scholar 

  31. Shiel B (2006) Evaluation of the social and economic costs of hearing impairment. In: Report for HEAR-IT. http://www.hear-it.org/multimedia/Hear_It_Report_October_ 2006.pdf. Accessed 15 Sep 2009

  32. Signing Savvy (2009) http://www.signingsavvy.com/. Accessed 15 Sep 2009

  33. Sign Language Interpreter Module Project (2010). http://www.slimodule.com/. Accessed 19 Feb 2010

  34. Smith C, Mayes T (1995) Telematics applications for education and training: usability guide. Version 2, DGXIII 3/c, Commission of the European Communities. Brussels, Belgium

  35. Snoek CGM, Worring M (2004) Multimodal video indexing: a review of the state-of-the-art. Multimedia Tools and Applications 25(1):5–35

    Article  Google Scholar 

  36. Straetz K, Kaibel A, Raithel V, Specht M, Grote K, Kramer F (2004) An e-learning environment for deaf adults. In: Proceedings of the 8th ERCIM Workshop “User Interfaces for All”. Vienna, Austria

  37. Stewart D, Schein J, Cartwright B (1998) Sign language interpreting: exploring its art and science. Allyn & Bacon, Needham Heights

    Google Scholar 

  38. United Nations (2006) Convention on the rights of persons with disabilities. New York: UN. http://www.un.org/disabilities/default.asp?navid=12&pid=150. Accessed 15 Sep 2009

  39. United Nations (2006) Global Audit of Web Accessibility. New York: UN. http://www.nomensa.com/resources/research/united-nationsglobal-audit-of-accessibility.html. Accessed 15 Sep 2009

  40. Uskov V, Uskov A (2006) Web-based education: 2006–2010 perspectives. International Journal of Advanced Technology for Learning 3(3):1–13

    MathSciNet  Google Scholar 

  41. Vanderheiden GC (1992) Full visual annotation of auditorially presented information for users who are deaf: ShowSounds. In: Proceedings of the RESNA International Conference. Toronto

  42. Vogler C, Metaxas D (2001) A framework for recognizing the simultaneous aspects of American sign language. Comput Vis Image Underst 81(3):358–384

    Article  MATH  Google Scholar 

  43. Web Content Accesssibility Guidelines (WCAG 1.0) W3C Recommendation 5 May 1999. Chisholm W, Vanderheiden G, Jacobs I (ed). http://www.w3.org/TR/WCAG10/. Accessed 15 Sep 2009

  44. Web Content Accesssibility Guidelines (WCAG 2.0) W3C Recommendation 11 December 2008. Caldwell B, Cooper M, Reid LG, Vanderheiden G (ed). http://www.w3.org/TR/WCAG20/. Accessed 15 Sep 2009

  45. World Federation of the Deaf (WFD) (2003) Position Paper regarding the United Nations Convention on the Rights of People with Disabilities. Ad Hoc Committee on a Comprehensive and Integral International Convention on the Protection and Promotion of the Rights and Dignity of Persons with Disabilities, 24 June 2003. http://www.un.org/esa/socdev/enable/rights/contrib-wfd.htm. Accessed 15 Sep 2009

  46. YouTube (2010). http://www.youtube.com/. Accessed 19 Feb 2010

Download references

Acknowledgement

The project is partially supported by the European Commission within the framework of the Lifelong Learning Programme, project DEAFVOC 2. It is also partially supported by the Slovenian Research Agency in the framework of the Science to Youth program which provides financial support to young researchers. Special thanks go to Petra Rezar from the Ljubljana School for the Deaf for her comments and suggestions while using the first prototype of the application, to the Association of the Deaf and Hard-Of-Hearing People of Podravje for their help in the evaluation of the application, as well as to Milan Rotovnik from University of Maribor for his work on transparency.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Matjaž Debevc.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Debevc, M., Kosec, P. & Holzinger, A. Improving multimodal web accessibility for deaf people: sign language interpreter module. Multimed Tools Appl 54, 181–199 (2011). https://doi.org/10.1007/s11042-010-0529-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-010-0529-8

Keywords

Navigation