Kimberly A. Weaver
Thad Starner
ABSTRACT
Language immersion from birth is crucial to a child's language development. However, language immersion can be particularly challenging for hearing parents of deaf children to provide as they may have to overcome many difficulties while learning sign language. We intend to create a mobile device-based system to help hearing parents learn sign language. The first step is to understand what level of detail (i.e., resolution) is necessary for novice signers to learn from video of signs. In this paper we present the results of a study designed to evaluate the ability of novices learning sign language to ascertain the details of a particular sign based on video presented on a mobile device. Four conditions were presented. Three conditions involve manipulation of video resolution (low, medium, and high). The fourth condition employs insets showing the sign handshapes along with the high resolution video. Subjects were tested on their ability to emulate the given sign over 80 signs commonly used between parents and their young children. Although participants noticed a reduction in quality in the low resolution condition, there was no significant effect of condition on ability to generate the sign. Sign difficulty had a significant correlation with ability to correctly reproduce the sign. Although the inset handshape condition did not improve the participants' ability to emulate the signs correctly, participant feedback provided insight into situations where insets would be more useful, as well as further suggestions to improve video intelligibility. Participants were able to reproduce even the most complex signs tested with relatively high accuracy.
- D. Anderson and R. Judy. The MacArthur communicative development inventory: Normative data for american sign language. Journal of Deaf Studies and Deaf Education, 7(2):83--106, 2002.Google Scholar
Cross Ref
- A. Cavender, R. Vanam, D. K. Barney, R. E. Ladner, and E. A. Riskin. MobileASL: intelligibility of sign language video over mobile phones. Disability and Rehabilitation: Assistive Technology, 3(1):93, 2008.Google ScholarCross Ref
- F. M. Ciaramello and S. S. Hemami. Can you see me now? an objective metric for predicting intelligibility of compressed american sign language video. In Proc. Human Vision and Electronic Imaging (HVEI) 2007, volume 6492, page 21, Mar. 2007.Google ScholarCross Ref
- R. Elliott, J. Glauert, J. Kennaway, I. Marshall, and E. Safar. Linguistic modelling and language-processing technologies for avatar-based sign language presentation. Universal Access in the Information Society, 6(4):375--391, Feb. 2008. Google ScholarDigital Library
- L. Fenson, V. A. Marchman, D. J. Thal, P. S. Dale, J. S. Reznick, and E. Bates. MacArthur-Bates communicative development inventories, 2004.Google Scholar
- Gallaudet Research Institute. Regional and national summary report of data from the 2007-08 annual survey of deaf and hard of hearing children and youth. Technical report, GRI, Gallaudet University, Washington DC, Nov. 2008.Google Scholar
- J. R. W. Glauert, R. Elliott, S. J. Cox, J. Tryggvason, and M. Sheard. VANESSA: a system for communication between deaf and hearing people. Technology & Disability, 18(4):207--216, Nov. 2006.Google ScholarCross Ref
- A. Grieve-Smith. SignSynth: a sign language synthesis application using Web3D and perl. In Gesture and Sign Language in Human-Computer Interaction, pages 37--53. 2002. Google ScholarDigital Library
- S. M. Halawani. Arabic sign language translation system on mobile devices. International Journal of Computer Science and Network Security, 8(1):251--256, 2008.Google Scholar
- V. Henderson-Summet. Facilitating Communication for Deaf Individuals with Mobile Technologies. Doctoral thesis, Georgia Institute of Technology, Atlanta, GA, USA, 2010.Google Scholar
- V. Henderson-Summet, K. Weaver, T. L. Westeyn, and T. E. Starner. American sign language vocabulary: computer aided instruction for non-signers. In Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) 2008, pages 281--282, Halifax, Nova Scotia, Canada, 2008. ACM. Google ScholarDigital Library
- M. Huenerfauth. Evaluation of a psycholinguistically motivated timing model for animations of american sign language. In Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) 2008, pages 129--136, Halifax, Nova Scotia, Canada, 2008. ACM. Google ScholarDigital Library
- B. F. Johnson and J. K. Caird. The effect of frame rate and video information redundancy on the perceptual learning of american sign language gestures. In Conference companion on Human factors in computing systems: common ground, pages 121--122, Vancouver, British Columbia, Canada, 1996. ACM. Google ScholarDigital Library
- K. Karpouzis, G. Caridakis, S. Fotinea, and E. Efthimiou. Educational resources and implementation of a greek sign language synthesis architecture. Computers & Education, 49(1):54--74, Aug. 2007. Google ScholarDigital Library
- R. I. Mayberry. When timing is everything: Age of First-Language acquisition effects on Second-Language learning. Applied Psycholinguistics, 28(03):537--549, 2007.Google ScholarCross Ref
- M. P. Moeller. Early intervention and language development in children who are deaf and hard of hearing. Pediatrics, 106(3):e43, Sept. 2000.Google ScholarCross Ref
- L. J. Muir and I. E. G. Richardson. Perception of sign language and its application to visual communications for deaf people. Journal of Deaf Studies and Deaf Education, 10(4):390--401, 2005.Google ScholarCross Ref
- H. Sagawa and M. Takeuchi. A teaching system of japanese sign language using sign language recognition and generation. In Proceedings of the Tenth ACM International Conference on Multimedia, pages 137--145, Juan-les-Pins, France, 2002. ACM. Google ScholarDigital Library
- J. L. Singleton and E. L. Newport. When learners surpass their models: The acquisition of American Sign Language from inconsistent input. Cognitive Psychology, 49(4):370--407, Dec. 2004.Google ScholarCross Ref
- T. E. Starner, C. M. Snoeck, B. A. Wong, and R. M. McGuire. Use of mobile appointment scheduling devices. In CHI '04 Extended Abstracts on Human Factors in Computing Systems, pages 1501--1504, Vienna, Austria, 2004. ACM. Google ScholarDigital Library
Index Terms
- An evaluation of video intelligibility for novice american sign language learners on a mobile device
Recommendations
Comparing Methods of Displaying Language Feedback for Student Videos of American Sign Language
ASSETS '15: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & AccessibilityDeaf children benefit from early exposure to language, and higher levels of written language literacy have been measured in deaf adults who were raised in homes using American Sign Language (ASL). Prior work has established that new parents of deaf ...
Comparing native signers' perception of American Sign Language animations and videos via eye tracking
ASSETS '13: Proceedings of the 15th International ACM SIGACCESS Conference on Computers and AccessibilityAnimations of American Sign Language (ASL) have accessibility benefits for signers with lower written-language literacy. Our lab has conducted prior evaluations of synthesized ASL animations: asking native signers to watch different versions of ...
Evaluation of Language Feedback Methods for Student Videos of American Sign Language
Special Issue (Part 2) of Papers from ASSETS 2015This research investigates how to best present video-based feedback information to students learning American Sign Language (ASL); these results are relevant not only for the design of a software tool for providing automatic feedback to students but ...
Comments