Skip to main content
SpringerLink
Log in

A gesture recogintion architecture for Arabic sign language communication system

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Sign language is the most natural and expressive way for the hearing impaired to communicate. With technological advances in multimedia systems and applications, technology-mediated sign language communication systems have long attracted researchers to enhance the communication capabilities for the speech and hearing impaired, promising improved social opportunities and integration. This paper introduces a framework for Arabic sign language communication using Microsoft Kinect device. The merit of the proposed framework is twofold: first, the framework supports an affordable and easily deployable real-time communication system using Arabic sign language, and secondly, it provides a real-time feedback about the signer performance via real-time avatar animation. A prototype application is developed to demonstrate the merits of the proposed framework. Experimental results show that the proposed Arabic sign language method enjoys a sign detection rate of 96 %. Furthermore, the average task completion time to complete an Arabic sign was about 2.2 s. This implies that the proposed method can be used to create a real-time Arabic sign language communication system. Finally, participants of the study highlighted that the proposed system is user-friendly and easy to use, and can be used at low cost to recognize and display Arabic signs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Agarwal A, Thakur MK Sign Language Recognition using Microsoft Kinect. 2013 Sixth International Conference on Contemporary Compouting (IC3):181–185

  2. Arabic Sign language Dictionary القاموس الإشاري العربي للصم http://www.menasy.com/index.htm Accessed November 15th, 2014.

  3. Assaleh K, Shanableh T, Fanaswala, Mustafa et al. Vision-based system for continuous Arabic Sign Language recognition in user dependent mode. 2008 5th International Symposium on Mechatronics and Its Applications (ISMA): 1–5

  4. Buttussi F, Chittaro L, Coppo M Using Web3D Technologies for Visualization and Search of Signs in an International Sign Language Dictionary. 2007 Proceedings of the Twelfth International Conference on 3D Web Technology: 61–70

  5. Chao S, Tianzhu Z, Bing-Kun B, Changseng X (2013) Discriminative Exemplar Coding for Sign Language Recognition With Kinect”. IEEE Transactions on Cybernetics 43(5):1418–1428

    Article  Google Scholar 

  6. Dell Latitude E7240 specs. Engadget, http://www.engadget.com/products/dell/latitude/-e7240/specs/ Accessed May 25th, 2015

  7. El-Bendary N, Zawbaa HM, Daoud MS, Hassanien AE, Nakamatsu K (2011) ArSLAT: Arabic Sign Language Alphabets Translator. International Journal of Computer, Information Systems and Industrial Management Applications 3:498–506, ISSN 2150–7988

    Google Scholar 

  8. Halawani SM, Zaitun AB (2012) An Avatar Based Translation System from Arabic Speech to Arabic Sign Language for Deaf People. Int J Inf Sci Educ 2(1):13–20

    Google Scholar 

  9. Jiang Y, Tao J, Ye W, Wang W, Ye Z (2014) An Isolated Sign Language Recognition System Using RGB-D Sensor with Sparse Coding. 2014 I.E. 17th Int Conf Comput Sci Eng (CSE): 21-26

  10. Kinect for Windows is now Available Kinect for Windows Blog, http://blogs.msdn.com/b/kinectforwindows/archive/2012/01/31/kinect-for-windows-is-now-available.aspx Accessed April 7th, 2015.

  11. Kinect with MS-SDK.Unity 3d Asset Store, www.assetstore.unity3d.com/en/#!/content/7747 Accessed November 15th, 2014.

  12. Meet Kinect. Kinect for Windows, https://www.microsoft.com/en-us/kinectforwindows/meetkinect/default.aspx Accessed April 10th, 2015.

  13. Memis A, Albayrak S Turkish Sign Language recognition using spatio-temporal features on Kinect RGB video sequences and depth maps. 2013 21st Signal Processing and Communications Applications Conference : 1–4

  14. Memis A, AlbayrakS (2013) Turkish Sign Language recognition using spatio-temporal features on Kinect RGB video sequences and depth maps. 21st Signal Process Commun Appl Conf (SIU): 1-4

  15. Mohandes M, Deriche M, Liu J (2014) Image-Based and Sensor-Based Approaches to Arabic Sign Language Recognition. IEEE Trans Human-Mach Syst 44(4):551–557

    Article  Google Scholar 

  16. Mohandes M, Liu J, Deriche M (2014) A survey of image-based Arabic sign language recognition. 2014 11th Int Multi-Conf Syst, Signals & Devices (SSD): 1,4

  17. Mohandes M, Quadri SI, Deriche M Arabic Sign Language Recognition an Image-based Approac. 2007 21st International Conference on Advanced Information Networking and Applications Wokshops (AINAW ’07):272–276

  18. Oszust M, Wysocki M (2013) Polish sign language words recognition with Kinect. 6th Int Conf Human Syst Interact (HSI): 219-226

  19. Quick Reference: Kinect 1 vs Kinect 2. The Imaginative Universal, http://www.imaginativeuniversal.com/blog/post/2014/03/05/Quick-Reference-Kinect-1-vs-Kinect-2.aspx Accessed May 25th, 2015

  20. Sagawa H, Takeuchi M A Teaching System of Japanese Sign Language Using Sign Language Recognition and Generatio. 2012 Proceedings of the Tenth ACM International Conference on Multimedia :137–145

  21. Samir A, Aboul-Ela M Error detection and correction approach for Arabic sign language recognition. 2012 Seventh International Conference on Computer Engineering & Systems (ICCES): 117-123

  22. Shanableh T, Assaleh K Video-based feature extraction techniques for isolated Arabic sign language recognition. 2007 9th International Symposium on Signal Processing and Its Applications (ISSPA): 1–4

  23. Sun C, Zhang T, Bao B-K, Xu C Latent support vector machine for sign language recognition with Kinect. 2013 20th IEEE International Conference on Image Processing (ICIP): 4190–4194

  24. Tennant, Brown, Tennant RA, Brown MG (1998) The American Sign Language handshape dictionary. Gallaudet University Press. p. 407. ISBN 1-56368-043-2

  25. The Best Development Platform For Creating Games.Unity, http://unity3d.com/unity Accessed April 8th, 2015.

  26. Tolba MF, Samir A, Abul-Ela M (2012) A proposed graph matching technique for Arabic sign language continuous sentences recognition. 2012 8th Int Conf Informatics Syst (INFOS): 14–20

  27. Tolba MF, Samir A, Abul-Ela M A proposed graph matching technique for Arabic sign language continuous sentences recognition. 2012 8th International Conference on Informatics and Systems (INFOS): MM-14 - MM-20

  28. Vatavu R-D, Anthony L, Wobbrock JO (2012) Gestures as point clouds: a $P recognizer for user interface prototypes. In Proceedings of the 14th ACM international conference on Multimodal interaction (ICMI '12). ACM, New York, NY, USA, 273–280.

  29. Verma HV, Aggarwal E, Chandra S (2013) Gesture recognition using kinect for sign language translation. IEEE Second Int Conf Image Inf Process (ICIIP): 96-100

  30. WHO (2008) The global burden of disease: 2004 update ([Online-Ausg.] ed.). Geneva, Switzerland: World Health Organization. p. 35. ISBN 9789241563710

  31. Yang Q Chinese Sign Language Recognition Based On Video Sequence Appearance Modeling. 2010 5th IEEE Conference on Industrial Electronics and Applications (ICIEA) :1537-1542

Download references

Acknowledgments

The authors would like to thank Usama Afzal and Umair Saad for help in conducting the usability study for this work.

Author information

Authors and Affiliations

  1. New York University Abu Dhabi, New York, NY, USA

    Tamás Aujeszky & Mohamad Eid

Authors
  1. Tamás Aujeszky
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamad Eid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohamad Eid.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aujeszky, T., Eid, M. A gesture recogintion architecture for Arabic sign language communication system. Multimed Tools Appl 75, 8493–8511 (2016). https://doi.org/10.1007/s11042-015-2767-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2767-2

Keywords

Search

Navigation