Top

Published in:

2021 | OriginalPaper | Chapter

Sign Language Recognition Using Leap Motion Based on Time-Frequency Characterization and Conventional Machine Learning Techniques

Authors : D. López-Albán, A. López-Barrera, D. Mayorca-Torres, D. Peluffo-Ordóñez

Published in: Applied Informatics

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

The abstract should briefly summarize the contents of the paper in Sign language is the form of communication between the deaf and hearing population, which uses the gesture-spatial configuration of the hands as a communication channel with their social environment. This work proposes the development of a gesture recognition method associated with sign language from the processing of time series from the spatial position of hand reference points granted by a Leap Motion optical sensor. A methodology applied to a validated American Sign Language (ASL) Dataset which involves the following sections: (i) preprocessing for filtering null frames, (ii) segmentation of relevant information, (iii) time-frequency characterization from the Discrete Wavelet Transform (DWT). Subsequently, the classification is carried out with Machine Learning algorithms (iv). It is graded by a 97.96% rating yield using the proposed methodology with the Fast Tree algorithm.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Object Detection Based Software System for Automatic Evaluation of Cursogramas Images

next chapter Solar Radiation Prediction Using Machine Learning Techniques

World Health Organization. Deafness and hearing loss. https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss. Accessed 27 May 2021

Al-Hammadi, M., et al.: Deep learning-based approach for sign language gesture recognition with efficient hand gesture representation. IEEE Access 8, 192527–192542 (2020). https://doi.org/10.1109/ACCESS.2020.3032140CrossRef

Cheok, M.J., Omar, Z., Jaward, M.H.: A review of hand gesture and sign language recognition techniques. Int. J. Mach. Learn. Cybern. 10(1), 131–153 (2017). https://doi.org/10.1007/s13042-017-0705-5CrossRef

Zafrulla, Z., Brashear, H., Starner, T., Hamilton, H., Presti, P.: American sign language recognition with the kinect. In: Proceedings of the 13th International Conference on Multimodal Interfaces, pp. 279–286, November 2011. https://doi.org/10.1145/2070481.2070532

Chong, T.W., Lee, B.G.: American sign language recognition using leap motion controller with a machine learning approach. Sensors 18(10), 3554 (2018). https://doi.org/10.3390/s18103554CrossRef

Weichert, F., Bachmann, D., Rudak, B., Fisseler, D.: Analysis of the accuracy and robustness of the leap motion controller. Sensors 13(5), 6380–6393 (2013). https://doi.org/10.3390/s130506380CrossRef

Lei, L., Dashun, Q.: Design of data-glove and Chinese sign language recognition system based on ARM9. In: 2015 12th IEEE International Conference on Electronic Measurement & Instruments (ICEMI), vol. 3, pp. 1130–1134. IEEE, July 2015. https://doi.org/10.1109/ICEMI.2015.7494440

Marin, G., Dominio, F., Zanuttigh, P.: Hand gesture recognition with leap motion and kinect devices. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 1565–1569. IEEE, October 2014. https://doi.org/10.1109/ICIP.2014.7025313

Shin, H., Kim, W.J., Jang, K.A.: Korean sign language recognition based on image and convolution neural network. In: Proceedings of the 2nd International Conference on Image and Graphics Processing, pp. 52–55, February 2019. https://doi.org/10.1145/3313950.3313967

10.

Weerasekera, C.S., Jaward, M.H., Kamrani, N.: Robust asl fingerspelling recognition using local binary patterns and geometric features. In: 2013 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–8. IEEE, November 2013. https://doi.org/10.1109/DICTA.2013.6691521

11.

Ravi, S., Suman, M., Kishore, P.V.V., Kumar, E.K., Kumar, M.T.K., et al.: Multi modal spatio temporal cotrained CNNs with single modal testing on RGB-D based sign language gesture recognition. J. Comput. Lang. 52, 88–102 (2019). https://doi.org/10.1016/j.cola.2019.04.002CrossRef

12.

Su, Y., Qing, Z.: Continuous Chinese sign language recognition with CNN-LSTM. In: Proceedings of SPIE 10420, Ninth International Conference on Digital Image Processing (ICDIP 2017), 104200F, 21 July 2017. https://doi.org/10.1117/12.2281671

13.

Hernandez, V., Suzuki, T., Venture, G.: Convolutional and recurrent neural network for human activity recognition: Application on American sign language. PLoS ONE 15(2), e0228869 (2020). https://doi.org/10.1371/journal.pone.0228869CrossRef

14.

Shanmuganathan, V., Yesudhas, H.R., Khan, M.S., Khari, M., Gandomi, A.H.: R-CNN and wavelet feature extraction for hand gesture recognition with EMG signals. Neural Comput. Appl. 32(21), 16723–16736 (2020). https://doi.org/10.1007/s00521-020-05349-wCrossRef

15.

Hernandez, V., Suzuki, T., Venture, G.: American Sign Language classification - LeapMotion - 25 subjects - 60 signs. Mendeley Data. V1 (2018). https://doi.org/10.17632/8yyp7gbg6z.1

16.

Vysocký, A., Grushko, S., Oščádal, P., Kot, T.; Babjak, J., Jánoš, R., Sukop, M., Bobovský, Z.: Analysis of precision and stability of hand tracking with leap motion sensor. Sensors 2020, 20, 4088 (2020) https://doi.org/10.3390/s20154088

Title: Sign Language Recognition Using Leap Motion Based on Time-Frequency Characterization and Conventional Machine Learning Techniques
Authors: D. López-Albán
A. López-Barrera
D. Mayorca-Torres
D. Peluffo-Ordóñez
Publisher: Springer International Publishing
Book: Applied Informatics
Print ISBN: 978-3-030-89653-9

Electronic ISBN: 978-3-030-89654-6

Copyright Year: 2021
DOI: https://doi.org/10.1007/978-3-030-89654-6_5

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner