Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Testing Modified Confusion Entropy as Split Criterion for Decision Trees Read chapter Read first chapter

2019 | OriginalPaper | Chapter

Deep CNN-Based Recognition of JSL Finger Spelling

Authors : Nam Tu Nguen, Shinji Sako, Bogdan Kwolek

Published in: Hybrid Artificial Intelligent Systems

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In this paper, we present a framework for recognition of static finger spelling in Japanese Sign Language on RGB images. The finger spelled signs were recognized by an ensemble consisting of a ResNet-based convolutional neural network and two ResNet quaternion convolutional neural networks. A 3D articulated hand model has been used to generate synthetic finger spellings and to extend a dataset consisting of real hand gestures. Twelve different gesture realizations were prepared for each of 41 signs. Ten images have been rendered for each realization through interpolations between the starting and end poses. Experimental results demonstrate that owing to sufficient amount of training data a high recognition rate can be attained on images from a single RGB camera. Results achieved by the ResNet quaternion convolutional neural network are better than results obtained by the ResNet CNN. The best recognition results were achieved by the ensemble. The JSL-rend dataset is available for download.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Orthogonal Properties of Asymmetric Neural Networks with Gabor Filters
next chapter Algorithm for Constructing a Classifier Team Using a Modified PCA (Principal Component Analysis) in the Task of Diagnosis of Acute Lymphocytic Leukaemia Type B-CLL
Literature
1.
Sagayam, M., Hemanth, J.: Hand posture and gesture recognition techniques for virtual reality applications: a survey. Virtual Reality 21(2), 91–107 (2017)CrossRef
2.
Chen, F., Zhong, Q., Cannella, F., Sekiyama, K., Fukuda, T.: Hand gesture modeling and recognition for human and robot interactive assembly using Hidden Markov Models. Int. J. Adv. Rob. Syst. 12(4), 48 (2015)CrossRef
3.
Raj, M.D., Gogul, I., Thangaraja, M., Kumar, V.: Static gesture recognition based precise positioning of 5-DOF robotic arm using FPGA. In: Trends in Industrial Measurement and Automation (TIMA), pp. 1–6 (2017)
4.
Liu, H., Wang, L.: Gesture recognition for human-robot collaboration: a review. Int. J. Ind. Ergon. 68, 355–367 (2018)CrossRef
5.
Patil, S., et al.: GesturePod: programmable gesture recognition for augmenting assistive devices, Technical report, Microsoft, May 2018
6.
Rautaray, S., Agrawal, A.: Vision based hand gesture recognition for human computer interaction: a survey. Artif. Intell. Rev. 43(1), 1–54 (2015)CrossRef
7.
Al-Shamayleh, A.S., Ahmad, R., Abushariah, M., Alam, K.A., Jomhari, N.: A systematic literature review on vision based gesture recognition techniques. Multimedia Tools Appl. 77(21), 28121–28184 (2018)CrossRef
8.
Ohn-Bar, E., Trivedi, M.: Hand gesture recognition in real time for automotive interfaces: a multimodal vision-based approach and evaluations. IEEE Trans. Intell. Transp. Syst. 15(6), 2368–2377 (2014)CrossRef
9.
Pisharady, P., Saerbeck, M.: Recent methods and databases in vision-based hand gesture recognition. Comput. Vis. Image Underst. 141, 152–165 (2015)CrossRef
10.
Oyedotun, O., Khashman, A.: Deep learning in vision-based static hand gesture recognition. Neural Comput. Appl., 1–11 (2016)
11.
Tompson, J., Stein, M., LeCun, Y., Perlin, K.: Real-time continuous pose recovery of human hands using convolutional networks. ACM Trans. Graph. 33(5) (2014)CrossRef
12.
Nagi, J., Ducatelle, F., et al.: Max-pooling convolutional neural networks for vision-based hand gesture recognition. In: IEEE ICSIP, pp. 342–347 (2011)
13.
14.
Koller, O., Ney, H., Bowden, R.: Deep hand: how to train a CNN on 1 million hand images when your data is continuous and weakly labelled. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3793–3802 (2016)
15.
Tabata, Y., Kuroda, T.: Finger spelling recognition using distinctive features of hand shape. In: International Conference on Disability, Virtual Reality and Associated Technologies with Art Abilitation, pp. 287–292 (2008)
16.
Kane, L., Khanna, P.: A framework for live and cross platform fingerspelling recognition using modified shape matrix variants on depth silhouettes. Comput. Vis. Image Underst. 141, 138–151 (2015)CrossRef
17.
18.
Rosalina, L.Y., Hadisukmana, N., Wahyu, R.B., Roestam, R., Wahyu, Y.: Implementation of real-time static hand gesture recognition using artificial neural network. In: CAIPT, pp. 1–6 (2017)
19.
Asad, M., Slabaugh, G.: SPORE: staged probabilistic regression for hand orientation inference. Comput. Vis. Image Underst. 161, 114–129 (2017)CrossRef
20.
Dawod, A.Y., Nordin, M.J., Abdullah, J.: Static fingerspelling recognition based on boundary tracing algorithm and chain code. In: International Conference on Intelligent Systems, Metaheuristics & Swarm Intelligence, pp. 104–109. ACM (2018)
21.
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
22.
Parcollet, T., et al.: Quaternion convolutional neural networks for end-to-end automatic speech recognition. In: Interspeech, ISCA, pp. 22–26 (2018)
23.
Popa, C.A.: Learning algorithms for quaternion-valued neural networks. Neural Process. Lett. 47(3), 949–973 (2018)CrossRef
24.
Nitta, T.: A quaternary version of the back-propagation algorithm. In: Proceedings of International Conference on Neural Networks, vol. 5, pp. 2753–2756 (1995)
25.
26.
Opitz, D., Maclin, R.: Popular ensemble methods: an empirical study. J. Artif. Int. Res. 11(1), 169–198 (1999)MATH
Metadata
Title
Deep CNN-Based Recognition of JSL Finger Spelling
Authors
Nam Tu Nguen
Shinji Sako
Bogdan Kwolek
Copyright Year
2019
Book
Hybrid Artificial Intelligent Systems

Print ISBN: 978-3-030-29858-6

Electronic ISBN: 978-3-030-29859-3

Copyright Year: 2019

DOI
https://doi.org/10.1007/978-3-030-29859-3_51

Premium Partner

    Image Credits