Skip to main content

2019 | OriginalPaper | Buchkapitel

Recognizing Hand Configurations of Brazilian Sign Language Using Convolutional Neural Networks

verfasst von : A. S. Oliveria, C. F. F. Costa Filho, M. G. F. Costa

Erschienen in: XXVI Brazilian Congress on Biomedical Engineering

Verlag: Springer Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper proposes evaluating three convolutional neural network architectures for recognizing hand configurations of the Brazilian Sign Language (Libras). To improve the generalization of neural networks, two techniques were employed: dropout and L2 regularization. A proprietary database consisting of 12.200 depth images, captured with the Kinect® sensor was used. Two hundred images were captured for each one of 61 Hand Configurations (HC) of Libras. The training and testing subsets were compounded using an interleave technique. An accuracy of 98% was achieved. This value is better than previous results obtained, with the same dataset, using the k-Nearest Neighbor (kNN) and Novelty classifiers, 95.41% and 96.31%, respectively.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Costa Filho, C.F.F., Santos, B.L., Souza, R.S., Santos, J.R., Costa, M.G.F.: A new method for recognizing hand configurations of Brazilian gesture language. In: 38th Annual International Conference of the IEEE-Engineering-in-Medicine-and- Biology-Society (EMBC), IEEE Engineering in Medicine and Biology Society Conference Proceedings, pp. 3829–3834 (2016) Costa Filho, C.F.F., Santos, B.L., Souza, R.S., Santos, J.R., Costa, M.G.F.: A new method for recognizing hand configurations of Brazilian gesture language. In: 38th Annual International Conference of the IEEE-Engineering-in-Medicine-and- Biology-Society (EMBC), IEEE Engineering in Medicine and Biology Society Conference Proceedings, pp. 3829–3834 (2016)
2.
Zurück zum Zitat Costa Filho, C.F.F, Santos, B.L., Souza, R.S., Santos, J.R., Costa, M.G.F.: A fully automatic method for recognizing hand configurations of Brazilian Sign Language. Res. Biomed. Eng. 33(1), 78–89 (2017) Costa Filho, C.F.F, Santos, B.L., Souza, R.S., Santos, J.R., Costa, M.G.F.: A fully automatic method for recognizing hand configurations of Brazilian Sign Language. Res. Biomed. Eng. 33(1), 78–89 (2017)
3.
Zurück zum Zitat Esobedo, E.C., Camara, G.C.: A new approach for dynamic gesture recognition using skeleton trajectory representation and histograms of cumulative magnitudes. In: 29th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 209–216 (2016) Esobedo, E.C., Camara, G.C.: A new approach for dynamic gesture recognition using skeleton trajectory representation and histograms of cumulative magnitudes. In: 29th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 209–216 (2016)
4.
Zurück zum Zitat Escobedo, E.C., Camara, G.C.: A robust gesture recognition using hand local data and skeleton trajectory. In: IEEE International Conference on Image Processing (ICIP), pp. 1240–1244 (2015) Escobedo, E.C., Camara, G.C.: A robust gesture recognition using hand local data and skeleton trajectory. In: IEEE International Conference on Image Processing (ICIP), pp. 1240–1244 (2015)
5.
Zurück zum Zitat Porfirio A.J., Wiggers K.L., Oliveira, L.E.S., Weingaertner, D.: Libras sign language hand configuration recognition based on 3D meshes. In: Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, Manchester, pp. 1588–1593 (2013) Porfirio A.J., Wiggers K.L., Oliveira, L.E.S., Weingaertner, D.: Libras sign language hand configuration recognition based on 3D meshes. In: Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, Manchester, pp. 1588–1593 (2013)
6.
Zurück zum Zitat Almeida, S.G.M., Guimaraes, F.G., Ramirez, J.A.: Feature extraction in Brazilian sign language recognition based on phonological structure and using RGB-D sensors. Expert Syst. Appl. 41(16), 7259–7271 (2014)CrossRef Almeida, S.G.M., Guimaraes, F.G., Ramirez, J.A.: Feature extraction in Brazilian sign language recognition based on phonological structure and using RGB-D sensors. Expert Syst. Appl. 41(16), 7259–7271 (2014)CrossRef
7.
Zurück zum Zitat Stiehl, D., Addams, L., Oliveira, L.S., Guimaraes, C., Brito Jr, A.S.: Towards a signwriting recognition system. In: 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 26–30 (2015) Stiehl, D., Addams, L., Oliveira, L.S., Guimaraes, C., Brito Jr, A.S.: Towards a signwriting recognition system. In: 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 26–30 (2015)
8.
Zurück zum Zitat Sutton, Y.: A way to analyze American Sign Language and any other sign language without translation into any spoken language. In: National Symposium on Sign Language Research and Teaching (1980) Sutton, Y.: A way to analyze American Sign Language and any other sign language without translation into any spoken language. In: National Symposium on Sign Language Research and Teaching (1980)
9.
Zurück zum Zitat Kang, B., Tripathi, S. Nguyen, T.Q.: Real-time sign language fingerspelling recognition using convolutional neural networks from depth map. In: 3rd IAPR Asian Conference on Pattern Recognition (ACPR), pp. 136–140 (2015) Kang, B., Tripathi, S. Nguyen, T.Q.: Real-time sign language fingerspelling recognition using convolutional neural networks from depth map. In: 3rd IAPR Asian Conference on Pattern Recognition (ACPR), pp. 136–140 (2015)
10.
Zurück zum Zitat Zhu, X., Liu, W., Jia, X., Wong, K.Y.K.: A two-stage detector for hand detection in ego-centric videos. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1–8 (2016) Zhu, X., Liu, W., Jia, X., Wong, K.Y.K.: A two-stage detector for hand detection in ego-centric videos. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1–8 (2016)
11.
Zurück zum Zitat Pigou, L., Dieleman, S., Kindermans, P.J., Schrauwen, B.: Sign language recognition using convolutional neural networks. In: Computer Vision—ECCV 2014 Workshops. Lecture Notes in Computer Science, vol. 8925, pp. 572–578 (2015) Pigou, L., Dieleman, S., Kindermans, P.J., Schrauwen, B.: Sign language recognition using convolutional neural networks. In: Computer Vision—ECCV 2014 Workshops. Lecture Notes in Computer Science, vol. 8925, pp. 572–578 (2015)
12.
Zurück zum Zitat Krizhevsky, A., Sutskever, I. and Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F. et al. (ed.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105. Curran Associates (2012) Krizhevsky, A., Sutskever, I. and Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F. et al. (ed.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105. Curran Associates (2012)
13.
Zurück zum Zitat Neris, M.N., Silva, A.J., Peres, S.M., Flores, F.C.: Self organizing maps and bit signature: a study applied on signal language recognition. In: IEEE International Joint Conference on Neural Networks, pp. 2934–2941 (2008) Neris, M.N., Silva, A.J., Peres, S.M., Flores, F.C.: Self organizing maps and bit signature: a study applied on signal language recognition. In: IEEE International Joint Conference on Neural Networks, pp. 2934–2941 (2008)
14.
Zurück zum Zitat Carneiro, T.S., Cortez, P.C., Costa, R.C.S.: Recognizing Libras gestures with neural classifiers, using Hu invariant moments. In: Interaction 09—South America, São Paulo, pp. 190–195 (2009) Carneiro, T.S., Cortez, P.C., Costa, R.C.S.: Recognizing Libras gestures with neural classifiers, using Hu invariant moments. In: Interaction 09—South America, São Paulo, pp. 190–195 (2009)
15.
Zurück zum Zitat Pizzolato, E.B., Anjo, M.S., Pedroso, G.C.: Automatic recognition of finger spelling for libras based on a two layer architecture. In: ACM Symposium on Applied Computing. Proceedings of the 25th Symposium on Applied Computing, pp. 970–974 (2010) Pizzolato, E.B., Anjo, M.S., Pedroso, G.C.: Automatic recognition of finger spelling for libras based on a two layer architecture. In: ACM Symposium on Applied Computing. Proceedings of the 25th Symposium on Applied Computing, pp. 970–974 (2010)
16.
Zurück zum Zitat Mehdi, S.A., Khan, Y.N.: Sign language recognition using sensor gloves. In: Proceedings of the 9th International Conference on IEEE Neural Information Processing, ICONIP’02, vol. 5, pp. 2204–2206 (2002) Mehdi, S.A., Khan, Y.N.: Sign language recognition using sensor gloves. In: Proceedings of the 9th International Conference on IEEE Neural Information Processing, ICONIP’02, vol. 5, pp. 2204–2206 (2002)
17.
Zurück zum Zitat Wang, H.: Nearest neighbors by neighborhood counting. IEEE Trans. Pattern Anal. Mach. Intell. 28(6), 942–953 (2006)CrossRef Wang, H.: Nearest neighbors by neighborhood counting. IEEE Trans. Pattern Anal. Mach. Intell. 28(6), 942–953 (2006)CrossRef
Metadaten
Titel
Recognizing Hand Configurations of Brazilian Sign Language Using Convolutional Neural Networks
verfasst von
A. S. Oliveria
C. F. F. Costa Filho
M. G. F. Costa
Copyright-Jahr
2019
Verlag
Springer Singapore
DOI
https://doi.org/10.1007/978-981-13-2517-5_64

Neuer Inhalt