nach oben

Wireless Personal Communications

Erschienen in:

13.09.2023

A Spatio-Temporal Framework for Dynamic Indian Sign Language Recognition

verfasst von: Sakshi Sharma, Sukhwinder Singh

Erschienen in: Wireless Personal Communications | Ausgabe 4/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

A sign language recognition system is a boon to the signer community as it eases the flow of information between the signer and non-signer communities. However, extracting timely detail from the video data is still a challenging task. In this paper, a deep learning based model consisting of trainable CNN and trainable stacked 2 bidirectional long short term memory (S2B-LSTM) has been proposed and tested to recognise the dynamic gestures of Indian sign language (ISL). The CNN architecture has been used as feature extractor to extract the spatial features from the input video data, whereas the temporal relation between the consecutive frames of input video is extracted using S2B-LSTM. This model has been trained and tested on self-developed dataset consisting of 360 videos of ISL dynamic gestures. The CNN-S2B-LSTM model outperforms the existing techniques of sign language recognition with best recognition accuracy of 97.6%.

Vorheriger Artikel Multi-view Feature Learning Based on Texture Description for Palm-Print Recognition

Nächster Artikel Development of an Enhanced Blockchain Mechanism for Internet of Things Authentication

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

AL-Rousan, M., Assaleh, K., & Tala'a, A. (2009).Video-based signer-independent Arabic sign language recognition using hidden Markov models. Appl. Soft Comput., 9(3), 990–999. https://doi.org/10.1016/j.asoc.2009.01.002.

Wadhawan, A., & Kumar, P. (2021). Sign language recognition systems: a decade systematic literature review. Archives of Computational Methods in Engineering, 28(3), 785–813. https://doi.org/10.1007/s11831-019-09384-2CrossRef

Moreira Almeida, S. G., Guimarães, F. G.& Arturo Ramírez, J. (2014). Feature extraction in Brazilian Sign Language Recognition based on phonological structure and using RGB-D sensors. Expert Systems with Applications, 41(16), 7259–7271. https://doi.org/10.1016/j.eswa.2014.05.024.

Sharma, S., & Singh, S. (2021). Recognition of indian sign language (ISL) using deep learning model. Wireless Personal Communications. https://doi.org/10.1007/s11277-021-09152-1CrossRef

Abhishek, K. S., Qubeley, L. C. F., & Ho, D. (2016). Glove-based hand gesture recognition sign language translator using capacitive touch sensor. In: 2016 IEEE International Conference on Electron Devices and Solid-State Circuits (EDSSC), Aug. 2016, pp. 334–337. https://doi.org/10.1109/EDSSC.2016.7785276.

Rautaray, S. S., & Agrawal, A. (2015). Vision based hand gesture recognition for human computer interaction: a survey. Artificial Intelligence Review, 43(1), 1–54. https://doi.org/10.1007/s10462-012-9356-9CrossRef

S. Sharma and S. Singh, 'Vision-based hand gesture recognition using deep learning for the interpretation of sign language', Expert Syst. Appl., vol. 182, p. 115657, Nov. 2021, doi: https://doi.org/10.1016/j.eswa.2021.115657.

Rekha, J., Bhattacharya, J., & Majumder, S. (2011). Shape, texture and local movement hand gesture features for Indian Sign Language recognition. In: 3rd International Conference on Trendz in Information Sciences Computing (TISC2011), Dec. 2011, pp. 30–35. https://doi.org/10.1109/TISC.2011.6169079.

Kishore, P. V. V., Prasad, M. V. D., Kumar, D. A., & Sastry, A. S. C. S. (2016). Optical flow hand tracking and active contour hand shape features for continuous sign language recognition with artificial neural networks. In: 2016 IEEE 6th International Conference on Advanced Computing (IACC), Feb. 2016, pp. 346–351. https://doi.org/10.1109/IACC.2016.71.

10.

Ahmed, W., Chanda, K., Mitra, S.( 2016). Vision based hand gesture recognition using dynamic time warping for indian sign language. In: 2016 International Conference on Information Science (ICIS), Aug. 2016, pp. 120–125. https://doi.org/10.1109/INFOSCI.2016.7845312.

11.

Naglot, D., & Kulkarni, M. (2016). ANN based Indian Sign Language numerals recognition using the leap motion controller. In: 2016 International Conference on Inventive Computation Technologies (ICICT), Aug. 2016, vol. 2, pp. 1–6. https://doi.org/10.1109/INVENTIVE.2016.7824830.

12.

Kumar, A., Thankachan, K., & Dominic, M. M. (Mar). Sign language recognition. In: 2016 3rd International Conference on Recent Advances in Information Technology (RAIT), Mar. 2016, pp. 422–428. https://doi.org/10.1109/RAIT.2016.7507939.

13.

Ibrahim, N. B., Selim, M. M., & Zayed, H. H. (2018). An Automatic arabic sign language recognition system (ArSLRS). Journal of King Saud University: Computer and Information Sciences, 30(4), 470–477, Oct. 2018. https://doi.org/10.1016/j.jksuci.2017.09.007.

14.

Kim, S. Y., Han, H. G., Kim, J. W., Lee, S., & Kim, T. W. (2017). A hand gesture recognition sensor using reflected impulses. IEEE Sensors Journal, 17(10), 2975–2976. https://doi.org/10.1109/JSEN.2017.2679220CrossRef

15.

Athira, P. K., Sruthi, C. J., & Lijiya, A. (2019). A signer independent sign language recognition with co-articulation elimination from live videos: An indian scenario. Journal of King Saud University: Computer and Information Sciences. https://doi.org/10.1016/j.jksuci.2019.05.002.

16.

P. M. Ferreira, J. S. Cardoso, and A. Rebelo, 'Multimodal Learning for Sign Language Recognition', in Pattern Recognition and Image Analysis, Cham, 2017, pp. 313–321. doi: https://doi.org/10.1007/978-3-319-58838-4_35.

17.

Darwish, S. M. (2017). Man-machine interaction system for subject independent sign language recognition. In: Proceedings of the 9th International Conference on Computer and Automation Engineering—ICCAE '17, Sydney, Australia, pp. 121–125. https://doi.org/10.1145/3057039.3057040.

18.

Kumar, P., Roy, P. P., & Dogra, D. P. (2018). Independent Bayesian classifier combination based sign language recognition using facial expression. Information Sciences, 428, 30–48. https://doi.org/10.1016/j.ins.2017.10.046MathSciNetCrossRef

19.

Vo, D.-H., Huynh, H.-H., Doan, P.-M., & Meunier, J. (2017). Dynamic gesture classification for vietnamese sign language recognition. International Journal of Advanced Computer Science and Applications, 8(3). https://doi.org/10.14569/IJACSA.2017.080357.

20.

Liao, Y., Xiong, P., Min, W., Min, W., & Lu, J. (2019). Dynamic sign language recognition based on video sequence with BLSTM-3D residual networks. IEEE Access, 7, 38044–38054. https://doi.org/10.1109/ACCESS.2019.2904749CrossRef

21.

Liu, Z., Chai, X., Liu, Z., & Chen, X. (2017). Continuous Gesture Recognition with Hand-Oriented Spatiotemporal Feature. In: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), Venice, Italy, Oct. 2017, pp. 3056–3064. https://doi.org/10.1109/ICCVW.2017.361.

22.

Hagemann, T., & Katsarou, K. (2020). A Systematic review on anomaly detection for cloud computing environments.In: 2020 3rd Artificial Intelligence and Cloud Computing Conference, New York, NY, USA, Dec. 2020, pp. 83–96. https://doi.org/10.1145/3442536.3442550.

23.

Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735CrossRef

24.

Funahashi, K., & Nakamura, Y. (1993). Approximation of dynamical systems by continuous time recurrent neural networks. Neural Networks, 6(6), 801–806.CrossRef

25.

Luo, S., Rao, Y., Chen, J., Wang, H., & Wang, Z. (2020). Short-term load forecasting model of distribution transformer based on CNN and LSTM. IEEE International Conference on High Voltage Engineering and Application (ICHVE), 2020, 1–4.

26.

Ullah, A., Ahmad, J., Muhammad, K., Sajjad, M., & Baik, S. W. (2018). Action recognition in video sequences using deep bi-directional LSTM With CNN features. IEEE Access, 6, 1155–1166. https://doi.org/10.1109/ACCESS.2017.2778011CrossRef

27.

Bhuyan, M. K., Ghosh, D., & Bora, P. K. (2005). Co-articulation Detection in Hand Gestures. In: TENCON 2005—2005 IEEE Region 10 Conference, Nov. 2005, pp. 1–4. https://doi.org/10.1109/TENCON.2005.300947.

Titel: A Spatio-Temporal Framework for Dynamic Indian Sign Language Recognition
verfasst von: Sakshi Sharma
Sukhwinder Singh
Publikationsdatum: 13.09.2023
Verlag: Springer US
Erschienen in: Wireless Personal Communications / Ausgabe 4/2023
Print ISSN: 0929-6212
Elektronische ISSN: 1572-834X
DOI: https://doi.org/10.1007/s11277-023-10730-8

Neuer Inhalt

Bildnachweise

VDI-Icon, Profil Icon, inhalt2, Springer Professional Modul/© Springer Fachmedien Wiesbaden GmbH, Internationaler Motorenkongress/© [M] ATZlive | Chisnikov / Fotolia.com, Search Icon, Banner Hanser, Benedikt Bonnmann von Adesso/© Adesso, Teilzeit/© Fokussiert / stock.adobe.com, Hans-Joachim Lefeld/© Lucht Probst Associates GmbH, Zeitschrift Wissensmanagement Cover, PatentFit-Logo/© Springer Fachmedien Wiesbaden GmbH, 2023_Antrieb/© supervisuell, ATZ-Webinar: Prototypenfreie Entwicklung durch Offline- und Driver-in-the-Loop-HiL-Tests /© (c) VI-grade, chassis.tech plus 2023/© [M] ATZlive / TÜV SÜD PRODUCT SERVICE GMBH

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 4/2023

Tamper Detection Technique for Text Images based on Vowels and Unicode Zero Length Characters

Review of Cosine Sum Window Functions and Recent Advances in FIR Filter Design Methods

An Energy-Efficient Trajectory Prediction for UAVs Using an Optimised 3D Improvised Protocol

EVACON-Rainsnow Computing: An Amalgamation of Cloud and Its Inherited Computing to Encourage End User for Both Localized and Globalized Remote Computing

Multi-view Feature Learning Based on Texture Description for Palm-Print Recognition

Comparative Analysis of Diversity Combining Schemes in Power Limited Cognitive Radio Under the Impact of Co-channel Interference

Neuer Inhalt

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.