Skip to main content

2024 | OriginalPaper | Buchkapitel

Vision-Based Human Activity Recognition Using CNN and LSTM Architecture

verfasst von : Neha Gupta, Payal Malik, Arun Kumar Dubey, Achin Jain, Sarita Yadav, Devansh Verma

Erschienen in: Advanced Computing

Verlag: Springer Nature Switzerland

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Technology’s growing use has facilitated the quality of living. Artificial Intelligence (AI) is the field that aims to define how human intelligence is mimicked by machines which are programmed to think or behave like humans. Modern approaches and tools for evaluating human behavior have been made possible by modern advancements in the fields of machine learning (ML) and artificial intelligence (AI). Due to its applicability in several industries, comprising of entertainment, security and surveillance, health, and intelligent environments, human activity recognition has gained prominence significantly. Human activity recognition (HAR) using video sensors typically involves analyzing the visual data captured by cameras to classify and identify the actions of individuals. In the following paper, we propose ConvLSTM and LRCN-based Human Action Recognition. A huge variety of films from the publicly accessible data set, UCF50 comprises a wide range of activity classes that are used to build a statistical model. For the model proposed in this paper, the accuracy has turned out to be 94%, the average f1-score is 0.93 and the average recall is calculated to be 0.925. The Loss curve has also been plotted along with the accuracy curve for the proposed model for recognizing human activities.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Abbaspour, S., Fotouhi, F., Sedaghatbaf, A., Fotouhi, H., Vahabi, M., Linden, M.: A comparative analysis of hybrid deep learning models for human activity recognition. Sensors 20, 5707 (2020)CrossRef Abbaspour, S., Fotouhi, F., Sedaghatbaf, A., Fotouhi, H., Vahabi, M., Linden, M.: A comparative analysis of hybrid deep learning models for human activity recognition. Sensors 20, 5707 (2020)CrossRef
2.
Zurück zum Zitat Ke, S.R., Thuc, H.L.U., Lee, Y.J., Hwang, J.N., Yoo, J.H., Choi, K.H.: A review on video-based human activity recognition. Computers 2(2), 88–131 (2013)CrossRef Ke, S.R., Thuc, H.L.U., Lee, Y.J., Hwang, J.N., Yoo, J.H., Choi, K.H.: A review on video-based human activity recognition. Computers 2(2), 88–131 (2013)CrossRef
3.
Zurück zum Zitat Singh, J., Sinha, S.: Video Based Human Activity Recognition Surveillance System Singh, J., Sinha, S.: Video Based Human Activity Recognition Surveillance System
4.
Zurück zum Zitat Hussain, Z., Sheng, Q.Z., Zhang, W.E.: A review and categorization of techniques on device-free human activity recognition. J. Netw. Comput. Appl. 167, 102738 (2020)CrossRef Hussain, Z., Sheng, Q.Z., Zhang, W.E.: A review and categorization of techniques on device-free human activity recognition. J. Netw. Comput. Appl. 167, 102738 (2020)CrossRef
5.
Zurück zum Zitat Gowda, S.N., Rohrbach, M., Sevilla-Lara, L.: Smart frame selection for action recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 2, pp. 1451–1459 (2021) Gowda, S.N., Rohrbach, M., Sevilla-Lara, L.: Smart frame selection for action recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 2, pp. 1451–1459 (2021)
6.
Zurück zum Zitat Mishra, S.R., Mishra, T.K., Sanyal, G., Sarkar, A., Satapathy, S.C.: Real time human action recognition using triggered frame extraction and a typical CNN heuristic. Pattern Recogn. Lett. 135, 329–336 (2020)CrossRef Mishra, S.R., Mishra, T.K., Sanyal, G., Sarkar, A., Satapathy, S.C.: Real time human action recognition using triggered frame extraction and a typical CNN heuristic. Pattern Recogn. Lett. 135, 329–336 (2020)CrossRef
7.
Zurück zum Zitat Yu, Z., Yan, W.Q.: Human action recognition using deep learning methods. In: 2020 35th International Conference on Image and Vision Computing New Zealand (IVCNZ), pp. 1–6. IEEE (2020) Yu, Z., Yan, W.Q.: Human action recognition using deep learning methods. In: 2020 35th International Conference on Image and Vision Computing New Zealand (IVCNZ), pp. 1–6. IEEE (2020)
8.
Zurück zum Zitat Deep, S., Zheng, X.: Hybrid model featuring CNN and LSTM architecture for human activity recognition on smartphone sensor data. In: 2019 20th International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT), pp. 259–264. IEEE (2019) Deep, S., Zheng, X.: Hybrid model featuring CNN and LSTM architecture for human activity recognition on smartphone sensor data. In: 2019 20th International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT), pp. 259–264. IEEE (2019)
9.
Zurück zum Zitat Mutegeki, R., Han, D.S.: A CNN-LSTM approach to human activity recognition. In: 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), pp. 362–366. IEEE (2020) Mutegeki, R., Han, D.S.: A CNN-LSTM approach to human activity recognition. In: 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), pp. 362–366. IEEE (2020)
10.
Zurück zum Zitat Semwal, V.B., Gupta, A., Lalwani, P.: An optimized hybrid deep learning model using ensemble learning approach for human walking activities recognition. J. Supercomput. 77(11), 1225612279 (2021)CrossRef Semwal, V.B., Gupta, A., Lalwani, P.: An optimized hybrid deep learning model using ensemble learning approach for human walking activities recognition. J. Supercomput. 77(11), 1225612279 (2021)CrossRef
11.
Zurück zum Zitat Pareek, P., Thakkar, A.: A survey on video-based human action recognition: recent updates, datasets, challenges, and applications. Artif. Intell. Rev. 54(3), 2259–2322 (2021)CrossRef Pareek, P., Thakkar, A.: A survey on video-based human action recognition: recent updates, datasets, challenges, and applications. Artif. Intell. Rev. 54(3), 2259–2322 (2021)CrossRef
12.
Zurück zum Zitat Hu, K., Zheng, F., Weng, L., Ding, Y., Jin, J.: Action recognition algorithm of spatio-temporal differential LSTM based on feature enhancement. Appl. Sci. 11(17), 7876 (2021)CrossRef Hu, K., Zheng, F., Weng, L., Ding, Y., Jin, J.: Action recognition algorithm of spatio-temporal differential LSTM based on feature enhancement. Appl. Sci. 11(17), 7876 (2021)CrossRef
13.
Zurück zum Zitat Donahue, J., et al.: Long-term recurrent convolutional networks for visual recognition and description. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2625–2634 (2015) Donahue, J., et al.: Long-term recurrent convolutional networks for visual recognition and description. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2625–2634 (2015)
14.
Zurück zum Zitat Sansano, E., Montoliu, R., Belmonte Fernandez, O.: A study of deep neural networks for human activity recognition. Comput. Intell. 36(3), 1113–1139 (2020)MathSciNetCrossRef Sansano, E., Montoliu, R., Belmonte Fernandez, O.: A study of deep neural networks for human activity recognition. Comput. Intell. 36(3), 1113–1139 (2020)MathSciNetCrossRef
15.
Zurück zum Zitat Tasnim, N., Islam, M.K., Baek, J.H.: Deep learning based human activity recognition using spatio-temporal image formation of skeleton joints. Appl. Sci. 11(6), 2675 (2021)CrossRef Tasnim, N., Islam, M.K., Baek, J.H.: Deep learning based human activity recognition using spatio-temporal image formation of skeleton joints. Appl. Sci. 11(6), 2675 (2021)CrossRef
16.
Zurück zum Zitat Hernández, F., Suárez, L.F., Villamizar, J., Altuve, M.: Human activity recognition on smartphones using a bidirectional LSTM network. In: 2019 XXII Symposium on Image, Signal Processing and Artificial Vision (STSIVA), pp. 1–5. IEEE (2019) Hernández, F., Suárez, L.F., Villamizar, J., Altuve, M.: Human activity recognition on smartphones using a bidirectional LSTM network. In: 2019 XXII Symposium on Image, Signal Processing and Artificial Vision (STSIVA), pp. 1–5. IEEE (2019)
17.
Zurück zum Zitat Kushwaha, A., Khare, A., Srivastava, P.: On integration of multiple features for human activity recognition in video sequences. Multimed. Tools Appl. 80(21), 32511–32538 (2021)CrossRef Kushwaha, A., Khare, A., Srivastava, P.: On integration of multiple features for human activity recognition in video sequences. Multimed. Tools Appl. 80(21), 32511–32538 (2021)CrossRef
18.
Zurück zum Zitat Murad, A., Pyun, J.Y.: Deep recurrent neural networks for human activity recognition. Sensors 17(11), 2556 (2017)CrossRef Murad, A., Pyun, J.Y.: Deep recurrent neural networks for human activity recognition. Sensors 17(11), 2556 (2017)CrossRef
19.
Zurück zum Zitat Peng, L., Chen, L., Ye, Z., Zhang, Y.: Aroma: a deep multi-task learning based simple and complex human activity recognition method using wearable sensors. Proc. ACM Interact. Mob. Wearable Ubiquit. Technol. 2(2), 1–16 (2018)CrossRef Peng, L., Chen, L., Ye, Z., Zhang, Y.: Aroma: a deep multi-task learning based simple and complex human activity recognition method using wearable sensors. Proc. ACM Interact. Mob. Wearable Ubiquit. Technol. 2(2), 1–16 (2018)CrossRef
20.
Zurück zum Zitat Almaslukh, B., Artoli, A.M., Al-Muhtadi, J.: A robust deep learning approach for position-independent smartphone-based human activity recognition. Sensors 18(11), 3726 (2018)CrossRef Almaslukh, B., Artoli, A.M., Al-Muhtadi, J.: A robust deep learning approach for position-independent smartphone-based human activity recognition. Sensors 18(11), 3726 (2018)CrossRef
21.
Zurück zum Zitat Wan, S., Qi, L., Xu, X., Tong, C., Gu, Z.: Deep learning models for real-time human activity recognition with smartphones. Mob. Netw. Appl. 25(2), 743–755 (2020)CrossRef Wan, S., Qi, L., Xu, X., Tong, C., Gu, Z.: Deep learning models for real-time human activity recognition with smartphones. Mob. Netw. Appl. 25(2), 743–755 (2020)CrossRef
22.
Zurück zum Zitat Mekruksavanich, S., Jitpattanakul, A.: Biometric user identification based on human activity recognition using wearable sensors: an experiment using deep learning models. Electronics 10(3), 308 (2021)CrossRef Mekruksavanich, S., Jitpattanakul, A.: Biometric user identification based on human activity recognition using wearable sensors: an experiment using deep learning models. Electronics 10(3), 308 (2021)CrossRef
23.
Zurück zum Zitat Agarwal, P., Alam, M.: A lightweight deep learning model for human activity recognition on edge devices. Procedia Comput. Sci. 167, 2364–2373 (2020)CrossRef Agarwal, P., Alam, M.: A lightweight deep learning model for human activity recognition on edge devices. Procedia Comput. Sci. 167, 2364–2373 (2020)CrossRef
25.
Zurück zum Zitat Bokhari, S.M., Sohaib, S., Khan, A.R., Shafi, M.: DGRU based human activity recognition using channel state information. Measurement 167, 108245 (2021)CrossRef Bokhari, S.M., Sohaib, S., Khan, A.R., Shafi, M.: DGRU based human activity recognition using channel state information. Measurement 167, 108245 (2021)CrossRef
26.
Zurück zum Zitat Paramasivam, K., Sindha, M.M.R., Balakrishnan, S.B.: KNN-based machine learning classifier used on deep learned spatial motion features for human action recognition. Entropy 25(6), 844 (2023)CrossRef Paramasivam, K., Sindha, M.M.R., Balakrishnan, S.B.: KNN-based machine learning classifier used on deep learned spatial motion features for human action recognition. Entropy 25(6), 844 (2023)CrossRef
27.
Zurück zum Zitat Khan, Y.A., Imaduddin, S., Singh, Y.P., Wajid, M., Usman, M., Abbas, M.: Artificial intelligence based approach for classification of human activities using MEMS sensors data. Sensors 23(3), 1275 (2023)CrossRef Khan, Y.A., Imaduddin, S., Singh, Y.P., Wajid, M., Usman, M., Abbas, M.: Artificial intelligence based approach for classification of human activities using MEMS sensors data. Sensors 23(3), 1275 (2023)CrossRef
Metadaten
Titel
Vision-Based Human Activity Recognition Using CNN and LSTM Architecture
verfasst von
Neha Gupta
Payal Malik
Arun Kumar Dubey
Achin Jain
Sarita Yadav
Devansh Verma
Copyright-Jahr
2024
DOI
https://doi.org/10.1007/978-3-031-56700-1_10

Premium Partner