Skip to main content
Erschienen in: Wireless Personal Communications 2/2021

07.05.2021

A Survey of Deep Learning Based Models for Human Activity Recognition

verfasst von: Nida Saddaf Khan, Muhammad Sayeed Ghani

Erschienen in: Wireless Personal Communications | Ausgabe 2/2021

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Human Activity Recognition (HAR) is a process of recognizing human activities automatically based on streaming data obtained from various sensors, such as, inertial sensors, physiological sensors, location sensors, camera, time and many more environmental sensors. HAR has proven to be beneficial in various fields of study especially in healthcare, aged-care, ambient living, personal care, social science, rehabilitation engineering and many other domains. Due to the recent advancements in computing power, deep learning-based algorithms have become most effective and efficient choice of algorithms for recognizing and solving HAR problems. In this survey, we categorize recent research work with respect to various factors and measures to investigate the recent trends in HAR using deep learning algorithms. The articles are analyzed in various aspects, such as those related to HAR, time series analysis, machine learning models, methods of dataset creation, and use of various other new trends such as transfer learning, active learning, etc.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
3.
Zurück zum Zitat Wang, J., Chen, Y., Hao, S., Peng, X., Hu, L. (2017). Deep learning for sensor-based activity recognition: a survey. Pattern Recognition Letter, Accessed: Dec. 07, 2018. [Online]. Available: /paper/Deep-Learning-for-Sensor-based-Activity-A-Survey-Wang-Chen/c2420b5f507015ba735f27f8709706cf01be1c2b. Wang, J., Chen, Y., Hao, S., Peng, X., Hu, L. (2017). Deep learning for sensor-based activity recognition: a survey. Pattern Recognition Letter, Accessed: Dec. 07, 2018. [Online]. Available: /paper/Deep-Learning-for-Sensor-based-Activity-A-Survey-Wang-Chen/c2420b5f507015ba735f27f8709706cf01be1c2b.
4.
Zurück zum Zitat Yang, J.B., Nguyen, M.N., San, P.P., Li, X.L., Krishnaswamy, S. (2015). Deep convolutional neural networks on multichannel time series for human activity recognition. Twenty-Fourth International Joint Conference on Artificial Intelligence, p. 7. Yang, J.B., Nguyen, M.N., San, P.P., Li, X.L., Krishnaswamy, S. (2015). Deep convolutional neural networks on multichannel time series for human activity recognition. Twenty-Fourth International Joint Conference on Artificial Intelligence, p. 7.
6.
Zurück zum Zitat Gani, M.O., Saha, A. K., Ahsan, G.M.T., Ahamed, S.I. (2018). A novel framework to recognize complex human activity, Accessed: Nov. 07, 2018. [Online]. Available: /paper/A-Novel-Framework-to-Recognize-Complex-Human-Gani-Saha/1024a5b27c233a452bbca412c322943ae02b9a7c. Gani, M.O., Saha, A. K., Ahsan, G.M.T., Ahamed, S.I. (2018). A novel framework to recognize complex human activity, Accessed: Nov. 07, 2018. [Online]. Available: /paper/A-Novel-Framework-to-Recognize-Complex-Human-Gani-Saha/1024a5b27c233a452bbca412c322943ae02b9a7c.
8.
Zurück zum Zitat Jeong, T., Klabjan, D., Starren, J. (2016). Predictive analytics using smartphone sensors for depressive episodes. ArXiv160307692 Cs Stat, Mar. 2016, Accessed: Mar. 11, 2019. [Online]. Available: http://arxiv.org/abs/1603.07692. Jeong, T., Klabjan, D., Starren, J. (2016). Predictive analytics using smartphone sensors for depressive episodes. ArXiv160307692 Cs Stat, Mar. 2016, Accessed: Mar. 11, 2019. [Online]. Available: http://​arxiv.​org/​abs/​1603.​07692.
10.
Zurück zum Zitat Yao, S., Hu, S., Zhao, Y., Zhang, A., Abdelzaher, T. (2016). DeepSense: A unified deep learning framework for time-series mobile sensing data processing. ArXiv161101942 Cs. Accessed: Nov. 07, 2018. [Online]. Available: http://arxiv.org/abs/1611.01942. Yao, S., Hu, S., Zhao, Y., Zhang, A., Abdelzaher, T. (2016). DeepSense: A unified deep learning framework for time-series mobile sensing data processing. ArXiv161101942 Cs. Accessed: Nov. 07, 2018. [Online]. Available: http://​arxiv.​org/​abs/​1611.​01942.
13.
Zurück zum Zitat Krizhevsky, A., Sutskever, I., Hinton, G.E. (2012). ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds. Curran Associates, Inc., pp. 1097–1105. Krizhevsky, A., Sutskever, I., Hinton, G.E. (2012). ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds. Curran Associates, Inc., pp. 1097–1105.
15.
Zurück zum Zitat Bao, L., Intille, S.S. (2004). Activity recognition from user-annotated acceleration data. Pervasive Computing, pp. 1–17. Bao, L., Intille, S.S. (2004). Activity recognition from user-annotated acceleration data. Pervasive Computing, pp. 1–17.
16.
Zurück zum Zitat Krause, A., Siewiorek, D. P., Smailagic, A., Farringdon, J. (2003). Unsupervised, dynamic identification of physiological and activity context in wearable computing. In Seventh IEEE International Symposium on Wearable Computers. Proceedings., Oct. 2003, pp. 88–97. Doi: https://doi.org/10.1109/ISWC.2003.1241398. Krause, A., Siewiorek, D. P., Smailagic, A., Farringdon, J. (2003). Unsupervised, dynamic identification of physiological and activity context in wearable computing. In Seventh IEEE International Symposium on Wearable Computers. Proceedings., Oct. 2003, pp. 88–97. Doi: https://​doi.​org/​10.​1109/​ISWC.​2003.​1241398.
20.
Zurück zum Zitat Alrazzak, U., Alhalabi, B. (2019). A survey on human activity recognition using accelerometer sensor. In Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), Spokane, WA, USA, pp. 152–159. Doi: https://doi.org/10.1109/ICIEV.2019.8858578. Alrazzak, U., Alhalabi, B. (2019). A survey on human activity recognition using accelerometer sensor. In Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), Spokane, WA, USA, pp. 152–159. Doi: https://​doi.​org/​10.​1109/​ICIEV.​2019.​8858578.
23.
Zurück zum Zitat Sakr, N.A., Abu-Elkheir, M., Atwan, A., Soliman, H.H. (2018). Current trends in complex human activity recognition. 14, 20. Sakr, N.A., Abu-Elkheir, M., Atwan, A., Soliman, H.H. (2018). Current trends in complex human activity recognition. 14, 20.
24.
Zurück zum Zitat Mobark, M., & Chuprat, S. (2018). Recognition of complex human activity using mobile phones: a systematic literature review. Journal of Theoretical and Applied Information Technology, 96(12), 24 Mobark, M., & Chuprat, S. (2018). Recognition of complex human activity using mobile phones: a systematic literature review. Journal of Theoretical and Applied Information Technology, 96(12), 24
28.
Zurück zum Zitat R. Elbasiony and W. Gomaa, (2020). A Survey on Human Activity Recognition Based on Temporal Signals of Portable Inertial Sensors. The International Conference on Advanced Machine Learning Technologies and Applications (AMLTA2019), Cham, vol. 921, pp. 734–745. Doi: https://doi.org/10.1007/978-3-030-14118-9_72. R. Elbasiony and W. Gomaa, (2020). A Survey on Human Activity Recognition Based on Temporal Signals of Portable Inertial Sensors. The International Conference on Advanced Machine Learning Technologies and Applications (AMLTA2019), Cham, vol. 921, pp. 734–745. Doi: https://​doi.​org/​10.​1007/​978-3-030-14118-9_​72.
30.
Zurück zum Zitat Stisen, A. et al. (2015). Smart devices are different: assessing and mitigatingmobile sensing heterogeneities for activity recognition. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems - SenSys ’15, Seoul, South Korea, pp. 127–140. Doi: https://doi.org/10.1145/2809695.2809718. Stisen, A. et al. (2015). Smart devices are different: assessing and mitigatingmobile sensing heterogeneities for activity recognition. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems - SenSys ’15, Seoul, South Korea, pp. 127–140. Doi: https://​doi.​org/​10.​1145/​2809695.​2809718.
31.
Zurück zum Zitat Zappi ,P. et al. (2008). Activity recognition from on-body sensors: accuracy-power trade-off by dynamic sensor selection. Wireless Sensor Networks, pp. 17–33. Zappi ,P. et al. (2008). Activity recognition from on-body sensors: accuracy-power trade-off by dynamic sensor selection. Wireless Sensor Networks, pp. 17–33.
35.
Zurück zum Zitat Bächlin, M., et al. (2010). Wearable assistant for parkinson’s disease patients with the freezing of gait symptom. Information Technology Biomedical IEEE Transactions On, 14, 436–446CrossRef Bächlin, M., et al. (2010). Wearable assistant for parkinson’s disease patients with the freezing of gait symptom. Information Technology Biomedical IEEE Transactions On, 14, 436–446CrossRef
40.
Zurück zum Zitat van Kasteren, T.L.M., Englebienne, G., Kröse, B.J.A. (2011). Human activity recognition from wireless sensor network data: benchmark and software. Activity Recognition in Pervasive Intelligent Environments, L. Chen, C. D. Nugent, J. Biswas, and J. Hoey, Eds. Paris: Atlantis Press, pp. 165–186. van Kasteren, T.L.M., Englebienne, G., Kröse, B.J.A. (2011). Human activity recognition from wireless sensor network data: benchmark and software. Activity Recognition in Pervasive Intelligent Environments, L. Chen, C. D. Nugent, J. Biswas, and J. Hoey, Eds. Paris: Atlantis Press, pp. 165–186.
42.
Zurück zum Zitat Zhang, Y., Zhang, Y., Zhang, Z., Song, Y. Human activity recognition based on time series analysis using U-Net. ArXiv preprint arXiv:1809.08113, p. 21. Zhang, Y., Zhang, Y., Zhang, Z., Song, Y. Human activity recognition based on time series analysis using U-Net. ArXiv preprint arXiv:1809.08113, p. 21.
44.
Zurück zum Zitat T. Sztyler and H. Stuckenschmidt, (2016). On-body localization of wearable devices: An investigation of position-aware activity recognition. in 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom), Sydney, Australia, pp. 1–9. Doi: https://doi.org/10.1109/PERCOM.2016.7456521. T. Sztyler and H. Stuckenschmidt, (2016). On-body localization of wearable devices: An investigation of position-aware activity recognition. in 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom), Sydney, Australia, pp. 1–9. Doi: https://​doi.​org/​10.​1109/​PERCOM.​2016.​7456521.
46.
Zurück zum Zitat Grzeszick, R., Lenk, J. M., Rueda, F. M., Fink, G. A., Feldhorst, S., ten Hompel, M. (2017). Deep neural network based human activity recognition for the order picking process. In Proceedings of the 4th international Workshop on Sensor-based Activity Recognition and Interaction - iWOAR ’17, Rostock, Germany, pp. 1–6. Doi: https://doi.org/10.1145/3134230.3134231. Grzeszick, R., Lenk, J. M., Rueda, F. M., Fink, G. A., Feldhorst, S., ten Hompel, M. (2017). Deep neural network based human activity recognition for the order picking process. In Proceedings of the 4th international Workshop on Sensor-based Activity Recognition and Interaction - iWOAR ’17, Rostock, Germany, pp. 1–6. Doi: https://​doi.​org/​10.​1145/​3134230.​3134231.
47.
Zurück zum Zitat Zhang, M., Sawchuk, A. A. (2012). USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing - UbiComp ’12, Pittsburgh, Pennsylvania, p. 1036. Doi: https://doi.org/10.1145/2370216.2370438. Zhang, M., Sawchuk, A. A. (2012). USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing - UbiComp ’12, Pittsburgh, Pennsylvania, p. 1036. Doi: https://​doi.​org/​10.​1145/​2370216.​2370438.
55.
Zurück zum Zitat Sample, A., Ranasinghe, D., Shi, Q. (2013). Sensor enabled wearable rfid technology for mitigating the risk of falls near beds. Sample, A., Ranasinghe, D., Shi, Q. (2013). Sensor enabled wearable rfid technology for mitigating the risk of falls near beds.
58.
Zurück zum Zitat Micucci, D., Mobilio, M., Napoletano, P. (2018). UniMiB SHAR: a new dataset for human activity recognition using acceleration data from smartphones. ArXiv161107688 Cs, Accessed: Oct. 30, 2018. [Online]. Available: http://arxiv.org/abs/1611.07688. Micucci, D., Mobilio, M., Napoletano, P. (2018). UniMiB SHAR: a new dataset for human activity recognition using acceleration data from smartphones. ArXiv161107688 Cs, Accessed: Oct. 30, 2018. [Online]. Available: http://​arxiv.​org/​abs/​1611.​07688.
61.
Zurück zum Zitat Ravi, D., Wong, C., Lo, B., Yang, G.-Z. (2016). Deep learning for human activity recognition: A resource efficient implementation on low-power devices. In 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN), pp. 71–76. Doi: https://doi.org/10.1109/BSN.2016.7516235. Ravi, D., Wong, C., Lo, B., Yang, G.-Z. (2016). Deep learning for human activity recognition: A resource efficient implementation on low-power devices. In 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN), pp. 71–76. Doi: https://​doi.​org/​10.​1109/​BSN.​2016.​7516235.
69.
Zurück zum Zitat Walse, K. H., Dharaskar, R. V., & Thakare, V. M. (2016). PCA based optimal ANN classifiers for human activity recognition using mobile sensors data. Proceedings of First International Conference on Information and Communication Technology for Intelligent Systems:, 1, 429–436 Walse, K. H., Dharaskar, R. V., & Thakare, V. M. (2016). PCA based optimal ANN classifiers for human activity recognition using mobile sensors data. Proceedings of First International Conference on Information and Communication Technology for Intelligent Systems:, 1, 429–436
78.
79.
81.
Zurück zum Zitat $$$F. Li, K. Shirahama, M. A. Nisar, L. Köping, and M. Grzegorzek, (2018). Comparison of feature learning methods for human activity recognition using wearable sensors. Sensors, vol. 18, no. 2, Art. no. 2, Feb. 2018. Doi: https://doi.org/10.3390/s18020679. $$$F. Li, K. Shirahama, M. A. Nisar, L. Köping, and M. Grzegorzek, (2018). Comparison of feature learning methods for human activity recognition using wearable sensors. Sensors, vol. 18, no. 2, Art. no. 2, Feb. 2018. Doi: https://​doi.​org/​10.​3390/​s18020679.
87.
Zurück zum Zitat Abedin, A., Rezatofighi, S. H., Shi, Q., Ranasinghe, D. C. (2019). SparseSense: Human activity recognition from highly sparse sensor data-streams using set-based neural networks. ArXiv190602399 Cs Stat Accessed: Apr. 06, 2020. [Online]. Available: http://arxiv.org/abs/1906.02399. Abedin, A., Rezatofighi, S. H., Shi, Q., Ranasinghe, D. C. (2019). SparseSense: Human activity recognition from highly sparse sensor data-streams using set-based neural networks. ArXiv190602399 Cs Stat Accessed: Apr. 06, 2020. [Online]. Available: http://​arxiv.​org/​abs/​1906.​02399.
90.
Zurück zum Zitat Russell, R., & Norvig, P. (1999). Artificial Intelligence: A Modern Approach. (3rd ed.). One Lake Street Upper Saddle River, NJUnited States: Prentice Hall Press.MATH Russell, R., & Norvig, P. (1999). Artificial Intelligence: A Modern Approach. (3rd ed.). One Lake Street Upper Saddle River, NJUnited States: Prentice Hall Press.MATH
91.
Zurück zum Zitat Rokni, S.-A., Nourollahi, M., Ghasemzadeh, H. Personalized human activity recognition using convolutional neural networks, p. 2. Rokni, S.-A., Nourollahi, M., Ghasemzadeh, H. Personalized human activity recognition using convolutional neural networks, p. 2.
92.
Zurück zum Zitat Hammerla, N. Y., Halloran, S., Plotz, T. (2020). Deep, convolutional, and recurrent models for human activity recognition using wearables. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence Accessed: Apr. 03, 2020. [Online]. Available: https://dl.acm.org/Doi/https://doi.org/10.5555/3060832.3060835. Hammerla, N. Y., Halloran, S., Plotz, T. (2020). Deep, convolutional, and recurrent models for human activity recognition using wearables. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence Accessed: Apr. 03, 2020. [Online]. Available: https://​dl.​acm.​org/​Doi/​https://​doi.​org/​10.​5555/​3060832.​3060835.
93.
Zurück zum Zitat San, P.P., Kakar, P., Li, X.-L., Krishnaswamy, S., Yang, J.-B., Nguyen, M. N. (2017). Chapter 9—Deep learning for human activity recognition. Big Data Analytics for Sensor-Network Collected Intelligence, H.-H. Hsu, C.-Y. Chang, and C.-H. Hsu, Eds. Academic Press, pp. 186–204. San, P.P., Kakar, P., Li, X.-L., Krishnaswamy, S., Yang, J.-B., Nguyen, M. N. (2017). Chapter 9—Deep learning for human activity recognition. Big Data Analytics for Sensor-Network Collected Intelligence, H.-H. Hsu, C.-Y. Chang, and C.-H. Hsu, Eds. Academic Press, pp. 186–204.
96.
Zurück zum Zitat Zeng, M., Gao, H., Yu, T., Mengshoel, O.J., Langseth, H., Lane, I. and Liu, X., (2018). Understanding and improving recurrent networks for human activity recognition by continuous attention. In Proceedings of the 2018 ACM International Symposium on Wearable Computers. https://dl.acm.org/Doi/https://doi.org/10.1145/3267242.3267286. Zeng, M., Gao, H., Yu, T., Mengshoel, O.J., Langseth, H., Lane, I. and Liu, X., (2018). Understanding and improving recurrent networks for human activity recognition by continuous attention. In Proceedings of the 2018 ACM International Symposium on Wearable Computers. https://​dl.​acm.​org/​Doi/​https://​doi.​org/​10.​1145/​3267242.​3267286.
97.
Zurück zum Zitat Chen, C., Li, K., Duan, M., Li, K. (2017). Chapter 6—extreme learning machine and its applications in big data processing. Big Data Analytics for Sensor-Network Collected Intelligence, H.-H. Hsu, C.-Y. Chang, and C.-H. Hsu, Eds. Academic Press, pp. 117–150. Chen, C., Li, K., Duan, M., Li, K. (2017). Chapter 6—extreme learning machine and its applications in big data processing. Big Data Analytics for Sensor-Network Collected Intelligence, H.-H. Hsu, C.-Y. Chang, and C.-H. Hsu, Eds. Academic Press, pp. 117–150.
100.
Zurück zum Zitat Cao, H., Nguyen, M.N., Phua, C., Krishnaswamy, S., Li, X.-L. (2012). An integrated framework for human activity classification. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, New York, NY, USA, pp. 331–340. Doi: https://doi.org/10.1145/2370216.2370268. Cao, H., Nguyen, M.N., Phua, C., Krishnaswamy, S., Li, X.-L. (2012). An integrated framework for human activity classification. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, New York, NY, USA, pp. 331–340. Doi: https://​doi.​org/​10.​1145/​2370216.​2370268.
102.
Zurück zum Zitat Kumar, R.C., Bharadwaj, S.S., Sumukha, George, K. (2016). Human activity recognition in cognitive environments using sequential ELM. In 2016 Second International Conference on Cognitive Computing and Information Processing (CCIP) (pp. 1–6). Doi: https://doi.org/10.1109/CCIP.2016.7802880. Kumar, R.C., Bharadwaj, S.S., Sumukha, George, K. (2016). Human activity recognition in cognitive environments using sequential ELM. In 2016 Second International Conference on Cognitive Computing and Information Processing (CCIP) (pp. 1–6). Doi: https://​doi.​org/​10.​1109/​CCIP.​2016.​7802880.
103.
Zurück zum Zitat Alsheikh, M. A., Selim, A., Niyato, D., Doyle, L., Lin, S., & Tan, H.-P. (2016). Deep Activity Recognition Models with Triaxial Accelerometers. (p. 6). AAAI Workshop Artif: Intell. Appl. Assist. Technol. Smart Environ. Alsheikh, M. A., Selim, A., Niyato, D., Doyle, L., Lin, S., & Tan, H.-P. (2016). Deep Activity Recognition Models with Triaxial Accelerometers. (p. 6). AAAI Workshop Artif: Intell. Appl. Assist. Technol. Smart Environ.
Metadaten
Titel
A Survey of Deep Learning Based Models for Human Activity Recognition
verfasst von
Nida Saddaf Khan
Muhammad Sayeed Ghani
Publikationsdatum
07.05.2021
Verlag
Springer US
Erschienen in
Wireless Personal Communications / Ausgabe 2/2021
Print ISSN: 0929-6212
Elektronische ISSN: 1572-834X
DOI
https://doi.org/10.1007/s11277-021-08525-w

Weitere Artikel der Ausgabe 2/2021

Wireless Personal Communications 2/2021 Zur Ausgabe

Neuer Inhalt