Skip to main content

2018 | OriginalPaper | Buchkapitel

Human Activity Recognition by Fusion of RGB, Depth, and Skeletal Data

verfasst von : Pushpajit Khaire, Javed Imran, Praveen Kumar

Erschienen in: Proceedings of 2nd International Conference on Computer Vision & Image Processing

Verlag: Springer Singapore

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

A significant increase in research of human activity recognition can be seen in recent years due to availability of low-cost RGB-D sensors and advancement of deep learning algorithms. In this paper, we augmented our previous work on human activity recognition (Imran et al., IEEE international conference on advances in computing, communications, and informatics (ICACCI), 2016) [1] by incorporating skeletal data for fusion. Three main approaches are used to fuse skeletal data with RGB, depth data, and the results are compared with each other. A challenging UTD-MHAD activity recognition dataset with intraclass variations, comprising of twenty-seven activities, is used for testing and experimentation. Proposed fusion results in accuracy of 95.38% (nearly 4% improvement over previous work), and it also justifies the fact that recognition improves with an increase in number of evidences in support.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Imran, J., Kumar, P.: Human Action Recognition using RGB-D Sensor and Deep Convolutional Neural Networks. In: IEEE International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 144–148. Jaipur, India (2016) Imran, J., Kumar, P.: Human Action Recognition using RGB-D Sensor and Deep Convolutional Neural Networks. In: IEEE International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 144–148. Jaipur, India (2016)
2.
Zurück zum Zitat Chen, C., Jafari, R., Kehtarnavaz, N.: UTD-MHAD: A Multimodal Dataset for Human Action Recognition Utilizing a Depth Camera and a Wearable Inertial Sensor. In: IEEE International Conference on Image Processing (ICIP), pp. 168–172 (2015) Chen, C., Jafari, R., Kehtarnavaz, N.: UTD-MHAD: A Multimodal Dataset for Human Action Recognition Utilizing a Depth Camera and a Wearable Inertial Sensor. In: IEEE International Conference on Image Processing (ICIP), pp. 168–172 (2015)
3.
Zurück zum Zitat Li, W., Zhang, Z., and Liu, Z.: Action recognition based on a bag of 3D points. In: Proc. IEEE Conf. Comput. Vis. Pattern Recog. Workshops, San Francisco, CA, USA, pp. 9–14. Jun. (2010) Li, W., Zhang, Z., and Liu, Z.: Action recognition based on a bag of 3D points. In: Proc. IEEE Conf. Comput. Vis. Pattern Recog. Workshops, San Francisco, CA, USA, pp. 9–14. Jun. (2010)
4.
Zurück zum Zitat Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., and Bajcsy R.: Berkeley MHAD: A Comprehensive Multimodal Human Action Database. In: Proc. IEEE Workshop Appl. Comput. Vision, pp. 53–60. Jan. (2013) Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., and Bajcsy R.: Berkeley MHAD: A Comprehensive Multimodal Human Action Database. In: Proc. IEEE Workshop Appl. Comput. Vision, pp. 53–60. Jan. (2013)
5.
Zurück zum Zitat Sung, J., Ponce, C., Selman, B., Saxena, A.: Unstructured Human Activity Detection from RGBD Images. In: IEEE International Conference on Robotics and Automation RiverCentre, Saint Paul, Minnesota, USA, pp. 842–849, (2012) Sung, J., Ponce, C., Selman, B., Saxena, A.: Unstructured Human Activity Detection from RGBD Images. In: IEEE International Conference on Robotics and Automation RiverCentre, Saint Paul, Minnesota, USA, pp. 842–849, (2012)
6.
Zurück zum Zitat Wang, P., Li, W., Gao, Z., Zhang, J., Tang, C., and Ogunbona, P. O.: Action Recognition From Depth Maps Using Deep Convolutional Neural Networks, IEEE Transactions on Human-Machine Systems, Vol. 46, No. 4, pp. 498–509 August (2016) Wang, P., Li, W., Gao, Z., Zhang, J., Tang, C., and Ogunbona, P. O.: Action Recognition From Depth Maps Using Deep Convolutional Neural Networks, IEEE Transactions on Human-Machine Systems, Vol. 46, No. 4, pp. 498–509 August (2016)
7.
Zurück zum Zitat Chen, C., Liu, M., Zhang, B., Han, J., Jiang, J., Liu, H.: 3D Action Recognition Using Multi-temporal Depth Motion Maps and Fisher Vector, In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence, pp. 3331–3337 (2016) Chen, C., Liu, M., Zhang, B., Han, J., Jiang, J., Liu, H.: 3D Action Recognition Using Multi-temporal Depth Motion Maps and Fisher Vector, In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence, pp. 3331–3337 (2016)
8.
Zurück zum Zitat Xia, L., Chen, C. C., and Aggarwal, J. K.: View Invariant Human Action Recognition Using Histograms of 3D Joints. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 20–27, Providence, RI, (2012) Xia, L., Chen, C. C., and Aggarwal, J. K.: View Invariant Human Action Recognition Using Histograms of 3D Joints. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 20–27, Providence, RI, (2012)
9.
Zurück zum Zitat Gaglio, S., Lo Re, G., and Morana, M.: Human Activity Recognition Process Using 3-D Posture Data. IEEE Transactions on Human-Machine Systems, Vol. 45, No. 5, pp. 586–597 (2015) Gaglio, S., Lo Re, G., and Morana, M.: Human Activity Recognition Process Using 3-D Posture Data. IEEE Transactions on Human-Machine Systems, Vol. 45, No. 5, pp. 586–597 (2015)
10.
Zurück zum Zitat Cippitelli, E., Gasparrini, S., Gambi, E., and Spinsante, S.: A Human Activity Recognition System Using Skeleton Data from RGBD Sensors, Computational Intelligence and Neuroscience, Article ID 4351435, pp. 1–14, Volume 2016 (2016) Cippitelli, E., Gasparrini, S., Gambi, E., and Spinsante, S.: A Human Activity Recognition System Using Skeleton Data from RGBD Sensors, Computational Intelligence and Neuroscience, Article ID 4351435, pp. 1–14, Volume 2016 (2016)
11.
Zurück zum Zitat Farhad, M. B., Jiang, Y., and Ma, J.: DMMs- Based Multiple Features Fusion for Human Action Recognition. International Journal of Multimedia Data Engineering and Management (IJMDEM) Volume 6, Issue 4, pp. 23–39 (2015) Farhad, M. B., Jiang, Y., and Ma, J.: DMMs- Based Multiple Features Fusion for Human Action Recognition. International Journal of Multimedia Data Engineering and Management (IJMDEM) Volume 6, Issue 4, pp. 23–39 (2015)
12.
Zurück zum Zitat Aggarwal, J.K., Xia, L.: Human Activity Recognition from 3D Data: A Review. Pattern Recognition Letters, 48, pp. 70–80 (2014) Aggarwal, J.K., Xia, L.: Human Activity Recognition from 3D Data: A Review. Pattern Recognition Letters, 48, pp. 70–80 (2014)
Metadaten
Titel
Human Activity Recognition by Fusion of RGB, Depth, and Skeletal Data
verfasst von
Pushpajit Khaire
Javed Imran
Praveen Kumar
Copyright-Jahr
2018
Verlag
Springer Singapore
DOI
https://doi.org/10.1007/978-981-10-7895-8_32