Skip to main content
Top
Published in: Multimedia Systems 3/2019

28-09-2018 | Regular Paper

Accurate body-part reconstruction from a single depth image

Authors: Arefi Farnoosh, Nadian-Ghomsheh Ali

Published in: Multimedia Systems | Issue 3/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Human pose reconstruction using depth images has received much attention for human-centric applications. Body-part labeling at pixel-level has shown to be efficient for human pose reconstruction. This paper presents an accurate human pose reconstruction method from a single depth image by combining body-part labeling and nearest pose-matching techniques. New pixel-level depth difference and local curvature-encoding features are introduced to provide more contextual depth information for pixel-level body-part labeling. To reduce the misclassification error, inspired by pose-matching techniques, a corrective step is also proposed. The method extracts depth region proposals from a reference pose and finds the best match using PCT coefficients to correct uncertain labels. Tests on a set of synthetic and natural depth poses showed improved accuracy of body-part labeling compared to the state-of-the-art methods. In addition, in comparison with the previous methods and the Kinect camera, an improved accuracy for human range of motion measurement was obtained .

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Literature
1.
go back to reference Xia, S., Gao, L., Lai, Y.-K., Yuan, M.-Z., Chai, J.: A survey on human performance capture and animation. J. Comput. Sci. Technol. 32(3), 536–554 (2017)CrossRef Xia, S., Gao, L., Lai, Y.-K., Yuan, M.-Z., Chai, J.: A survey on human performance capture and animation. J. Comput. Sci. Technol. 32(3), 536–554 (2017)CrossRef
2.
go back to reference Mündermann, L., Corazza, S., Andriacchi, T.P.: The evolution of methods for the capture of human movement leading to markerless motion capture for biomechanical applications. J. NeuroEng. Rehabil. 3(1), 6 (2006)CrossRef Mündermann, L., Corazza, S., Andriacchi, T.P.: The evolution of methods for the capture of human movement leading to markerless motion capture for biomechanical applications. J. NeuroEng. Rehabil. 3(1), 6 (2006)CrossRef
3.
go back to reference Han, F., Reily, B., Hoff, W., Zhang, H.: Space-time representation of people based on 3D skeletal data: a review. Comput. Vis. Image Underst. 158, 85–105 (2017)CrossRef Han, F., Reily, B., Hoff, W., Zhang, H.: Space-time representation of people based on 3D skeletal data: a review. Comput. Vis. Image Underst. 158, 85–105 (2017)CrossRef
4.
go back to reference Presti, L.L., La Cascia, M.: 3D skeleton-based human action classification: a survey. Pattern Recognit. 53, 130–147 (2016)CrossRef Presti, L.L., La Cascia, M.: 3D skeleton-based human action classification: a survey. Pattern Recognit. 53, 130–147 (2016)CrossRef
5.
go back to reference Wang, Y., Liu, Y., Tong, X., Dai, Q., Tan, P.: Outdoor markerless motion capture with sparse handheld video cameras. IEEE Trans. Vis. Comput. Graph. PP(99), 1 (2017) Wang, Y., Liu, Y., Tong, X., Dai, Q., Tan, P.: Outdoor markerless motion capture with sparse handheld video cameras. IEEE Trans. Vis. Comput. Graph. PP(99), 1 (2017)
6.
go back to reference Li, J., Grandner, M.A., Chang, Y.-P., Jungquist, C., Porock, D.: Person-centered dementia care and sleep in assisted living residents with dementia: a pilot study. Behav. Sleep Med. 15(2), 97–113 (2017)CrossRef Li, J., Grandner, M.A., Chang, Y.-P., Jungquist, C., Porock, D.: Person-centered dementia care and sleep in assisted living residents with dementia: a pilot study. Behav. Sleep Med. 15(2), 97–113 (2017)CrossRef
7.
go back to reference Knippenberg, E., Verbrugghe, J., Lamers, I., Palmaers, S., Timmermans, A., Spooren, A.: Markerless motion capture systems as training device in neurological rehabilitation: a systematic review of their use, application, target population and efficacy. J. Neuroengi. Rehabil. 14(1), 61 (2017)CrossRef Knippenberg, E., Verbrugghe, J., Lamers, I., Palmaers, S., Timmermans, A., Spooren, A.: Markerless motion capture systems as training device in neurological rehabilitation: a systematic review of their use, application, target population and efficacy. J. Neuroengi. Rehabil. 14(1), 61 (2017)CrossRef
8.
go back to reference Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., Moore, R.: Real-time human pose recognition in parts from single depth images. Commun. ACM. 56(1), 116–124 (2013)CrossRef Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., Moore, R.: Real-time human pose recognition in parts from single depth images. Commun. ACM. 56(1), 116–124 (2013)CrossRef
10.
go back to reference Mousavi Hondori, H., Khademi, M.: A review on technical and clinical impact of microsoft kinect on physical therapy and rehabilitation. J. Med. Eng. (2014) Mousavi Hondori, H., Khademi, M.: A review on technical and clinical impact of microsoft kinect on physical therapy and rehabilitation. J. Med. Eng. (2014)
11.
go back to reference Zhou, H., Hu, H.: Human motion tracking for rehabilitation: a survey. Biomed. Signal Process. Control 3(1), 1–18 (2008)CrossRef Zhou, H., Hu, H.: Human motion tracking for rehabilitation: a survey. Biomed. Signal Process. Control 3(1), 1–18 (2008)CrossRef
12.
go back to reference Lee, S.H., Yoon, C., Chung, S.G., Kim, H.C., Kwak, Y., Park, H.: Kim K.: Measurement of shoulder range of motion in patients with adhesive capsulitis using a kinect. PloS ONE. 10 (6), e0129398 (2015) Lee, S.H., Yoon, C., Chung, S.G., Kim, H.C., Kwak, Y., Park, H.: Kim K.: Measurement of shoulder range of motion in patients with adhesive capsulitis using a kinect. PloS ONE. 10 (6), e0129398 (2015)
13.
go back to reference Pastor, I., Hayes, H.A., Bamberg, S.J.: A feasibility study of an upper limb rehabilitation system using kinect and computer games. In: Engineering in Medicine and Biology Society (EMBC), Annual International Conference of the IEEE, pp 1286–1289 (2012) Pastor, I., Hayes, H.A., Bamberg, S.J.: A feasibility study of an upper limb rehabilitation system using kinect and computer games. In: Engineering in Medicine and Biology Society (EMBC), Annual International Conference of the IEEE, pp 1286–1289 (2012)
14.
go back to reference Da Gama, A., Chaves, T., Figueiredo, L., Teichrieb, V. Guidance and movement correction based on therapeutics movements for motor rehabilitation support systems. In: Virtual and Augmented Reality (SVR), 14th Symposium on IEEE, pp 191–200 (2012) Da Gama, A., Chaves, T., Figueiredo, L., Teichrieb, V. Guidance and movement correction based on therapeutics movements for motor rehabilitation support systems. In: Virtual and Augmented Reality (SVR), 14th Symposium on IEEE, pp 191–200 (2012)
15.
go back to reference Wei, X., Zhang, P., Chai, J.: Accurate realtime full-body motion capture using a single depth camera. ACM Trans. Graph. (TOG). 31(6), 188 (2012)CrossRef Wei, X., Zhang, P., Chai, J.: Accurate realtime full-body motion capture using a single depth camera. ACM Trans. Graph. (TOG). 31(6), 188 (2012)CrossRef
16.
go back to reference Liang, H., Yuan, J., Thalmann, D.: Parsing the hand in depth images. IEEE Trans. Multimedi. 16(5), 1241–1253 (2014)CrossRef Liang, H., Yuan, J., Thalmann, D.: Parsing the hand in depth images. IEEE Trans. Multimedi. 16(5), 1241–1253 (2014)CrossRef
17.
go back to reference Gajdosik, R.L., Bohannon, R.W.: Clinical measurement of range of motion: review of goniometry emphasizing reliability and validity. Phys. Ther. 67(12), 1867–1872 (1987)CrossRef Gajdosik, R.L., Bohannon, R.W.: Clinical measurement of range of motion: review of goniometry emphasizing reliability and validity. Phys. Ther. 67(12), 1867–1872 (1987)CrossRef
18.
go back to reference Zulkarnain, R.F., Kim, G.-Y., Adikrishna, A., Hong, H.P., Kim, Y.J., Jeon, I.-H.: Digital data acquisition of shoulder range of motion and arm motion smoothness using Kinect v2. J. Shoulder Elb. Surg. 26(5), 895–901 (2017)CrossRef Zulkarnain, R.F., Kim, G.-Y., Adikrishna, A., Hong, H.P., Kim, Y.J., Jeon, I.-H.: Digital data acquisition of shoulder range of motion and arm motion smoothness using Kinect v2. J. Shoulder Elb. Surg. 26(5), 895–901 (2017)CrossRef
19.
go back to reference Manghisi, V.M., Uva, A.E., Fiorentino, M., Bevilacqua, V., Trotta, G.F., Monno, G.: Real time RULA assessment using Kinect v2 sensor. Appl. Ergon. 65, 481–491 (2017)CrossRef Manghisi, V.M., Uva, A.E., Fiorentino, M., Bevilacqua, V., Trotta, G.F., Monno, G.: Real time RULA assessment using Kinect v2 sensor. Appl. Ergon. 65, 481–491 (2017)CrossRef
20.
go back to reference Reither, L.R., Foreman, M.H., Migotsky, N., Haddix, C., Engsberg, J.R.: Upper extremity movement reliability and validity of the Kinect version 2. Disabil. Rehabil. Assist. Technol. 13(1), 54–59 (2018)CrossRef Reither, L.R., Foreman, M.H., Migotsky, N., Haddix, C., Engsberg, J.R.: Upper extremity movement reliability and validity of the Kinect version 2. Disabil. Rehabil. Assist. Technol. 13(1), 54–59 (2018)CrossRef
21.
go back to reference Bonnechere, B., Jansen, B., Salvia, P., Bouzahouene, H., Omelina, L., Moiseev, F., Sholukha, V., Cornelis, J., Rooze, M., Jan, S.V.S.: Validity and reliability of the Kinect within functional assessment activities: comparison with standard stereophotogrammetry. Gait Posture 39(1), 593–598 (2014)CrossRef Bonnechere, B., Jansen, B., Salvia, P., Bouzahouene, H., Omelina, L., Moiseev, F., Sholukha, V., Cornelis, J., Rooze, M., Jan, S.V.S.: Validity and reliability of the Kinect within functional assessment activities: comparison with standard stereophotogrammetry. Gait Posture 39(1), 593–598 (2014)CrossRef
22.
go back to reference Ye, M., Zhang, Q., Wang, L., Zhu, J., Yang, R., Gall, J.: A survey on human motion analysis from depth data. In: Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications. Springer, Berlin, Heidelberg, pp 149–187 (2013) Ye, M., Zhang, Q., Wang, L., Zhu, J., Yang, R., Gall, J.: A survey on human motion analysis from depth data. In: Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications. Springer, Berlin, Heidelberg, pp 149–187 (2013)
23.
go back to reference Zhang, Q., Song, X., Shao, X., Shibasaki, R., Zhao, H.: Unsupervised skeleton extraction and motion capture from 3D deformable matching. Neurocomputing 100, 170–182 (2013)CrossRef Zhang, Q., Song, X., Shao, X., Shibasaki, R., Zhao, H.: Unsupervised skeleton extraction and motion capture from 3D deformable matching. Neurocomputing 100, 170–182 (2013)CrossRef
24.
go back to reference Baak, A., Müller, M., Bharaj, G., Seidel, H.-P., Theobalt, C.: A data-driven approach for real-time full body pose reconstruction from a depth camera. In: IEEE International Conference on Computer Vision, Barcelona, Spain (2011) Baak, A., Müller, M., Bharaj, G., Seidel, H.-P., Theobalt, C.: A data-driven approach for real-time full body pose reconstruction from a depth camera. In: IEEE International Conference on Computer Vision, Barcelona, Spain (2011)
25.
go back to reference Taylor, J., Shotton, J., Sharp, T., Fitzgibbon, A.: The vitruvian manifold: inferring dense correspondences for one-shot human pose estimation. In: Computer Vision and Pattern Recognition (CVPR), IEEE Conference on, pp 103–110 (2012)b b Taylor, J., Shotton, J., Sharp, T., Fitzgibbon, A.: The vitruvian manifold: inferring dense correspondences for one-shot human pose estimation. In: Computer Vision and Pattern Recognition (CVPR), IEEE Conference on, pp 103–110 (2012)b b
26.
go back to reference Ganapathi, V., Plagemann, C., Koller, D., Thrun, S.: Real-time human pose tracking from range data. In: European Conference on Computer Vision, Springer, pp 738–751 (2012) Ganapathi, V., Plagemann, C., Koller, D., Thrun, S.: Real-time human pose tracking from range data. In: European Conference on Computer Vision, Springer, pp 738–751 (2012)
27.
go back to reference Ye, M., Wang, X., Yang, R., Ren, L., Pollefeys, M.: Accurate 3d pose estimation from a single depth image. In: IEEE International Conference on Computer Vision (ICCV), pp 731–738 (2011) Ye, M., Wang, X., Yang, R., Ren, L., Pollefeys, M.: Accurate 3d pose estimation from a single depth image. In: IEEE International Conference on Computer Vision (ICCV), pp 731–738 (2011)
28.
go back to reference Cheng, K.-L., Tong, R.-F., Tang, M., Qian, J.-Y., Sarkis, M.: Parametric human body reconstruction based on sparse key points. IEEE Trans. Vis. Comput. Graph. 22(11), 2467–2479 (2016)CrossRef Cheng, K.-L., Tong, R.-F., Tang, M., Qian, J.-Y., Sarkis, M.: Parametric human body reconstruction based on sparse key points. IEEE Trans. Vis. Comput. Graph. 22(11), 2467–2479 (2016)CrossRef
29.
go back to reference Shafaei, A., Little, J.J.: Real-time human motion capture with multiple depth cameras. In: 13th Conference on IEEE Computer and Robot Vision (CRV), pp 24–31 (2016) Shafaei, A., Little, J.J.: Real-time human motion capture with multiple depth cameras. In: 13th Conference on IEEE Computer and Robot Vision (CRV), pp 24–31 (2016)
30.
go back to reference Jiu, M., Wolf, C., Taylor, G., Baskurt, A.: Human body part estimation from depth images via spatially-constrained deep learning. Pattern Recognit. Lett. 50, 122–129 (2014)CrossRef Jiu, M., Wolf, C., Taylor, G., Baskurt, A.: Human body part estimation from depth images via spatially-constrained deep learning. Pattern Recognit. Lett. 50, 122–129 (2014)CrossRef
31.
go back to reference Sun, M., Kohli, P., Shotton, J.: Conditional regression forests for human pose estimation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3394–3401 (2012) Sun, M., Kohli, P., Shotton, J.: Conditional regression forests for human pose estimation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3394–3401 (2012)
32.
go back to reference Girshick, R., Shotton, J., Kohli, P., Criminisi, A., Fitzgibbon, A.: Efficient regression of general-activity human poses from depth images. In: IEEE International Conference on Computer Vision (ICCV), pp 415–422 (2011) Girshick, R., Shotton, J., Kohli, P., Criminisi, A., Fitzgibbon, A.: Efficient regression of general-activity human poses from depth images. In: IEEE International Conference on Computer Vision (ICCV), pp 415–422 (2011)
33.
go back to reference Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., sstrunk, S.: SLIC Superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)CrossRef Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., sstrunk, S.: SLIC Superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)CrossRef
35.
go back to reference Xia, S., Zhang, Z., Su, L.: Cascade of full-body pose regression from single depth image at 100 FPS. arXiv:1711.08126v2 (2018) Xia, S., Zhang, Z., Su, L.: Cascade of full-body pose regression from single depth image at 100 FPS. arXiv:1711.08126v2 (2018)
36.
go back to reference Yap, P.-T., Jiang, X., Kot, A.C.: Two-dimensional polar harmonic transforms for invariant image representation. IEEE Trans. Pattern Anal. Mach. Intell. 32(7), 1259–1270 (2010)CrossRef Yap, P.-T., Jiang, X., Kot, A.C.: Two-dimensional polar harmonic transforms for invariant image representation. IEEE Trans. Pattern Anal. Mach. Intell. 32(7), 1259–1270 (2010)CrossRef
38.
go back to reference Jiu, M., Wolf, C., Baskurt, A.: Integrating Spatial Layout of object parts into classification without pairwise terms-application to fast body parts estimation from depth images. In: VISAPP, pp 626–631 (2013) Jiu, M., Wolf, C., Baskurt, A.: Integrating Spatial Layout of object parts into classification without pairwise terms-application to fast body parts estimation from depth images. In: VISAPP, pp 626–631 (2013)
39.
go back to reference Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Scene parsing with multiscale feature learning, purity trees, and optimal covers. arXiv preprint arXiv 1202.2160 (2012) Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Scene parsing with multiscale feature learning, purity trees, and optimal covers. arXiv preprint arXiv 1202.2160 (2012)
Metadata
Title
Accurate body-part reconstruction from a single depth image
Authors
Arefi Farnoosh
Nadian-Ghomsheh Ali
Publication date
28-09-2018
Publisher
Springer Berlin Heidelberg
Published in
Multimedia Systems / Issue 3/2019
Print ISSN: 0942-4962
Electronic ISSN: 1432-1882
DOI
https://doi.org/10.1007/s00530-018-0594-9

Other articles of this Issue 3/2019

Multimedia Systems 3/2019 Go to the issue