Skip to main content
Erschienen in: International Journal of Computer Vision 3/2017

23.02.2017

Lie-X: Depth Image Based Articulated Object Pose Estimation, Tracking, and Action Recognition on Lie Groups

verfasst von: Chi Xu, Lakshmi Narasimhan Govindarajan, Yu Zhang, Li Cheng

Erschienen in: International Journal of Computer Vision | Ausgabe 3/2017

Einloggen

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Pose estimation, tracking, and action recognition of articulated objects from depth images are important and challenging problems, which are normally considered separately. In this paper, a unified paradigm based on Lie group theory is proposed, which enables us to collectively address these related problems. Our approach is also applicable to a wide range of articulated objects. Empirically it is evaluated on lab animals including mouse and fish, as well as on human hand. On these applications, it is shown to deliver competitive results compared to the state-of-the-arts, and non-trivial baselines including convolutional neural networks and regression forest methods. Moreover, new sets of annotated depth data of articulated objects are created which, together with our code, are made publicly available.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Fußnoten
1
Our datasets, code, and detailed information pertaining to the project can be found at a dedicated project webpage http://​web.​bii.​a-star.​edu.​sg/​~xuchi/​Lie-X.​html.
 
Literatur
Zurück zum Zitat Agarwal, A., & Triggs, B. (2006). Recovering 3D human pose from monocular images. IEEE Transanction on PAMI 28(1), 44–58. Agarwal, A., & Triggs, B. (2006). Recovering 3D human pose from monocular images. IEEE Transanction on PAMI 28(1), 44–58.
Zurück zum Zitat Ali, K., Fleuret, F., Hasler, D., & Fua, P. (2009). Joint pose estimator and feature learning for object detection. In ICCV. Ali, K., Fleuret, F., Hasler, D., & Fua, P. (2009). Joint pose estimator and feature learning for object detection. In ICCV.
Zurück zum Zitat Altafini, C. (2000). Nonlinear control in year 2000, chap. The De Casteljau algorithm on SE(3) (pp. 1–12). Springer, Berlin. Altafini, C. (2000). Nonlinear control in year 2000, chap. The De Casteljau algorithm on SE(3) (pp. 1–12). Springer, Berlin.
Zurück zum Zitat Andriluka, M., Roth, S., & Schiele, B. (2008). People-tracking-by-detection and people-detection-by-tracking. In CVPR. Andriluka, M., Roth, S., & Schiele, B. (2008). People-tracking-by-detection and people-detection-by-tracking. In CVPR.
Zurück zum Zitat Andrychowicz, M., Denil, M., Gomez, S., Hoffman, M., Pfau, D., Schaul, T., Shillingford, B., & de Freitas, N. (2016). Learning to learn by gradient descent by gradient descent (pp. 1–50). Andrychowicz, M., Denil, M., Gomez, S., Hoffman, M., Pfau, D., Schaul, T., Shillingford, B., & de Freitas, N. (2016). Learning to learn by gradient descent by gradient descent (pp. 1–50).
Zurück zum Zitat Arnol’d, V. I. (2013). Mathematical methods of classical mechanics. Berlin: Springer. Arnol’d, V. I. (2013). Mathematical methods of classical mechanics. Berlin: Springer.
Zurück zum Zitat Ballan, L., Taneja, A., Gall, J., Gool, L.V., & Pollefeys, M. (2012). Motion capture of hands in action using discriminative salient points. In ECCV. Ballan, L., Taneja, A., Gall, J., Gool, L.V., & Pollefeys, M. (2012). Motion capture of hands in action using discriminative salient points. In ECCV.
Zurück zum Zitat Bookstein, F. (1977). The study of shape transformation after D’Arcy Thompson. Mathematical Biosciences, 34(3–4), 177–219. Bookstein, F. (1977). The study of shape transformation after D’Arcy Thompson. Mathematical Biosciences, 34(3–4), 177–219.
Zurück zum Zitat Bourdev, L., & Malik, J. (2009). Poselets: Body part detectors trained using 3D human pose annotations. In ICCV. Bourdev, L., & Malik, J. (2009). Poselets: Body part detectors trained using 3D human pose annotations. In ICCV.
Zurück zum Zitat Branson, K., & Belongie, S. (2005). Tracking multiple mouse contours (without too many samples). In CVPR. Branson, K., & Belongie, S. (2005). Tracking multiple mouse contours (without too many samples). In CVPR.
Zurück zum Zitat Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005). Learning to rank using gradient descent. In ICML. Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005). Learning to rank using gradient descent. In ICML.
Zurück zum Zitat Chen, L., Wei, H., & Ferryman, J. (2013). A survey on model based approaches for 2D and 3D visual human pose recovery. PRL, 34(15), 1995–2006.CrossRef Chen, L., Wei, H., & Ferryman, J. (2013). A survey on model based approaches for 2D and 3D visual human pose recovery. PRL, 34(15), 1995–2006.CrossRef
Zurück zum Zitat Dollar, P., Rabaud, V., Cottrell, G., & Belongie, S. (2005). Behavior recognition via sparse spatio-temporal features. In IEEE Workshop on PETS. Dollar, P., Rabaud, V., Cottrell, G., & Belongie, S. (2005). Behavior recognition via sparse spatio-temporal features. In IEEE Workshop on PETS.
Zurück zum Zitat Dollar, P., Welinder, P., & Perona, P. (2010). Cascaded pose regression. In CVPR. Dollar, P., Welinder, P., & Perona, P. (2010). Cascaded pose regression. In CVPR.
Zurück zum Zitat Felzenszwalb, P., & Huttenlocher, D. (2005). Pictorial structures for object recognition. International Journal of Computer Vision, 61(1), 55–79.CrossRef Felzenszwalb, P., & Huttenlocher, D. (2005). Pictorial structures for object recognition. International Journal of Computer Vision, 61(1), 55–79.CrossRef
Zurück zum Zitat Fleuret, F., & Geman, D. (2008). Stationary features and cat detection. JMLR, 9, 2549–2578.MathSciNetMATH Fleuret, F., & Geman, D. (2008). Stationary features and cat detection. JMLR, 9, 2549–2578.MathSciNetMATH
Zurück zum Zitat Gall, J., Yao, A., Razavi, N., van Gool, L., & Lempitsky, V. (2011). Hough forests for object detection, tracking, and action recognition. IEEE Transactions on PAMI, 33(11), 2188–2202.CrossRef Gall, J., Yao, A., Razavi, N., van Gool, L., & Lempitsky, V. (2011). Hough forests for object detection, tracking, and action recognition. IEEE Transactions on PAMI, 33(11), 2188–2202.CrossRef
Zurück zum Zitat Hinterstoisser, S., Lepetit, V., Ilic, S., Fua, P., & Navab, N. (2010). Dominant orientation templates for real-time detection of textureless objects. In CVPR. Hinterstoisser, S., Lepetit, V., Ilic, S., Fua, P., & Navab, N. (2010). Dominant orientation templates for real-time detection of textureless objects. In CVPR.
Zurück zum Zitat Hough, P. (1959). Machine analysis of bubble chamber pictures. In Proceedings of International Conference on High Energy Accelerators and Instrumentation. Hough, P. (1959). Machine analysis of bubble chamber pictures. In Proceedings of International Conference on High Energy Accelerators and Instrumentation.
Zurück zum Zitat Huang, C., Allain, B., Franco, J., Navab, N., & Boyer, E. (2016). Volumetric 3D tracking by detection. In CVPR. Huang, C., Allain, B., Franco, J., Navab, N., & Boyer, E. (2016). Volumetric 3D tracking by detection. In CVPR.
Zurück zum Zitat Isard, M., & Blake, A. (1998). Condensation—Conditional density propagation for visual tracking. International Journal of Computer Vision, 29(1), 5–28. Isard, M., & Blake, A. (1998). Condensation—Conditional density propagation for visual tracking. International Journal of Computer Vision, 29(1), 5–28.
Zurück zum Zitat Kalueff, A., Gebhardt, M., Stewart, A., Cachat, J., Brimmer, M., Chawla, J., et al. (2013). Towards a comprehensive catalog of zebrafish behavior 1.0 and beyond. Zebrafish, 10(1), 70–86.CrossRef Kalueff, A., Gebhardt, M., Stewart, A., Cachat, J., Brimmer, M., Chawla, J., et al. (2013). Towards a comprehensive catalog of zebrafish behavior 1.0 and beyond. Zebrafish, 10(1), 70–86.CrossRef
Zurück zum Zitat Lazebnik, S., Schmid, C., & Ponce, J. (2006). Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In CVPR. Lazebnik, S., Schmid, C., & Ponce, J. (2006). Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In CVPR.
Zurück zum Zitat Leibe, B., Leonardis, A., & Schiele, B. (2004). Combined object categorization and segmentation with an implicit shape model (pp. 17–32). In ECCV workshop on statistical learning in computer vision. Leibe, B., Leonardis, A., & Schiele, B. (2004). Combined object categorization and segmentation with an implicit shape model (pp. 17–32). In ECCV workshop on statistical learning in computer vision.
Zurück zum Zitat Mahasseni, B., & Todorovic, S. (2016). Regularizing long short term memory with 3D human-skeleton sequences for action recognition. In CVPR. Mahasseni, B., & Todorovic, S. (2016). Regularizing long short term memory with 3D human-skeleton sequences for action recognition. In CVPR.
Zurück zum Zitat Manton, J. (2013). A primer on stochastic differential geometry for signal processing. IEEE Journal of Selected Topics in Signal Processing, 7(4), 681–699.CrossRef Manton, J. (2013). A primer on stochastic differential geometry for signal processing. IEEE Journal of Selected Topics in Signal Processing, 7(4), 681–699.CrossRef
Zurück zum Zitat Mikic, I., Trivedi, M. M., Hunter, E., & Cosman, P. C. (2003). Human body model acquisition and tracking using voxel data. International Journal of Computer Vision, 53(3), 199–223.CrossRefMATH Mikic, I., Trivedi, M. M., Hunter, E., & Cosman, P. C. (2003). Human body model acquisition and tracking using voxel data. International Journal of Computer Vision, 53(3), 199–223.CrossRefMATH
Zurück zum Zitat Murray, R., Sastry, S., & Li, Z. (1994). A mathematical introduction to robotic manipulation. boca raton: CRC Press.MATH Murray, R., Sastry, S., & Li, Z. (1994). A mathematical introduction to robotic manipulation. boca raton: CRC Press.MATH
Zurück zum Zitat Nie, X., Xiong, C., & Zhu, S. (2015). Joint action recognition and pose estimation from video. In CVPR. Nie, X., Xiong, C., & Zhu, S. (2015). Joint action recognition and pose estimation from video. In CVPR.
Zurück zum Zitat Oberweger, M., Wohlhart, P., & Lepetit, V. (2015a). Hands deep in deep learning for hand pose estimation. In Computer Vision Winter Workshop. Oberweger, M., Wohlhart, P., & Lepetit, V. (2015a). Hands deep in deep learning for hand pose estimation. In Computer Vision Winter Workshop.
Zurück zum Zitat Oberweger, M., Wohlhart, P., & Lepetit, V. (2015b). Training a feedback loop for hand pose estimation. In ICCV. Oberweger, M., Wohlhart, P., & Lepetit, V. (2015b). Training a feedback loop for hand pose estimation. In ICCV.
Zurück zum Zitat Oikonomidis, N., & Argyros, A. (2011). Efficient model-based 3D tracking of hand articulations using Kinect. In BMVC. Oikonomidis, N., & Argyros, A. (2011). Efficient model-based 3D tracking of hand articulations using Kinect. In BMVC.
Zurück zum Zitat Perez-Sala, X., Escalera, S., Angulo, C., & Gonzalez, J. (2014). Survey of human motion analysis using depth imagery. Sensors, 14, 4189–4210.CrossRef Perez-Sala, X., Escalera, S., Angulo, C., & Gonzalez, J. (2014). Survey of human motion analysis using depth imagery. Sensors, 14, 4189–4210.CrossRef
Zurück zum Zitat Poppe, R. (2007). Vision-based human motion analysis: An overview. Computer Vision and Image Understanding, 108(1–2), 4–18. Poppe, R. (2007). Vision-based human motion analysis: An overview. Computer Vision and Image Understanding, 108(1–2), 4–18.
Zurück zum Zitat Procesi, C. (2007). Lie groups: An approach through invariants and representations. Berlin: Springer.MATH Procesi, C. (2007). Lie groups: An approach through invariants and representations. Berlin: Springer.MATH
Zurück zum Zitat Qian, C., Sun, X., Wei, Y., Tang, X., & Sun, J. (2014). Realtime and robust hand tracking from depth. In CVPR. Qian, C., Sun, X., Wei, Y., Tang, X., & Sun, J. (2014). Realtime and robust hand tracking from depth. In CVPR.
Zurück zum Zitat Rahmani, H., & Mian, A. (2016). 3D action recognition from novel viewpoints. In CVPR. Rahmani, H., & Mian, A. (2016). 3D action recognition from novel viewpoints. In CVPR.
Zurück zum Zitat Shotton, J., Girshick, R., Fitzgibbon, A., Sharp, T., Cook, M., Finocchio, M., et al. (2013). Efficient human pose estimation from single depth images. IEEE TPAMI, 35(12), 2821–40.CrossRef Shotton, J., Girshick, R., Fitzgibbon, A., Sharp, T., Cook, M., Finocchio, M., et al. (2013). Efficient human pose estimation from single depth images. IEEE TPAMI, 35(12), 2821–40.CrossRef
Zurück zum Zitat Sinha, A., Choi, C., & Ramani, K. (2016). Deephand: Robust hand pose estimation by completing a matrix imputed with deep features. In CVPR. Sinha, A., Choi, C., & Ramani, K. (2016). Deephand: Robust hand pose estimation by completing a matrix imputed with deep features. In CVPR.
Zurück zum Zitat Srivastava, A., Turaga, P., & Kurtek, S. (2012). On advances in differential-geometric approaches for 2D and 3D shape analyses and activity recognition. Image Vision Computing, 30(6–7), 398–416.CrossRef Srivastava, A., Turaga, P., & Kurtek, S. (2012). On advances in differential-geometric approaches for 2D and 3D shape analyses and activity recognition. Image Vision Computing, 30(6–7), 398–416.CrossRef
Zurück zum Zitat Sun, X., Wei, Y., Liang, S., Tang, X., & Sun, J. (2015). Cascaded hand pose regression. In CVPR. Sun, X., Wei, Y., Liang, S., Tang, X., & Sun, J. (2015). Cascaded hand pose regression. In CVPR.
Zurück zum Zitat Tan, D., Cashman, T., Taylor, J., Fitzgibbon, A., Tarlow, D., Khamis, S., Izadi, S., & Shotton, J. (2016). Fits like a glove: Rapid and reliable hand shape personalization. In CVPR. Tan, D., Cashman, T., Taylor, J., Fitzgibbon, A., Tarlow, D., Khamis, S., Izadi, S., & Shotton, J. (2016). Fits like a glove: Rapid and reliable hand shape personalization. In CVPR.
Zurück zum Zitat Tang, D., Taylor, J., Kohli, P., Keskin, C., Kim, T., & Shotton, J. (2015). Opening the black box: Hierarchical sampling optimization for estimating human hand pose. In ICCV. Tang, D., Taylor, J., Kohli, P., Keskin, C., Kim, T., & Shotton, J. (2015). Opening the black box: Hierarchical sampling optimization for estimating human hand pose. In ICCV.
Zurück zum Zitat Tompson, J., Jain, A., LeCun, Y., & Bregler, C. (2014). Joint training of a convolutional network and a graphical model for human pose estimation. In NIPS. Tompson, J., Jain, A., LeCun, Y., & Bregler, C. (2014). Joint training of a convolutional network and a graphical model for human pose estimation. In NIPS.
Zurück zum Zitat Tompson, J., Stein, M., Lecun, Y., & Perlin, K. (2014). Real-time continuous pose recovery of human hands using convolutional networks. SIGGRAPH. Tompson, J., Stein, M., Lecun, Y., & Perlin, K. (2014). Real-time continuous pose recovery of human hands using convolutional networks. SIGGRAPH.
Zurück zum Zitat Tuzel, O., Porikli, F., & Meer, P. (2008). Learning on Lie groups for invariant detection and tracking. In CVPR. Tuzel, O., Porikli, F., & Meer, P. (2008). Learning on Lie groups for invariant detection and tracking. In CVPR.
Zurück zum Zitat Vemulapalli, R., Arrate, F., & Chellappa, R. (2014). Human action recognition by representing 3D skeletons as points in a Lie group. In CVPR. Vemulapalli, R., Arrate, F., & Chellappa, R. (2014). Human action recognition by representing 3D skeletons as points in a Lie group. In CVPR.
Zurück zum Zitat Vemulapalli, R., & Chellappa, R. (2016). Rolling rotations for recognizing human actions from 3D skeletal data. In CVPR. Vemulapalli, R., & Chellappa, R. (2016). Rolling rotations for recognizing human actions from 3D skeletal data. In CVPR.
Zurück zum Zitat Wiltschko, A., Johnson, M., Iurilli, G., Peterson, R., Katon, J., Pashkovski, S., et al. (2015). Mapping sub-second structure in mouse behavior. Neuron, 88(6), 1121–35.CrossRef Wiltschko, A., Johnson, M., Iurilli, G., Peterson, R., Katon, J., Pashkovski, S., et al. (2015). Mapping sub-second structure in mouse behavior. Neuron, 88(6), 1121–35.CrossRef
Zurück zum Zitat Xiong, X., & la Torre, F.D. (2013). Supervised descent method and its applications to face alignment. In CVPR. Xiong, X., & la Torre, F.D. (2013). Supervised descent method and its applications to face alignment. In CVPR.
Zurück zum Zitat Xu, C., & Cheng, L. (2013). Efficient hand pose estimation from a single depth image. In ICCV. Xu, C., & Cheng, L. (2013). Efficient hand pose estimation from a single depth image. In ICCV.
Zurück zum Zitat Xu, C., Nanjappa, A., Zhang, X., & Cheng, L. (2015). Estimate hand poses efficiently from single depth images. International Journal of Computer Vision, 1–25. Xu, C., Nanjappa, A., Zhang, X., & Cheng, L. (2015). Estimate hand poses efficiently from single depth images. International Journal of Computer Vision, 1–25.
Zurück zum Zitat Yang, Y., & Ramanan, D. (2011). Articulated pose estimation with flexible mixtures-of-parts. In CVPR. Yang, Y., & Ramanan, D. (2011). Articulated pose estimation with flexible mixtures-of-parts. In CVPR.
Zurück zum Zitat Zhou, X., Wan, Q., Zhang, W., Xue, X. & Wei, Y. (2016). Model-based deep hand pose estimation. In IJCAI. Zhou, X., Wan, Q., Zhang, W., Xue, X. & Wei, Y. (2016). Model-based deep hand pose estimation. In IJCAI.
Metadaten
Titel
Lie-X: Depth Image Based Articulated Object Pose Estimation, Tracking, and Action Recognition on Lie Groups
verfasst von
Chi Xu
Lakshmi Narasimhan Govindarajan
Yu Zhang
Li Cheng
Publikationsdatum
23.02.2017
Verlag
Springer US
Erschienen in
International Journal of Computer Vision / Ausgabe 3/2017
Print ISSN: 0920-5691
Elektronische ISSN: 1573-1405
DOI
https://doi.org/10.1007/s11263-017-0998-6

Weitere Artikel der Ausgabe 3/2017

International Journal of Computer Vision 3/2017 Zur Ausgabe