Skip to main content
main-content
Top

Hint

Swipe to navigate through the chapters of this book

2019 | OriginalPaper | Chapter

3D Hand Tracking by Employing Probabilistic Principal Component Analysis to Model Action Priors

Authors: Emmanouil Oulof Porfyrakis, Alexandros Makris, Antonis Argyros

Published in: Computer Vision Systems

Publisher: Springer International Publishing

share
SHARE

Abstract

This paper addresses the problem of 3D hand pose estimation by modeling specific hand actions using probabilistic Principal Component Analysis. For each of the considered actions, a parametric subspace is learned based on a dataset of sample action executions. The developed method tracks the 3D hand pose either in the case of unconstrained hand motion or in the case that the hand is engaged in some of the modelled actions. The tracker uses gradient descent optimization to fit a 3D hand model to the available observations. An online criterion is used to automatically switch between tracking the hand in the unconstrained case and tracking it in the case of learned action sub-spaces. To train and evaluate the proposed method, we captured a new dataset that contains sample executions of 5 different grasp-like hand actions and hand/object interactions. We tested the proposed method both quantitatively and qualitatively. For the quantitative evaluation we relied on our dataset to create synthetic sequences from which we artificially removed observations to simulate occlusions. The obtained results show that the proposed method improves 3D hand pose estimation over existing approaches, especially in the presence of occlusions, where the employed action models assist the accurate recovery of the 3D hand pose despite the missing observations.
Literature
1.
go back to reference Agarwal, S., Mierle, K., et al.: Ceres solver (2012) Agarwal, S., Mierle, K., et al.: Ceres solver (2012)
3.
go back to reference Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7291–7299 (2017) Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7291–7299 (2017)
4.
go back to reference de La Gorce, M., Fleet, D.J., Paragios, N.: Model-based 3D hand pose estimation from monocular video. IEEE Trans. Pattern Anal. Mach. Intell. 33(9), 1793–1805 (2011) CrossRef de La Gorce, M., Fleet, D.J., Paragios, N.: Model-based 3D hand pose estimation from monocular video. IEEE Trans. Pattern Anal. Mach. Intell. 33(9), 1793–1805 (2011) CrossRef
6.
go back to reference Jenkins, O.C., Matarić, M.J.: A spatio-temporal extension to isomap nonlinear dimension reduction. In: Proceedings of the Twenty-first International Conference on Machine Learning, p. 56. ACM (2004) Jenkins, O.C., Matarić, M.J.: A spatio-temporal extension to isomap nonlinear dimension reduction. In: Proceedings of the Twenty-first International Conference on Machine Learning, p. 56. ACM (2004)
7.
go back to reference Kato, M., Chen, Y.W., Xu, G.: Articulated hand tracking by PCA-ICA approach. In: 7th International Conference on Automatic Face and Gesture Recognition (FGR06), pp. 329–334. IEEE (2006) Kato, M., Chen, Y.W., Xu, G.: Articulated hand tracking by PCA-ICA approach. In: 7th International Conference on Automatic Face and Gesture Recognition (FGR06), pp. 329–334. IEEE (2006)
8.
go back to reference Köpüklü, O., Gunduz, A., Kose, N., Rigoll, G.: Real-time hand gesture detection and classification using convolutional neural networks. arXiv preprint arXiv:​1901.​10323 (2019) Köpüklü, O., Gunduz, A., Kose, N., Rigoll, G.: Real-time hand gesture detection and classification using convolutional neural networks. arXiv preprint arXiv:​1901.​10323 (2019)
9.
go back to reference Liang, C., Song, Y., Zhang, Y.: Hand gesture recognition using view projection from point cloud. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 4413–4417. IEEE (2016) Liang, C., Song, Y., Zhang, Y.: Hand gesture recognition using view projection from point cloud. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 4413–4417. IEEE (2016)
10.
go back to reference Makris, A., Argyros, A.: Model-based 3D hand tracking with on-line shape adaptation, pp. 77.1-77.12. British Machine Vision Association (2015) Makris, A., Argyros, A.: Model-based 3D hand tracking with on-line shape adaptation, pp. 77.1-77.12. British Machine Vision Association (2015)
11.
go back to reference Makris, A., Kyriazis, N., Argyros, A.A.: Hierarchical particle filtering for 3D hand tracking. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 8–17. IEEE, June 2015 Makris, A., Kyriazis, N., Argyros, A.A.: Hierarchical particle filtering for 3D hand tracking. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 8–17. IEEE, June 2015
12.
go back to reference Mueller, F., et al.: Ganerated hands for real-time 3D hand tracking from monocular RGB. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018 Mueller, F., et al.: Ganerated hands for real-time 3D hand tracking from monocular RGB. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018
13.
go back to reference Oikonomidis, I., Kyriazis, N., Argyros, A.A.: Efficient model-based 3D tracking of hand articulations using kinect. In: BmVC, vol. 1, p. 3 (2011) Oikonomidis, I., Kyriazis, N., Argyros, A.A.: Efficient model-based 3D tracking of hand articulations using kinect. In: BmVC, vol. 1, p. 3 (2011)
14.
go back to reference Ormoneit, D., Sidenbladh, H., Black, M.J., Hastie, T.: Learning and tracking cyclic human motion. In: Advances in Neural Information Processing Systems, pp. 894–900 (2001) Ormoneit, D., Sidenbladh, H., Black, M.J., Hastie, T.: Learning and tracking cyclic human motion. In: Advances in Neural Information Processing Systems, pp. 894–900 (2001)
15.
go back to reference Oyedotun, O.K., Khashman, A.: Deep learning in vision-based static hand gesture recognition. Neural Comput. Appl. 28(12), 3941–3951 (2017) CrossRef Oyedotun, O.K., Khashman, A.: Deep learning in vision-based static hand gesture recognition. Neural Comput. Appl. 28(12), 3941–3951 (2017) CrossRef
16.
go back to reference Panteleris, P., Oikonomidis, I., Argyros, A.A.: Using a single RGB frame for real time 3D hand pose estimation in the wild. In: IEEE Winter Conference on Applications of Computer Vision (WACV 2018), also available at Arxiv, pp. 436–445. IEEE, lake Tahoe, March 2018 Panteleris, P., Oikonomidis, I., Argyros, A.A.: Using a single RGB frame for real time 3D hand pose estimation in the wild. In: IEEE Winter Conference on Applications of Computer Vision (WACV 2018), also available at Arxiv, pp. 436–445. IEEE, lake Tahoe, March 2018
17.
go back to reference Poier, G., Schinagl, D., Bischof, H.: Learning pose specific representations by predicting different views, April 2018 Poier, G., Schinagl, D., Bischof, H.: Learning pose specific representations by predicting different views, April 2018
18.
go back to reference Qian, C., Sun, X., Wei, Y., Tang, X., Sun, J.: Realtime and robust hand tracking from depth, pp. 1106–1113, June 2014 Qian, C., Sun, X., Wei, Y., Tang, X., Sun, J.: Realtime and robust hand tracking from depth, pp. 1106–1113, June 2014
19.
go back to reference Raskin, L.: Dimensionality reduction for 3D articulated body tracking and human action analysis. Technion-Israel Institute of Technology, Faculty of Computer Science (2010) Raskin, L.: Dimensionality reduction for 3D articulated body tracking and human action analysis. Technion-Israel Institute of Technology, Faculty of Computer Science (2010)
20.
go back to reference Sun, X., Wei, Y., Liang, S., Tang, X., Sun, J.: Cascaded hand pose regression. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015 Sun, X., Wei, Y., Liang, S., Tang, X., Sun, J.: Cascaded hand pose regression. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015
21.
go back to reference Tan, D.J., et al.: Fits like a glove: rapid and reliable hand shape personalization. Microsoft Research, June 2016 Tan, D.J., et al.: Fits like a glove: rapid and reliable hand shape personalization. Microsoft Research, June 2016
22.
go back to reference Tian, T.P., Li, R., Sclaroff, S.: Tracking human body pose on a learned smooth space. Technical report, Boston University Computer Science Department (2005) Tian, T.P., Li, R., Sclaroff, S.: Tracking human body pose on a learned smooth space. Technical report, Boston University Computer Science Department (2005)
23.
go back to reference Tipping, M.E., Bishop, C.M.: Mixtures of probabilistic principal component analyzers. Neural Comput. 11(2), 443–482 (1999) CrossRef Tipping, M.E., Bishop, C.M.: Mixtures of probabilistic principal component analyzers. Neural Comput. 11(2), 443–482 (1999) CrossRef
Metadata
Title
3D Hand Tracking by Employing Probabilistic Principal Component Analysis to Model Action Priors
Authors
Emmanouil Oulof Porfyrakis
Alexandros Makris
Antonis Argyros
Copyright Year
2019
DOI
https://doi.org/10.1007/978-3-030-34995-0_48

Premium Partner