Abstract
Tracking, recognition and interaction based on 3D freehand are a part of our virtual assembly system, in which monocular camera is used to input online freehand videos and the hand pose tracker requires a reliable initial pose in the first frame. A novel approach to initializing 3D pose and position of freehand is put forward in this paper visualization of 3D hand model and modeling the operators’ cognitive behaviors. Our approach is composed of three phases: hand posture recognition, coarse-tuning and fine-tuning. The operator moves his/her hand onto the to meet the needs of our virtual assembly system. The main contribution of this paper is that the three core techniques are for the first time integrated together, including human–computer interaction (HCI) in the process of initializing, projection of the 3D hand model in the period of coarse-tuning time. Then, the computer repeatedly fine-tunes the 3D hand model until the projection of the 3D hand model is completely superimposed onto the operator’s hand image. We focus on exploring and modeling cognitive behavior of operator’s hand upon which we design our initialization algorithm. Our research shows that cognitive behavioral models are not only beneficial to reducing cognitive loads for operators, because it makes the computers cater for the changes of the operators’ hand poses, but also helpful to address high dimensionality of articulated 3D hand model. Our experimental results also show that the approach presented in this paper is easier, more pleasurable and satisfactory experience for the operators. Our initialization system has successfully been applied to our 3D freehand tracking system and a simulation virtual assembly system.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Erol, A., et al.: Vision-based hand pose estimation: a review. Comput. Vis. Image Underst. 108, 52–73 (2007)
Julier, S.J., Uhlmann, J.K.: A new extension of the Kalman filter to nonlinear systems. In: Procedure of AeroSense: The 11th International Symposium on Aerospace/Defence Sensing, Simulation and Controls, pp. 82–193. SPIE, Bellingham (1997)
Cui, J.S.: Studies on three-dimensional model based posture estimation and tracking of articulated objects. PhD Thesis, Tsinghua University, Beijing, China (2004)
Erol, A., Bebis, G., Nicolescu, M., Boyle, R., Twombly, X.: A review on vision-based full DOF hand motion estimation. In: Proceedings of the IEEE Workshop on Vision for Human–Computer Interaction (V4HCI), San Diego, California, vol. 3, pp. 75–83 (2005)
Triesch, J., von der Malsurg, C.: A system for person-independent hand posture recognition against complex background. IEEE Trans. Pattern Anal. Mach. Intell. 23(12), 84–95 (2001)
Shimada, N., Kimura, K., Shirai, Y.: Real-time 3-D hand posture estimation based on 2-D appearance retrieval using monocular camera. In: Proc. Int. Workshop RATFG-RTS. pp. 23–30 (2001)
Rosales, R.: The specialized mappings architecture with applications to vision-based estimation of articulated body pose. PhD Thesis. Boston University, Boston (2002)
Athitsos, V., Sclaroff, S.: Estimating 3D hand pose from a cluttered image. Proc. IEEE Int. Conf. Comput. Vis. Pattern Recogn. 2, 432–439 (2003)
Stenger, B., Thayananthan, A., Torr, P.H.S., Cipolla, R.: Filtering using a tree-based estimator. Proc. IEEE Int. Conf. Comput. Vis. 2, 1063–1070 (2003)
Tomasi, C., Petrov, S., Sastry, A.: 3D tracking = classification + interpolation. In: The Ninth IEEE International Conference on Computer Vision, pp. 1441–1448 (2003)
Bray, M., Koller-Meier, E., Goo, L.V.: Smart particle filtering for 3D hand tracking. In: The Sixth IEEE International Conference on Automatic Face and Pose Recognition, pp. 675–680. IEEE Computer Society, Los Alamitos (2004)
LaGorce, M., Paragios, N., Fleet, D.: Model based hand tracking with texture, shading and self-occlusions. IEEE Proc. IEEE Conf. Comput. Vis. Pattern Recogn. pp. 1–8 (2008)
John, B.E.: Cognitive modeling in human–computer interaction. In: Proceedings of Graphics Interface, pp. 161–167 (1998)
Card, S.K., Moran, T.P., Ewell, A.: The Psychology of Human–Computer Interaction. Lawrence Erlbaum Associates Inc., Hillsdale (1993)
John, B.E., Kieras, D.E.: Using GOMS for operator interface design and evaluation: which technique? ACM Trans. Comput. Hum. Interact. 3(4), 287–319 (1996)
Anderson, J.R., et al.: Cognitive tutors: lessons learned. J. Learn. Sci. 4(2), 167–207 (1995)
Ritter, F.E., Baxter, G.D., Jones, G., Young, R.M.: Supporting cognitive models as operators. ACM Trans. Hum. Comput. Int. 7(2), 141–173 (2000)
Tang, Z.: Visualization of 3D Datasets [M]. Singhua University Press, Beijing (1999)
Cleveland, W.S.: Visualizing data [M]. Hobart Press, New Jersey (1993)
Jaimes, A., Sebe, N.: Multimodal human–computer interaction: a survey. Comput. Vis. Image Underst. 108(1–2), 116–134 (2007)
Karray, F., Alemzadeh, M., Saleh, J.A., Arab, M.N.: Human–computer interaction: overview on state of the art. Int. J. Smart Intell. Syst. 1(1), 137–159 (2008)
Chapanis, A.: Man Machine Engineering. Wadsworth, Belmont (1965)
Norman, D.: Cognitive engineering. In: Norman, D., Draper, S. (eds.) Operator Centered Design: New Perspective on Human–Computer Interaction. Lawrence Erlbaum, Hillsdale (1986)
Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)
Te’eni, D., Carey, J., Zhang, P.: Human Computer Interaction: Developing Effective Organizational Information Systems. Wiley, Hoboken (2007)
John, B.E., Kieras, D.E.: The GOMS family of operator interface analysis techniques: comparison and contrast. ACM Trans. Comput. Hum. Interact., 320–351 (1996)
Gordon, N., Salmond, D.J., Smith, A.F.M.: Novel approach to nonlinear and non-Gaussian Bayesian state estimation. IEE Proc. F 140, 107–113 (1993)
Feng el al, Z.: Research on features extraction from frame image sequences. In: International Symposium on Computer Science and Computational Technology (ISCSCT’2008), pp. 762–766 (2008)
Stefan, A., Athitsos, V., Alon, J., Sclaroff, S.: Translation and scale-invariant gesture recognition in complex scenes. In: Proceedings of the 1st International Conference on PErvasive Technologies Related to Assistive Environments, Athens, Greece, pp. 1–8 (2008)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Feng, Z., Zhang, M., Pan, Z. et al. 3D-freehand-pose initialization based on operator’s cognitive behavioral models. Vis Comput 26, 607–617 (2010). https://doi.org/10.1007/s00371-010-0452-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-010-0452-z