Min Je Park
Skip Abstract Section
Abstract
Video taken from a single monocular camera is the most common means of recording human motion. In this article, we present a practical, semiautomatic method for synthesizing a human motion that is guided by such video. After preprocessing an input video, we select a precaptured motion clip called a reference motion from a motion library. We then compute the sequence of body configurations of a virtual character by deforming this motion, according to spacetime constraints derived from a sequence of 2D features in the input video. Experimental results show that our method can synthesize highly dynamic motions, such as kicking and header motions of soccer players. We also showed the potential of our scheme as a new paradigm for motion capture, that is, capturing motions from videos taken with a monocular camera, even outside a motion capture studio.
- Barron, C. and Kakadiaris, I. A. 2000. Estimating anthropometry and pose from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 669--676.]]Google Scholar
- Ben-Arie, J., Pnadit, P., and Rajaram, S. 2001a. Design of a digital library for human movement. In Proceedings of the 1st ACM/IEEE-CS Joint Conference on Digital Libraries. 300--309.]] Google Scholar
- Ben-Arie, J., Pnadit, P., and Rajaram, S. 2001b. View-Based human activity recognition by indexing and sequencing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 78--83.]]Google Scholar
- Brand, M. 1999. Shadow puppetry. In Proceedings of the International Conference on Computer Vision (ICCV).]] Google Scholar
- Bregler, C., Loeb, L., Chuang, E., and Deshpande, H. 2002. Turning to the masters: Motion capturing cartoons. Comput. Graph. 36, 399--407.]]Google Scholar
- Bregler, C. and Malik, J. 1998. Tracking people with twists and exponential maps. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8--15.]] Google Scholar
- Bregler, C., Malik, J., and Pullen, K. 2004. Twist-Based acquisition and tracking of animal and human kinematics. Int. J. Comput. Vision 56, 3, 179--194.]] Google Scholar
- Bruderin, A. and Williams, L. 1995. Motion signal processing. Comput. Graph. 29, 4, 97--104.]]Google Scholar
- Cohen, M. F. 1992. Interactive spacetime control for animation. Comput. Graph. 26, 2, 293--302.]] Google Scholar
- Cormen, T. H., Leiserson, C. E., and Rivest, R. L. 1999. Introduction to Algorithms. MIT Press, Cambridge, MA.]] Google Scholar
- Delamarre, Q. and Faugeras, O. 2001. 3d articulated models and multiview tracking with physical forces. Comput. Vision Image Understanding 81, 3, 328--357.]] Google Scholar
- Demirdjian, D., Ko, T., and Darrell, T. 2003. Constraining human body tracking. In Proceedings of the International Conference on Computer Vision.]] Google Scholar
- Demori, R. and Probst, D. 1986. Handbook of Pattern Recognition and Image Processing, 1st ed. Academic Press, London.]]Google Scholar
- Difranco, D. E., Cham, T. J., and Rehg, J. M. 1999. Recovery of 3D articulated motion from 2D correspondences. Tech. Rep. TR-CRL-99-7, Cambridge Research Laboratory.]]Google Scholar
- Drummond, T. and Cipolla, R. 2001. Real-Time tracking of highly articulated structures in the presence of noisy measurements. In Proceedings of the International Conference on Computer Vision. 315--320.]]Google Scholar
- Fang, A. C. and Pollard, N. C. 2003. Efficient synthesis of physically valid human motion. Comput. Graph. 37.]]Google Scholar
- Fletcher, R. 1980. Practical Methods of Optimization, 1st ed. Wiley and Sons, New York.]] Google Scholar
- Gavrila, D. M. 1999. The visual analysis of human movement: A survey. Comput. Vision. Image Understanding 3, 1, 82--98.]] Google Scholar
- Gill, P. E. and Murray, W. 1974. Numerical Methods for Constrained Optimization, 1st ed. Academic Press, London.]]Google Scholar
- Gleicher, M. 1997. Motion editing with spacetime constraints. In Proceedings of the Symposium on Interactive 3D Graphics. 139--148.]] Google Scholar
- Gleicher, M. 1998. Retargetting motion to new characters. Comput. Graph. 32, 33--42.]]Google Scholar
- Gleicher, M. and Ferrier, N. 2002. Evaluating video-based motion capture. In Proceedings of the Computer Animation Conference. 75--80.]] Google Scholar
- Grochow, K., Martin, S. L., Hertzmann, A., and Popovic, Z. 2004. Style-Based inverse kinematics. Comput. Graph. 38, 522--531.]]Google Scholar
- Howe, N. R., Leventon, M. E., and Freeman, W. T. 2000. Bayesian reconstruction of 3D human motion from single-camera video. Advances in Neural Information Processing Systems. 820--826.]]Google Scholar
- Ju, S. X., Black, M. J., and Yacoob, Y. 1996. Cardboard people: A parameterized model of articulated image motion. In Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition.]] Google Scholar
- Kakadiaris, I. and Metaxas, D. 1996. Model-Based estimation of 3D human motion with occlusion based on active multi-viewpoint selection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 81--89.]] Google Scholar
- Kirk, A. G., O'Brien, J. F., and Forsyth, D. A. 2005. Skeletal parameter estimation from optical motion capture data. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 782--788.]] Google Scholar
- Lee, J., Chai, J., Reitsma, P., Hodgins, J., and Pollard, N. 2002. Interactive control of avatars aniamted with human motion data. Comput. Graph. 36, 491--500.]]Google Scholar
- Lee, J. and Shin, S. Y. 1999. A hierarchical approach to interactive motion editing for human-like figures. Comput. Graph. 33, 39--48.]]Google Scholar
- Lee, S., Wolberg, G., and Shin, S. Y. 1997. Scattered data interpolation with multilevel b-splines. IEEE Trans. Visualization Comput. Graph. 3, 3, 228--244.]] Google Scholar
- Leung, M. K. and Yang, Y. H. 1995. First sight: A human body outline labeling system. IEEE Trans. Pattern Anal. Mach. Intell. 17, 4, 359--377.]] Google Scholar
- Liebowitz, D. and Carlsson, S. 2001. Uncalibrated motion capture exploiting articulated structure constraints. In Proceedings of the 8th International Conference on Computer Vision. 230--237.]]Google Scholar
- Liu, C. K. and Popović, Z. 2002. Synthesis of complex dynamic character motion from simple animations. Comput. Graph. 36, 4, 408--416.]]Google Scholar
- Moeslund, T. B. and Granum, E. 2001. A survey of computer vision-based human motion capture. Comput. Vision Image Understanding 81, 3, 231--268.]] Google Scholar
- Morris, D. D. and Rehg, J. 1998. Singularity analysis for articulated object tracking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).]] Google Scholar
- MotionBuilder. Autodesk MotionBuilderTM. www.autodesk.com.]]Google Scholar
- Noma, T., Oishi, K., Futsuhara, H., Baba, H., Ohashi, T., and Ejima, T. 1999. Motion generator approach to translating human motion from video to animation. In Proceedings of the Pacific Graphics Conference. 50--59.]] Google Scholar
- Park, M. J., Choi, M. G., and Shin, S. Y. 2002. Human motion reconstruction from inter-frame feature correspondences of a single video stream. In Proceedings of the ACM Symposium on Computer Animation. 113--120.]] Google Scholar
- Pavlovic, V., Rehg, J. M., Cham, T. J., and Murphy, K. 1999. A dynamic Bayesian network approach to figure tracking using learned dynamic models. In Proceedings of the International Conference on Computer Vision 1, 94--101.]]Google Scholar
- Pheasant, S. 1996. Bodyspace: Anthropometry, Ergonomics and the Design of Work, 2nd ed. Taylor and Francis, Bristol, PA.]]Google Scholar
- Plaencker, R. and Fua, P. 2001. Tracking and modeling people in video sequences. Comput. Vision Image Understanding 81, 3, 285--302.]] Google Scholar
- Popović, Z. and Witkin, A. 1999. Physically-Based motion transformation. Comput. Graph. 33, 11--20.]]Google Scholar
- Poser. e frontier PoserTM. www.e-frontier.com.]]Google Scholar
- Press, W. H., Teukolsky, S. A., Vetterling, W. T., and Flannery, B. P. 1992. Numerical Recipes in C: The Art of Scientific Computing, 2nd ed. Cambridge University Press, New York.]] Google Scholar
- Ramanan, D. and Forsyth, D. A. 2003. Automatic annotation of everyday movements. Neural Information Processing Systems (NIPS).]]Google Scholar
- Rehg, J. M. and Kanade, T. 1994. Visual tracking of high DOF articulated structures: An application to human hand tracking. In Proceedings of the European Conference on Computer Vision. 35--46.]] Google Scholar
- Rose, C., Guenter, B., Bodenheimer, B., and Cohen, M. F. 1996. Efficient generation of motion transitions using spacetime constraints. Comput. Graph. 30, 147--154.]]Google Scholar
- Safonova, A., Hodgins, J., and Pollard, N. 2004. Synthesizing physically realisitic human motion in low-dimensional, behavior-specific spaces. Comput. Graph. 38.]]Google Scholar
- Shoemake, K. 1985. Animating rotation with quaternion curves. Comput. Graph. 19, 245--254.]] Google Scholar
- Sidenbladh, H., Black, M. J., and Fleet, D. J. 2000a. Stochastic tracking of 3D human figures using 2D image motion. In Proceedings of the European Conference on Computer Vision. 702--718.]] Google Scholar
- Sidenbladh, H., Black, M. J., and Fleet, D. J. 2000b. Stochastic tracking of 3d human figures using 2d image motion. In Proceedings of the European Conference on Computer Vision.]] Google Scholar
- Sidenbladh, H., Black, M. J., and Sigal, L. 2002. Implicit probabilistic models of human motion for synthesis and tracking. In Proceedings of the European Conference on Computer Vision. 784--800.]] Google Scholar
- Sminchisescu, C. and Triggs, B. 2001. Covariance scaled sampling for monocular 3D body trakcing. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition. 447--454.]]Google Scholar
- Song, Y., Goncalves, L., and Perona, P. 2003. Unsupervised learning of human motion. IEEE Trans. Pattern Anal. Mach. Intell. 25, 7, 814--827.]] Google Scholar
- Starck, J. and Hilton, A. 2003. Model-Based multiple view reconstruction of people. In Proceedings of the International Conference on Computer Vision.]] Google Scholar
- Tak, S., Song, O., and Ko, H. 2000. Motion balancing filtering. Comput. Graph. Forum 9, 3, 437--446.]]Google Scholar
- Taylor, C. J. 2000. Reconstruction of articulated objects from point correspondences in a single uncalibrated image. Comput. Vision Image Understanding 80, 8, 349--363.]] Google Scholar
- Tian, T.-P., Li, R., and Sclaroff, S. 2005. Articulated pose estimation in a learned smooth space of feasible solutions. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2005.]] Google Scholar
- Urtasun, R., Fleet, D., and Fua, P. 2005. Monocular 3-d tracking of the golf swing. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2005.]] Google Scholar
- Veenman, C. J., Hendriks, E. A., and Reinders, M. J. T. 1998. A fast and robust point tracking algorithm. In Proceedings of the International Conference on Image Processing.]]Google Scholar
- Wang, L., Hu, W., and Tan, T. 2003. Recent development in human motion analysis. Pattern Recogn. 36, 585--601.]]Google Scholar
- Witkin, A. and Kass, M. 1988. Spacetime constraints. Comput. Graph. 22, 4, 159--168.]] Google Scholar
- Witkin, A. and Popović, Z. 1995. Motion warping. Comput. Graph. 29, 105--108.]]Google Scholar
- Wren, C., Azarbayejani, A., and Pentland, A. 1997. Pfinder: Real-Time 3-D tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19, 7, 780--785.]] Google Scholar
- Yamane, K., Kuffner, J., and Hodgins, J. 2004. Synthesizing animations of human manipulation tasks. Comput. Graph. 38.]]Google Scholar
- Zheng, J. Y. and Suezaki, S. 1998. A model-Based approach in extracting and generating human motion. In Proceedings of the 14th International Conference on Pattern Recognition. 1201--1205.]] Google Scholar
Index Terms
- Video-guided motion synthesis using example motions
Recommendations
Human motion reconstruction from inter-frame feature correspondences of a single video stream using a motion library
SCA '02: Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animationVideos taken from a single camera are a most common source of human motions. In this paper, we present a novel method to reconstruct the motion of a human-like figure from inter-frame feature correspondences of a single video stream. We exploit a motion ...
Using motion capture for interactive motion editing
VRCAI '14: Proceedings of the 13th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in IndustryMotion capture technology has been widely used for creating character motions. Motion editing is usually also required to adjust captured motions. Because character poses which include joint rotations, body positions, and orientations are high-...
Motion synthesis by example
Proceedings of the Eurographics workshop on Computer animation and simulation '96
Comments