Matthew Flagg
Sing Bing Kang
Irfan Essa
ABSTRACT
This paper describes a data-driven approach for generating photorealistic animations of human motion. Each animation sequence follows a user-choreographed path and plays continuously by seamlessly transitioning between different segments of the captured data. To produce these animations, we capitalize on the complementary characteristics of motion capture data and video. We customize our capture system to record motion capture data that are synchronized with our video source. Candidate transition points in video clips are identified using a new similarity metric based on 3-D marker trajectories and their 2-D projections into video. Once the transitions have been identified, a video-based motion graph is constructed. We further exploit hybrid motion and video data to ensure that the transitions are seamless when generating animations. Motion capture marker projections serve as control points for segmentation of layers and nonrigid transformation of regions. This allows warping and blending to generate seamless in-between frames for animation. We show a series of choreographed animations of walks and martial arts scenes as validation of our approach.
Supplemental Material
- Agarwala, A., Zheng, K. C., Pal, C., Agrawala, M., Cohen, M., Curless, B., Salesin, D., and Szeliski, R. 2005. Panoramic video textures. ACM Transactions on Graphics 24, 3, 821--827. 2 Google Scholar
Digital Library
- Beier, T., and Neely, S. 1992. Feature-based image metamorphosis. In Computer Graphics (Proceedings of ACM SIGGRAPH 92) 26, 2, 35--42. 3 Google ScholarDigital Library
- Besl, P. J., and McKay, N. D. 1992. A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14, 2, 239--256. 4 Google ScholarDigital Library
- Borshukov, G., Hable, J., and Montgomery, J. 2007. Playable Universal Capture chapter in GPU Gems 3. Addison Wesley. 6Google Scholar
- Boykov, Y., and Kolmogorov, V. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. on Pattern Analysis and Machine Intelligence 26, 9, 1124--1137. 5 Google ScholarDigital Library
- Carranza, J., Theobalt, C., Magnor, M. A., and Seidel, H.-P. 2003. Free-viewpoint video of human actors. ACM Transactions on Graphics 22, 3, 569--577. 3 Google ScholarDigital Library
- Celly, B., and Zordan, V. 2004. Animated people textures. In 17th International Conference on Computer Animation and Social Agents. 2, 3Google Scholar
- Cobzas, D., Yerex, K., and Jgersand, M. 2002. Dynamic textures for image-based rendering of fine-scale 3d structure and animation of non-rigid motion. In Eurographics, 1067--7055. 3Google Scholar
- de Aguiar, E., Stoll1, C., Theobalt, C., Ahmed1, N., Seidel, H., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Transactions on Graphics 27, 3, 4:1--4:10. 1 Google ScholarDigital Library
- Efros, A. A., and Leung, T. K. 1999. Texture synthesis by non-parametric sampling. In ICCV (2), 1033--1038. 6 Google ScholarDigital Library
- Felzenszwalb, P. F., and Huttenlocher, D. P. 2004. Efficient graph-based image segmentation. Int. J. Comput. Vision 59, 2, 167--181. 5 Google ScholarDigital Library
- Hartley, R. I., and Zisserman, A. 2004. Multiple View Geometry in Computer Vision, second ed. Cambridge University Press. 8 Google ScholarDigital Library
- Hornung, A., Dekkers, E., and Kobbelt, L. 2007. Character animation from 2d pictures and 3d motion data. ACM Transactions on Graphics 26, 1. 3, 7 Google ScholarDigital Library
- Kovar, L., Gleicher, M., and Pighin, F. 2002. Motion graphs. ACM Transactions on Graphics 21, 3, 473--482. 3 Google ScholarDigital Library
- Li, S. Z. 1995. Markov random field modeling in computer vision. Springer-Verlag, London, UK. 5 Google ScholarDigital Library
- Sand, P., McMillan, L., and Popovic, J. 2003. Continuous capture of skin deformation. In ACM Transactions on Graphics, vol. 22, 578--586. 3 Google ScholarDigital Library
- Schaefer, S., McPhail, T., and Warren, J. 2006. Image deformation using moving least squares. ACM Transactions on Graphics 25, 3, 533--540. 4 Google ScholarDigital Library
- Schödl, A., and Essa, I. A. 2002. Controlled animation of video sprites. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA), 121--127. 2, 3 Google ScholarDigital Library
- Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. In Proceedings of ACM SIGGRAPH 2000, ACM Press / ACM SIGGRAPH, 489--498. 2, 3 Google ScholarDigital Library
- Starck, J., and Hilton, A. 2007. Surface capture for performance-based animation. IEEE Comput. Graph. Appl. 27, 3, 21--31. 3 Google ScholarDigital Library
- Zhao, L., and Safonova, A. 2008. Achieving good connectivity in motion graphs. In Proceedings of the 2008 ACM/Eurographics Symposium on Computer Animation. 7 Google ScholarDigital Library
- Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-quality video view interpolation using a layered representation. ACM Transactions on Graphics 23, 3, 600--608. 1, 2, 7 Google ScholarDigital Library
Index Terms
- Human video textures
Recommendations
Video textures
SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniquesThis paper introduces a new type of medium, called a video texture, which has qualities somewhere between those of a photograph and a video. A video texture provides a continuous infinitely varying stream of images. While the individual frames of a ...
Free-viewpoint video of human actors
In free-viewpoint video, the viewer can interactively choose his viewpoint in 3-D space to observe the action of a dynamic real-world scene from arbitrary perspectives. The human body and its motion plays a central role in most visual media and its ...
Capturing and animating skin deformation in human motion
SIGGRAPH '06: ACM SIGGRAPH 2006 PapersDuring dynamic activities, the surface of the human body moves in many subtle but visually significant ways: bending, bulging, jiggling, and stretching. We present a technique for capturing and animating those motions using a commercial motion capture ...
Comments