Carsten Stoll
Abstract
We present a new performance capture approach that incorporates a physically-based cloth model to reconstruct a rigged fully-animatable virtual double of a real person in loose apparel from multi-view video recordings. Our algorithm only requires a minimum of manual interaction. Without the use of optical markers in the scene, our algorithm first reconstructs skeleton motion and detailed time-varying surface geometry of a real person from a reference video sequence. These captured reference performance data are then analyzed to automatically identify non-rigidly deforming pieces of apparel on the animated geometry. For each piece of apparel, parameters of a physically-based real-time cloth simulation model are estimated, and surface geometry of occluded body regions is approximated. The reconstructed character model comprises a skeleton-based representation for the actual body parts and a physically-based simulation model for the apparel. In contrast to previous performance capture methods, we can now also create new real-time animations of actors captured in general apparel.
Supplemental Material
Available for Download
- Allen, B., Curless, B., Popović, Z., and Hertzmann, A. 2006. Learning a correlated model of identity and posedependent body shape variation for real-time synthesis. In Proc. of SCA, 147--156. Google Scholar
Digital Library
- Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: Shape completion and animation of people. In ACM TOG (Proc. SIGGRAPH '05). Google ScholarDigital Library
- Balan, A. O., and Black, M. J. 2008. The naked truth: Estimating body shape under clothing. In ECCV (2), 15--29. Google ScholarDigital Library
- Baran, I., and Popović, J. 2007. Automatic rigging and animation of 3d characters. ACM TOG (Proc. SIGGRAPH '07). Google ScholarDigital Library
- Bhat, K. S., Twigg, C. D., Hodgins, J. K., Khosla, P. K., Popović, Z. Z., and Seitz, S. M. 2003. Estimating cloth simulation parameters from video. In Proc. of SCA. Google ScholarDigital Library
- Bradley, D., Popa, T., Sheffer, A., Heidrich, W., and Boubekeur, T. 2008. Markerless garment capture. ACM TOG (Proc. SIGGRAPH '08). Google ScholarDigital Library
- Carranza, J., Theobalt, C., Magnor, M., and Seidel, H.-P. 2003. Free-viewpoint video of human actors. In ACM TOG (Proc. SIGGRAPH '03). Google ScholarDigital Library
- Choi, K.-J., and Ko, H.-S. 2005. Research problems in clothing simulation. Computer-Aided Design 37, 6, 585--592. Google ScholarDigital Library
- de Aguiar, E., Theobalt, C., Thrun, S., and Seidel, H.-P. 2008. Automatic conversion of mesh animations into skeleton-based animations. Proc. Eurographics EG'08.Google Scholar
- de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. In ACM TOG (Proc. SIGGRAPH'08). Google ScholarDigital Library
- Gall, J., Stoll, C., de Aguiar, E., Theobalt, C., Rosenhahn, B., and Seidel, H.-P. 2009. Motion capture using simultaneous skeleton tracking and surface estimation. In Proc. IEEE CVPR.Google Scholar
- Hansen, N., Niederberger, A. S. P., Guzzella, L., and Koumoutsakos, P. 2009. A method for handling uncertainty in evolutionary optimization with an application to feedback control of combustion. IEEE TEC 13, 1. Google ScholarDigital Library
- Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H.-P. 2009. A statistical model of human pose and body shape. CGF (Proc. Eurographics 2009) 28, 2.Google Scholar
- Hasler, N., Stoll, C., Rosenhahn, B., Thormählen, T., and Seidel, H.-P. 2009. Estimating body shape of dressed humans. Comput. Graph. 33, 3, 211--216. Google ScholarDigital Library
- James, D. L., and Twigg, C. D. 2005. Skinning mesh animations. ACM TOG (Proc. SIGGRAPH'05). Google ScholarDigital Library
- Kavan, L., Collins, S., Žára, J., and O'Sullivan, C. 2007. Skinning with dual quaternions. In Symposium on Interactive 3D graphics and games, 39--46. Google ScholarDigital Library
- Kawabata, S. 1980. The standardization and analysis of hand evaluation. Textile Machinery Society of Japan.Google Scholar
- Kircher, S., and Garland, M. 2006. Editing arbitrarily deforming surface animations. In ACM TOG (Proc. SIGGRAPH'06). Google ScholarDigital Library
- Lowe, D. G. 1999. Object recognition from local scale-invariant features. In Proc. ICCV, vol. 2, 1150ff. Google ScholarDigital Library
- Matusik, W., Buehler, C., Raskar, R., Gortler, S., and McMillan, L. 2000. Image-based visual hulls. In ACM TOG (Proc. SIGGRAPH'00). Google ScholarDigital Library
- Müller, M., Heidelberger, B., Hennix, M., and Ratcliff, J. 2007. Position based dynamics. Journal Vis. Com. Google ScholarDigital Library
- Park, S. I., and Hodgins, J. K. 2008. Data-driven modeling of skin and muscle deformation. ACM TOG (Proc. SIGGRAPH'08). Google ScholarDigital Library
- Popa, T., Zhou, Q., Bradley, D., Kraevoy, V., Fu, H., Sheffer, A., and Heidrich, W. 2009. Wrinkling captured garments using space-time data-driven deformation. Computer Graphics Forum (Proc. Eurographics) 28, 2.Google ScholarCross Ref
- Poppe, R. 2007. Vision-based human motion analysis: An overview. CVIU 108, 1--2, 4--18. Google ScholarDigital Library
- Pritchard, D., and Heidrich, W. 2003. Cloth motion capture. In Proc. Eurographics EG'03.Google Scholar
- Sand, P., McMillan, L., and Popović, J. 2003. Continuous capture of skin deformation. ACM TOG (Proc. SIGGRAPH'03). Google ScholarDigital Library
- Scholz, V., Stich, T., Keckeisen, M., Wacker, M., and Magnor, M. 2005. Garment motion capture using color-coded patterns. In Proc. Eurographics EG'05.Google Scholar
- Shi, X., Zhou, K., Tong, Y., Desbrun, M., Bao, H., and Guo, B. 2008. Example-based dynamic skinning in real time. ACM TOG (Proc. SIGGRAPH '08). Google ScholarDigital Library
- Starck, J., and Hilton, A. 2007. Surface capture for performance based animation. IEEE CGAA 27(3), 21--31. Google ScholarDigital Library
- Vlasic, D., Baran, I., Matusik, W., and Popović, J. 2008. Articulated mesh animation from multi-view silhouettes. ACM TOG (Proc. SIGGRAPH '08). Google ScholarDigital Library
- Vlasic, D., Peers, P., Baran, I., Debevec, P., Popović, J., Rusinkiewicz, S., and Matusik, W. 2009. Dynamic shape capture using multi-view photometric stereo. In ACM TOG (Proc. SIGGRAPH Asia '09). Google ScholarDigital Library
- Waschbüsch, M., Würmlin, S., Cotting, D., Sadlo, F., and Gross, M. 2005. Scalable 3D video of dynamic scenes. In Proc. Pacific Graphics, 629--638.Google Scholar
- White, R., Crane, K., and Forsyth, D. 2007. Capturing and animating occluded cloth. In ACM TOG (Proc. SIGGRAPH'07). Google ScholarDigital Library
- Xu, W., Zhou, K., Yu, Y., Tan, Q., Peng, Q., and Guo, B. 2007. Gradient domain editing of deforming mesh sequences. In ACM TOG (Proc. SIGGRAPH '07). Google ScholarDigital Library
- Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-quality video view interpolation using a layered representation. ACM TOG (Proc. SIGGRAPH '04). Google ScholarDigital Library
Index Terms
- Video-based reconstruction of animatable human characters
Recommendations
Video-based reconstruction of animatable human characters
SIGGRAPH ASIA '10: ACM SIGGRAPH Asia 2010 papersWe present a new performance capture approach that incorporates a physically-based cloth model to reconstruct a rigged fully-animatable virtual double of a real person in loose apparel from multi-view video recordings. Our algorithm only requires a ...
Data-driven detailed hair animation for game characters
We propose a data-driven method to realize high-quality detailed hair animations in interactive applications like games. By devising an error metric method to evaluate hair animation similarities, we take hair features into consideration as much as ...
Non-humanoid creature performance from human acting
SIGGRAPH '16: ACM SIGGRAPH 2016 PostersWe propose a framework for using human acting as input for the animation of non-humanoid creatures; captured motion is classified using machine learning techniques, and a combination of preexisting clips and motion retargeting are used to synthetize new ...
Comments