Matthew Loper
Abstract
Marker-based motion capture (mocap) is widely criticized as producing lifeless animations. We argue that important information about body surface motion is present in standard marker sets but is lost in extracting a skeleton. We demonstrate a new approach called MoSh (Motion and Shape capture), that automatically extracts this detail from mocap data. MoSh estimates body shape and pose together using sparse marker data by exploiting a parametric model of the human body. In contrast to previous work, MoSh solves for the marker locations relative to the body and estimates accurate body shape directly from the markers without the use of 3D scans; this effectively turns a mocap system into an approximate body scanner. MoSh is able to capture soft tissue motions directly from markers by allowing body shape to vary over time. We evaluate the effect of different marker sets on pose and shape accuracy and propose a new sparse marker set for capturing soft-tissue motion. We illustrate MoSh by recovering body shape, pose, and soft-tissue motion from archival mocap data and using this to produce animations with subtlety and realism. We also show soft-tissue motion retargeting to new characters and show how to magnify the 3D deformations of soft tissue to create animations with appealing exaggerations.
Supplemental Material
Available for Download
Supplemental material.
- Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. ACM Trans. Graph. (Proc. SIGGRAPH) 22, 3, 587--594. Google Scholar
Digital Library
- Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: Shape Completion and Animation of PEople. ACM Trans. Graph. (Proc. SIGGRAPH 24, 3, 408--416. Google ScholarDigital Library
- Bogo, F., Romero, J., Loper, M., and Black, M. J. 2014. FAUST: Dataset and evaluation for 3D mesh registration. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). Google ScholarDigital Library
- de Aguiar, E., Theobalt, C., Stoll, C., and Seidel, H.-P. 2007. Marker-less deformable mesh tracking for human shape and motion capture. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 1--8.Google Scholar
- de Aguiar, E., Zayer, R., Theobalt, C., Seidel, H. P., and Magnor, M. 2007. A simple framework for natural animation of digitized models. In Computer Graphics and Image Processing, 2007. SIBGRAPI 2007. XX Brazilian Symposium on, 3--10. Google ScholarDigital Library
- de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Trans. Graph. (Proc. SIGGRAPH) 27, 3 (Aug.), 98:1--98:10. Google ScholarDigital Library
- Griewank, A., and Walther, A. 2008. Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, second ed. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA. Google ScholarDigital Library
- Hirshberg, D. A., Loper, M., Rachlin, E., and Black, M. J. 2012. Coregistration: Simultaneous alignment and modeling of articulated 3d shape. In Computer Vision ECCV 2012, A. Fitzgibbon, S. Lazebnik, P. Perona, Y. Sato, and C. Schmid, Eds., vol. 7577 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 242--255. Google ScholarDigital Library
- Hong, Q. Y., Park, S. I., and Hodgins, J. K. 2010. A data-driven segmentation for the shoulder complex. Computer Graphics Forum 29, 2, 537--544.Google ScholarCross Ref
- Jain, A., Thormählen, T., Seidel, H.-P., and Theobalt, C. 2010. MovieReshape: Tracking and reshaping of humans in videos. ACM Transactiosn on Graphics (Proc. SIGGRAPH) 29, 6 (Dec.), 148:1--148:10. Google ScholarDigital Library
- Kwon, J.-Y., and Lee, I.-K. 2007. Rubber-like exaggeration for character animation. In Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, IEEE Computer Society, Washington, DC, USA, PG '07, 18--26. Google ScholarDigital Library
- Leardini, A., Chiari, L., Croce, U. D., and Cappozzo, A. 2005. Human movement analysis using stereophotogrammetry: Part 3. soft tissue artifact assessment and compensation. Gait & Posture 21, 2, 212--225. Google ScholarDigital Library
- Liu, Y., Gall, J., Stoll, C., Dai, Q., Seidel, H.-P., and Theobalt, C. 2013. Markerless motion capture of multiple characters using multiview image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 11, 2720--2735. Google ScholarDigital Library
- Livne, M., Sigal, L., Troje, N., and Fleet, D. 2012. Human attributes from 3D pose tracking. Computer Vision and Image Understanding 116, 5, 648--660. Google ScholarDigital Library
- Loper, M., 2014. Chumpy autodifferentiation library. http://chumpy.org/.Google Scholar
- Neumann, T., Varanasi, K., Hasler, N., Wacker, M., Magnor, M., and Theobalt, C. 2013. Capture and statistical modeling of arm-muscle deformations. Computer Graphics Forum 32, 2 (May), 285--294.Google ScholarCross Ref
- Neumann, T., Varanasi, K., Wenger, S., Wacker, M., Magnor, M., and Theobalt, C. 2013. Sparse localized deformation components. ACM Trans. Graph. 32, 6 (Nov.), 179:1--179:10. Google ScholarDigital Library
- Nocedal, J., and Wright, S. J. 2006. Numerical Optimization, 2nd ed. Springer, New York.Google Scholar
- Park, S. I., and Hodgins, J. K. 2006. Capturing and animating skin deformation in human motion. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3 (July), 881--889. Google ScholarDigital Library
- Park, S. I., and Hodgins, J. K. 2008. Data-driven modeling of skin and muscle deformation. ACM Trans. Graph. (Proc. SIGGRAPH) 27, 3 (Aug.), 96:1--96:6. Google ScholarDigital Library
- Robinette, K., Blackwell, S., Daanen, H., Boehmer, M., Fleming, S., Brill, T., Hoeferlin, D., and Burnsides, D. 2002. Civilian American and European Surface Anthropometry Resource (CAESAR) final report. Tech. Rep. AFRL-HE-WP-TR-2002-0169, US Air Force Research Laboratory. Google ScholarDigital Library
- Stark, J., and Hilton, A. 2007. Surface capture for performance-based animation. IEEE Computer Graphics and Applications 27, 3, 21--31. Google ScholarDigital Library
- Tsoli, A., Mahmood, N., and Black, M. J. 2014. Breathing life into shape: Capturing, modeling and animating 3D human breathing. ACM Trans. Graph., (Proc. SIGGRAPH) 33, 4 (July), 52:1--52:11. Google ScholarDigital Library
- Wadhwa, N., Rubinstein, M., Durand, F., and Freeman, W. T. 2013. Phase-based video motion processing. ACM Trans. Graph., (Proc. SIGGRAPH) 32, 4 (July), 80:1--80:10. Google ScholarDigital Library
- Wang, H., Xu, N., Raskar, R., and Ahuja, N. 2007. Videoshop: A new framework for spatio-temporal video editing in gradient domain. Graph. Models 69, 1, 57--70. Google ScholarDigital Library
- Wu, H.-Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., and Freeman, W. T. 2012. Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph. (Proc. SIGGRAPH) 31, 4 (July), 65:1--65:8. Google ScholarDigital Library
Index Terms
- MoSh: motion and shape capture from sparse markers
Recommendations
Dyna: a model of dynamic human shape in motion
To look human, digital full-body avatars need to have soft-tissue deformations like those of real people. We learn a model of soft-tissue deformations from examples using a high-resolution 4D capture system and a method that accurately registers a ...
Capturing and animating skin deformation in human motion
SIGGRAPH '06: ACM SIGGRAPH 2006 PapersDuring dynamic activities, the surface of the human body moves in many subtle but visually significant ways: bending, bulging, jiggling, and stretching. We present a technique for capturing and animating those motions using a commercial motion capture ...
Capturing and animating skin deformation in human motion
During dynamic activities, the surface of the human body moves in many subtle but visually significant ways: bending, bulging, jiggling, and stretching. We present a technique for capturing and animating those motions using a commercial motion capture ...
Comments