ABSTRACT
We present a fast, fully automatic morphing algorithm for creating simulatable flesh and muscle models for human and humanoid faces. Current techniques for creating such models require a significant amount of time and effort, making them infeasible or impractical. In fact, the vast majority of research papers use only a floating mask with no inner lips, teeth, tongue, eyelids, eyes, head, ears, etc.---and even those that build the full visual model would typically still lack the cranium, jaw, muscles, and other internal anatomy. Our method requires only the target surface mesh as input and can create a variety of models in only a few hours with no user interaction. We start with a symmetric, high resolution, anatomically accurate template model that includes auxiliary information such as feature points and curves. Then given a target mesh, we automatically orient it to the template, detect feature points, and use these to bootstrap the detection of corresponding feature curves. These curve correspondences are used to deform the surface mesh of the template model to match the target mesh. Then, the calculated displacements of the template surface mesh are used to drive a three-dimensional morph of the full template model including all interior anatomy. The resulting target model can be simulated to generate a large range of expressions that are consistent across characters using the same muscle activations. Full automation of this entire process makes it readily available to a wide range of users.
Supplemental Material
Available for Download
- 3Lateral, 2015. 3Lateral Studio. 3lateral.com.Google Scholar
- Akimoto, T., Suenaga, Y., and Wallace, R. S. 1993. Automatic creation of 3d facial models. IEEE Comput. Graph. Appl. 13, 5 (Sept.), 16--22. Google ScholarDigital Library
- Ali-Hamadi, D., Liu, T., Gilles, B., Kavan, L., Faure, F., Palombi, O., and Cani, M.-P. 2013. Anatomy transfer. In ACM SIGGRAPH Asia 2013 papers, SIGGRAPH ASIA '13, 188:1--188:8. Google ScholarDigital Library
- Autodesk, 2011. Face robot. http://softimage.wiki.softimage.com/xsidocs/face_cover.htm.Google Scholar
- Baraff, D., Witkin, A., and Kass, M. 2003. Untangling cloth. ACM Trans. Graph. (SIGGRAPH Proc.) 22, 862--870. Google ScholarDigital Library
- Beeler, T., and Bradley, D. 2014. Rigid stabilization of facial expressions. ACM Trans. Graph. 33, 4 (July), 44:1--44:9. Google ScholarDigital Library
- Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graph. (SIGGRAPH Proc.) 29, 3, 40:1--40:9. Google ScholarDigital Library
- Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R. W., and Gross, M. 2011. High-quality passive facial performance capture using anchor frames. ACM Trans. Graph. (SIGGRAPH Proc.) 30, 4, 75:1--75:10. Google ScholarDigital Library
- Besl, P. J., and McKay, N. D. 1992. Method for registration of 3-d shapes. In Robotics-DL tentative, International Society for Optics and Photonics, 586--606.Google Scholar
- Bhat, K. S., Goldenthal, R., Ye, Y., Mallet, R., and Koperwas, M. 2013. High fidelity facial animation capture and retargeting with contours. In ACM SIGGRAPH/Eurographics Symp. on Comput. Anim., 7--14. Google ScholarDigital Library
- Bridson, R., Fedkiw, R., and Anderson, J. 2002. Robust treatment of collisions, contact and friction for cloth animation. ACM Trans. Graph. (SIGGRAPH Proc.) 21, 3, 594--603. Google ScholarDigital Library
- Brown, B. J., and Rusinkiewicz, S. 2007. Global non-rigid alignment of 3-d scans. In ACM TOG, vol. 26, ACM, 21. Google ScholarDigital Library
- Fyffe, G., Jones, A., Alexander, O., Ichikari, R., and Debevec, P. 2015. Driving high-resolution facial scans with video performance capture. ACM TOG 34, 1. Google ScholarDigital Library
- Garrido, P., Valgaerts, L., Wu, C., and Theobalt, C. 2013. Reconstructing detailed dynamic face geometry from monocular video. ACM Trans. Graph. 32, 6, 158. Google ScholarDigital Library
- Gibou, F., Fedkiw, R., Cheng, L.-T., and Kang, M. 2002. A second order accurate symmetric discretization of the Poisson equation on irregular domains. J. Comput. Phys. 176, 205--227. Google ScholarDigital Library
- Kemelmacher-Shlizerman, I., and Basri, R. 2011. 3d face reconstruction from a single image using a single reference face shape. Pattern Analysis and Machine Intelligence, IEEE Transactions on 33, 2 (Feb), 394--405. Google ScholarDigital Library
- Lewis, J. P., Anjyo, K., Rhee, T., Zhang, M., Pighin, F., and Deng, Z. 2014. Practice and Theory of Blendshape Facial Models. In Eurographics 2014 - State of the Art Reports, The Eurographics Association, S. Lefebvre and M. Spagnuolo, Eds.Google Scholar
- Li, H., Adams, B., Guibas, L. J., and Pauly, M. 2009. Robust single-view geometry and motion reconstruction. In ACM TOG, vol. 28, ACM, 175. Google ScholarDigital Library
- Molino, N., Bridson, R., Teran, J., and Fedkiw, R. 2003. A crystalline, red green strategy for meshing highly deformable objects with tetrahedra. In 12th Int. Meshing Roundtable, 103--114.Google Scholar
- Noh, J., and Neumann, U. 2001. Expression cloning. In Proc. of ACM SIGGRAPH, ACM Press, E. Fiume, Ed., 277--288. Google ScholarDigital Library
- Osher, S., and Fedkiw, R. 2002. Level Set Methods and Dynamic Implicit Surfaces. Springer-Verlag. New York, NY.Google Scholar
- Pauly, M., Mitra, N. J., Giesen, J., Gross, M., and Guibas, L. J. 2005. Example-based 3d scan completion. In Proceedings of the Third Eurographics Symposium on Geometry Processing, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SGP '05. Google ScholarDigital Library
- Qiu, L., Yu, Y., and Fedkiw, R. 2015. On thin gaps between rigid bodies two-way coupled to incompressible flow. J. Comput. Phys. 292, 0, 1--29. Google ScholarDigital Library
- Selle, A., Lentine, M., and Fedkiw, R. 2008. A mass spring model for hair simulation. ACM Trans. Graph. (SIGGRAPH Proc.) 27, 3 (Aug.), 64.1--64.11. Google ScholarDigital Library
- Shi, F., Wu, H.-T., Tong, X., and Chai, J. 2014. Automatic acquisition of high-fidelity facial performances using monocular videos. ACM Trans. Graph. 33, 6 (Nov.), 222:1--222:13. Google ScholarDigital Library
- Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. (SIGGRAPH Proc.) 24, 3. Google ScholarDigital Library
- Sifakis, E., Selle, A., Robinson-Mosher, A., and Fedkiw, R. 2006. Simulating speech with a physics-based facial muscle model. ACM SIGGRAPH/Eurographics Symp. on Comput. Anim., 261--270. Google ScholarDigital Library
- Sorkine, O., and Alexa, M. As-rigid-as-possible surface modeling.Google Scholar
- Sumner, R., and Popović, J. 2004. Deformation transfer for triangle meshes. In ACM Trans. on Graph. (Proc. ACM SIGGRAPH), vol. 23, 399--405. Google ScholarDigital Library
- Taubin, G. 1995. Estimating the tensor of curvature of a surface from a polyhedral approximation. ICCV, 902--907. Google ScholarDigital Library
- Teran, J., Blemker, S., Ng, V., and Fedkiw, R. 2003. Finite volume methods for the simulation of skeletal muscle. In Proc. of the 2003 ACM SIGGRAPH/Eurographics Symp. on Comput. Anim., 68--74. Google ScholarDigital Library
- Teran, J., Sifakis, E., Irving, G., and Fedkiw, R. 2005. Robust quasistatic finite elements and flesh simulation. Proc. of the 2005 ACM SIGGRAPH/Eurographics Symp. on Comput. Anim., 181--190. Google ScholarDigital Library
- Zhang, L., Snavely, N., Curless, B., and Seitz, S. 2004. Spacetime faces: High resolution capture for modeling and animation. In ACM Trans. Graph. (Proc. ACM SIGGRAPH), ACM Press, vol. 23, 548--558. Google ScholarDigital Library
- Zhao, H.-K., Osher, S., and Fedkiw, R. 2001. Fast surface reconstruction using the level set method. In 1st IEEE Wrkshp. on Variational and Level Set Meth., 8th Int. Conf. on Comput. Vis., 194--202. Google ScholarDigital Library
Index Terms
- Fully automatic generation of anatomical face simulation models
Recommendations
Muscle-based facial retargeting with anatomical constraints
SIGGRAPH '19: ACM SIGGRAPH 2019 TalksWe present a physically based facial retargeting algorithm that is suitable for use in high-end production. Given an actor's facial performance, we first run a targeted muscle simulation on the actor in order to determine the actor blendshape muscles ...
Muscle simulation for facial animation in Kong: Skull Island
SIGGRAPH '17: ACM SIGGRAPH 2017 TalksFor Kong: Skull Island, Industrial Light & Magic created an anatomically motivated facial simulation model for Kong that includes the facial skeleton and musculature. We applied a muscle simulation framework that allowed us to target facial shapes while ...
Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models
An approach to the analysis of dynamic facial images for the purposes of estimating and resynthesizing dynamic facial expressions is presented. The approach exploits a sophisticated generative model of the human face originally developed for realistic ...
Comments