Eftychios Sifakis
Skip Abstract Section
Skip Supplemental Material SectionAbstract
We built an anatomically accurate model of facial musculature, passive tissue and underlying skeletal structure using volumetric data acquired from a living male subject. The tissues are endowed with a highly nonlinear constitutive model including controllable anisotropic muscle activations based on fiber directions. Detailed models of this sort can be difficult to animate requiring complex coordinated stimulation of the underlying musculature. We propose a solution to this problem automatically determining muscle activations that track a sparse set of surface landmarks, e.g. acquired from motion capture marker data. Since the resulting animation is obtained via a three dimensional nonlinear finite element method, we obtain visually plausible and anatomically correct deformations with spatial and temporal coherence that provides robustness against outliers in the motion capture data. Moreover, the obtained muscle activations can be used in a robust simulation framework including contact and collision of the face with external objects.
Supplemental Material
- Basu, S., Oliver, N., and Pentland, A. 1998. 3D lip shapes from video: a combined physical-statistical model. Speech Communication 26, 131--148. Google Scholar
Digital Library
- Basu, S., Oliver, N. and Pentland, A. 1998. 3D modeling and tracking of human lip motions. IEEE Computer Society, 337--343. Google ScholarDigital Library
- Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Proc. of ACM SIGGRAPH, ACM Press, 187--194. Google ScholarDigital Library
- Blanz, V., and Vetter, T. 2003. Face recognition based on fitting a 3D morphable model. IEEE Trans. on Pattern Analysis and Machine Intelligence 29, 9, 1063--1074. Google ScholarDigital Library
- Blanz, V., Basso, C., Poggio, T., and Vetter, T. 2003. Reanimating faces in images and video. In Proc. of Eurographics, vol. 22.Google Scholar
- Blanz, V., Scherbaum, K., Vetter, T., and Seidel, H. P. 2004. Exchanging faces in images. In Proc. of Eurographics, vol. 23.Google Scholar
- Borshukov, G., Piponi, D., Larsen, O., Lewis, J. P., and Tempelaar-Lietz, C. 2003. Universal Capture - image-based facial animation for "The Matrix Reloaded". In ACM SIGGRAPH 2003 Sketches & Applications, ACM Press, 1--1. Google ScholarDigital Library
- Brand, M. 1999. Voice puppetry. In Proc. of ACM SIGGRAPH, 21--28. Google ScholarDigital Library
- Bregler, C., Covell, M., and Slaney, M. 1997. Video Rewrite: driving visual speech with audio. In Proc. of ACM SIGGRAPH, 353--360. Google ScholarDigital Library
- Byun, M., and Badler, N. I. 2002. FacEMOTE: Qualitative parametric modifiers for facial animations. In Proc. of ACM SIGGRAPH/Eurographics Symp. on Comput. Anim., ACM Press, 65--71. Google ScholarDigital Library
- Cao, Y., Faloutsos, P., and Pighin, F. 2003. Unsupervised learning for speech motion editing. In Proc. of the ACM SIGGRAPH/Eurographics Symp. on Comput. Anim., 225--231. Google ScholarDigital Library
- Cao, Y., Faloutsos, P., Kohler, E., and Pighin, F. 2004. Real-time speech motion synthesis from recorded motions. In Proc. of 2003 ACM SIGGRAPH/Eurographics Symp. on Comput. Anim., 347--355. Google ScholarDigital Library
- Cassell, J., Pelachaud, C., Badler, N., Steedman, M., Achorn, B., Becket, T., Doubille, B., Prevost, S., and Stone, M. 1994. Animated conversation: Rule-based generation of facial expression, gesture and spoken intonation for multiple conversational agents. In Proc. of ACM SIGGRAPH, ACM Press, 413--420. Google ScholarDigital Library
- Cassell, J., Vilhjálmsson, H. H., and Bickmore, T. 2001. BEAT: the Behavior Expression Animation Toolkit. In Proc. of ACM SIGGRAPH, 477--486. Google ScholarDigital Library
- Chai, J., Xiao, J., and Hodgins, J. 2003. Vision-based control of 3D facial animation. In Proc. of ACM SIGGRAPH/Eurographics Symp. on Comput. Anim., 193--206. Google ScholarDigital Library
- Choe, B., and Ko, H.-S. 2001. Analysis and synthesis of facial expressions with hand-generated muscle actuation basis. In Proc. of Comput. Anim., 12--19. Google ScholarDigital Library
- Choe, B., Lee, H., and Ko, H.-S. 2001. Performance-driven muscle-based facial animation. J. Vis. and Comput. Anim. 12, 67--79.Google ScholarCross Ref
- DeCarlo, D., Metaxas, D., and Stone, M. 1998. An anthropometric face model using variational techniques. In Proc. of ACM SIGGRAPH, ACM Press, 67--74. Google ScholarDigital Library
- Ekman, P., and Friesen, W. V. 1978. Facial Action Coding System. Consulting Psychologist Press, Palo Alto.Google Scholar
- Essa, I., and Pentland, A. 1997. Coding, analysis, interpretation, and recognition of facial expressions. IEEE Trans. on Pattern Analysis and Machine Intelligence 19, 7, 757-763. Google ScholarDigital Library
- Essa, I., Basu, S., Darrell, T., and Pentland, A. 1996. Modeling, tracking and interactive animation of faces and heads using input from video. In Proc. of Computer Animation, IEEE Computer Society, 68--79. Google ScholarDigital Library
- Ezzat, T., Geiger, G., and Poggio, T. 2002. Trainable videorealistic speech animation. In ACM Transactions on Graphics, ACM Press, vol. 21, 388--398. Google ScholarDigital Library
- Gill, P. E., Murray, W., and Wright, M. H. 1981. Practical Optimization. Academic Press, San Diego, USA.Google Scholar
- Guenter, B., Grimm, C., Wood, D., Malvar, H., and Pighin, F. 1998. Making faces. In Proc. ACM SIGGRAPH. ACM Press, 55--66. Google ScholarDigital Library
- Heidelberger, B., Teschner, M., Keiser, R., Müller, M., and Gross, M. 2004. Consistent penetration depth estimation for deformable collision response. In Proc. of Vision, Modeling, Visualization (VMV), 339--346.Google Scholar
- Joshi, P., Tien, W. C., Desbrun, M., and Pighin, F. 2003. Learning controls for blend shape based realistic facial animation. In Proc. ACM SIGGRAPH/Eurographics Symp. on Comput. Anim., 365--373. Google ScholarDigital Library
- Kahler, K., Haber, J., and Seidel, H.-P. 2001. Geometry-based muscle modeling for facial animation. In Proc. of Graphics Interface, 37--46. Google ScholarDigital Library
- Kahler, K., Haber, J., Yamauchi, H., and Seidel, H.-P. 2002. Head shop: Generating animated head models with anatomical structure. In Proc. of ACM SIGGRAPH/Eurographics Symp. on Comput. Anim., 55--63. Google ScholarDigital Library
- Kahler, K., Haber, J., and Seidel, H.-P. 2003. Reanimating the dead: Reconstruction of expressive faces from skull data. In ACM Trans. on Graphics, vol. 22, 554--561. Google ScholarDigital Library
- Kalra, P., Mangili, A., Magnetat-Thalmann, N., and Thalmann, D. 1992. Simulation of facial muscle actions based on rational free form deformations. In Proc. of Eurographics, 59--69.Google Scholar
- Keeve, E., Girod, S., Pfeifle, P., and Girod, B. 1996. Anatomy-based facial tissue modeling using the finite element method. In Proc. of Visualization, 21--28. Google ScholarDigital Library
- Koch, R. M., Gross, M. H., Carls, F. R., von Buren, D. F., Fankhauser, G., and Parish, Y. I. H. 1996. Simulating facial surgery using finite element models. Computer Graphics 30, Annual Conference Series, 421--428. Google ScholarDigital Library
- Koch, R., Gross, M., and Bosshard, A. 1998. Emotion editing using finite elements. Proceedings of Eurographics 1998 17, 3.Google Scholar
- Kshirsagar, S., and Magnenat-Thalmann, N. 2003. Visyllable based speech animation. In Proc. of Eurographics, vol. 22.Google Scholar
- Lee, Y., Terzopoulos, D., and Waters, K. 1995. Realistic modeling for facial animation. Comput. Graph. (SIGGRAPH Proc.), 55--62. Google ScholarDigital Library
- Lee, S. P., Badler, J. B., and Badler, N. I. 2002. Eyes alive. In Proc. of ACM SIGGRAPH, ACM Press, 637--644. Google ScholarDigital Library
- Magnenat-Thalmann, N., Primeau, E., and Thalmann, D. 1988. Abstract muscle action procedures for human face animation. The Visual Computer 3, 5, 290--297.Google ScholarCross Ref
- Morishima, S., Ishikawa, T., and Terzopoulos, D. 1998. Facial muscle parameter decision from 2D frontal image. In Proc. of the Int. Conf. on Pattern Recognition, vol. 1, 160--162. Google ScholarDigital Library
- Moving Picture Experts Group, 1998. Information technology - coding of audio-visual objects part 2: Visual. Final draft of international standard ISO/IEC JTC1/SC29/WG11 N2501 14496--2.Google Scholar
- Na, K., and Jung, M. 2004. Hierarchical retargetting of fine facial motions. In Proc. of Eurographics, vol. 23.Google Scholar
- Noh, J., and Neumann, U. 2001. Expression cloning. In Proc. of ACM SIGGRAPH, ACM Press, E. Fiume, Ed., 277--288. Google ScholarDigital Library
- Parke, F. I., and Waters, K. 1996. Computer Facial Animation. AK Peters, Ltd. Google ScholarDigital Library
- Parke, F. I. 1972. Computer generated animation of faces. In Proc. of ACM Conference, ACM Press, 451--457. Google ScholarDigital Library
- Pieper, S., Rosen, J., and Zeltzer, D. 1992. Interactive graphics for plastic surgery: A task-level analysis and implementation. In Proc. of Symp. on Interactive 3D graphics, ACM Press, 127--134. Google ScholarDigital Library
- Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., and Salesin, D. H. 1998. Synthesizing realistic facial expressions from photographs. In Proc. of ACM SIGGRAPH, ACM Press, 75--84. Google ScholarDigital Library
- Pighin, F., Szeliski, R., and Salesin, D. 1999. Resynthesizing facial animation through 3D model-based tracking. In Proc. of Int. Conf. on Comput. Vision, 143--150.Google Scholar
- Platt, S. M., and Badler, N. I. 1981. Animating facial expressions. Comput. Graph. (SIGGRAPH Proc.), 245--252. Google ScholarDigital Library
- Pyun, H., Kim, Y., Chae, W., Kang, H. W., and Shin, S. Y. 2003. An example-based approach for facial expression cloning. In Proc. of ACM SIGGRAPH/Eurographics Symp. on Comput. Anim., 167--176. Google ScholarDigital Library
- Roth, S. H., Gross, M., Turello, M. H., and Carls, S. 1998. A Bernstein-Bézier based approach to soft tissue simulation. In Proc. of Eurographics, vol. 17, 285--294.Google ScholarCross Ref
- Sumner, R., and Popović, J. 2004. Deformation transfer for triangle meshes. In Proc. of ACM SIGGRAPH, vol. 23, 32--39. Google ScholarDigital Library
- Teran, J., Blemker, S., Ng, V., and Fedkiw, R. 2003. Finite volume methods for the simulation of skeletal muscle. In Proc. of the 2003 ACM SIGGRAPH/Eurographics Symp. on Comput. Anim., 68--74. Google ScholarDigital Library
- Teran, J., Sifakis, E., Irving, G., and Fedkiw, R. 2005. Robust quasistatic finite elements and flesh simulation. ACM Trans. on Graphics (to appear).Google Scholar
- Teran, J., Sifakis, E., Salinas-Blemker, S., Ng-Thow-Hing, V., Lau, C., and Fedkiw, R. 2005. Creating and simulating skeletal muscle from the visible human data set. IEEE Trans. on Vis. and Comput. Graph. 11, 3, 317--328. Google ScholarDigital Library
- Terzopoulos, D., and Waters, K. 1993. Analysis and synthesis of facial image sequences using physical and anatomical models. IEEE Trans. on Pattern Analysis and Machine Intelligence 15, 6. Google ScholarDigital Library
- Teschner, M., Girod, S., and Girod, B. 2000. Direct computation of nonlinear soft-tissue deformation. In Proc. of Vision, Modeling, and Visualization, 383--390.Google Scholar
- Teschner, M., Heidelberger, B., Müller, M., Pomeranets, D., and Gross, M. 2003. Optimized spatial hashing for collision detection of deformable objects. In Proc. of Vision, Modeling, Visualization (VMV), 47--54.Google Scholar
- U.S. NATIONAL LIBRARY OF MEDICINE, 1994. The visible human project. http://www.nlm.nih.gov/research/visible/.Google Scholar
- Wang, Y., Huang, X., Lee, C. S., Zhang, S., Li, Z., Samaras, D., Metaxas, D., Elgammal, A., and Huang, P. 2004. High resolution acquisition, learning and transfer of dynamic 3-D facial expressions. In Proc. of Eurographics, 677--686.Google Scholar
- Waters, K., and Frisbie, J. 1995. A coordinated muscle model for speech animation. In Proc. of Graphics Interface, 163--170.Google Scholar
- Waters, K. 1987. A muscle model for animating three-dimensional facial expressions. Comput. Graph. (SIGGRAPH Proc.), 17--24. Google ScholarDigital Library
- Williams, L. 1990. Performance-driven facial animation. In Computer Graphics (Proc. of Int. Conf. on Computer Graphics and Interactive Techniques), ACM Press, 235--242. Google ScholarDigital Library
- Wu, Y., Kalra, P., Magnenat-Thalmann, N., and Thalmann, D. 1999. Simulating wrinkles and skin aging. The Visual Computer 15, 4, 183--198.Google ScholarCross Ref
- Zajac, F. 1989. Muscle and tendon: Properties, models, scaling, and application to biomechanics and motor control. Critical Reviews in Biomed. Eng. 17, 4, 359--411.Google Scholar
- Zhang, Q., Liu, Z., Guo, B., and Shum, H. 2003. Geometry-driven photorealistic facial expression synthesis. In Proc. of ACM SIGGRAPH/Eurographics Symp. on Comput. Anim., ACM Press, 16--22. Google ScholarDigital Library
- Zhang, L., Snavely, N., Curless, B., and Seitz, S. 2004. Spacetime faces: High resolution capture for modeling and animation. In Proc. of ACM SIGGRAPH, ACM Press, vol. 23, 548--558. Google ScholarDigital Library
Index Terms
- Automatic determination of facial muscle activations from sparse motion capture marker data
Recommendations
Automatic determination of facial muscle activations from sparse motion capture marker data
SIGGRAPH '05: ACM SIGGRAPH 2005 PapersWe built an anatomically accurate model of facial musculature, passive tissue and underlying skeletal structure using volumetric data acquired from a living male subject. The tissues are endowed with a highly nonlinear constitutive model including ...
Muscle-based facial retargeting with anatomical constraints
SIGGRAPH '19: ACM SIGGRAPH 2019 TalksWe present a physically based facial retargeting algorithm that is suitable for use in high-end production. Given an actor's facial performance, we first run a targeted muscle simulation on the actor in order to determine the actor blendshape muscles ...
Orthogonal-Blendshape-Based Editing System for Facial Motion Capture Data
The authors present a novel data-driven 3D facial motion capture data editing system using automated construction of an orthogonal blendshape face model and constrained weight propagation, aiming to bridge the popular facial motion capture technique and ...
Comments