Frédéric Pighin
Dani Lischinski
Richard Szeliski
ABSTRACT
We present new techniques for creating photorealistic textured 3D facial models from photographs of a human subject, and for creating smooth transitions between different facial expressions by morphing between these different models. Starting from several uncalibrated views of a human subject, we employ a user-assisted technique to recover the camera poses corresponding to the views as well as the 3D coordinates of a sparse set of chosen locations on the subject's face. A scattered data interpolation technique is then used to deform a generic face mesh to fit the particular geometry of the subject's face. Having recovered the camera poses and the facial geometry, we extract from the input images one or more texture maps for the model. This process is repeated for several facial expressions of a particular subject. To generate transitions between these facial expressions we use 3D shape morphing between the corresponding face models, while at the same time blending the corresponding textures. Using our technique, we have been able to generate highly realistic face models and natural looking animations.
- Takaaki Akimoto, Yasuhito Suenaga, and Richard S. Wallace. Automatic Creation of 3D Facial Models. IEEE Computer Graphics and Applications, 13(5):16--22, September 1993. Google Scholar
Digital Library
- Alias | Wavefront, Toronto, Ontario. Alias V7.0, 1995.Google Scholar
- Thaddeus Beier and Shawn Neely. Feature-based Image Metamorphosis. In SIGGRAPH 92 Conference Proceedings, pages 35--42. ACM SIGGRAPH, July 1992. Google ScholarDigital Library
- Philippe Bergeron and Pierre Lachapelle. Controlling Facial Expressions and Body Movements in the Computer-Generated Animated Short "Tony De Peltrie". In SIGGRAPH 85 Advanced Computer Animation seminar notes. July 1985.Google Scholar
- Christoph Bregler, Michele Covell, and Malcolm Slaney. Video Rewrite: Driving Visual Speech with Audio. In SIGGRAPH 97 Conference Proceedings, pages 353--360. ACM SIGGRAPH, August 1997. Google ScholarDigital Library
- David T. Chen, Andrei State, and David Banks. Interactive Shape Metamorphosis. In 1995 Symposium on Interactive 3D Graphics, pages 43--44. ACM SIGGRAPH, April 1995. Google ScholarDigital Library
- Chang S. Choi, Kiyoharu, Hiroshi Harashima, and Tsuyoshi Takebe. Analysis and Synthesis of Facial Image Sequences in Model-Based Image Coding. In IEEE Transactions on Circuits and Systems for Video Technology, volume 4, pages 257--275. June 1994.Google ScholarDigital Library
- Cyberware Laboratory, Inc, Monterey, California. 4020/RGB 3D Scanner with Color Digitizer, 1990.Google Scholar
- Paul E. Debevec, Camillo J. Taylor, and Jitendra Malik. Modeling and Rendering Architecture from Photographs: A Hybrid Geometry- and Image-Based Approach. In SIGGRAPH 96 Conference Proceedings, pages 11--20. ACM SIGGRAPH, August 1996. Google ScholarDigital Library
- Eben Ostby, Pixar Animation Studios. Personal communication, January 1997.Google Scholar
- Paul Ekman and Wallace V. Friesen. Unmasking the Face. A guide to recognizing emotions fron facial clues. Prentice-Hall, Inc., Englewood Cliffs, New Jersey, 1975.Google Scholar
- Paul Ekman and Wallace V. Friesen. Manual for the Facial Action Coding System. Consulting Psychologists Press, Inc., Palo Alto, California, 1978.Google Scholar
- Man Essa, Sumit Basu, Trevor Darrell, and Alex Pentland. Modeling, Tracking and Interactive Animation of Faces and Heads Using Input from Video. In Computer Animation Conference, pages 68--79. June 1996. Google ScholarDigital Library
- Gary Faigin. The Artist's Complete Guide to Facial Expression. Watson-Guptill Publications, New York, 1990.Google Scholar
- Olivier Faugeras. Three-Dimensional Computer Vision: A Geometric Viewpoint. MIT Press, Cambridge, Massachusetts, 1993. Google ScholarDigital Library
- G. Golub and C. F. Van Loan. Matrix Computation, third edition. The John Hopkins University Press, Baltimore and London, 1996. Google ScholarDigital Library
- Brian Guenter, Cindy Grimm, Daniel Wood, Henrique Malvar, and Frédéric Pighin. Making Faces. In SIGGRAPH 98 Conference Proceedings. ACM SIGGRAPH, July 1998. Google ScholarDigital Library
- Pat Hanrahan and Wolfgang Krueger. Reflection from Layered Surfaces Due to Subsurface Scattering. In SIGGRAPH 93 Conference Proceedings, volume 27, pages 165--174. ACM SIGGRAPH, August 1993. Google ScholarDigital Library
- Bright Star Technologies Inc. Beginning Reading Software. Sierra On-Line, Inc., 1993.Google Scholar
- Horace H. S. Ip and Lijun Yin. Constructing a 3D Individualized Head Model from Two Orthogonal Views. The Visual Computer, 12:254--266, 1996.Google ScholarCross Ref
- Gregory Ward J., Francis M. Rubinstein, and Robert D. Clear. A Ray Tracing Solution for Diffuse Interreflection. In SIGGRAPH 88 Conference Proceedings, volume 22, pages 85--92. August 1988. Google ScholarDigital Library
- James R. Kent, Wayne E. Carlson, and Richard E. Parent. Shape Transformation for Polyhedral Objects. In SIGGRAPH 92 Proceedings Conference, volume 26, pages 47--54. ACM SIGGRAPH, July 1992. Google ScholarDigital Library
- Rolf M. Koch, Markus H. Gross, Friedrich R. Carls, Daniel F. von Büren, George Fankhauser, and Yoav I. H. Parish, Simulating Facial Surgery Using Finite Element Methods. In SIGGRAPH 96 Conference Proceedings, pages 421--428. ACM SIGGRAPH, August 1996. Google ScholarDigital Library
- Tsuneya Kurihara and Kiyoshi Arai. A Transformation Method for Modeling and Animation of the Human Face from Photographs. In Nadia Magnenat Thalmann and Daniel Thalmann, editors, Computer Animation 91, pages 45--58. Springer-Verlag, Tokyo, 1991.Google ScholarCross Ref
- A. Lanitis, C. J. Taylor, and T. F. Cootes. A Unified Approach for Coding and Interpreting Face Images. In Fifth International Conference on Computer Vision (ICCV95), pages 368--373. Cambridge, Massachusetts, June 1995. Google ScholarDigital Library
- C. L. Lawson and R. J. Hansen. Solving Least Squares Problems. Prentice-Hall, Englewood Cliffs, 1974.Google Scholar
- Seung-Yong Lee, Kyung-Yong Chwa, Sung Yong Shin, and George Wolberg. Image Metamorphosis Using Snakes and Free-Form Deformations. In SIGGRAPH 95 Conference Proceedings, pages 439--448. ACM SIGGRAPH, August 1995. Google ScholarDigital Library
- Seung-Yong Lee, George Wolberg, Kyung-Yong Chwa, and Sung Yong Shin. Image Metamorphosis with Scattered Feature Constraints. IEEE Transactions on Visualization and Computer Graphics, 2(4), December 1996. Google ScholarDigital Library
- Yuencheng Lee, Demetri Terzopoulos, and Keith Waters. Realistic Modeling for Facial Animation. In SIGGRAPH 95 Conference Proceedings, pages 55--62. ACM SIGGRAPH, August 1995. Google ScholarDigital Library
- Yuencheng C. Lee, Demetri Terzopoulos, and Keith Waters. Constructing Physics-Based Facial Models of Individuals. In Proceedings of Graphics Interface 93, pages 1--8. May 1993.Google Scholar
- Francis H. Moffitt and Edward M. Mikhail. Photogrammetry. Harper & Row, New York, 3 edition, 1980.Google Scholar
- Shree K. Nayar, Katsushi Ikeuchi, and Takeo Kanade. Shape from Interreflections. International Journal of Computer Vision, 6:173--195, 1991. Google ScholarDigital Library
- Gregory M. Nielson. Scattered Data Modeling. IEEE Computer Graphics and Applications, 13(1):60--70, January 1993. Google ScholarDigital Library
- Frederic I. Parke. Computer Generated Animation of Faces. Proceedings ACM annual conference., August 1972. Google ScholarDigital Library
- Frederic I. Parke. A Parametric Model for Human Faces. PhD thesis, University of Utah, Salt Lake City, Utah, December 1974. UTEC-CSc-75-047. Google ScholarDigital Library
- Frederic I. Parke and Keith Waters. Computer Facial Animation. A K Peters, Wellesley, Massachusetts, 1996. Google ScholarDigital Library
- W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling. Numerical Recipes in C: The Art of Scientific Computing. Cambridge University Press, Cambridge, England, second edition, 1992. Google ScholarDigital Library
- Kari Pulli, Michael Cohen, Tom Duchamp, Hugues Hoppe, Linda Shapiro, and Werner Stuetzle. View-based rendering: Visualizing real objects from scanned range and color data. In Proc. 8th Eurographics Workshop on Rendering. June 1997. Google ScholarDigital Library
- Steven M. Seitz and Charles R. Dyer. View Morphing. In SIGGRAPH 96 Conference Proceedings, Annual Conference Series, pages 21--30. ACM SIGGRAPH, August 1996. Google ScholarDigital Library
- Chester C. Slama, editor. Manual of Photogrammetry. American Society of Photogrammetry, Falls Church, Virginia, fourth edition, 1980.Google Scholar
- Richard Szeliski and Sing Bing Kang. Recovering 3D Shape and Motion from Image Streams using Nonlinear Least Squares. Journal of Visual Communication and Image Representation, 5(1):10--28, March 1994.Google Scholar
- Richard Szeliski and Heung-Yeung Shum. Creating Full View Panoramic Image Mosaics and Texture-Mapped Models. In SIGGRAPH 97 Conference Proceedings, pages 251--258. ACM SIGGRAPH, August 1997. Google ScholarDigital Library
- Demetri Terzopoulos and Keith Waters. Physically-based Facial Modeling, Analysis, and Animation. Journal of Visualization and Computer Animation, 1(4):73--80, March 1990.Google ScholarCross Ref
- Kristinn R. Thórisson. Gandalf: An Embodied Humanoid Capable of Real-Time Multimodal Dialogue with People. In First ACM International Conference on Autonomous Agents. 1997. Google ScholarDigital Library
- Michael W. Vannier, Jeffrey F. Marsh, and James O. Warren. Three-dimentional Computer Graphics for Craniofacial Surgical Planning and Evaluation. In SIGGRAPH 83 Conference Proceedings, volume 17, pages 263--273. ACM SIGGRAPH, August 1983. Google ScholarDigital Library
- Keith Waters. A Muscle Model for Animating Three-Dimensional Facial Expression. In SIGGRAPH 87 Conference Proceedings), volume 21, pages 17--24. ACM SIGGRAPH, July 1987. Google ScholarDigital Library
- Lance Williams. Performance-Driven Facial Animation. In SIGGRAPH 90 Conference Proceedings, volume 24, pages 235--242. August 1990. Google ScholarDigital Library
- Z. Zhang, K. Isono, and S. Akamatsu. Euclidean Structure from Uncalibrated Images Using Fuzzy Domain Knowledge: Application to Facial Images Synthesis. In Proc. International Conference on Computer Vision (ICCV'98). January 1998. Google ScholarDigital Library
Index Terms
- Synthesizing realistic facial expressions from photographs
Recommendations
Real-time 3D neural facial animation from binocular video
We present a method for performing real-time facial animation of a 3D avatar from binocular video. Existing facial animation methods fail to automatically capture precise and subtle facial motions for driving a photo-realistic 3D avatar "in-the-wild" (...
A morphable model for the synthesis of 3D faces
SIGGRAPH '99: Proceedings of the 26th annual conference on Computer graphics and interactive techniquesIn this paper, a new technique for modeling textured 3D faces is introduced. 3D faces can either be generated automatically from one or more photographs, or modeled directly through an intuitive user interface. Users are assisted in two key problems of ...
Synthesizing realistic facial expressions from photographs
SIGGRAPH '98: Proceedings of the 25th annual conference on Computer graphics and interactive techniques
Comments