Oleg Alexander
Paul Debevec
ABSTRACT
This course describes how high-resolution face scanning, advanced character rigging, and performance-driven facial animation were combined to create Digital Emily, a believably photorealistic digital actor. Actress Emily O'Brien was scanned in the USC ICT light stage in 35 different facial poses using a new high-resolution face-scanning process capable of capturing geometry and textures down to the level of skin pores and fine wrinkles. These scans were assembled into a rigged digital character, which could then be driven by Image Metrics video-based facial animation technology. The real Emily was captured speaking on a small set, and her movements were used to drive a complete digital face replacement of her character, including its diffuse, specular, and animated displacement maps. HDRI lighting reconstruction techniques were used to reproduce the lighting on her original performance. The most recent results show new real-time animation and rendering research for the Digital Emily character.
Supplemental Material
Available for Download
- Baker, S., and Matthews, I. 2004. Lucas-kanade 20 years on: A unifying framework. Int. J. Comput. Vision 56, 3, 221--255. Google Scholar
Digital Library
- Borshukov, G., and Lewis, J. P. 2003. Realistic human face rendering for 'The Matrix Reloaded'. In ACM SIGGRAPH 2003 Sketches&Applications. Google ScholarDigital Library
- Chang. 2007. Beowulf. Variety (Nov).Google Scholar
- Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Proceedings of ACM SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, 145--156. Google ScholarDigital Library
- Debevec, P., Tchou, C., Gardner, A., Hawkins, T., Poullis, C., Stumpfel, J., Jones, A., Yun, N., Einarsson, P., Lundgren, T., Fajardo, M., and Martinez, P. 2004. Estimating surface reflectance properties of a complex scene under captured natural illumination. Tech. Rep. ICT-TR-06.2004, USC ICT, Marina del Rey, CA, USA, Jun. http://gl.ict.usc.edu/Research/reflectance/Parth-ICTTR-06.2004.pdf.Google Scholar
- Debevec, P. 1998. Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In Proceedings of SIGGRAPH 98, Computer Graphics Proceedings, Annual Conference Series, 189--198. Google ScholarDigital Library
- Ekman, P., and Friesen, W. 1978. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto.Google Scholar
- Fleishman, S., Drori, I., and Cohen-Or, D. 2003. Bilateral mesh denoising. ACM Transactions on Graphics 22, 3 (July), 950--953. Google ScholarDigital Library
- Ghosh, A., Hawkins, T., Peers, P., Frederiksen, S., and Debevec, P. 2008. Practical modeling and acquisition of layered facial reflectance. ACM Transactions on Graphics 27, 5 (Dec.), 139:1--139:10. Google ScholarDigital Library
- Guenter, B., Grimm, C., Wood, D., Malvar, H., and Pighin, F. 1998. Making faces. In Proceedings of SIGGRAPH 98, Computer Graphics Proceedings, Annual Conference Series, 55--66. Google ScholarDigital Library
- Hawkins, T., Wenger, A., Tchou, C., Gardner, A., Göransson, F., and Debevec, P. 2004. Animatable facial reflectance fields. In Rendering Techniques 2004: 15th Eurographics Workshop on Rendering, 309--320. Google ScholarDigital Library
- Hyneman, W., Itokazu, H., Williams, L., and Zhao, X. 2005. Human face project. In ACM SIGGRAPH 2005 Course #9: Digital Face Cloning, ACM, New York, NY, USA. Google ScholarDigital Library
- ICT-Graphics-Laboratory, 2001. Realistic human face scanning and rendering. Web site. http://gl.ict.usc.edu/Research/facescan/.Google Scholar
- Jensen, H. W., Marschner, S. R., Levoy, M., and Hanrahan, P. 2001. A practical model for subsurface light transport. In Proceedings of ACM SIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, 511--518. Google ScholarDigital Library
- Kaufman, D. 1999. Photo genesis. WIRED 7, 7 (July). http://www.wired.com/wired/archive/7.07/jester.html.Google Scholar
- Landis, H. 2002. Production-ready global illumination. In Notes for ACM SIGGRAPH 2005 Course #16: RenderMan in Production, ACM, New York, NY, USA.Google Scholar
- Ma, W.-C., Hawkins, T., Peers, P., Chabert, C.-F., Weiss, M., and Debevec, P. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. In Rendering Techniques, 183--194. Google ScholarDigital Library
- Ma, W.-C., Jones, A., Chiang, J.-Y., Hawkins, T., Frederiksen, S., Peers, P., Vukovic, M., Ouhyoung, M., and Debevec, P. 2008. Facial performance synthesis using deformation-driven polynomial displacement maps. ACM Transactions on Graphics 27, 5 (Dec.), 121:1--121:10. Google ScholarDigital Library
- Ma, W.-C. 2008. A Framework for Capture and Synthesis of High Resolution Facial Geometry and Performance. PhD thesis, National Taiwan University.Google Scholar
- Mori, M. 1970. Bukimi no tani (the uncanny valley). Energy 7, 4, 33--35.Google Scholar
- Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3d geometry. ACM Transactions on Graphics 24, 3 (Aug.), 536--543. Google ScholarDigital Library
- Perlman, S. 2006. Volumetric cinematography: The world no longer flat. Mova White Paper (Oct).Google Scholar
- Plantec, P. 2008. The digital eye: Image metrics attempts to leap the uncanny valley. VFXWorld magazine (August). http://www.vfxworld.com/?atype=articles&id=3723.Google Scholar
- Robertson, B. 2009. What's old is new again. Computer Graphics World 32, 1 (Jan).Google Scholar
- Sagar, M., Monos, J., Schmidt, J., Ziegler, D., Foo, S.-C., Scott, R., Stern, J., Waegner, C., Nofz, P., Hawkins, T., and Debevec, P. 2004. Reflectance field rendering of human faces for Spider-man 2. In SIGGRAPH '04: ACM SIGGRAPH 2004 Technical Sketches, ACM, New York, NY, USA. Google ScholarDigital Library
- Wang, Y., Gupta, M., Zhang, S., Wang, S., Gu, X., Samaras, D., and Huang, P. 2008. High resolution tracking of non-rigid motion of densely sampled 3d data using harmonic maps. Int. J. Comput. Vision 76, 3, 283--300. Google ScholarDigital Library
- Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., and Debevec, P. 2005. Performance relighting and reflectance transformation with time-multiplexed illumination. ACM Transactions on Graphics 24, 3 (Aug.), 756--764. Google ScholarDigital Library
- Wolff, E. 2003. Creating virtual performers: Disney's human face project. Millimeter magazine (April).Google Scholar
- Zhang, Z. 2000. A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22, 11, 1330--1334. Google ScholarDigital Library
Index Terms
- The Digital Emily project: photoreal facial modeling and animation
Recommendations
The Digital Emily Project: Achieving a Photorealistic Digital Actor
The Digital Emily Project uses advanced face scanning, character rigging, performance capture, and compositing to achieve one of the world's first photorealistic digital facial performances. The project scanned the geometry and reflectance of actress ...
Digital Ira: creating a real-time photoreal digital actor
SIGGRAPH '13: ACM SIGGRAPH 2013 PostersOverview In 2008, the "Digital Emily" project [Alexander et al. 2009] showed how a set of high-resolution facial expressions scanned in a light stage could be rigged into a real-time photoreal digital character and driven with video-based facial ...
Creating a Photoreal Digital Actor: The Digital Emily Project
CVMP '09: Proceedings of the 2009 Conference for Visual Media ProductionThe Digital Emily Project is a collaboration between facial animation company Image Metrics and the Graphics Laboratory at the University of Southern California's Institute for Creative Technologies to achieve one of the world's first photorealistic ...
Comments