Thabo Beeler
Bernd Bickel
Abstract
This paper describes a passive stereo system for capturing the 3D geometry of a face in a single-shot under standard light sources. The system is low-cost and easy to deploy. Results are submillimeter accurate and commensurate with those from state-of-the-art systems based on active lighting, and the models meet the quality requirements of a demanding domain like the movie industry. Recovered models are shown for captures from both high-end cameras in a studio setting and from a consumer binocular-stereo camera, demonstrating scalability across a spectrum of camera deployments, and showing the potential for 3D face modeling to move beyond the professional arena and into the emerging consumer market in stereoscopic photography.
Our primary technical contribution is a modification of standard stereo refinement methods to capture pore-scale geometry, using a qualitative approach that produces visually realistic results. The second technical contribution is a calibration method suited to face capture systems. The systemic contribution includes multiple demonstrations of system robustness and quality. These include capture in a studio setup, capture off a consumer binocular-stereo camera, scanning of faces of varying gender and ethnicity and age, capture of highly-transient facial expression, and scanning a physical mask to provide ground-truth validation.
Supplemental Material
Available for Download
- Alexander, O., Rogers, M., Lambeth, W., Chiang, M., and Debevec, P. 2009. The Digital Emily Project: Photoreal facial modeling and animation. ACM Trans. Graph.Google Scholar
- Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. 2010. High-quality single shot capture of facial geometry: Implementation details. Tech. Rep. 671, ETH Zurich.Google Scholar
- Bradley, D., Heidrich, W., Popa, T., and Sheffer, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph. Google ScholarDigital Library
- Chen, T., Goesele, M., and Seidel, H. 2006. Mesostructure from specularity. CVPR. Google ScholarDigital Library
- DI3D. 2009. Dimensional imaging. http://www.di3d.com.Google Scholar
- Donner, C., and Jensen, H. 2006. A spectral bssrdf for shading human skin. Eurographics Symposium on Rendering.Google Scholar
- Donner, C., Weyrich, T., d'Eon, E., Ramamoorthi, R., and Rusinkiewicz, S. 2008. A layered, heterogeneous reflectance model for acquiring and rendering human skin. ACM Trans. Graph. Google ScholarDigital Library
- Furukawa, Y., and Ponce, J. 2007. Accurate, dense, and robust multi-view stereopsis. CVPR.Google Scholar
- Glencross, M., Ward, G., Melendez, F., Jay, C., Liu, J., and Hubbold, R. 2008. A perceptually validated model for surface depth hallucination. ACM Trans. Graph. Google ScholarDigital Library
- Golovinskiy, A., Matusik, W., Pfister, H., Rusinkiewicz, S., and Funkhouser, T. 2006. A statistical model for synthesis of detailed facial geometry. ACM Trans. Graph. Google ScholarDigital Library
- Hartley, R., and Zisserman, A. 2000. Multiple View Geometry, second ed. Cambridge University Press. Google ScholarDigital Library
- Hernandez, C., Vogiatzis, G., and Cipolla, R. 2008. Shadows in three-source photometric stereo. ECCV. Google ScholarDigital Library
- Hiep, V., Keriven, R., Labatut, P., and Pons, J. 2009. Towards high-resolution large-scale multi-view stereo. CVPR.Google Scholar
- Hyneman, W., Itokazu, H., Williams, L., and Zhao, X. 2005. Human face project. SIGGRAPH 2005 Courses. Google ScholarDigital Library
- Intel. 2001. Opencv reference manual. http://developer.intel.com.Google Scholar
- Kazhdan, M., Bolitho, M., and Hoppe, H. 2006. Poisson surface reconstruction. In SGP. Google ScholarDigital Library
- Ma, W., Hawkins, T., Peers, P., Chabert, C., Weiss, M., and Debevec, P. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. Rendering Techniques. Google ScholarDigital Library
- Merrell, P., Akbarzadeh, A., Wang, L., Mordohai, P., Frahm, J., Yang, R., Nister, D., and Pollefeys, M. 2007. Real-time visibility-based fusion of depth maps. ICCV.Google Scholar
- Meyer, M., Desbrun, M., Schröder, P., and Barr, A. H. 2003. Discrete differential-geometry operators for triangulated 2-manifolds. In Visualization and Mathematics III.Google Scholar
- Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3d geometry. ACM Trans. Graph. Google ScholarDigital Library
- Parke, F. 1974. A parametric model for human faces. PhD Thesis, University of Utah. Google ScholarDigital Library
- Pighin, F., and Lewis, J. 2005. Digital face cloning. ACM Trans. Graph.Google Scholar
- Robert, L., and Deriche, R. 1996. Dense depth map reconstruction: A minimization and regularization approach which preserves discontinuities. In ECCV. Google ScholarDigital Library
- Scharstein, D., and Szeliski, R. 1996. Stereo matching with non-linear diffusion. CVPR. Google ScholarDigital Library
- Scharstein, D., and Szeliski, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV. Google ScholarDigital Library
- Seitz, S., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. CVPR. Google ScholarDigital Library
- Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. Google ScholarDigital Library
- Svoboda, T. Multi camera self-calibration. http://cmp.felk.cvut.cz/svoboda/SelfCal/index.html.Google Scholar
- Torralba, A., and Freeman, W. 2003. Properties and applications of shape recipes. CVPR.Google Scholar
- Weise, T., Leibe, B., and Gool, L. V. 2007. Fast 3D scanning with automatic motion compensation. CVPR.Google Scholar
- Weise, T., Li, H., Gool, L., and Pauly, M. 2009. Face/off: live facial puppetry. SCA. Google ScholarDigital Library
- Weyrich, T., Matusik, W., Pfister, H., Bickel, B., Donner, C., Tu, C., McAndless, J., Lee, J., Ngan, A., Jensen, H., and Gross, M. 2006. Analysis of human faces using a measurement-based skin reflectance model. ACM Trans. Graph. Google ScholarDigital Library
- Woodford, O., Torr, P., Reid, I., and Fitzgibbon, A. 2008. Global stereo reconstruction under second order smoothness priors. CVPR.Google Scholar
Index Terms
- High-quality single-shot capture of facial geometry
Recommendations
Single-shot high-quality facial geometry and skin appearance capture
We propose a new light-weight face capture system capable of reconstructing both high-quality geometry and detailed appearance maps from a single exposure. Unlike currently employed appearance acquisition systems, the proposed technology does not ...
High-quality single-shot capture of facial geometry
SIGGRAPH '10: ACM SIGGRAPH 2010 papersThis paper describes a passive stereo system for capturing the 3D geometry of a face in a single-shot under standard light sources. The system is low-cost and easy to deploy. Results are submillimeter accurate and commensurate with those from state-of-...
Epipolar geometry for prism-based single-lens stereovision
In order to simplify the design and implementation of a stereo vision system, prism has been used to capture stereo images with a single camera. This kind of system not only provides advantages over traditional two-camera stereo, but also reduces the ...
Comments