Thabo Beeler
Bernd Bickel
Abstract
Although facial hair plays an important role in individual expression, facial-hair reconstruction is not addressed by current face-capture systems. Our research addresses this limitation with an algorithm that treats hair and skin surface capture together in a coupled fashion so that a high-quality representation of hair fibers as well as the underlying skin surface can be reconstructed. We propose a passive, camera-based system that is robust against arbitrary motion since all data is acquired within the time period of a single exposure. Our reconstruction algorithm detects and traces hairs in the captured images and reconstructs them in 3D using a multiview stereo approach. Our coupled skin-reconstruction algorithm uses information about the detected hairs to deliver a skin surface that lies underneath all hairs irrespective of occlusions. In dense regions like eyebrows, we employ a hair-synthesis method to create hair fibers that plausibly match the image data. We demonstrate our scanning system on a number of individuals and show that it can successfully reconstruct a variety of facial-hair styles together with the underlying skin surface.
Supplemental Material
Available for Download
Supplemental material.
- Alexander, O., Rogers, M., Lambeth, W., Chiang, M., and Debevec, P. 2009. The digital Emily project: photo-real facial modeling and animation. In ACM SIGGRAPH 2009 Courses, ACM, New York, NY, USA, SIGGRAPH '09, 12:1--12:15. Google Scholar
Digital Library
- Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graph. (Proc. SIGGRAPH) 29, 4 (July), 40:1--40:9. Google ScholarDigital Library
- Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R. W., and Gross, M. 2011. High-quality passive facial performance capture using anchor frames. ACM Trans. Graph. (Proc. SIGGRAPH) 30, 4 (Aug.), 75:1--75:10. Google ScholarDigital Library
- Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, SIGGRAPH '99, 187--194. Google ScholarDigital Library
- Bradley, D., Heidrich, W., Popa, T., and Sheffer, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph. (Proc. SIGGRPAH) 29 (July), 41:1--41:10. Google ScholarDigital Library
- Canny, J. 1986. A computational approach to edge detection. Pattern Analysis and Machine Intelligence, IEEE Trans. PAMI-8, 6 (nov.), 679--698. Google ScholarDigital Library
- Dalí, S., and Halsman, P. 1954. Dali's mustache: a photographic interview. Flammarion.Google Scholar
- Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, SIGGRAPH '00, 145--156. Google ScholarDigital Library
- Furukawa, Y., and Ponce, J. 2009. Dense 3D motion capture for human faces. In Computer Vision and Pattern Recognition, 1674--1681.Google Scholar
- Ghosh, A., Fyffe, G., Tunwattanapong, B., Busch, J., Yu, X., and Debevec, P. 2011. Multiview face capture using polarized spherical gradient illumination. ACM, New York, NY, USA, vol. 30, 129:1--129:10. Google ScholarDigital Library
- Grabli, S., Sillion, F. X., Marschner, S. R., and Lengyel, J. E. 2002. Image-based hair capture by inverse lighting. In Proceedings of Graphics Interface (GI), 51--58.Google Scholar
- Jakob, W., Moon, J. T., and Marschner, S. 2009. Capturing hair assemblies fiber by fiber. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 28, 5 (Dec.), 164:1--164:9. Google ScholarDigital Library
- Kazhdan, M., Bolitho, M., and Hoppe, H. 2006. Poisson surface reconstruction. Proceedings of the fourth Eurographics symposium on Geometry processing, 61--70. Google ScholarDigital Library
- Ma, W.-C., Hawkins, T., Peers, P., Chabert, C.-F., Weiss, M., and Debevec, P. E. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. Rendering Techniques 9. Google ScholarDigital Library
- Paris, S., Briceño, H. M., and Sillion, F. X. 2004. Capture of hair geometry from multiple images. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3 (Aug.), 712--719. Google ScholarDigital Library
- Paris, S., Chang, W., Kozhushnyan, O. I., Jarosz, W., Matusik, W., Zwicker, M., and Durand, F. 2008. Hair photobooth: Geometric and photometric acquisition of real hairstyles. ACM Trans. Graph. (Proc. SIGGRAPH) 27, 3 (Aug.), 30:1--30:9. Google ScholarDigital Library
- Wei, Y., Ofek, E., Quan, L., and Shum, H.-Y. 2005. Modeling hair from multiple views. ACM Trans. Graph. (Proc. SIGGRAPH) 24, 3 (July), 816--820. Google ScholarDigital Library
- Weyrich, T., Matusik, W., Pfister, H., Bickel, B., Donner, C., Tu, C., McAndless, J., Lee, J., Ngan, A., Jensen, H. W., and Gross, M. 2006. Analysis of human faces using a measurement-based skin reflectance model. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3 (July), 1013--1024. Google ScholarDigital Library
Index Terms
- Coupled 3D reconstruction of sparse facial hair and skin
Recommendations
Facial hair tracking for high fidelity performance capture
Facial hair is a largely overlooked topic in facial performance capture. Most production pipelines in the entertainment industry do not have a way to automatically capture facial hair or track the skin underneath it. Thus, actors are asked to shave ...
Single View Facial Hair 3D Reconstruction
Pattern Recognition and Image AnalysisAbstractIn this work, we introduce a novel energy-based framework that addresses the challenging problem of 3D reconstruction of facial hair from a single RGB image. To this end, we identify hair pixels over the image via texture analysis and then ...
3D reconstruction framework via combining one 3D scanner and multiple stereo trackers
This paper presents a novel 3D reconstruction framework of large objects, where we adopt one 3D scanner to reconstruct partial sections of large objects, and employ multiple stereo trackers to extend reconstruction range. Both the 3D scanner and stereo ...
Comments