Abstract
We propose a novel system to reconstruct a high-quality hair depth map from a single portrait photo with minimal user input. We achieve this by combining depth cues such as occlusions, silhouettes, and shading, with a novel 3D helical structural prior for hair reconstruction. We fit a parametric morphable face model to the input photo and construct a base shape in the face, hair and body regions using occlusion and silhouette constraints. We then estimate the normals in the hair region via a Shape-from-Shading-based optimization that uses the lighting inferred from the face model and enforces an adaptive albedo prior that models the typical color and occlusion variations of hair. We introduce a 3D helical hair prior that captures the geometric structure of hair, and show that it can be robustly recovered from the input photo in an automatic manner. Our system combines the base shape, the normals estimated by Shape from Shading, and the 3D helical hair prior to reconstruct high-quality 3D hair models. Our single-image reconstruction closely matches the results of a state-of-the-art multi-view stereo applied on a multi-view stereo dataset. Our technique can reconstruct a wide variety of hairstyles ranging from short to long and from straight to messy, and we demonstrate the use of our 3D hair models for high-quality portrait relighting, novel view synthesis and 3D-printed portrait reliefs.
Supplemental Material
Available for Download
Supplemental files.
- Barron, J. T., and Malik, J. 2012. Color constancy, intrinsic images, and shape estimation. In ECCV, Springer, 57--70. Google ScholarDigital Library
- Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graph. 29, 4, 40:1--40:9. Google ScholarDigital Library
- Beeler, T., Bickel, B., Noris, G., Beardsley, P., Marschner, S., Sumner, R. W., and Gross, M. 2012. Coupled 3D reconstruction of sparse facial hair and skin. ACM Trans. Graph. 31, 4, 117:1--117:10. Google ScholarDigital Library
- Bertails, F., Audoly, B., Cani, M.-P., Querleux, B., Leroy, F., and Lévêque, J.-L. 2006. Super-helices for predicting the dynamics of natural hair. ACM Trans. Graph. 25, 3, 1180--1187. Google ScholarDigital Library
- Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Proceedings of SIGGRAPH, ACM, 187--194. Google ScholarDigital Library
- Bonneel, N., Paris, S., van de Panne, M., Durand, F., and Drettakis, G. 2009. Single photo estimation of hair appearance. In EGSR, Eurographics Association, 1171--1180. Google ScholarDigital Library
- Cao, C., Weng, Y., Zhou, S., Tong, Y., and Zhou, K. 2014. Face Warehouse: A 3D facial expression database for visual computing. IEEE Trans. TVCG 20, 3, 413--425. Google ScholarDigital Library
- Chai, M., Wang, L., Weng, Y., Yu, Y., Guo, B., and Zhou, K. 2012. Single-view hair modeling for portrait manipulation. ACM Trans. Graph. 31, 4, 116:1--116:8. Google ScholarDigital Library
- Chai, M., Wang, L., Weng, Y., Jin, X., and Zhou, K. 2013. Dynamic hair manipulation in images and videos. ACM Trans. Graph. 32, 4, 75:1--75:8. Google ScholarDigital Library
- Cherin, N., Cordier, F., and Melkemi, M. 2014. Modeling piecewise helix curves from 2D sketches. Comput. Aided Des. 46, 258--262. Google ScholarDigital Library
- Durou, J.-D., Falcone, M., and Sagona, M. 2008. Numerical methods for shape-from-shading: A new survey with benchmarks. Comput. Vis. Image Underst. 109, 1, 22--43. Google ScholarDigital Library
- Echevarria, J. I., Bradley, D., Gutierrez, D., and Beeler, T. 2014. Capturing and stylizing hair for 3D fabrication. ACM Trans. Graph. 33, 4, 125:1--125:11. Google ScholarDigital Library
- Fuhrmann, S., Langguth, F., and Goesele, M. 2014. MVE - a multi-view reconstruction environment. In Eurographics Workshop on Graphics and Cultural Heritage, The Eurographics Association, 11--18.Google Scholar
- Garrido, P., Valgaert, L., Wu, C., and Theobalt, C. 2013. Reconstructing detailed dynamic face geometry from monocular video. ACM Trans. Graph. 32, 6, 158:1--158:10. Google ScholarDigital Library
- Han, Y., Lee, J.-Y., and Kweon, I. S. 2013. High quality shape from a single RGB-D image under uncalibrated natural illumination. In ICCV, IEEE, 1617--1624. Google ScholarDigital Library
- Haque, S. M., Chatterjee, A., and Govindu, V. M. 2014. High quality photometric reconstruction using a depth camera. In CVPR, IEEE, 2283--2290. Google ScholarDigital Library
- Hu, L., Ma, C., Luo, L., and Li, H. 2014. Robust hair capture using simulated examples. ACM Trans. Graph. 33, 4, 126:1--126:10. Google ScholarDigital Library
- Hu, L., Ma, C., Luo, L., Wei, L.-Y., and Li, H. 2014. Capturing braided hairstyles. ACM Trans. Graph. 33, 6, 225:1--225:9. Google ScholarDigital Library
- Hu, L., Ma, C., Luo, L., and Li, H. 2015. Single-view hair modeling using a hairstyle database. ACM Trans. Graph. 34, 4, 125:1--125:9. Google ScholarDigital Library
- Johnson, M. K., and Adelson, E. H. 2011. Shape estimation in natural illumination. In CVPR, IEEE, 2553--2560. Google ScholarDigital Library
- Karsch, K., Liao, Z., Rock, J., Barron, J. T., and Hoiem, D. 2013. Boundary cues for 3D object shape recovery. In CVPR, IEEE, 2163--2170. Google ScholarDigital Library
- Karypis, G., and Kumar, V. 1998. Multilevel K-way partitioning scheme for irregular graphs. J. Parallel Distrib. Comput. 48, 1, 96--129. Google ScholarDigital Library
- Luo, L., Li, H., and Rusinkiewicz, S. 2013. Structure-aware hair capture. ACM Trans. Graph. 32, 4, 76:1--76:12. Google ScholarDigital Library
- Marschner, S. R., Jensen, H. W., Cammarano, M., Worley, S., and Hanrahan, P. 2003. Light scattering from human hair fibers. ACM Trans. Graph. 22, 3, 780--791. Google ScholarDigital Library
- Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3D geometry. ACM Trans. Graph. 24, 3, 536--543. Google ScholarDigital Library
- Oxholm, G., and Nishino, K. 2012. Shape and reflectance from natural illumination. In ECCV, Springer, 528--541. Google ScholarDigital Library
- Paris, S., Chang, W., Kozhushnyan, O. I., Jarosz, W., Matusik, W., Zwicker, M., and Durand, F. 2008. Hair photobooth: Geometric and photometric acquisition of real hairstyles. ACM Trans. Graph. 27, 3, 30:1--30:9. Google ScholarDigital Library
- Suwajanakorn, S., Kemelmacher-Shlizerman, I., and Seitz, S. M. 2014. Total moving face reconstruction. In ECCV, Springer, 796--812.Google Scholar
- Sýkora, D., Kavan, L., Čadik, M., Jamriška, O., Jacobson, A., Whited, B., Simmons, M., and Sorkine-Hornung, O. 2014. Ink-and-ray: Bas-relief meshes for adding global illumination effects to hand-drawn characters. ACM Trans. Graph. 33, 2, 16:1--16:15. Google ScholarDigital Library
- Valgaerts, L., Wu, C., Bruhn, A., Seidel, H.-P., and Theobalt, C. 2012. Lightweight binocular facial performance capture under uncontrolled lighting. ACM Trans. Graph. 31, 6, 187:1--187:11. Google ScholarDigital Library
- Vlasic, D., Brand, M., Pfister, H., and Popović, J. 2005. Face transfer with multilinear models. ACM Trans. Graph. 24, 3, 426--433. Google ScholarDigital Library
- Wither, J., Bertails, F., and Cani, M.-P. 2007. Realistic hair from a sketch. In SMI, IEEE, 33--42. Google ScholarDigital Library
- Wu, C., Varanasi, K., Liu, Y., Seidel, H.-P., and Theobalt, C. 2011. Shading-based dynamic shape refinement from multi-view video under general illumination. In ICCV, IEEE, 1108--1115. Google ScholarDigital Library
- Zhang, R., Tsai, P.-S., Cryer, J. E., and Shah, M. 1999. Shape from shading: A survey. IEEE Trans. PAMI 21, 8, 690--706. Google ScholarDigital Library
Index Terms
- High-quality hair modeling from a single portrait photo
Recommendations
Single-view hair modeling for portrait manipulation
Human hair is known to be very difficult to model or reconstruct. In this paper, we focus on applications related to portrait manipulation and take an application-driven approach to hair modeling. To enable an average user to achieve interesting ...
Capture of hair geometry from multiple images
SIGGRAPH '04: ACM SIGGRAPH 2004 PapersHair is a major feature of digital characters. Unfortunately, it has a complex geometry which challenges standard modeling tools. Some dedicated techniques exist, but creating a realistic hairstyle still takes hours. Complementary to user-driven methods,...
Capture of hair geometry from multiple images
Hair is a major feature of digital characters. Unfortunately, it has a complex geometry which challenges standard modeling tools. Some dedicated techniques exist, but creating a realistic hairstyle still takes hours. Complementary to user-driven methods,...
Comments