Daniel Vlasic
Paul Debevec
Szymon Rusinkiewicz
Wojciech Matusik
Skip Abstract Section
Abstract
We describe a system for high-resolution capture of moving 3D geometry, beginning with dynamic normal maps from multiple views. The normal maps are captured using active shape-from-shading (photometric stereo), with a large lighting dome providing a series of novel spherical lighting configurations. To compensate for low-frequency deformation, we perform multi-view matching and thin-plate spline deformation on the initial surfaces obtained by integrating the normal maps. Next, the corrected meshes are merged into a single mesh using a volumetric method. The final output is a set of meshes, which were impossible to produce with previous methods. The meshes exhibit details on the order of a few millimeters, and represent the performance over human-size working volumes at a temporal resolution of 60Hz.
- Ahmed, N., Theobalt, C., Dobre, P., Seidel, H.-P., and Thrun, S. 2008. Robust fusion of dynamic shape and normal capture for high-quality reconstruction of time-varying geometry. In Computer Vision and Pattern Recognition, 1--8.Google Scholar
- Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. Scape: Shape Completion and Animation of People. ACM Transactions on Graphics 24, 3 (Aug.), 408--416. Google ScholarDigital Library
- Balan, A. O., Sigal, L., Black, M. J., Davis, J. E., and Haussecker, H. W. 2007. Detailed human shape and pose from images. In Computer Vision and Pattern Recognition.Google Scholar
- Bernardini, F., Rushmeier, H., Martin, I. M., Mittleman, J., and Taubin, G. 2002. Building a digital model of michelangelo's florentine pietà. IEEE Computer Graphics&Applications 22, 1 (Jan./Feb.), 59--67. Google ScholarDigital Library
- Bradley, D., Popa, T., Sheffer, A., Heidrich, W., and Boubekeur, T. 2008. Markerless garment capture. ACM Transactions on Graphics 27, 3 (Aug.), 99. Google ScholarDigital Library
- Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. 2004. High accuracy optical flow estimation based on a theory for warping. In Proceedings of the 8th European Conference on Computer Vision, 25--36.Google Scholar
- Campbell, N., Vogiatzis, G., Hernandez, C., and Cipolla, R. 2007. Automatic 3d object segmentation in multiple views using volumetric graph-cuts. In British Machine Vision Conference.Google Scholar
- Carranza, J., Theobalt, C., Magnor, M. A., and Seidel, H.-P. 2003. Free-viewpoint video of human actors. ACM Transactions on Graphics 22, 3 (July), 569--577. Google ScholarDigital Library
- Chang, W., and Zwicker, M. 2008. Automatic registration for articulated shapes. Computer Graphics Forum (Proceedings of SGP 2008) 27, 5, 1459--1468. Google ScholarDigital Library
- Corazza, S., Mündermann, L., Chaudhari, A., Demattio, T., Cobelli, C., and Andriacchi, T. P. 2006. A markerless motion capture system to study musculoskeletal biomechanics: Visual hull and simulated annealing approach. Annals of Biomedical Engineering 34, 6 (July), 1019--1029.Google ScholarCross Ref
- Criminisi, A., Blake, A., Rother, C., Shotton, J., and Torr, P. H. 2007. Efficient dense stereo with occlusions for new view-synthesis by four-state dynamic programming. Int. Journal of Computer Vision 71, 1, 89--110. Google ScholarDigital Library
- Curless, B., and Levoy, M. 1996. A volumetric method for building complex models from range images. In Proceedings of SIGGRAPH 96, Computer Graphics Proceedings, Annual Conference Series, 303--312. Google ScholarDigital Library
- Davis, J., Marschner, S. R., Garr, M., and Levoy, M. 2002. Filling holes in complex surfaces using volumetric diffusion. In Symposium on 3D Data Processing, Visualization, and Transmission, 428--438.Google Scholar
- Davis, J., Ramamoorthi, R., and Rusinkiewicz, S. 2005. Spacetime stereo: A unifying framework for depth from triangulation. In IEEE Transactions on Pattern Analysis and Machine Intelligence, 196--302. Google ScholarDigital Library
- de Aguiar, E., Theobalt, C., Stoll, C., and Seidel, H.-P. 2007. Marker-less deformable mesh tracking for human shape and motion capture. In Computer Vision and Pattern Recognition.Google Scholar
- de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Transactions on Graphics 27, 3 (Aug.), 98. Google ScholarDigital Library
- Einarsson, P., Chabert, C.-F., Jones, A., Ma, W.-C., Lamond, B., Hawkins, T., Bolas, M., Sylwan, S., and Debevec, P. 2006. Relighting human locomotion with flowed reflectance fields. In Proc. of Eurographics Symposium on Rendering, 183--194. Google ScholarDigital Library
- Furukawa, Y., and Ponce, J. 2006. Carved visual hulls for image-based modeling. In European Conference on Computer Vision, 564--577. Google ScholarDigital Library
- Hernandez, C., and Schmitt, F. 2004. Silhouette and stereo fusion for 3D object modeling. Computer Vision and Image Understanding 96, 3 (Dec.), 367--392. Google ScholarDigital Library
- Hernandez, C., Vogiatzis, G., and Cipolla, R. 2008. Multiview photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 30, 3, 548--554. Google ScholarDigital Library
- Hertzmann, A., and Seitz, S. M. 2005. Example-based photometric stereo: Shape reconstruction with general, varying brdfs. IEEE Trans. Pattern Anal. Mach. Intell. 27, 8, 1254--1264. Google ScholarDigital Library
- Hornung, A., and Kobbelt, L. 2006. Hierarchical volumetric multi-view stereo reconstruction of manifold surfaces based on dual graph embedding. In Computer Vision and Pattern Recognition, 503--510. Google ScholarDigital Library
- Huang, Q.-X., Adams, B., Wicke, M., and Guibas, L. J. 2008. Non-rigid registration under isometric deformations. Computer Graphics Forum (Proc. SGP'08) 27, 5, 1449--1457. Google ScholarDigital Library
- Joshi, N., and Kriegman, D. 2007. Shape from varying illumination and viewpoint. In International Conference on Computer Vision.Google Scholar
- Kazhdan, M., Bolitho, M., and Hoppe, H. 2006. Poisson surface reconstruction. In Symposium on Geometry Processing. Google ScholarDigital Library
- Li, H., Sumner, R. W., and Pauly, M. 2008. Global correspondence optimization for non-rigid registration of depth scans. Computer Graphics Forum 27, 5, 1421--1430.Google ScholarDigital Library
- Lim, J., Ho, J., Yang, M.-H., and Kriegman, D. 2005. Passive photometric stereo from motion. In International Conference on Computer Vision. Google ScholarDigital Library
- Ma, W.-C., Hawkins, T., Peers, P., Chabert, C.-F., Weiss, M., and Debevec, P. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. In Rendering Techniques, 183--194. Google ScholarCross Ref
- Mitra, N. J., Flory, S., Ovsjanikov, M., Gelfand, N., Guibas, L., and Pottmann, H. 2007. Dynamic geometry registration. In Proc. Symposium on Geometry Processing, 173--182. Google ScholarDigital Library
- Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3d geometry. ACM Transactions on Graphics 24, 3 (Aug.), 536--543. Google ScholarDigital Library
- Okutomi, M., and Kanade, T. 1993. A multiple-baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence 15, 4, 353--363. Google ScholarDigital Library
- Pekelny, Y., and Gotsman, C. 2008. Articulated object reconstruction and markerless motion capture from depth video. Computer Graphics Forum 27, 2 (Apr.), 399--408.Google ScholarCross Ref
- Rander, P. W., Narayanan, P., and Kanade, T. 1997. Virtualized reality: Constructing time-varying virtual worlds from real world events. In IEEE Visualization, 277--284. Google ScholarDigital Library
- Rusinkiewicz, S., Hall-Holt, O., and Levoy, M. 2002. Real-time 3D model acquisition. ACM Transactions on Graphics 21, 3 (July), 438--446. Google ScholarDigital Library
- Sagawa, R., Osawa, N., and Yagi, Y. 2007. Deformable registration of textured range images by using texture and shape features. In 3DIM '07: Proceedings of the Sixth International Conference on 3-D Digital Imaging and Modeling, 65--72. Google ScholarDigital Library
- Seitz, S. M., and Dyer, C. R. 1999. Photorealistic scene reconstruction by voxel coloring. International Journal of Computer Vision 35, 2, 151--173. Google ScholarDigital Library
- Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multiview stereo reconstruction algorithms. In Computer Vision and Pattern Recognition, 519--528. Google ScholarDigital Library
- Sharf, A., Alcantara, D. A., Lewiner, T., Greif, C., Sheffer, A., Amenta, N., and Cohen-Or, D. 2008. Space-time surface reconstruction using incompressible flow. ACM Trans. Graph. 27, 5, 1--10. Google ScholarDigital Library
- Starck, J., and Hilton, A. 2003. Model-based multiple view reconstruction of people. In International Conference on Computer Vision, 915--922. Google ScholarDigital Library
- Starck, J., and Hilton, A. 2007. Surface capture for performance based animation. IEEE Computer Graphics and Applications 27(3), 21--31. Google ScholarDigital Library
- Svoboda, T., Martinec, D., and Pajdla, T. 2005. A convenient multi-camera self-calibration for virtual environments. PRESENCE: Teleoperators and Virtual Environments 14, 4 (August), 407--422. Google ScholarDigital Library
- Theobalt, C., Ahmed, N., Lensch, H., Magnor, M., and Seidel, H.-P. 2007. Seeing people in different light-joint shape, motion, and reflectance capture. IEEE Transactions on Visualization and Computer Graphics 13, 4 (July/Aug.), 663--674. Google ScholarDigital Library
- Vlasic, D., Baran, I., Matusik, W., and Popović, J. 2008. Articulated mesh animation from multi-view silhouettes. ACM Transactions on Graphics 27, 3 (Aug.), 97. Google ScholarDigital Library
- Vogiatzis, G., Torr, P. H. S., and Cipolla, R. 2005. Multiview stereo via volumetric graph-cuts. In Computer Vision and Pattern Recognition, 391--398. Google ScholarDigital Library
- Vogiatzis, G., Hernandez, C., and Cipolla, R. 2006. Reconstruction in the round using photometric normals and silhouettes. In 2006 Conference on Computer Vision and Pattern Recognition (CVPR 2006), 1847--1854. Google ScholarDigital Library
- Wand, M., Jenke, P., Huang, Q., Bokeloh, M., Guibas, L., and Schilling, A. 2007. Reconstruction of deforming geometry from time-varying point clouds. In Proc. Symposium on Geometry Processing, 49--58. Google ScholarDigital Library
- Wand, M., Adams, B., Ovsjanikov, M., Berner, A., Bokeloh, M., Jenke, P., Guibas, L., Seidel, H.-P., and Schilling, A. 2009. Efficient reconstruction of nonrigid shape and motion from real-time 3d scanner data. ACM Transactions on Graphics 28, 2 (Apr.), 15. Google ScholarDigital Library
- Woodham, R. J. 1978. Photometric stereo: A reflectance map technique for determining surface orientation from image intensity. In Proc. SPIE's 22nd Annual Technical Symposium, vol. 155.Google Scholar
- Zhang, S., and Huang, P. 2006. High-resolution real-time three-dimensional shape measurement. Optical Engineering 45, 12.Google Scholar
- Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Transactions on Graphics 23, 3 (Aug.), 548--558. Google ScholarDigital Library
- Zhang, H., Sheffer, A., Cohen-Or, D., Zhou, Q., van Kaick, O., and Tagliasacchi, A. 2008. Deformation-driven shape correspondence. Proc. Symposium on Geometry Processing 27, 5 (July), 1431--1439. Google ScholarDigital Library
Index Terms
- Dynamic shape capture using multi-view photometric stereo
Recommendations
Dynamic shape capture using multi-view photometric stereo
SIGGRAPH Asia '09: ACM SIGGRAPH Asia 2009 papersWe describe a system for high-resolution capture of moving 3D geometry, beginning with dynamic normal maps from multiple views. The normal maps are captured using active shape-from-shading (photometric stereo), with a large lighting dome providing a ...
Deep Reflectance Volumes: Relightable Reconstructions from Multi-view Photometric Images
Computer Vision – ECCV 2020AbstractWe present a deep learning approach to reconstruct scene appearance from unstructured images captured under collocated point lighting. At the heart of Deep Reflectance Volumes is a novel volumetric scene representation consisting of opacity, ...
Topology-adaptive multi-view photometric stereo
CVPR '11: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern RecognitionIn this paper, we present a novel technique that enables capturing of detailed 3D models from flash photographs integrating shading and silhouette cues. Our main contribution is an optimization framework which not only captures subtle surface details ...
Comments