Michael Wand
Skip Abstract Section
Skip Supplemental Material SectionAbstract
We present a new technique for reconstructing a single shape and its nonrigid motion from 3D scanning data. Our algorithm takes a set of time-varying unstructured sample points that capture partial views of a deforming object as input and reconstructs a single shape and a deformation field that fit the data. This representation yields dense correspondences for the whole sequence, as well as a completed 3D shape in every frame. In addition, the algorithm automatically removes spatial and temporal noise artifacts and outliers from the raw input data. Unlike previous methods, the algorithm does not require any shape template but computes a fitting shape automatically from the input data. Our reconstruction framework is based upon a novel topology-aware adaptive subspace deformation technique that allows handling long sequences with complex geometry efficiently. The algorithm accesses data in multiple sequential passes, so that long sequences can be streamed from hard disk, not being limited by main memory. We apply the technique to several benchmark datasets, significantly increasing the complexity of the data that can be handled efficiently in comparison to previous work.
Supplemental Material
Available for Download
- Adams, B., Ovsjanikov, M., Wand, M., Seidel, H.-P., and Guibas, L. J. 2008. Meshless modeling of deformable shapes and their motion. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Google Scholar
Digital Library
- Ahmed, N., Theobalt, C., Roessl, C., Thrun, S., and Seidel, H.-P. 2008. Dense correspondence finding for parametrization-free animation reconstruction from video. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
- Allen, B., Curless, B., and Popović, Z. 2002. Articulated body deformation from range scan data. Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 612--619. Google ScholarDigital Library
- Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. In ACM SIGGRAPH Papers. ACM, New York, 587--594. Google ScholarDigital Library
- Anguelov, D., Koller, D., Srinivasan, P., Thrun, S., Pang, H.-C., and Davis, J. 2005. The correlated correspondence algorithm for unsupervised registration of nonrigid surfaces. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS ).Google Scholar
- Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: Shape completion and animation of people. In Proceedings of the 32nd International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH). Google ScholarDigital Library
- Anuar, N. and Guskov, I. 2004. Extracting animated meshes with adaptive motion estimation. In Proceedings of the Vision, Modeling and Visualization Conference. 63--71.Google Scholar
- Au, O. K.-C., Tai, C.-L., Liu, L., and Fu, H. 2006. Dual laplacian editing for meshes. IEEE Trans. Visualiz. Comput. Graph. 12, 3, 386--395. Google ScholarDigital Library
- Avriel, M. 2003. Nonlinear Programming: Analysis and Methods. Dover Publishing.Google Scholar
- Besl, P. J. and McKay, N. D. 1992. A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14, 2, 239--256. Google ScholarDigital Library
- Botsch, M., Pauly, M., Gross, M., and Kobbelt, L. 2006. Primo: Coupled prisms for intuitive surface modeling. In SGP '06: Proceedings of the 4th Eurographics Symposium on Geometry Processing. Eurographics Association. 11--20. Google ScholarDigital Library
- Botsch, M., Pauly, M., Wicke, M., and Gross, M. 2007. Adaptive space deformations based on rigid cells. Comput. Graph. Forum 26, 3, 339--347.Google ScholarCross Ref
- Botsch, M. and Sorkine, O. 2008. On linear variational surface deformation methods. IEEE Trans. Visualiz. Comput. Graph. 14, 1, 213--230. Google ScholarDigital Library
- Carceroni, R. L. and Kutulakos, K. N. 2002. Multi-View scene capture by surfel sampling: From video streams to non-rigid 3d motion, shape and reflectance. Int. J. Comput. Vision 49, 2-3, 175--214. Google ScholarDigital Library
- Carranza, J., Theobalt, C., Magnor, M. A., and Seidel, H.-P. 2003. Free-Viewpoint video of human actors. In ACM SIGGRAPH Papers. ACM, New York, 569--577. Google ScholarDigital Library
- Chang, W. and Zwicker, M. 2008. Automatic registration for articulated shapes. Comput. Graph. Forum 27, 5.Google ScholarDigital Library
- Davis, J., Nehab, D., Ramamoorthi, R., and Rusinkiewicz, S. 2005. Spacetime stereo: A unifying framework for depth from triangulation. IEEE Trans. Pattern Anal. Mach. Intell. 27, 2, 296--302. Google ScholarDigital Library
- de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Trans. Graph. 27, 3, 1--10. Google ScholarDigital Library
- Fong, P. and Buron, F. 2005. High-Resolution three-dimensional sensing of fast deforming objects. In Proceedings of the IEEE/RSI International Conference on Intelligent Robots and Systems, (IROS). 1606--1611.Google Scholar
- Fries, T.-P. and Matthies, H. G. 2003. Classification and overview of meshfree methods. Tech. rep., TU Brunswick, Germany Nr. 2003-03.Google Scholar
- Huang, J., Shi, X., Liu, X., Zhou, K., Wei, L.-Y., Teng, S.-H., Bao, H., Guo, B., and Shum, H.-Y. 2006. Subspace gradient domain mesh deformation. ACM Trans. Graph. 25, 3, 1126--1134. Google ScholarDigital Library
- Huang, Q.-X., Adams, B., and Wand, M. 2007. Bayesian surface reconstruction via iterative scan alignment to an optimized prototype. In Proceedings of the 5th Eurographics Symposium on Geometry Processing (SCP). Eurographics Association, 213--223. Google ScholarDigital Library
- Huang, Q.-X., Adams, B., Wicke, M., and Guibas, L. J. 2008. Non-Rigid registration under isometric deformations. Comput. Graph. Forum 27, 5.Google ScholarCross Ref
- König, S. and Gumhold, S. 2007. Image-Based motion compensation for structured light scanning of dynamic scenes. In EG Workshop on Dynamic 3D Imaging (Dyn3D '07).Google Scholar
- Li, H., Sumner, R. W., and Pauly, M. 2008. Global correspondence optimization for non-rigid registration of depth scans. Comput. Graph. Forum 27, 5. Google ScholarDigital Library
- Medioni, G., Lee, M., and Tang, C. 2000. A Computational Framework for Segmentation and Grouping. Elsevier. Google ScholarDigital Library
- MESA. 2009. http://www.mesa-imaging.ch.Google Scholar
- Mitra, N. J., Flöry, S., Ovsjanikov, M., Gelfand, N., Guibas, L., and Pottmann, H. 2007. Dynamic geometry registration. In Proceedings of the 5th Eurographics Symposium on Geometry Processing (SGP '07). Eurographics Association, 173--182. Google ScholarDigital Library
- Müller, M., Keiser, R., Nealen, A., Pauly, M., Gross, M., and Alexa, M. 2004. Point based animation of elastic, plastic and melting objects. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '04). Eurographics Association, 141--151. Google ScholarDigital Library
- Nealen, A., Müller, M., Keiser, R., Boxerman, E., and Carlson, M. 2005. Physically based deformable models in computer graphics. In Eurographics '05 STAR Reports, 71--94.Google Scholar
- Park, S. I. and Hodgins, J. K. 2006. Capturing and animating skin deformation in human motion. ACM Trans. Graph. 25, 3, 881--889. Google ScholarDigital Library
- Pauly, M., Keiser, R., Adams, B., Dutré, P., Gross, M., and Guibas, L. J. 2005. Meshless animation of fracturing solids. ACM Trans. Graph. 24, 3, 957--964. Google ScholarDigital Library
- Pekelny, Y. and Gotsman, C. 2008. Articulated object reconstruction and markerless motion capture from depth video. Comput. Graph. Forum (Proceedings of the Eurographics EG'08) 27, 2, 4.Google Scholar
- PMD. 2009. http://www.pmdtec.com.Google Scholar
- Sand, P., McMillan, L., and Popović, J. 2003. Continuous capture of skin deformation. ACM Trans. Graph. 22, 3, 578--586. Google ScholarDigital Library
- Sharf, A., Alcantara, D. A., Lewiner, T., Greif, C., Sheffer, A., Amenta, N., and Cohen-Or, D. 2008. Space-Time surface reconstruction using incompressible flow. ACM Trans. Graph. (Proc. Siggraph Asia). Google ScholarDigital Library
- Sheffer, A. and Kraevoy, V. 2004. Pyramid coordinates for morphing and deformation. In Proceedings of the 3D Data Processing, Visualization, and Transmission, 2nd International Symposium on (3DPVT'04). IEEE Computer Society, 68--75. Google ScholarDigital Library
- Shewchuk, J. R. 1994. An introduction to the conjugate gradient method without the agonizing pain. Tech. rep., Carnegie Mellon University, School of Computer Science. Google ScholarDigital Library
- Shi, L., Yu, Y., Bell, N., and Feng, W.-W. 2006. A fast multigrid algorithm for mesh deformation. ACM Trans. Graph. 25, 3, 1108--1117. Google ScholarDigital Library
- Shi, X., Zhou, K., Tong, Y., Desbrun, M., Bao, H., and Guo, B. 2007. Mesh puppetry: Cascading optimization of mesh deformation with inverse kinematics. In ACM SIGGRAPH Papers. ACM, New York, 81. Google ScholarDigital Library
- Starck, J. and Hilton, A. 2005. Spherical matching for temporal correspondence of non-rigid surfaces. In Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV'05). IEEE Computer Society, 1387--1394. Google ScholarDigital Library
- Sumner, R. W., Schmid, J., and Pauly, M. 2007. Embedded deformation for shape manipulation. In ACM SIGGRAPH Papers. ACM, New York, 80. Google ScholarDigital Library
- Sumner, R. W., Zwicker, M., Gotsman, C., and Popović, J. 2005. Mesh-Based inverse kinematics. ACM Trans. Graph. 24, 3, 488--495. Google ScholarDigital Library
- Süssmuth, J., Winter, M., and Greiner, G. 2008. Reconstructing animated meshes from time-varying point clouds. Comput. Graph. Forum 27, 5, 1469--1476. Google ScholarDigital Library
- Terzopoulos, D., Platt, J., Barr, A., and Fleischer, K. 1987. Elastically deformable models. In Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques. ACM, New York, 205--214. Google ScholarDigital Library
- Varanasi, K., Zaharescu, A., Boyer, E., and Horaud, R. P. 2008. Temporal surface tracking using mesh evolution. In Proceedings of the 10th European Conference on Computer Vision. Lecture Notes in Computer Science. Part II. Springer-Verlag, 30--43. Google ScholarDigital Library
- Wand, M., Jenke, P., Huang, Q., Bokeloh, M., Guibas, L., and Schilling, A. 2007. Reconstruction of deforming geometry from time-varying point clouds. In Proceedings of the 5th Eurographics Symposium on Geometry Processing (SGP'07). Eurographics Association, 49--58. Google ScholarDigital Library
- Weise, T., Leibe, B., and Gool, L. V. 2007. Fast 3D scanning with automatic motion compensation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'07).Google Scholar
- Würmlin, S., Lamboray, E., Staadt, O. G., and Gross, M. H. 2002. 3D video recorder. In Proceedings of the 10th Pacific Conference on Computer Graphics and Applications (PG '02) . IEEE Computer Society, 325. Google ScholarDigital Library
- Xu, W., Zhou, K., Yu, Y., Tan, Q., Peng, Q., and Guo, B. 2007. Gradient domain editing of deforming mesh sequences. In ACM SIGGRAPH Papers. ACM, New York, 84. Google ScholarDigital Library
- Zhang, L., Curless, B., and Seitz, S. M. 2003. Spacetime stereo: Shape recovery for dynamic scenes. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 367--374.Google Scholar
- Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: High-Resolution capture for modeling and animation. In ACM Annual Conference on Computer Graphics. 548--558. Google ScholarDigital Library
- Zhou, K., Huang, J., Snyder, J., Liu, X., Bao, H., Guo, B., and Shum, H.-Y. 2005. Large mesh deformation using the volumetric graph Laplacian. ACM Trans. Graph. 24, 3, 496--503. Google ScholarDigital Library
- Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-Quality video view interpolation using a layered representation. In ACM SIGGRAPH Papers. ACM, New York, 600--608. Google ScholarDigital Library
- Zwicker, M., Pfister, H., van Baar, J., and Gross, M. 2001. Surface splatting. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. ACM, New York, 371--378. Google ScholarDigital Library
Index Terms
- Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data
Recommendations
Real-time non-rigid reconstruction using an RGB-D camera
We present a combined hardware and software solution for markerless reconstruction of non-rigidly deforming physical objects with arbitrary shape in real-time. Our system uses a single self-contained stereo camera unit built from off-the-shelf ...
Robust single-view geometry and motion reconstruction
We present a framework and algorithms for robust geometry and motion reconstruction of complex deforming shapes. Our method makes use of a smooth template that provides a crude approximation of the scanned object and serves as a geometric and ...
3D surface reconstruction from multiview photographic images using 2D edge contours
Most techniques for reconstructing 3D shapes from multi-view 2D photographic images require a large number of images. In this paper, we present a new method for reconstructing 3D surfaces, represented by sets of polygons, using a small number, e.g. 10, ...
Comments