Skip to main content
SpringerLink
Log in

Image-Based Rendering Using Image-Based Priors

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Given a set of images acquired from known viewpoints, we describe a method for synthesizing the image which would be seen from a new viewpoint. In contrast to existing techniques, which explicitly reconstruct the 3D geometry of the scene, we transform the problem to the reconstruction of colour rather than depth. This retains the benefits of geometric constraints, but projects out the ambiguities in depth estimation which occur in textureless regions.

On the other hand, regularization is still needed in order to generate high-quality images. The paper’s second contribution is to constrain the generated views to lie in the space of images whose texture statistics are those of the input images. This amounts to an image-based prior on the reconstruction which regularizes the solution, yielding realistic synthetic views. Examples are given of new view generation for cameras interpolated between the acquisition viewpoints—which enables synthetic steadicam stabilization of a sequence with a high level of realism.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • 2d3 Ltd. http://www.2d3.com, 2002.

  • Besag, J. 1986. On the statistical analysis of dirty pictures. J. Royal Stat. Soc. B, 48(3):259–302.

    Google Scholar 

  • Broadhurst, A. and Cipolla, R. 2001. A statistical consistency check for the space carving algorithm. In Proc. ICCV.

  • Debevec, P.E., Taylor, C.J., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. In Proceedings, ACM SIGGRAPH, pp. 11–20.

  • Efros, A. and Leung, T. 1999. Texture synthesis by non-parametric sampling. In Proc. ICCV, pp. 1039–1046.

  • Freeman, W. and Pasztor, E. 1999. Learning low-level vision. In ICCV, pp. 1182–1189.

  • Gortler, S.J., Grzeszczuk, R., Szeliski, R., and Cohen, M.F. 1996. The lumigraph. In SIG-GRAPH96.

  • Grenander, U. and Srivastava, A. 2001. Probability models for clutter in natural images. IEEE PAMI, 23(4):424–429.

    Google Scholar 

  • Huang, J. and Mumford, D. 1999. Statistics of natural images and models. In Proc. CVPR, pp. 1541–1547.

  • Irani, M., Hassner, T., and Anandan, P. 2002. What does the scene look like from a scene point? In Proc. ECCV.

  • Koch, R. 1995. 3D surface reconstruction from stereoscopic image sequences. In Proc. ICCV, pp. 109–114.

  • Koch, R., Heigl, B., and Pollefeys, Marc 2001. Image-based rendering from uncalibrated light-fields with scalable geometry. In G. Gimel’farb (Eds.) R. Klette, T. Huang, {Multi-Image Analysis, Springer LNCS 2032}, pp. 51–66.

  • Kutulakos, K. and Seitz, S. 1999. A theory of shape by space carving. In Proc. ICCV, pp. 307–314.

  • Levoy, M. and Hanrahan, P. 1996. Light field rendering. In SIGGRAPH96.

  • Matusik, W., Buehler, C., Raskar, R., McMillan, L., and Gortler, S. 2000. Image-based visual hulls. In Proc. ACMSIGGRAPH, pp. 369–374.

  • Matusik, W., Pfister, H., Beardsley, P.A., Ngan, A., Ziegler, R., and McMillan, L. 2002. Image-based 3d photography using opacity hulls. In Proc. ACMSIGGRAPH.

  • McMillan, L. and Bishop, G. 1995. Plenoptic modeling: An image-based rendering system. In SIGGRAPH95.

  • Rademacher, P. 1999. View-dependent geometry. In Proc. ACM. SIGGRAPH pp. 439–446.

  • Scharstein, D. 1999. View Synthesis Using Stereo Vision, Vol. 1583 of LNCS. Springer-Verlag.

  • Scharstein, D. and Szeliski, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV, 47(l):7–42.

    Google Scholar 

  • Seitz, S.M. and Dyer, C.R. 1997. Photorealistic scene reconstruction by voxel coloring. In Proc. CVPR, pp. 1067–1073.

  • Sminchisescu, C. and Triggs, B. 2002. Building roadmaps of local minima of visual models. In Proc. ECCV, Vol. 1, pp. 566–582.

    Google Scholar 

  • Srivastava, A., Lee, A., Simoncelli, E., and Zhu, S. 2003. On advances in statistical modeling of natural images, 18(l):17–33.

  • Szeliski, R. and Golland, P. 1998. Stereo matching with transparency and matting. In Proc. ICCV, pp. 517–524.

  • Wei, L.-Y. and Levoy, M. 2000. Fast texture synthesis using tree-structured vector quantization. In Proc. ACM SIGGRAPH, pp. 479–488.

  • Wexler, Y. and Chellappa, R. 2001. View synthesis using convex and visual hulls. In Proc. BMVC.

Download references

Author information

Authors and Affiliations

  1. Engineering Science, The University of Oxford, UK

    Andrew Fitzgibbon

  2. Computer Science and Applied Math, The Weizmann Institute of Science, Israel

    Yonatan Wexler

  3. Engineering Science, The University of Oxford, UK

    Andrew Zisserman

Authors
  1. Andrew Fitzgibbon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yonatan Wexler
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrew Zisserman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrew Fitzgibbon.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fitzgibbon, A., Wexler, Y. & Zisserman, A. Image-Based Rendering Using Image-Based Priors. Int J Comput Vision 63, 141–151 (2005). https://doi.org/10.1007/s11263-005-6643-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-005-6643-9

Keywords

Search

Navigation