Skip to main content
SpringerLink
Log in

Interactive image completion with perspective correction

  • Special Issue Paper
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

We present an interactive system for fragment-based image completion which exploits information about the approximate 3D structure in a scene in order to estimate and apply perspective corrections when copying a source fragment to a target position. Even though implicit 3D information is used, the interaction is strictly 2D, which makes the user interface very simple and intuitive. We propose different interaction metaphors in our system for providing 3D information interactively. Our search and matching procedure is done in the Fourier domain, and hence it is very fast and it allows us to use large fragments and multiple source images with high resolution while still obtaining interactive response times. Our image completion technique also takes user-specified structure information into account where we generalize the concept of feature curves to arbitrary sets of feature pixels. We demonstrate our technique on a number of difficult completion tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Arya, S., Mount, D.M.: Approximate nearest neighbor queries in fixed dimensions. In: SODA ’93: Proceedings of the fourth annual ACM-SIAM Symposium on Discrete algorithms, pp. 271–280. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA (1993)

    Google Scholar 

  2. Ashikhmin, M.: Synthesizing natural textures. In: SI3D ’01: Proceedings of the 2001 symposium on Interactive 3D graphics, pp. 217–226. ACM Press, New York, NY, USA (2001)

    Chapter  Google Scholar 

  3. Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and gray levels. In: IEEE Transactions on Image Processing, vol. 10, pp. 1200–1211 (2001)

  4. Barrett, W.A., Cheney, A.S.: Object-based image editing. In: SIGGRAPH ’02: Proceedings of the 29th annual conference on Computer graphics and interactive techniques, pp. 777–784 (2002)

  5. Bertalmio, M., Bertozzi, A., Sapiro, G.: Navier-stokes, fluid dynamics, and image and video inpainting. In: Proc. of Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 355–362 (2001)

  6. Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: SIGGRAPH ’00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pp. 417–424. ACM Press/Addison-Wesley Publishing Co., New York, NY, USA (2000)

    Chapter  Google Scholar 

  7. Bertalmio, M., Vese, L.A., Sapiro, G., Osher, S.: Simultaneous structure and texture image inpainting. In: CVPR (2), pp. 707–712 (2003)

  8. Boehm, W., Prautzsch, H.: Geometric Concepts for Geometric Design. A K Peters, Ltd. (1994)

  9. Criminisi, A., Pérez, P., Toyama, K.: Object removal by exemplar-based inpainting. In: CVPR (2), pp. 721–728 (2003)

  10. Datar, M., Immorlica, N., Indyk, P., Mirrokni, V.S.: Locality-sensitive hashing scheme based on p-stable distributions. In: SCG ’04: Proceedings of the twentieth annual symposium on Computational geometry, pp. 253–262. ACM Press, New York, NY, USA (2004)

    Chapter  Google Scholar 

  11. Debevec, P.E., Taylor, C.J., Malik, J.: Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach. In: SIGGRAPH ’96: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pp. 11–20 (1996)

  12. Drori, I., Cohen-Or, D., Yeshurun, H.: Fragment-based image completion. ACM Transactions on Graphics 22(3), 303–312 (2003)

    Article  Google Scholar 

  13. Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: SIGGRAPH ’01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pp. 341–346. ACM Press, New York, NY, USA (2001)

    Chapter  Google Scholar 

  14. Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: ICCV ’99: Proceedings of the International Conference on Computer Vision-Volume 2, p. 1033 (1999)

  15. Frigo, M., Johnson, S.G.: The design and implementation of FFTW3. Proceedings of the IEEE 93(2), 216–231 (2005)

  16. Harrison, P.: A non-hierarchical procedure for re-synthesis of complex textures. In: International Conference in Central Europe on Computer Graphics and Visualization, pp. 190–197 (2001)

  17. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2003)

  18. Hel-Or, Y., Hel-Or, H.: Real time pattern matching using projection kernels. In: ICCV, pp. 1486–1493 (2003)

  19. Horry, Y., Anjyo, K.I., Arai, K.: Tour into the picture: using a spidery mesh interface to make animation from a single image. In: SIGGRAPH ’97: Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pp. 225–232 (1997)

  20. Jia, J., Tang, C.K.: Image repairing: Robust image synthesis by adaptive nd tensor voting. In: CVPR (1), pp. 643–650 (2003)

  21. Kasson, J.M., Plouffe, W.: An analysis of selected computer interchange color spaces. ACM Trans. Graph. 11(4), 373–405 (1992)

    Article  MATH  Google Scholar 

  22. Komodakis, N., Tziritas, G.: Image completion using global optimization. In: CVPR06, pp. I: 442–452 (2006)

  23. Kwatra, V., Essa, I., Bobick, A., Kwatra, N.: Texture optimization for example-based synthesis. ACM Trans. Graph. 24(3), 795–802 (2005)

    Article  Google Scholar 

  24. Kwatra, V., Schoedl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: Image and video synthesis using graph cuts. ACM Transactions on Graphics, SIGGRAPH 2003 22(3), 277–286 (2003)

    Google Scholar 

  25. Liang, L., Liu, C., Xu, Y.Q., Guo, B., Shum, H.Y.: Real-time texture synthesis by patch-based sampling. ACM Trans. Graph. 20(3), 127–150 (2001)

    Article  Google Scholar 

  26. Liu, Y., Lin, W.C., Hays, J.: Near-regular texture analysis and manipulation. ACM Trans. Graph. 23(3), 368–376 (2004)

    Article  Google Scholar 

  27. Oh, B.M., Chen, M., Dorsey, J., Durand, F.: Image-based modeling and photo editing. In: SIGGRAPH ’01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pp. 433–442. ACM Press, New York, NY, USA (2001)

    Chapter  Google Scholar 

  28. Oliveira, M.M., Bowen, B., McKenna, R., Chang, Y.S.: Fast digital image inpainting. In: Proceedings of the International Conference on Visualization, Imaging and Image Processing, pp. 261–266 (2001)

  29. Pavić, D., Schönefeld, V., Kobbelt, L.: Accompanying video. http://www.rwth-graphics.de/downloads/

  30. Pérez, P., Gangnet, M., Blake, A.: Patchworks: Example-based region tiling for image editing. Tech. Rep. MSR-TR-2004-04, Microsoft Research (2004)

  31. Sun, J., Yuan, L., Jia, J., Shum, H.Y.: Image completion with structure propagation. ACM Trans. Graph. 24(3), 861–868 (2005)

    Article  Google Scholar 

  32. Wei, L.Y., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: SIGGRAPH ’00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pp. 479–488. ACM Press/Addison-Wesley Publishing Co., New York, NY, USA (2000)

    Chapter  Google Scholar 

  33. Wilczkowiak, M., Brostow, G.J., Tordoff, B., Cipolla, R.: Hole filling through photomontage. In: 16th British Machine Vision Conference 2005 - BMVC’2005, Oxford, United Kingdom, pp. 492–501 (2005)

Download references

Author information

Authors and Affiliations

  1. Computer Graphics Group, RWTH Aachen University, Aachen, Germany

    Darko Pavić, Volker Schönefeld & Leif Kobbelt

Authors
  1. Darko Pavić
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Volker Schönefeld
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Leif Kobbelt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Darko Pavić.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pavić, D., Schönefeld, V. & Kobbelt, L. Interactive image completion with perspective correction. Visual Comput 22, 671–681 (2006). https://doi.org/10.1007/s00371-006-0050-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-006-0050-2

Keywords

Search

Navigation