Skip to main content

2017 | OriginalPaper | Buchkapitel

Exemplar-Based Pixel by Pixel Inpainting Based on Patch Shift

verfasst von : Zhenping Qiang, Libo He, Dan Xu

Erschienen in: Computer Vision

Verlag: Springer Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper presents a novel exemplar-based image inpainting method. It performs patch-shift based information statistics and incorporates the result into the data filling process. The central idea to ensure the confidence pixel value to fill is based on patch shift that captures the statistic information from the expanded filling patches’ most similar patches. Our approach outperforms the original exemplar-based inpainting in a large number of examples on removing large occluding objects and thin scratches in real and synthetic images, at the cost of some additional computation. It inherits well-known advantages of the exemplar-based inpainting method, such as the capability of maintaining the integrity and consistency of image structure information, and the adaptability to many application scenarios. Experiments on synthetic and natural images show the advantages of our approach, especially, in the big target removing task our method yields generally better results.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)CrossRefMATH Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)CrossRefMATH
2.
Zurück zum Zitat Arias, P., Facciolo, G., Caselles, V., Sapiro, G.: A variational framework for exemplar-based image inpainting. Int. J. Comput. Vision 93(3), 319–347 (2011)MathSciNetCrossRefMATH Arias, P., Facciolo, G., Caselles, V., Sapiro, G.: A variational framework for exemplar-based image inpainting. Int. J. Comput. Vision 93(3), 319–347 (2011)MathSciNetCrossRefMATH
3.
Zurück zum Zitat Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans. Image Process. 10(8), 1200–1211 (2001)MathSciNetCrossRefMATH Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans. Image Process. 10(8), 1200–1211 (2001)MathSciNetCrossRefMATH
4.
Zurück zum Zitat Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.: Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph.-TOG 28(3), 24 (2009) Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.: Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph.-TOG 28(3), 24 (2009)
5.
Zurück zum Zitat Bertalmio, M., Bertozzi, A.L., Sapiro, G.: Navier-stokes, fluid dynamics, and image and video inpainting. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 1, pp. I-355–I-362 (2001) Bertalmio, M., Bertozzi, A.L., Sapiro, G.: Navier-stokes, fluid dynamics, and image and video inpainting. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 1, pp. I-355–I-362 (2001)
6.
Zurück zum Zitat Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 417–424. ACM Press/Addison-Wesley Publishing Co. (2000) Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 417–424. ACM Press/Addison-Wesley Publishing Co. (2000)
7.
Zurück zum Zitat Bertalmio, M., Vese, L., Sapiro, G., Osher, S.: Simultaneous structure and texture image inpainting. IEEE Trans. Image Process. 12(8), 882–889 (2003)CrossRef Bertalmio, M., Vese, L., Sapiro, G., Osher, S.: Simultaneous structure and texture image inpainting. IEEE Trans. Image Process. 12(8), 882–889 (2003)CrossRef
8.
Zurück zum Zitat Bornemann, F., März, T.: Fast image inpainting based on coherence transport. J. Math. Imaging Vision 28(3), 259–278 (2007)MathSciNetCrossRef Bornemann, F., März, T.: Fast image inpainting based on coherence transport. J. Math. Imaging Vision 28(3), 259–278 (2007)MathSciNetCrossRef
9.
Zurück zum Zitat Bugeau, A., Bertalmío, M., Caselles, V., Sapiro, G.: A comprehensive framework for image inpainting. IEEE Trans. Image Process. 19(10), 2634–2645 (2010)MathSciNetCrossRefMATH Bugeau, A., Bertalmío, M., Caselles, V., Sapiro, G.: A comprehensive framework for image inpainting. IEEE Trans. Image Process. 19(10), 2634–2645 (2010)MathSciNetCrossRefMATH
10.
Zurück zum Zitat Chan, T.F., Shen, J.: Nontexture inpainting by curvature-driven diffusions. J. Vis. Commun. Image Represent. 12(4), 436–449 (2001)CrossRef Chan, T.F., Shen, J.: Nontexture inpainting by curvature-driven diffusions. J. Vis. Commun. Image Represent. 12(4), 436–449 (2001)CrossRef
11.
Zurück zum Zitat Criminisi, A., Perez, P., Toyama, K.: Object removal by exemplar-based inpainting. In: Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. II-721–II-728. IEEE (2003) Criminisi, A., Perez, P., Toyama, K.: Object removal by exemplar-based inpainting. In: Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. II-721–II-728. IEEE (2003)
12.
Zurück zum Zitat Daisy, M., Buyssens, P., Tschumperlé, D., Lézoray, O.: A smarter exemplar-based inpainting algorithm using local and global heuristics for more geometric coherence. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 4622–4626. IEEE (2014) Daisy, M., Buyssens, P., Tschumperlé, D., Lézoray, O.: A smarter exemplar-based inpainting algorithm using local and global heuristics for more geometric coherence. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 4622–4626. IEEE (2014)
13.
Zurück zum Zitat Drori, I., Cohen-Or, D., Yeshurun, H.: Fragment-based image completion. ACM Trans. Graph. (TOG) 22, 303–312 (2003). ACMCrossRef Drori, I., Cohen-Or, D., Yeshurun, H.: Fragment-based image completion. ACM Trans. Graph. (TOG) 22, 303–312 (2003). ACMCrossRef
14.
Zurück zum Zitat Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: The Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 2, pp. 1033–1038 (1999) Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: The Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 2, pp. 1033–1038 (1999)
15.
Zurück zum Zitat Elad, M., Starck, J.L., Querre, P., Donoho, D.L.: Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Appl. Comput. Harmon. Anal. 19(3), 340–358 (2005)MathSciNetCrossRefMATH Elad, M., Starck, J.L., Querre, P., Donoho, D.L.: Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Appl. Comput. Harmon. Anal. 19(3), 340–358 (2005)MathSciNetCrossRefMATH
16.
Zurück zum Zitat Fadili, M.J., Starck, J.L., Murtagh, F.: Inpainting and zooming using sparse representations. Comput. J. 52(1), 64–79 (2009)CrossRef Fadili, M.J., Starck, J.L., Murtagh, F.: Inpainting and zooming using sparse representations. Comput. J. 52(1), 64–79 (2009)CrossRef
17.
Zurück zum Zitat Fedorov, V., Facciolo, G., Arias, P.: Variational framework for non-local inpainting. Image Process. On Line 5, 362–386 (2015)MathSciNetCrossRef Fedorov, V., Facciolo, G., Arias, P.: Variational framework for non-local inpainting. Image Process. On Line 5, 362–386 (2015)MathSciNetCrossRef
18.
Zurück zum Zitat Guillemot, C., Le Meur, O.: Image inpainting: overview and recent advances. IEEE Signal Process. Mag. 31(1), 127–144 (2014)CrossRef Guillemot, C., Le Meur, O.: Image inpainting: overview and recent advances. IEEE Signal Process. Mag. 31(1), 127–144 (2014)CrossRef
19.
Zurück zum Zitat He, K., Sun, J.: Image completion approaches using the statistics of similar patches. IEEE Trans. Pattern Anal. Mach. Intell. 36(12), 2423–2435 (2014)CrossRef He, K., Sun, J.: Image completion approaches using the statistics of similar patches. IEEE Trans. Pattern Anal. Mach. Intell. 36(12), 2423–2435 (2014)CrossRef
20.
Zurück zum Zitat Koh, M.S., Rodriguez-Marek, E.: Turbo inpainting: iterative K-SVD with a new dictionary. In: IEEE International Workshop on Multimedia Signal Processing, MMSP 2009, pp. 1–6. IEEE (2009) Koh, M.S., Rodriguez-Marek, E.: Turbo inpainting: iterative K-SVD with a new dictionary. In: IEEE International Workshop on Multimedia Signal Processing, MMSP 2009, pp. 1–6. IEEE (2009)
21.
Zurück zum Zitat Komodakis, N.: Image completion using global optimization. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 442–452. IEEE (2006) Komodakis, N.: Image completion using global optimization. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 442–452. IEEE (2006)
22.
Zurück zum Zitat Komodakis, N., Tziritas, G.: Image completion using efficient belief propagation via priority scheduling and dynamic pruning. IEEE Trans. Image Process. 16(11), 2649–2661 (2007)MathSciNetCrossRef Komodakis, N., Tziritas, G.: Image completion using efficient belief propagation via priority scheduling and dynamic pruning. IEEE Trans. Image Process. 16(11), 2649–2661 (2007)MathSciNetCrossRef
23.
Zurück zum Zitat Le Meur, O., Gautier, J., Guillemot, C.: Examplar-based inpainting based on local geometry. In: 2011 18th IEEE International Conference on Image Processing (ICIP), pp. 3401–3404. IEEE (2011) Le Meur, O., Gautier, J., Guillemot, C.: Examplar-based inpainting based on local geometry. In: 2011 18th IEEE International Conference on Image Processing (ICIP), pp. 3401–3404. IEEE (2011)
25.
Zurück zum Zitat Pritch, Y., Kav-Venaki, E., Peleg, S.: Shift-map image editing. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 151–158. IEEE (2009) Pritch, Y., Kav-Venaki, E., Peleg, S.: Shift-map image editing. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 151–158. IEEE (2009)
26.
Zurück zum Zitat Sun, J., Yuan, L., Jia, J., Shum, H.Y.: Image completion with structure propagation. ACM Trans. Graph. (ToG) 24(3), 861–868 (2005)CrossRef Sun, J., Yuan, L., Jia, J., Shum, H.Y.: Image completion with structure propagation. ACM Trans. Graph. (ToG) 24(3), 861–868 (2005)CrossRef
27.
Zurück zum Zitat Telea, A.: An image inpainting technique based on the fast marching method. J. Graph. Tools 9(1), 23–34 (2004)CrossRef Telea, A.: An image inpainting technique based on the fast marching method. J. Graph. Tools 9(1), 23–34 (2004)CrossRef
28.
Zurück zum Zitat Wei, L.Y., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 479–488. ACM Press/Addison-Wesley Publishing Co. (2000) Wei, L.Y., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 479–488. ACM Press/Addison-Wesley Publishing Co. (2000)
29.
Zurück zum Zitat Wexler, Y., Shechtman, E., Irani, M.: Space-time video completion. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 1, pp. I-120–I-127. IEEE (2004) Wexler, Y., Shechtman, E., Irani, M.: Space-time video completion. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 1, pp. I-120–I-127. IEEE (2004)
30.
Zurück zum Zitat Xu, Z., Sun, J.: Image inpainting by patch propagation using patch sparsity. IEEE Trans. Image Process. 19(5), 1153–1165 (2010)MathSciNetCrossRefMATH Xu, Z., Sun, J.: Image inpainting by patch propagation using patch sparsity. IEEE Trans. Image Process. 19(5), 1153–1165 (2010)MathSciNetCrossRefMATH
Metadaten
Titel
Exemplar-Based Pixel by Pixel Inpainting Based on Patch Shift
verfasst von
Zhenping Qiang
Libo He
Dan Xu
Copyright-Jahr
2017
Verlag
Springer Singapore
DOI
https://doi.org/10.1007/978-981-10-7302-1_31

Premium Partner