Skip to main content

2016 | OriginalPaper | Buchkapitel

MARLow: A Joint Multiplanar Autoregressive and Low-Rank Approach for Image Completion

verfasst von : Mading Li, Jiaying Liu, Zhiwei Xiong, Xiaoyan Sun, Zongming Guo

Erschienen in: Computer Vision – ECCV 2016

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, we propose a novel multiplanar autoregressive (AR) model to exploit the correlation in cross-dimensional planes of a similar patch group collected in an image, which has long been neglected by previous AR models. On that basis, we then present a joint multiplanar AR and low-rank based approach (MARLow) for image completion from random sampling, which exploits the nonlocal self-similarity within natural images more effectively. Specifically, the multiplanar AR model constraints the local stationarity in different cross-sections of the patch group, while the low-rank minimization captures the intrinsic coherence of nonlocal patches. The proposed approach can be readily extended to multichannel images (e.g. color images), by simultaneously considering the correlation in different channels. Experimental results demonstrate that the proposed approach significantly outperforms state-of-the-art methods, even if the pixel missing rate is as high as 90 %.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Buades, A., Coll, B., Morel, J.M.: A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 4(2), 490–530 (2005)MathSciNetCrossRefMATH Buades, A., Coll, B., Morel, J.M.: A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 4(2), 490–530 (2005)MathSciNetCrossRefMATH
2.
Zurück zum Zitat Cai, J.F., Cands, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010)MathSciNetCrossRefMATH Cai, J.F., Cands, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010)MathSciNetCrossRefMATH
4.
Zurück zum Zitat Chen, Y.L., Hsu, C.T., Liao, H.Y.: Simultaneous tensor decomposition and completion using factor priors. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 577–591 (2014)CrossRef Chen, Y.L., Hsu, C.T., Liao, H.Y.: Simultaneous tensor decomposition and completion using factor priors. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 577–591 (2014)CrossRef
5.
Zurück zum Zitat Chierchia, G., Pustelnik, N., Pesquet-Popescu, B., Pesquet, J.C.: A nonlocal structure tensor-based approach for multicomponent image recovery problems. IEEE Trans. Image Process. 23(12), 5531–5544 (2014)MathSciNetCrossRef Chierchia, G., Pustelnik, N., Pesquet-Popescu, B., Pesquet, J.C.: A nonlocal structure tensor-based approach for multicomponent image recovery problems. IEEE Trans. Image Process. 23(12), 5531–5544 (2014)MathSciNetCrossRef
6.
Zurück zum Zitat Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)MathSciNetCrossRef Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)MathSciNetCrossRef
7.
Zurück zum Zitat Dong, W., Shi, G., Li, X.: Nonlocal image restoration with bilateral variance estimation: a low-rank approach. IEEE Trans. Image Process. 22(2), 700–711 (2013)MathSciNetCrossRef Dong, W., Shi, G., Li, X.: Nonlocal image restoration with bilateral variance estimation: a low-rank approach. IEEE Trans. Image Process. 22(2), 700–711 (2013)MathSciNetCrossRef
8.
Zurück zum Zitat Dong, W., Zhang, L., Lukac, R., Shi, G.: Sparse representation based image interpolation with nonlocal autoregressive modeling. IEEE Trans. Image Process. 22(4), 1382–1394 (2013)MathSciNetCrossRef Dong, W., Zhang, L., Lukac, R., Shi, G.: Sparse representation based image interpolation with nonlocal autoregressive modeling. IEEE Trans. Image Process. 22(4), 1382–1394 (2013)MathSciNetCrossRef
9.
Zurück zum Zitat Goh, W., Chong, M., Kalra, S., Krishnan, D.: Bi-directional 3D auto-regressive model approach to motion picture restoration. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 4, pp. 2275–2278, May 1996 Goh, W., Chong, M., Kalra, S., Krishnan, D.: Bi-directional 3D auto-regressive model approach to motion picture restoration. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 4, pp. 2275–2278, May 1996
10.
Zurück zum Zitat He, L., Wang, Y.: Iterative support detection-based split bregman method for wavelet frame-based image inpainting. IEEE Trans. Image Process. 23(12), 5470–5485 (2014)MathSciNetCrossRef He, L., Wang, Y.: Iterative support detection-based split bregman method for wavelet frame-based image inpainting. IEEE Trans. Image Process. 23(12), 5470–5485 (2014)MathSciNetCrossRef
11.
Zurück zum Zitat Heide, F., Heidrich, W., Wetzstein, G.: Fast and flexible convolutional sparse coding. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5135–5143, June 2015 Heide, F., Heidrich, W., Wetzstein, G.: Fast and flexible convolutional sparse coding. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5135–5143, June 2015
12.
Zurück zum Zitat Ji, H., Liu, C., Shen, Z., Xu, Y.: Robust video denoising using low rank matrix completion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1791–1798, June 2010 Ji, H., Liu, C., Shen, Z., Xu, Y.: Robust video denoising using low rank matrix completion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1791–1798, June 2010
13.
Zurück zum Zitat Kokaram, A., Rayner, P.: Detection and interpolation of replacement noise in motion picture sequences using 3D autoregressive modelling. In: IEEE International Symposium on Circuits and Systems, vol. 3, pp. 21–24, May 1994 Kokaram, A., Rayner, P.: Detection and interpolation of replacement noise in motion picture sequences using 3D autoregressive modelling. In: IEEE International Symposium on Circuits and Systems, vol. 3, pp. 21–24, May 1994
14.
Zurück zum Zitat Li, X., Orchard, M.: New edge-directed interpolation. IEEE Trans. Image Process. 10(10), 1521–1527 (2001)CrossRef Li, X., Orchard, M.: New edge-directed interpolation. IEEE Trans. Image Process. 10(10), 1521–1527 (2001)CrossRef
15.
Zurück zum Zitat Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values in visual data. In: IEEE International Conference on Computer Vision, pp. 2114–2121, September 2009 Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values in visual data. In: IEEE International Conference on Computer Vision, pp. 2114–2121, September 2009
16.
Zurück zum Zitat Maggioni, M., Boracchi, G., Foi, A., Egiazarian, K.: Video denoising, deblocking, and enhancement through separable 4-D nonlocal spatiotemporal transforms. IEEE Trans. Image Process. 21(9), 3952–3966 (2012)MathSciNetCrossRef Maggioni, M., Boracchi, G., Foi, A., Egiazarian, K.: Video denoising, deblocking, and enhancement through separable 4-D nonlocal spatiotemporal transforms. IEEE Trans. Image Process. 21(9), 3952–3966 (2012)MathSciNetCrossRef
17.
Zurück zum Zitat Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Non-local sparse models for image restoration. In: IEEE International Conference on Computer Vision, pp. 2272–2279, September 2009 Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Non-local sparse models for image restoration. In: IEEE International Conference on Computer Vision, pp. 2272–2279, September 2009
18.
Zurück zum Zitat Ono, S., Miyata, T., Yamada, I.: Cartoon-texture image decomposition using blockwise low-rank texture characterization. IEEE Trans. Image Process. 23(3), 1128–1142 (2014)MathSciNetCrossRef Ono, S., Miyata, T., Yamada, I.: Cartoon-texture image decomposition using blockwise low-rank texture characterization. IEEE Trans. Image Process. 23(3), 1128–1142 (2014)MathSciNetCrossRef
19.
Zurück zum Zitat Roth, S., Black, M.: Fields of experts: a framework for learning image priors. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 860–867, June 2005 Roth, S., Black, M.: Fields of experts: a framework for learning image priors. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 860–867, June 2005
20.
Zurück zum Zitat Roth, S., Black, M.J.: Fields of experts. Int. J. Comput. Vis. 82(2), 205–229 (2009)CrossRef Roth, S., Black, M.J.: Fields of experts. Int. J. Comput. Vis. 82(2), 205–229 (2009)CrossRef
21.
Zurück zum Zitat Takeda, H., Farsiu, S., Milanfar, P.: Robust kernel regression for restoration and reconstruction of images from sparse noisy data. In: IEEE International Conference on Image Processing, pp. 1257–1260 (2006) Takeda, H., Farsiu, S., Milanfar, P.: Robust kernel regression for restoration and reconstruction of images from sparse noisy data. In: IEEE International Conference on Image Processing, pp. 1257–1260 (2006)
22.
Zurück zum Zitat Zhang, D., Hu, Y., Ye, J., Li, X., He, X.: Matrix completion by truncated nuclear norm regularization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2192–2199, June 2012 Zhang, D., Hu, Y., Ye, J., Li, X., He, X.: Matrix completion by truncated nuclear norm regularization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2192–2199, June 2012
23.
Zurück zum Zitat Zhang, J., Zhao, D., Gao, W.: Group-based sparse representation for image restoration. IEEE Trans. Image Process. 23(8), 3336–3351 (2014)MathSciNetCrossRef Zhang, J., Zhao, D., Gao, W.: Group-based sparse representation for image restoration. IEEE Trans. Image Process. 23(8), 3336–3351 (2014)MathSciNetCrossRef
24.
Zurück zum Zitat Zhang, J., Zhao, D., Xiong, R., Ma, S., Gao, W.: Image restoration using joint statistical modeling in a space-transform domain. IEEE Trans. Circ. Syst. Video Technol. 24(6), 915–928 (2014)CrossRef Zhang, J., Zhao, D., Xiong, R., Ma, S., Gao, W.: Image restoration using joint statistical modeling in a space-transform domain. IEEE Trans. Circ. Syst. Video Technol. 24(6), 915–928 (2014)CrossRef
25.
Zurück zum Zitat Zhang, X., Wu, X.: Image interpolation by adaptive 2-d autoregressive modeling and soft-decision estimation. IEEE Trans. Image Process. 17(6), 887–896 (2008)MathSciNetCrossRef Zhang, X., Wu, X.: Image interpolation by adaptive 2-d autoregressive modeling and soft-decision estimation. IEEE Trans. Image Process. 17(6), 887–896 (2008)MathSciNetCrossRef
26.
Zurück zum Zitat Zhang, Z., Ely, G., Aeron, S., Hao, N., Kilmer, M.: Novel methods for multilinear data completion and de-noising based on tensor-svd. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3842–3849, June 2014 Zhang, Z., Ely, G., Aeron, S., Hao, N., Kilmer, M.: Novel methods for multilinear data completion and de-noising based on tensor-svd. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3842–3849, June 2014
27.
Zurück zum Zitat Zhou, D., Shen, X., Dong, W.: Image zooming using directional cubic convolution interpolation. IET Image Process. 6(6), 627–634 (2012)MathSciNetCrossRef Zhou, D., Shen, X., Dong, W.: Image zooming using directional cubic convolution interpolation. IET Image Process. 6(6), 627–634 (2012)MathSciNetCrossRef
28.
Zurück zum Zitat Zhou, M., Chen, H., Paisley, J., Ren, L., Li, L., Xing, Z., Dunson, D., Sapiro, G., Carin, L.: Nonparametric bayesian dictionary learning for analysis of noisy and incomplete images. IEEE Trans. Image Process. 21(1), 130–144 (2012)MathSciNetCrossRef Zhou, M., Chen, H., Paisley, J., Ren, L., Li, L., Xing, Z., Dunson, D., Sapiro, G., Carin, L.: Nonparametric bayesian dictionary learning for analysis of noisy and incomplete images. IEEE Trans. Image Process. 21(1), 130–144 (2012)MathSciNetCrossRef
Metadaten
Titel
MARLow: A Joint Multiplanar Autoregressive and Low-Rank Approach for Image Completion
verfasst von
Mading Li
Jiaying Liu
Zhiwei Xiong
Xiaoyan Sun
Zongming Guo
Copyright-Jahr
2016
DOI
https://doi.org/10.1007/978-3-319-46478-7_50