Skip to main content

2015 | OriginalPaper | Buchkapitel

Simple, Accurate, and Robust Nonparametric Blind Super-Resolution

verfasst von : Wen-Ze Shao, Michael Elad

Erschienen in: Image and Graphics

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper proposes a simple, accurate, and robust approach to single image blind super-resolution (SR). This task is formulated as a functional to be minimized with respect to both an intermediate super-resolved image and a non- parametric blur-kernel. The proposed method includes a convolution consistency constraint which uses a non-blind learning-based SR result to better guide the estimation process. Another key component is the bi-0-2-norm regularization placed on the super-resolved, sharp image and the blur-kernel, which is shown to be quite beneficial for accurate blur-kernel estimation. The numerical optimization is implemented by coupling the splitting augmented Lagrangian and the conjugate gradient. With the pre-estimated blur-kernel, the final SR image is reconstructed using a simple TV-based non-blind SR method. The new method is demonstrated to achieve better performance than Michaeli and Irani [2] in both terms of the kernel estimation accuracy and image SR quality.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
I.e., knowing the blur kernel.
 
2
In [31] the bi-0-2-norm regularization is shown to achieve state-of-the-art kernel estimation performance. Due to this reason as well as the similarity between blind deblurring and blind SR, we extend the bi-0-2-norm regularization for the nonparametric blind SR problem.
 
3
BCCB: block-circulant with circulant blocks.
 
4
The same meaning applies to \( c_{{\mathbf{k}}}^{i} \).
 
5
We should note that we have also selected a uniform set of parameter values for each of the formulations (8), (9) and (10), respectively, in order to optimize the obtained blind SR performance on a series of experiments. However, it was found that these alternative are still inferior to (7), just similar to the observation made in blind motion deblurring [31].
 
6
Experiments reported in this paper are performed with MATLAB v7.0 on a computer with an Intel i7-4600 M CPU (2.90 GHz) and 8 GB memory.
 
7
In [2] blur-kernels are typically solved with size 9 × 9, 11 × 11 or 13 × 13 for various blind SR problems.
 
Literatur
1.
Zurück zum Zitat Efrat, N., Glasner, D., Apartsin, A., Nadler, B., Levin, A.: Accurate blur models vs. image priors in single image super-resolution. In: Proceedings of IEEE Conference on Computer Vision, pp. 2832–2839. IEEE Press, Washington (2013) Efrat, N., Glasner, D., Apartsin, A., Nadler, B., Levin, A.: Accurate blur models vs. image priors in single image super-resolution. In: Proceedings of IEEE Conference on Computer Vision, pp. 2832–2839. IEEE Press, Washington (2013)
2.
Zurück zum Zitat Michaeli, T., Irani, M.: Nonparametric blind super-resolution. In: Proceedings of IEEE Conference on Computer Vision, pp. 945–952. IEEE Press, Washington (2013) Michaeli, T., Irani, M.: Nonparametric blind super-resolution. In: Proceedings of IEEE Conference on Computer Vision, pp. 945–952. IEEE Press, Washington (2013)
3.
Zurück zum Zitat Freeman, W.T., Pasztor, E.C.: Learning to estimate scenes from images. In: Proceedings of Advances in Neural Information Processing Systems, pp. 775–781. MIT Press, Cambridge (1999) Freeman, W.T., Pasztor, E.C.: Learning to estimate scenes from images. In: Proceedings of Advances in Neural Information Processing Systems, pp. 775–781. MIT Press, Cambridge (1999)
4.
Zurück zum Zitat Baker, S., Kanade, T.: Hallucinating faces. In: Proceedings of IEEE Conference on Automatic Face and Gesture Recognition, pp. 83–88. IEEE Press, Washington (2000) Baker, S., Kanade, T.: Hallucinating faces. In: Proceedings of IEEE Conference on Automatic Face and Gesture Recognition, pp. 83–88. IEEE Press, Washington (2000)
5.
Zurück zum Zitat Nasrollahi, K., Moeslund, T.B.: Super-resolution: a comprehensive survey. Mach. Vis. Appl. 25, 1423–1468 (2014)CrossRef Nasrollahi, K., Moeslund, T.B.: Super-resolution: a comprehensive survey. Mach. Vis. Appl. 25, 1423–1468 (2014)CrossRef
6.
Zurück zum Zitat Mudenagudi, U., Singla, R., Kalra, P.K., Banerjee, S.: Super resolution using graph-cut. In: Narayanan, P.J., Nayar, S.K., Shum, H.-Y. (eds.) ACCV 2006. LNCS, vol. 3852, pp. 385–394. Springer, Heidelberg (2006)CrossRef Mudenagudi, U., Singla, R., Kalra, P.K., Banerjee, S.: Super resolution using graph-cut. In: Narayanan, P.J., Nayar, S.K., Shum, H.-Y. (eds.) ACCV 2006. LNCS, vol. 3852, pp. 385–394. Springer, Heidelberg (2006)CrossRef
7.
Zurück zum Zitat Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Advances and challenges in super-resolution. Int. J. Imaging Syst. Technol. 14, 47–57 (2004)CrossRef Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Advances and challenges in super-resolution. Int. J. Imaging Syst. Technol. 14, 47–57 (2004)CrossRef
8.
Zurück zum Zitat Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution via sparse representation. IEEE TIP 19, 2861–2873 (2010)MathSciNetMATH Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution via sparse representation. IEEE TIP 19, 2861–2873 (2010)MathSciNetMATH
9.
Zurück zum Zitat Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: Proceedings of IEEE International Conference on Computer Vision, pp. 349–356. IEEE Press, Washington (2009) Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: Proceedings of IEEE International Conference on Computer Vision, pp. 349–356. IEEE Press, Washington (2009)
10.
Zurück zum Zitat Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Boissonnat, J.-D., Chenin, P., Cohen, A., Gout, C., Lyche, T., Mazure, M.-L., Schumaker, L. (eds.) Curves and Surfaces 2011. LNCS, vol. 6920, pp. 711–730. Springer, Heidelberg (2012)CrossRef Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Boissonnat, J.-D., Chenin, P., Cohen, A., Gout, C., Lyche, T., Mazure, M.-L., Schumaker, L. (eds.) Curves and Surfaces 2011. LNCS, vol. 6920, pp. 711–730. Springer, Heidelberg (2012)CrossRef
11.
Zurück zum Zitat Timofte, R., Smet, V.D., Gool, L.V.: Anchored neighborhood regression for fast exampled-based super-resolution. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1920–1927. IEEE Press, Washington (2013) Timofte, R., Smet, V.D., Gool, L.V.: Anchored neighborhood regression for fast exampled-based super-resolution. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1920–1927. IEEE Press, Washington (2013)
12.
Zurück zum Zitat Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part IV. LNCS, vol. 8692, pp. 184–199. Springer, Heidelberg (2014)CrossRef Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part IV. LNCS, vol. 8692, pp. 184–199. Springer, Heidelberg (2014)CrossRef
13.
Zurück zum Zitat Peleg, T., Elad, M.: A statistical prediction model based on sparse representations for single image super-resolution. IEEE TIP 23, 2569–2582 (2014)MathSciNetMATH Peleg, T., Elad, M.: A statistical prediction model based on sparse representations for single image super-resolution. IEEE TIP 23, 2569–2582 (2014)MathSciNetMATH
14.
Zurück zum Zitat Yang, J., Lin, Z., Cohen, S.: Fast image super-resolution based on in-place example regression. In: Proceedings of IEEE Conference on CVPR, pp. 1059–1066. IEEE Press, Washington (2013) Yang, J., Lin, Z., Cohen, S.: Fast image super-resolution based on in-place example regression. In: Proceedings of IEEE Conference on CVPR, pp. 1059–1066. IEEE Press, Washington (2013)
15.
Zurück zum Zitat Fattal, R.: Image upsampling via imposed edge statistics. In: ACM Transactions on Graphics, vol. 26, Article No. 95 (2007) Fattal, R.: Image upsampling via imposed edge statistics. In: ACM Transactions on Graphics, vol. 26, Article No. 95 (2007)
16.
Zurück zum Zitat Begin, I., Ferrie, F.R.: PSF recovery from examples for blind super-resolution. In: Proceedings of IEEE Conference on Image Processing, pp. 421–424. IEEE Press, Washington (2007) Begin, I., Ferrie, F.R.: PSF recovery from examples for blind super-resolution. In: Proceedings of IEEE Conference on Image Processing, pp. 421–424. IEEE Press, Washington (2007)
17.
Zurück zum Zitat Wang, Q., Tang, X., Shum, H.: Patch based blind image super resolution. In: Proceedings of IEEE Conference on Computer Vision, pp. 709–716. IEEE Press, Washington (2005) Wang, Q., Tang, X., Shum, H.: Patch based blind image super resolution. In: Proceedings of IEEE Conference on Computer Vision, pp. 709–716. IEEE Press, Washington (2005)
18.
Zurück zum Zitat Joshi, N., Szeliski, R., Kriegman, D.J.: PSF estimation using sharp edge prediction. In: Proceedings of IEEE Conference on CVPR, pp. 1–8. IEEE Press, Washington (2008) Joshi, N., Szeliski, R., Kriegman, D.J.: PSF estimation using sharp edge prediction. In: Proceedings of IEEE Conference on CVPR, pp. 1–8. IEEE Press, Washington (2008)
19.
Zurück zum Zitat Levin, A., Weiss, Y., Durand, F., Freeman, W.T.: Understanding blind deconvolution algorithms. IEEE PAMI 33, 2354–2367 (2011)CrossRef Levin, A., Weiss, Y., Durand, F., Freeman, W.T.: Understanding blind deconvolution algorithms. IEEE PAMI 33, 2354–2367 (2011)CrossRef
20.
Zurück zum Zitat Fergus, R., Singh, B., Hertzmann, A., Roweis, S.T., Freeman, W.T.: Removing camera shake from a single photograph. ACM Trans. Graph. 25, 787–794 (2006)CrossRef Fergus, R., Singh, B., Hertzmann, A., Roweis, S.T., Freeman, W.T.: Removing camera shake from a single photograph. ACM Trans. Graph. 25, 787–794 (2006)CrossRef
21.
Zurück zum Zitat Chang, H., Yeung, D.-Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of IEEE International Conference on Computer Vision, pp. 275–282. IEEE Press, Washington (2004) Chang, H., Yeung, D.-Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of IEEE International Conference on Computer Vision, pp. 275–282. IEEE Press, Washington (2004)
22.
Zurück zum Zitat Wipf, D.P., Zhang, H.: Revisiting bayesian blind deconvolution. J. Mach. Learn. Res. 15, 3595–3634 (2014)MathSciNetMATH Wipf, D.P., Zhang, H.: Revisiting bayesian blind deconvolution. J. Mach. Learn. Res. 15, 3595–3634 (2014)MathSciNetMATH
23.
Zurück zum Zitat Roth, S., Black, M.J.: Fields of experts. Int. J. Comput. Vis. 82, 205–229 (2009)CrossRef Roth, S., Black, M.J.: Fields of experts. Int. J. Comput. Vis. 82, 205–229 (2009)CrossRef
24.
Zurück zum Zitat Weiss, Y., Freeman, W.T.: What makes a good model of natural images? In: Proceedings of IEEE Conference on CVPR, pp. 1–8. IEEE Press, Washington (2007) Weiss, Y., Freeman, W.T.: What makes a good model of natural images? In: Proceedings of IEEE Conference on CVPR, pp. 1–8. IEEE Press, Washington (2007)
25.
Zurück zum Zitat Krishnan, D., Tay, T., Fergus, R.: Blind deconvolution using a normalized sparsity measure. In: Proceedings of IEEE Conference on CVPR, pp. 233–240. IEEE Press, Washington (2011) Krishnan, D., Tay, T., Fergus, R.: Blind deconvolution using a normalized sparsity measure. In: Proceedings of IEEE Conference on CVPR, pp. 233–240. IEEE Press, Washington (2011)
26.
Zurück zum Zitat Xu, L., Zheng, S., Jia, J.: Unnatural L0 sparse representation for natural image deblurring. In: Proceedings of IEEE Conference on CVPR, pp. 1107–1114. IEEE Press, Washington (2013) Xu, L., Zheng, S., Jia, J.: Unnatural L0 sparse representation for natural image deblurring. In: Proceedings of IEEE Conference on CVPR, pp. 1107–1114. IEEE Press, Washington (2013)
27.
Zurück zum Zitat Babacan, S.D., Molina, R., Do, M.N., Katsaggelos, A.K.: Bayesian blind deconvolution with general sparse image priors. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 341–355. Springer, Heidelberg (2012)CrossRef Babacan, S.D., Molina, R., Do, M.N., Katsaggelos, A.K.: Bayesian blind deconvolution with general sparse image priors. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 341–355. Springer, Heidelberg (2012)CrossRef
28.
Zurück zum Zitat Blumensath, T., Davies, M.E.: Iterative hard thresholding for compressed sensing. Appl. Comput. Harmonic Anal. 27, 265–274 (2009)MathSciNetCrossRef Blumensath, T., Davies, M.E.: Iterative hard thresholding for compressed sensing. Appl. Comput. Harmonic Anal. 27, 265–274 (2009)MathSciNetCrossRef
29.
Zurück zum Zitat Storath, M., Weinmann, A., Demaret, L.: Jump-sparse and sparse recovery using potts functionals. IEEE Sig. Process. 62, 3654–3666 (2014)MathSciNetCrossRef Storath, M., Weinmann, A., Demaret, L.: Jump-sparse and sparse recovery using potts functionals. IEEE Sig. Process. 62, 3654–3666 (2014)MathSciNetCrossRef
30.
Zurück zum Zitat Shao, W.-Z., Deng, H.-S., Wei, Z.-H.: The magic of split augmented Lagrangians applied to K-frame-based ℓ0-ℓ2 minimization image restoration. SIViP 8, 975–983 (2014)CrossRef Shao, W.-Z., Deng, H.-S., Wei, Z.-H.: The magic of split augmented Lagrangians applied to K-frame-based 0-2 minimization image restoration. SIViP 8, 975–983 (2014)CrossRef
31.
Zurück zum Zitat Shao, W.-Z., Li, H.-B., Elad, M.: Bi-ℓ0-ℓ2-norm regularization for blind motion deblurring. arxiv.org/ftp/arxiv/papers/1408/1408.4712.pdf (2014) Shao, W.-Z., Li, H.-B., Elad, M.: Bi-0-2-norm regularization for blind motion deblurring. arxiv.​org/​ftp/​arxiv/​papers/​1408/​1408.​4712.​pdf (2014)
32.
Zurück zum Zitat Levin, A., Fergus, R., Durand, F., Freeman, W.T.: Image and depth from a conventional camera with a coded aperture. In: ACM Transactions on Graphics, vol. 26, Article No. 70 (2007) Levin, A., Fergus, R., Durand, F., Freeman, W.T.: Image and depth from a conventional camera with a coded aperture. In: ACM Transactions on Graphics, vol. 26, Article No. 70 (2007)
33.
Zurück zum Zitat Yang, C.-Y., Ma, C., Yang, M.-H.: Single-image super-resolution: a benchmark. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part IV. LNCS, vol. 8692, pp. 372–386. Springer, Heidelberg (2014)CrossRef Yang, C.-Y., Ma, C., Yang, M.-H.: Single-image super-resolution: a benchmark. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part IV. LNCS, vol. 8692, pp. 372–386. Springer, Heidelberg (2014)CrossRef
34.
Zurück zum Zitat Timofte, R., Smet, V.D., Gool, L.V.: A+: adjusted anchored neighborhood regression for fast super- resolution. In: Proceedings of Asian Conference on Computer Vision, pp. 111–126 (2014) Timofte, R., Smet, V.D., Gool, L.V.: A+: adjusted anchored neighborhood regression for fast super- resolution. In: Proceedings of Asian Conference on Computer Vision, pp. 111–126 (2014)
35.
Zurück zum Zitat Yang, J., Wang, Z., Lin, Z., Cohen, S., Huang, T.: Coupled dictionary training for image super-resolution. IEEE Trans. Image Process. 21, 3467–3478 (2012)MathSciNetCrossRef Yang, J., Wang, Z., Lin, Z., Cohen, S., Huang, T.: Coupled dictionary training for image super-resolution. IEEE Trans. Image Process. 21, 3467–3478 (2012)MathSciNetCrossRef
36.
Zurück zum Zitat He, Y., Yap, K.H., Chen, L., Chau, L.P.: A soft MAP framework for blind super-resolution image reconstruction. Image Vis. Comput. 27, 364–373 (2009)CrossRef He, Y., Yap, K.H., Chen, L., Chau, L.P.: A soft MAP framework for blind super-resolution image reconstruction. Image Vis. Comput. 27, 364–373 (2009)CrossRef
37.
Zurück zum Zitat Krishnan, D., Fergus, R.: Fast image deconvolution using hyper-Laplacian priors. In: Proceedings of Advances in Neural Information Processing Systems, pp. 1033–1041. MIT Press, Cambridge (2009) Krishnan, D., Fergus, R.: Fast image deconvolution using hyper-Laplacian priors. In: Proceedings of Advances in Neural Information Processing Systems, pp. 1033–1041. MIT Press, Cambridge (2009)
38.
Zurück zum Zitat Marquina, A., Osher, S.J.: Image super-resolution by TV-regularization and Bregman iteration. J. Sci. Comput. 37, 367–382 (2008)MathSciNetCrossRef Marquina, A., Osher, S.J.: Image super-resolution by TV-regularization and Bregman iteration. J. Sci. Comput. 37, 367–382 (2008)MathSciNetCrossRef
Metadaten
Titel
Simple, Accurate, and Robust Nonparametric Blind Super-Resolution
verfasst von
Wen-Ze Shao
Michael Elad
Copyright-Jahr
2015
Verlag
Springer International Publishing
DOI
https://doi.org/10.1007/978-3-319-21969-1_29

Premium Partner