Skip to main content
Erschienen in: Neural Computing and Applications 9/2020

21.11.2018 | Original Article

Single-image super-resolution via joint statistic models-guided deep auto-encoder network

verfasst von: Rong Chen, Yanyun Qu, Cuihua Li, Kun Zeng, Yuan Xie, Ce Li

Erschienen in: Neural Computing and Applications | Ausgabe 9/2020

Einloggen

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Recent researches on super-resolution (SR) with deep learning networks have achieved amazing results. However, most of the existing studies neglect the internal distinctiveness of an image and the output of most methods tends to be of blurring, smoothness and implausibility. In this paper, we proposed a unified model which combines the deep model with the image restoration model for single-image SR. This model can not only reconstruct the SR image, but also keep the distinct fine structures for the low-resolution image. Two statistic priors are used to guide the updating of the output of the deep neural network: One is the non-local similarity and the other is the local smoothness. The former is modeled as the non-local total variation regularization, and the latter as the steering kernel regression total variation regularization. For this unified model, a new optimization function is formulated under a regularization framework. To optimize the total variation problem, a novel algorithm based on split Bregman iteration is developed with the theoretical proof of convergence. The experimental results demonstrate that the proposed unified model improves the peak signal-to-noise ratio of the deep SR model. Quantitative and qualitative results on four benchmark datasets show that the proposed model achieves better performance than the deep SR model without regularization terms.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: British Machine Vision Conference (BMVC), 2012 Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: British Machine Vision Conference (BMVC), 2012 
2.
Zurück zum Zitat Buades A, Coll B, Morel JM (2005) A non-local algorithm for image denoising. In: IEEE Computer Society conference on computer vision and pattern recognition, 2005. CVPR 2005, vol 2, pp 60–65. IEEE Buades A, Coll B, Morel JM (2005) A non-local algorithm for image denoising. In: IEEE Computer Society conference on computer vision and pattern recognition, 2005. CVPR 2005, vol 2, pp 60–65. IEEE
3.
Zurück zum Zitat Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. In: Proceedings of the 2004 IEEE Computer Society conference on computer vision and pattern recognition, 2004. CVPR 2004, vol 1, p I. IEEE Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. In: Proceedings of the 2004 IEEE Computer Society conference on computer vision and pattern recognition, 2004. CVPR 2004, vol 1, p I. IEEE
4.
Zurück zum Zitat Cui Z, Chang H, Shan S, Zhong B, Chen X (2014) Deep network cascade for image super-resolution. Springer, Berlin, pp 49–64 Cui Z, Chang H, Shan S, Zhong B, Chen X (2014) Deep network cascade for image super-resolution. Springer, Berlin, pp 49–64
5.
Zurück zum Zitat Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307CrossRef Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307CrossRef
6.
Zurück zum Zitat Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. Springer, Berlin, pp 391–407 Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. Springer, Berlin, pp 391–407
7.
Zurück zum Zitat Dong W, Zhang L, Lukac R, Shi G (2013) Sparse representation based image interpolation with nonlocal autoregressive modeling. IEEE Trans Image Process 22(4):1382–1394MathSciNetCrossRef Dong W, Zhang L, Lukac R, Shi G (2013) Sparse representation based image interpolation with nonlocal autoregressive modeling. IEEE Trans Image Process 22(4):1382–1394MathSciNetCrossRef
8.
Zurück zum Zitat Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65CrossRef Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65CrossRef
9.
Zurück zum Zitat Goldstein T, Osher S (2009) The split Bregman method for l1-regularized problems. SIAM J Imaging Sci 2(2):323–343MathSciNetCrossRef Goldstein T, Osher S (2009) The split Bregman method for l1-regularized problems. SIAM J Imaging Sci 2(2):323–343MathSciNetCrossRef
10.
Zurück zum Zitat Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp 5197–5206 Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp 5197–5206
11.
Zurück zum Zitat Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp 1646–1654 Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp 1646–1654
12.
Zurück zum Zitat Kim J, Kwon Lee J, Mu Lee K (2016) Deeply-recursive convolutional network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp 1637–1645 Kim J, Kwon Lee J, Mu Lee K (2016) Deeply-recursive convolutional network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp 1637–1645
13.
Zurück zum Zitat Kim KI, Kwon Y (2010) Single-image super-resolution using sparse regression and natural image prior. IEEE Trans Pattern Anal Mach Intell 32(6):1127–1133CrossRef Kim KI, Kwon Y (2010) Single-image super-resolution using sparse regression and natural image prior. IEEE Trans Pattern Anal Mach Intell 32(6):1127–1133CrossRef
14.
Zurück zum Zitat Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep Laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 624–632 Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep Laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 624–632
15.
Zurück zum Zitat Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z et al (2016) Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z et al (2016) Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint
16.
Zurück zum Zitat Li J, Qu Y, Li C, Xie Y, Wu Y, Fan J (2015) Learning local Gaussian process regression for image super-resolution. Neurocomputing 154:284–295CrossRef Li J, Qu Y, Li C, Xie Y, Wu Y, Fan J (2015) Learning local Gaussian process regression for image super-resolution. Neurocomputing 154:284–295CrossRef
17.
Zurück zum Zitat Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: The IEEE conference on computer vision and pattern recognition (CVPR) Workshops, vol 1, p 3 Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: The IEEE conference on computer vision and pattern recognition (CVPR) Workshops, vol 1, p 3
18.
Zurück zum Zitat Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Eighth IEEE international conference on computer vision, 2001. ICCV 2001. Proceedings, vol 2, pp 416–423. IEEE Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Eighth IEEE international conference on computer vision, 2001. ICCV 2001. Proceedings, vol 2, pp 416–423. IEEE
19.
Zurück zum Zitat Ni KS, Nguyen TQ (2007) Image superresolution using support vector regression. IEEE Trans Image Process 16(6):1596–1610MathSciNetCrossRef Ni KS, Nguyen TQ (2007) Image superresolution using support vector regression. IEEE Trans Image Process 16(6):1596–1610MathSciNetCrossRef
20.
Zurück zum Zitat Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp 1874–1883 Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp 1874–1883
21.
Zurück zum Zitat Takeda H, Farsiu S, Milanfar P (2007) Kernel regression for image processing and reconstruction. IEEE Trans Image Process 16(2):349–366MathSciNetCrossRef Takeda H, Farsiu S, Milanfar P (2007) Kernel regression for image processing and reconstruction. IEEE Trans Image Process 16(2):349–366MathSciNetCrossRef
22.
Zurück zum Zitat Timofte R, De V, Van Gool L (2013) Anchored neighborhood regression for fast example-based super-resolution. In: 2013 IEEE international conference on computer vision (ICCV), pp 1920–1927. IEEE Timofte R, De V, Van Gool L (2013) Anchored neighborhood regression for fast example-based super-resolution. In: 2013 IEEE international conference on computer vision (ICCV), pp 1920–1927. IEEE
23.
Zurück zum Zitat Wang R, Tao D (2016) Non-local auto-encoder with collaborative stabilization for image restoration. IEEE Trans Image Process 25(5):2117–2129MathSciNetCrossRef Wang R, Tao D (2016) Non-local auto-encoder with collaborative stabilization for image restoration. IEEE Trans Image Process 25(5):2117–2129MathSciNetCrossRef
24.
Zurück zum Zitat Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deep networks for image super-resolution with sparse prior. In: Proceedings of the IEEE international conference on computer vision, pp 370–378 Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deep networks for image super-resolution with sparse prior. In: Proceedings of the IEEE international conference on computer vision, pp 370–378
25.
Zurück zum Zitat Yang J, Wang Z, Lin Z, Cohen S, Huang T (2012) Coupled dictionary training for image super-resolution. IEEE Trans Image Process 21(8):3467–3478MathSciNetCrossRef Yang J, Wang Z, Lin Z, Cohen S, Huang T (2012) Coupled dictionary training for image super-resolution. IEEE Trans Image Process 21(8):3467–3478MathSciNetCrossRef
26.
Zurück zum Zitat Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873MathSciNetCrossRef Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873MathSciNetCrossRef
27.
Zurück zum Zitat Zeng K, Yu J, Wang R, Li C, Tao D (2017) Coupled deep autoencoder for single image super-resolution. IEEE Trans Cybern 47(1):27–37CrossRef Zeng K, Yu J, Wang R, Li C, Tao D (2017) Coupled deep autoencoder for single image super-resolution. IEEE Trans Cybern 47(1):27–37CrossRef
28.
Zurück zum Zitat Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. Springer, Berlin, pp 711–730MATH Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. Springer, Berlin, pp 711–730MATH
29.
Zurück zum Zitat Zhang H, Cao X, Ho JKL, Chow TWS (2017) Object-level video advertising: an optimization framework. IEEE Trans Ind Inform 13(2):520–531CrossRef Zhang H, Cao X, Ho JKL, Chow TWS (2017) Object-level video advertising: an optimization framework. IEEE Trans Ind Inform 13(2):520–531CrossRef
30.
Zurück zum Zitat Zhang H, Ji Y, Huang W, Liu L (2018) Sitcom-star-based clothing retrieval for video advertising: a deep learning framework. Neural Comput Appl 1–20 Zhang H, Ji Y, Huang W, Liu L (2018) Sitcom-star-based clothing retrieval for video advertising: a deep learning framework.  Neural Comput Appl 1–20
31.
Zurück zum Zitat Zhang K, Gao X, Tao D, Li X (2012) Single image super-resolution with non-local means and steering kernel regression. IEEE Trans Image Process 21(11):4544–4556MathSciNetCrossRef Zhang K, Gao X, Tao D, Li X (2012) Single image super-resolution with non-local means and steering kernel regression. IEEE Trans Image Process 21(11):4544–4556MathSciNetCrossRef
Metadaten
Titel
Single-image super-resolution via joint statistic models-guided deep auto-encoder network
verfasst von
Rong Chen
Yanyun Qu
Cuihua Li
Kun Zeng
Yuan Xie
Ce Li
Publikationsdatum
21.11.2018
Verlag
Springer London
Erschienen in
Neural Computing and Applications / Ausgabe 9/2020
Print ISSN: 0941-0643
Elektronische ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-018-3886-2

Weitere Artikel der Ausgabe 9/2020

Neural Computing and Applications 9/2020 Zur Ausgabe

Emerging Trends of Applied Neural Computation - E_TRAINCO

Improved zeroing neural networks for finite time solving nonlinear equations

Emerging Trends of Applied Neural Computation - E_TRAINCO

Spatiotemporal neural networks for action recognition based on joint loss