Skip to main content
Erschienen in: International Journal of Machine Learning and Cybernetics 8/2020

22.02.2020 | Original Article

Deep quantification down-plain-upsampling residual learning for single image super-resolution

verfasst von: Shuying Huang, Haijun Zhu, Yong Yang, Yifan Zuo, Yingjun Tang

Erschienen in: International Journal of Machine Learning and Cybernetics | Ausgabe 8/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Deep convolutional neural networks have been widely used in single image super-resolution (SISR) with great success. However, the performance and efficiency of such models need to be improved for practical applications. In this paper, a novel deep quantification down-plain-upsampling (QDPU) network for SISR is proposed. In the framework, a down-plain-upsampling (DPU) residual block based on U-Net is firstly designed to reduce the computational cost by transforming the spatial scale of feature maps without sacrificing the reconstruction performance. Then, to better transmit low-level features to the reconstruction layer, we construct quantification skip-connection modules to integrate the outputs of the DPU residual blocks. Finally, QDPU is developed by stacking the DPU residual blocks with multiple skip-connections to reconstruct high-resolution images and reduce the computational burden. Quantitative and qualitative evaluations of the reconstruction results on four benchmark datasets show that the proposed method can achieve better performance compared with several state-of-the-art SISR methods.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Weitere Produktempfehlungen anzeigen
Literatur
1.
Zurück zum Zitat Nasrollahi K, Moeslund TB (2014) Super-resolution: a comprehensive survey. Mach Vis Appl 25(6):1423–1468CrossRef Nasrollahi K, Moeslund TB (2014) Super-resolution: a comprehensive survey. Mach Vis Appl 25(6):1423–1468CrossRef
2.
Zurück zum Zitat Li X, Orchard MT (2001) New edge-directed interpolation. IEEE Trans Image Process 10(10):1521–1527CrossRef Li X, Orchard MT (2001) New edge-directed interpolation. IEEE Trans Image Process 10(10):1521–1527CrossRef
3.
Zurück zum Zitat Zhang L, Wu X (2006) An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans Image Process 15(8):2226–2238CrossRef Zhang L, Wu X (2006) An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans Image Process 15(8):2226–2238CrossRef
4.
Zurück zum Zitat Protter M, Elad M, Takeda H, Milanfar P (2008) Generalizing the Nonlocal-means to super-resolution reconstruction. IEEE Trans Image Process 18(1):36–51MathSciNetMATHCrossRef Protter M, Elad M, Takeda H, Milanfar P (2008) Generalizing the Nonlocal-means to super-resolution reconstruction. IEEE Trans Image Process 18(1):36–51MathSciNetMATHCrossRef
5.
Zurück zum Zitat Sun J, Xu Z, Shum HY (2008) Image super-resolution using gradient profile prior. In: IEEE conference on computer vision and pattern recognition. IEEE, Anchorage, AK, pp 1–8 Sun J, Xu Z, Shum HY (2008) Image super-resolution using gradient profile prior. In: IEEE conference on computer vision and pattern recognition. IEEE, Anchorage, AK, pp 1–8
6.
Zurück zum Zitat Sun J, Xu Z, Shum HY (2011) Gradient profile prior and its applications in image super-resolution and enhancement. IEEE Trans Image Process 20(6):1529MathSciNetMATHCrossRef Sun J, Xu Z, Shum HY (2011) Gradient profile prior and its applications in image super-resolution and enhancement. IEEE Trans Image Process 20(6):1529MathSciNetMATHCrossRef
7.
Zurück zum Zitat Tai YW, Liu S, Brown MS, Lin S (2010) Super resolution using edge prior and single image detail synthesis. In: IEEE conference on computer vision and pattern recognition. IEEE, San Francisco, CA, pp 2400–2407 Tai YW, Liu S, Brown MS, Lin S (2010) Super resolution using edge prior and single image detail synthesis. In: IEEE conference on computer vision and pattern recognition. IEEE, San Francisco, CA, pp 2400–2407
8.
Zurück zum Zitat Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873MathSciNetMATHCrossRef Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873MathSciNetMATHCrossRef
9.
Zurück zum Zitat Huang S, Sun J, Yang Y, Fang Y, Lin P, Que Y (2019) Robust single-image super-resolution based on adaptive edge-preserving smoothing regularization. IEEE Trans Image Process 27(6):2650MathSciNetMATHCrossRef Huang S, Sun J, Yang Y, Fang Y, Lin P, Que Y (2019) Robust single-image super-resolution based on adaptive edge-preserving smoothing regularization. IEEE Trans Image Process 27(6):2650MathSciNetMATHCrossRef
10.
Zurück zum Zitat Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, pp 105–114 Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, pp 105–114
11.
Zurück zum Zitat Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: IEEE conference on computer vision and pattern recognition. IEEE, Las Vegas, NV, pp 1646–1654 Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: IEEE conference on computer vision and pattern recognition. IEEE, Las Vegas, NV, pp 1646–1654
12.
Zurück zum Zitat Yang W, Feng J, Yang J, Zhao F, Liu J, Guo Z, Yan S (2017) Deep edge guided recurrent residual learning for image super-resolution. IEEE Trans Image Process 26(12):5895–5907MathSciNetCrossRef Yang W, Feng J, Yang J, Zhao F, Liu J, Guo Z, Yan S (2017) Deep edge guided recurrent residual learning for image super-resolution. IEEE Trans Image Process 26(12):5895–5907MathSciNetCrossRef
13.
Zurück zum Zitat Park SC, Min KP, Kang MG (2003) Super-resolution image reconstruction: a technical overview. IEEE Signal Process Mag 20(3):21–36CrossRef Park SC, Min KP, Kang MG (2003) Super-resolution image reconstruction: a technical overview. IEEE Signal Process Mag 20(3):21–36CrossRef
14.
Zurück zum Zitat Huang H, He R, Sun Z, Tan T (2017) Wavelet-SRNet: a wavelet-based CNN for multi-scale face super resolution. In: IEEE conference on computer vision. IEEE, Venice, pp 1698–1706 Huang H, He R, Sun Z, Tan T (2017) Wavelet-SRNet: a wavelet-based CNN for multi-scale face super resolution. In: IEEE conference on computer vision. IEEE, Venice, pp 1698–1706
15.
Zurück zum Zitat Dong C, Chen CL, He K, Tang X (2015) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307CrossRef Dong C, Chen CL, He K, Tang X (2015) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307CrossRef
16.
Zurück zum Zitat Guo T, Mousavi HS, Vu TH, Monga V (2017) Deep wavelet prediction for image super-resolution. In: IEEE conference on computer vision and pattern recognition workshops. IEEE, Honolulu, HI, pp 1100–1109 Guo T, Mousavi HS, Vu TH, Monga V (2017) Deep wavelet prediction for image super-resolution. In: IEEE conference on computer vision and pattern recognition workshops. IEEE, Honolulu, HI, pp 1100–1109
17.
Zurück zum Zitat Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep Laplacian pyramid networks for fast and accurate super-resolution. In: IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, pp 5835–5843 Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep Laplacian pyramid networks for fast and accurate super-resolution. In: IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, pp 5835–5843
18.
Zurück zum Zitat Zhang Y, Sun L, Yan C, Ji X, Dai Q (2018) Adaptive residual networks for high-quality image restoration. IEEE Trans Image Process 99:1MathSciNetMATH Zhang Y, Sun L, Yan C, Ji X, Dai Q (2018) Adaptive residual networks for high-quality image restoration. IEEE Trans Image Process 99:1MathSciNetMATH
19.
Zurück zum Zitat Dong C, Chen CL, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision. Amsterdam, pp 391–407 Dong C, Chen CL, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision. Amsterdam, pp 391–407
20.
Zurück zum Zitat Shi W, Caballero J, Huszar F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: IEEE conference on computer vision and pattern recognition. IEEE, Las Vegas, NV, pp 1874–1883 Shi W, Caballero J, Huszar F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: IEEE conference on computer vision and pattern recognition. IEEE, Las Vegas, NV, pp 1874–1883
21.
Zurück zum Zitat Gregor K, Lecun Y (2010) Learning fast approximations of sparse coding. In: International conference on international conference on machine learning. Haifa, Israel, pp 399–406 Gregor K, Lecun Y (2010) Learning fast approximations of sparse coding. In: International conference on international conference on machine learning. Haifa, Israel, pp 399–406
22.
Zurück zum Zitat Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deep networks for image super-resolution with sparse prior. In: IEEE international conference on computer vision. IEEE, Santiago, Chile, pp 370–378 Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deep networks for image super-resolution with sparse prior. In: IEEE international conference on computer vision. IEEE, Santiago, Chile, pp 370–378
23.
Zurück zum Zitat Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: IEEE conference on computer vision and pattern recognition. IEEE, Las Vegas, NV, pp 1637–1645 Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: IEEE conference on computer vision and pattern recognition. IEEE, Las Vegas, NV, pp 1637–1645
24.
Zurück zum Zitat Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations. San Diego, CA, pp 1–14 Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations. San Diego, CA, pp 1–14
25.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition. IEEE, Las Vegas, NV, pp 770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition. IEEE, Las Vegas, NV, pp 770–778
26.
Zurück zum Zitat Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: IEEE conference on computer vision and pattern recognition workshops. IEEE, Honolulu, HI, pp 1132–1140 Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: IEEE conference on computer vision and pattern recognition workshops. IEEE, Honolulu, HI, pp 1132–1140
27.
Zurück zum Zitat Timofte R, Agustsson E, Gool LV, Yang MH, Zhang L, Lim B, Son S, Kim H, Nah S, Lee KM (2017) NTIRE 2017 challenge on single image super-resolution: methods and results. In: IEEE conference on computer vision and pattern recognition workshops. IEEE, Honolulu, HI, pp 1110–1121 Timofte R, Agustsson E, Gool LV, Yang MH, Zhang L, Lim B, Son S, Kim H, Nah S, Lee KM (2017) NTIRE 2017 challenge on single image super-resolution: methods and results. In: IEEE conference on computer vision and pattern recognition workshops. IEEE, Honolulu, HI, pp 1110–1121
28.
Zurück zum Zitat Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, pp 2790–2798 Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, pp 2790–2798
29.
Zurück zum Zitat Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: IEEE conference on computer vision and pattern recognition. IEEE, Salt Lake City, UT, pp 2472–2481 Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: IEEE conference on computer vision and pattern recognition. IEEE, Salt Lake City, UT, pp 2472–2481
31.
Zurück zum Zitat Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size. arXiv:1602.07360 Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size. arXiv:​1602.​07360
32.
Zurück zum Zitat Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: International conference on neural information processing systems. Lake Tahoe, NV, pp 1097–1105 Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: International conference on neural information processing systems. Lake Tahoe, NV, pp 1097–1105
33.
Zurück zum Zitat Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer assisted intervention. Munich, pp 234–241 Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer assisted intervention. Munich, pp 234–241
34.
Zurück zum Zitat Iglovikov V, Shvets A (2018) TernausNet: U-Net with VGG11 encoder pre-trained on ImageNet for image segmentation. arXiv:1801.05746 Iglovikov V, Shvets A (2018) TernausNet: U-Net with VGG11 encoder pre-trained on ImageNet for image segmentation. arXiv:​1801.​05746
35.
Zurück zum Zitat Lin M, Chen Q, Yan S (2014) Network in network. In: International conference on learning representations. Banff, Canada, pp 1–10 Lin M, Chen Q, Yan S (2014) Network in network. In: International conference on learning representations. Banff, Canada, pp 1–10
36.
Zurück zum Zitat Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: IEEE conference on computer vision and pattern recognition. IEEE, Salt Lake City, UT, pp 7132–7141 Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: IEEE conference on computer vision and pattern recognition. IEEE, Salt Lake City, UT, pp 7132–7141
37.
Zurück zum Zitat Tai Y, Yang J, Liu X, Xu C (2017) MemNet: a persistent memory network for image restoration. In: IEEE conference on computer vision. IEEE, Venice, Italy, pp 4549–4557 Tai Y, Yang J, Liu X, Xu C (2017) MemNet: a persistent memory network for image restoration. In: IEEE conference on computer vision. IEEE, Venice, Italy, pp 4549–4557
38.
Zurück zum Zitat Kamnitsas K, Ledig C, Newcombe VFJ, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2016) Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 36:61CrossRef Kamnitsas K, Ledig C, Newcombe VFJ, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2016) Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 36:61CrossRef
39.
Zurück zum Zitat Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder–decoder architecture for scene segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495CrossRef Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder–decoder architecture for scene segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495CrossRef
40.
Zurück zum Zitat Nah S, Kim TH, Lee KM (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, pp 257–265 Nah S, Kim TH, Lee KM (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, pp 257–265
42.
Zurück zum Zitat Szegedy C, Ioffe S, Vanhoucke V, Alemi A (2017) Inception-v4, Inception-ResNet and the impact of residual connections on learning. In: AAAI conference on artificial intelligence. San Francisco, CA, pp 4278–4284 Szegedy C, Ioffe S, Vanhoucke V, Alemi A (2017) Inception-v4, Inception-ResNet and the impact of residual connections on learning. In: AAAI conference on artificial intelligence. San Francisco, CA, pp 4278–4284
43.
Zurück zum Zitat Zeyde R, Elad M, Protter M (2012) On single image scale-up using sparse-representations. In: International conference on curves and surfaces. Avignon, France, pp 711–730 Zeyde R, Elad M, Protter M (2012) On single image scale-up using sparse-representations. In: International conference on curves and surfaces. Avignon, France, pp 711–730
45.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: IEEE international conference on computer vision. IEEE, Santiago, Chile, pp 1026–1034 He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: IEEE international conference on computer vision. IEEE, Santiago, Chile, pp 1026–1034
46.
Zurück zum Zitat Martin DR, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: IEEE international conference on computer vision. IEEE, Vancouver, BC, pp 416–423 Martin DR, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: IEEE international conference on computer vision. IEEE, Vancouver, BC, pp 416–423
47.
Zurück zum Zitat Agustsson E, Timofte R (2017) NTIRE 2017 challenge on single image super-resolution: dataset and study. In: IEEE conference on computer vision pattern and recognition workshops. IEEE, Honolulu, HI, pp 1122–1131 Agustsson E, Timofte R (2017) NTIRE 2017 challenge on single image super-resolution: dataset and study. In: IEEE conference on computer vision pattern and recognition workshops. IEEE, Honolulu, HI, pp 1122–1131
48.
Zurück zum Zitat Bevilacqua M, Roumy A, Guillemot C, Morel A (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Electronic proceedings of the British machine vision conference. Guildford, Surrey, pp 1–10 Bevilacqua M, Roumy A, Guillemot C, Morel A (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Electronic proceedings of the British machine vision conference. Guildford, Surrey, pp 1–10
49.
Zurück zum Zitat Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: IEEE conference on computer vision and pattern recognition. IEEE, Boston, MA, pp 5197–5206 Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: IEEE conference on computer vision and pattern recognition. IEEE, Boston, MA, pp 5197–5206
Metadaten
Titel
Deep quantification down-plain-upsampling residual learning for single image super-resolution
verfasst von
Shuying Huang
Haijun Zhu
Yong Yang
Yifan Zuo
Yingjun Tang
Publikationsdatum
22.02.2020
Verlag
Springer Berlin Heidelberg
Erschienen in
International Journal of Machine Learning and Cybernetics / Ausgabe 8/2020
Print ISSN: 1868-8071
Elektronische ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-020-01083-w

Weitere Artikel der Ausgabe 8/2020

International Journal of Machine Learning and Cybernetics 8/2020 Zur Ausgabe

Neuer Inhalt