Skip to main content
Erschienen in: Pattern Recognition and Image Analysis 3/2020

01.07.2020 | APPLIED PROBLEMS

Deep ResNet Based Remote Sensing Image Super-Resolution Reconstruction in Discrete Wavelet Domain

verfasst von: Q. Qin, J. Dou, Z. Tu

Erschienen in: Pattern Recognition and Image Analysis | Ausgabe 3/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

We present a single-image super-resolution (SR) method for Remote Sensing Image based on deep learning within Discrete Wavelet Domain in this paper. Our method is inspired Residual Learning. Firstly, an input image is decomposed by single level 2D Discrete wavelet transform to get four sub-bands. The four sub-bands coefficients are feeding into the Deep Learning Residual Network to predict correspondingly residual images; Adding four sub-band images and residual images as the new sub-bands of 2D wavelet transform; Finally, uses the inverse 2D Discrete wavelet transform to get the final output Super Resolution HR image. Our proposed method performs better than existing methods in accuracy and visual improvements in our results are easily noticeable.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat R. Timofte, E. Agustsson, L. Van Gool, M.-H. Yang, L. Zhang, B. Lim, et al., “NTIRE 2017 challenge on single image super-resolution: Methods and results,” in Proc. 2017 30th IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2017) (Honolulu, HI, USA, 2017), pp. 1110–1121. R. Timofte, E. Agustsson, L. Van Gool, M.-H. Yang, L. Zhang, B. Lim, et al., “NTIRE 2017 challenge on single image super-resolution: Methods and results,” in Proc. 2017 30th IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2017) (Honolulu, HI, USA, 2017), pp. 1110–1121.
2.
Zurück zum Zitat M. Irani and S. Peleg, “Improving resolution by image registration,” CVGIP: Graphical Models Image Process. 53 (3), 231–239 (1991). M. Irani and S. Peleg, “Improving resolution by image registration,” CVGIP: Graphical Models Image Process. 53 (3), 231–239 (1991).
3.
Zurück zum Zitat W. Shi, J. Caballero, C. Ledig, X. Zhuang, W. Bai, K. Bhatia, A. M. Simoes Monteiro de Marvao, T. Dawes, D. O’Regan, and D. Rueckert, “Cardiac image super-resolution with global correspondence using multi-atlas PatchMatch,” in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2013, Proc. 16th International Conference, Part III, Ed by. K. Mori, I. Sakuma, Y. Sato, C. Barillot, and N. Navab, Lecture Notes in Computer Science (Springer, Berlin, Heidelberg, 2013), Vol. 8151, pp. 9–16. W. Shi, J. Caballero, C. Ledig, X. Zhuang, W. Bai, K. Bhatia, A. M. Simoes Monteiro de Marvao, T. Dawes, D. O’Regan, and D. Rueckert, “Cardiac image super-resolution with global correspondence using multi-atlas PatchMatch,” in Medical Image Computing and Computer-Assisted InterventionMICCAI 2013, Proc. 16th International Conference, Part III, Ed by. K. Mori, I. Sakuma, Y. Sato, C. Barillot, and N. Navab, Lecture Notes in Computer Science (Springer, Berlin, Heidelberg, 2013), Vol. 8151, pp. 9–16.
4.
Zurück zum Zitat M. W. Thornton, P. M. Atkinson, and D. A. Holland, “Sub-pixel mapping of rural land cover objects from fine spatial resolution satellite sensor imagery using super-resolution pixel-swapping,” Int. J. Remote Sens. 27 (3), 473–491 (2006).CrossRef M. W. Thornton, P. M. Atkinson, and D. A. Holland, “Sub-pixel mapping of rural land cover objects from fine spatial resolution satellite sensor imagery using super-resolution pixel-swapping,” Int. J. Remote Sens. 27 (3), 473–491 (2006).CrossRef
5.
Zurück zum Zitat W. W. W. Zou and P. C. Yuen, “Very low resolution face recognition problem,” IEEE Trans. Image Process. 21 (1), 327–340 (2012).MathSciNetCrossRef W. W. W. Zou and P. C. Yuen, “Very low resolution face recognition problem,” IEEE Trans. Image Process. 21 (1), 327–340 (2012).MathSciNetCrossRef
6.
Zurück zum Zitat C.-Y. Yang, C. Ma, and M.-H Yang, “Single-image super-resolution: A benchmark,” in Computer Vision − ECCV 2014, Proc. 13th European Conference, Zurich, Switzerland,2014, Part IV, Ed. by D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Lecture Notes in Computer Science (Springer, Cham, 2014), Vol. 8692, pp. 372–386. C.-Y. Yang, C. Ma, and M.-H Yang, “Single-image super-resolution: A benchmark,” in Computer VisionECCV 2014, Proc. 13th European Conference, Zurich, Switzerland,2014, Part IV, Ed. by D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Lecture Notes in Computer Science (Springer, Cham, 2014), Vol. 8692, pp. 372–386.
7.
Zurück zum Zitat G. Freedman and R. Fattal, “Image and video upscaling from local self-examples,” ACM Trans. Graphics 28 (3), Article No. 12, 1–10 (2010). G. Freedman and R. Fattal, “Image and video upscaling from local self-examples,” ACM Trans. Graphics 28 (3), Article No. 12, 1–10 (2010).
8.
Zurück zum Zitat J. Yang, Z. Lin, and S. Cohen, “Fast image super-resolution based on in-place example regression,” in Proc. 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2013) (Portland, OR, USA, 2013), pp. 1059–1066. J. Yang, Z. Lin, and S. Cohen, “Fast image super-resolution based on in-place example regression,” in Proc. 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2013) (Portland, OR, USA, 2013), pp. 1059–1066.
9.
Zurück zum Zitat K. I. Kim and Y. Kwon, “Single-image super-resolution using sparse regression and natural image prior,” IEEE Trans. Pattern Anal. Mach. Intell. 32 (6), 1127–1133 (2010).CrossRef K. I. Kim and Y. Kwon, “Single-image super-resolution using sparse regression and natural image prior,” IEEE Trans. Pattern Anal. Mach. Intell. 32 (6), 1127–1133 (2010).CrossRef
10.
Zurück zum Zitat R. Timofte, V. De, and L. Van Gool, “Anchored neighborhood regression for fast example-based super-resolution,” in Proc. 2013 IEEE International Conference on Computer Vision (ICCV 2013) (Sydney, NSW, Australia, 2013), pp. 1920–1927. R. Timofte, V. De, and L. Van Gool, “Anchored neighborhood regression for fast example-based super-resolution,” in Proc. 2013 IEEE International Conference on Computer Vision (ICCV 2013) (Sydney, NSW, Australia, 2013), pp. 1920–1927.
11.
Zurück zum Zitat S. Mallat, A Wavelet Tour of Signal Processing, 2nd ed. (Academic Press, New York, 1999).MATH S. Mallat, A Wavelet Tour of Signal Processing, 2nd ed. (Academic Press, New York, 1999).MATH
12.
Zurück zum Zitat D. Glasner, S. Bagon, and M. Irani, “Super-resolution from a single image,” in Proc. 2009 IEEE 12th International Conference on Computer Vision (ICCV) (Kyoto, Japan, 2009), pp. 349–356. D. Glasner, S. Bagon, and M. Irani, “Super-resolution from a single image,” in Proc. 2009 IEEE 12th International Conference on Computer Vision (ICCV) (Kyoto, Japan, 2009), pp. 349–356.
13.
Zurück zum Zitat J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19 (11), 2861–2873 (2010).MathSciNetCrossRef J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19 (11), 2861–2873 (2010).MathSciNetCrossRef
14.
Zurück zum Zitat E. Pérez-Pellitero, J. Salvador, J. Ruiz-Hidalgo, and B. Rosenhahn, “PSyCo: Manifold span reduction for super resolution,” in Proc. 2016 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016) (Las Vegas, NV, USA, 2016), pp. 1837–1845. E. Pérez-Pellitero, J. Salvador, J. Ruiz-Hidalgo, and B. Rosenhahn, “PSyCo: Manifold span reduction for super resolution,” in Proc. 2016 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016) (Las Vegas, NV, USA, 2016), pp. 1837–1845.
15.
Zurück zum Zitat R. Timofte, V. De Smet, and L. Van Gool, “A+: Adjusted anchored neighborhood regression for fast super-resolution,” in Computer Vision – ACCV 2014, Proc. 12th Asian Conference on Computer Vision, Singapore,2014, Part IV, Ed. by D. Cremers, I. Reid, H. Saito, and M.-H. Yang, Lecture Notes in Computer Science (Springer, Cham, 2014), Vol. 9006, pp. 111–126. R. Timofte, V. De Smet, and L. Van Gool, “A+: Adjusted anchored neighborhood regression for fast super-resolution,” in Computer VisionACCV 2014, Proc. 12th Asian Conference on Computer Vision, Singapore,2014, Part IV, Ed. by D. Cremers, I. Reid, H. Saito, and M.-H. Yang, Lecture Notes in Computer Science (Springer, Cham, 2014), Vol. 9006, pp. 111–126.
16.
Zurück zum Zitat C. Dong, C. C. Loy, K. He, and X. Tang, “Learning a deep convolutional network for image super-resolution,” in Computer Vision − ECCV 2014, Proc. 13th European Conference, Zurich, Switzerland,2014, Part IV, Ed. by D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Lecture Notes in Computer Science (Springer, Cham, 2014), Vol. 8692, pp. 184–199. C. Dong, C. C. Loy, K. He, and X. Tang, “Learning a deep convolutional network for image super-resolution,” in Computer VisionECCV 2014, Proc. 13th European Conference, Zurich, Switzerland,2014, Part IV, Ed. by D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Lecture Notes in Computer Science (Springer, Cham, 2014), Vol. 8692, pp. 184–199.
17.
Zurück zum Zitat C.-Y. Yang, C. Ma, and M.-H. Yang, “Single-image super-resolution: A benchmark,” in Proc. European Conference on Computer Vision, Zurich, Switzerland, 2014, pp. 372–386. C.-Y. Yang, C. Ma, and M.-H. Yang, “Single-image super-resolution: A benchmark,” in Proc. European Conference on Computer Vision, Zurich, Switzerland, 2014, pp. 372–386.
18.
Zurück zum Zitat K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. 2016 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016) (Las Vegas, NV, USA, 2016), pp. 770–778. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. 2016 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016) (Las Vegas, NV, USA, 2016), pp. 770–778.
19.
Zurück zum Zitat A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems 25: Proc. NIPS 2012 Conf. (Lake Tahoe, NV, USA, 2012), Vol. 1, pp. 1097–1105. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems 25: Proc. NIPS 2012 Conf. (Lake Tahoe, NV, USA, 2012), Vol. 1, pp. 1097–1105.
20.
Zurück zum Zitat J. Kim, J. K. Lee, and K. M. Lee, “Accurate image super-resolution using very deep convolutional networks,” in Proc. 2016 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016) (Las Vegas, NV, USA, 2016), pp. 1646–1654. J. Kim, J. K. Lee, and K. M. Lee, “Accurate image super-resolution using very deep convolutional networks,” in Proc. 2016 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016) (Las Vegas, NV, USA, 2016), pp. 1646–1654.
21.
Zurück zum Zitat LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D, “Backpropagation applied to handwritten zip code recognition,” Neural Comput. 1 (4), 541–551 (1989).CrossRef LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D, “Backpropagation applied to handwritten zip code recognition,” Neural Comput. 1 (4), 541–551 (1989).CrossRef
22.
Zurück zum Zitat V. Nair and G. E. Hinton, “Rectified linear units improve restricted Boltzmann machines,” in ICML’10: Proc. 27th International Conference on Machine Learning (Haifa, Israel, 2010) (Omnipress, 2010), pp. 807–814. V. Nair and G. E. Hinton, “Rectified linear units improve restricted Boltzmann machines,” in ICML10: Proc. 27th International Conference on Machine Learning (Haifa, Israel, 2010) (Omnipress, 2010), pp. 807–814.
23.
Zurück zum Zitat D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature 323 (6088), 533–536 (1986).CrossRef D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature 323 (6088), 533–536 (1986).CrossRef
24.
Zurück zum Zitat A. Cotter, O. Shamir, N. Srebro, and K. Sridharan, “Better mini-batch algorithms via accelerated gradient methods,” in Advances in Neural Information Processing Systems 24: Proc. NIPS 2011 Conf. (Granada, Spain, 2011), pp. 1647–1655. A. Cotter, O. Shamir, N. Srebro, and K. Sridharan, “Better mini-batch algorithms via accelerated gradient methods,” in Advances in Neural Information Processing Systems 24: Proc. NIPS 2011 Conf. (Granada, Spain, 2011), pp. 1647–1655.
25.
Zurück zum Zitat T. Zhang, “Solving large scale linear prediction problems using stochastic gradient descent algorithms,” in ICML’04: Proc. 21st International Conference on Machine Learning (Banff, Canada, 2004) (ACM, New York, 2004), pp. 919–926. T. Zhang, “Solving large scale linear prediction problems using stochastic gradient descent algorithms,” in ICML04: Proc. 21st International Conference on Machine Learning (Banff, Canada, 2004) (ACM, New York, 2004), pp. 919–926.
26.
Zurück zum Zitat S. Schulter, C. Leistner, and H. Bischof, “Fast and accurate image upscaling with super-resolution forests,” in Proc. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015) (Boston, MA, USA, 2015), pp. 3791–3799. S. Schulter, C. Leistner, and H. Bischof, “Fast and accurate image upscaling with super-resolution forests,” in Proc. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015) (Boston, MA, USA, 2015), pp. 3791–3799.
27.
Zurück zum Zitat D. Martin, C. Fowlkes, D. Tal, and J. Malik, “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” in Proc. 8th IEEE International Conference on Computer Vision (ICCV 2001) (Vancouver, Canada, 2001), Vol. 2, pp. 416–423. D. Martin, C. Fowlkes, D. Tal, and J. Malik, “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” in Proc. 8th IEEE International Conference on Computer Vision (ICCV 2001) (Vancouver, Canada, 2001), Vol. 2, pp. 416–423.
28.
Zurück zum Zitat M. Bevilacqua, A. Roumy, C. Guillemot, and M.-L. Alberi Morel, “Low-complexity single-image super-resolution based on nonnegative neighbor embedding,” in Proc. 23rd British Machine Vision Conference BMVC 2012 (Surrey, UK, 2012) (BMVA Press, 2012), pp. 135.1 – 135.10. M. Bevilacqua, A. Roumy, C. Guillemot, and M.-L. Alberi Morel, “Low-complexity single-image super-resolution based on nonnegative neighbor embedding,” in Proc. 23rd British Machine Vision Conference BMVC 2012 (Surrey, UK, 2012) (BMVA Press, 2012), pp. 135.1 – 135.10.
29a.
Zurück zum Zitat G. Cheng, J. Han, and X. Lu, “Remote sensing image scene classification: Benchmark and state of the art,” arXiv preprint arXiv:1703.00121 (2017); G. Cheng, J. Han, and X. Lu, “Remote sensing image scene classification: Benchmark and state of the art,” arXiv preprint arXiv:1703.00121 (2017);
29b.
Zurück zum Zitat Proc. IEEE 105 (10), 1865–1883 (2017). Proc. IEEE 105 (10), 1865–1883 (2017).
Metadaten
Titel
Deep ResNet Based Remote Sensing Image Super-Resolution Reconstruction in Discrete Wavelet Domain
verfasst von
Q. Qin
J. Dou
Z. Tu
Publikationsdatum
01.07.2020
Verlag
Pleiades Publishing
Erschienen in
Pattern Recognition and Image Analysis / Ausgabe 3/2020
Print ISSN: 1054-6618
Elektronische ISSN: 1555-6212
DOI
https://doi.org/10.1134/S1054661820030232

Weitere Artikel der Ausgabe 3/2020

Pattern Recognition and Image Analysis 3/2020 Zur Ausgabe

MATHEMATICAL THEORY OF IMAGES AND SIGNALS REPRESENTING, PROCESSING, ANALYSIS, RECOGNITION, AND UNDERSTANDING

3D Shape Recognition Based on 1D Signal Processing for Real-Time Applications

MATHEMATICAL THEORY OF PATTERN RECOGNITION

Comparison of Different Dichotomous Classification Algorithms

MATHEMATICAL THEORY OF IMAGES AND SIGNALS REPRESENTING, PROCESSING, ANALYSIS, RECOGNITION, AND UNDERSTANDING

Learning-Based Single Image Super-Resolution with Improved Edge Information

Premium Partner