Skip to main content

2020 | OriginalPaper | Buchkapitel

Hyperspectral Image Super-Resolution Using Multi-scale Feature Pyramid Network

verfasst von : He Sun, Zhiwei Zhong, Deming Zhai, Xianming Liu, Junjun Jiang

Erschienen in: Digital TV and Wireless Multimedia Communication

Verlag: Springer Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Hyperspectral (HS) images are captured with rich spectral information, which have been proved to be useful in many real-world applications, such as earth observation. Due to the limitations of HS cameras, it is difficult to obtain HS images with high-resolution (HR). Recent advances in deep learning (DL) for single image super-resolution (SISR) task provide a powerful tool for restoring high-frequency details from low-resolution (LR) input image. Inspired by this progress, in this paper, we present a novel DL-based model for single HS image super-resolution in which a feature pyramid block is designed to extract multi-scale features of the input HS image. Our method does not need auxiliary inputs which further extends the application scenes. Experiment results show that our method outperforms state-of-the-arts on both objective quality indices and subjective visual results.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Bioucas-Dias, J.M., et al.: Hyperspectral unmixing overview: geometrical, statistical, and sparse regression-based approaches. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 5, 354–379 (2012)CrossRef Bioucas-Dias, J.M., et al.: Hyperspectral unmixing overview: geometrical, statistical, and sparse regression-based approaches. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 5, 354–379 (2012)CrossRef
2.
Zurück zum Zitat Haut, J.M., Paoletti, M.E., Plaza, J., Li, J.Y., Plaza, A.J.: Active learning with convolutional neural networks for hyperspectral image classification using a new bayesian approach. IEEE Trans. Geosci. Remote Sens. 56, 6440–6461 (2018)CrossRef Haut, J.M., Paoletti, M.E., Plaza, J., Li, J.Y., Plaza, A.J.: Active learning with convolutional neural networks for hyperspectral image classification using a new bayesian approach. IEEE Trans. Geosci. Remote Sens. 56, 6440–6461 (2018)CrossRef
3.
Zurück zum Zitat Camps-Valls, G., Bruzzone, L.: Kernel-based methods for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 43, 1351–1362 (2005)CrossRef Camps-Valls, G., Bruzzone, L.: Kernel-based methods for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 43, 1351–1362 (2005)CrossRef
4.
Zurück zum Zitat Nguyen, H.V., Banerjee, A., Chellappa, R.: Tracking via object reflectance using a hyperspectral video camera. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, pp. 44–51 (2010) Nguyen, H.V., Banerjee, A., Chellappa, R.: Tracking via object reflectance using a hyperspectral video camera. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, pp. 44–51 (2010)
5.
Zurück zum Zitat Huete, A.R., Miura, T., Gao, X.: Land cover conversion and degradation analyses through coupled soil-plant biophysical parameters derived from hyperspectral EO-1 hyperion. IEEE Trans. Geosci. Remote Sens. 41, 1268–1276 (2003)CrossRef Huete, A.R., Miura, T., Gao, X.: Land cover conversion and degradation analyses through coupled soil-plant biophysical parameters derived from hyperspectral EO-1 hyperion. IEEE Trans. Geosci. Remote Sens. 41, 1268–1276 (2003)CrossRef
6.
Zurück zum Zitat Roberts, D.A., Dennison, P.E., Gardner, M.E., Hetzel, Y., Ustin, S.L., Lee, C.T.: Evaluation of the potential of hyperion for fire danger assessment by comparison to the airborne visible/infrared imaging spectrometer. IEEE Trans. Geosci. Remote Sens. 41, 1297–1310 (2003)CrossRef Roberts, D.A., Dennison, P.E., Gardner, M.E., Hetzel, Y., Ustin, S.L., Lee, C.T.: Evaluation of the potential of hyperion for fire danger assessment by comparison to the airborne visible/infrared imaging spectrometer. IEEE Trans. Geosci. Remote Sens. 41, 1297–1310 (2003)CrossRef
7.
Zurück zum Zitat Simões, M., Bioucas-Dias, J.M., Almeida, L.B., Chanussot, J.: A convex formulation for hyperspectral image superresolution via subspace-based regularization. IEEE Trans. Geosci. Remote Sens. 53, 3373–3388 (2015)CrossRef Simões, M., Bioucas-Dias, J.M., Almeida, L.B., Chanussot, J.: A convex formulation for hyperspectral image superresolution via subspace-based regularization. IEEE Trans. Geosci. Remote Sens. 53, 3373–3388 (2015)CrossRef
8.
Zurück zum Zitat Wei, Q., Bioucas-Dias, J.M., Dobigeon, N., Tourneret, J.Y.: Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Trans. Geosci. Remote Sens. 53, 3658–3668 (2014)CrossRef Wei, Q., Bioucas-Dias, J.M., Dobigeon, N., Tourneret, J.Y.: Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Trans. Geosci. Remote Sens. 53, 3658–3668 (2014)CrossRef
9.
Zurück zum Zitat Yokoya, N., Yairi, T., Iwasaki, A.: Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion. IEEE Trans. Geosci. Remote Sens. 50, 528–537 (2012)CrossRef Yokoya, N., Yairi, T., Iwasaki, A.: Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion. IEEE Trans. Geosci. Remote Sens. 50, 528–537 (2012)CrossRef
10.
Zurück zum Zitat Zhang, K., Wang, M., Yang, S.Y., Jiao, L.: Spatialspectral-graph-regularized low-rank tensor decomposition for multispectral and hyperspectral image fusion. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 11, 1030–1040 (2018)CrossRef Zhang, K., Wang, M., Yang, S.Y., Jiao, L.: Spatialspectral-graph-regularized low-rank tensor decomposition for multispectral and hyperspectral image fusion. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 11, 1030–1040 (2018)CrossRef
11.
Zurück zum Zitat Qu, Y., Qi, H., Kwan, C.: Unsupervised sparse Dirichlet-Net for hyperspectral image super-resolution. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2511–2520 (2018) Qu, Y., Qi, H., Kwan, C.: Unsupervised sparse Dirichlet-Net for hyperspectral image super-resolution. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2511–2520 (2018)
12.
Zurück zum Zitat Xie, Q., Zhou, M., Zhao, Q., Meng, D., Zuo, W., Xu, Z.: Multispectral and hyperspectral image fusion by MS/HS fusion net. ArXiv abs/1901.03281 (2019) Xie, Q., Zhou, M., Zhao, Q., Meng, D., Zuo, W., Xu, Z.: Multispectral and hyperspectral image fusion by MS/HS fusion net. ArXiv abs/1901.03281 (2019)
13.
Zurück zum Zitat Loncan, L., et al.: Hyperspectral pansharpening: a review. IEEE Geosci. Remote Sens. Mag. 3, 27–46 (2015)CrossRef Loncan, L., et al.: Hyperspectral pansharpening: a review. IEEE Geosci. Remote Sens. Mag. 3, 27–46 (2015)CrossRef
14.
Zurück zum Zitat Akgun, T., Altunbasak, Y., Mersereau, R.M.: Super-resolution reconstruction of hyperspectral images. IEEE Trans. Image Process. 14, 1860–1875 (2005)CrossRef Akgun, T., Altunbasak, Y., Mersereau, R.M.: Super-resolution reconstruction of hyperspectral images. IEEE Trans. Image Process. 14, 1860–1875 (2005)CrossRef
15.
Zurück zum Zitat Mei, S., Yuan, X., Ji, J., Zhang, Y., Wan, S., Du, Q.: Hyperspectral image spatial super-resolution via 3D full convolutional neural network. Remote Sens. 9, 1139 (2017)CrossRef Mei, S., Yuan, X., Ji, J., Zhang, Y., Wan, S., Du, Q.: Hyperspectral image spatial super-resolution via 3D full convolutional neural network. Remote Sens. 9, 1139 (2017)CrossRef
16.
Zurück zum Zitat Yuan, Y., Zheng, X., Lu, X.: Hyperspectral image superresolution by transfer learning. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10(5), 1963–1974 (2017)CrossRef Yuan, Y., Zheng, X., Lu, X.: Hyperspectral image superresolution by transfer learning. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10(5), 1963–1974 (2017)CrossRef
17.
Zurück zum Zitat Li, Y., Zhang, L., Ding, C., Wei, W., Zhang, Y.: Single hyperspectral image super-resolution with grouped deep recursive residual network. In: 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM), pp. 1–4 (2018) Li, Y., Zhang, L., Ding, C., Wei, W., Zhang, Y.: Single hyperspectral image super-resolution with grouped deep recursive residual network. In: 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM), pp. 1–4 (2018)
18.
Zurück zum Zitat Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38, 295–307 (2014)CrossRef Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38, 295–307 (2014)CrossRef
19.
Zurück zum Zitat Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1646–1654 (2015) Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1646–1654 (2015)
20.
Zurück zum Zitat Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1132–1140 (2017) Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1132–1140 (2017)
21.
Zurück zum Zitat Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Deep Laplacian pyramid networks for fast and accurate super-resolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5835–5843 (2017) Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Deep Laplacian pyramid networks for fast and accurate super-resolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5835–5843 (2017)
22.
Zurück zum Zitat Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2472–2481 (2018) Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2472–2481 (2018)
23.
Zurück zum Zitat Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. ArXiv abs/1807.02758 (2018) Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. ArXiv abs/1807.02758 (2018)
24.
Zurück zum Zitat Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2017) Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2017)
25.
Zurück zum Zitat Haris, M., Shakhnarovich, G., Ukita, N.: Deep back-projection networks for super-resolution. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1664–1673 (2018) Haris, M., Shakhnarovich, G., Ukita, N.: Deep back-projection networks for super-resolution. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1664–1673 (2018)
26.
Zurück zum Zitat Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004) CrossRef Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004) CrossRef
27.
Zurück zum Zitat Lin, T.Y., Dollár, P., Girshick, R.B., He, K., Hariharan, B., Belongie, S.J.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936–944 (2016) Lin, T.Y., Dollár, P., Girshick, R.B., He, K., Hariharan, B., Belongie, S.J.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936–944 (2016)
28.
Zurück zum Zitat He, K., Gkioxari, G., Dollar, P., Girshick, R.B.: Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. (2018) He, K., Gkioxari, G., Dollar, P., Girshick, R.B.: Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. (2018)
29.
Zurück zum Zitat Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1874–1883 (2016) Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1874–1883 (2016)
30.
Zurück zum Zitat Yokoya, N., Iwasaki, A.: Airborne hyperspectral data over Chikusei. Technical report SAL-2016-05-27, Space Application Laboratory, University of Tokyo, Japan (May 2016) Yokoya, N., Iwasaki, A.: Airborne hyperspectral data over Chikusei. Technical report SAL-2016-05-27, Space Application Laboratory, University of Tokyo, Japan (May 2016)
31.
Zurück zum Zitat Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014) Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014)
32.
Zurück zum Zitat Sidorov, O., Hardeberg, J.Y.: Deep hyperspectral prior: denoising, inpainting, super-resolution. ArXiv abs/1902.00301 (2019) Sidorov, O., Hardeberg, J.Y.: Deep hyperspectral prior: denoising, inpainting, super-resolution. ArXiv abs/1902.00301 (2019)
33.
Zurück zum Zitat Ulyanov, D., Vedaldi, A., Lempitsky, V.S.: Deep image prior. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9446–9454 (2017) Ulyanov, D., Vedaldi, A., Lempitsky, V.S.: Deep image prior. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9446–9454 (2017)
34.
Zurück zum Zitat Wald, L.: Data Fusion, Definitions and Architectures - Fusion of Images of Different Spatial Resolutions. Les Presses de l’Ecole des Mines, Paris (2002) Wald, L.: Data Fusion, Definitions and Architectures - Fusion of Images of Different Spatial Resolutions. Les Presses de l’Ecole des Mines, Paris (2002)
Metadaten
Titel
Hyperspectral Image Super-Resolution Using Multi-scale Feature Pyramid Network
verfasst von
He Sun
Zhiwei Zhong
Deming Zhai
Xianming Liu
Junjun Jiang
Copyright-Jahr
2020
Verlag
Springer Singapore
DOI
https://doi.org/10.1007/978-981-15-3341-9_5

Neuer Inhalt