Skip to main content
Erschienen in: Neural Processing Letters 1/2020

27.09.2019

Deep Feature Fusion for High-Resolution Aerial Scene Classification

verfasst von: Heng Wang, Yunlong Yu

Erschienen in: Neural Processing Letters | Ausgabe 1/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The rapid development of remote sensing technology let us acquire a large collection of remote sensing scene images with high resolution. Aerial scene classification has become a crucial problem for understanding high-resolution remote sensing imagery. In this letter, we propose a novel framework for aerial scene classification. Unlike some traditional methods in which the features are produced by using handcrafted feature descriptors, our proposed method uses the raw RGB network stream and the saliency coded network stream to extract two different types of informative features. Then, we further propose a deep feature fusion model to fuse these two sets of features for final classification. The comprehensive performance evaluation of our proposed method is tested on two publicly available remote sensing scene classification benchmarks, i.e., the UC-Merced dataset and the AID dataset. Experimental results show that our proposed method achieves satisfactory results and outperforms the state-of-the-art approaches.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Anwer RM, Khan FS, van de Weijer J, Molinier M, Laaksonen J (2018) Binary patterns encoded convolutional neural networks for texture recognition and remote sensing scene classification. arXiv preprint arXiv:1706.01171v2 Anwer RM, Khan FS, van de Weijer J, Molinier M, Laaksonen J (2018) Binary patterns encoded convolutional neural networks for texture recognition and remote sensing scene classification. arXiv preprint arXiv:​1706.​01171v2
2.
Zurück zum Zitat Bian X, Chen C, Sheng Y, Xu Y, Du Q (2017) Fusing two convolutional neural networks for high-resolution scene classification. In: 2017 IEEE International geoscience and remote sensing symposium (IGARSS). IEEE, pp 3242–3245 Bian X, Chen C, Sheng Y, Xu Y, Du Q (2017) Fusing two convolutional neural networks for high-resolution scene classification. In: 2017 IEEE International geoscience and remote sensing symposium (IGARSS). IEEE, pp 3242–3245
3.
Zurück zum Zitat Bian X, Chen C, Tian L, Du Q (2017) Fusing local and global features for high-resolution scene classification. IEEE J Sel Top Appl Earth Obs Remote Sens 10:2889–2901 Bian X, Chen C, Tian L, Du Q (2017) Fusing local and global features for high-resolution scene classification. IEEE J Sel Top Appl Earth Obs Remote Sens 10:2889–2901
4.
Zurück zum Zitat Blei DM, Ng AY, Jordan MI (2003) Latent Dirichlet allocation. J Mach Learn Res 3(Jan):993–1022 Blei DM, Ng AY, Jordan MI (2003) Latent Dirichlet allocation. J Mach Learn Res 3(Jan):993–1022
5.
Zurück zum Zitat Bosch A, Zisserman A, Muñoz X (2006) Scene classification via pLSA. In: Computer vision—ECCV 2006, pp 517–530 Bosch A, Zisserman A, Muñoz X (2006) Scene classification via pLSA. In: Computer vision—ECCV 2006, pp 517–530
6.
Zurück zum Zitat Chaib S, Liu H, Gu Y, Yao H (2017) Deep feature fusion for VHR remote sensing scene classification. IEEE Trans Geosci Remote Sens 55:4775–4784 Chaib S, Liu H, Gu Y, Yao H (2017) Deep feature fusion for VHR remote sensing scene classification. IEEE Trans Geosci Remote Sens 55:4775–4784
7.
Zurück zum Zitat Ibarrola-Ulzurrun E, Marcello J, Gonzalo-Martin C (2018) Advanced classification of remote sensing high resolution imagery. an application for the management of natural resources. In: Rocha Á (ed) Developments and advances in intelligent systems and applications. Springer, Berlin, pp 1–13 Ibarrola-Ulzurrun E, Marcello J, Gonzalo-Martin C (2018) Advanced classification of remote sensing high resolution imagery. an application for the management of natural resources. In: Rocha Á (ed) Developments and advances in intelligent systems and applications. Springer, Berlin, pp 1–13
8.
Zurück zum Zitat Jegou H, Perronnin F, Douze M, Sánchez J, Perez P, Schmid C (2012) Aggregating local image descriptors into compact codes. IEEE Trans Pattern Anal Mach Intell 34(9):1704–1716 Jegou H, Perronnin F, Douze M, Sánchez J, Perez P, Schmid C (2012) Aggregating local image descriptors into compact codes. IEEE Trans Pattern Anal Mach Intell 34(9):1704–1716
9.
Zurück zum Zitat Ji W, Li X, Lu X (2017) Bidirectional adaptive feature fusion for remote sensing scene classification. In: CCF Chinese conference on computer vision. Springer, Berlin, pp 486–497 Ji W, Li X, Lu X (2017) Bidirectional adaptive feature fusion for remote sensing scene classification. In: CCF Chinese conference on computer vision. Springer, Berlin, pp 486–497
10.
Zurück zum Zitat Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM international conference on multimedia. ACM, pp 675–678 Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM international conference on multimedia. ACM, pp 675–678
11.
Zurück zum Zitat Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE computer society conference on computer vision and pattern recognition, vol 2. IEEE, pp 2169–2178 Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE computer society conference on computer vision and pattern recognition, vol 2. IEEE, pp 2169–2178
12.
Zurück zum Zitat Liu C, Wechsler H (2001) A shape-and texture-based enhanced Fisher classifier for face recognition. IEEE Trans Image Process 10(4):598–608 Liu C, Wechsler H (2001) A shape-and texture-based enhanced Fisher classifier for face recognition. IEEE Trans Image Process 10(4):598–608
13.
Zurück zum Zitat Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110 Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110
14.
Zurück zum Zitat Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987 Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987
15.
Zurück zum Zitat Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vis 42(3):145–175 Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vis 42(3):145–175
16.
Zurück zum Zitat Perronnin F, Sánchez J, Mensink T (2010) Improving the fisher kernel for large-scale image classification. In: Computer vision—ECCV 2010, pp 143–156 Perronnin F, Sánchez J, Mensink T (2010) Improving the fisher kernel for large-scale image classification. In: Computer vision—ECCV 2010, pp 143–156
17.
Zurück zum Zitat Ranganath C, Rainer G (2003) Cognitive neuroscience: neural mechanisms for detecting and remembering novel events. Nat Rev Neurosci 4(3):193 Ranganath C, Rainer G (2003) Cognitive neuroscience: neural mechanisms for detecting and remembering novel events. Nat Rev Neurosci 4(3):193
18.
Zurück zum Zitat Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252 Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252
19.
Zurück zum Zitat Sheng G, Yang W, Xu T, Sun H (2012) High-resolution satellite scene classification using a sparse coding based multiple feature combination. Int J Remote Sens 33(8):2395–2412 Sheng G, Yang W, Xu T, Sun H (2012) High-resolution satellite scene classification using a sparse coding based multiple feature combination. Int J Remote Sens 33(8):2395–2412
20.
Zurück zum Zitat Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:​1409.​1556
21.
Zurück zum Zitat Sivic J, Zisserman A (2003) Video google: a text retrieval approach to object matching in videos. In: IEEE International conference on computer vision. IEEE, p 1470 Sivic J, Zisserman A (2003) Video google: a text retrieval approach to object matching in videos. In: IEEE International conference on computer vision. IEEE, p 1470
22.
Zurück zum Zitat Swain MJ, Ballard DH (1991) Color indexing. Int J Comput Vis 7(1):11–32 Swain MJ, Ballard DH (1991) Color indexing. Int J Comput Vis 7(1):11–32
23.
Zurück zum Zitat Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9 Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9
24.
Zurück zum Zitat Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: 2010 IEEE Conference on computer vision and pattern recognition (CVPR). IEEE, pp 3360–3367 Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: 2010 IEEE Conference on computer vision and pattern recognition (CVPR). IEEE, pp 3360–3367
25.
Zurück zum Zitat Wang Y, Zhang L, Tong X, Nie F, Huang H, Mei J (2018) LRAGE: learning latent relationships with adaptive graph embedding for aerial scene classification. IEEE Trans Geosci Remote Sens 56(2):621–634 Wang Y, Zhang L, Tong X, Nie F, Huang H, Mei J (2018) LRAGE: learning latent relationships with adaptive graph embedding for aerial scene classification. IEEE Trans Geosci Remote Sens 56(2):621–634
26.
Zurück zum Zitat Weng Q, Mao Z, Lin J, Guo W (2017) Land-use classification via extreme learning classifier based on deep convolutional features. IEEE Geosci Remote Sens Lett 14(5):704–708 Weng Q, Mao Z, Lin J, Guo W (2017) Land-use classification via extreme learning classifier based on deep convolutional features. IEEE Geosci Remote Sens Lett 14(5):704–708
27.
Zurück zum Zitat Xia GS, Hu J, Hu F, Shi B, Bai X, Zhong Y, Zhang L, Lu X (2017) AID: a benchmark data set for performance evaluation of aerial scene classification. IEEE Trans Geosci Remote Sens 55:3965–3981 Xia GS, Hu J, Hu F, Shi B, Bai X, Zhong Y, Zhang L, Lu X (2017) AID: a benchmark data set for performance evaluation of aerial scene classification. IEEE Trans Geosci Remote Sens 55:3965–3981
28.
Zurück zum Zitat Yang J, Yang JY, Zhang D, Lu JF (2003) Feature fusion: parallel strategy versus serial strategy. Pattern Recognit 36(6):1369–1381 Yang J, Yang JY, Zhang D, Lu JF (2003) Feature fusion: parallel strategy versus serial strategy. Pattern Recognit 36(6):1369–1381
29.
Zurück zum Zitat Yang Y, Newsam S (2010) Bag-of-visual-words and spatial extensions for land-use classification. In: Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems. ACM, pp 270–279 Yang Y, Newsam S (2010) Bag-of-visual-words and spatial extensions for land-use classification. In: Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems. ACM, pp 270–279
30.
Zurück zum Zitat Yu Y, Liu F (2018) Aerial scene classification via multilevel fusion based on deep convolutional neural networks. IEEE Geosci Remote Sens Lett 15(2):287–291 Yu Y, Liu F (2018) Aerial scene classification via multilevel fusion based on deep convolutional neural networks. IEEE Geosci Remote Sens Lett 15(2):287–291
31.
Zurück zum Zitat Yu Y (2018) Liu F (2018) A two-stream deep fusion framework for high-resolution aerial scene classification. Comput Intell Neurosci 2018:1–13 Yu Y (2018) Liu F (2018) A two-stream deep fusion framework for high-resolution aerial scene classification. Comput Intell Neurosci 2018:1–13
32.
Zurück zum Zitat Zheng Z, Zhang T, Yan L (2012) Saliency model for object detection: searching for novel items in the scene. Opt Lett 37(9):1580–1582 Zheng Z, Zhang T, Yan L (2012) Saliency model for object detection: searching for novel items in the scene. Opt Lett 37(9):1580–1582
33.
Zurück zum Zitat Zou Q, Ni L, Zhang T, Wang Q (2015) Deep learning based feature selection for remote sensing scene classification. IEEE Geosci Remote Sens Lett 12(11):2321–2325 Zou Q, Ni L, Zhang T, Wang Q (2015) Deep learning based feature selection for remote sensing scene classification. IEEE Geosci Remote Sens Lett 12(11):2321–2325
Metadaten
Titel
Deep Feature Fusion for High-Resolution Aerial Scene Classification
verfasst von
Heng Wang
Yunlong Yu
Publikationsdatum
27.09.2019
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 1/2020
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-019-10119-4

Weitere Artikel der Ausgabe 1/2020

Neural Processing Letters 1/2020 Zur Ausgabe

Neuer Inhalt