Skip to main content
Top
Published in: Neural Computing and Applications 18/2020

07-05-2020 | S.I. : Deep Learning Approaches for RealTime Image Super Resolution (DLRSR)

Arbitrary-oriented object detection via dense feature fusion and attention model for remote sensing super-resolution image

Authors: Fuhao Zou, Wei Xiao, Wanting Ji, Kunkun He, Zhixiang Yang, Jingkuan Song, Helen Zhou, Kai Li

Published in: Neural Computing and Applications | Issue 18/2020

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In this paper, we aim at developing a new arbitrary-oriented end-to-end object detection method to further push the frontier of object detection for remote sensing image. The proposed method comprehensively takes into account multiple strategies, such as attention mechanism, feature fusion, rotation region proposal as well as super-resolution pre-processing simultaneously to boost the performance in terms of localization and classification under the faster RCNN-like framework. Specifically, a channel attention network is integrated for selectively enhancing useful features and suppressing useless ones. Next, a dense feature fusion network is designed based on multi-scale detection framework, which fuses multiple layers of features to improve the sensitivity to small objects. In addition, considering the objects for detection are often densely arranged and appear in various orientations, we design a rotation anchor strategy to reduce the redundant detection regions. Extensive experiments on two remote sensing public datasets DOTA, NWPU VHR-10 and scene text dataset ICDAR2015 demonstrate that the proposed method can be competitive with or even superior to the state-of-the-art ones, like R2CNN and R2CNN++.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition (CVPR 2016), 2016, pp 770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition (CVPR 2016), 2016, pp 770–778
2.
go back to reference Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the 2014 IEEE conference on computer vision and pattern recognition (CVPR), 2014, pp 580–587 Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the 2014 IEEE conference on computer vision and pattern recognition (CVPR), 2014, pp 580–587
3.
go back to reference He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Proceedings of the 13th European conference on computer vision (ECCV 2014), 2014, pp 346–361 He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Proceedings of the 13th European conference on computer vision (ECCV 2014), 2014, pp 346–361
4.
go back to reference Girshick R (2015) Fast R-CNN [region-based Convolutional Neural Network]. In: Proceedings of the 2015 IEEE international conference on computer vision (ICCV), 2015, pp 1440–1448 Girshick R (2015) Fast R-CNN [region-based Convolutional Neural Network]. In: Proceedings of the 2015 IEEE international conference on computer vision (ICCV), 2015, pp 1440–1448
5.
go back to reference Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149CrossRef Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149CrossRef
6.
go back to reference Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified real-time object detection. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR), 2016, pp 779–788 Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified real-time object detection. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR), 2016, pp 779–788
7.
go back to reference Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) SSD: single shot multibox detector. In: Proceedings of the 14th European conference computer vision (ECCV2016), 9905, pp 21–37 Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) SSD: single shot multibox detector. In: Proceedings of the 14th European conference computer vision (ECCV2016), 9905, pp 21–37
8.
go back to reference Dai J, Li Y, He K, Sun J (2016) R-FCN: object detection via region-based fully convolutional networks. In: Proceedings of the 2016 conference on advances in neural information processing systems (NIPS), pp 379–387 Dai J, Li Y, He K, Sun J (2016) R-FCN: object detection via region-based fully convolutional networks. In: Proceedings of the 2016 conference on advances in neural information processing systems (NIPS), pp 379–387
9.
go back to reference He K, Gkioxari G, Dollar P, Girshick R (2017) Mask R-CNN. In: Prcoeedings of the 2017 IEEE international conference on computer vision (ICCV), 2017, pp 2980–2988 He K, Gkioxari G, Dollar P, Girshick R (2017) Mask R-CNN. In: Prcoeedings of the 2017 IEEE international conference on computer vision (ICCV), 2017, pp 2980–2988
10.
go back to reference Yang X, Sun H, Kun F, Yang J, Sun X, Yan M, Guo Z (2018) Automatic ship detection in remote sensing images from google earth of complex scenes based on multiscale rotation dense feature pyramid networks. Remote Sens 10(1):132–146CrossRef Yang X, Sun H, Kun F, Yang J, Sun X, Yan M, Guo Z (2018) Automatic ship detection in remote sensing images from google earth of complex scenes based on multiscale rotation dense feature pyramid networks. Remote Sens 10(1):132–146CrossRef
11.
go back to reference Yang X, Sun H, Sun X, Yan M, Zhi G, Kun F (2018) Position detection and direction prediction for arbitrary-oriented ships via multitask rotation region convolutional neural network. IEEE Access 6:50839–50849CrossRef Yang X, Sun H, Sun X, Yan M, Zhi G, Kun F (2018) Position detection and direction prediction for arbitrary-oriented ships via multitask rotation region convolutional neural network. IEEE Access 6:50839–50849CrossRef
12.
go back to reference Jiang Y, Zhu X, Wang X, Yang S, Li W, Wang H, Fu P, Luo Z, R2CNN: rotational region CNN for orientation robust scene text detection. arXiv:1706.09579 Jiang Y, Zhu X, Wang X, Yang S, Li W, Wang H, Fu P, Luo Z, R2CNN: rotational region CNN for orientation robust scene text detection. arXiv:​1706.​09579
13.
go back to reference Yang X, Fu K, Sun H, Yang J, Guo Z, Yan M, Zhang T, Xian S, R2CNN++: multi-dimensional attention based rotation invariant detector with robust anchor strategy. arXiv:1811.07126 Yang X, Fu K, Sun H, Yang J, Guo Z, Yan M, Zhang T, Xian S, R2CNN++: multi-dimensional attention based rotation invariant detector with robust anchor strategy. arXiv:​1811.​07126
14.
go back to reference Ma J, Shao W, Ye H, Wang L, Wang H, Zheng Y, Xue X (2018) Arbitrary-oriented scene text detection via rotation proposals. IEEE Trans Multimedia 20(11):3111–3122CrossRef Ma J, Shao W, Ye H, Wang L, Wang H, Zheng Y, Xue X (2018) Arbitrary-oriented scene text detection via rotation proposals. IEEE Trans Multimedia 20(11):3111–3122CrossRef
15.
go back to reference Shermeyer J, Van Etten A (2019) The effects of super-resolution on object detection performance in satellite imagery. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops Shermeyer J, Van Etten A (2019) The effects of super-resolution on object detection performance in satellite imagery. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops
16.
go back to reference Haris M, Shakhnarovich G, Ukita N (2018) Task-driven super resolution: object detection in low-resolution images. arXiv preprint arXiv:1803.11316 Haris M, Shakhnarovich G, Ukita N (2018) Task-driven super resolution: object detection in low-resolution images. arXiv preprint arXiv:​1803.​11316
17.
go back to reference Chen Y, Li J, Xiao H, Jin X, Yan S, Feng J (2017) Dual path networks. In: Proceedings of the 2017 conference on advances in neural information processing systems, 2017, pp 4468–4476 Chen Y, Li J, Xiao H, Jin X, Yan S, Feng J (2017) Dual path networks. In: Proceedings of the 2017 conference on advances in neural information processing systems, 2017, pp 4468–4476
18.
go back to reference Lin T-Y, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition (CVPR), 2017, 936–944 Lin T-Y, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition (CVPR), 2017, 936–944
19.
go back to reference Hu J, Shen L, Sun G (2018) Squeeze-and-excitation Networks. In: Proceedings of the 2018 IEEE conference on computer vision and pattern recognition, 2018, pp 7132–7141 Hu J, Shen L, Sun G (2018) Squeeze-and-excitation Networks. In: Proceedings of the 2018 IEEE conference on computer vision and pattern recognition, 2018, pp 7132–7141
20.
go back to reference Uijlings JRR, van de Sande KEA, Gevers T, Smeulders AWM (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171CrossRef Uijlings JRR, van de Sande KEA, Gevers T, Smeulders AWM (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171CrossRef
21.
go back to reference Shamsolmoali P, Zareapoor M, Wang R et al (2019) A novel deep structure U-net for sea-land segmentation in remote sensing images. IEEE J Sel Top Appl Earth Observ Remote Sens 12(9):3219–3232CrossRef Shamsolmoali P, Zareapoor M, Wang R et al (2019) A novel deep structure U-net for sea-land segmentation in remote sensing images. IEEE J Sel Top Appl Earth Observ Remote Sens 12(9):3219–3232CrossRef
22.
go back to reference Shamsolmoali P, Zareapoor M, Wang R et al (2019) G-GANISR: gradual generative adversarial network for image super resolution. Neurocomputing 366:140–153CrossRef Shamsolmoali P, Zareapoor M, Wang R et al (2019) G-GANISR: gradual generative adversarial network for image super resolution. Neurocomputing 366:140–153CrossRef
23.
go back to reference Li F et al (2017) Super-resolution for GaoFen-4 remote sensing images. IEEE Geosci Remote Sens Lett 15(1):28–32CrossRef Li F et al (2017) Super-resolution for GaoFen-4 remote sensing images. IEEE Geosci Remote Sens Lett 15(1):28–32CrossRef
24.
go back to reference Wu W et al (2016) A new framework for remote sensing image super-resolution: sparse representation-based method by processing dictionaries with multi-type features. J Syst Archit 64:63–75CrossRef Wu W et al (2016) A new framework for remote sensing image super-resolution: sparse representation-based method by processing dictionaries with multi-type features. J Syst Archit 64:63–75CrossRef
25.
go back to reference Xie S, Girshick R, Dollar P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition (CVPR), 2017, pp 5987–5995 Xie S, Girshick R, Dollar P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition (CVPR), 2017, pp 5987–5995
26.
go back to reference Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition (CVPR), 2017, pp 2261–2269 Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition (CVPR), 2017, pp 2261–2269
27.
go back to reference Xia G-S, Bai X, Ding J, Zhu Z, Belongie S, Luo J, Datcu M, Pelillo M, Zhang L (2018) DOTA: a large-scale dataset for object detection in aerial images. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) 2018, pp 3974–3983 Xia G-S, Bai X, Ding J, Zhu Z, Belongie S, Luo J, Datcu M, Pelillo M, Zhang L (2018) DOTA: a large-scale dataset for object detection in aerial images. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) 2018, pp 3974–3983
28.
go back to reference Cheng G, Zhou P, Han J (2016) Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images. IEEE Trans Geosci Remote Sens 54(12):7405–7415CrossRef Cheng G, Zhou P, Han J (2016) Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images. IEEE Trans Geosci Remote Sens 54(12):7405–7415CrossRef
29.
go back to reference Zhang Y, Li K, Li K, Wang L, Zhong B, Yun F (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the IEEE conference on ECCV 2018, pp 1–16 Zhang Y, Li K, Li K, Wang L, Zhong B, Yun F (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the IEEE conference on ECCV 2018, pp 1–16
30.
go back to reference Eirikur A, Radu T (2017) NTIRE 2017 challenge on single image super-resolution: dataset and study. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) 2017, pp 1–10 Eirikur A, Radu T (2017) NTIRE 2017 challenge on single image super-resolution: dataset and study. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) 2017, pp 1–10
31.
go back to reference Tang T, Zhou S, Deng Z, Lei L, Zou H (2017) Arbitrary oriented vehicle detection in aerial imagery with single convolutional neural networks. Remote Sens 9(11):1170CrossRef Tang T, Zhou S, Deng Z, Lei L, Zou H (2017) Arbitrary oriented vehicle detection in aerial imagery with single convolutional neural networks. Remote Sens 9(11):1170CrossRef
32.
go back to reference Azimi SM, Vig E, Bahmanyar R, Körner M, Reinartz P (2018) Towards multi-class object detection in unconstrained remote sensing imagery. arXiv preprint, arXiv:1807.02700 Azimi SM, Vig E, Bahmanyar R, Körner M, Reinartz P (2018) Towards multi-class object detection in unconstrained remote sensing imagery. arXiv preprint, arXiv:​1807.​02700
33.
go back to reference Zhou X et al (2017) EAST: an efficient and accurate scene text detector. In: Proceedings of the IEEE conference on computer vision and pattern recognition Zhou X et al (2017) EAST: an efficient and accurate scene text detector. In: Proceedings of the IEEE conference on computer vision and pattern recognition
34.
go back to reference Ren Y, Zhu C, Xiao S (2018) Deformable faster R-CNN with aggregating multi-layer features for partially occluded object detection in optical remote sensing images. Remote Sens 10(9):1470CrossRef Ren Y, Zhu C, Xiao S (2018) Deformable faster R-CNN with aggregating multi-layer features for partially occluded object detection in optical remote sensing images. Remote Sens 10(9):1470CrossRef
35.
36.
go back to reference Han X, Zhong Y, Zhang L (2017) An efficient and robust integrated geospatial object detection framework for high spatial resolution remote sensing imagery. Remote Sens 9(7):666CrossRef Han X, Zhong Y, Zhang L (2017) An efficient and robust integrated geospatial object detection framework for high spatial resolution remote sensing imagery. Remote Sens 9(7):666CrossRef
37.
go back to reference Liu W et al (2016) SSD: single shot multibox detector. In: European conference on computer vision. Springer, Cham Liu W et al (2016) SSD: single shot multibox detector. In: European conference on computer vision. Springer, Cham
39.
go back to reference Shen Z et al (2017) DSOD: learning deeply supervised object detectors from scratch. In: Proceedings of the IEEE international conference on computer vision Shen Z et al (2017) DSOD: learning deeply supervised object detectors from scratch. In: Proceedings of the IEEE international conference on computer vision
40.
go back to reference Xu Z et al (2017) Deformable convnet with aspect ratio constrained NMS for object detection in remote sensing imagery. Remote Sens 9(12):1312CrossRef Xu Z et al (2017) Deformable convnet with aspect ratio constrained NMS for object detection in remote sensing imagery. Remote Sens 9(12):1312CrossRef
41.
go back to reference Li K et al (2017) Rotation-insensitive and context-augmented object detection in remote sensing images. IEEE Trans Geosci Remote Sens 56(4):2337–2348CrossRef Li K et al (2017) Rotation-insensitive and context-augmented object detection in remote sensing images. IEEE Trans Geosci Remote Sens 56(4):2337–2348CrossRef
42.
go back to reference Tian Z et al (2016) Detecting text in natural image with connectionist text proposal network. In: European conference on computer vision. Springer, Cham Tian Z et al (2016) Detecting text in natural image with connectionist text proposal network. In: European conference on computer vision. Springer, Cham
43.
go back to reference Shi B, Bai X, Belongie S (2017) Detecting oriented text in natural images by linking segments. In: Proceedings of the IEEE conference on computer vision and pattern recognition Shi B, Bai X, Belongie S (2017) Detecting oriented text in natural images by linking segments. In: Proceedings of the IEEE conference on computer vision and pattern recognition
Metadata
Title
Arbitrary-oriented object detection via dense feature fusion and attention model for remote sensing super-resolution image
Authors
Fuhao Zou
Wei Xiao
Wanting Ji
Kunkun He
Zhixiang Yang
Jingkuan Song
Helen Zhou
Kai Li
Publication date
07-05-2020
Publisher
Springer London
Published in
Neural Computing and Applications / Issue 18/2020
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-020-04893-9

Other articles of this Issue 18/2020

Neural Computing and Applications 18/2020 Go to the issue

S.I.: Deep Learning Approaches for RealTime Image Super Resolution (DLRSR)

GAN-Poser: an improvised bidirectional GAN model for human motion prediction

Premium Partner