Top

Neural Computing and Applications

Published in:

21-02-2024 | Original Article

Cross-dimensional feature attention aggregation network for cloud and snow recognition of high satellite images

Authors: Kai Hu, Enwei Zhang, Min Xia, Huiqin Wang, Xiaoling Ye, Haifeng Lin

Published in: Neural Computing and Applications | Issue 14/2024

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Cloud and snow in remote sensing images typically block the underlying surface information and interfere with the extraction of available information, so detecting cloud and snow becomes a critical problem in remotely sensed image processing. The current methods for detecting clouds and snow are susceptible to interference from complex background, making it difficult to recover cloud edge details and causing missing and false detection phenomena. To address these issues, a cross-dimensional feature attention aggregation network is suggested to realize the segmentation of clouds and snow. To address the problem of interference induced by the similar spectral characteristics of clouds and snow, the context attention aggregation module is added to conflate feature maps of various dimensions and screen the information. Multi-scale strip convolution module (MSSCM) and its improved version MSSCMs are used to extract edge characteristics at different scales and improve the harsh segmentation border. Also, adding deep feature semantic information extraction module to deep features to guide the classification of the model to avoid the interference of complex background. Finally, a ’los beatles’ module is used to replace the traditional linear combination in the decoding stage, and the feature information of different granularity is fused and extracted to enhance the model’s detection efficiency. In this paper, experiments are carried out on the public datasets: CSWV, HRC\(\_\)WHU and L8\(\_\)SPARCS. The MIOU scores on the three datasets are 89.507\(\%\), 91.674\(\%\) and 80.722\(\%\), respectively. Comparative experiment findings demonstrate that the network presented in this article can attain the highest detection accuracy and good detection efficiency with low parameters.

previous article Dynamic bias alignment and discrimination enhancement for unsupervised domain adaptation

next article GeoTrans: a transfer learning approach for estimating petrophysical properties from geophysical sensors data

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Chen X, Liang S, Cao Y, He T, Wang D (2015) Observed contrast changes in snow cover phenology in northern middle and high latitudes from 2001–2014. Sci Rep 5(1):1–9

Miao S, Xia M, Qian M, Zhang Y, Liu J, Lin H (2022) Cloud/shadow segmentation based on multi-level feature enhanced network for remote sensing imagery. Int J Remote Sens 43(15–16):5940–5960

Zhu Z, Qiu S, He B, Deng C (2018) Cloud and cloud shadow detection for Landsat images: the fundamental basis for analyzing Landsat time series. In: Weng Q (ed) Remote sensing time series image processing. CRC Press, Boca Raton, pp 3–23CrossRef

Paltridge GW, Platt CMR (1976) Radiative processes in meteorology and climatology. Elsevier Scientific Pub. C., Amsterdam

Chen L, Xia M, Qian M, Chen B (2022) Dual-branch network for cloud and cloud shadow segmentation. IEEE Trans Geosci Remote Sens 60:5410012

Song L, Xia M, Jin J, Qian M, Zhang Y (2021) SUACDNet: attentional change detection network based on Siamese u-shaped structure. Int J Appl Earth Obs Geoinf 105:102597

Junchang J, Roy DP (2008) The availability of cloud-free Landsat ETM+ data over the conterminous united states and globally. Remote Sens Environ 112(3):1196–1211

Dozier J (1989) Spectral signature of alpine snow cover from the Landsat thematic mapper. Remote Sens Environ 28:9–22

Roy DP, Junchang J, Kline K, Scaramuzza PL, Kovalskyy V, Hansen M, Loveland TR, Vermote E, Zhang C (2010) Web-enabled Landsat data (weld): Landsat ETM+ composited mosaics of the conterminous United States. Remote Sens Environ 114(1):35–49

10.

Huete A, Didan K, Tomoaki ME, Rodriguez P, Gao X, Ferreira LG (2002) Overview of the radiometric and biophysical performance of the MODIS vegetation indices. Remote Sens Environ 83(1–2):195–213

11.

Zhang Y, Guindon B, Cihlar J (2002) An image transform to characterize and compensate for spatial variations in thin cloud contamination of Landsat images. Remote Sens Environ 82(2–3):173–187CrossRef

12.

Weng L, Pang K, Xia M, Lin H, Qian M, Zhu C (2023) Sgformer: a local and global features coupling network for semantic segmentation of land cover. IEEE J Sel Top Appl Earth Obs Remote Sens 16:6812–6824

13.

Zhu Z, Woodcock CE (2014) Automated cloud, cloud shadow, and snow detection in multitemporal Landsat data: an algorithm designed specifically for monitoring land cover change. Remote Sens Environ 152:217–234

14.

Xie F, Shi M, Shi Z, Yin J, Zhao D (2017) Multilevel cloud detection in remote sensing images based on deep learning. IEEE J Sel Top Appl Earth Obs Remote Sens 10(8):3631–3640

15.

Zhang C, Weng L, Ding L, Xia M, Lin H (2023) CRSNet: cloud and cloud shadow refinement segmentation networks for remote sensing imagery. Remote Sens 15(6):96

16.

Ji H, Xia M, Zhang D, Lin H (2023) Multi-supervised feature fusion attention network for clouds and shadows detection. ISPRS Int J Geo-Inf 12(6):247

17.

Braaten JD, Cohen WB, Yang Z (2015) Automated cloud and cloud shadow identification in Landsat MSS imagery for temperate ecosystems. Remote Sens Environ 169:128–138

18.

Li Z, Shen H, Li H, Xia G, Gamba P, Zhang L (2017) Multi-feature combined cloud and cloud shadow detection in GaoFen-1 wide field of view imagery. Remote Sens Environ 191:342–358

19.

Tapakis R, Charalambides AG (2013) Equipment and methodologies for cloud detection and classification: a review. Solar Energy 95:392–430CrossRef

20.

Ping BS, Yunshan FM (2020) A cloud and cloud shadow detection method based on fuzzy c-means algorithm. IEEE J Sel Top Appl Earth Obs Remote Sens 13:1714–1727

21.

An Z, Shi Z (2015) Scene learning for cloud detection on remote-sensing images. IEEE J Sel Top Appl Earth Obs Remote Sens 8:4206–4222

22.

Fang Z, Ji W, Wang X, Li L, Li Y (2021) Automatic cloud and snow detection for GF-1 and PRSS-1 remote sensing images. J Appl Remote Sens 9:024516

23.

Corneliu OD, Mihai D (2013) Information content of very high resolution SAR images: study of feature extraction and imaging parameters. IEEE Trans Geosci Remote Sens 51(8):4591–4610CrossRef

24.

Geng J, Fan J, Wang H, Ma X, Li B, Chen F (2015) High-resolution SAR image classification via deep convolutional autoencoders. IEEE Geosci Remote Sens Lett 12(11):2351–2355

25.

Liu M, Yan W, Zhao W, Zhang Q, Li M, Liao G (2013) Dempster–Shafer fusion of multiple sparse representation and statistical property for SAR target configuration recognition. IEEE Geosci Remote Sens Lett 11(6):1106–1110

26.

Asner Gregory P (2001) Cloud cover in Landsat observations of the Brazilian amazon. Int J Remote Sens 22(18):3855–3862CrossRef

27.

Bossu J, Hautiere N, Tarel J-P (2011) Rain or snow detection in image sequences through use of a histogram of orientation of streaks. Int J Comput Vis 93(3):348–367

28.

Zhu Z, Woodcock CE (2012) Object-based cloud and cloud shadow detection in Landsat imagery. Remote Sens Environ 118:83–94CrossRef

29.

Joseph HM, Hayes DJ (2014) Automated detection of cloud and cloud shadow in single-date Landsat imagery using neural networks and spatial post-processing. Remote Sens 6(6):4907–4926CrossRef

30.

Zhu Z, Wang S, Woodcock CE (2015) Improvement and expansion of the Fmask algorithm: cloud, cloud shadow, and snow detection for Landsat’s 4–7, 8, and sentinel 2 images. Remote Sens Environ 159:269–277

31.

Le Goff M , Tourneret J-Y, H Wendt, M Ortner, M Spigai (2017) Deep learning for cloud detection. In: 8th International conference of pattern recognition systems (ICPRS 2017). IET, pp 1–6

32.

Chen Q, Zhang Z, Chen S, Wen S, Ma H, Zhihua X (2022) A self-attention based global feature enhancing network for semantic segmentation of large-scale urban street-level point clouds. Int J Appl Earth Obs Geoinf 113:102974

33.

Gao J, Weng L, Xia M, Lin H (2022) MLNet: multichannel feature fusion lozenge network for land segmentation. J Appl Remote Sens 16(1):1–19CrossRef

34.

Jiang X, Li Y, Jiang T, Xie J, Yilong W, Cai Q, Jiang J, Jiaming X, Zhang H (2022) Roadformer: pyramidal deformable vision transformers for road network extraction with remote sensing images. Int J Appl Earth Obs Geoinf 113:102987

35.

Song L, Xia M, Weng L, Lin H, Qian M, Chen B (2023) Axial cross attention meets CNN: bi-branch fusion network for change detection. IEEE J Sel Top Appl Earth Obs Remote Sen 16:32–43

36.

Peng D, Bruzzone L, Zhang Y, Guan H, He P (2021) SCDNet: a novel convolutional network for semantic change detection in high resolution optical remote sensing imagery. Int J Appl Earth Obs Geoinf 103:102465

37.

Gao W, Li X, Han Y, Liu Y (2022) Multi-scale vertical cross-layer feature aggregation and attention fusion network for object detection. In: International conference on artificial neural networks. Springer, pp 139–150

38.

Dai X, Chen K, Xia M, Weng L, Lin H (2023) LPMSNet: location pooling multi-scale network for cloud and cloud shadow segmentation. Remote Sens 15:4005CrossRef

39.

Ma Z, Xia M, Weng L, Lin H (2023) Local feature search network for building and water segmentation of remote sensing image. Sustainability 15(4):3034

40.

Kai H, Weng C, Shen C, Wang T, Weng L, Xia M (2023) A multi-stage underwater image aesthetic enhancement algorithm based on a generative adversarial network. Eng Appl Artif Intell 123:106196CrossRef

41.

Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 3431–3440

42.

Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

43.

Badrinarayanan V, Handa A, Cipolla R (2015) SEGNet: a deep convolutional encoder–decoder architecture for robust semantic pixel-wise labelling. Comput Sci

44.

Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 2881–2890

45.

Li H, Xiong P, An J, Wang L (2018) Pyramid attention network for semantic segmentation. arXiv preprint arXiv:1805.10180

46.

Sun K, Xiao B, Liu D, Wang J (2019) Deep high-resolution representation learning for human pose estimation. arXiv e-prints,

47.

Chai D, Newsam S, Zhang HK, Qiu Y, Huang J (2019) Cloud and cloud shadow detection in Landsat imagery based on deep convolutional neural networks. Remote Sens Environ 225:307–316CrossRef

48.

Zhang G, Gao X, Yang Y, Wang M, Ran S (2021) Controllably deep supervision and multi-scale feature fusion network for cloud and snow detection based on medium-and high-resolution imagery dataset. Remote Sens 13(23):4805CrossRef

49.

Li Y, Chen W, Zhang Y, Tao C, Xiao R, Tan Y (2020) Accurate cloud detection in high-resolution remote sensing imagery by weakly supervised deep learning. Remote Sens Environ 250:112045CrossRef

50.

Hongcai D, Li K, Guo J, Zhang J, Yang J (2019) Cloud and snow detection from remote sensing imagery based on convolutional neural network. In: Optoelectronic imaging and multimedia technology VI

51.

Xi W, Shi Z, Zou Z (2021) A geographic information-driven method and a new large scale dataset for remote sensing cloud/snow detection. ISPRS J Photogramm Remote Sens 174:87–104CrossRef

52.

Li Z, Shen H, Cheng Q, Liu Y, You S, He Z (2019) Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors. ISPRS J Photogramm Remote Sens 150:197–212CrossRef

53.

He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778,

54.

Li Z, Shen H, Cheng Q, Liu Y, You S, He Z (2018) Deep learning based cloud detection for remote sensing images by the fusion of multi-scale convolutional features. arXiv preprint arXiv:1810.05801,

55.

Wang Z, Xia M, Min L, Pan L, Liu J (2022) Parameter identification in power transmission systems based on graph convolution network. IEEE Trans Power Deliv 37(4):3155–3163CrossRef

56.

Howard A, Zhmoginov A, Chen L-C, Sandler M, Menglong Z (2018) Mobile networks for classification, detection and segmentation, inverted residuals and linear bottlenecks

57.

Liu A, Li S, Chang Y (2022) Cross-resolution feature attention network for image super-resolution. Vis Comput 39(9):3837–3849CrossRef

58.

Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122,

59.

Chen B, Xia M, Qian M, Huang J (2022) MANet: a multi-level aggregation network for semantic segmentation of high-resolution remote sensing images. Int J Remote Sens 43(15–16):5874–5894CrossRef

60.

Dai Y, Gieseke F, Oehmcke S, Wu Y, Barnard K (2021) Attentional feature fusion. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 3560–3569

61.

Woo S, Park J, Lee J-Y, Kweon IS (2018) Cbam: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

62.

Li H, Xiong P, Fan H, Sun J (2019) DFANet: deep feature aggregation for real-time semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9522–9531

63.

Yang M, Kun Y, Zhang C, Li Z, Yang K (2018) Denseaspp for semantic segmentation in street scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3684–3692

64.

Haiping W, Bin X, Noel C, Mengchen L, Xiyang D, Lu Y, Lei Z (2021) Cvt: introducing convolutions to vision transformers. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 22–31

65.

Chen L, Xia M, Lin H (2022) Multi-scale strip pooling feature aggregation network for cloud and cloud shadow segmentation. Neural Comput Appl 34(8):6149–6162CrossRef

66.

Paszke A, Chaurasia A, Kim S, Culurciello E (2016) ENet: A deep neural network architecture for real-time semantic segmentation. arXiv preprint arXiv:1606.02147,

67.

Xia M, Yi Q, Lin H (2021) PANDA: parallel asymmetric network with double attention for cloud and its shadow detection. J Appl Remote Sens 15(4):046512CrossRef

68.

Li G, Yun I, Kim J, Kim J (2019) DABNet: depth-wise asymmetric bottleneck for real-time semantic segmentation. arXiv preprint arXiv:1907.11357,

69.

Mehta S, Rastegari M, Shapiro L, Hajishirzi H (2019) Espnetv2: a light-weight, power efficient, and general purpose convolutional neural network, pp 9190–9200

70.

Huang Z, Wang X, Huang L, Huang C, Wei Y, Liu, W (2019) CCNet: Criss-cross attention for semantic segmentation. In: International conference on computer vision

71.

Wang W, Xie E, Li X, Fan DP, Shao L (2021) Pyramid vision transformer: a versatile backbone for dense prediction without convolutions, pp 568–578

72.

Hong Y, Pan H, Sun W, Member S, Jia Y (2021) Deep dual-resolution networks for real-time and accurate semantic segmentation of road scenes. arXiv preprint arXiv:2101.06085,

73.

Yu C, Gao C, Wang J, Yu G, Shen C, Sang N (2021) Bisenet v2: bilateral network with guided aggregation for real-time semantic segmentation. Int J Comput Vis 129(11):3051–3068CrossRef

74.

Zhang F, Chen Y, Li Z, Hong Z, Ding E (2019) ACFnet: attentional class feature network for semantic segmentation. IEEE

75.

Yuan Y, Chen X, Chen X, Wang J (2019) Segmentation transformer: object-contextual representations for semantic segmentation. arXiv preprint arXiv:1909.11065

76.

Yu C, Wang J, Peng C, Gao C, Yu G, Sang N (2018) Learning a discriminative feature network for semantic segmentation. IEEE

77.

Chen LC, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder–decoder with atrous separable convolution for semantic image segmentation. Springer, ChamCrossRef

78.

Yi Qu, Xia Min, Zhang Yonghong (2021) Strip pooling channel spatial attention network for the segmentation of cloud and cloud shadow. Comput Geosci 157:104940CrossRef

79.

Chen Bingyu, Xia Min, Huang Junqing (2021) CDUNet: cloud detection Unet for remote sensing imagery. Remote Sens 13:731CrossRef

80.

Xia Min, Wang Tao, Zhang Yonghong, Liu Jia, Yiqing Xu (2021) Cloud/shadow segmentation based on global attention feature fusion residual network for remote sensing imagery. Int J Remote Sens 42(6):2022–2045CrossRef

Title: Cross-dimensional feature attention aggregation network for cloud and snow recognition of high satellite images
Authors: Kai Hu
Enwei Zhang
Min Xia
Huiqin Wang
Xiaoling Ye
Haifeng Lin
Publication date: 21-02-2024
Publisher: Springer London
Published in: Neural Computing and Applications / Issue 14/2024
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI: https://doi.org/10.1007/s00521-024-09477-5

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 14/2024

Adaptive temperature scaling for Robust calibration of deep neural networks

Emotion recognition based on phase-locking value brain functional network and topological data analysis

Siamada: visual tracking based on Siamese adaptive learning network

Enhancing cardiac diagnostics through semantic-driven image synthesis: a hybrid GAN approach

On the adversarial robustness of generative autoencoders in the latent space

Cascade 2D attentional decoders with context-enhanced encoder for scene text recognition

Premium Partner