Skip to main content
Top
Published in: Multimedia Systems 5/2023

17-06-2023 | Regular Paper

A deraining with detail-recovery network via context aggregation

Authors: Weihao Gao, Yongjun Zhang, Wei Long, Zhongwei Cui

Published in: Multimedia Systems | Issue 5/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

As one representative object of bad weather, rain streaks can have a bad influence on image capture. Worse still, computer vision tasks can be badly influenced when working in rainy scenes. Therefore, it makes sense to conduct research on rain removal. This paper introduces a new deraining network (DDRNet). Specifically, rain streaks contain both frequency and spatial information, so we propose a rain removal module (RRM) consisting of a rain channel attention module (RCAM) and a rain spatial attention module (RSAM) for extracting rain information. Furthermore, we design a detail-recovery module (DRM) to extract the background feature so that the original background image details deleted by mistake during the deraining process can be made up by it. Moreover, a two-branch aggregation (TBA) mechanism is adopted to promote the process of information flow that effectively enhances the execution of our DDRNet. The evaluation of our DDRNet on various benchmark datasets has shown the superiority of our algorithm. In addition, a range of low-level experiments, such as object detection and semantic segmentation, provide further evidence of the effectiveness of the proposed DDRNet.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Zamanakos, G., Tsochatzidis, L., Amanatiadis, A., Pratikakis, I.: A comprehensive survey of lidar-based 3d object detection methods with deep learning for autonomous driving. Comput. Grap. 99, 153–181 (2021)CrossRef Zamanakos, G., Tsochatzidis, L., Amanatiadis, A., Pratikakis, I.: A comprehensive survey of lidar-based 3d object detection methods with deep learning for autonomous driving. Comput. Grap. 99, 153–181 (2021)CrossRef
2.
go back to reference Mei, R., Wang, H., Men, A.: Attention-enhanced and more balanced r-cnn for object detection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 2136–2140 . IEEE (2020) Mei, R., Wang, H., Men, A.: Attention-enhanced and more balanced r-cnn for object detection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 2136–2140 . IEEE (2020)
3.
go back to reference Vanian, V., Zamanakos, G., Pratikakis, I.: Improving performance of deep learning models for 3d point cloud semantic segmentation via attention mechanisms. Comput. Graph. 106, 277–287 (2022)CrossRef Vanian, V., Zamanakos, G., Pratikakis, I.: Improving performance of deep learning models for 3d point cloud semantic segmentation via attention mechanisms. Comput. Graph. 106, 277–287 (2022)CrossRef
4.
go back to reference Kang, L.-W., Lin, C.-W., Fu, Y.-H.: Automatic single-image-based rain streaks removal via image decomposition. IEEE Trans. Image Process. 21(4), 1742–1755 (2011)MathSciNetCrossRefMATH Kang, L.-W., Lin, C.-W., Fu, Y.-H.: Automatic single-image-based rain streaks removal via image decomposition. IEEE Trans. Image Process. 21(4), 1742–1755 (2011)MathSciNetCrossRefMATH
5.
go back to reference Huang, D.-A., Kang, L.-W., Wang, Y.-C.F., Lin, C.-W.: Self-learning based image decomposition with applications to single image denoising. IEEE Trans. Multimedia 16(1), 83–93 (2013)CrossRef Huang, D.-A., Kang, L.-W., Wang, Y.-C.F., Lin, C.-W.: Self-learning based image decomposition with applications to single image denoising. IEEE Trans. Multimedia 16(1), 83–93 (2013)CrossRef
6.
go back to reference Kim, J.-H., Lee, C., Sim, J.-Y., Kim, C.-S.: Single-image deraining using an adaptive nonlocal means filter. In: 2013 IEEE International Conference on Image Processing, pp. 914–917. IEEE (2013) Kim, J.-H., Lee, C., Sim, J.-Y., Kim, C.-S.: Single-image deraining using an adaptive nonlocal means filter. In: 2013 IEEE International Conference on Image Processing, pp. 914–917. IEEE (2013)
7.
go back to reference Chen, D.-Y., Chen, C.-C., Kang, L.-W.: Visual depth guided color image rain streaks removal using sparse coding. IEEE Trans. Circuits Syst. Video Technol. 24(8), 1430–1455 (2014)CrossRef Chen, D.-Y., Chen, C.-C., Kang, L.-W.: Visual depth guided color image rain streaks removal using sparse coding. IEEE Trans. Circuits Syst. Video Technol. 24(8), 1430–1455 (2014)CrossRef
8.
go back to reference Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3397–3405 (2015) Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3397–3405 (2015)
9.
go back to reference Li, Y., Tan, R.T., Guo, X., Lu, J., Brown, M.S.: Rain streak removal using layer priors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2736–2744 (2016) Li, Y., Tan, R.T., Guo, X., Lu, J., Brown, M.S.: Rain streak removal using layer priors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2736–2744 (2016)
10.
go back to reference Chang, Y., Yan, L., Zhong, S.: Transformed low-rank model for line pattern noise removal. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1726–1734 (2017) Chang, Y., Yan, L., Zhong, S.: Transformed low-rank model for line pattern noise removal. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1726–1734 (2017)
11.
go back to reference Zhu, L., Fu, C.-W., Lischinski, D., Heng, P.-A.: Joint bi-layer optimization for single-image rain streak removal. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2526–2534 (2017) Zhu, L., Fu, C.-W., Lischinski, D., Heng, P.-A.: Joint bi-layer optimization for single-image rain streak removal. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2526–2534 (2017)
12.
go back to reference Fu, X., Huang, J., Ding, X., Liao, Y., Paisley, J.: Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans. Image Process. 26(6), 2944–2956 (2017)MathSciNetCrossRefMATH Fu, X., Huang, J., Ding, X., Liao, Y., Paisley, J.: Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans. Image Process. 26(6), 2944–2956 (2017)MathSciNetCrossRefMATH
13.
go back to reference Li, R., Cheong, L.-F., Tan, R.T.: Single image deraining using scale-aware multi-stage recurrent network. arXiv preprint arXiv:1712.06830 (2017) Li, R., Cheong, L.-F., Tan, R.T.: Single image deraining using scale-aware multi-stage recurrent network. arXiv preprint arXiv:​1712.​06830 (2017)
14.
go back to reference Fan, Z., Wu, H., Fu, X., Huang, Y., Ding, X.: Residual-guide network for single image deraining. In: Proceedings of the 26th ACM International Conference on Multimedia, pp. 1751–1759 (2018) Fan, Z., Wu, H., Fu, X., Huang, Y., Ding, X.: Residual-guide network for single image deraining. In: Proceedings of the 26th ACM International Conference on Multimedia, pp. 1751–1759 (2018)
15.
go back to reference Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 254–269 (2018) Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 254–269 (2018)
16.
go back to reference Liu, J., Yang, W., Yang, S., Guo, Z.: Erase or fill? deep joint recurrent rain removal and reconstruction in videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3233–3242 (2018) Liu, J., Yang, W., Yang, S., Guo, Z.: Erase or fill? deep joint recurrent rain removal and reconstruction in videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3233–3242 (2018)
17.
go back to reference Zhang, H., Patel, V.M.: Density-aware single image de-raining using a multi-stream dense network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 695–704 (2018) Zhang, H., Patel, V.M.: Density-aware single image de-raining using a multi-stream dense network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 695–704 (2018)
18.
go back to reference Li, R., Cheong, L.-F., Tan, R.T.: Heavy rain image restoration: Integrating physics model and conditional adversarial learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1633–1642 (2019) Li, R., Cheong, L.-F., Tan, R.T.: Heavy rain image restoration: Integrating physics model and conditional adversarial learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1633–1642 (2019)
19.
go back to reference Wang, T., Yang, X., Xu, K., Chen, S., Zhang, Q., Lau, R.W.: Spatial attentive single-image deraining with a high quality real rain dataset. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12270–12279 (2019) Wang, T., Yang, X., Xu, K., Chen, S., Zhang, Q., Lau, R.W.: Spatial attentive single-image deraining with a high quality real rain dataset. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12270–12279 (2019)
20.
go back to reference Wei, W., Meng, D., Zhao, Q., Xu, Z., Wu, Y.: Semi-supervised transfer learning for image rain removal. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3877–3886 (2019) Wei, W., Meng, D., Zhao, Q., Xu, Z., Wu, Y.: Semi-supervised transfer learning for image rain removal. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3877–3886 (2019)
21.
go back to reference Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. IEEE Trans. Circuits Syst. Video Technol. 30(11), 3943–3956 (2019)CrossRef Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. IEEE Trans. Circuits Syst. Video Technol. 30(11), 3943–3956 (2019)CrossRef
22.
go back to reference Zheng, Y., Yu, X., Liu, M., Zhang, S.: Residual multiscale based single image deraining. In: Bmvc, p. 147 (2019) Zheng, Y., Yu, X., Liu, M., Zhang, S.: Residual multiscale based single image deraining. In: Bmvc, p. 147 (2019)
23.
go back to reference Jiang, K., Wang, Z., Yi, P., Chen, C., Huang, B., Luo, Y., Ma, J., Jiang, J.: Multi-scale progressive fusion network for single image deraining. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8346–8355 (2020) Jiang, K., Wang, Z., Yi, P., Chen, C., Huang, B., Luo, Y., Ma, J., Jiang, J.: Multi-scale progressive fusion network for single image deraining. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8346–8355 (2020)
24.
go back to reference Wang, G., Sun, C., Sowmya, A.: Context-enhanced representation learning for single image deraining. Int. J. Comput. Vision 129, 1650–1674 (2021)CrossRef Wang, G., Sun, C., Sowmya, A.: Context-enhanced representation learning for single image deraining. Int. J. Comput. Vision 129, 1650–1674 (2021)CrossRef
25.
go back to reference Liang, S., Meng, X., Su, Z., Zhou, F.: Multi-receptive field aggregation network for single image deraining. J. Vis. Commun. Image Represent. 84, 103469 (2022)CrossRef Liang, S., Meng, X., Su, Z., Zhou, F.: Multi-receptive field aggregation network for single image deraining. J. Vis. Commun. Image Represent. 84, 103469 (2022)CrossRef
26.
go back to reference He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016) He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
27.
go back to reference Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, Springer. pp. 234–241 (2015) Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, Springer. pp. 234–241 (2015)
28.
go back to reference Cao, M., Gao, Z., Ramesh, B., Mei, T., Cui, J.: Single image deraining integrating physics model and density-oriented conditional gan refinement. IEEE Signal Process. Lett. 28, 1635–1639 (2021)CrossRef Cao, M., Gao, Z., Ramesh, B., Mei, T., Cui, J.: Single image deraining integrating physics model and density-oriented conditional gan refinement. IEEE Signal Process. Lett. 28, 1635–1639 (2021)CrossRef
29.
go back to reference Xu, J., Ai, B., Chen, W., Yang, A., Sun, P., Rodrigues, M.: Wireless image transmission using deep source channel coding with attention modules. IEEE Trans. Circuits Syst. Video Technol. 32(4), 2315–2328 (2021)CrossRef Xu, J., Ai, B., Chen, W., Yang, A., Sun, P., Rodrigues, M.: Wireless image transmission using deep source channel coding with attention modules. IEEE Trans. Circuits Syst. Video Technol. 32(4), 2315–2328 (2021)CrossRef
30.
go back to reference Charbonnier, P., Blanc-Feraud, L., Aubert, G., Barlaud, M.: Two deterministic half-quadratic regularization algorithms for computed imaging. In: Proceedings of 1st International Conference on Image Processing, vol. 2, pp. 168–172 (1994). IEEE Charbonnier, P., Blanc-Feraud, L., Aubert, G., Barlaud, M.: Two deterministic half-quadratic regularization algorithms for computed imaging. In: Proceedings of 1st International Conference on Image Processing, vol. 2, pp. 168–172 (1994). IEEE
31.
go back to reference Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., Yan, S.: Deep joint rain detection and removal from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1357–1366 (2017) Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., Yan, S.: Deep joint rain detection and removal from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1357–1366 (2017)
32.
go back to reference Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J.: Removing rain from single images via a deep detail network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3855–3863 (2017) Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J.: Removing rain from single images via a deep detail network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3855–3863 (2017)
33.
go back to reference Li, S., Araujo, I.B., Ren, W., Wang, Z., Tokuda, E.K., Junior, R.H., Cesar-Junior, R., Zhang, J., Guo, X., Cao, X.: Single image deraining: A comprehensive benchmark analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3838–3847 (2019) Li, S., Araujo, I.B., Ren, W., Wang, Z., Tokuda, E.K., Junior, R.H., Cesar-Junior, R., Zhang, J., Guo, X., Cao, X.: Single image deraining: A comprehensive benchmark analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3838–3847 (2019)
34.
go back to reference Hu, X., Fu, C.-W., Zhu, L., Heng, P.-A.: Depth-attentional features for single-image rain removal. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8022–8031 (2019) Hu, X., Fu, C.-W., Zhu, L., Heng, P.-A.: Depth-attentional features for single-image rain removal. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8022–8031 (2019)
35.
go back to reference Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRef Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRef
36.
go back to reference Mittal, A., Soundararajan, R., Bovik, A.C.: Making a completely blind" image quality analyzer. IEEE Signal Process. Lett. 20(3), 209–212 (2012)CrossRef Mittal, A., Soundararajan, R., Bovik, A.C.: Making a completely blind" image quality analyzer. IEEE Signal Process. Lett. 20(3), 209–212 (2012)CrossRef
37.
go back to reference Liu, L., Liu, B., Huang, H., Bovik, A.C.: No-reference image quality assessment based on spatial and spectral entropies. Signal Process. 29(8), 856–863 (2014) Liu, L., Liu, B., Huang, H., Bovik, A.C.: No-reference image quality assessment based on spatial and spectral entropies. Signal Process. 29(8), 856–863 (2014)
39.
go back to reference Yasarla, R., Patel, V.M.: Uncertainty guided multi-scale residual learning-using a cycle spinning cnn for single image de-raining. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8405–8414 (2019) Yasarla, R., Patel, V.M.: Uncertainty guided multi-scale residual learning-using a cycle spinning cnn for single image de-raining. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8405–8414 (2019)
40.
go back to reference Ren, D., Zuo, W., Hu, Q., Zhu, P., Meng, D.: Progressive image deraining networks: A better and simpler baseline. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3937–3946 (2019) Ren, D., Zuo, W., Hu, Q., Zhu, P., Meng, D.: Progressive image deraining networks: A better and simpler baseline. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3937–3946 (2019)
41.
go back to reference Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.-H., Shao, L.: Multi-stage progressive image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14821–14831 (2021) Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.-H., Shao, L.: Multi-stage progressive image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14821–14831 (2021)
42.
go back to reference Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 801–818 (2018) Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 801–818 (2018)
Metadata
Title
A deraining with detail-recovery network via context aggregation
Authors
Weihao Gao
Yongjun Zhang
Wei Long
Zhongwei Cui
Publication date
17-06-2023
Publisher
Springer Berlin Heidelberg
Published in
Multimedia Systems / Issue 5/2023
Print ISSN: 0942-4962
Electronic ISSN: 1432-1882
DOI
https://doi.org/10.1007/s00530-023-01116-8

Other articles of this Issue 5/2023

Multimedia Systems 5/2023 Go to the issue