Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Neural Computing and Applications
Published in: Neural Computing and Applications 5/2023

02-08-2021 | S.I. : Deep Geospatial Data Understanding

Scale-free heterogeneous cycleGAN for defogging from a single image for autonomous driving in fog

Authors: Hang Sun, Yan Zhang, Peng Chen, Zhiping Dan, Shuifa Sun, Jun Wan, Weisheng Li

Published in: Neural Computing and Applications | Issue 5/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In recent years, convolutional neural networks have been widely used in image defogging and achieved remarkable performance. However, most of the learning-based defogging methods using pairs of synthetic foggy and corresponding ground-truth images for training. Due to the gap between the distribution of real-word foggy and synthetic foggy images, there are limits to apply these defogging methods to autonomous driving in practice. CycleGAN is a two-way GANs network that could using unpaired real-word images to train image defogging models. However, there are several problems when CycleGAN is directly used for image defogging: (1) Using two different distributed data to train the same generator will confuse the learning of generators, which reduce the convergence speed and defogging results of the network. (2) The generator ignores the global features exploration which is very important for learning the scene and lighting information and needs the post-processing to restore the original image scales, which will result in a decrease in the quality of the generated image. To address these issues, we propose a Scale-free Heterogeneous CycleGAN (SH-CycleGAN) to utilize unpaired real-word images for boosting image defogging. The SH-CycleGAN contains a Heterogeneous Learning CycleGAN (HLCG) framework, and a generator with a Global Features Fusion module and an Adaptive Pooling module(GFFAP). Specifically, in the proposed HLCF framework, each BatchNorm layer learning in the generator is independent, which can solve the problem of confusion caused by learning different distributed data. Furthermore, the development of the GFFAP model can deal with image of any scales and improve the generated images. The experiments compared with eight state-of-the-art image defogging methods on both synthetic and real-world images demonstrate that our proposed method outperforms state-of-the-art performance and obtains more pleasing visually defogging results.
previous article PAENL: personalized attraction enhanced network learning for recommendation
next article Efficient locality-sensitive hashing over high-dimensional streaming data

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
Wang Y, Chao W, Garg D, Hariharan B, Campbell ME, Weinberger KQ (2019) Pseudo-lidar from visual depth estimation: bridging the gap in 3d object detection for autonomous driving. In: IEEE conference on computer vision and pattern recognition, CVPR 2019, Long Beach, CA, USA, 16–20 June 2019, Computer vision foundation/IEEE, pp 8445–8453
2.
Ma X, Wang Z, Li H, Zhang P, Ouyang W, Fan X (2019) Accurate monocular 3d object detection via color-embedded 3d reconstruction for autonomous driving. In: 2019 IEEE/CVF international conference on computer vision, ICCV 2019, Seoul, Korea (South), 27 Oct–2 Nov 2019, IEEE, pp 6850–6859
3.
Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In: IEEE conference on computer vision and pattern recognition, CVPR 2019, Long Beach, CA, USA, 16–20 June 2019, Computer vision foundation/IEEE, pp 3146–3154
4.
Bochkovskiy A, Wang C, Liao HM (2020) Yolov4: optimal speed and accuracy of object detection. CoRR, vol abs/2004.10934
5.
Ding M, Huo Y, Yi H, Wang Z, Shi J, Lu Z, Luo P (2020) Learning depth-guided convolutions for monocular 3d object detection. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, CVPR 2020, Seattle, WA, USA, 13–19 June 2020, IEEE, pp 11669–11678
6.
Cao J, Cholakkal H, Anwer RM, Khan FS, Pang Y, Shao L (2020) D2det: towards high quality object detection and instance segmentation. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, CVPR 2020, Seattle, WA, USA, 13–19 June 2020, IEEE , pp 11482–11491
7.
Sun H, Li J, Chang J, Du B, Su Z (2016) Efficient compressive sensing tracking via mixed classifier decision. Sci China Inf Sci 59(7):072102:1–072102:15CrossRef
8.
Guo D, Wang J, Cui Y, Wang Z, Chen S (2020) Siamcar: siamese fully convolutional classification and regression for visual tracking. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, CVPR 2020, Seattle, WA, USA, 13–19 June 2020, IEEE, pp 6268–6276
9.
Xu Y, Osep A, Ban Y, Horaud R, Leal-Taixé L, Alameda-Pineda X (2020) How to train your deep multi-object tracker. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, CVPR 2020, Seattle, WA, USA, 13–19 June 2020, IEEE, pp 6786–6795
10.
Weng X, Wang Y, Man Y, Kitani KM (2020) GNN3DMOT: graph neural network for 3d multi-object tracking with 2d-3d multi-feature learning. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, CVPR 2020, Seattle, WA, USA, 13–19 June 2020, IEEE, pp 6498–6507
11.
Wan J, Li J, Lai Z, Du B, Zhang L (2020) Robust face alignment by cascaded regression and de-occlusion. Neural Networks 123:261–272CrossRef
12.
Liu J, Chen Z, Du B, Tao D (2020) ASTS: a uni-fied framework for arbitrary shape text spotting. IEEE Trans Image Process 29:5924–5936CrossRefMATH
13.
McCartney E, Hall J, Freeman F, Optics of the atmosphere: Scattering by molecules and particles. Phys Today, 1977
14.
Nayar SK, Narasimhan SG (1999) Vision in bad weather. In: proceedings of the international conference on computer vision, Kerkyra, Corfu, Greece, 20–25 Sept 1999, IEEE computer society, pp 820–827
15.
16.
Ancuti CO, Ancuti C, Hermans C, Bekaert P (2010) A fast semi-inverse approach to detect and remove the haze from a single image. In: Kimmel R, Klette R, Sugimoto A (eds) Computer vision—ACCV 2010—10th Asian conference on computer vision, Queenstown, New Zealand, 8–12 Nov 2010, Revised selected papers, Part II, vol 6493 of lecture notes in computer science, Springer, pp 501–514
17.
He K, Sun J, Tang X (2011) Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell 33(12):2341–2353CrossRef
18.
Fattal R (2014) Dehazing using color-lines. ACM Trans Graph 34(1):13:1-13:14CrossRef
19.
Zhu Q, Mai J, Shao L (2015) A fast single image haze removal algorithm using color attenuation prior. IEEE Trans Image Process 24(11):3522–3533CrossRefMATH
20.
Berman D, Treibitz T, Avidan S (2016) Non-local image dehazing. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016, IEEE computer society, pp 1674–1682
21.
Cai B, Xu X, Jia K, Qing C, Tao D (2016) Dehazenet: an end-to-end system for single image haze removal. IEEE Trans Image Process 25(11):5187–5198CrossRefMATH
22.
Ren W, Liu S, Zhang H, Pan J, Cao X, Yang M (2016) Single image dehazing via multi-scale convolutional neural networks. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer vision—ECCV 2016—14th European conference, Amsterdam, The Netherlands, 11–14 Oct 2016, Proceedings, Part II, vol 9906 of lecture notes in computer science, Springer, pp 154–169
23.
Zhang H, Patel VM (2018) Densely connected pyramid dehazing network. In: 2018 IEEE conference on computer vision and pattern recognition, CVPR 2018, Salt Lake City, UT, USA, 18–22 June 2018, IEEE computer society, pp 3194–3203
24.
Yang X, Xu Z, Luo J (2018) Towards perceptual image dehazing by physics-based disentanglement and adversarial training. In: McIlraith SA, Weinberger KQ (eds) Proceedings of the thirty-second AAAI conference on artificial intelligence, (AAAI-18), the 30th innovative applications of artificial intelligence (IAAI-18), and the 8th AAAI symposium on educational advances in artificial intelligence (EAAI-18), New Orleans, Louisiana, USA, 2–7 Feb 2018, AAAI Press, pp 7485–7492
25.
Pan J, Liu S, Sun D, Zhang J, Liu Y, Ren JSJ, Li Z, Tang J, Lu H, Tai Y, Yang M (2018) Learning dual convolutional neural networks for low-level vision. In: 2018 IEEE conference on computer vision and pattern recognition, CVPR 2018, Salt Lake City, UT, USA, 18–22 June 2018, IEEE computer society, pp 3070–3079
26.
Li B, Peng X, Wang Z, Xu J, Feng D (2017) Aod-net: All-in-one dehazing network. In: IEEE international conference on computer vision, ICCV 2017, Venice, Italy, 22–29 Oct 2017, IEEE computer society, pp 4780–4788
27.
Li R, Pan J, Li Z, Tang J (2018) Single image dehazing via conditional generative adversarial network. In: 2018 IEEE conference on computer vision and pattern recognition, CVPR 2018, Salt Lake City, UT, USA, 18–22 June 2018, IEEE computer society, pp 8202–8211
28.
Qu Y, Chen Y, Huang J, Xie Y (2019) Enhanced pix2pix dehazing network. In: IEEE conference on computer vision and pattern recognition, CVPR 2019, Long Beach, CA, USA, 16–20 June 2019, Computer vision foundation/IEEE, pp 8160–8168
29.
Zhang H, Sindagi V, Patel VM (2018) Multi-scale single image dehazing using perceptual pyramid deep network. In: 2018 IEEE conference on computer vision and pattern recognition workshops, CVPR Workshops 2018, Salt Lake City, UT, USA, 18–22 June 2018, IEEE computer society, pp 902–911
30.
Mei K, Jiang A, Li J, Wang M (2018) Progressive feature fusion network for realistic image dehazing. In: Jawahar CV, Li H, Mori G, Schindler K (eds) Computer vision—ACCV 2018—14th Asian conference on computer vision, Perth, Australia, 2–6 Dec 2018, Revised selected papers, Part I, vol 11361 of lecture notes in computer science, Springer, pp 203–215
31.
Liu X, Suganuma M, Sun Z, Okatani T (2019) Dual residual networks leveraging the potential of paired operations for image restoration. In: IEEE conference on computer vision and pattern recognition, CVPR 2019, Long Beach, CA, USA, 16–20 June 2019, Computer vision foundation/IEEE, pp 7007–7016
32.
Liu X, Ma Y, Shi Z, Chen J (2019) Griddehazenet: attention-based multi-scale network for image dehazing. In: 2019 IEEE/CVF international conference on computer vision, ICCV 2019, Seoul, Korea (South), 27 Oct–2 Nov 2019, IEEE, pp 7313–7322
33.
Liu Y, Pan J, Ren JSJ, Su Z (2019) Learning deep priors for image dehazing. In: 2019 IEEE/CVF international conference on computer vision, ICCV 2019, Seoul, Korea (South), 27 Oct–2 Nov 2019, IEEE, pp 2492–2500
34.
Chen Y, Li J, Xiao H, Jin X, Yan S, Feng J (2017) Dual path networks. In: Guyon I, von Luxburg U, Bengio S, Wallach HM, Fergus R, Vishwanathan SVN, Garnett R (eds) Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, 4–9 Dec 2017, Long Beach, CA, USA, pp 4467–4475
35.
Engin D, Genç A, Ekenel HK (2018) Cycle-dehaze: enhanced cyclegan for single image dehazing. In: 2018 IEEE conference on computer vision and pattern recognition workshops, CVPR Workshops 2018, Salt Lake City, UT, USA,18–22 June 2018, IEEE computer society, pp 825–833
36.
Dong Y, Liu Y, Zhang H, Chen S, Qiao Y (2020) FD-GAN: generative adversarial networks with fusion-discriminator for single image dehazing. In: The thirty-fourth AAAI conference on artificial intelligence, AAAI 2020, The thirty-second innovative applications of artificial intelligence conference, IAAI 2020, The tenth AAAI symposium on educational advances in artificial intelligence, EAAI 2020, New York, NY, USA, 7–12 Feb 2020, AAAI Press, pp 10729–10736
37.
Ancuti C, Ancuti CO, Timofte R (2018) NTIRE 2018 challenge on image dehazing: methods and results. In: 2018 IEEE conference on computer vision and pattern recognition workshops, CVPR workshops 2018, Salt Lake City, UT, USA, 18–22 June 2018, IEEE computer society, pp 891–901
38.
Li B, Ren W, Fu D, Tao D, Feng D, Zeng W, Wang Z (2017) RESIDE: a benchmark for single image dehazing. CoRR, vol abs/1712.04143
39.
Bychkovsky V, Paris S, Chan E, Durand F (2011) Learning photographic global tonal adjustment with a database of input/output image pairs. In: The 24th IEEE conference on computer vision and pattern recognition, CVPR 2011, Colorado Springs, CO, USA, 20–25 June 2011, IEEE computer society, pp 97–104
40.
Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville AC, Bengio Y (2014) Generative adversarial nets. In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger, KQ (eds) Advances in neural information processing systems 27: annual conference on neural information processing systems 2014, 8–13 Dec 2014, Montreal, Quebec, Canada, pp 2672–2680
41.
Arjovsky M, Chintala S, Bottou L (2017) Wasserstein GAN. CoRR, vol abs/1701.07875
42.
Zhu J, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE international conference on computer vision, ICCV 2017, Venice, Italy, 22–29 Oct 2017, IEEE Computer Societ, pp 2242–2251
43.
Brock A, Donahue J, Simonyan K (2019) Large scale GAN training for high fidelity natural image synthesis. In: 7th international conference on learning representations, ICLR 2019, New Orleans, LA, USA, 6–9 May 2019, OpenReview.net
44.
Chen X, Duan Y, Houthooft R, Schulman J, Sutskever I, Abbeel P (2016) Infogan: interpretable representation learning by information maximizing generative adversarial nets. In: Lee DD, Sugiyama M, von Luxburg U, Guyon I, Garnett R (eds) Advances in neural information processing systems 29: annual conference on neural information processing systems 2016, 5–10 Dec 2016, Barcelona, Spain, pp 2172–2180
45.
Mao X, Li Q, Xie H, Lau RYK, Wang Z, Smolley SP (2017) Least squares generative adversarial networks. In: IEEE international conference on computer vision, ICCV 2017, Venice, Italy, 22–29 Oct 2017, IEEE computer society, pp 2813–2821
46.
Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Qiao Y, Loy CC (2018) ESRGAN: enhanced super-resolution generative adversarial networks. In: Leal-Taixé L, Roth S (eds) Computer vision—ECCV 2018 Workshops—Munich, Germany, 8–14 Sept 2018, Proceedings, Part V, vol 11133 of lecture notes in computer science, Springer, pp 63–79
47.
Bell-Kligler S, Shocher A, Irani M (2019) Blind super-resolution kernel estimation using an internal-gan. In: Wallach HM, Larochelle H, Beygelzimer A, d’Alché-Buc F, Fox EB, Garnett R (eds) Advances in neural information processing systems 32: annual conference on neural information processing systems 2019, NeurIPS 2019, 8–14 Dec 2019, Vancouver, BC, Canada, pp 284–293
48.
Ji X, Cao Y, Tai Y, Wang C, Li J, Huang F (2020) Real-world super-resolution via kernel estimation and noise injection. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, CVPR workshops 2020, Seattle, WA, USA, 14–19 June 2020, IEEE, pp 1914–1923
49.
Islam MJ, Xia Y, Sattar J (2020) Fast underwater image enhancement for improved visual perception. IEEE Robotics Autom Lett 5(2):3227–3234CrossRef
50.
Jiang Y, Gong X, Liu D, Cheng Y, Fang C, Shen X, Yang J, Zhou P, Wang Z (2019) Enlightengan: deep light enhancement without paired supervision. CoRR, vol abs/1906.06972
51.
Karras T, Laine S, Aittala M, Hellsten J, Lehtinen J, Aila T (2020) Analyzing and improving the image quality of stylegan. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, CVPR 2020, Seattle, WA, USA, 13–19 June 2020, IEEE, pp 8107–8116
52.
Liu W, Hou X, Duan J, Qiu G (2020) End-to-end single image fog removal using enhanced cycle consistent adversarial networks. IEEE Trans Image Process 29:7819–7833CrossRefMATH
53.
54.
Tan RT (2008) Visibility in bad weather from a single image. In: CVPR 2008 IEEE computer society conference on computer vision and pattern recognition, 24–26 June 2008. Alaska, USA, IEEE computer society, Anchorage
55.
Meng G, Wang Y, Duan J, Xiang S, Pan C (2013) Efficient image dehazing with boundary constraint and contextual regularization. In: IEEE international conference on computer vision, ICCV 2013, Sydney, Australia, 1–8 Dec 2013, IEEE computer society, pp 617–624
56.
Tang K, Yang J, Wang J (2014) Investigating haze-relevant features in a learning framework for image dehazing. In: 2014 IEEE conference on computer vision and pattern recognition, CVPR 2014, Columbus, OH, USA, 23–28 June 2014, IEEE computer society, pp 2995–3002
57.
Zhang H, Sindagi V, Patel VM (2020) Joint transmission map estimation and dehazing using deep networks. IEEE Trans Circuits Syst Video Technol 30(7):1975–1986
58.
Sun H, Ang MH, Rus D (2019) A convolutional network for joint deraining and dehazing from a single image for autonomous driving in rain. In: 2019 IEEE/RSJ international conference on intelligent robots and systems, IROS 2019, Macau, SAR, China, 3–8 Nov 2019, IEEE, pp 962–969
59.
Zhang Y, Ding L, Sharma G (2017) Hazerd: an outdoor scene dataset and benchmark for single image dehazing. In: 2017 IEEE international conference on image processing, ICIP 2017, Beijing, China, 17–20 Sept 2017, IEEE, pp 3205–3209
60.
Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: Bengio Y, LeCun Y (eds) 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, conference track proceedings
Metadata
Title
Scale-free heterogeneous cycleGAN for defogging from a single image for autonomous driving in fog
Authors
Hang Sun
Yan Zhang
Peng Chen
Zhiping Dan
Shuifa Sun
Jun Wan
Weisheng Li
Publication date
02-08-2021
Publisher
Springer London
Published in
Neural Computing and Applications / Issue 5/2023
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-021-06296-w

Other articles of this Issue 5/2023

Neural Computing and Applications 5/2023 Go to the issue

Original Article

Automated fabric inspection through convolutional neural network: an approach

Original Article

Few-shot learning for modeling cyber physical systems in non-stationary environments

Original Article

Segmentation on remote sensing imagery for atmospheric air pollution using divergent differential evolution algorithm

Original Article

Multiobjective problem modeling of the capacitated vehicle routing problem with urgency in a pandemic period

S.I. : Deep Geospatial Data Understanding

SmartDL: energy-aware decremental learning in a mobile-based federation for geo-spatial system

Original Article

A novel whale optimization algorithm optimized XGBoost regression for estimating bearing capacity of concrete piles

Premium Partner

    Image Credits