Skip to main content
Top
Published in: Neural Processing Letters 6/2023

05-06-2023

Flow Learning Based Dual Networks for Low-Light Image Enhancement

Authors: Siyu Wang, Changhui Hu, Weilin Yi, Ziyun Cai, Mingliang Zhai, Wankou Yang

Published in: Neural Processing Letters | Issue 6/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The deep learning-based low-light image enhancement task aims to learn a mapping that converts low-light images to normally exposed images by training with paired or unpaired datasets. Most of these existing methods are based on convolutional neural networks, which largely limits the network’s ability to learn global information. Meanwhile, the reconstruction loss or adversarial loss they adopt often cannot accurately measure the visual distance between the prediction and the target, resulting in blurred regions in the enhancement results. In this paper, we propose a novel flow learning based dual networks (FDN), which consists of a dual network and a flow learning based model. The dual network is mainly composed of a reidual-based Unet encoder and a residual-based Swin Transformer encoder, which can make up for the lack of global information processing and has more advantages in processing deep and shallow information. Moreover, we use a single loss function named negative log-likelihood to train the entire network, which enables the flow models to adequately learn the complex conditional distribution of normally exposed images and avoid blurry outputs. Experimental results on two benchmark datasets show that the proposed FDN method can achieve the sate-of-the-art performances on low-light image enhancement task.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Li C, Guo C, Han L-H, Jiang J, Cheng M-M, Gu J, Loy CC (2021) Low-light image and video enhancement using deep learning: a survey. IEEE Trans Pattern Anal Mach Intell 44:9396–9416CrossRef Li C, Guo C, Han L-H, Jiang J, Cheng M-M, Gu J, Loy CC (2021) Low-light image and video enhancement using deep learning: a survey. IEEE Trans Pattern Anal Mach Intell 44:9396–9416CrossRef
2.
go back to reference Liu J, Xu D, Yang W, Fan M, Huang H (2021) Benchmarking low-light image enhancement and beyond. Int J Comput Vis 129(4):1153–1184CrossRef Liu J, Xu D, Yang W, Fan M, Huang H (2021) Benchmarking low-light image enhancement and beyond. Int J Comput Vis 129(4):1153–1184CrossRef
3.
go back to reference Wang H, Peng J, Chen D, Jiang G, Zhao T, Fu X (2020) Attribute-guided feature learning network for vehicle reidentification. IEEE Multimedia 27(4):112–121CrossRef Wang H, Peng J, Chen D, Jiang G, Zhao T, Fu X (2020) Attribute-guided feature learning network for vehicle reidentification. IEEE Multimedia 27(4):112–121CrossRef
4.
go back to reference Ibrahim H, Kong NSP (2007) Brightness preserving dynamic histogram equalization for image contrast enhancement. IEEE Trans Consum Electron 53(4):1752–1758CrossRef Ibrahim H, Kong NSP (2007) Brightness preserving dynamic histogram equalization for image contrast enhancement. IEEE Trans Consum Electron 53(4):1752–1758CrossRef
6.
go back to reference Wang S, Zheng J, Hu H-M, Li B (2013) Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans Image Process 22(9):3538–3548CrossRef Wang S, Zheng J, Hu H-M, Li B (2013) Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans Image Process 22(9):3538–3548CrossRef
7.
go back to reference Guo X, Li Y, Ling H (2016) Lime: low-light image enhancement via illumination map estimation. IEEE Trans Image Process 26(2):982–993MathSciNetCrossRefMATH Guo X, Li Y, Ling H (2016) Lime: low-light image enhancement via illumination map estimation. IEEE Trans Image Process 26(2):982–993MathSciNetCrossRefMATH
8.
go back to reference Lore KG, Akintayo A, Sarkar S (2017) LLNet: a deep autoencoder approach to natural low-light image enhancement. Pattern Recognit 61:650–662CrossRef Lore KG, Akintayo A, Sarkar S (2017) LLNet: a deep autoencoder approach to natural low-light image enhancement. Pattern Recognit 61:650–662CrossRef
10.
go back to reference Zhang Y, Zhang J, Guo X (2019) Kindling the darkness: a practical low-light image enhancer. In: Proceedings of the 27th ACM international conference on multimedia, pp 1632–1640 Zhang Y, Zhang J, Guo X (2019) Kindling the darkness: a practical low-light image enhancer. In: Proceedings of the 27th ACM international conference on multimedia, pp 1632–1640
11.
go back to reference Triantafyllidou D, Moran S, McDonagh S, Parisot S, Slabaugh G (2020) Low light video enhancement using synthetic data produced with an intermediate domain mapping. In: European conference on computer vision. Springer, pp 103–119 Triantafyllidou D, Moran S, McDonagh S, Parisot S, Slabaugh G (2020) Low light video enhancement using synthetic data produced with an intermediate domain mapping. In: European conference on computer vision. Springer, pp 103–119
12.
go back to reference Yang W, Wang S, Fang Y, Wang Y, Liu J (2020) From fidelity to perceptual quality: A semi-supervised approach for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3063–3072 Yang W, Wang S, Fang Y, Wang Y, Liu J (2020) From fidelity to perceptual quality: A semi-supervised approach for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3063–3072
13.
go back to reference Jiang Y, Gong X, Liu D, Cheng Y, Fang C, Shen X, Yang J, Zhou P, Wang Z (2021) EnlightenGAN: deep light enhancement without paired supervision. IEEE Trans Image Process 30:2340–2349CrossRef Jiang Y, Gong X, Liu D, Cheng Y, Fang C, Shen X, Yang J, Zhou P, Wang Z (2021) EnlightenGAN: deep light enhancement without paired supervision. IEEE Trans Image Process 30:2340–2349CrossRef
14.
go back to reference Guo C, Li C, Guo J, Loy CC, Hou J, Kwong S, Cong R (2020) Zero-reference deep curve estimation for low-light image enhancement. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1780–1789 Guo C, Li C, Guo J, Loy CC, Hou J, Kwong S, Cong R (2020) Zero-reference deep curve estimation for low-light image enhancement. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1780–1789
15.
go back to reference Jaini P, Kobyzev I, Yu Y, Brubaker M (2020) Tails of Lipschitz triangular flows. In: International conference on machine learning. PMLR, pp 4673–4681 Jaini P, Kobyzev I, Yu Y, Brubaker M (2020) Tails of Lipschitz triangular flows. In: International conference on machine learning. PMLR, pp 4673–4681
18.
go back to reference Kingma DP, Dhariwal P (2018) Glow: generative flow with invertible 1x1 convolutions. Adv Neural Inf Process Syst 31 Kingma DP, Dhariwal P (2018) Glow: generative flow with invertible 1x1 convolutions. Adv Neural Inf Process Syst 31
19.
go back to reference Lugmayr A, Danelljan M, Gool LV, Timofte R (2020) Srflow: learning the super-resolution space with normalizing flow. In: European conference on computer vision. Springer, pp 715–732 Lugmayr A, Danelljan M, Gool LV, Timofte R (2020) Srflow: learning the super-resolution space with normalizing flow. In: European conference on computer vision. Springer, pp 715–732
20.
go back to reference Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241 Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241
21.
go back to reference He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
22.
go back to reference Wang H, Yao M, Jiang G, Mi Z, Fu X (2023) Graph-collaborated auto-encoder hashing for multi-view binary clustering. arXiv preprint arXiv:2301.02484 Wang H, Yao M, Jiang G, Mi Z, Fu X (2023) Graph-collaborated auto-encoder hashing for multi-view binary clustering. arXiv preprint arXiv:​2301.​02484
23.
go back to reference Wang H, Jiang G, Peng J, Deng R, Fu X (2022) Towards adaptive consensus graph: multi-view clustering via graph collaboration. IEEE Trans Multimedia Wang H, Jiang G, Peng J, Deng R, Fu X (2022) Towards adaptive consensus graph: multi-view clustering via graph collaboration. IEEE Trans Multimedia
24.
go back to reference Wang H, Wang Y, Zhang Z, Fu X, Zhuo L, Xu M, Wang M (2020) Kernelized multiview subspace analysis by self-weighted learning. IEEE Trans Multimedia 23:3828–3840CrossRef Wang H, Wang Y, Zhang Z, Fu X, Zhuo L, Xu M, Wang M (2020) Kernelized multiview subspace analysis by self-weighted learning. IEEE Trans Multimedia 23:3828–3840CrossRef
25.
go back to reference Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 10012–10022 Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 10012–10022
26.
go back to reference Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30 Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30
27.
go back to reference Wang Y, Wan R, Yang W, Li H, Chau L-P, Kot A (2022) Low-light image enhancement with normalizing flow. In: Proceedings of the AAAI conference on artificial intelligence, vol 36, pp 2604–2612 Wang Y, Wan R, Yang W, Li H, Chau L-P, Kot A (2022) Low-light image enhancement with normalizing flow. In: Proceedings of the AAAI conference on artificial intelligence, vol 36, pp 2604–2612
28.
go back to reference Zamir SW, Arora A, Khan S, Hayat M, Khan FS, Yang M-H, Shao L (2020) Learning enriched features for real image restoration and enhancement. In: European conference on computer vision. Springer, pp 492–511 Zamir SW, Arora A, Khan S, Hayat M, Khan FS, Yang M-H, Shao L (2020) Learning enriched features for real image restoration and enhancement. In: European conference on computer vision. Springer, pp 492–511
29.
go back to reference Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141 Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141
30.
go back to reference Woo S, Park J, Lee J-Y, Kweon, IS (2018) CBAM: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19 Woo S, Park J, Lee J-Y, Kweon, IS (2018) CBAM: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19
31.
go back to reference Ying Z, Li G, Ren Y, Wang R, Wang W (2017) A new low-light image enhancement algorithm using camera response model. In: The IEEE international conference on computer vision (ICCV) Ying Z, Li G, Ren Y, Wang R, Wang W (2017) A new low-light image enhancement algorithm using camera response model. In: The IEEE international conference on computer vision (ICCV)
32.
go back to reference Hu C-H, Yu J, Wu F, Zhang Y, Jing X-Y, Lu X-B, Liu P (2021) Face illumination recovery for the deep learning feature under severe illumination variations. Pattern Recognit 111:107724CrossRef Hu C-H, Yu J, Wu F, Zhang Y, Jing X-Y, Lu X-B, Liu P (2021) Face illumination recovery for the deep learning feature under severe illumination variations. Pattern Recognit 111:107724CrossRef
33.
go back to reference Risheng L, Long M, Jiaao Z, Xin F, Zhongxuan L (2021) Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. In: Proceedings of the IEEE conference on computer vision and pattern recognition Risheng L, Long M, Jiaao Z, Xin F, Zhongxuan L (2021) Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. In: Proceedings of the IEEE conference on computer vision and pattern recognition
34.
go back to reference Zhang Y, Guo X, Ma J, Liu W, Zhang J (2021) Beyond brightening low-light images. Int J Comput Vis 129(4):1013–1037CrossRef Zhang Y, Guo X, Ma J, Liu W, Zhang J (2021) Beyond brightening low-light images. Int J Comput Vis 129(4):1013–1037CrossRef
35.
go back to reference Xu X, Wang R, Fu C-W, Jia J (2022) SNR-aware low-light image enhancement. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 17714–17724 Xu X, Wang R, Fu C-W, Jia J (2022) SNR-aware low-light image enhancement. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 17714–17724
36.
go back to reference Yang W, Wang W, Huang H, Wang S, Liu J (2021) Sparse gradient regularized deep Retinex network for robust low-light image enhancement. IEEE Trans Image Process 30:2072–2086CrossRef Yang W, Wang W, Huang H, Wang S, Liu J (2021) Sparse gradient regularized deep Retinex network for robust low-light image enhancement. IEEE Trans Image Process 30:2072–2086CrossRef
37.
go back to reference Bychkovsky V, Paris S, Chan E, Durand F (2011) Learning photographic global tonal adjustment with a database of input/output image pairs. In: CVPR 2011. IEEE, pp 97–104 Bychkovsky V, Paris S, Chan E, Durand F (2011) Learning photographic global tonal adjustment with a database of input/output image pairs. In: CVPR 2011. IEEE, pp 97–104
38.
go back to reference Park J, Lee J-Y, Yoo D, Kweon IS (2018) Distort-and-recover: color enhancement using deep reinforcement learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5928–5936 Park J, Lee J-Y, Yoo D, Kweon IS (2018) Distort-and-recover: color enhancement using deep reinforcement learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5928–5936
39.
go back to reference Gharbi M, Chen J, Barron JT, Hasinoff SW, Durand F (2017) Deep bilateral learning for real-time image enhancement. ACM Trans Graph TOG 36(4):1–12CrossRef Gharbi M, Chen J, Barron JT, Hasinoff SW, Durand F (2017) Deep bilateral learning for real-time image enhancement. ACM Trans Graph TOG 36(4):1–12CrossRef
40.
go back to reference Chen Y-S, Wang Y-C, Kao M-H, Chuang Y-Y (2018) Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6306–6314 Chen Y-S, Wang Y-C, Kao M-H, Chuang Y-Y (2018) Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6306–6314
41.
go back to reference Wang R, Zhang Q, Fu C-W, Shen X, Zheng W-S, Jia J (2019) Underexposed photo enhancement using deep illumination estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6849–6857 Wang R, Zhang Q, Fu C-W, Shen X, Zheng W-S, Jia J (2019) Underexposed photo enhancement using deep illumination estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6849–6857
42.
go back to reference Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612CrossRef Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612CrossRef
43.
go back to reference Loh YP, Chan CS (2019) Getting to know low-light images with the exclusively dark dataset. Comput Vis Image Underst 178:30–42CrossRef Loh YP, Chan CS (2019) Getting to know low-light images with the exclusively dark dataset. Comput Vis Image Underst 178:30–42CrossRef
Metadata
Title
Flow Learning Based Dual Networks for Low-Light Image Enhancement
Authors
Siyu Wang
Changhui Hu
Weilin Yi
Ziyun Cai
Mingliang Zhai
Wankou Yang
Publication date
05-06-2023
Publisher
Springer US
Published in
Neural Processing Letters / Issue 6/2023
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-023-11303-3

Other articles of this Issue 6/2023

Neural Processing Letters 6/2023 Go to the issue