1 Introduction
-
A GAN-based unsupervised image fusion algorithm for fusing extreme-exposure images is proposed. The adversarial relationship enables fused image to have more details from source images.
-
Different from FusionGAN, a novel structure of GAN is developed, which is more suitable for the task of MEF.
-
We design a new loss function for MEF which can help fused image to preserve more information from source images.
-
We construct a new training dataset which contains all kinds of conditions. This dataset could enhance the robustness of our method.
2 Related works
2.1 Fusion methods based on deep learning
2.2 The basic theory of GAN
2.3 Variants of GAN and their applications
3 Proposed method
3.1 GANFuse
3.2 Loss function
3.3 Network structure
3.3.1 Generator
3.3.2 Discriminator
3.4 Training
3.5 Testing
4 Experiments
4.1 Comparison methods
4.2 Qualitative comparisons
4.3 Quantitative comparisons
4.3.1 Standard deviation (SD)
4.3.2 Peak signal-to-noise ratio (PSNR)
4.3.3 Correlation coefficient (CC)
4.3.4 Mean structural similarity (MSSIM)
4.3.5 The sum of the correlations of differences (SCD)
Metrics | SD | PSNR | SSIM | CC | SCD |
---|---|---|---|---|---|
GANFuse | 0.148115426 | 57.33800149 | 1.965263834 | 0.850954307 | 1.584032931 |
GBM | 0.13316501 | 57.28301727 | 1.96499124 | 0.792732864 | 1.031510881 |
DeepFuse | 0.147293475 | 57.30245913 | 1.964967841 | 0.847856712 | 1.56252029 |
GFF | 0.196556506 | 55.59830182 | 1.949333007 | 0.072599073 | − 0.398211353 |
FLER | 0.121066135 | 56.74671019 | 1.960781543 | 0.344774143 | − 0.211503979 |
DSIFT | 0.175588204 | 55.88504831 | 1.952625491 | 0.094026786 | − 0.417428915 |
4.4 Comparative experiment
Metrics | SD | PSNR | SSIM | CC | SCD |
---|---|---|---|---|---|
GANFuse | 0.148115426 | 57.33800149 | 1.965263834 | 0.850954307 | 1.584032931 |
The comparative experiment 1 | 0.148030308 | 57.30291199 | 1.964888646 | 0.850155029 | 1.574653232 |
The comparative experiment 2 | 0.139084259 | 57.30443939 | 1.964849778 | 0.846627754 | 1.449303726 |
Metrics | SD | PSNR | SSIM | CC | SCD |
---|---|---|---|---|---|
GANFuse | 0.148115426 | 57.33800149 | 1.965263834 | 0.850954307 | 1.584032931 |
The comparative experiment 3 | 0.147009365 | 57.33418144 | 1.965185013 | 0.850091644 | 1.565150411 |
The comparative experiment 4 | 0.14853375 | 57.33491155 | 1.965202621 | 0.849739465 | 1.579227282 |