Skip to main content
Top
Published in: Arabian Journal for Science and Engineering 8/2022

27-01-2022 | Research Article-Computer Engineering and Computer Science

Infrared-visible Image Fusion Using Accelerated Convergent Convolutional Dictionary Learning

Authors: Chengfang Zhang, Ziliang Feng

Published in: Arabian Journal for Science and Engineering | Issue 8/2022

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Techniques for the fusion of infrared and visible images have gradually become a popular research topic in the field of computer vision. In our paper, accelerated convergent convolutional dictionary learning (CDL) is first introduced for infrared-visible image fusion. The proposed method combines the advantages of CDL and convolutional sparse representation (CSR) while also compensating for model mismatches between the training and fusion stages. Each image is decomposed into a base layer and a detail layer, for which different fusion strategies are used. Unlike previous CSR/CDL-based fusion methods, we introduce a practical and convergent Fast Block Proximal Gradient Using a Diagonal Majorizer (FBPG-M) method with two-block and multiblock schemes into the detail layer. Influenced by various imaging mechanisms, an ‘averaging’ fusion strategy is used for the base layer. Our method is evaluated and compared qualitatively and quantitatively with five typical fusion methods on 10 public datasets. The model is both subjectively and objectively evaluated, and the results show that the proposed method achieves notable success in terms of preserving details and focusing on targets.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Wang, Zhishe; Xu, Jiawei; Jiang,: Xiaolin and Yan, Xiaomei.: Infrared and visible image fusion via hybrid decomposition of NSCT and morphological sequential toggle operator. Optik International Journal for Light Electron Optics 201,(2019) Wang, Zhishe; Xu, Jiawei; Jiang,: Xiaolin and Yan, Xiaomei.: Infrared and visible image fusion via hybrid decomposition of NSCT and morphological sequential toggle operator. Optik International Journal for Light Electron Optics 201,(2019)
2.
go back to reference Duan, Chaowei; Wang, Zhisheng; Xing, Changda; Lu, Shanshan: Infrared and visible image fusion using multi-scale edge-preserving decomposition and multiple saliency features. Optik 228,(2021) Duan, Chaowei; Wang, Zhisheng; Xing, Changda; Lu, Shanshan: Infrared and visible image fusion using multi-scale edge-preserving decomposition and multiple saliency features. Optik 228,(2021)
3.
go back to reference Chen, Jun; Li, Xuejiao; Luo, Linbo; Mei, Xiaoguang; Ma, Jiayi: Infrared and visible image fusion based on target-enhanced multiscale transform decomposition. Information Sciences 508, 64–78 (2020)CrossRef Chen, Jun; Li, Xuejiao; Luo, Linbo; Mei, Xiaoguang; Ma, Jiayi: Infrared and visible image fusion based on target-enhanced multiscale transform decomposition. Information Sciences 508, 64–78 (2020)CrossRef
4.
go back to reference Ma, Jiayi; Chen, Chen; Li, Chang; Huang, Jun: Infrared and visible image fusion via gradient transfer and total variation minimization. Information Fusion 31, 100–109 (2016)CrossRef Ma, Jiayi; Chen, Chen; Li, Chang; Huang, Jun: Infrared and visible image fusion via gradient transfer and total variation minimization. Information Fusion 31, 100–109 (2016)CrossRef
5.
go back to reference Dai, Liyang; Liu, Gang; Huang, Lei; Xiao, Gang; Xu, Zhao; Ruan, Junjin: Feature transfer method for infrared and visible image fusion via fuzzy lifting scheme. Infrared Physics & Technology (2021) Dai, Liyang; Liu, Gang; Huang, Lei; Xiao, Gang; Xu, Zhao; Ruan, Junjin: Feature transfer method for infrared and visible image fusion via fuzzy lifting scheme. Infrared Physics & Technology (2021)
6.
go back to reference Qilei Li,Wei Wu,Lu Lu,Zuoyong Li,Awais Ahmad ,Gwanggil Jeon.: Infrared and visible images fusion by using sparse representation and guided filter, Journal of Intelligent Transportation Systems, 254-263, (2020) Qilei Li,Wei Wu,Lu Lu,Zuoyong Li,Awais Ahmad ,Gwanggil Jeon.: Infrared and visible images fusion by using sparse representation and guided filter, Journal of Intelligent Transportation Systems, 254-263, (2020)
7.
go back to reference Yin, Haitao; Li, Shutao: Multimodal image fusion with joint sparsity model. Optical Engineering 50,(2011) Yin, Haitao; Li, Shutao: Multimodal image fusion with joint sparsity model. Optical Engineering 50,(2011)
8.
go back to reference Visual attention guided image fusion with sparse representation: Yang, Bin and Li, Shutao. Optik - International Journal for Light and Electron Optics 125, 4881–4888 (2014) Visual attention guided image fusion with sparse representation: Yang, Bin and Li, Shutao. Optik - International Journal for Light and Electron Optics 125, 4881–4888 (2014)
9.
go back to reference Gao, Zhisheng; Zhang, Chengfang: Texture clear multi-modal image fusion with joint sparsity model. Optik - International Journal for Light and Electron Optics 121305, S0030402616310725 (2016) Gao, Zhisheng; Zhang, Chengfang: Texture clear multi-modal image fusion with joint sparsity model. Optik - International Journal for Light and Electron Optics 121305, S0030402616310725 (2016)
10.
go back to reference Zhang, C.F.; Yi, L.Z.: Multimodal image fusion with adaptive joint sparsity model. J. Electron. Imaging 28, 013043 (2019) Zhang, C.F.; Yi, L.Z.: Multimodal image fusion with adaptive joint sparsity model. J. Electron. Imaging 28, 013043 (2019)
11.
go back to reference Chengfang Zhang, Ziliang Feng, Zhisheng Gao, Xin Jin, Dan Yan, Liangzhong Yi.: Salient feature multimodal image fusion with a joint sparse model and multiscale dictionary learning, Opt. Eng, vol. 59, pp.051402,(2020) Chengfang Zhang, Ziliang Feng, Zhisheng Gao, Xin Jin, Dan Yan, Liangzhong Yi.: Salient feature multimodal image fusion with a joint sparse model and multiscale dictionary learning, Opt. Eng, vol. 59, pp.051402,(2020)
12.
go back to reference Liu, Y.; Wang, Z.: Simultaneous image fusion and denoising with adaptive sparse representation. IET Image Processing 9(5), 347–357 (2015)CrossRef Liu, Y.; Wang, Z.: Simultaneous image fusion and denoising with adaptive sparse representation. IET Image Processing 9(5), 347–357 (2015)CrossRef
13.
go back to reference Wohlberg, B.: Efficient algorithms for convolutional sparse representations. IEEE Trans. Image Process 25, 301–315 (2016)MathSciNetCrossRef Wohlberg, B.: Efficient algorithms for convolutional sparse representations. IEEE Trans. Image Process 25, 301–315 (2016)MathSciNetCrossRef
14.
go back to reference Garcia-Cardona, Cristina; Wohlberg, Brendt: Convolutional Dictionary Learning: A Comparative Review and New Algorithms, IEEE Transactions on Computational. Imaging 4(3), 366–381 (2018)MathSciNet Garcia-Cardona, Cristina; Wohlberg, Brendt: Convolutional Dictionary Learning: A Comparative Review and New Algorithms, IEEE Transactions on Computational. Imaging 4(3), 366–381 (2018)MathSciNet
15.
go back to reference Il and Yong, Jeffrey A and Fessler.: Convolutional Dictionary Learning: Acceleration and Convergence. IEEE Transactions on Image Processing 27(4), 1697–1712 (2018) Il and Yong, Jeffrey A and Fessler.: Convolutional Dictionary Learning: Acceleration and Convergence. IEEE Transactions on Image Processing 27(4), 1697–1712 (2018)
16.
go back to reference Chun, Il Yong; Fessler, Jeffrey A.: Convergent convolutional dictionary learning using Adaptive Contrast Enhancement (CDL-ACE): Application of CDL to image denoising,2017 International Conference on Sampling Theory and Applications (SampTA), (2017) Chun, Il Yong; Fessler, Jeffrey A.: Convergent convolutional dictionary learning using Adaptive Contrast Enhancement (CDL-ACE): Application of CDL to image denoising,2017 International Conference on Sampling Theory and Applications (SampTA), (2017)
17.
go back to reference Liu, Y.; Chen, X.; Ward, R.K.: Image fusion with convolutional sparse representation. IEEE Signal Process. Lett.vol. 23, 1882–1886 (2016)CrossRef Liu, Y.; Chen, X.; Ward, R.K.: Image fusion with convolutional sparse representation. IEEE Signal Process. Lett.vol. 23, 1882–1886 (2016)CrossRef
18.
go back to reference Wu, Minghui; Ma, Yong; Fan, Fan; Mei, Xiaoguang; Huang, Jun: Infrared and visible image fusion via joint convolutional sparse representation. Journal of the Optical Society of America A.vol. 37(7), 1105–1115 (2020)CrossRef Wu, Minghui; Ma, Yong; Fan, Fan; Mei, Xiaoguang; Huang, Jun: Infrared and visible image fusion via joint convolutional sparse representation. Journal of the Optical Society of America A.vol. 37(7), 1105–1115 (2020)CrossRef
19.
go back to reference Chengrui, Gao; Liu, Feiqiang; Hua, Yan: Infrared and visible image fusion using dual-tree complex wavelet transform and convolutional sparse representation. Journal of Intelligent & Fuzzy Systems 39(3), 4617–4629 (2020)CrossRef Chengrui, Gao; Liu, Feiqiang; Hua, Yan: Infrared and visible image fusion using dual-tree complex wavelet transform and convolutional sparse representation. Journal of Intelligent & Fuzzy Systems 39(3), 4617–4629 (2020)CrossRef
20.
go back to reference Shao, Luling; Wu, Jin; Wu, Minghui: Infrared and Visible Image Fusion Based on Spatial Convolution Sparse representation. Journal of Physics: Conference Series 1634,(2020) Shao, Luling; Wu, Jin; Wu, Minghui: Infrared and Visible Image Fusion Based on Spatial Convolution Sparse representation. Journal of Physics: Conference Series 1634,(2020)
21.
go back to reference Jian, W.; Chunxia, Q.; Xiufei, Z.; Ke, Y.; Ping, R.: A multi-source image fusion algorithm based on gradient regularized convolution sparse representation. Journal of Systems Engineering and Electronics 31(3), 447–459 (2020)CrossRef Jian, W.; Chunxia, Q.; Xiufei, Z.; Ke, Y.; Ping, R.: A multi-source image fusion algorithm based on gradient regularized convolution sparse representation. Journal of Systems Engineering and Electronics 31(3), 447–459 (2020)CrossRef
22.
go back to reference Li, by Linguo., Tan, Ling., Li, Shujing., Ye, Qing.: Image fusion based on convolution sparse representation and pulse coupled neural network in non-subsampled contourlet domain. International Journal of Embedded Systems 12(1), 447–459 (2020) Li, by Linguo., Tan, Ling., Li, Shujing., Ye, Qing.: Image fusion based on convolution sparse representation and pulse coupled neural network in non-subsampled contourlet domain. International Journal of Embedded Systems 12(1), 447–459 (2020)
23.
go back to reference Zhang, Chengfang.: Convolutional dictionary learning using global matching tracking (CDL-GMT) Application to visible-infrared image fusion,4th International Conference on Mechatronics and Intelligent Robotics,(2021) Zhang, Chengfang.: Convolutional dictionary learning using global matching tracking (CDL-GMT) Application to visible-infrared image fusion,4th International Conference on Mechatronics and Intelligent Robotics,(2021)
24.
go back to reference Li, H.; Wu, X.J.; Kittler, J.: Infrared and Visible Image Fusion using a Deep Learning Framework,24rd International Conference on. IEEE,pp.2705–2710,(2018) Li, H.; Wu, X.J.; Kittler, J.: Infrared and Visible Image Fusion using a Deep Learning Framework,24rd International Conference on. IEEE,pp.2705–2710,(2018)
25.
go back to reference Liu, Yu and Chen, Xun and Cheng, Juan and Peng, Hu and Wang, Zengfu.: Infrared and visible image fusion with convolutional neural networks,International Journal of Wavelets, Multiresolution and Information Processing,(2018) Liu, Yu and Chen, Xun and Cheng, Juan and Peng, Hu and Wang, Zengfu.: Infrared and visible image fusion with convolutional neural networks,International Journal of Wavelets, Multiresolution and Information Processing,(2018)
26.
go back to reference Ma, Jiayi., Yu, Wei., Liang, Pengwei., Li, Chang., and Jiang, Junjun.: FusionGAN: A generative adversarial network for infrared and visible image fusion,Information Fusion,48,11-26, (2019) Ma, Jiayi., Yu, Wei., Liang, Pengwei., Li, Chang., and Jiang, Junjun.: FusionGAN: A generative adversarial network for infrared and visible image fusion,Information Fusion,48,11-26, (2019)
27.
go back to reference Xydeas, C.S.; P. V.V.: Objective image fusion performance measure. Military Technical. Courier 56(4), 181–193 (2000) Xydeas, C.S.; P. V.V.: Objective image fusion performance measure. Military Technical. Courier 56(4), 181–193 (2000)
28.
go back to reference Piella, Gemma; Heijmans, H.: A new quality metric for image fusion. International Conference on Image Processing IEEE, (2003) Piella, Gemma; Heijmans, H.: A new quality metric for image fusion. International Conference on Image Processing IEEE, (2003)
29.
go back to reference Zhao, Jiying., Laganiere, R., and Liu, Z.: Performance assessment of combinative pixel-level image fusion based on an absolute feature measurement. International Journal of Innovative Computing Information Control Ijicic 3.6(2006) Zhao, Jiying., Laganiere, R., and Liu, Z.: Performance assessment of combinative pixel-level image fusion based on an absolute feature measurement. International Journal of Innovative Computing Information Control Ijicic 3.6(2006)
30.
go back to reference Chen, Y.; Blum, R.S.: A New Automated Quality Assessment Algorithm for Image Fusion. Image and Vision Computing 27, 1421–1432 (2009)CrossRef Chen, Y.; Blum, R.S.: A New Automated Quality Assessment Algorithm for Image Fusion. Image and Vision Computing 27, 1421–1432 (2009)CrossRef
31.
go back to reference Liu, Z.; et al.: Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative study. IEEE Trans. Pattern Anal. Machine Intell. 34(1), 94–109 (2011)CrossRef Liu, Z.; et al.: Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative study. IEEE Trans. Pattern Anal. Machine Intell. 34(1), 94–109 (2011)CrossRef
32.
go back to reference Heide, F.; Heidrich, W.; Wetzstein, G.: Fast and flexible convolutional sparse coding, in Proc, pp. 5135–5143. IEEE CVPR, Boston, MA (2015) Heide, F.; Heidrich, W.; Wetzstein, G.: Fast and flexible convolutional sparse coding, in Proc, pp. 5135–5143. IEEE CVPR, Boston, MA (2015)
33.
go back to reference Zeiler, M.D.; Krishnan, D.; Taylor, G.W.; Fergus, R.: Deconvolutional networks, in Proc, pp. 2528–2535. IEEE CVPR, San Francisco, CA (2010) Zeiler, M.D.; Krishnan, D.; Taylor, G.W.; Fergus, R.: Deconvolutional networks, in Proc, pp. 2528–2535. IEEE CVPR, San Francisco, CA (2010)
37.
go back to reference Li, S.; Kang, X.; Hu, J.: Image Fusion with Guided Filtering. IEEE Transactions on Image Processing 22(7), 2864–2875 (2013)CrossRef Li, S.; Kang, X.; Hu, J.: Image Fusion with Guided Filtering. IEEE Transactions on Image Processing 22(7), 2864–2875 (2013)CrossRef
Metadata
Title
Infrared-visible Image Fusion Using Accelerated Convergent Convolutional Dictionary Learning
Authors
Chengfang Zhang
Ziliang Feng
Publication date
27-01-2022
Publisher
Springer Berlin Heidelberg
Published in
Arabian Journal for Science and Engineering / Issue 8/2022
Print ISSN: 2193-567X
Electronic ISSN: 2191-4281
DOI
https://doi.org/10.1007/s13369-021-06380-2

Other articles of this Issue 8/2022

Arabian Journal for Science and Engineering 8/2022 Go to the issue

Research Article-Computer Engineering and Computer Science

Optimization of Product Switching Processes in Assembly Lines

Research Article-Computer Engineering and Computer Science

Disposition-Based Concept Drift Detection and Adaptation in Data Stream

Research Article-Computer Engineering and Computer Science

Learning Deep Pyramid-based Representations for Pansharpening

Premium Partners