Skip to main content
Top

2021 | OriginalPaper | Chapter

Multi-level Fusion Based Deep Convolutional Network for Image Quality Assessment

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Image quality assessment aims to design effective models to automatically predict the perceptual quality score of a given image that is consistent with human cognition. In this paper, we propose a novel end-to-end multi-level fusion based deep convolutional neural network for full-reference image quality assessment (FR-IQA), codenamed MF-IQA. In MF-IQA, we first extract features with the help of edge feature fusion for both distorted images and the corresponding reference images. Afterwards, we apply multi-level feature fusion to evaluate a number of local quality indices, and then they would be pooled into a global quality score. With the proposed multi-level fusion and edge feature fusion strategy, the input images and the corresponding feature maps can be better learned and thus help produce more accurate and meaningful visual perceptual predictions. The experimental results and statistical comparisons on three IQA datasets demonstrate that our framework achieves the state-of-the-art prediction accuracy in contrast to most existing algorithms.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Bosse, S., Maniry, D., Müller, K.R., Wiegand, T., Samek, W.: Deep neural networks for no-reference and full-reference image quality assessment. IEEE TIP (2017) Bosse, S., Maniry, D., Müller, K.R., Wiegand, T., Samek, W.: Deep neural networks for no-reference and full-reference image quality assessment. IEEE TIP (2017)
2.
go back to reference Bosse, S., Siekmann, M., Samek, W., Wiegand, T.: A perceptually relevant shearlet-based adaptation of the PSNR. In: IEEE ICIP (2017) Bosse, S., Siekmann, M., Samek, W., Wiegand, T.: A perceptually relevant shearlet-based adaptation of the PSNR. In: IEEE ICIP (2017)
3.
go back to reference Canny, J.: A computational approach to edge detection. IEEE PAMI 8, 679–698 (1986)CrossRef Canny, J.: A computational approach to edge detection. IEEE PAMI 8, 679–698 (1986)CrossRef
5.
go back to reference Ding, K., Ma, K., Wang, S., Simoncelli, E.P.: Image quality assessment: unifying structure and texture similarity. arXiv preprint arXiv:2004.07728 (2020) Ding, K., Ma, K., Wang, S., Simoncelli, E.P.: Image quality assessment: unifying structure and texture similarity. arXiv preprint arXiv:​2004.​07728 (2020)
6.
go back to reference Gao, F., Wang, Y., Li, P., Tan, M., Yu, J., Zhu, Y.: Deepsim: deep similarity for image quality assessment. Neurocomputing (2017) Gao, F., Wang, Y., Li, P., Tan, M., Yu, J., Zhu, Y.: Deepsim: deep similarity for image quality assessment. Neurocomputing (2017)
7.
go back to reference Girod, B.: What’s wrong with mean-squared error? Digital images and human vision (1993) Girod, B.: What’s wrong with mean-squared error? Digital images and human vision (1993)
8.
go back to reference Larson, E.C., Chandler, D.M.: Most apparent distortion: full-reference image quality assessment and the role of strategy. SPIE IS&T J. Electron. Imaging (2010) Larson, E.C., Chandler, D.M.: Most apparent distortion: full-reference image quality assessment and the role of strategy. SPIE IS&T J. Electron. Imaging (2010)
10.
go back to reference Ponomarenko, N., et al.: Color image database tid2013: peculiarities and preliminary results. In: IEEE EUVIP (2013) Ponomarenko, N., et al.: Color image database tid2013: peculiarities and preliminary results. In: IEEE EUVIP (2013)
11.
go back to reference Prashnani, E., Cai, H., Mostofi, Y., Sen, P.: Pieapp: perceptual image-error assessment through pairwise preference. In: IEEE CVPR (2018) Prashnani, E., Cai, H., Mostofi, Y., Sen, P.: Pieapp: perceptual image-error assessment through pairwise preference. In: IEEE CVPR (2018)
12.
go back to reference Sara, U., Akter, M., Uddin, M.S.: Image quality assessment through FSIM, SSIM, MSE and PSNR-a comparative study. JCC 07, 8–18 (2019)CrossRef Sara, U., Akter, M., Uddin, M.S.: Image quality assessment through FSIM, SSIM, MSE and PSNR-a comparative study. JCC 07, 8–18 (2019)CrossRef
13.
go back to reference Sheikh, H.R., Bovik, A.C.: Image information and visual quality. IEEE TIP 15, 430–444 (2006) Sheikh, H.R., Bovik, A.C.: Image information and visual quality. IEEE TIP 15, 430–444 (2006)
14.
go back to reference Sheikh, H.R., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE TIP 15, 3440–3451 (2006) Sheikh, H.R., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE TIP 15, 3440–3451 (2006)
15.
go back to reference Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014) Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:​1409.​1556 (2014)
16.
go back to reference Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE TIP 13, 600–612 (2004) Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE TIP 13, 600–612 (2004)
17.
go back to reference Wang, Z., Li, Q.: Information content weighting for perceptual image quality assessment. IEEE TIP 20, 1185–1198 (2010)MathSciNetMATH Wang, Z., Li, Q.: Information content weighting for perceptual image quality assessment. IEEE TIP 20, 1185–1198 (2010)MathSciNetMATH
18.
go back to reference Xue, W., Zhang, L., Mou, X., Bovik, A.C.: Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE TIP 23, 684–695 (2013)MathSciNetMATH Xue, W., Zhang, L., Mou, X., Bovik, A.C.: Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE TIP 23, 684–695 (2013)MathSciNetMATH
19.
go back to reference Zhang, L., Li, H.: Sr-sim: A fast and high performance IQA index based on spectral residual. In: IEEE ICIP (2012) Zhang, L., Li, H.: Sr-sim: A fast and high performance IQA index based on spectral residual. In: IEEE ICIP (2012)
20.
go back to reference Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE TIP 20, 2378–2386 (2011)MathSciNetMATH Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE TIP 20, 2378–2386 (2011)MathSciNetMATH
Metadata
Title
Multi-level Fusion Based Deep Convolutional Network for Image Quality Assessment
Authors
Qianyu Guo
Jing Wen
Copyright Year
2021
DOI
https://doi.org/10.1007/978-3-030-68780-9_51

Premium Partner