Skip to main content
SpringerLink
Log in

Composition-preserving deep approach to full-reference image quality assessment

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Image quality assessment is an important element of a broad spectrum of applications ranging from automatic video streaming to display technology. In this study, we present a novel full-reference image quality assessment algorithm relying on a Siamese layout of pretrained convolutional neural networks (CNNs), feature pooling, and a neural network. Unlike previous methods, our algorithm handles input images without resizing, cropping, or any modifications. As a consequence, it effectively learns the fine-grained, quality-aware features of images. The proposed model derives its core performance from pretrained CNNs, being trained at a higher resolution than that in previous works. The presented architecture was trained on the recently published KADID-10k, which is the largest image quality database and contains 10,125 digital images. Experimental results on KADID-10k demonstrate that the proposed method outperforms other state-of-the-art algorithms. These results are also confirmed with cross-database tests using other publicly available datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Bae, S.H., Kim, M.: A novel image quality assessment with globally and locally consilient visual quality perception. IEEE Trans. Image Process. 25(5), 2392–2406 (2016)

    MathSciNet  MATH  Google Scholar 

  2. Bianco, S., Celona, L., Napoletano, P., Schettini, R.: On the use of deep learning for blind image quality assessment. SIViP 12(2), 355–362 (2018)

    Google Scholar 

  3. Bosse, S., Maniry, D., Müller, K.R., Wiegand, T., Samek, W.: Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans. Image Process. 27(1), 206–219 (2017)

    MathSciNet  MATH  Google Scholar 

  4. Chang, H.W., Yang, H., Gan, Y., Wang, M.H.: Sparse feature fidelity for perceptual image quality assessment. IEEE Trans. Image Process. 22(10), 4007–4018 (2013)

    MathSciNet  MATH  Google Scholar 

  5. Chou, C.H., Li, Y.C.: A perceptually tuned subband image coder based on the measure of just-noticeable-distortion profile. IEEE Trans. Circuits Syst. Video Technol. 5(6), 467–476 (1995)

    Google Scholar 

  6. Daly, S.J.: Visible differences predictor: an algorithm for the assessment of image fidelity. In: Human Vision, Visual Processing, and Digital Display III, vol. 1666, pp. 2–16. International Society for Optics and Photonics (1992)

  7. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

  8. Dosselmann, R., Yang, X.D.: A comprehensive assessment of the structural similarity index. SIViP 5(1), 81–91 (2011)

    Google Scholar 

  9. Gao, F., Wang, Y., Li, P., Tan, M., Yu, J., Zhu, Y.: Deepsim: deep similarity for image quality assessment. Neurocomputing 257, 104–114 (2017)

    Google Scholar 

  10. Gu, K., Wang, S., Zhai, G., Lin, W., Yang, X., Zhang, W.: Analysis of distortion distribution for pooling in image quality prediction. IEEE Trans. Broadcast. 62(2), 446–456 (2016)

    Google Scholar 

  11. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  12. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)

  13. Kang, L., Ye, P., Li, Y., Doermann, D.: Convolutional neural networks for no-reference image quality assessment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1733–1740 (2014)

  14. Kim, J., Lee, S.: Deep learning of human visual sensitivity in image quality assessment framework. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1676–1684 (2017)

  15. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  16. Kolaman, A., Yadid-Pecht, O.: Quaternion structural similarity: a new quality index for color images. IEEE Trans. Image Process. 21(4), 1526–1536 (2011)

    MathSciNet  MATH  Google Scholar 

  17. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

  18. Larson, E.C., Chandler, D.M.: Most apparent distortion: full-reference image quality assessment and the role of strategy. J. Electron. Imaging 19(1), 011006 (2010)

    Google Scholar 

  19. Li, C., Bovik, A.C.: Three-component weighted structural similarity index. In: Image Quality and System Performance VI, vol. 7242, p. 72420Q. International Society for Optics and Photonics (2009)

  20. Li, J., Zou, L., Yan, J., Deng, D., Qu, T., Xie, G.: No-reference image quality assessment using prewitt magnitude based on convolutional neural networks. SIViP 10(4), 609–616 (2016)

    Google Scholar 

  21. Liang, Y., Wang, J., Wan, X., Gong, Y., Zheng, N.: Image quality assessment using similar scene as reference. In: European Conference on Computer Vision, pp. 3–18. Springer, Berlin (2016)

  22. Lin, H., Hosu, V., Saupe, D.: Kadid-10k: A large-scale artificially distorted iqa database. In: 2019 Tenth International Conference on Quality of Multimedia Experience (QoMEX), pp. 1–3. IEEE (2019)

  23. Liu, A., Lin, W., Narwaria, M.: Image quality assessment based on gradient similarity. IEEE Trans. Image Process. 21(4), 1500–1512 (2011)

    MathSciNet  MATH  Google Scholar 

  24. Ma, L., Li, S., Ngan, K.N.: Reduced-reference image quality assessment in reorganized dct domain. Sig. Process. Image Commun. 28(8), 884–902 (2013)

    Google Scholar 

  25. Moorthy, A.K., Bovik, A.C.: A two-step framework for constructing blind image quality indices. IEEE Signal Process. Lett. 17(5), 513–516 (2010)

    Google Scholar 

  26. Nafchi, H.Z., Shahkolaei, A., Hedjam, R., Cheriet, M.: Mean deviation similarity index: efficient and reliable full-reference image quality evaluator. IEEE Access 4, 5579–5590 (2016)

    Google Scholar 

  27. Ponomarenko, N., Jin, L., Ieremeiev, O., Lukin, V., Egiazarian, K., Astola, J., Vozel, B., Chehdi, K., Carli, M., Battisti, F., et al.: Image database TID2013: peculiarities, results and perspectives. Sig. Process. Image Commun. 30, 57–77 (2015)

    Google Scholar 

  28. Ponomarenko, N., Lukin, V., Zelensky, A., Egiazarian, K., Carli, M., Battisti, F.: TID2008-a database for evaluation of full-reference visual quality assessment metrics. Adv. Mod. Radioelectron. 10(4), 30–45 (2009)

    Google Scholar 

  29. Sampat, M.P., Wang, Z., Gupta, S., Bovik, A.C., Markey, M.K.: Complex wavelet structural similarity: a new image similarity index. IEEE Trans. Image Process. 18(11), 2385–2401 (2009)

    MathSciNet  MATH  Google Scholar 

  30. Sheikh, H.R., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15(11), 3440–3451 (2006)

    Google Scholar 

  31. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  32. Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)

  33. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)

  34. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Google Scholar 

  35. Wang, Z., Li, Q.: Information content weighting for perceptual image quality assessment. IEEE Trans. Image Process. 20(5), 1185–1198 (2010)

    MathSciNet  MATH  Google Scholar 

  36. Wang, Z., Simoncelli, E.P.: Reduced-reference image quality assessment using a wavelet-domain natural image statistic model. In: Human Vision and Electronic Imaging X, vol. 5666, pp. 149–160. International Society for Optics and Photonics (2005)

  37. Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: The Thrity-Seventh Asilomar Conference on Signals, Systems and Computers, 2003, vol. 2, pp. 1398–1402. IEEE (2003)

  38. Watson, A.B., Borthwick, R., Taylor, M.: Image quality and entropy masking. In: Human Vision and Electronic Imaging II, vol. 3016, pp. 2–13. International Society for Optics and Photonics (1997)

  39. Xue, W., Zhang, L., Mou, X., Bovik, A.C.: Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE Trans. Image Process. 23(2), 684–695 (2013)

    MathSciNet  MATH  Google Scholar 

  40. Zhang, L., Shen, Y., Li, H.: VSI: a visual saliency-induced index for perceptual image quality assessment. IEEE Trans. Image Process. 23(10), 4270–4281 (2014)

    MathSciNet  MATH  Google Scholar 

  41. Zhang, L., Zhang, L., Mou, X.: RFSIM: a feature based image quality assessment metric using Riesz transforms. In: 2010 IEEE International Conference on Image Processing, pp. 321–324. IEEE (2010)

  42. Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20(8), 2378–2386 (2011)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The author thanks the anonymous reviewers whose comments have greatly improved this manuscript.

Author information

Authors and Affiliations

  1. Department of Networked Systems and Services, Budapest University of Technology and Economics, Budapest, Hungary

    Domonkos Varga

Authors
  1. Domonkos Varga
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Domonkos Varga.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Varga, D. Composition-preserving deep approach to full-reference image quality assessment. SIViP 14, 1265–1272 (2020). https://doi.org/10.1007/s11760-020-01664-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-020-01664-w

Keywords

Search

Navigation