Skip to main content

2020 | OriginalPaper | Buchkapitel

Blind 3D Image Quality Assessment Based on Multi-scale Feature Learning

verfasst von : Yongfang Wang, Shuai Yuan, Yumeng Xia, Ping An

Erschienen in: Digital TV and Wireless Multimedia Communication

Verlag: Springer Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

3D image quality assessment (3D-IQA) plays an important role in 3D multimedia applications. In recent years, convolutional neural networks (CNN) have been widely used in various images processing tasks and achieve excellent performance. In this paper, we propose a blind 3D-IQA metric based on multi-scale feature learning by using multi-column convolutional neural networks (3D-IQA-MCNN). To address the problem of limited 3D-IQA dataset size, we take patches from the left image and right image as input and use the full-reference (FR) IQA metric to approximate a reference ground-truth for training the 3D-IQA-MCNN. Then we put the patches from left image and right image into the pre-trained 3D-IQA-MCNN and obtain two quality feature vectors based on multi-scale. Finally, by regressing the quality feature vectors onto the subjective mean opinion score (MOS), the visual quality of 3D images is predicted. Experimental results show that the proposed method achieves high consistency with human subjective assessment and outperforms several state-of-the-art 3D-IQA methods.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Chen, L., Zhao, J.: Robust contourlet-based watermarking for depth-image-based rendering 3D images. In: 2016 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Nara, pp. 1–4 (2016) Chen, L., Zhao, J.: Robust contourlet-based watermarking for depth-image-based rendering 3D images. In: 2016 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Nara, pp. 1–4 (2016)
2.
Zurück zum Zitat Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRef Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRef
3.
Zurück zum Zitat You, J., et al.: Perceptual quality assessment for stereoscopic images based on 2D image quality metrics and disparity analysis. In: Proceedings of the International Workshop Video Processing Quality Metrics Consumer Electronics, pp. 1–6 (2010) You, J., et al.: Perceptual quality assessment for stereoscopic images based on 2D image quality metrics and disparity analysis. In: Proceedings of the International Workshop Video Processing Quality Metrics Consumer Electronics, pp. 1–6 (2010)
4.
Zurück zum Zitat Benoit, A., et al.: Quality assessment of stereoscopic images. EURASIP J. Image Video Process. 2008, 1–13 (2009)CrossRef Benoit, A., et al.: Quality assessment of stereoscopic images. EURASIP J. Image Video Process. 2008, 1–13 (2009)CrossRef
5.
Zurück zum Zitat Tam, W.J., Speranza, F., Yano, S., Shimono, K., Ono, H.: Stereoscopic 3D-TV: visual comfort. IEEE Trans. Broadcast. 57(2), 335–346 (2011)CrossRef Tam, W.J., Speranza, F., Yano, S., Shimono, K., Ono, H.: Stereoscopic 3D-TV: visual comfort. IEEE Trans. Broadcast. 57(2), 335–346 (2011)CrossRef
6.
Zurück zum Zitat Lebreton, P., Raake, A., Barkowsky, M., Le Callet, P.: Evaluating depth perception of 3D stereoscopic videos. IEEE J. Sel. Top. Signal Process. 6(6), 710–720 (2012)CrossRef Lebreton, P., Raake, A., Barkowsky, M., Le Callet, P.: Evaluating depth perception of 3D stereoscopic videos. IEEE J. Sel. Top. Signal Process. 6(6), 710–720 (2012)CrossRef
7.
Zurück zum Zitat Shao, F., Lin, W., Wang, S., Jiang, G., Yu, M.: Blind image quality assessment for stereoscopic images using binocular guided quality lookup and visual codebook. IEEE Trans. Broadcast. 61(2), 154–165 (2015)CrossRef Shao, F., Lin, W., Wang, S., Jiang, G., Yu, M.: Blind image quality assessment for stereoscopic images using binocular guided quality lookup and visual codebook. IEEE Trans. Broadcast. 61(2), 154–165 (2015)CrossRef
8.
Zurück zum Zitat Gu, K., et al.: No-reference stereoscopic IQA approach: from nonlinear effect to parallax compensation. J. Elect. Comput. Eng 2012(pt.3), 436031.1–436031.12 (2012) Gu, K., et al.: No-reference stereoscopic IQA approach: from nonlinear effect to parallax compensation. J. Elect. Comput. Eng 2012(pt.3), 436031.1–436031.12 (2012)
9.
Zurück zum Zitat Gu, K., Zhai, G., Lin, W., Yang, X., Zhang, W.: No-reference image sharpness assessment in autoregressive parameter space. IEEE Trans. Image Process. 24(10), 3218–3231 (2015)MathSciNetCrossRef Gu, K., Zhai, G., Lin, W., Yang, X., Zhang, W.: No-reference image sharpness assessment in autoregressive parameter space. IEEE Trans. Image Process. 24(10), 3218–3231 (2015)MathSciNetCrossRef
10.
Zurück zum Zitat Su, C., Cormack, L.K., Bovik, A.C.: Oriented correlation models of distorted natural images with application to natural stereopair quality evaluation. IEEE Trans. Image Process. 24(5), 1685–1699 (2015)MathSciNetCrossRef Su, C., Cormack, L.K., Bovik, A.C.: Oriented correlation models of distorted natural images with application to natural stereopair quality evaluation. IEEE Trans. Image Process. 24(5), 1685–1699 (2015)MathSciNetCrossRef
11.
Zurück zum Zitat Oh, H., Ahn, S., Kim, J., Lee, S.: Blind deep S3D image quality evaluation via local to global feature aggregation. IEEE Trans. Image Process. 26(10), 4923–4936 (2017)MathSciNetCrossRef Oh, H., Ahn, S., Kim, J., Lee, S.: Blind deep S3D image quality evaluation via local to global feature aggregation. IEEE Trans. Image Process. 26(10), 4923–4936 (2017)MathSciNetCrossRef
12.
Zurück zum Zitat Chen, M., Cormack, L.K., Bovik, A.C.: No-reference quality assessment of natural stereopairs. IEEE Trans. Image Process. 22(9), 3379–3391 (2013)MathSciNetCrossRef Chen, M., Cormack, L.K., Bovik, A.C.: No-reference quality assessment of natural stereopairs. IEEE Trans. Image Process. 22(9), 3379–3391 (2013)MathSciNetCrossRef
13.
Zurück zum Zitat Sazzad, Z.M., et al.: Objective no-reference stereoscopic image quality prediction based on 2D image features and relative disparity. Adv. Multimed. 2012(8), 1–16 (2012)CrossRef Sazzad, Z.M., et al.: Objective no-reference stereoscopic image quality prediction based on 2D image features and relative disparity. Adv. Multimed. 2012(8), 1–16 (2012)CrossRef
14.
Zurück zum Zitat Lin, Y., Wu, J.: Quality assessment of stereoscopic 3D image compression by binocular integration behaviors. IEEE Trans. Image Process. 23(4), 1527–1542 (2014)MathSciNetCrossRef Lin, Y., Wu, J.: Quality assessment of stereoscopic 3D image compression by binocular integration behaviors. IEEE Trans. Image Process. 23(4), 1527–1542 (2014)MathSciNetCrossRef
15.
Zurück zum Zitat Ciregan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, pp. 3642–3649 (2012) Ciregan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, pp. 3642–3649 (2012)
16.
Zurück zum Zitat Shuai, Y., Wang, Y., Peng, Y., Xia, Y.: Accurate image super-resolution using cascaded multi-column convolutional neural networks. In: 2018 IEEE International Conference on Multimedia and Expo (ICME 2018), pp. 1–6, 23–27 July (2018) Shuai, Y., Wang, Y., Peng, Y., Xia, Y.: Accurate image super-resolution using cascaded multi-column convolutional neural networks. In: 2018 IEEE International Conference on Multimedia and Expo (ICME 2018), pp. 1–6, 23–27 July (2018)
17.
Zurück zum Zitat Mass, A.L., et al.: Rectifier nonlinearities improve neural network acoustic models. In: ICMLW, vol. 30, no. 1 (2013) Mass, A.L., et al.: Rectifier nonlinearities improve neural network acoustic models. In: ICMLW, vol. 30, no. 1 (2013)
19.
Zurück zum Zitat Moorthy, A.K., et al.: Subjective evaluation of stereoscopic image quality. Signal Process. Image Commun. 28(8), 870–883 (2013)CrossRef Moorthy, A.K., et al.: Subjective evaluation of stereoscopic image quality. Signal Process. Image Commun. 28(8), 870–883 (2013)CrossRef
20.
Zurück zum Zitat Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRef Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRef
21.
Zurück zum Zitat He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, pp. 1026–1034 (2015) He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, pp. 1026–1034 (2015)
22.
Zurück zum Zitat Pascanu, R., et al.: On the difficulty of training recurrent neural networks. In: ICML, pp. 1310–1318 (2013) Pascanu, R., et al.: On the difficulty of training recurrent neural networks. In: ICML, pp. 1310–1318 (2013)
23.
Zurück zum Zitat Chen, M.-J., et al.: Full-reference quality assessment of stereopairs accounting for rivalry. Signal Process. Image Commun. 28(9), 1143–1155 (2013)CrossRef Chen, M.-J., et al.: Full-reference quality assessment of stereopairs accounting for rivalry. Signal Process. Image Commun. 28(9), 1143–1155 (2013)CrossRef
24.
Zurück zum Zitat Gorley, P., et al.: Stereoscopic image quality metrics and compression. In: Proceedings of the SPIE, vol. 6803 (2008) Gorley, P., et al.: Stereoscopic image quality metrics and compression. In: Proceedings of the SPIE, vol. 6803 (2008)
Metadaten
Titel
Blind 3D Image Quality Assessment Based on Multi-scale Feature Learning
verfasst von
Yongfang Wang
Shuai Yuan
Yumeng Xia
Ping An
Copyright-Jahr
2020
Verlag
Springer Singapore
DOI
https://doi.org/10.1007/978-981-15-3341-9_22

Neuer Inhalt