Skip to main content
Top

2023 | OriginalPaper | Chapter

Panoramic Video Quality Assessment Based on Spatial-Temporal Convolutional Neural Networks

Authors : Tingting An, Songlin Sun, Rui Liu

Published in: Signal and Information Processing, Networking and Computers

Publisher: Springer Nature Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The development of 5G technology and Ultra HD video provide the basis for panoramic video, namely virtual reality (VR). At present, the traditional VQA method is not effective on panoramic video. Therefore, it is crucial to design objective VQA models for the standardization of panoramic video industry. With the development of deep learning, excellent algorithms of VQA methods based on convolutional neural network have emerged. In this paper, we propose a full reference VQA model based on spatial-temporal 3D convolutional neural network, the feature extraction combined the time and spatial information. we verify and optimize the proposed VQA model based on VQA-ODV panoramic video database, its objective score has a higher correlation with subjective scores than that of traditional VQA methods.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference 《5G高新视频-沉浸式视频技术白皮书. 国家广播电视总局科技司 (2020) 《5G高新视频-沉浸式视频技术白皮书. 国家广播电视总局科技司 (2020)
2.
go back to reference Gaubatz, M., Hemami, S.: Visual Quality Assessment Package. http://foulard,ece.cornell.Edu/gaubatz/metrix_mux Gaubatz, M., Hemami, S.: Visual Quality Assessment Package. http://​foulard,ece.cornell.Edu/gaubatz/metrix_mux
3.
go back to reference Sheikh, H.R., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15(11), 3440–3451 (2006)CrossRef Sheikh, H.R., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15(11), 3440–3451 (2006)CrossRef
4.
go back to reference Pinson, M.H., Wolf, S.: A new standardized method for objectively measuring video quality. IEEE Trans. Broadcast. 50(3), 312–322 (2004)CrossRef Pinson, M.H., Wolf, S.: A new standardized method for objectively measuring video quality. IEEE Trans. Broadcast. 50(3), 312–322 (2004)CrossRef
5.
go back to reference Sheikh, H.R., Bovik, A.C.: A visual information fidelity approach to video quality assessment. In: Proceedings of the 1st International Workshop on Video Processing and Quality Metrics for Consumer Electronics, pp. 23–25 (2005) Sheikh, H.R., Bovik, A.C.: A visual information fidelity approach to video quality assessment. In: Proceedings of the 1st International Workshop on Video Processing and Quality Metrics for Consumer Electronics, pp. 23–25 (2005)
6.
go back to reference Vu, P.V., Vu, C.T., Chandler, D.M.: A spatiotemporal most apparent-distortion model for video quality assessment. In: Proceedings of the 18th IEEE International Conference on Image Processing (ICIP), pp. 2505–2508 (2011) Vu, P.V., Vu, C.T., Chandler, D.M.: A spatiotemporal most apparent-distortion model for video quality assessment. In: Proceedings of the 18th IEEE International Conference on Image Processing (ICIP), pp. 2505–2508 (2011)
7.
go back to reference Seshadrinathan, K., Bovik, A.C.: Motion tuned spatiotemporal quality assessment of natural videos. IEEE Trans. Image Process. 19(2), 335–350 (2010)MathSciNetCrossRef Seshadrinathan, K., Bovik, A.C.: Motion tuned spatiotemporal quality assessment of natural videos. IEEE Trans. Image Process. 19(2), 335–350 (2010)MathSciNetCrossRef
8.
go back to reference Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: Proceedings of the 27th Asilomar Conference on Signals, Systems & Computers, vol. 2, pp. 1398–1402 (2003) Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: Proceedings of the 27th Asilomar Conference on Signals, Systems & Computers, vol. 2, pp. 1398–1402 (2003)
9.
go back to reference Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20(8), 2378–2386 (2011)MathSciNetCrossRef Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20(8), 2378–2386 (2011)MathSciNetCrossRef
10.
go back to reference Zakharchenko, V., Choi, K.P., Park, J.H.: Quality metric for spherical panoramic video. In: Optics and Photonics for Information Processing X, vol. 9970. International Society for Optics and Photonics, p. 99700C (2016) Zakharchenko, V., Choi, K.P., Park, J.H.: Quality metric for spherical panoramic video. In: Optics and Photonics for Information Processing X, vol. 9970. International Society for Optics and Photonics, p. 99700C (2016)
11.
go back to reference Yu, M., Lakshman, H., Girod, B.: A framework to evaluate omnidirectional video coding schemes. In: IEEE ISMAR. IEEE, pp. 31–36 (2015) Yu, M., Lakshman, H., Girod, B.: A framework to evaluate omnidirectional video coding schemes. In: IEEE ISMAR. IEEE, pp. 31–36 (2015)
14.
go back to reference Kim, W., Kim, J., Ahn, S., Kim, J., Lee, S.: Deep Video Quality Assessor: From Spatio-Temporal Visual Sensitivity to a Convolutional Neural Aggregation Network. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11205, pp. 224–241. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01246-5_14CrossRef Kim, W., Kim, J., Ahn, S., Kim, J., Lee, S.: Deep Video Quality Assessor: From Spatio-Temporal Visual Sensitivity to a Convolutional Neural Aggregation Network. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11205, pp. 224–241. Springer, Cham (2018). https://​doi.​org/​10.​1007/​978-3-030-01246-5_​14CrossRef
15.
go back to reference Hanhart, P., Boyce, J., Choi, K.: JVET common test conditions and evaluation procedures for 360 video. In: Joint Video Exploration Team of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, JVET-K1012 (2018) Hanhart, P., Boyce, J., Choi, K.: JVET common test conditions and evaluation procedures for 360 video. In: Joint Video Exploration Team of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, JVET-K1012 (2018)
16.
go back to reference Duan, H., Zhai, G., Yang, X., et al.: IVQAD 2017: an immersive video quality assessment database. In: 2017 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 1–5 (2017) Duan, H., Zhai, G., Yang, X., et al.: IVQAD 2017: an immersive video quality assessment database. In: 2017 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 1–5 (2017)
17.
go back to reference Yang, J.C., Liu, T.L., Jiang, B., et al.: 3D panoramic virtual reality video quality assessment based on 3D convolutional neural networks. IEEE Access 6, 38669–38682 (2018)CrossRef Yang, J.C., Liu, T.L., Jiang, B., et al.: 3D panoramic virtual reality video quality assessment based on 3D convolutional neural networks. IEEE Access 6, 38669–38682 (2018)CrossRef
18.
go back to reference Li, C., Xu, M., Du, X., Wang, Z.: Bridge the gap between VQA and human behavior on omnidirectional video: a large-scale dataset and a deep learning model. In: 2018 ACM Multimedia Conference (MM ’18), October 22–26, 2018, Seoul, Republic of Korea. ACM, New York, NY, USA, p. 9 (2018) Li, C., Xu, M., Du, X., Wang, Z.: Bridge the gap between VQA and human behavior on omnidirectional video: a large-scale dataset and a deep learning model. In: 2018 ACM Multimedia Conference (MM ’18), October 22–26, 2018, Seoul, Republic of Korea. ACM, New York, NY, USA, p. 9 (2018)
19.
go back to reference LeCun, Y., et al.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)CrossRef LeCun, Y., et al.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)CrossRef
20.
go back to reference Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.:Learning spatiotemporal features with 3D convolutional networks. In: 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, pp.4489–4497 (2015) Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.:Learning spatiotemporal features with 3D convolutional networks. In: 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, pp.4489–4497 (2015)
21.
go back to reference 孙宇乐. 全景视频质量评价与编码研究. 浙江:浙江大学 (2020) 孙宇乐. 全景视频质量评价与编码研究. 浙江:浙江大学 (2020)
22.
go back to reference Pearson, K.: Note on regression and inheritance in the case of two parents. Proc. R. Soc. London 58(347–352), 240–242 (1895)CrossRef Pearson, K.: Note on regression and inheritance in the case of two parents. Proc. R. Soc. London 58(347–352), 240–242 (1895)CrossRef
23.
go back to reference Yang, J.C., Lin, Y.C., Gao, Z.Q., Lv, Z.H., Wei, W., Song, H.B.: Quality index for stereoscopic images by separately evaluating adding and subtracting. PLoS ONE 10(12), e0145800 (2015)CrossRef Yang, J.C., Lin, Y.C., Gao, Z.Q., Lv, Z.H., Wei, W., Song, H.B.: Quality index for stereoscopic images by separately evaluating adding and subtracting. PLoS ONE 10(12), e0145800 (2015)CrossRef
Metadata
Title
Panoramic Video Quality Assessment Based on Spatial-Temporal Convolutional Neural Networks
Authors
Tingting An
Songlin Sun
Rui Liu
Copyright Year
2023
Publisher
Springer Nature Singapore
DOI
https://doi.org/10.1007/978-981-19-3387-5_161