Xavier Corbillon
Alisa Devlic
Gwendal Simon
ABSTRACT
With the decreasing price of Head-Mounted Displays (HMDs), 360-degree videos are becoming popular. The streaming of such videos through the Internet with state of the art streaming architectures requires, to provide high immersion feeling, much more bandwidth than the median user's access bandwidth. To decrease the need for bandwidth consumption while providing high immersion to users, scientists and specialists proposed to prepare and encode 360-degree videos into quality-variable video versions and to implement viewport-adaptive streaming. Quality-variable versions are different versions of the same video with non-uniformly spread quality: there exists some so-called Quality Emphasized Regions (QERs). With viewport-adaptive streaming the client, based on head movement prediction, downloads the video version with the high quality region closer to where the user will watch. In this paper we propose a generic theoretical model to find out the optimal set of quality-variable video versions based on traces of head positions of users watching a 360-degree video. We propose extensions to adapt the model to popular quality-variable version implementations such as tiling and offset projection. We then solve a simplified version of the model with two quality levels and restricted shapes for the QER. With this simplified model, we show that an optimal set of four quality-variable video versions prepared by a streaming server, together with a perfect head movement prediction, allow for 45% bandwidth savings to display video with the same average quality as state of the art solutions or allows an increase of 102% of the displayed quality for the same bandwidth budget.
- A. Aminlou, K. Kammachi Sreedhar, A. Zare, and M. Hannuksela. Testing methodology for viewport-dependent encoding and streaming. MPEG meeting, Oct. 2016. m39081.Google Scholar
- G. Boccignone, A. Marcelli, P. Napoletano, G. Di Fiore, G. Iacovoni, and S. Morsa. Bayesian integration of face and low-level cues for foveated video coding. IEEE Transactions on Circuits and Systems for Video Technology, 2008. Google ScholarDigital Library
- A. Borji and L. Itti. State-of-the-art in visual attention modeling. transactions on pattern analysis and machine intelligence, 2013. Google ScholarDigital Library
- J. M. Boyce, Y. Ye, J. Chen, and A. K. Ramasubramonian. Overview of SHVC: scalable extensions of the high efficiency video coding standard. IEEE Trans. Circuits Syst. Video Techn., 26(1):20--34, 2016.Google ScholarDigital Library
- C. Concolato, J. Le Feuvre, F. Denoual, F. Maze, N. Ouedraogo, and J. Taquet. Adaptive streaming of hevc tiled videos using mpeg-dash. IEEE Transactions on Circuits and Systems for Video Technology, 2017.Google Scholar
- X. Corbillon, A. Devlic, G. Simon, and J. Chakareski. Viewport-adaptive navigable 360-degree video delivery. CoRR, abs/1609.08042, 2016.Google Scholar
- X. Corbillon, F. De Simone, and G. Simon. 360-degree video head movement dataset. In Proc of ACM Multimedia Systems (MMSys). ACM, 2017. Google ScholarDigital Library
- P. Di, Q. Xie, and J. Alvarez. Adaptive streaming for fov switching. MPEG meeting, Oct. 2016. m39207.Google Scholar
- S. Dodge and L. Karam. Visual saliency prediction using a mixture of deep neural networks. arXiv preprint arXiv:1702.00372, 2017.Google Scholar
- V. Gaddam, H. Ngo, R. Langseth, C. Griwodz, D. Johansen, and P. Halvorsen. Tiling of Panorama Video for Interactive Virtual Cameras: Overheads and Potential Bandwidth Requirement Reduction. In Picture Coding Symposium (PCS), 2015.Google Scholar
- M. Hosseini and V. Swaminathan. Adaptive 360 VR video streaming based on MPEG-DASH SRD. In Proc. of IEEE ISM, pages 407--408, 2016.Google ScholarCross Ref
- ISO/IEC 23000-20. Omnidirectional media application format (omaf) committe draft, January 2017. ISO/IEC JTC1/SC29/W11.Google Scholar
- L. Itti. Automatic foveation for video compression using a neurobiological model of visual attention. Transactions on Image Processing, 2004. Google ScholarDigital Library
- E. Kuzyakov. End-to-end optimizations for dynamic streaming. Blogpost, February 2017. https://code.facebook.com/posts/637561796428084.Google Scholar
- E. Kuzyakov and D. Pio. Next-generation video encoding techniques for 360 video and vr. Blogpost, January 2016. https://code.facebook.com/posts/1126354007399553.Google Scholar
- J. Le Feuvre and C. Concolato. Tiled-based Adaptive Streaming using MPEG-DASH. In ACM MMSys, 2016. Google ScholarDigital Library
- J.-S. Lee, F. De Simone, and T. Ebrahimi. Efficient video coding based on audio-visual focus of attention. Journal of Visual Communication and Image Representation, 22(8):704--711, 2011. Google ScholarDigital Library
- K. M. Misra, C. A. Segall, M. Horowitz, S. Xu, A. Fuldseth, and M. Zhou. An overview of tiles in HEVC. J. Sel. Topics Signal Proc., 7(6):969--977, 2013.Google ScholarCross Ref
- O. A. Niamut, E. Thomas, L. D'Acunto, C. Concolato, F. Denoual, and S. Y. Lim. MPEG DASH SRD: spatial relationship description. In ACM MMSys, 2016. Google ScholarDigital Library
- D. Ochi, Y. Kunita, A. Kameda, A. Kojima, and S. Iwaki. Live streaming system for omnidirectional video. In IEEE Virtual Reality (VR), 2015.Google ScholarCross Ref
- F. Quan, B. Han, L. Ji, and V. Gopalakrishnan. Optimizing 360 video delivery over cellular networks. In ACM SIGCOMM AllThingsCellular, 2016. Google ScholarDigital Library
- Y. Sánchez, R. Skupin, and T. Schierl. Compressed domain video processing for tile based panoramic streaming using HEVC. In IEEE ICIP, 2015.Google ScholarDigital Library
- K. K. Sreedhar, A. Aminlou, M. M. Hannuksela, and M. Gabbouj. Viewport-adaptive encoding and streaming of 360-degree video for virtual reality applications. In Proc. of IEEE ISM, pages 583--586, 2016.Google ScholarCross Ref
- E. Thomas. Draft for ve on region and point description in omnidirectional content. MPEG meeting, Oct. 2016. m39576.Google Scholar
- H. Wang, V.-T. Nguyen, W. T. Ooi, and M. C. Chan. Mixing Tile Resolutions in Tiled Video: A Perceptual Quality Assessment. In Proc. of ACM NOSSDAV, 2014. Google ScholarDigital Library
- R. G. Youvalari, A. Aminlou, M. M. Hannuksela, and M. Gabbouj. Efficient coding of 360-degree pseudo-cylindrical panoramic video for virtual reality applications. In Proc. of IEEE ISM, pages 525--528, 2016.Google ScholarCross Ref
- M. Yu, H. Lakshman, and B. Girod. A Framework to Evaluate Omnidirectional Video Coding Schemes. In IEEE ISMAR, 2015. Google ScholarDigital Library
- A. Zare, A. Aminlou, M. M. Hannuksela, and M. Gabbouj. Hevc-compliant tile-based streaming of panoramic video for virtual reality applications. In Proc. of ACM Conf. on Multimedia MM, 2016. Google ScholarDigital Library
- A. Zare, K. K. Sreedhar, V. K. M. Vadakital, A. Aminlou, M. M. Hannuksela, and M. Gabbouj. Hevc-compliant viewport-adaptive streaming of stereoscopic panoramic video. In Picture Coding Symposium (PCS). IEEE, 2016.Google ScholarCross Ref
- C. Zhou, Z. Li, and Y. Liu. A measurement study of oculus 360 degree video streaming. In Proc. of ACM MMSys, 2017. Google ScholarDigital Library
Index Terms
- Optimal Set of 360-Degree Videos for Viewport-Adaptive Streaming
Recommendations
Adaptive 360-Degree Video Streaming using Scalable Video Coding
MM '17: Proceedings of the 25th ACM international conference on MultimediaVirtual reality and 360-degree video streaming are growing rapidly, yet, streaming high-quality 360-degree video is still challenging due to high bandwidth requirements. Existing solutions reduce bandwidth consumption by streaming high-quality video ...
360-Degree Video Head Movement Dataset
MMSys'17: Proceedings of the 8th ACM on Multimedia Systems ConferenceWhile Virtual Reality applications are increasingly attracting the attention of developers and business analysts, the behaviour of users watching 360-degree (i.e. omnidirectional) videos has not been thoroughly studied yet. This paper introduces a ...
Tile Priorities in Adaptive 360-Degree Video Streaming
Smart MultimediaAbstractFor video applications, tiled streaming is a popular way to deliver viewport dependent 360-degree video. Unfortunately, dynamic adaptation to network bandwidth fluctuations of such video streams is still a challenge. This paper proposes a method ...
Comments