Jangwoo Son
Eun-Seok Ryu
ABSTRACT
The computing power and bandwidth of the current VR are limited when compared to the high-quality VR. To overcome these limits, this study proposes a new viewport dependent streaming method that transmits 360-degree videos using the high efficiency video coding (HEVC) and the scalability extension of HEVC (SHVC). The proposed SHVC and HEVC encoders generate the bitstream that can transmit tiles independently. Therefore, the bitstream generated by the proposed encoder can be extracted in units of tiles. In accordance with what is discussed in the standard, the proposed extractor extracts the bitstream of the tiles corresponding to the viewport. SHVC video bitstream extracted by the proposed methods consist of (i) an SHVC base layer (BL) which represents the entire 360-degree area and (ii) an SHVC enhancement layer (EL) for selective streaming with viewport (region of interest (ROI)) tiles. When the proposed HEVC encoder is used, low and high resolution sequences are separately encoded as the BL and EL of SHVC. By streaming the BL(low resolution) and selective EL(high resolution) tiles with ROI instead of streaming whole high quality 360-degree video, the proposed method can reduce the network bandwidth as well as the computational complexity on the decoder side. Experimental results show more than 47% bandwidth reduction.
- M. Champel, T. Stockhammer, T. Fautier, E. Thomas and R. Koenen. 2016. Quality Requirements for VR. Technical Report ISO/IEC JTC1/SC29/WG11/MPEG 116/m39532.Google Scholar
- A. Zare, A. Aminlou, M. Hannuksela and M. Gabbouj. 2016. HEVC-compliant Tile-based Streaming of Panoramic Video for Virtual Reality Applications. In Proceedings of the 24th ACM Multimedia Conference. Netherlands, 601--605. Google ScholarDigital Library
- Y. Sánchez, R. Skupin and T. Schierl. 2017. Video processing for panoramic streaming using HEVC and its scalable extensions. Multimedia Tools and Applications. 5631--5659. Google ScholarDigital Library
- Y. K. Wang, Hendry and M. Karczewicz. 2016. Viewport dependent processing in VR: partial video decoding. Technical Report ISO/IEC JTC1/SC29/WG11/MPEG 116/ m38559.Google Scholar
- R. Skupin, Y. Sanchez, K. Suhrmg, T. Schierl, E. S. Ryu and J. Son. 2017. Temporal MCTS Coding Constraints Implementation. Technical Report ISO/IEC JTC1/SC29/WG11/MPEG 120/ m41626.Google Scholar
- J. Boyce, A. Ramasubramanian, R. Skupin, Gary J. Sullivan, A. Tourapis and Y. K. Wang. 2017. HEVC Additional Supplemental Enhancement Information (Draft 4). Technical Report ISO/IEC JTC1/SC29/WG11/JCTVC-AC1005.Google Scholar
- J. Boyce, Y. Ye, J. Chen and A. K. Ramasubramoian. 2015. Overview of SHVC: scalable extensions of the high efficiency video coding standard. IEEE Transactions on Circuits and Systems for Video Technology. 26, 1(2015), 20--34.Google ScholarDigital Library
- R. Skupin and Y. Sanchez. 2017. MCTS extraction with optional slice reordering. Technical Report ISO/IEC JTC1/SC29/WG11/MPEG 119/m40902.Google Scholar
- C. Feldmann, C. Bulla and B. Cellarius. 2013. Efficient stream-reassembling for video conferencing applications using tiles in HEVC. Proc. of International Conferences on Advances in Multimedia (MMEDIA). 130--135.Google Scholar
- B. Bross, P. Helle, H. Lakshman and K. Ugur. 2014. Inter-Picture Prediction in HEVC. Integrated Circuits and Systems. Springer, 113--140.Google Scholar
- R. Sjoberg, Y. Chen, A. Fujibayashi, M. M. Hannuksela, J. Samuelsson, T. K. Tan, T. K., Y. K. Wang and S. Wenger. 2012. Overview of HEVC high-level syntax and reference picture management. IEEE Transactions on Circuits and Systems for Video Technology. 22, 12(2012), 1858--1870. Google ScholarDigital Library
- J. Boyce, E. Alshina, A. Abbas and Y. Ye. 2017. JVET common test conditions and evaluation procedures for 360° video. Technical Report ISO/IECJTC1/SC29/WG11/JVET-H1030.Google Scholar
- Wikipedia. 2018. Comparison of virtual reality headsets. Retrieved from https://en.wikipedia.org/wiki/Comparison_of_virtual_reality_headsetsGoogle Scholar
- Github. 2018. HEVCESBrowser. Retrieved from https://github.com/virinext/hevcesbrowserGoogle Scholar
- Github. 2018. FFmpeg. Retrieved from https://github.com/FFmpeg/FFmpeg.Google Scholar
- High Efficiency Video Coding (HEVC) reference software HM. 2017. Retrieved from https://hevc.hhi.fraunhofer.de/Google Scholar
Recommendations
Towards Viewport-dependent 6DoF 360 Video Tiled Streaming for Virtual Reality Systems
MM '20: Proceedings of the 28th ACM International Conference on MultimediaPrevious studies of 360-degree video streaming with regard to virtual reality allowed users to move their head freely, while their position is fixed according to the camera's location in virtual reality. One of the approaches to overcome the problem is ...
Implementing Motion-Constrained Tile and Viewport Extraction for VR Streaming
NOSSDAV '18: Proceedings of the 28th ACM SIGMM Workshop on Network and Operating Systems Support for Digital Audio and Video1 360-degree video streaming for virtual reality (VR) is emerging. However, the computing power and bandwidth of the current VR are limited when compared to the high-quality VR. To overcome these limits, this study proposes 360 video tiled streaming ...
Immersive 360° VR tiled streaming system for esports service
MMSys '18: Proceedings of the 9th ACM Multimedia Systems ConferenceThis paper shows how a system was designed and implemented for the live tiled streaming of 360° Virtual Reality(VR) eSports based on HEVC. A virtual camera technique was designed and implemented based on the Unity and Unreal engines as a means of ...
Comments