ABSTRACT
Viewport adaptive streaming is emerging as a promising way to deliver high quality 360-degree video. It is still a critical issue to predict user's viewpoint and deliver partial video within the viewport. Current widely-used motion-based or content-saliency methods have low precision, especially for long-term prediction. In this paper, benefiting from data-driven learning, we propose a Cross-user Learning based System (CLS) to improve the precision of viewport prediction. Since users have similar region-of-interest (ROI) when watching a same video, it is possible to exploit cross-users' ROI behavior to predict viewport. We use a machine learning algorithm to group users according to historical fixations, and predict the viewing probability by the class. Additionally, we present a QoE-driven rate allocation to minimize the expected streaming distortion under bandwidth constraint, and give a Multiple-Choice Knapsack solution. Experiments demonstrate that CLS provides 2dB quality improvement than full-image streaming and 1.5 dB quality improvement than linear regression (LR) method. On average, the precision of viewpoint prediction improve 15% compared with LR.
- Ali Borji, Ming-Ming Cheng, Qibin Hou, Huaizu Jiang, and Jia Li. 2014. Salient object detection: A survey. arXiv preprint arXiv:1411.5878 (2014).Google Scholar
- Xavier Corbillon, Alisa Devlic, Gwendal Simon, and Jacob Chakareski. 2017. Optimal Set of 360-Degree Videos for Viewport-Adaptive Streaming, In Proc. of ACM Multimedia (MM) (2017). Google ScholarDigital Library
- Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, and others. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In Kdd, Vol. 96. 226--231. Google ScholarDigital Library
- Ching-Ling Fan, Jean Lee, Wen-Chih Lo, Chun-Ying Huang, Kuan-Ta Chen, and Cheng-Hsin Hsu. 2017. Fixation Prediction for 360 Video Streaming to Head-Mounted Displays. (2017).Google Scholar
- M. Coban G. V. Auwera and M. Karczewicz. 2016. VR/360 Video Truncated Square Pyramid Geometry for OMAF. ISO/IEC JTC1/SC29/WG11/M (2016).Google Scholar
- Mario Graf, Christian Timmerer, and Christopher Mueller. 2017. Towards bandwidth efficient adaptive streaming of omnidirectional video over http: Design, implementation, and evaluation. In Proceedings of the 8th ACM on Multimedia Systems Conference. ACM, 261--271. Google ScholarDigital Library
- Marti A. Hearst, Susan T Dumais, Edgar Osuna, John Platt, and Bernhard Scholkopf. 1998. Support vector machines. IEEE Intelligent Systems and their applications, Vol. 13, 4 (1998), 18--28. Google ScholarDigital Library
- Mohammad Hosseini and Viswanathan Swaminathan. 2016. Adaptive 360 VR video streaming based on MPEG-DASH SRD. In Multimedia (ISM), 2016 IEEE International Symposium on. IEEE, 407--408.Google ScholarCross Ref
- Xing Liu, Qingyang Xiao, Vijay Gopalakrishnan, Bo Han, Feng Qian, and Matteo Varvello. 2017. 360 Innovations for Panoramic Video Streaming. In Proceedings of the 16th ACM Workshop on Hot Topics in Networks. ACM, 50--56. Google ScholarDigital Library
- Stefano Petrangeli, Viswanathan Swaminathan, Mohammad Hosseini, and Filip De Turck. 2017. An HTTP/2-Based Adaptive Streaming Framework for 360 Virtual Reality Videos. In Proceedings of the 2017 ACM on Multimedia Conference. ACM, 306--314. Google ScholarDigital Library
- Feng Qian, Lusheng Ji, Bo Han, and Vijay Gopalakrishnan. 2016. Optimizing 360 video delivery over cellular networks. In Proceedings of the 5th Workshop on All Things Cellular: Operations, Applications and Challenges. ACM, 1--6. Google ScholarDigital Library
- Haakon Riiser, Tore Endestad, Paul Vigmostad, Carsten Griwodz, and Pâl Halvorsen. 2012. Video streaming using a location-based bandwidth-lookup service for bitrate planning. ACM Trans. Multimedia Comput. Commun. Appl (TOMCCAP), Vol. 8, 3 (2012), 24. Google ScholarDigital Library
- Luigi Rizzo. 1997. Dummynet: a simple approach to the evaluation of network protocols. ACM SIGCOMM Computer Communication Review, Vol. 27, 1 (1997), 31--41. Google ScholarDigital Library
- Patrice Rondao Alface, Maarten Aerts, Donny Tytgat, Sammy Lievens, Christoph Stevens, Nico Verzijp, and Jean-Francois Macq. 2017. 16K Cinematic VR Streaming. In Proceedings of the 2017 ACM on Multimedia Conference. ACM, 1105--1112. Google ScholarDigital Library
- Prabhakant Sinha and Andris A Zoltners. 1979. The multiple-choice knapsack problem. Operations Research, Vol. 27, 3 (1979), 503--515. Google ScholarDigital Library
- Kashyap Kammachi Sreedhar, Alireza Aminlou, Miska M Hannuksela, and Moncef Gabbouj. 2016. Viewport-Adaptive Encoding and Streaming of 360-Degree Video for Virtual Reality Applications. In Multimedia (ISM), 2016 IEEE International Symposium on. IEEE, 583--586.Google ScholarCross Ref
- ISO/IEC JTC1/SC29/WG11 W13533. 2012. MPEG DASH: The Standard for Multimedia Streaming over the Internet.Google Scholar
- Chenglei Wu, Zhihao Tan, Zhi Wang, and Shiqiang Yang. 2017. A Dataset for Exploring User Behaviors in VR Spherical Video Streaming. In Proceedings of the 8th ACM on Multimedia Systems Conference. ACM, 193--198. Google ScholarDigital Library
- Lan Xie, Zhimin Xu, Yixuan Ban, Xinggong Zhang, and Zongming Guo. 2017. 360ProbDASH: Improving QoE of 360 Video Streaming Using Tile-based HTTP Adaptive Streaming. In Proceedings of the 2017 ACM on Multimedia Conference. ACM, 315--323. Google ScholarDigital Library
- M. Yu, H. Lakshman, and B. Girod. 2015. A Framework to Evaluate Omnidirectional Video Coding Schemes. In 2015 IEEE ISMAR. 31--36. Google ScholarDigital Library
Index Terms
- CLS: A Cross-user Learning based System for Improving QoE in 360-degree Video Adaptive Streaming
Recommendations
360ProbDASH: Improving QoE of 360 Video Streaming Using Tile-based HTTP Adaptive Streaming
MM '17: Proceedings of the 25th ACM international conference on MultimediaRecently, there has been a significant interest towards 360-degree panorama video. However, such videos usually require extremely high bitrate which hinders their widely spread over the Internet. Tile-based viewport adaptive streaming is a promising way ...
Machine Learning Based Content-Agnostic Viewport Prediction for 360-Degree Video
Accurate and fast estimations or predictions of the (near) future location of the users of head-mounted devices within the virtual omnidirectional environment open a plethora of opportunities in application domains such as interactive immersive gaming and ...
Efficient viewport prediction and tiling schemes for 360 degree video streaming
MMSys '24: Proceedings of the 15th ACM Multimedia Systems Conference360-degree video streaming for VR visualisation is characterised by large transmission data volume and stringent interactive latency demands; hence guaranteeing suitable transmission quality, while meeting the existing constraints, is highly challenging. ...
Comments