Miguel Fabian Romero-Rondón
ABSTRACT
While Virtual Reality (VR) represents a revolution in the user experience, current VR systems are flawed on different aspects. The difficulty to focus naturally in current headsets incurs visual discomfort and cognitive overload, while high-end headsets require tethered powerful hardware for scene synthesis. One of the major solutions envisioned to address these problems is foveated rendering. We consider the problem of streaming stored 360° videos to a VR headset equipped with eye-tracking and foveated rendering capabilities. Our end research goal is to make high-performing foveated streaming systems allowing the playback buffer to build up to absorb the network variations, which is permitted in none of the current proposals. We present our foveated streaming prototype based on the FOVE, one of the first commercially available headsets with an integrated eye-tracker. We build on the FOVE's Unity API to design a gaze-adaptive streaming system using one low- and one high-resolution segment from which the foveal region is cropped with per-frame filters. The low- and high-resolution frames are then merged at the client to approach the natural focusing process.
- K. Carnegie and T. Rhee. 2015. Reducing visual discomfort with HMDs using dynamic depth of field. IEEE Computer Graphics and Appl. 35, 5 (2015), p. 34--41.Google Scholar
Cross Ref
- X. Corbillon, G. Simon, A. Devlic, and J. Chakareski. 2017. Viewport-adaptive navigable 360-degree video delivery. In IEEE ICC.Google Scholar
- G. Illahi, M. Siekkinen, and E. Masala. 2017. Foveated video streaming for cloud gaming. In IEEE Int. Workshop on Multimedia Signal Proc. (MMSP).Google Scholar
- A. Patney, J. Kim, M. Salvi, A. Kaplanyan, C. Wyman, et al. 2016. Perceptually-based foveated virtual reality. In ACM SIGGRAPH Emerging Technologies. Google ScholarDigital Library
- A. Patney, M. Salvi, J. Kim, A. Kaplanyan, C. Wyman, et al. 2016. Towards foveated rendering for gaze-tracked virtual reality. ACM Trans. on Graphics 35, 6 (2016), p. 179. Google ScholarDigital Library
- P. Rondao Alface, M. Aerts, D. Tytgat, S. Lievens, C. Stevens, et al. 2017. 16K Cinematic VR Streaming. In ACM Multimedia Conf. (MM). Google ScholarDigital Library
- J. Ryoo, K. Yun, D. Samaras, S. R. Das, and G. Zelinsky. 2016. Design and Evaluation of a Foveated Video Streaming Service for Commodity Client Devices. In ACM Int. Conf. on Multimedia Sys. (MMSys). Google ScholarDigital Library
Index Terms
- Foveated streaming of virtual reality videos
Recommendations
Towards foveated rendering for gaze-tracked virtual reality
Foveated rendering synthesizes images with progressively less detail outside the eye fixation region, potentially unlocking significant speedups for wide field-of-view displays, such as head mounted displays, where target framerate and resolution is ...
Perceptually-based foveated virtual reality
SIGGRAPH '16: ACM SIGGRAPH 2016 Emerging TechnologiesHumans have two distinct vision systems: foveal and peripheral vision. Foveal vision is sharp and detailed, while peripheral vision lacks fidelity. The difference in characteristics of the two systems enable recently popular foveated rendering systems, ...
Real vs Simulated Foveated Rendering to Reduce Visual Discomfort in Virtual Reality
Human-Computer Interaction – INTERACT 2021AbstractIn this paper, a study aimed at investigating the effects of real (using eye tracking to determine the fixation) and simulated foveated blurring in immersive Virtual Reality is presented. Techniques to reduce the optical flow perceived at the ...
Comments