Brian Guenter
Desney Tan
Abstract
We exploit the falloff of acuity in the visual periphery to accelerate graphics computation by a factor of 5-6 on a desktop HD display (1920x1080). Our method tracks the user's gaze point and renders three image layers around it at progressively higher angular size but lower sampling rate. The three layers are then magnified to display resolution and smoothly composited. We develop a general and efficient antialiasing algorithm easily retrofitted into existing graphics code to minimize "twinkling" artifacts in the lower-resolution layers. A standard psychophysical model for acuity falloff assumes that minimum detectable angular size increases linearly as a function of eccentricity. Given the slope characterizing this falloff, we automatically compute layer sizes and sampling rates. The result looks like a full-resolution image but reduces the number of pixels shaded by a factor of 10-15.
We performed a user study to validate these results. It identifies two levels of foveation quality: a more conservative one in which users reported foveated rendering quality as equivalent to or better than non-foveated when directly shown both, and a more aggressive one in which users were unable to correctly label as increasing or decreasing a short quality progression relative to a high-quality foveated reference. Based on this user study, we obtain a slope value for the model of 1.32-1.65 arc minutes per degree of eccentricity. This allows us to predict two future advantages of foveated rendering: (1) bigger savings with larger, sharper displays than exist currently (e.g. 100 times speedup at a field of view of 70° and resolution matching foveal acuity), and (2) a roughly linear (rather than quadratic or worse) increase in rendering cost with increasing display field of view, for planar displays at a constant sharpness.
Supplemental Material
Available for Download
Supplemental Materials for Foveated 3D graphics
- Baudisch, P., DeCarlo, D., Duchowski, A. T., and Geisler, W. S. 2003. Focusing on the essential: considering attention in display design. Commun. ACM 46 (March), 60--66. Google Scholar
Digital Library
- Bergström, P. 2003. Eye-movement Controlled Image Coding. PhD thesis, Linköping University, Linköping, Sweden.Google Scholar
- Cheng, I. 2003. Foveated 3D model simplification. In Proceedings, Signal Processing and its Applications, 241--244.Google ScholarCross Ref
- Colenbrander, A. 2001. Vision and vision loss. In Duane's Clinical Opthalmology. Lippincott, Williams and Wilkins, ch. 51.Google Scholar
- Cook, R. L., Porter, T., and Carpenter, L. 1984. Distributed ray tracing. SIGGRAPH Comput. Graph. 18 (January), 137--145. Google ScholarDigital Library
- Duchowski, A. T., and Çöltekin, A. 2007. Foveated gaze-coningent displays for periperal LOD management, 3D visualization, and stereo imaging. ACM Trans. on Multimedia, Communications and Applications 3, 4 (Dec.). Google ScholarDigital Library
- Duchowski, A. T. 2002. A breadth-first survey of eye tracking applications. Behavior Research Methods, Instruments, and Computers 34, 455--70.Google ScholarCross Ref
- Funkhouser, T., and Sequin, C. 1993. Adaptive display algorithm for interactive frame rates during visualization of complex virtual environments. In Computer Graphics (SIGGRAPH '93 Proceedings), vol. 27, 247--254. Google ScholarDigital Library
- Geisler, W. S., and Perry, J. S. 2002. Real-time simulation of arbitrary visual fields. In Proceedings of the symposum on eye tracking research applications ETRA. Google ScholarDigital Library
- Guenter, B. 1994. Motion compensated noise reduction. Tech. rep., Microsoft Research.Google Scholar
- Han, C., Sun, B., Ramamoorthi, R., and Grinspun, E. 2007. Frequency domain normal map filtering. ACM Trans. Graph. 26 (July). Google ScholarDigital Library
- Horvitz, E., and Lengyel, J. 1997. Perception, attention, and resources: a decision-theoretic approach to graphics rendering. In Proceedings of the Thirteenth conference on Uncertainty in artificial intelligence, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, UAI'97, 238--249. Google ScholarDigital Library
- Levoy, M., and Whitaker, R. 1989. Gaze-directed volume rendering. Tech. rep., University of North Carolina.Google Scholar
- Loschky, L. C., and McConkie, G. W. 2000. User performance with gaze contingent multiresolutional displays. In Proceedings of the 2000 symposium on Eye tracking research & applications, ETRA '00, 97--103. Google ScholarDigital Library
- Luebke, D., Halen, B., Newfield, D., and Watson, B. 2000. Perceptually driven simplification using gaze-directed rendering. Tech. rep., University of Virginia.Google Scholar
- Murphy, H., and Duchowski, A. T. 2001. Gaze-contingent level of detail rendering. In Eurographics 2001 (Short Presentations).Google Scholar
- Murphy, H. A., and Duchowski, A. T. 2007. Hybrid image/model based gaze-contingent rendering. Applied Perception in Graphics and Visualization. Google ScholarDigital Library
- Myszkowski, K. 2002. Perception-based global illumination, rendering, and animation techniques. In SCCG '02: Proceedings of the 18th Spring Conference on Computer Graphics, 13--24. Google ScholarDigital Library
- Nehab, D., Sander, P. V., Lawrence, J. D., Tatarchuk, N., and Isidoro, J. R. 2007. Accelerating real-time shading with reverse reprojection caching. In ACM SIGGRAPH/Eurographics Symposium on Graphics Hardware, 25--35. Google ScholarDigital Library
- Ohshima, T., Yamamoto, H., and Tamura, H. 1996. Gaze-directed adaptive rendering for interacting with virtual space. In Proceedings of the 1996 Virtual Reality Annual International Symposium (VRAIS 96), IEEE Computer Society, Washington, DC, USA, VRAIS '96, 103--110. Google ScholarDigital Library
- Olano, M., and Baker, D. 2010. LEAN mapping. In Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games, ACM, New York, NY, USA, I3D '10, 181--188. Google ScholarDigital Library
- O'Sullivan, C., Dingliana, J., and Howlett, S. 2002. Gaze-contingent algorithms for interactive graphics. In The Mind's Eyes: Cognitive and Applied Aspects of Eye Movement Research, J. Hyönä, R. Radach, and H. Deubel, Eds. Elsevier Science, Oxford.Google Scholar
- Ramanarayanan, G., Ferwerda, J., Walter, B., and Bala, K. 2007. Visual equivalence: towards a new standard for image fidelity. ACM Trans. Graph. 26 (July). Google ScholarDigital Library
- Reddy, M. 1998. Specification and evaluation of level of detail selection criteria. Virtual Reality 3, 2, 132--143.Google ScholarDigital Library
- Reddy, M. 2001. Perceptually optimized 3D graphics. Computer Graphics and Applications, IEEE 21, 5 (sep/oct), 68--75. Google ScholarDigital Library
- Reingold, E., Loschky, L. C., McConkie, G. W., and Stampe, D. M. 2003. Gaze-contingent multiresolution displays: an integrative review. Human Factors 45, 2, 307--28.Google ScholarCross Ref
- Strasburger, H., Rentschler, I., and Jüttner, M. 2011. Peripheral vision and pattern recognition: a review. Journal of Vision 11, 5, 1--82.Google ScholarCross Ref
- Thibos, L. N. 1989. Image processing by the human eye. In Visual Communications and Image Processing IV, W. A. Pearlman, Ed., Proc. SPIE 1199, 1148--1153.Google Scholar
- Virsu, V., and Romano, J. 1979. Visual resolution, contrast sensitivity, and the cortical magnification factor. Experimental Brain Research 37, 475--494.Google ScholarCross Ref
- Watson, B., Walker, N., Hodges, L. F., and Worden, A. 1997. Managing level of detail through peripheral degradation: effects on search performance with a head-mounted display. ACM Trans. Comput.-Hum. Interact. 4 (December), 323--346. Google ScholarDigital Library
- Watson, B., Walker, N., and Hodges, L. F. 2004. Supra-threshold control of peripheral LOD. ACM Trans. Graph. 23 (Aug.), 750--759. Google ScholarDigital Library
- Weaver, K. A. 2007. Design and evaluation of a perceptually adaptive rendering system for immersive virtual reality environments. Master's thesis, Iowa State University.Google Scholar
- Weymouth, F. W. 1958. Visual sensory units and the minimum angle of resolution. American Journal of Opthalmology 46, 102--113.Google ScholarCross Ref
- Yee, H., Pattanaik, S., and Greenberg, D. P. 2001. Spatiotemporal sensitivity and visual attention for efficient rendering of dynamic environments. ACM Trans. Graph. 20 (January), 39--65. Google ScholarDigital Library
Index Terms
- Foveated 3D graphics
Recommendations
Kernel Foveated Rendering
Foveated rendering coupled with eye-tracking has the potential to dramatically accelerate interactive 3D graphics with minimal loss of perceptual detail. In this paper, we parameterize foveated rendering by embedding polynomial kernel functions in the ...
Decoupled sampling for graphics pipelines
We propose a generalized approach to decoupling shading from visibility sampling in graphics pipelines, which we call decoupled sampling. Decoupled sampling enables stochastic supersampling of motion and defocus blur at reduced shading cost, as well as ...
Filtering and Rendering of Resolution-Dependent Reflectance Models
The apparent reflectance of a surface depends upon the resolution at which it is imaged. Conventional reflectance models represent reflection at a single predetermined resolution; however, a low-resolution pixel that views a greater surface area often ...
Comments