Ashok Veeraraghavan
Amit Agrawal
Abstract
We describe a theoretical framework for reversibly modulating 4D light fields using an attenuating mask in the optical path of a lens based camera. Based on this framework, we present a novel design to reconstruct the 4D light field from a 2D camera image without any additional refractive elements as required by previous light field cameras. The patterned mask attenuates light rays inside the camera instead of bending them, and the attenuation recoverably encodes the rays on the 2D sensor. Our mask-equipped camera focuses just as a traditional camera to capture conventional 2D photos at full sensor resolution, but the raw pixel values also hold a modulated 4D light field. The light field can be recovered by rearranging the tiles of the 2D Fourier transform of sensor values into 4D planes, and computing the inverse Fourier transform. In addition, one can also recover the full resolution image information for the in-focus parts of the scene.
We also show how a broadband mask placed at the lens enables us to compute refocused images at full sensor resolution for layered Lambertian scenes. This partial encoding of 4D ray-space data enables editing of image contents by depth, yet does not require computational recovery of the complete 4D light field.
Supplemental Material
- Accorsi, R., Gasparini, F., and Lanza, R. C. 2001. Optimal coded aperture patterns for improved SNR in nuclear medicine imaging. Nuclear Instruments and Methods in Physics Research A 474 (Dec.), 273--284.Google Scholar
Cross Ref
- Adelson, T., and Wang, J. 1992. Single lens stereo with a plenoptic camera. IEEE Trans. Pattern Anal. Machine Intell. 14, 99--106. Google ScholarDigital Library
- Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. ACM Trans. Graph. 23, 3, 294--302. Google ScholarDigital Library
- Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization using graph cuts. IEEE Trans. Pattern Anal. Machine Intell. 23, 1222--1239. Google ScholarDigital Library
- Chaudhuri, S., and Rajagopalan, A. 1999. Depth from Defocus: A Real Aperture Imaging Approach. Springer.Google ScholarCross Ref
- Dowski, E. R., and Cathey, W. 1995. Extended depth of field through wavefront coding. Appl. Optics 34, 11 (Apr.), 1859--1866.Google ScholarCross Ref
- Dowski, E. R., and Johnson, G. E. 1999. Wavefront coding: A modern method of achieving high performance and/or low cost imaging systems. In SPIE Annual Meeting.Google Scholar
- Farid, H., and Simoncelli, E. 1998. Range estimation by optical differentiation. J. Opt. Soc. of America A 15, 1, 1777--1786.Google ScholarCross Ref
- Fergus, R., Torralba, A., and Freeman, W. 2006. Random lens imaging. Tech. rep., MIT.Google Scholar
- Fessenden, R. 1908. Wireless telephony. Trans. American Institute of Electrical Engineers 27, 553--629.Google ScholarCross Ref
- Field, D. 1994. What is the goal of sensory coding? Neural Comput. 6, 559--601. Google ScholarDigital Library
- Georgiev, T., Zheng, C., Nayar, S., Curless, B., Salasin, D., and Intwala, C. 2006. Spatio-angular resolution trade-offs in integral photography. In Eurographics Symposium on Rendering, 263--272. Google ScholarCross Ref
- Gortler, S., Grzeszczuk, R., Szeliski, R., and Cohen, M. 1996. The lumigraph. In SIGGRAPH, 43--54. Google ScholarDigital Library
- Gottesman, S. R., and Fenimore, E. E. 1989. New family of binary arrays for coded aperture imaging. Appl. Optics 28, 20 (Oct), 4344--4352.Google ScholarCross Ref
- Haeberli, P., 1994. A multifocus method for controlling depth of field. GraficaObscura.Google Scholar
- Hiura, S., and Matsuyama, T. 1998. Depth measurement by the multi-focus camera. In Proc. Conf. Computer Vision and Pattern Recognition, 953--961. Google ScholarDigital Library
- Imatest. Image quality evaluation software. http://www.imatest.com/.Google Scholar
- Isaksen, A., McMillan, L., and Gortler, S. 2000. Dynamically reparameterized light fields. In SIGGRAPH, 297--306. Google ScholarDigital Library
- Ives, H. 1928. Camera for making parallax panoramagrams. J. Opt. Soc. Amer. 17, 435--439.Google ScholarCross Ref
- Javidi, B., and Okano, F., Eds. 2002. Three-Dimensional Television, Video and Display Technologies. Springer-Verlag. Google ScholarDigital Library
- Johnson, G. E., Dowski, E. R., and Cathey, W. T. 2000. Passive ranging through wave-front coding: Information and application. Applied Optics 39, 1700--1710.Google ScholarCross Ref
- Levoy, M., and Hanrahan, P. 1996. Light field rendering. In SIGGRAPH 96, 31--42. Google ScholarDigital Library
- Levoy, M., Chen, B., Vaish, V., Horowitz, M., McDowall, M., and Bolas, M. 2004. Synthetic aperture confocal imaging. ACM Trans. Graph. 23, 825--834. Google ScholarDigital Library
- Lippmann, G. 1908. Epreuves reversible donnant la sensation du relief. J. Phys 7, 821--825.Google Scholar
- Lucy, L. 1974. An iterative technique for the rectification of observed distributions. J. Astronomy 79, 745--754.Google ScholarCross Ref
- Martnez-Corral, M., Javidi, B., Martnez-Cuenca, R., and Saavedra, G. 2004. Integral imaging with improved depth of field by use of amplitude-modulated microlens arrays. Applied Optics 43, 5806--5813.Google ScholarCross Ref
- Nayar, S., and Mitsunaga, T. 2000. High dynamic range imaging: spatially varying pixel exposures. In Proc. Conf. Computer Vision and Pattern Recognition, vol. 1, 472--479.Google Scholar
- Nayar, S. K., Branzoi, V., and Boult, T. E. 2006. Programmable imaging: Towards a flexible camera. Int'l J. Computer Vision 70, 1, 7--22. Google ScholarDigital Library
- Ng, R., Levoy, M., Brdif, M., Duval, G., Horowitz, M., and Hanrahan, P. 2005. Light field photography with a hand-held plenoptic camera. Tech. rep., Stanford Univ.Google Scholar
- Ng, R. 2005. Fourier slice photography. ACM Trans. Graph. 24, 735--744. Google ScholarDigital Library
- Okano, F., Hoshino, H., and Yuyama, A. 1997. Real-time pickup method for a three-dimensional image based on integral photography. Applied Optics 36, 15981603.Google ScholarCross Ref
- Okano, F., Arai, J., Hoshino, H., and Yuyama, I. 1999. Three dimensional video system based on integral photography. Optical Engineering 38, 1072--1077.Google ScholarCross Ref
- Oppenheim, A. V., Schafer, R. W., and Buck, J. R. 1999. Discrete-Time Signal Processing. Prentice-Hall. Google ScholarDigital Library
- Raskar, R., Agrawal, A., and Tumblin, J. 2006. Coded exposure photography: motion deblurring using fluttered shutter. ACM Trans. Graph. 25, 3, 795--804. Google ScholarDigital Library
- Richardson, W. 1972. Bayesian-based iterative method of image restoration. J. Opt. Soc. of Am. 62, 1, 55--59.Google ScholarCross Ref
- Skinner, G. K. 1988. X-Ray Imaging with Coded Masks. Scientific American 259 (Aug.), 84.Google ScholarCross Ref
- Sun, W., and Barbastathis, G. 2005. Rainbow volume holographic imaging. Opt. Lett. 30, 976--978.Google ScholarCross Ref
- Vaish, V., Wilburn, B., Joshi, N., and Levoy, M. 2004. Using plane + parallax for calibrating dense camera arrays. In Proc. Conf. Computer Vision and Pattern Recognition, 2--9.Google Scholar
- van der Gracht, J., Dowski, E., Taylor, M., and Deaver, D. 1996. Broadband behavior of an optical-digital focus-invariant system. Optics Letters 21, 13 (July), 919--921.Google ScholarCross Ref
- Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., and Tumblin, J. 2007. Non-refractive modulators for coding and capturing scene appearance. Tech. Rep. UMIACS-TR-2007-21, Univ. of Maryland.Google Scholar
- Wang, S., and Heidrich, W. 2004. The design of an inexpensive very high resolution scan camera system. Eurographics 23, 441--450.Google ScholarCross Ref
- Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., and Levoy, M. 2005. High performance imaging using large camera arrays. ACM Trans. Graph. 24, 3, 765--776. Google ScholarDigital Library
- Yang, J. C. 2000. A Light Field Camera For Image Based Rendering. Master's thesis, Massachussettes Institute of Technology.Google Scholar
- Zomet, A., and Nayar, S. 2006. Lensless imaging with a controllable aperture. In Proc. Conf. Computer Vision and Pattern Recognition, 339--346. Google ScholarDigital Library
Index Terms
- Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing
Recommendations
Programmable aperture photography: multiplexed light field acquisition
In this paper, we present a system including a novel component called programmable aperture and two associated post-processing algorithms for high-quality light field acquisition. The shape of the programmable aperture can be adjusted and used to ...
Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing
SIGGRAPH '07: ACM SIGGRAPH 2007 papersWe describe a theoretical framework for reversibly modulating 4D light fields using an attenuating mask in the optical path of a lens based camera. Based on this framework, we present a novel design to reconstruct the 4D light field from a 2D camera ...
Dual photography
SIGGRAPH '05: ACM SIGGRAPH 2005 PapersWe present a novel photographic technique called dual photography, which exploits Helmholtz reciprocity to interchange the lights and cameras in a scene. With a video projector providing structured illumination, reciprocity permits us to generate ...
Comments