Marc Levoy
Mark Horowitz
Abstract
By inserting a microlens array into the optical train of a conventional microscope, one can capture light fields of biological specimens in a single photograph. Although diffraction places a limit on the product of spatial and angular resolution in these light fields, we can nevertheless produce useful perspective views and focal stacks from them. Since microscopes are inherently orthographic devices, perspective views represent a new way to look at microscopic specimens. The ability to create focal stacks from a single photograph allows moving or light-sensitive specimens to be recorded. Applying 3D deconvolution to these focal stacks, we can produce a set of cross sections, which can be visualized using volume rendering. In this paper, we demonstrate a prototype light field microscope (LFM), analyze its optical performance, and show perspective views, focal stacks, and reconstructed volumes for a variety of biological specimens. We also show that synthetic focusing followed by 3D deconvolution is equivalent to applying limited-angle tomography directly to the 4D light field.
Supplemental Material
- Adelson, T., Wang, J. Y. A. 1992. Single lens stereo with a plenoptic camera. IEEE Transactions on Pattern Analysis and Machine Intelligence 14, 2, 99--106. Google Scholar
Digital Library
- Agard, D. A. 1984. Optical sectioning microscopy: Cellular architecture in three dimensions. Ann. Rev. Biophys. Bioeng 13, 191--219.Google ScholarCross Ref
- Andersen, A. H., Kak, A. C., 1984. Simultaneous algebraic reconstruction technique (SART): A superior implementation of the ART algorithm. Ultrasonic Imaging 6, 81--94.Google ScholarCross Ref
- Arridge, S. R. 2001. Methods for the inverse problem in optical tomography. Proc. Waves and Imaging Through Complex Media. Kluwer, 307--329.Google ScholarCross Ref
- Castleman, K. R. 1979. Digital Image Processing. Prentice Hall. Google ScholarDigital Library
- Chamgoulov, R. O., Lane, P. M., Macaulay, C. E. 2004. Optical computed-tomography microscope using digital spatial light modulation. Proc. SPIE 5324, 182--190.Google Scholar
- Colsher, J. G. 1980. Fully three-dimensional positron emission tomography. Phys. Med. Biol. 25, 1, 103--115.Google ScholarCross Ref
- Corle, T. R., Kino, G. S. 1996. Confocal Scanning Optical Microscopy and Related Imaging Systems. Academic Press.Google Scholar
- Ellis, G. W. 1966. Holomicrography: transformation of image during reconstruction a posteriori. Science 143, 1195--1196.Google ScholarCross Ref
- Goldberg, N. 1992. Camera technology: the dark side of the lens. Academic Press.Google Scholar
- Goodman, J. 1996. Introduction to Fourier optics. 2nd edition, McGraw-Hill.Google Scholar
- Gustafsson, M. G. L. 2005. Nonlinear structured-illumination microscopy: Wide-field fluorescence imaging with theoretically unlimited resolution. Proc. National Academy of Sciences 102, 37.Google ScholarCross Ref
- Holmes, T. J., Bhattacharyya, S., et al. 1995. Light microscopic images reconstructed by maximum likelihood deconvolution. In Handbook of Biological Confocal Microscopy, ed. J. B. Pawley, Plenum Press, 389--402.Google Scholar
- Inoue, S., Oldenbourg, R. 1995. Microscopes. In Handbook of Optics, 2nd edition, McGraw-Hill.Google Scholar
- Inoue, S. and Spring, K. R. 1997. Video Microscopy. 2nd edition, Plenum Press.Google Scholar
- Gabor, D. 1948. A new microscopic principle. Nature 161, 777--778.Google ScholarCross Ref
- Isaksen, A., Mcmillan, L., Gortler, S. J. 2000. Dynamically reparameterized light fields. Proc. SIGGRAPH 2000. Google ScholarDigital Library
- Javidi, B., Okano, F., eds. 2002. Three-Dimensional Television, Video and Display Technologies. Springer-Verlag. Google ScholarDigital Library
- Kak, A. C., Slaney, M. 1988. Principles of Computerized Tomographic Imaging. IEEE Press.Google Scholar
- Kawata, S., Nakamura, 0., Minami, S. 1987. Optical microscope tomography. I. Support constraint. J. Opt. Soc. Am. A 4, 1, 292--297.Google ScholarCross Ref
- Kingslake, R. 1983. Optical system design. Academic Press.Google Scholar
- Levoy, M., Hanrahan, P. 1996. Light field rendering. Proc. SIGGRAPH 1996. Google ScholarDigital Library
- Levoy, M., Chen, B., Vaish, V., Horowitz, M., Mcdowall, I., Bolas, M. 2004. Synthetic aperture confocal imaging. ACM Transactions on Graphics (Proc. SIGGRAPH) 23, 3, 825--834. Google ScholarDigital Library
- Markham, J., Conchello, J.-A. 2001. Artefacts in restored images due to intensity loss in three-dimensional fluorescence microscopy. J. Microscopy 204, 2, 93--98.Google ScholarCross Ref
- Mcnally, J. G., Preza, C., Conchello, J. A., Thomas, L. J. Jr. 1994. Artifacts in computational optical-sectioning microscopy. J. Opt. Soc. Am. A 11, 3, 1056--67.Google ScholarCross Ref
- Nayar, S. K., Nakagawa, Y. 1990. Shape from focus: An effective approach for rough surfaces. Proc. International Conference on Robotics and Automation (ICRA), Vol. 2, 218--225.Google ScholarCross Ref
- Nayar, S. K., Narasimhan, S. G. 2002. Assorted pixels: Multi-sampled imaging with structural models. Proc. ECCV. Google ScholarDigital Library
- Ng, R. 2005. Fourier slice photography. ACM Transactions on Graphics (Proc. SIGGRAPH) 24, 3, 735--744. Google ScholarDigital Library
- Ng, R., Levoy, M., Bredif, M., Duval, G., Horowitz, M., Hanrahan, P. 2005. Light Field Photography with a Hand-Held Plenoptic Camera. Stanford Tech Report CTSR 2005-02.Google Scholar
- Ng, R. 2006. Digital Light Field Photography. PhD dissertation, Stanford University. Google ScholarDigital Library
- Noguchi, M., Nayar, S. 1994. Microscopic shape from focus using active illumination. Proc. IAPR International Conference on Pattern Recognition (ICPR), Vol. A, 147--152.Google ScholarCross Ref
- Okoshi, T. 1976. Three-Dimensional Imaging Techniques. Academic Press.Google Scholar
- Piller, H. 1977. Microscope Photometry. Springer-Verlag.Google Scholar
- Pluta, M. 1988. Advanced Light Microscopy (in 3 volumes). Elsevier.Google Scholar
- Schechner, Y., Kiryati, N. 1999. The optimal axial interval for estimating depth from defocus. Proc. ICCV. Google ScholarDigital Library
- Schechner, Y., Kiryati, N. 2000. Depth from defocus vs. stereo: How different really are they? IJCV 39, 2, 141--162. Google ScholarDigital Library
- Schechner, Y. Y., Kiryati, N., Basri, R. 2000. Separation of transparent layers using focus. IJCV 39, 1, 25--39. Google ScholarDigital Library
- Schechner, Y., Nayar, S. 2001. Generalized Mosaicing. Proc. ICCV.Google Scholar
- Shah, U. B., Nayar, S. K. 1992. Extracting 3-D structure and focused images using an optical microscope. Proc. IEEE Symposium on Computer-Based Medical Systems.Google ScholarCross Ref
- Streibl, N. 1984. Depth transfer by an imaging system. Optica Acta 31, 11, 1233--1241.Google ScholarCross Ref
- Streibl, N. 1985. Three-dimensional imaging by a microscope. J. Opt. Soc. Am. A 2, 2, 121--127.Google ScholarCross Ref
- Swedlow, J. R., Sedat, J. W., Agard, D. A. 1997. Deconvolution in optical microscopy. In Deconvolution of Images and Spectra, ed. P. A. Jansson, Academic Press, 284--309. Google ScholarDigital Library
- Vaish, V., Garg, G., Talvala, E., Antunez, E., Wilburn, B., Horowitz, M., Levoy, M., Synthetic aperture focusing using a shear-warp factorization of the viewing transform. Proc. Workshop on Advanced 3D Imaging for Safety and Security, in conjunction with CVPR 2005. Google ScholarDigital Library
- Weinstein, R. S., Descour, M. R., et al. 2004. An array microscope for ultrarapid virtual slide processing and telepathology. Design, fabrication, and validation study. Human Pathology 35, 11, 1303--1314.Google ScholarCross Ref
Index Terms
- Light field microscopy
Recommendations
Light field microscopy
SIGGRAPH '06: ACM SIGGRAPH 2006 PapersBy inserting a microlens array into the optical train of a conventional microscope, one can capture light fields of biological specimens in a single photograph. Although diffraction places a limit on the product of spatial and angular resolution in ...
Light-field supported fast volume rendering
SIGGRAPH '12: ACM SIGGRAPH 2012 PostersWe present a combination of light-field rendering and volume rendering to enable the interactive exploration of large volumetric data sets. We recycle previously rendered images and use the idle times of the volume renderer for filling a cached-managed ...
Dynamically reparameterized light fields
SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniquesThis research further develops the light field and lumigraph image-based rendering methods and extends their utility. We present alternate parameterizations that permit 1) interactive rendering of moderately sampled light fields of scenes with ...
Comments