Donald G. Dansereau
Skip Abstract Section
Skip Supplemental Material SectionAbstract
We demonstrate that the redundant information in light field imagery allows volumetric focus, an improvement of signal quality that maintains focus over a controllable range of depths. To do this, we derive the frequency-domain region of support of the light field, finding it to be the 4D hyperfan at the intersection of a dual fan and a hypercone, and design a filter with correspondingly shaped passband. Drawing examples from the Stanford Light Field Archive and images captured using a commercially available lenslet-based plenoptic camera, we demonstrate that the hyperfan outperforms competing methods including planar focus, fan-shaped antialiasing, and nonlinear image and video denoising techniques. We show the hyperfan preserves depth of field, making it a single-step all-in-focus denoising filter suitable for general-purpose light field rendering. We include results for different noise types and levels, through murky water and particulate matter, in real-world scenarios, and evaluated using a variety of metrics. We show that the hyperfan's performance scales with aperture count, and demonstrate the inclusion of aliased components for high-quality rendering.
Supplemental Material
- E. H. Adelson and J. Y. A. WANG. 2002. Single lens stereo with a plenoptic camera. IEEE Trans. Pattern Anal. Mach. Intell. 14, 2, 99--106. Google Scholar
Digital Library
- A. Agrawal, Y. Xu, and R. Raskar. 2009. Invertible motion blur in video. ACM Trans. Graph. 28, 3. Google ScholarDigital Library
- M. Aharon, M. Elad, and A. Bruckstein. 2006. K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54, 11, 4311--4322. Google ScholarDigital Library
- R. Ansari. 1987. Efficient IIR and FIR fan filters. IEEE Trans. Circ. Syst. 34, 8, 941--945.Google ScholarCross Ref
- J. Berent and P. L. Dragotti. 2007. Plenoptic manifolds. Signal Process. Mag. 24, 6, 34--44.Google ScholarCross Ref
- T. E. Bishop and P. Favaro. 2012. The light field camera: Extended depth of field, aliasing, and superresolution. IEEE Trans. Pattern Anal. Mach. Intell. 34, 5, 972--986. Google ScholarDigital Library
- K. Bitsakos and C. Fermuller. 2006. Depth estimation using the compound eye of dipteran flies. Biol. Cybernet. 95, 5, 487--501. Google ScholarDigital Library
- R. Bolles, H. Baker, and D. Marimont. 1987. Epipolar-plane image analysis: An approach to determining structure from motion. Intl. J. Comput. Vis. 1, 1, 7--55.Google ScholarCross Ref
- A. Buades, B. Coll, and J.-M. Morel. 2005. A review of image denoising algorithms, with a new one. SIAM J. Multiscale Model. Simul. 4, 2, 490--530.Google ScholarCross Ref
- A. Cetin, O. Gerek, and Y. Yardimci. 1997. Equiripple FIR filter design by the FFT algorithm. IEEE Signal Process. Mag. 14, 2, 60--64.Google ScholarCross Ref
- J. Chai, X. Tong, S. Chan, and H. Shum. 2000. Plenoptic sampling. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'00). 307--318. Google ScholarDigital Library
- S.-C. Chan and H.-Y. Shum. 2000. A spectral analysis for light field rendering. In Proceedings of the International Conference on Image Processing (ICIP'00). Vol. 2, 25--28.Google ScholarCross Ref
- O. Cossairt, M. Gupta, and S. Nayar. 2012. When does computational imaging improve performance? IEEE Trans. Image Process. 22, 2, 447--458. Google ScholarDigital Library
- K. Dabov, A. Foi, and K. Egiazarian. 2007. Video denoising by sparse 3D transform-domain collaborative filtering. In Proceedings of the 15th European Signal Processing Conference (EURIPCO'07). 7.Google Scholar
- K. J. Dana, B. Van Ginneken, S. K. Nayar, and J. J. Koenderink. 1999. Reflectance and texture of real-world surfaces. ACM Trans. Graph. 18, 1, 1--34. Google ScholarDigital Library
- D. G. Dansereau. 2003. 4D light field processing and its application to computer vision. M.S. thesis, Department of Electrical and Computer Engineering, University of Calgary. http://www-personal.acfr.usyd.edu.au/ddan1654/dansereau2003MscThesis.pdf.Google Scholar
- D. G. Dansereau. 2014. Plenoptic signal processing for robust vision in field robotics. Ph.D. thesis, Australian Centre for Field Robotics, School of Aerospace, Mechanical and Mechatronic Engineering, The University of Sydney.Google Scholar
- D. G. Dansereau, D. L. Bongiorno, O. Pizarro, and S. B. Williams. 2013a. Light field image denoising using a linear 4D frequency-hyperfan all-in-focus filter. In Proceedings of the SPIE Conference on Computational Imaging (SPIE'13). Vol. 8657.Google Scholar
- D. G. Dansereau, O. Pizarro, and S. B. Williams. 2013b. Decoding, calibration and rectification for lenselet-based plenoptic cameras. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'13). 1027--1034. Google ScholarDigital Library
- D. G. Dansereau and L. T. Bruton. 2003. A 4D frequency-planar IIR filter and its application to light field processing. In Proceedings of the International Symposium on Circuits and Systems (ISCAS'03). Vol. 4. 476--479.Google Scholar
- D. G. Dansereau and L. T. Bruton. 2004. Gradient-based depth estimation from 4D light fields. In Proceedings of the International Symposium on Circuits and Systems (ISCAS'04). Vol. 3, 549--552.Google Scholar
- D. G. Dansereau and L. T. Bruton. 2007. A 4-D dual-fan filter bank for depth filtering in light fields. IEEE Trans. Signal Process. 55, 2, 542--549. Google ScholarCross Ref
- D. G. Dansereau, I. Mahon, O. Pizarro, and S. B. Williams. 2011. Plenoptic flow: Closed-form visual odometry for light field cameras. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'11). 4455--4462.Google Scholar
- D. G. Dansereau and S. B. Williams. 2011. Seabed modeling and distractor extraction for mobile AUVS using light field filtering. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA'11). 1634--1639.Google Scholar
- F. Durand, N. Holzschuch, C. Soler, E. Chan, and F. Sillion. 2005. A frequency analysis of light transport. In Proceedings of the 32nd International Conference on Computer Graphics and Interactive Techniques: Papers (SIGGRAPH'05). 1115--1126. Google ScholarDigital Library
- M. Elad and M. Aharon. 2006. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15, 12, 3736--3745. Google ScholarDigital Library
- W. Freeman, A. Levin, S. Hasinoff, P. Green, and F. Durand. 2009. 4D frequency analysis of computational cameras for depth of field extension. Tech. rep. MIT-CSAIL-TR-2009-019. http://dspace.mit.edu/handle/1721.1/45513.Google Scholar
- W. S. Geisler. 2008. Visual perception and the statistical properties of natural scenes. Ann. Rev. Psychol. 59, 167--192.Google ScholarCross Ref
- B. Goldluecke and S. Wanner. 2013. The variational structure of disparity and regularization of 4D light fields. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'13). 1003--1010. Google ScholarDigital Library
- X. Gu, S. J. Gortler, and M. F. Cohen. 1997. Polyhedral geometry and the two-plane parameterization. In Proceedings of the Eurographics Workshop on Rendering Techniques. Springer, 1--12. Google ScholarDigital Library
- O. G. Guleryuz. 2007. Weighted averaging for denoising with overcomplete dictionaries. IEEE Trans. Image Process. 16, 12, 3020--3034. Google ScholarDigital Library
- A. Isaksen, L. McMillan, and S. Gortler. 2000. Dynamically reparameterized light fields. In Proceedings of the International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'00). 297--306. Google ScholarDigital Library
- Y. Ji, J. Ye, and J. Yu. 2013. Reconstructing gas flows using light-path approximation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'13). 2507--2514. Google ScholarDigital Library
- J.-H. Lambert. 1760. Photometria, sive de Mensura et gradibus luminis, colorum et umbrae. Eberhard Klett.Google Scholar
- A. Levin and F. Durand. 2010. Linear view synthesis using a dimensionality gap light field prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'10). 1831--1838.Google Scholar
- A. Levin, S. Hasinoff, P. Green, F. Durand, and W. Freeman. 2009. 4D frequency analysis of computational cameras for depth of field extension. ACM Trans. Graph. 28, 3. Google ScholarDigital Library
- M. Levoy, B. Chen, V. Vaish, M. Horowitz, I. McDowall, and M. Bolas. 2004. Synthetic aperture confocal imaging. ACM Trans. Graph. 23, 3, 825--834. Google ScholarDigital Library
- M. Levoy and P. Hanrahan. 1996. Light field rendering. In Proceedings of the International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'96). 31--42. Google ScholarDigital Library
- A. Lumsdaine and T. Georgiev. 2008. Full resolution lightfield rendering. Tech. rep., Adobe Systems. http://tgeorgiev.net/FullResolution.pdf.Google Scholar
- A. Lumsdaine and T. Georgiev. 2009. The focused plenoptic camera. In Proceedings of the IEEE International Conference on Computational Photography (ICCP'09). 1--8.Google Scholar
- K. Maeno, H. Nagahara, A. Shimada, and R.-I. Taniguchi. 2013. Light field distortion feature for transparent object recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'13). 2786--2793. Google ScholarDigital Library
- K. Mitra, O. Cossairt, and A. Veeraraghavan. 2013. A frame-work for the analysis of computational imaging systems with practical applications. http://arxiv.org/pdf/1308.1981.pdf.Google Scholar
- H. Nagahara, S. Kuthirummal, C. Zhou, and S. K. Nayar. 2008. Flexible depth of field photography. In Proceedings of the European Conference on Computer Vision (ECCV'08). Springer, 60--73. Google ScholarDigital Library
- J. Neumann, C. Fermuller, Y. Aloimonos, and V. Brajovic. 2005. Compound eye sensor for 3D ego motion estimation. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'05). Vol. 4. 3712--3717.Google Scholar
- R. Ng. 2005. Fourier slice photography. ACM Trans. Graph. 24, 3, 735--744. Google ScholarDigital Library
- R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan. 2005. Light field photography with a hand-held plenoptic camera. Tech. rep. CSTR 2, Department of Computer Science, Stanford University. https://graphics.stanford.edu/papers/lfcamera/lfcamera-150dpi.pdf.Google Scholar
- M. O'TOOLE, R. Raskar, and K. N. Kutulakos. 2012. Primal-dual coding to probe light transport. In Proceedings of the International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'12). Vol. 31. 39. Google ScholarDigital Library
- J. G. Proakis and M. Salehi. 2007. Digital Communications 5th Ed. McGraw-Hill.Google Scholar
- L. R. Rabiner and B. Gold. 1975. Theory and Application of Digital Signal Processing, Volume 1. Prentice-Hall.Google Scholar
- R. Raskar, A. Agrawal, and J. Tumblin. 2006. Coded exposure photography: Motion deblurring using fluttered shutter. ACM Trans. Graph. 25, 3, 795--804. Google ScholarDigital Library
- R. Raskar, A. Agrawal, C. A. Wilson, and A. Veeraraghavan. 2008. Glare aware photography: 4D ray sampling for reducing glare effects of camera lenses. ACM Trans. Graph. 27, 3. Google ScholarDigital Library
- D. L. Ruderman. 1997. Origins of scaling in natural images. Vis. Res. 37, 23, 3385--3398.Google ScholarCross Ref
- A. Shnayderman, A. Gusev, and A. Eskicioglu. 2006. An SVD-based grayscale image quality measure for local and global assessment. IEEE Trans. Image Process. 15, 2, 422--429. Google ScholarDigital Library
- J. Stewart, J. Yu, S. J. Gortler, and L. McMillan. 2003. A new reconstruction filter for undersampled light fields. In Proceedings of the 14th Eurographics Workshop on Rendering (EGRW'03). Eurographics Association, 150--156. Google ScholarDigital Library
- V. Vaish, M. Levoy, R. Szeliski, C. Zitnick, and S. Kang. 2006. Reconstructing occluded surfaces using synthetic apertures: Stereo, focus and robust measures. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'06). Vol. 2. 2331--2338. Google ScholarDigital Library
- A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, and J. Tumblin. 2007. Dappled photography: Mask enhanced cameras for heterodyned light fields and coded aperture refocusing. ACM Trans. Graph. 26, 3. Google ScholarDigital Library
- Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13, 4, 600--612. Google ScholarDigital Library
- S. Wanner and B. Goldluecke. 2013. Variational light field analysis for disparity estimation and super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36, 3, 606--619. Google ScholarDigital Library
- B. Wilburn, N. Joshi, V. Vaish, E. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy. 2005. High performance imaging using large camera arrays. ACM Trans. Graph. 24, 3, 765--776. Google ScholarDigital Library
- H. Yang, M. Pollefeys, G. Welch, J. Frahm, and A. Ilie. 2007. Differential camera tracking through linearizing the local appearance manifold. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'07). 1--8.Google Scholar
- Z. Yu, X. Guo, X. Chen, and Y. Yu. 2013. Catadioptric array photography for low light imaging. In Proceedings of the 2nd IEEE International Workshop on Computational Cameras and Displays (CCD'13).Google Scholar
Index Terms
- Linear Volumetric Focus for Light Field Cameras
Recommendations
Hand-held 3D light field photography and applications
We propose a method to acquire 3D light fields using a hand-held camera, and describe several computational photography applications facilitated by our approach. As our input we take an image sequence from a camera translating along an approximately ...
Axial-cones: modeling spherical catadioptric cameras for wide-angle light field rendering
Catadioptric imaging systems are commonly used for wide-angle imaging, but lead to multi-perspective images which do not allow algorithms designed for perspective cameras to be used. Efficient use of such systems requires accurate geometric ray modeling ...
Axial-cones: modeling spherical catadioptric cameras for wide-angle light field rendering
SIGGRAPH ASIA '10: ACM SIGGRAPH Asia 2010 papersCatadioptric imaging systems are commonly used for wide-angle imaging, but lead to multi-perspective images which do not allow algorithms designed for perspective cameras to be used. Efficient use of such systems requires accurate geometric ray modeling ...
Comments