Matthew O'Toole
Skip Abstract Section
Skip Supplemental Material SectionAbstract
We present a general framework for analyzing the transport matrix of a real-world scene at full resolution, without capturing many photos. The key idea is to use projectors and cameras to directly acquire eigenvectors and the Krylov subspace of the unknown transport matrix. To do this, we implement Krylov subspace methods partially in optics, by treating the scene as a "black box subroutine" that enables optical computation of arbitrary matrix-vector products. We describe two methods---optical Arnoldi to acquire a low-rank approximation of the transport matrix for relighting; and optical GMRES to invert light transport. Our experiments suggest that good quality relighting and transport inversion are possible from a few dozen low-dynamic range photos, even for scenes with complex shadows, caustics, and other challenging lighting effects.
Supplemental Material
Available for Download
- Amano, T., and Kato, H. 2008. Real world dynamic appearance enhancement with procam feedback. Proc. IEEE PRO-CAMS. Google Scholar
Digital Library
- Ambs, P. 2009. A short history of optical computing: rise, decline, and evolution. Proc. SPIE 7388.Google ScholarCross Ref
- Athale, R. A., and Collins, W. C., 1982. Optical matrixmatrix multiplier based on outer product decomposition.Google Scholar
- Bai, J., Chandraker, M., Ng, T.-T., and Ramamoorthi, R. 2010. A dual theory of inverse and forward light transport. Proc. ECCV. Google ScholarDigital Library
- Basri, R., and Jacobs, D. 2003. Lambertian reflectance and linear subspaces. IEEE T-PAMI 25, 2, 218--233. Google ScholarDigital Library
- Casasent, D. 1981. Hybrid processors. Optical Information Processing Fundamentals, 181--233.Google Scholar
- Cederquist, J., and Lee, S. 1979. The use of feedback in optical information processing. Applied Physics A: Materials Science & Processing.Google Scholar
- Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. Proc. SIGGRAPH. Google ScholarDigital Library
- Fuchs, M., Blanz, V., Lensch, H., and Seidel, H.-P. 2007. Adaptive sampling of reflectance fields. ACM TOG 26, 2 (Jun). Google ScholarDigital Library
- Garg, G., Talvala, E., Levoy, M., and Lensch, H. 2006. Symmetric photography: Exploiting data-sparseness in reflectance fields. Proc. Eurographics Symp. Rendering. Google ScholarDigital Library
- Goodman, J. W. 2005. Introduction to fourier optics, 3rd edition. Roberts & Company Publishers.Google Scholar
- Greenbaum, A., and Trefethen, L. N. 1994. Gmres/cr and arnoldi/lanczos as matrix approximation problems. SIAM J. Sci. Comput. 15, 2, 359--368. Google ScholarDigital Library
- Grossberg, M., Peri, H., Nayar, S., and Belhumeur, P. 2004. Making one object look like another: controlling appearance using a projector-camera system. Proc. CVPR, 452--459.Google Scholar
- Guilfoyle, P., and Stone, R. 1991. Digital optical computer ii. Proc. SPIE 1563, 214--222.Google Scholar
- Gutknecht, M. H. 2007. Block krylov space methods for linear systems with multiple right-hand sides: an introduction. In Modern Mathematical Models, Methods and Algorithms for Real World Systems. 420--447.Google Scholar
- Koenderink, J. J., and van Doorn, A. J. 1983. Geometrical modes as a general method to treat diffuse interreflections in radiometry. J. Opt. Soc. Am 73, 6, 843--850.Google ScholarCross Ref
- Kumar, B. V. K. V., and Casasent, D. 1981. Eigenvector determination by iterative optical methods. Applied Optics 20, 21, 3707--3710.Google ScholarCross Ref
- Langer, M. 1999. When shadows become interreflections. Int. J. Computer Vision 34, 2, 193--204. Google ScholarDigital Library
- Larsen, R. M. http://soi.stanford.edu/ rmunk/propack/.Google Scholar
- Leith, E. 2000. The evolution of information optics. IEEE J. Select Topics in Quantum Electronics 6, 6, 1297--1304.Google ScholarCross Ref
- Liesen, J., and Tichy, P. 2004. Convergence analysis of krylov subspace methods. GAMM Mitt. Ges. Angew. Math. Mech. 27, 2, 153--173 (2005).Google ScholarCross Ref
- Mahajan, D., Shlizerman, I., Ramamoorthi, R., and Belhumeur, P. 2007. A theory of locally low dimensional light transport. Proc. SIGGRAPH. Google ScholarDigital Library
- Matusik, W., Pfister, H., Ziegler, R., and Ngan, A. 2002. Acquisition and rendering of transparent and refractive objects. Proc. Eurographics Symp. on Rendering. Google ScholarDigital Library
- Ng, R., Ramamoorthi, R., and Hanrahan, P. 2003. All-frequency shadows using non-linear wavelet lighting approximation. Proc. SIGGRAPH. Google ScholarDigital Library
- Ng, T., Pahwa, R., Bai, J., Quek, T., and Tan, K. 2009. Radiometric compensation using stratified inverses. Proc. ICCV.Google Scholar
- Peers, P., and Dutré, P. 2003. Wavelet environment matting. Proc. Eurographics Symp. on Rendering. Google ScholarDigital Library
- Peers, P., and Dutré, P. 2005. Inferring reflectance functions from wavelet noise. Proc. Eurographics Symp. Rendering. Google ScholarDigital Library
- Peers, P., Mahajan, D., Lamond, B., Ghosh, A., Matusik, W., Ramamoorthi, R., and Debevec, P. 2009. Compressive light transport sensing. ACM TOG 28, 1. Google ScholarDigital Library
- Psaltis, D., and Athale, R. A. 1986. High accuracy computation with linear analog optical systems: a cricitcal study. Applied Optics 25, 18, 3071--3077.Google ScholarCross Ref
- Rajbenbach, H., Fainman, Y., and Lee, S. H. 1987. Optical implementation of an interative algorithm for matrix-inversion. Applied Optics 26, 6, 1024--1031.Google ScholarCross Ref
- Saad, Y. 2003. Iterative methods for sparse linear systems. Google ScholarDigital Library
- Salvi, J., Pages, J., and Batlle, J. 2004. Pattern codification strategies in structured light systems. Pattern Recogn 37, 4, 827--849.Google ScholarCross Ref
- Schechner, Y., Nayar, S., and Belhumeur, P. 2007. Multiplexing for optimal lighting. IEEE T-PAMI 29, 8, 1339--1354. Google ScholarDigital Library
- Seitz, S., Matsushita, Y., and Kutulakos, K. 2005. A theory of inverse light transport. Proc. ICCV, 1440--1447. Google ScholarDigital Library
- Sen, P., and Darabi, S. 2009. Compressive dual photography. Proc. Eurographics.Google Scholar
- Sen, P., Chen, B., Garg, G., Marschner, S., Horowitz, M., Levoy, M., and Lensch, H. 2005. Dual photography. Proc. SIGGRAPH. Google ScholarDigital Library
- Simon, H. D., and Zha, H. 2000. Low-rank matrix approximation using the lanczos bidiagonalization process with applications. SIAM J. Sci. Comput. 21, 6, 2257--2274. Google ScholarDigital Library
- Simoncini, V., and Szyld, D. B. 2003. Theory of inexact krylov subspace methods and applications to scientific computing. SIAM J. Sci. Comput. 25, 2, 454--477. Google ScholarDigital Library
- Trefethen, L. N., and Bau, I. 1997. Numerical linear algebra. SIAM, xii+361.Google Scholar
- Wang, J., Dong, Y., Tong, X., Lin, Z., and Guo, B. 2009. Kernel nyström method for light transport. Proc. SIGGRAPH. Google ScholarDigital Library
- Wang, O., Fuchs, M., Fuchs, C., Davis, J., Seidel, H.-P., and Lensch, H. P. A. 2010. A context-aware light source. Proc. ICCP.Google Scholar
- Wetzstein, G., and Bimber, O. 2007. Radiometric compensation through inverse light transport. Pacific Graphics, 391--399. Google ScholarDigital Library
- Zhang, L., and Nayar, S. 2006. Projection defocus analysis for scene capture and image display. Proc. SIGGRAPH. Google ScholarDigital Library
Recommendations
Optical computing for fast light transport analysis
SIGGRAPH ASIA '10: ACM SIGGRAPH Asia 2010 papersWe present a general framework for analyzing the transport matrix of a real-world scene at full resolution, without capturing many photos. The key idea is to use projectors and cameras to directly acquire eigenvectors and the Krylov subspace of the ...
Comments