Abstract
We present a variational framework to generate super-resolved novel views from 4D light field data sampled at low resolution, for example by a plenoptic camera. In contrast to previous work, we formulate the problem of view synthesis as a continuous inverse problem, which allows us to correctly take into account foreshortening effects caused by scene geometry transformations. High-accuracy depth maps for the input views are locally estimated using epipolar plane image analysis, which yields floating point depth precision without the need for expensive matching cost minimization. The disparity maps are further improved by increasing angular resolution with synthesized intermediate views. Minimization of the super-resolution model energy is performed with state of the art convex optimization algorithms within seconds.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Bolles, R., Baker, H., Marimont, D.: Epipolar-plane image analysis: An approach to determining structure from motion. International Journal of Computer Vision 1, 7–55 (1987) 1, 6
Authors: -. In: Additional Material, reference_2.pdf (2012, under review) 1, 3, 6, 8, 9, 10, 11, 12
Vaish, V., Wilburn, B., Joshi, N., Levoy, M.: Using plane + parallax for calibrating dense camera arrays. In: Proc. International Conference on Computer Vision and Pattern Recognition (2004) 1
Levoy, M., Hanrahan, P.: Light field rendering. In: Proc. SIGGRAPH, pp. 31–42 (1996) 1, 2, 8
Perwass, C., Wietzke, L.: The next generation of photography (2010) 1, www.raytrix.de
Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., Hanrahan, P.: Light field photography with a hand-held plenoptic camera. Technical Report CSTR 2005-02, Stanford University (2005) 1
Bishop, T., Favaro, P.: Full-Resolution Depth Map Estimation from an Aliased Plenoptic Light Field. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part II. LNCS, vol. 6493, pp. 186–200. Springer, Heidelberg (2011) 1, 2
Georgiev, T., Lumsdaine, A.: Focused plenoptic camera and rendering. Journal of Electronic Imaging 19, 021106 (2010) 1
McMillan, L., Bishop, G.: Plenoptic modeling: An image-based rendering system. In: Proc. SIGGRAPH, pp. 39–46 (1995) 1, 2
Protter, M., Elad, M.: Super-resolution with probabilistic motion estimation. IEEE Trans. Image Processing 18, 1899–1904 (2009) 1, 2, 8
Goldluecke, B., Cremers, D.: Superresolution texture maps for multiview reconstruction. In: Proc. ICCV (2009) 1, 2, 3, 5
Adelson, E., Bergen, J.: The plenoptic function and the elements of early vision. Computational Models of Visual Processing 1 (1991) 2
Gortler, S., Grzeszczuk, R., Szeliski, R., Cohen, M.: The Lumigraph. In: Proc. SIGGRAPH, pp. 43–54 (1996) 2
Shum, H., Chan, S., Kang, S.: Image-based rendering. Springer, New York (2007) 2
Kubota, A., Aizawa, K., Chen, T.: Reconstructing dense light field from array of multifocus images for novel view synthesis. IEEE Trans. Image Processing 16, 269–279 (2007) 2, 8
Chai, J.X., Tong, X., Chany, S.C., Shum, H.Y.: Plenoptic sampling. In: Proc. SIGGRAPH, pp. 307–318 (2000) 2, 4
Siu, A., Lau, E.: Image registration for image-based rendering. IEEE Trans. Image Processing 14, 241–252 (2005) 2
Geys, I., Koninckx, T.P., Gool, L.V.: Fast interpolated cameras by combining a GPU based plane sweep with a max-flow regularisation algorithm. In: 3DPVT, pp. 534–541 (2004) 2, 8
Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Trans. on Pattern Analysis and Machine Intelligence 24, 1167–1183 (2002) 3, 4
Beck, A., Teboulle, M.: Fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sciences 2, 183–202 (2009) 8, 9
Bermùdez, A., Moreno, C.: Duality methods for solving variational inequalities. Comp. and Maths. with Appls. 7, 43–58 (1981) 9
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wanner, S., Goldluecke, B. (2012). Spatial and Angular Variational Super-Resolution of 4D Light Fields. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds) Computer Vision – ECCV 2012. ECCV 2012. Lecture Notes in Computer Science, vol 7576. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33715-4_44
Download citation
DOI: https://doi.org/10.1007/978-3-642-33715-4_44
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33714-7
Online ISBN: 978-3-642-33715-4
eBook Packages: Computer ScienceComputer Science (R0)