Top

Published in:

2018 | OriginalPaper | Chapter

Inverse Lightfield Rendering for Shape, Reflection and Natural Illumination

Authors : Antonin Sulc, Ole Johannsen, Bastian Goldluecke

Published in: Energy Minimization Methods in Computer Vision and Pattern Recognition

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

We propose an inverse rendering model for light fields to recover surface normals, depth, reflectance and natural illumination. Our setting is fully uncalibrated, with the reflectance modeled with a spatially-constant Blinn-Phong model and illumination as an environment map. While previous work makes strong assumptions in this difficult scenario, focusing solely on specific types of objects like faces or imposing very strong priors, our approach leverages only the light field structure, where a solution consistent across all subaperture views is sought. The optimization is based primarily on shading, which is sensitive to fine geometric details which are propagated to the initial coarse depth map. Despite the problem being inherently ill-posed, we achieve encouraging results on synthetic as well as real-world data.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Sharpening Hyperspectral Images Using Spatial and Spectral Priors in a Plug-and-Play Algorithm

next chapter Shadow and Specularity Priors for Intrinsic Light Field Decomposition

Available online http://gl.ict.usc.edu/Data/HighResProbes/.

Aldrian, O., Smith, W.A.P.: Inverse rendering of faces on a cloudy day. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7574, pp. 201–214. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33712-3_15 CrossRef

Aldrian, O., Smith, W.A.: Inverse rendering of faces with a 3D Morphable model. IEEE Trans. Pattern Anal. Mach. Intell. 35(5), 1080–1093 (2013)CrossRef

Barron, J.T., Malik, J.: Shape, illumination, and reflectance from shading. IEEE Trans. Pattern Anal. Mach. Intell. 37(8), 1670–1687 (2015)CrossRef

Bolles, R., Baker, H., Marimont, D.: Epipolar-plane image analysis: an approach to determining structure from motion. Int. J. Comput. Vis. 1(1), 7–55 (1987)CrossRef

Branch, M.A., Coleman, T.F., Li, Y.: A subspace, interior, and conjugate gradient method for large-scale bound-constrained minimization problems. SIAM J. Sci. Comput. 21(1), 1–23 (1999)MathSciNetCrossRefMATH

Byrd, R.H., Lu, P., Nocedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16(5), 1190–1208 (1995)MathSciNetCrossRefMATH

Graber, G., Balzer, J., Soatto, S., Pock, T.: Efficient minimal-surface regularization of perspective depth maps in variational stereo. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2015)

Heber, S., Pock, T.: Shape from light field meets robust PCA. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 751–767. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10599-4_48

Honauer, K., Johannsen, O., Kondermann, D., Goldluecke, B.: A dataset and evaluation methodology for depth estimation on 4D light fields. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10113, pp. 19–34. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54187-7_2 CrossRef

10.

Huang, R., Smith, W.A.: Shape-from-shading under complex natural illumination. In: 2011 18th IEEE International Conference on Image Processing (ICIP), pp. 13–16. IEEE (2011)

11.

Jeon, H., Park, J., Choe, G., Park, J., Bok, Y., Tai, Y., Kweon, I.: Accurate depth map estimation from a lenslet light field camera. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2015)

12.

Johannsen, O., Honauer, K., Goldluecke, B., Alperovich, A., Battisti, F., Bok, Y., Brizzi, M., Carli, M., Choe, G., Diebold, M., Gutsche, M., Jeon, H.G., Kweon, I.S., Park, J., Schilling, H., Sheng, H., Si, L., Strecke, M., Sulc, A., Tai, Y.W., Wang, Q., Wang, T.C., Wanner, S., Xiong, Z., Yu, J., Zhang, S., Zhu, H.: A taxonomy and evaluation of dense light field depth estimation algorithms. In: Proceedings of the 2nd Workshop on Light Fields for Computer Vision at CVPR (2017)

13.

Lombardi, S., Nishino, K.: Reflectance and natural illumination from a single image. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 582–595. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33783-3_42 CrossRef

14.

Matusik, W., Pfister, H., Brand, M., McMillan, L.: A data-driven reflectance model. ACM Trans. Graph. 22(3), 759–769 (2003)CrossRef

15.

Oxholm, G., Nishino, K.: Shape and reflectance from natural illumination. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7572, pp. 528–541. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33718-5_38 CrossRef

16.

Oxholm, G., Nishino, K.: Multiview shape and reflectance from natural illumination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2155–2162 (2014)

17.

Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: International Conference on Computer Vision (ICCV 2011) (2011)

18.

Ramamoorthi, R., Hanrahan, P.: A signal-processing framework for inverse rendering. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 117–128. ACM (2001)

19.

Romeiro, F., Zickler, T.: Blind reflectometry. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 45–58. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15549-9_4 CrossRef

20.

Rusinkiewicz, S.M.: A new change of variables for efficient BRDF representation. In: Drettakis, G., Max, N. (eds.) Rendering Techniques ’98, pp. 11–22. Springer, Heidelberg (1998). https://doi.org/10.1007/978-3-7091-6453-2_2 CrossRef

21.

Strecke, M., Alperovich, A., Goldluecke, B.: Accurate depth and normal maps from occlusion-aware focal stack symmetry. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2017)

22.

Tao, M., Hadap, S., Malik, J., Ramamoorthi, R.: Depth from combining defocus and correspondence using light-field cameras. In: Proceedings of the International Conference on Computer Vision (2013)

23.

Wang, T., Efros, A., Ramamoorthi, R.: Occlusion-aware depth estimation using light-field cameras. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3487–3495 (2015)

24.

Wang, T.C., Chandraker, M., Efros, A.A., Ramamoorthi, R.: SVBRDF-invariant shape and reflectance estimation from light-field cameras. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5451–5459 (2016)

25.

Wanner, S., Goldluecke, B.: Variational light field analysis for disparity estimation and super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 606–619 (2014)CrossRef

26.

Wanner, S., Straehle, C., Goldluecke, B.: Globally consistent multi-label assignment on the ray space of 4D light fields. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2013)

27.

Zeisl, B., Zach, C., Pollefeys, M.: Variational regularization and fusion of surface normal maps. In: Proceedings of the International Conference on 3D Vision (3DV) (2014)

Title: Inverse Lightfield Rendering for Shape, Reflection and Natural Illumination
Authors: Antonin Sulc
Ole Johannsen
Bastian Goldluecke
Publisher: Springer International Publishing
Book: Energy Minimization Methods in Computer Vision and Pattern Recognition
Print ISBN: 978-3-319-78198-3

Electronic ISBN: 978-3-319-78199-0

Copyright Year: 2018
DOI: https://doi.org/10.1007/978-3-319-78199-0_25

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner