Skip to main content
Erschienen in:
Buchtitelbild

2017 | OriginalPaper | Buchkapitel

Layered Scene Reconstruction from Multiple Light Field Camera Views

verfasst von : Ole Johannsen, Antonin Sulc, Nico Marniok, Bastian Goldluecke

Erschienen in: Computer Vision – ACCV 2016

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

We propose a framework to infer complete geometry of a scene with strong reflections or hidden by partially transparent occluders from a set of 4D light fields captured with a hand-held light field camera. For this, we first introduce a variant of bundle adjustment specifically tailored to 4D light fields to obtain improved pose parameters. Geometry is recovered in a global framework based on convex optimization for a weighted minimal surface. To allow for non-Lambertian materials and semi-transparent occluders, the point-wise costs are not based on the principle of photo-consistency. Instead, we perform a layer analysis of the light field obtained by finding superimposed oriented patterns in epipolar plane image space to obtain a set of depth hypotheses and confidence scores, which are integrated into a single functional.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Gortler, S., Grzeszczuk, R., Szeliski, R., Cohen, M.: The lumigraph. In: Proceedings of SIGGRAPH, pp. 43–54 (1996) Gortler, S., Grzeszczuk, R., Szeliski, R., Cohen, M.: The lumigraph. In: Proceedings of SIGGRAPH, pp. 43–54 (1996)
2.
Zurück zum Zitat Bolles, R., Baker, H., Marimont, D.: Epipolar-plane image analysis: an approach to determining structure from motion. Int. J. Comput. Vis. 1, 7–55 (1987)CrossRef Bolles, R., Baker, H., Marimont, D.: Epipolar-plane image analysis: an approach to determining structure from motion. Int. J. Comput. Vis. 1, 7–55 (1987)CrossRef
3.
Zurück zum Zitat Johannsen, O., Sulc, A., Goldluecke, B.: What sparse light field coding reveals about scene structure. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2016) Johannsen, O., Sulc, A., Goldluecke, B.: What sparse light field coding reveals about scene structure. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2016)
4.
Zurück zum Zitat Wanner, S., Goldluecke, B.: Reconstructing reflective and transparent surfaces from epipolar plane images. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 1–10. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40602-7_1 CrossRef Wanner, S., Goldluecke, B.: Reconstructing reflective and transparent surfaces from epipolar plane images. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 1–10. Springer, Heidelberg (2013). doi:10.​1007/​978-3-642-40602-7_​1 CrossRef
5.
Zurück zum Zitat Johannsen, O., Sulc, A., Goldluecke, B.: On linear structure from motion for light field cameras. In: Proceedings of the International Conference on Computer Vision (2015) Johannsen, O., Sulc, A., Goldluecke, B.: On linear structure from motion for light field cameras. In: Proceedings of the International Conference on Computer Vision (2015)
6.
Zurück zum Zitat Johannsen, O., Sulc, A., Goldluecke, B.: Variational separation of light field layers. In: Vision, Modelling and Visualization (VMV) (2015) Johannsen, O., Sulc, A., Goldluecke, B.: Variational separation of light field layers. In: Vision, Modelling and Visualization (VMV) (2015)
7.
Zurück zum Zitat Wanner, S., Goldluecke, B.: Variational light field analysis for disparity estimation and super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36, 606–619 (2014)CrossRef Wanner, S., Goldluecke, B.: Variational light field analysis for disparity estimation and super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36, 606–619 (2014)CrossRef
8.
Zurück zum Zitat Chandraker, M., Reddy, D., Wang, Y., Ramamoorthi, R.: What object motion reveals about shape with unknown BRDF and lighting. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2013) Chandraker, M., Reddy, D., Wang, Y., Ramamoorthi, R.: What object motion reveals about shape with unknown BRDF and lighting. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2013)
9.
Zurück zum Zitat Tao, M., Wang, T.C., Malik, J., Ramamoorthi, R.: Depth estimation for glossy surfaces with light-field cameras. In: Proceedings of the European Conference on Computer Vision Workshops (2014) Tao, M., Wang, T.C., Malik, J., Ramamoorthi, R.: Depth estimation for glossy surfaces with light-field cameras. In: Proceedings of the European Conference on Computer Vision Workshops (2014)
10.
Zurück zum Zitat Chen, C., Lin, H., Yu, Z., Kang, S.B., J., Y.: Light field stereo matching using bilateral statistics of surface cameras. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2014) Chen, C., Lin, H., Yu, Z., Kang, S.B., J., Y.: Light field stereo matching using bilateral statistics of surface cameras. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2014)
11.
Zurück zum Zitat Stich, T., Tevs, A., Magnor, M.: Global depth from epipolar volumes - a general framework for reconstructing non-Lambertian surfaces. In: 3DPVT (2006) Stich, T., Tevs, A., Magnor, M.: Global depth from epipolar volumes - a general framework for reconstructing non-Lambertian surfaces. In: 3DPVT (2006)
12.
Zurück zum Zitat Laurentini, A.: The visual hull concept for visual-based image understanding. IEEE Trans. Pattern Anal. Mach. Intell. 16, 150–162 (1994)CrossRef Laurentini, A.: The visual hull concept for visual-based image understanding. IEEE Trans. Pattern Anal. Mach. Intell. 16, 150–162 (1994)CrossRef
13.
Zurück zum Zitat Zuo, X., Du, C., Wang, S., Zheng, J., Yang, R.: Interactive visual hull refinement for specular and transparent object surface reconstruction. In: Proceedings of the International Conference on Computer Vision (2015) Zuo, X., Du, C., Wang, S., Zheng, J., Yang, R.: Interactive visual hull refinement for specular and transparent object surface reconstruction. In: Proceedings of the International Conference on Computer Vision (2015)
14.
Zurück zum Zitat Kutulakos, K.N., Seitz, S.M.: A theory of shape by space carving. Int. J. Comput. Vis. 38, 199–218 (2000)CrossRefMATH Kutulakos, K.N., Seitz, S.M.: A theory of shape by space carving. Int. J. Comput. Vis. 38, 199–218 (2000)CrossRefMATH
15.
Zurück zum Zitat Yang, Y., Pollefeys, M., Welch, G.: Dealing with textureless regions and specular highlights - a progressive space carving scheme using a novel photo-consistency measure. In: Proceedings of the International Conference on Computer Vision (2003) Yang, Y., Pollefeys, M., Welch, G.: Dealing with textureless regions and specular highlights - a progressive space carving scheme using a novel photo-consistency measure. In: Proceedings of the International Conference on Computer Vision (2003)
16.
Zurück zum Zitat Faugeras, O., Keriven, R.: Variational principles, surface evolution, PDE’s, level set methods, and the stereo problem. IEEE Trans. Image Process. 7, 336–344 (1998)MathSciNetCrossRefMATH Faugeras, O., Keriven, R.: Variational principles, surface evolution, PDE’s, level set methods, and the stereo problem. IEEE Trans. Image Process. 7, 336–344 (1998)MathSciNetCrossRefMATH
17.
Zurück zum Zitat Jin, H., Soatto, S., Yezzi, A.: Multi-view stereo reconstruction of dense shape and complex appearance. Int. J. Comput. Vis. 63, 175–189 (2005)CrossRef Jin, H., Soatto, S., Yezzi, A.: Multi-view stereo reconstruction of dense shape and complex appearance. Int. J. Comput. Vis. 63, 175–189 (2005)CrossRef
18.
Zurück zum Zitat Yu, T., Narendra, A., Chen, W.C.: SDG cut: 3D reconstruction of non-Lambertian objects using graph cuts on surface distance grid. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2006) Yu, T., Narendra, A., Chen, W.C.: SDG cut: 3D reconstruction of non-Lambertian objects using graph cuts on surface distance grid. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2006)
19.
Zurück zum Zitat Ihrke, I., Goldluecke, B., Magnor, M.: Reconstructing the geometry of flowing water. In: Proceedings of the International Conference on Computer Vision, pp. 1055–1060 (2005) Ihrke, I., Goldluecke, B., Magnor, M.: Reconstructing the geometry of flowing water. In: Proceedings of the International Conference on Computer Vision, pp. 1055–1060 (2005)
20.
Zurück zum Zitat Goldman, D., Curless, B., Hertzmann, A., Seitz, S.: Shape and spatially-varying BRDFs from photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1060–1071 (2010)CrossRef Goldman, D., Curless, B., Hertzmann, A., Seitz, S.: Shape and spatially-varying BRDFs from photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1060–1071 (2010)CrossRef
21.
Zurück zum Zitat Vogiatzis, G., Hernandez, C., Cipolla, R.: Reconstruction in the round using photometric normals and silhouettes. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2006) Vogiatzis, G., Hernandez, C., Cipolla, R.: Reconstruction in the round using photometric normals and silhouettes. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2006)
22.
Zurück zum Zitat Kolev, K., Pock, T., Cremers, D.: Anisotropic minimal surfaces integrating photoconsistency and normal information for multiview stereo. In: Proceedings of the European Conference on Computer Vision (2010) Kolev, K., Pock, T., Cremers, D.: Anisotropic minimal surfaces integrating photoconsistency and normal information for multiview stereo. In: Proceedings of the European Conference on Computer Vision (2010)
23.
Zurück zum Zitat Cremers, D., Kolev, K.: Multiview stereo and silhouette consistency via convex functionals over convex domains. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1161–1174 (2011)CrossRef Cremers, D., Kolev, K.: Multiview stereo and silhouette consistency via convex functionals over convex domains. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1161–1174 (2011)CrossRef
24.
Zurück zum Zitat Federer, H.: Geometric Measure Theory. Springer, Heidelberg (1969)MATH Federer, H.: Geometric Measure Theory. Springer, Heidelberg (1969)MATH
25.
Zurück zum Zitat Zach, C., Pock, T., Bischof, H.: A globally optimal algorithm for robust TV-L1 range image integration. In: Proceedings of the International Conference on Computer Vision (2007) Zach, C., Pock, T., Bischof, H.: A globally optimal algorithm for robust TV-L1 range image integration. In: Proceedings of the International Conference on Computer Vision (2007)
26.
Zurück zum Zitat Graber, G., Pock, T., Bischof, H.: Online 3D reconstruction using convex optimization. In: Proceedings of the International Conference on Computer Vision Workshops (2011) Graber, G., Pock, T., Bischof, H.: Online 3D reconstruction using convex optimization. In: Proceedings of the International Conference on Computer Vision Workshops (2011)
27.
Zurück zum Zitat Zickler, T., Belhumeur, P., Kriegman, D., Stereopsis, H.: Exploiting reciprocity for surface reconstruction. Int. J. Comput. Vis. 49, 215–227 (2002)CrossRefMATH Zickler, T., Belhumeur, P., Kriegman, D., Stereopsis, H.: Exploiting reciprocity for surface reconstruction. Int. J. Comput. Vis. 49, 215–227 (2002)CrossRefMATH
28.
Zurück zum Zitat Dansereau, D., Pizarro, O., Williams, S.: Decoding, calibration and rectification for lenselet-based plenoptic cameras. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 1027–1034 (2013) Dansereau, D., Pizarro, O., Williams, S.: Decoding, calibration and rectification for lenselet-based plenoptic cameras. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 1027–1034 (2013)
29.
Zurück zum Zitat Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: International Conference on Computer Vision (ICCV 2011) (2011) Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: International Conference on Computer Vision (ICCV 2011) (2011)
30.
Zurück zum Zitat Chan, T., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM J. Appl. Math. 66, 1632–1648 (2006)MathSciNetCrossRefMATH Chan, T., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM J. Appl. Math. 66, 1632–1648 (2006)MathSciNetCrossRefMATH
Metadaten
Titel
Layered Scene Reconstruction from Multiple Light Field Camera Views
verfasst von
Ole Johannsen
Antonin Sulc
Nico Marniok
Bastian Goldluecke
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-54187-7_1

Premium Partner