Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
Erschienen in:
A Comprehensive Introduction to Photometric 3D-Reconstruction Kapitel lesen Erstes Kapitel lesen

2020 | OriginalPaper | Buchkapitel

4. Non-Rigid Structure-from-Motion and Shading

verfasst von : Mathias Gallardo, Toby Collins, Adrien Bartoli

Erschienen in: Advances in Photometric 3D-Reconstruction

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

We show how photometric and motion-based approaches can be combined to reconstruct the 3D shape of deformable objects from monocular images. We start by motivating the problem using real-world applications. We give a comprehensive overview of the state-of-the-art approaches and discuss their limitations for practical use in these applications. We then introduce the problem of Non-Rigid Structure-from-Motion and Shading (NRSfMS), where photometric and geometric information is used for reconstruction, without prior knowledge about the shape of the deformable object. We present in detail the first technical solution to NRSfMS and close the chapter with the main remaining open problems.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Vorheriges Kapitel RGBD-Fusion: Depth Refinement for Diffuse and Specular Objects
Nächstes Kapitel On the Well-Posedness of Uncalibrated Photometric Stereo Under General Lighting
Fußnoten
Literatur
1.
2.
3.
Ahmed AH, Farag AA (2007) Shape from shading under various imaging conditions. In: CVPR
4.
Akhter I, Sheikh Y, Khan S, Kanade T (2009) Nonrigid structure from motion in trajectory space. In: NIPS
5.
Barron JT, Malik J (2015) Shape, illumination, and reflectance from shading. IEEE Trans Pattern Anal Mach Intell 37(8):1670–1687
6.
Bartoli A, Gérard Y, Chadebecq F, Collins T, Pizarro D (2015) Shape-from-template. IEEE Trans Pattern Anal Mach Intell 37(10):2099–2118
7.
Bartoli A, Özgür E (2016) A perspective on non-isometric shape-from-template. In: ISMAR
8.
Beeler T, Bickel B, Beardsley P, Sumner B, Gross M (2010) High-quality single-shot capture of facial geometry. In: SIGGRAPH
9.
Belhumeur PN, Kriegman DJ, Yuille AL (1997) The bas-relief ambiguity. In: CVPR
10.
Bell S, Bala K, Snavely N (2014) Intrinsic images in the wild. ACM Trans Graph (SIGGRAPH) 33(4)
11.
Bregler C, Hertzmann A, Biermann H (2000) Recovering non-rigid 3D shape from image streams. In: CVPR
12.
Brostow GJ, Hernández C, Vogiatzis G, Stenger B, Cipolla R (2001) Video normals from colored lights. IEEE Trans Pattern Anal Mach Intell 33(10):2104–2114
13.
Brunet F, Hartley R, Bartoli A (2014) Monocular template-based 3D surface reconstruction: convex inextensible and nonconvex isometric methods. Comput Vis Image Underst 125:138–154 August
14.
Chhatkuli A, Pizarro D, Bartoli A (2014) Non-rigid shape-from-motion for isometric surfaces using infinitesimal planarity. In: BMVC
15.
Chhatkuli A, Pizarro D, Bartoli A, Collins T (2017) A stable analytical framework for isometric shape-from-template by surface integration. IEEE Trans Pattern Anal Mach Intell 39(5):833–850
16.
Chhatkuli A, Pizarro D, Collins T, Bartoli A (2016) Inextensible non-rigid shape-from-motion by second-order cone programming. In: CVPR
17.
Collins T, Bartoli A (2015) Realtime shape-from-template: system and applications. In: ISMAR
18.
Dai Y, Li H, He M (2014) A simple prior-free method for non-rigid structure-from-motion factorization. Int J Comput Vis 107(2):101–122MathSciNetMATH
19.
20.
Durou J-D, Falcone M, Sagona M (2008) Numerical methods for shape-from-shading: a new survey with benchmarks. Comput Vis Image Underst 109(1):22–43 January
21.
Ecker A, Jepson AD (2010) Polynomial shape from shading. In: CVPR
22.
Falcone M, Sagona M, Seghini A (2003) A scheme for the shape-from-shading model with “black shadows”. Numerical mathematics and advanced applications, pp 503–512
23.
Fayad J, Agapito L, Del Bue A (2010) Piecewise quadratic reconstruction of non-rigid surfaces from monocular sequences. In: ECCV
24.
Fukunaga K, Hostetler L (1975) The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Trans Inf Theory 21(1):32–40
25.
26.
Gallardo M, Collins T, Bartoli A (2016) Can we jointly register and reconstruct creased surfaces by shape-from-template accurately? In: ECCV
27.
Gallardo M, Collins T, Bartoli A (2016) Using shading and a 3D template to reconstruct complex surface deformations. In: BMVC
28.
Gallardo M, Collins T, Bartoli A (2017) Dense non-rigid structure-from-motion and shading with unknown albedos. In: ICCV
29.
Garg R, Roussos A, Agapito L (2013) Dense variational reconstruction of non-rigid surfaces from monocular video. In: CVPR
30.
Gotardo PFU, Martinez AM (2011) Kernel non-rigid structure from motion. In: ICCV
31.
Gotardo PFU, Simon T, Sheikh Y, Matthews I (2015) Photogeometric scene flow for high-detail dynamic 3D reconstruction. In: ICCV
32.
Gumerov N, Zandifar A, Duraiswami R, Davis LS (2004) Structure of applicable surfaces from single views. In: ECCV
33.
Haefner B, Quéau Y, Möllenhoff T, Cremers D (2018) Fight ill-posedness with ill-posedness: single-shot variational depth super-resolution from shading. In: CVPR
34.
Haefner B, Ye Z, Gao M, Wu T, Quéau Y, Cremers D (2019) Variational uncalibrated photometric stereo under general lighting. In: ICCV
35.
Haouchine N, Dequidt J, Berger MO, Cotin S (2014) Single view augmentation of 3D elastic objects. In: ISMAR
36.
Hartley RI, Zisserman A (2003) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, Cambridge
37.
Horn BKP (1970) Shape from shading: a method for obtaining the shape of a smooth shape of a smooth opaque object from one view. Technical report, Cambridge, MA, USA
38.
Horn BKP (1989) Shape from shading, pp 123–171
39.
Ikeuchi K, Horn BKP (1981) Numerical shape from shading and occluding boundaries. Artif Intell 17(1):141–184
40.
Innmann M, Zollhöfer M, Nießner M, Theobalt C, Stamminger M (2016) VolumeDeform: real-time volumetric non-rigid reconstruction. In: ECCV
41.
Jin H, Cremers D, Wang D, Prados E, Yezzi A, Soatto S (2008) 3-D reconstruction of shaded objects from multiple images under unknown illumination. Int J Comput Vis 76(3):245–256
42.
Kim K, Torii A, Okutomi M (2016) Multi-view inverse rendering under arbitrary illumination and albedo. In: ECCV
43.
Kimmel R, Bruckstein AM (1994) Global shape-from-shading. In: ICPR
44.
Koo B, Özgür E, Le Roy B, Buc E, Bartoli A (2017) Deformable registration of a preoperative 3D liver volume to a laparoscopy image using contour and shading cues. In: MICCAI
45.
Langguth F, Sunkavalli K, Hadap S, Goesele M (2016) Shading-aware Multi-view Stereo. In: ECCV
46.
Lee KM, Kuo C-C (1997) Shape from shading with a generalized reflectance map model. Comput Vis Image Underst 67(2):143–160
47.
Lee KM, Kuo CCJ (1993) Shape from shading with a linear triangular element surface model. IEEE Trans Pattern Anal Mach Intell 15(8):815–822
48.
Liu-Yin Q, Yu R, Agapito L, Fitzgibbon A, Russell C (2016) Better together: joint reasoning for non-rigid 3D reconstruction with specularities and shading. In: BMVC
49.
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110
50.
Magnenat S, Ngo DT, Zund F, Ryffel M, Noris G, Rothlin G, Marra A, Nitti M, Fua P, Gross M, Sumner R (2015) Live texturing of augmented reality characters from colored drawings. IEEE Trans Vis Comput Graph 21(11):1201–1210
51.
Malti A, Bartoli A (2014) Combining conformal deformation and cook-torrance shading for 3D reconstruction in laparoscopy. IEEE Trans Biol Eng 61(6):1684–1692
52.
Malti A, Bartoli A, Collins T (2011) A pixel-based approach to template-based monocular 3D reconstruction of deformable surfaces. In: Proceedings of the IEEE international workshop on dynamic shape capture and analysis at ICCV
53.
Malti A, Bartoli A, Hartley RI (2015) A linear least-squares solution to elastic shape-from-template. In: CVPR
54.
Moreno-Noguer F, Salzmann M, Lepetit V, Fua P (2009) Capturing 3D stretchable surfaces from single images in closed form. In: CVPR
55.
Newcombe RA, Fox D, Seitz SM (2015) DynamicFusion: reconstruction and tracking of non-rigid scenes in real-time. In: CVPR
56.
Ngo TD, Östlund J, Fua P (2016) Template-based monocular 3D shape recovery using laplacian meshes. IEEE Trans Pattern Anal Mach Intell 38(1):172–187
57.
Ngo TD, Park S, Jorstad AA, Crivellaro A, Yoo C, Fua P (2015) Dense image registration and deformable surface reconstruction in presence of occlusions and minimal texture. In: ICCV
58.
Okatani T, Deguchi K (1996) Shape reconstruction from an endoscope image by SfS technique for a point light source at the projection Center. Comput Vis Image Underst 66(2):119–131 July
59.
Özgür E, Bartoli A (2017) Particle-SfT: a provably-convergent, fast shape-from-template algorithm. Int J Comput Vis 123(2):184–205 June
60.
Parashar S, Pizarro D, Bartoli A (2016) Isometric non-rigid shape-from-motion in linear time. In: CVPR
61.
Pentland AP (1984) Local shading analysis. IEEE Trans Pattern Anal Mach Intell 6(2):170–187
62.
Pentland AP (1988) Shape information from shading: a theory about human perception. In: ICCV
63.
Perriollat M, Hartley R, Bartoli A (2011) Monocular template-based reconstruction of inextensible surfaces. Int J Comput Vis 95(2):124–137 NovemberMathSciNetMATH
64.
Prados E, Faugeras O (2005) Shape from shading: a well-posed problem? In: CVPR
65.
Pumarola A, Agudo A, Porzi L, Sanfeliu A, Lepetit V, Moreno-Noguer F (2018) Geometry-aware network for non-rigid shape prediction from a single view. In: CVPR
66.
Quéau Y, Mélou J, Castan F, Cremers D, Durou J (2017) A variational approach to shape-from-shading under natural illumination. In: EMMCVPR
67.
Richardson E, Sela M, Kimmel R (2016) 3D face reconstruction by learning from synthetic data. In: 3DV
68.
Richter SR, Roth S (2015) Discriminative shape from shading in uncalibrated illumination. In: CVPR
69.
Rouy E, Tourin A (1992) A viscosity solutions approach to shape-from-shading. SIAM J Numer Anal 29(3):867–884 JuneMathSciNetMATH
70.
Salzmann M, Fua P (2009) Reconstructing sharply folding surfaces: a convex formulation. In: CVPR
71.
Salzmann M, Fua P (2011) Linear local models for monocular reconstruction of deformable surfaces. IEEE Trans Pattern Anal Mach Intell 33(5):931–944
72.
Salzmann M, Pilet J, Ilic S, Fua P (2007) Surface deformation models for nonrigid 3D shape recovery. IEEE Trans Pattern Anal Mach Intell 29(8):1481–1487
73.
Sorkine O, Alexa M (2007) As-rigid-as-possible surface modeling. In: Symposium on geometry processing, pp 109–116
74.
Sundaram N, Brox T, Keutzer K (2010) Dense point trajectories by GPU-accelerate large displacement optical flow. In: ECCV
75.
Tankus A, Sochen N, Yeshurun Y (2005) Shape-from-shading under perspective projection. Int J Comput Vis 63(1):21–43 June
76.
Taylor J, Jepson AD, Kutulakos K (2010) Non-rigid structure from locally-rigid motion. In: CVPR
77.
Thies J, Zollhöfer M, Stamminger M, Theobalt C, Nießner M (2016) Face2Face: real-time face capture and reenactment of RGB videos. In: CVPR
78.
Tsai P-S, Shah M (1994) Shape from shading using linear approximation. Image Vis Comput 12(8):487–498
79.
Valgaerts L, Wu C, Bruhn A, Seidel H-P, Theobalt C (2012) Lightweight binocular facial performance capture under uncontrolled lighting. In: SIGGRAPH
80.
Varol A, Salzmann M, Fua P, Urtasun R (2012) A constrained latent variable model. In: CVPR
81.
Varol A, Salzmann M, Tola E, Fua P (2009) Template-free monocular reconstruction of deformable surfaces. In: ICCV
82.
Varol A, Shaji A, Salzmann M, Fua P (2012) Monocular 3D reconstruction of locally textured surfaces. IEEE Trans Pattern Anal Mach Intell 34(6)
83.
Vicente S, Agapito L (2012) Soft inextensibility constraints for template-free non-rigid reconstruction. In: ECCV
84.
Vicente S, Agapito L (2013) Balloon shapes: reconstructing and deforming objects with volume from images. In: 3DV
85.
Wang X, Salzmann M, Wang F, Zhao J (2016) Template-free 3D reconstruction of poorly-textured nonrigid surfaces. In: ECCV
86.
Wu C, Narasimhan SG, Jaramaz B (2010) A multi-image shape-from-shading framework for near-lighting perspective endoscopes. Int J Comput Vis 86(2):211–228MathSciNet
87.
Wu C, Varanasi K, Liu Y, Seidel HP, Theobalt C (2011) Shading-based dynamic shape refinement from multi-view video under general illumination. In: ICCV
88.
Xiong Y, Chakrabarti A, Basri R, Gortler SJ, Jacobs DW, Zickler T (2015) From shading to local shape. IEEE Trans Pattern Anal Mach Intell 37(1)
89.
Yu R, Russell C, Campbell NDF, Agapito L (2015) Direct, dense, and deformable: template-based non-rigid 3D reconstruction from RGB video. In: ICCV
90.
Zhang Z (1997) Parameter estimation techniques: a tutorial with application to conic fitting. Image Vis Comput 15(1):59–76
91.
Zhou Z, Wu Z, Tan P (2013) Multi-view photometric stereo with spatially varying isotropic materials. In: CVPR
Metadaten
Titel
Non-Rigid Structure-from-Motion and Shading
verfasst von
Mathias Gallardo
Toby Collins
Adrien Bartoli
Copyright-Jahr
2020
Buch
Advances in Photometric 3D-Reconstruction

Print ISBN: 978-3-030-51865-3

Electronic ISBN: 978-3-030-51866-0

Copyright-Jahr: 2020

DOI
https://doi.org/10.1007/978-3-030-51866-0_4

Premium Partner

    Bildnachweise