nach oben

International Journal of Computer Vision

Erschienen in:

30.04.2018

An Approximate Shading Model with Detail Decomposition for Object Relighting

verfasst von: Zicheng Liao, Kevin Karsch, Hongyi Zhang, David Forsyth

Erschienen in: International Journal of Computer Vision | Ausgabe 1/2019

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

We present an object relighting system that allows an artist to select an object from an image and insert it into a target scene. Through simple interactions, the system can adjust illumination on the inserted object so that it appears naturally in the scene. To support image-based relighting, we build object model from the image, and propose a perceptually-inspired approximate shading model for the relighting. It decomposes the shading field into (a) a rough shape term that can be reshaded, (b) a parametric shading detail that encodes missing features from the first term, and (c) a geometric detail term that captures fine-scale material properties. With this decomposition, the shading model combines 3D rendering and image-based composition and allows more flexible compositing than image-based methods. Quantitative evaluation and a set of user studies suggest our method is a promising alternative to existing methods of object insertion.

Vorheriger Artikel Structural Constraint Data Association for Online Multi-object Tracking

Nächster Artikel Combining Multiple Cues for Visual Madlibs Question Answering

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nur mit Berechtigung zugänglich

Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., et al. (2004). Interactive digital photomontage. ACM Transactions on Graphics, 23(3), 294–302.CrossRef

Arsalan Soltani, A., Huang, H., Wu, J., Kulkarni, T. D., & Tenenbaum, J. B. (2017). Synthesizing 3D shapes via modeling multi-view depth maps and silhouettes with deep generative networks. In The IEEE conference on computer vision and pattern recognition (CVPR).

Barron, J. T., & Malik, J. (2012). Color constancy, intrinsic images, and shape estimation. In ECCV.

Basri, R., & Jacobs, D. (2003). Lambertian reflectance and linear subspaces. In PAMI.

Beck, J., & Prazdny, S. (1981). Highlights and the perception of glossiness. Attention, Perception and Psychophysics, 30, 407–410.CrossRef

Berzhanskaya, J., Swaminathan, G., Beck, J., & Mingolla, E. (2005). Remote effects of highlights on gloss perception. Perception, 34, 565–575.CrossRef

Burt, P. J., & Adelson, E. H. (1983). A multiresolution spline with application to image mosaics. ACM Transactions on Graphics, 2(4), 217–236.CrossRef

Cavanagh, P. (2005). The artist as neuroscientist. Nature, 434, 301–307.CrossRef

Chen, T., Cheng, M.-M., Tan, P., Shamir, A., & Hu, S.-M. (2009). Sketch2photo: internet image montage. ACM Transactions on Graphics, 28(5), 124:1–124:10.

Debevec, P. (1998). Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In SIGGRAPH’98 (pp. 189–198). ACM.

Deshpande, A., Lu, J., Yeh, M.-C., Jin Chong, M., & Forsyth, D. (2017). Learning diverse image colorization. In The IEEE conference on computer vision and pattern recognition (CVPR).

Durou, J.-D., Falcone, M., & Sagona, M. (2008). Numerical methods for shape-from-shading: A new survey with benchmarks. Computer Vision and Image Understanding, 109(1), 22–43.CrossRef

Esteban, C. H., Vogiatzis, G., & Cipolla, R. (2008). Multiview photometric stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(3), 548–554.CrossRef

Furukawa, Y., & Hernandez, C. (2015). Multi-vew Stereo: A tutorial. Foundations and trends?. In Computer graphics and vision.

Fyffe, G., Jones, A., Alexander, O., Ichikari, R., Graham, P., Nagano, K., Busch, J., & Debevec P. (2013). Driving high-resolution facial blendshapes with video performance capture. In ACM SIGGRAPH 2013 Talks, SIGGRAPH ’13 (p. 33:1), New York, NY, USA, 2013. ACM.

Fyffe, G., Nagano, K., Huynh, L., Saito, S., Busch, J., Jones, A., Li, H., & Debevec, P. (2017). Multi-view stereo on consistent face topology. In Computer Graphics Forum.

Ghosh, A., Fyffe, G., Tunwattanapong, B., Busch, J., Yu, X., & Debevec, P. (2011). Multiview face capture using polarized spherical gradient illumination. ACM Transactions on Graphics, 30(6), 129:1–129:10.CrossRef

Grosse, R., Johnson, M. K., Adelson, E. H., & Freeman, W. T. (2009). Ground-truth dataset and baseline evaluations for intrinsic image algorithms. In ICCV (pp. 2335–2342).

Hartley, R. I., & Zisserman, A. (2004). Multiple view geometry in computer vision (2nd ed.). Cambridge: Cambridge University Press. (ISBN: 0521540518).CrossRefMATH

Johnston, S. F. (2002). Lumo: Illumination for cel animation. In NPAR ’02.

Karsch, K., Hedau, V., Forsyth, D., & Hoiem, D. (2011). Rendering synthetic objects into legacy photographs. ACM Transactions on Graphics (SIGGRAPH Asia), 30(6), 157:1–157:12.

Karsch, K., Liu, C., Kang, S. B., & England, N. (2012). Depth extraction from video using non-parametric sampling. In ECCV.

Karsch, K., Sunkavalli, K., Hadap, S., Carr, N., Jin, H., Fonte, R., et al. (2014). Automatic scene inference for 3D object compositing. ACM Transactions on Graphics, 33(3), 32:1–32:15.CrossRefMATH

Kim, S., Park, K., Sohn, K., & Lin, S. (2016). Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields. In European conference on computer vision.

Lalonde, J.-F., Hoiem, D., Efros, A. A., Rother, C., Winn, J., & Criminisi, A. (2007). Photo clip art. ACM Transactions on Graphics (SIGGRAPH 2007), 26(3), 3.CrossRef

Lettry, L., Vanhoey, K., & Van Gool L. (2016). Darn: a deep adversial residual network for intrinsic image decomposition. arXiv:1612.07899.

Liao, Z., Karsch, K., & Forsyth, D. (2015). An approximate shading model for object relighting. In CVPR.

Liao, Z., Rock, J., Wang, Y., & Forsyth, D. (2013). Non-parametric filtering for geometric detail extraction and material representation. In CVPR (pp. 963–970).

Narihira, T., Maire, M., & Yu, S. X. (2015). Direct intrinsics: Learning albedo-shading decomposition by convolutional regression. In Proceedings of the IEEE international conference on computer vision (pp. 2992–2992).

Niebner, M., Keinert, B., Fisher, M., Stamminger, M., Loop, C., & Schäfer, H. (2016). Real-time rendering techniques with hardware tessellation. Computer Graphics Forum, 35(1), 113–137.CrossRef

Ostrovsky, Y., Cavanagh, P., & Sinha, P. (2005). Perceiving illumination inconsistencies in scenes. Perception, 34, 1301–1314.CrossRef

Pérez, P., Gangnet, M., & Blake, A. (2003). Poisson image editing. ACM Transactions on Graphics, 22(3), 313–318.CrossRef

Prasad, M., & Fitzgibbon, A. (2006). Single view reconstruction of curved surfaces. In CVPR (pp. 1345–1354).

Ramamoorthi, R., & Hanrahan, P. (2001). An efficient representation for irradiance environment maps. In Proceedings of the 28th annual conference on computer graphics and interactive techniques, SIGGRAPH’01 (pp. 497–500).

Richardson, E., Sela, M., Or-El, R., & Kimmel R. (July 2017). Learning detailed face reconstruction from a single image. In The IEEE conference on computer vision and pattern recognition (CVPR).

Shu, Z., Yumer, E., Hadap, S., Sunkavalli, K., Shechtman, E., & Samaras, D. (2017). Neural face editing with intrinsic image disentangling. In The IEEE conference on computer vision and pattern recognition (CVPR).

Tarr, M. J., Kersten, D., & Bülthoff, H. H. (1998). Why the visual recognition system might encode the effects of illumination. Vision Research, 38, 2259–2275.CrossRef

Trigeorgis, G., Snape, P., Kokkinos, I., & Zafeiriou, S. (2017). Face normals “in-the-wild” using fully convolutional networks. In The IEEE conference on computer vision and pattern recognition (CVPR).

Twarog, N., Tappen, M., & Adelson, E. (2012). Playing with puffball: simple scale-invariant inflation for use in vision and graphics. In Proceedings of the ACM symposium on applied perception, SAP’12 (pp. 45–54).

Wu, T.-P., Sun, J., Tang, C.-K., & Shum, H.-Y. (2008). Interactive normal reconstruction from a single image. ACM Transactions on Graphics, 27(5), 119:1–119:9.CrossRef

Xia, T., Liao, B., & Yu, Y. (2009). Patch-based image vectorization with automatic curvilinear feature alignment. ACM Transactions on Graphics, 28(5), 115:1–115:10.CrossRef

Zhang, R., Tsai, P.-S., Cryer, J., & Shah, M. (1999). Shape-from-shading: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(8), 690–706.CrossRefMATH

Titel: An Approximate Shading Model with Detail Decomposition for Object Relighting
verfasst von: Zicheng Liao
Kevin Karsch
Hongyi Zhang
David Forsyth
Publikationsdatum: 30.04.2018
Verlag: Springer US
Erschienen in: International Journal of Computer Vision / Ausgabe 1/2019
Print ISSN: 0920-5691
Elektronische ISSN: 1573-1405
DOI: https://doi.org/10.1007/s11263-018-1090-6

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Weitere Artikel der Ausgabe 1/2019

Fast Diffeomorphic Image Registration via Fourier-Approximated Lie Algebras

Combining Multiple Cues for Visual Madlibs Question Answering

From BoW to CNN: Two Decades of Texture Representation for Texture Classification

Structural Constraint Data Association for Online Multi-object Tracking