Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Introduction to Dense Optical Flow Read chapter Read first chapter

2016 | OriginalPaper | Chapter

From Images to Depths and Back

Authors : Tal Hassner, Ronen Basri

Published in: Dense Image Correspondences for Computer Vision

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

This chapter describes what is possibly the earliest use of dense correspondence estimation for transferring semantic information between images of different scenes. The method described in this chapter was designed for non-parametric, “example-based” depth estimation of objects appearing in single photos. It consults a database of example 3D geometries and associated appearances, searching for those which look similar to the object in the photo. This is performed at the pixel level, in similar spirit to the more recent methods described in the following chapters. Those newer methods, however, use robust, generic dense correspondence estimation engines. By contrast, the method described here uses a hard-EM optimization to optimize a well-defined target function over the similarity of appearance/depth pairs in the database to appearance/estimated-depth pairs of a query photo. Results are presented demonstrating how depths associated with diverse reference objects may be assigned to different objects appearing in query photos. Going beyond visible shape, we show that the method can be employed for the surprising task of estimating shapes of occluded objects’ backsides. This, so long as the reference database contains examples of mappings from appearances to backside shapes. Finally, we show how the duality of appearance and shape may be exploited in order to “paint colors” on query shapes (“colorize” them) by simply reversing the matching from appearances to depths.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter In Defense of Gradient-Based Alignment on Densely Sampled Sparse Features
next chapter Depth Transfer: Depth Extraction from Videos Using Nonparametric Sampling
Literature
1.
Assa, J., Wolf, L.: Diorama construction from a single image. In: Eurographics, pp. 599–608 (2007)
2.
Atick, J., Griffin, P., Redlich, A.: Statistical approach to shape from shading: reconstruction of three-dimensional face surfaces from single two-dimensional images. Neural Comput. 8(6), 1321–1340 (1996)CrossRef
3.
Barron, J.T., Malik, J.: Shape, albedo, and illumination from a single image of an unknown object. In: Proceedings of Conference on Computer Vision Pattern Recognition, pp. 334–341. IEEE, Providence (2012)
4.
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of ACM SIGGRAPH Conference on Computer Graphics, pp. 187–194. ACM/Addison-Wesley, New York (1999)
5.
Burt, P., Adelson, E.: The laplacian pyramid as a compact image code. IEEE Trans. Commun. 30, 532–540 (1983)CrossRef
6.
Chen, Y., Kim, T.K., Cipolla, R.: Inferring 3D shapes and deformations from single views. In: European Conference on Computer Vision, pp. 300–313. Springer, Heidelberg (2010)
7.
Chen, X., Guo, Y., Zhou, B., Zhao, Q.: Deformable model for estimating clothed and naked human shapes from a single image. Vis. Comput. 29(11), 1187–1196 (2013)CrossRef
8.
Cipolla, R., Fletcher, G., Giblin, P.: Surface geometry from cusps of apparent contours. In: Proceedings of International Conference on Computer Vision, pp. 858–863 (1995)
9.
Cole, F., Isola, P., Freeman, W.T., Durand, F., Adelson, E.H.: Shapecollage: occlusion-aware, example-based shape interpretation. In: European Conference on Computer Vision, pp. 665–678. Springer, Heidelberg (2012)
10.
Criminisi, A., Reid, I., Zisserman, A.: Single view metrology. Int. J. Comput. Vis. 40(2), 123–148 (2000)CrossRefMATH
11.
12.
Delage, E., Lee, H., Ng, A.: Automatic single-image 3D reconstructions of indoor manhattan world scenes. In: Proceedings of the International Symposium of Robotics Research (ISRR), pp. 305–321 (2005)
13.
Dovgard, R., Basri, R.: Statistical symmetric shape from shading for 3D structure recovery of faces. In: European Conference on Computer Vision, vol. 2, pp. 99–113. Springer, Berlin/Heidelberg (2004)
14.
Eigen, D., Fergus, R.: Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. arXiv preprint (2014) [arXiv:1411.4734]
15.
Gupta, A., Satkin, S., Efros, A.A., Hebert, M.: From 3D scene geometry to human workspace. In: Proceedings of Conference on Computer Vision Pattern Recognition, pp. 1961–1968. IEEE, Providence (2011)
16.
Han, J., Zhou, K., Wei, L., Gong, M., Bao, H., Zhang, X., Guo, B.: Fast example-based surface texture synthesis via discrete optimization. Vis. Comput. 22, 918–925 (2006)CrossRef
17.
18.
Hassner, T., Basri, R.: Automatic depth-map colorization. In: Eurographics (short), pp. 73–76 (2006)
19.
Hassner, T., Basri, R.: Example based 3D reconstruction from single 2D images. In: Beyond Patches Workshop at CVPR, p. 15 (2006)
20.
Hassner, T., Basri, R.: Single view depth estimation from examples. arXiv preprint (2013) [arXiv:1304.3915]
21.
Hassner, T., Zelnik-Manor, L., Leifman, G., Basri, R.: Minimal-cut model composition. In: International Conference on Shape Modeling and Applications (SMI’ 05), pp. 72–81 (2005)
22.
Hassner, T., Mayzels, V., Zelnik-Manor, L.: On sifts and their scales. In: Proceedings of Conference on Computer Vision Pattern Recognition, pp. 1522–1528. IEEE, Providence (2012)
23.
Hassner, T., Assif, L., Wolf, L.: When standard RANSAC is not enough: cross-media visual matching with hypothesis relevancy. Mach. Vis. Appl. 25(4), 971–983 (2014)CrossRef
24.
Hassner, T., Harel, S., Paz, E., Enbar, R.: Effective face frontalization in unconstrained images. In: Proceedings of Conference on Computer Vision Pattern Recognition (2015)CrossRef
25.
Hoiem, D., Efros, A., Hebert, M.: Automatic photo pop-up. ACM Trans. Graph. 24(3), 577–584 (2005)CrossRef
26.
Hoiem, D., Efros, A., Hebert, M.: Geometric context from a single image. In: Proceedings of International Conference on Computer Vision, pp. 654–661. IEEE Computer Society, Beijing (2005)
27.
Horn, B.: Obtaining Shape from Shading Information. The Psychology of Computer Vision. McGraw-Hill, New York (1975)
28.
Karsch, K., Liu, C., Kang, S.B.: Depth extraction from video using non-parametric sampling. In: European Conference on Computer Vision, pp. 775–788. Springer, Heidelberg (2012)
29.
Kearns, M., Mansour, Y., Ng, A.: An information-theoretic analysis of hard and soft assignment methods for clustering. In: Proceedings of the NATO Advanced Study Institute on Learning in Graphical Models, pp. 495–520. Kluwer Academic, Norwell (1998)
30.
Kemelmacher, I., Basri, R.: Molding face shapes by example. In: European Conference on Computer Vision, p. 2006 (277–288)
31.
Kemelmacher-Shlizerman, I., Seitz, S.: Face reconstruction in the wild. In: Proceedings of International Conference on Computer Vision, pp. 1746–1753. IEEE, Washington (2011)
32.
Kraevoy, V., Sheffer, A.: Cross-parameterization and compatible remeshing of 3D models. ACM Trans. Graph. 23(3), 861–869 (2004)CrossRef
33.
Leifman, G., Tal, A.: Mesh colorization. Comput. Graph. Forum 31(2), 421–430 (2012)CrossRef
34.
35.
36.
Liu, F., Shen, C., Lin, G., Reid, I.: Learning depth from single monocular images using deep convolutional neural fields. arXiv preprint (2015) [arXiv:1502.07411]
37.
Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRef
38.
Oswald, M.R., Toppe, E., Cremers, D.: Fast and globally optimal single view reconstruction of curved objects. In: Proceedings of Conference on Computer Vision Pattern Recognition, pp. 534–541. IEEE, Washington (2012)
39.
Panagopoulos, A., Hadap, S., Samaras, D.: Reconstructing shape from dictionaries of shading primitives. In: Asian Conference on Computer Vision, pp. 80–94. Springer, Heidelberg (2013)
40.
Praun, E., Sweldens, W., Schröder, P.: Consistent mesh parameterizations. In: Proceedings of ACM SIGGRAPH Conference on Computer Graphics, pp. 179–184. ACM, New York (2001)
41.
Qiu, W., Wang, X., Bai, X., Yuille, A., Tu, Z.: Scale-space sift flow. In: Proceedings of Winter Conference on Applications of Computer Vision, pp. 1112–1119. IEEE, Steamboat Springs (2014)
42.
Romdhani, S., Vetter, T.: Efficient, robust and accurate fitting of a 3D morphable model. In: Proceedings of International Conference on Computer Vision, p. 59 (2003)
43.
Saxena, A., Sun, M., Ng, A.: Make3d: learning 3-D scene structure from a single still image. Trans. Pattern Anal. Mach. Intell. 31(5), 824–840 (2009)CrossRef
44.
Schwing, A.G., Urtasun, R.: Efficient exact inference for 3D indoor scene understanding. In: European Conference on Computer Vision, pp. 299–313. Springer, Heidelberg (2012)
45.
Tau, M., Hassner, T.: Dense correspondences across scenes and scales. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1 (2015)
46.
Thomas, A., Ferrari, V., Leibe, B., Tuytelaars, T., Gool, L.: Shape-from-recognition: recognition enables meta-data transfer. Comput. Vis. Image Underst. 113(12), 1222–1234 (2009)CrossRef
47.
48.
Turk, G.: Texture synthesis on surfaces. In: Proceedings of ACM SIGGRAPH Conference on Computer Graphics, pp. 347–354. ACM, New York (2001)
49.
50.
Vedaldi, A., Fulkerson, B.: VLFeat: an open and portable library of computer vision algorithms. In: Proceedings of International Conference on Multimedia, pp. 1469–1472 (2010). Available: www.​vlfeat.​org/
51.
Wei, L.Y., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: Proceedings of ACM SIGGRAPH Conference on Computer Graphics, pp. 479–488. ACM/Addison-Wesley, New York (2000)
52.
Wei, L.Y., Levoy, M.: Texture synthesis over arbitrary manifold surfaces. In: Proceedings of ACM SIGGRAPH Conference on Computer Graphics, pp. 355–360. ACM, New York (2001)
53.
Witkin, A.: Recovering surface shape and orientation from texture. Artif. Intell. 17(1–3), 17–45 (1981)CrossRef
54.
Xiong, Y., Chakrabarti, A., Basri, R., Gortler, S.J., Jacobs, D.W., Zickler, T.: From shading to local shape. arXiv preprint (2014) [arXiv:1310.2916]
55.
Ying, L., Hertzmann, A., Biermann, H., Zorin, D.: Texture and shape synthesis on surfaces. In: Proceedings of the 12th Eurographics Workshop on Rendering Techniques, pp. 301–312. Springer, Heidelberg (2001)
56.
Zhou, K., Wang, X., Tong, Y., Desbrun, M., Guo, B., Shum, H.Y.: Texturemontage: seamless texturing of arbitrary surfaces from multiple images. ACM Trans. Graph. 24(3), 1148–1155 (2005)CrossRef
Metadata
Title
From Images to Depths and Back
Authors
Tal Hassner
Ronen Basri
Copyright Year
2016
Book
Dense Image Correspondences for Computer Vision

Print ISBN: 978-3-319-23047-4

Electronic ISBN: 978-3-319-23048-1

Copyright Year: 2016

DOI
https://doi.org/10.1007/978-3-319-23048-1_8