Skip to main content
SpringerLink
Log in

Analysis and Approximation of Some Shape-from-Shading Models for Non-Lambertian Surfaces

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

The reconstruction of a 3D object or a scene is a classical inverse problem in Computer Vision. In the case of a single image this is called the Shape-from-Shading (SfS) problem and it is known to be ill-posed even in a simplified version like the vertical light source case. A huge number of works deals with the orthographic SfS problem based on the Lambertian reflectance model, the most common and simplest model which leads to an eikonal-type equation when the light source is on the vertical axis. In this paper, we want to study non-Lambertian models since they are more realistic and suitable whenever one has to deal with different kind of surfaces, rough or specular. We will present a unified mathematical formulation of some popular orthographic non-Lambertian models, considering vertical and oblique light directions as well as different viewer positions. These models lead to more complex stationary non-linear partial differential equations of Hamilton–Jacobi type which can be regarded as the generalization of the classical eikonal equation corresponding to the Lambertian case. However, all the equations corresponding to the models considered here (Oren–Nayar and Phong) have a similar structure so we can look for weak solutions to this class in the viscosity solution framework. Via this unified approach, we are able to develop a semi-Lagrangian approximation scheme for the Oren–Nayar and the Phong model and to prove a general convergence result. Numerical simulations on synthetic and real images will illustrate the effectiveness of this approach and the main features of the scheme, also comparing the results with previous results in the literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. Ahmed, A., Farag, A.: A new formulation for shape from shading for non-Lambertian surfaces. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1817–1824 (2006)

  2. Ahmed, A.H., Farag, A.A.: Shape from Shading under various imaging conditions. In: IEEE International Conference on Computer Vision and Pattern Recognition CVPR’07, pp. X1–X8. Minneapolis (2007)

  3. Bakshi, S., Yang, Y.H.: Shape-from-Shading for non-Lambertian surfaces. In: Proceedings of IEEE International Conference on Image Processing (ICIP), pp. 130–134 (1994)

  4. Barles, G.: Solutions de viscosité des équations de Hamilton-Jacobi. Springer, Berlin (1994)

    MATH  Google Scholar 

  5. Biswas, S., Aggarwal, G., Chellappa, R.: Robust estimation of albedo for illumination-invariant matching and shape recovery. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 884–899 (2009)

    Article  Google Scholar 

  6. Brooks, M., Chojnacki, W., Kozera, R.: Impossible and ambiguous shading patterns. Int. J. Comput. Vis. 7(2), 119–126 (1992)

    Article  Google Scholar 

  7. Brooks, M., Horn, B.: Shape and source from shading. In: Proceedings of the Ninth International Joint Conference on Artificial Intelligence (vol. II), pp. 932–936. Los Angeles (1985)

  8. Bruss, A.: The image irradiance equation: Its solution and application. Technical report ai-tr-623, Massachusetts Institute of Technology (1981)

  9. Camilli, F., Falcone, M.: An approximation scheme for the maximal solution of the shape from shading model. In: Proceedings of the IEEE International Conference on Image Processing (vol. I), vol. 1, pp. 49–52. Lausanne (1996). doi:10.1109/ICIP.1996.559430

  10. Camilli, F., Grüne, L.: Numerical approximation of the maximal solutions for a class of degenerate Hamilton-Jacobi equations. SIAM J. Numer. Anal. 38(5), 1540–1560 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  11. Camilli, F., Siconolfi, A.: Maximal subsolutions of a class of degenerate Hamilton-Jacobi problems. Indiana Univ. Math. J. 48(3), 1111–1131 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  12. Carlini, E., Falcone, M., Festa, A.: A brief survey on semi-lagrangian schemes for image processing. In: M. Breuss, A. Bruckstein, P. Maragos (eds.) Innovations for Shape Analysis: Models and Algorithms, pp. 191–218. Springer (2013). ISBN: 978-3-642-34140-3

  13. Chambolle, A.: A uniqueness result in the theory of stereo vision: coupling shape from shading and binocular information allows unambiguous depth reconstruction. Ann. l’Ist. Henri Poincarè 11(1), 1–16 (1994)

    MathSciNet  MATH  Google Scholar 

  14. Courteille, F., Crouzil, A., Durou, J.D., Gurdjos, P.: Towards shape from Shading under realistic photographic conditions. In: Proceedings of the 17th International Conference on Pattern Recognition, vol. 2, pp. 277–280. Cambridge (2004)

  15. Courteille, F., Crouzil, A., Durou, J.D., Gurdjos, P.: Shape from shading for the digitization of curved documents. Mach. Vis. Appl. 18(5), 301–316 (2007)

    Article  MATH  Google Scholar 

  16. Daniel, P., Durou, J.D.: From deterministic to stochastic methods for shape from shading. In: Proceedings of the Fourth Asian Conference on Computer Vision, pp. 187–192. Taipei, Taiwan (2000)

  17. Diggelen, J.V.: A photometric investigation of the slopes and heights of the ranges of hills in the maria of the moon. Bull. Astron. Inst. Neth. 11(423), 283–290 (1951)

    Google Scholar 

  18. Durou, J.D., Aujol, J.F., Courteille, F.: Integrating the normal field of a surface in the presence of discontinuities. Energy Minim. Methods Comput. Vis. Pattern Recogn. (EMMCVPR) 5681, 261–273 (2009)

    Article  Google Scholar 

  19. Durou, J.D., Falcone, M., Sagona, M.: Numerical methods for shape from shading: a new survey with benchmarks. Comput. Vis. Image Underst. 109(1), 22–43 (2008)

    Article  Google Scholar 

  20. Falcone, M.: Numerical solution of dynamic programming equations. In Bardi, M., Dolcetta, I.C., Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations, pp. 471–504 (1997)

  21. Falcone, M., Ferretti, R.: Semi-Lagrangian approximation schemes for linear and hamilton-jacobi equations. SIAM (2014)

  22. Falcone, M., Sagona, M.: An algorithm for the global solution of the shape from shading model. In: Proceedings of the Ninth International Conference on Image Analysis and Processing (vol. I), vol. 1310 of Lecture Notes in Computer Science, pp. 596–603. Florence (1997)

  23. Falcone, M., Sagona, M., Seghini, A.: A scheme for the shape–from–shading model with “black shadows”. Numer. Math. Adv. Appl. 503–512 (2003)

  24. Fassold, H., Danzl, R., Schindler, K., Bischof, H.: Reconstruction of archaeological finds using shape from stereo and shape from shading. In: Proceedings of the 9th Computer Vision Winter Workshop. Piran, pp. 21–30 (2004)

  25. Favaro, P., Papadhimitri, T.: A closed-form solution to uncalibrated photometric stereo via diffuse maxima. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 821–828 (2012)

  26. Frankot, R., Chellappa, R.: A method for enforcing integrability in shape from shading algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 10(4), 439–451 (1988)

    Article  MATH  Google Scholar 

  27. Grumpe, A., Belkhir, F., Wöhler, C.: Construction of lunar DEMs based on reflectance modelling. Adv. Sp. Res. 53(12), 1735–1767 (2014)

    Article  Google Scholar 

  28. Horn, B.: Shape from shading: a method for obtaining the shape of a smooth opaque object from one view. Ph.D. Thesis, Massachusetts Institute of Technology (1970)

  29. Horn, B.: Obtaining shape from shading information. In: Winston, P.H. (ed.) The Psychology of Computer Vision, vol. 4, pp. 115–155. McGraw-Hill, New York (1975)

    Google Scholar 

  30. Horn, B., Brooks, M.: The variational approach to shape from shading. Comput. Vis. Graph. Image Process. 33(2), 174–208 (1986)

    Article  MATH  Google Scholar 

  31. Horn, B., Brooks, M.: Shape from Shading (Artificial Intelligence). The MIT Press, Cambridge (1989)

    Google Scholar 

  32. Ikeuchi, K., Horn, B.K.: Numerical shape from shading and occluding boundaries. Artif. Intell. 17(1–3), 141–184 (1981)

    Article  Google Scholar 

  33. Ishii, H., Ramaswamy, M.: Uniqueness results for a class of Hamilton–Jacobi equations with singular coefficients. Commun. Partial Differ. Equ. 20, 2187–2213 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  34. Ju, Y., Breuß, M., Bruhn, A., Galliani, S.: Shape from shading for rough surfaces: analysis of the Oren-Nayar model. In: Proceedings of the British Machine Vision Conference (BMVC), pp. 104.1–104.11. BMVA Press (2012). doi:10.5244/C.26.104

  35. Ju, Y., Tozza, S., Breuß, M., Bruhn, A., Kleefeld, A.: Generalised perspective shape from shading with Oren-Nayar reflectance. In: Proceedings of the 24th British Machine Vision Conference (BMVC 2013), pp. 42.1–42.11. BMVA Press, Bristol (2013)

  36. Kain, J., Ostrov, D.: Numerical shape from shading for discontinuous photographic images. Int. J. Comput. Vis. 44(3), 163–173 (2001)

    Article  MATH  Google Scholar 

  37. Kozera, R.: Existence and uniqueness in photometric stereo. Appl. Math. Comput. 44(1), 103 (1991)

    MathSciNet  MATH  Google Scholar 

  38. Kozera, R.: Uniqueness in shape from shading revisited. J. Math. Imaging Vis. 7(2), 123–138 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  39. Kruzkov, S.N.: The generalized solution of the hamilton-jacobi equations of eikonal type i. Math. USSR Sbornik 27, 406–446 (1975)

    Article  MATH  Google Scholar 

  40. Lions, P., Rouy, E., Tourin, A.: Shape-from-shading, viscosity solutions and edges. Numer. Math. 64(3), 323–353 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  41. Lohse, V., Heipke, C., Kirk, R.L.: Derivation of planetary topography using multi-image shape-from-shading. Planetary Sp. Sci. 54(7), 661–674 (2006)

    Article  Google Scholar 

  42. Mecca, R., Falcone, M.: Uniqueness and approximation of a photometric shape-from-shading model. SIAM J. Imaging Sci. 6(1), 616–659 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  43. Mecca, R., Tozza, S.: Shape reconstruction of symmetric surfaces using photometric stereo. In: M. Breuss, A. Bruckstein, P. Maragos (eds.) Innovations for Shape Analysis: Models and Algorithms, pp. 219–243. Springer (2013). ISBN: 978-3-642-34140-3

  44. Okatani, T., Deguchi, K.: Shape reconstruction from an endoscope image by shape from shading technique for a point light source at the projection center. Comput. Vis. Image Underst. 66(2), 119–131 (1997)

    Article  Google Scholar 

  45. Oliensis, J.: Shape from shading as a partially well-constrained problem. Comput. Vis. Graph. Image Process. 54(2), 163–183 (1991)

    MATH  Google Scholar 

  46. Oliensis, J.: Uniqueness in shape from shading. Int. J. Comput. Vis. 6(2), 75–104 (1991)

    Article  MATH  Google Scholar 

  47. Oren, M., Nayar, S.: Diffuse reflectance from rough surfaces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 763–764 (1993)

  48. Oren, M., Nayar, S.: Generalization of Lambert’s reflectance model. In: Proceedings of the International Conference and Exhibition on Computer Graphics and Interactive Techniques (SIGGRAPH), pp. 239–246 (1994)

  49. Oren, M., Nayar, S.: Seeing beyond Lambert’s law. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 269–280 (1994)

  50. Oren, M., Nayar, S.: Generalization of the Lambertian model and implications for machine vision. Int. J. Comput. Vis. 14(3), 227–251 (1995)

    Article  Google Scholar 

  51. Phong, B.T.: Illumination for computer generated pictures. Commun. ACM 18(6), 311–317 (1975)

    Article  Google Scholar 

  52. Prados, E., Camilli, F., Faugeras, O.: A viscosity solution method for shape-from-shading without image boundary data. ESAIM Math. Model. Numer. Anal. 40(2), 393–412 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  53. Prados, E., Faugeras, O.: A mathematical and algorithmic study of the lambertian sfs problem for orthographic and pinhole cameras. Rapport de recherche 5005, Institut National de Recherche en Informatique et en Automatique, Sophia Antipolis (2003)

  54. Prados, E., Faugeras, O.: Perspective shape from shading and viscosity solutions. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 826–831 (2003)

  55. Prados, E., Faugeras, O.: Shape from shading: a well-posed problem? In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 870–877 (2005)

  56. Prados, E., Faugeras, O., Rouy, E.: Shape from shading andviscosity solutions. In: A. Heyden, G. Sparr, M. Nielsen,P. Johansen (eds.) Computer Vision, ECCV 2002, Lecture Notes in Computer Science, vol. 2351, pp. 790–804. Springer, Berlin/Heidelberg (2002)

  57. Quéau, Y., Lauze, F., Durou, J.: Solving uncalibrated photometric stereo using total variation. J. Math. Imaging Vis. 52(1), 87–107 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  58. Ragheb, H., Hancock, E.: Surface normals and height from non-lambertian image data. In: Proceedings of the International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT), pp. 18–25 (2004)

  59. Ragheb, H., Hancock, E.: Surface radiance correction for shape from shading. Pattern Recogn. 38(10), 1574–1595 (2005)

    Article  Google Scholar 

  60. Rindfleisch, T.: Photometric method for lunar topography. Photogramm. Eng. 32(2), 262–277 (1966)

    Google Scholar 

  61. Rouy, E., Tourin, A.: A viscosity solutions approach to shape-from-shading. SIAM J. Numer. Anal. 29(3), 867–884 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  62. Sagona, M.: Numerical methods for degenerate eikonal type equations and applications. Ph.D. thesis, Dipartimento di Matematica dell‘Università di Napoli Federico II, Napoli (2001)

  63. Samaras, D., Metaxas, D.: Incorporating illumination constraints in deformable models for shape from shading and light direction estimation. IEEE Trans. Pattern Anal. Mach. Intell. 25(2), 247–264 (2003)

    Article  Google Scholar 

  64. Smith, W., Hancock, E.: Recovering facial shape and albedo using a statistical model of surface normal direction. In: 10th IEEE International Conference on Computer Vision (ICCV 2005), 2005, Beijing, pp. 588–595 (2005)

  65. Smith, W., Hancock, E.: Estimating facial albedo from a single image. IJPRAI 20(6), 955–970 (2006)

    Google Scholar 

  66. Steffen, W., Koning, N., Wenger, S., Morisset, C., Magnor, M.: Shape: a 3d modeling tool for astrophysics. IEEE Trans. Visual. Comput. Graph. 17(4), 454–465 (2011)

    Article  Google Scholar 

  67. Strat, T.: A numerical method for shape from shading from a single image. Master’s thesis, Department of Electrical Engineering and Computer Science, Massachussetts Institute of Technology, Cambridge (1979)

  68. Szeliski, R.: Fast shape from shading. Comput. Vis. Graph. Image Process. 53(2), 129–153 (1991)

    MATH  Google Scholar 

  69. Tankus, A., Kiryati, N.: Photometric stereo under perspective projection. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), vol. 1, pp. 611–616 (2005)

  70. Tankus, A., Sochen, N., Yeshurun, Y.: Shape-from-shading under perspective projection. Int. J. Comput. Vis. 63(1), 21–43 (2005)

    Article  Google Scholar 

  71. Tozza, S.: Analysis and approximation of non-Lambertian shape-from-shading models. Ph.D. thesis, Dipartimento di Matematica, Sapienza - Universitá di Roma, Rome (2015)

  72. Tozza, S., Falcone, M.: A semi-Lagrangian approximation of the Oren-Nayar PDE for the orthographic shape-from-shading problem. In: S. Battiato, J. Braz (eds.) Proceedings of the 9th International Conference on Computer Vision Theory and Applications (VISAPP), vol. 3, pp. 711–716. SciTePress (2014)

  73. Tozza, S., Falcone, M.: A comparison of non-lambertian models for the shape-from-shading problem. In: M. Breuss, A. Bruckstein, P. Maragos, S. Wuhrer (eds.) New Perspectives in Shape Analysis. Springer (2015)

  74. Tozza, S., Mecca, R., Duocastella, M., Bue, A.D.: Direct differential photometric-stereo shape recovery of diffuse and specular surfaces. J. Math. Imaging Vis. (JMIV) (2016). doi:10.1007/s10851-016-0633-0

  75. Vogel, O., Breuß, M., Leichtweis, T., Weickert, J.: Fast shape from shading for phong-type surfaces. In: X.C. Tai, K. Mørken, M. Lysaker, K.A. Lie (eds.) Scale Space and Variational Methods in Computer Vision, Lecture Notes in Computer Science, vol. 5567, pp. 733–744. Springer (2009)

  76. Vogel, O., Breuß, M., Weickert, J.: Perspective shape from shading with non-Lambertian reflectance. In: Proceedings of the DAGM Symposium on Pattern Recognition, pp. 517–526 (2008)

  77. Vogel, O., Valgaerts, L., Breuß, M., Weickert, J.: Making shape from shading work for real-world images. In: J. Denzler, G. Notni,H. Süße (eds.) Pattern Recognition, Lecture Notes in Computer Science, vol. 5748, pp. 191–200. Springer, Berlin/Heidelberg (2009)

  78. Wikipedia: Mask of Agamemnon. http://en.wikipedia.org/wiki/Mask_of_Agamemnon

  79. Wolff, L.: Diffuse-reflectance model for smooth dielectric surfaces. J. Opt. Soc. Am. A 11(11), 2956–2968 (1994)

    Article  Google Scholar 

  80. Wolff, L.B., Nayar, S.K., Oren, M.: Improved diffuse reflection models for computer vision. Int. J. Comput. Vis. 30(1), 55–71 (1998)

    Article  Google Scholar 

  81. Woodham, R.J.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19(1), 134–144 (1980)

    Article  Google Scholar 

  82. Worthington, P., Hancock, E.: New constraints on data-closeness and needle map consistency for shape-from-shading. IEEE Trans. Pattern Anal. Mach. Intell. 21(12), 1250–1267 (1999)

    Article  Google Scholar 

  83. Wu, C., Narasimhan, S., Jaramaz, B.: A multi-image shape-from-shading framework for near-lighting perspective endoscopes. Int. J. Comput. Vis. 86, 211–228 (2010)

    Article  MathSciNet  Google Scholar 

  84. Yoon, K.J., Prados, E., Sturm, P.: Joint estimation of shape and reflectance using multiple images with known illumination conditions. Int. J. Comput. Vis. 86, 192–210 (2010)

    Article  MathSciNet  Google Scholar 

  85. Zhang, R., Tsai, P.S., Cryer, J., Shah, M.: Shape from shading: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 21(8), 690–706 (1999)

    Article  MATH  Google Scholar 

  86. Zheng, Q., Chellappa, R.: Estimation of illuminant direction, albedo, and shape from shading. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1991. Proceedings CVPR ’91. pp. 540–545 (1991)

Download references

Acknowledgments

The first author wishes to acknowledge the support obtained by the INDAM under the GNCS research project “Metodi ad alta risoluzione per problemi evolutivi fortemente nonlineari” and the hospitality obtained by Prof. Edwin Hancock of the University of York, Department of Computer Science, during her scholarship “Borsa di perfezionamento all’estero” paid by Sapienza—Università di Roma.

Author information

Authors and Affiliations

  1. Dipartimento di Matematica, Sapienza - Università di Roma, Piazzale Aldo Moro, 5, 00185, Rome, Italy

    Silvia Tozza & Maurizio Falcone

Authors
  1. Silvia Tozza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maurizio Falcone
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Silvia Tozza.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tozza, S., Falcone, M. Analysis and Approximation of Some Shape-from-Shading Models for Non-Lambertian Surfaces. J Math Imaging Vis 55, 153–178 (2016). https://doi.org/10.1007/s10851-016-0636-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-016-0636-x

Keywords

Search

Navigation