Top

International Journal of Computer Vision

Published in:

12-02-2019

One-Day Outdoor Photometric Stereo Using Skylight Estimation

Authors: Jiyoung Jung, Joon-Young Lee, In So Kweon

Published in: International Journal of Computer Vision | Issue 8/2019

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

We present an outdoor photometric stereo method using images captured in a single day. We simulate a sky hemisphere for each image according to its GPS and timestamp and parameterize the obtained sky hemisphere into quadratic skylight and Gaussian sunlight distributions. Our previous work recovered an outdoor scene on a clear day, whereas the current paper shows that cloudy days can provide better illumination conditions for surface orientation recovery, and hence we propose a modified sky model to represent a well-conditioned skylight distribution for outdoor photometric stereo. The proposed method models natural illumination according to a sky model, providing sufficient constraints for shape reconstruction from 1-day images. We tested the proposed method to recover various sized objects and scenes from real-world outdoor daylight images and verified the method using synthetic and real data experiments.

previous article Video Enhancement with Task-Oriented Flow

next article Correction to: One-Day Outdoor Photometric Stereo Using Skylight Estimation

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Abrams, A., Hawley, C., & Pless, R. (2012). Heliometric stereo: Shape from sun position. In Proceedings of European conference on computer vision (ECCV).

Abrams, A., Hawley, C., Miskell, K., Stoica, A., Jacobs, N., & Pless, R. (2013a). Shadow estimation method for “the episolar constraint: Monocular shape from shadow correspondence”. arXiv:1304.4112.

Abrams, A., Miskell, K., & Pless, R. (2013b). The episolar constraint: Monocular shape from shadow correspondence. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR).

Ackermann, J., Langguth, F., Fuhrmann, S., & Goesele, M. (2012). Photometric stereo for outdoor webcams. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR).

Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., & Seitz, S. M. (2011). Building Rome in a day. Communications of the ACM, 54(10), 105–112.CrossRef

Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multi-view stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 32(8), 1362–1376.CrossRef

Google Earth. (2015). https://earth.google.com.

Grossmann, P. (1987). Depth from focus. Pattern Recognition Letters, 5(1), 63–69.MathSciNetCrossRef

Han, Y., Lee, J. Y., & Kweon, I. S. (2013). High quality shape from a single RGB-D image under uncalibrated natural illumination. In Proceedings of international conference on computer vision (ICCV).

Hold-Geoffroy, Y., Zhang, J., Gotardo, P. F. U., & LaLonde, J. F. (2015a). What is a good day for outdoor photometric stereo? In International conference on computational photography.

Hold-Geoffroy, Y., Zhang, J., Gotardo, P. F. U., & LaLonde, J. F. (2015b). x-hour outdoor photometric stereo. In International conference on 3D vision.

Hold-Geoffroy, Y., Sunkavalli, K., Hadap, S., Gambaretto, E., & LaLonde, J. F. (2017). Deep outdoor illumination estimation. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR).

Horn, B. K. P. (1989). Obtaining shape from shading information. In Shape from shading (pp. 123–171). Cambridge, MA, USA: MIT Press.

Hošek, L., & Wilkie, A. (2013). Adding a solar radiance function to the Hošek–Wilkie skylight model. IEEE Computer Graphics and Applications, 33(3), 44–52.CrossRef

Huang, R., & Smith, W. A. P. (2011). Shape-from-shading under complex natural illumination. In Proceedings of international conference on image processing (ICIP).

Hung, C. H., Wu, T. P., Matsushita, Y., Xu, L., Jia, J., & Tang, C. K. (2015). Photometric stereo in the wild. In Proceedings of IEEE Winter Conference on Applications of Computer Vision (WACV). Waikoloa, HI: IEEE.

Inose, K., Shimizu, S., Kawakami, R., Mukaigawa, Y., & Ikeuchi, K. (2013). Refining outdoor photometric stereo based on sky model. IPSJ Transaction on Computer Vision and Applications, 5(1), 104–108.CrossRef

Jacobs, N., Roman, N., & Pless, R. (2007). Consistent temporal variations in many outdoor scenes. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR).

Johnson, M. K., & Adelson, E. H. (2011). Shape estimation in natural illumination. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR).

Jung, J. (2016). Outdoor photometric stereo for 3D scene recovery. Ph.D. Thesis, KAIST, Daejeon, Republic of Korea.

Jung, J., Lee, J. Y., Jeong, Y., & Kweon, I. S. (2015a). Time-of-flight sensor calibration for a color and depth camera pair. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 37(7), 1501–1513.CrossRef

Jung, J., Lee, J. Y., & Kweon, I. S. (2015b). One-day outdoor photometric stereo via skylight estimation. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR).

Kawakami, R., Zhao, H., Tan, R. T., & Ikeuchi, K. (2013). Camera spectral sensitivity and white balance estimation from sky images. International Journal on Computer Vision (IJCV), 105(3), 187–204.MATHCrossRef

Koenderink, J. J., van Doorn, A. J., & Kappers, A. M. L. (1992). Surface perception in pictures. Perception and Psychophysics, 52(5), 487–496.CrossRef

Kolmogorov, V., & Zabih, R. (2004). What energy functions can be minimized via graph cuts? IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 26(2), 147–159.MATHCrossRef

Lalonde, J. F., Narasimhan, S. G., & Efros, A. A. (2010). What do the sun and the sky tell us about the camera? International Journal on Computer Vision (IJCV), 88(1), 24–51.CrossRef

Lalonde, J. F., Efros, A. A., & Narasimhan, S. G. (2012). Estimating natural illumination from a single outdoor image. International Journal on Computer Vision (IJCV), 98(2), 123–145.MathSciNetCrossRef

Lalonde, J. F., Asselin, L. P., Becirovski, J., Hold-Geoffroy, Y., Garon, M., Gardner, M. A., & Zhang, J. (2016). The Laval HDR sky database. http://www.hdrdb.com. Accessed 1 October 2015.

Lee, J. Y., Matsushita, Y., Shi, B., & Kweon, I. S. (2013). Radiometric calibration by rank minimization. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 35(1), 114–156.CrossRef

Lombardi, S., & Nishino, K. (2012). Reflectance and illumination from a single image. In Proceedings of European conference on computer vision (ECCV).

Lu, F., Matsushita, Y., Sato, I., Okabe, T., & Sato, Y. (2013). Uncalibrated photometric stereo for unknown isotropic reflectances. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR).

Okutomi, M., & Kanade, T. (1993). A multiple-baseline stereo system. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 15(4), 353–363.CrossRef

Oxholm, G., & Nishino, K. (2012). Shape and reflectance from natural illumination. In Proceedings of European conference on computer vision (ECCV).

Preetham, A. J., Shirley, P., & Smits, B. (1999). A practical analytic model for daylight. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques (SIGGRAPH ’99) (pp. 91–100). ACM Press/Addison-Wesley Publishing Co.

Ramamoorthi, R., & Hanrahan, P. (2001). On the relationship between radiance and irradiance: Determining the illumination from images of a convex lambertian object. Journal of the Optical Society of America A (JOSA A), 18(10), 2448–2459.MathSciNetCrossRef

Saff, E., & Kuijlaars, A. B. J. (1997). Distributing many points on a sphere. The Mathematical Intelligencer, 19(1), 5–11.MathSciNetMATHCrossRef

Scharstein, D., & Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal on Computer Vision (IJCV), 47(13), 7–42.MATHCrossRef

Shan, Q., Adams, R., Curless, B., Furukawa, Y., & Seitz, S. M. (2013). The visual turing test for scene reconstruction. In Proceedings of international conference on 3D vision.

Shen, F., Sunkavalli, K., Bonneel, N., Rusinkiewicz, S., Pfister, H., & Tong, X. (2014). Time-lapse photometric stereo and applications. Computer Graphics Forum, 33(7), 359–367.CrossRef

Shi, B., Matsushita, Y., Wei, Y., Xu, C., & Tan, P. (2010). Self-calibrating photometric stereo. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR).

Simchony, T., Chellappa, R., & Shao, M. (1990). Direct analytical methods for solving poisson equations in computer vision problems. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 12(5), 435–446.MATHCrossRef

Stumpfel, J., Tchou, C., Jones, A., Hawkins, T., Wenger, A., & Debevec, P. (2004). Direct HDR capture of the sun and sky. In Proceedings of the 3rd International Conference on Computer Graphics, Virtual Reality, Visualisation and Interaction in Africa (AFRIGRAPH) (pp. 145–149). ACM.

Subbarao, M., & Surya, G. (1994). Depth from defocus: A spatial domain approach. International Journal on Computer Vision (IJCV), 13(3), 271–294.CrossRef

Sunkavalli, K., Matusik, W., Pfister, H., & Rusinkiewicz, S. (2007). Factored time-lapse video. ACM Transaction on Graphics (TOG), 26(3), 101–111.CrossRef

Sunkavalli, K., Romeiro, F., Matusik, W., Zickler, T., & Pfister, H. (2008). What do color changes reveal about an outdoor scene? In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR).

Turk, G., & Levoy, M. (1994). Zippered polygon meshes from range images. In Proceedings of the 21st annual conference on Computer graphics and interactive techniques (SIGGRAPH ’94) (pp. 311-318). ACM.

Woodham, R. (1980). Photometric method for determining surface orientation from multiple images. Optical Engineering, 19(1), 139–144.CrossRef

Yoon, K. J., & Kweon, I. S. (2006). Adaptive support-weight approach for correspondence search. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 28(4), 650–656.CrossRef

Yu, L. F., Yeung, S. K., Tai, Y. W., & Lin, S. (2013). Shading-based shape refinement of RGB-D images. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR).

Zhang, Q., Ye, M., Yang, R., Matsushita, Y., Wilburn, B., & Yu, H. (2012). Edge-preserving photometric stereo via depth fusion. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR).

Title: One-Day Outdoor Photometric Stereo Using Skylight Estimation
Authors: Jiyoung Jung
Joon-Young Lee
In So Kweon
Publication date: 12-02-2019
Publisher: Springer US
Published in: International Journal of Computer Vision / Issue 8/2019
Print ISSN: 0920-5691
Electronic ISSN: 1573-1405
DOI: https://doi.org/10.1007/s11263-018-01145-1

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 8/2019

Visibility Enhancement by Integrating Refocusing and Direct-Global Separation with Contact Imaging

Fast Abnormal Event Detection

A Robust Monocular 3D Object Tracking Method Combining Statistical and Photometric Constraints

Contrast Invariant SNR and Isotonic Regressions

Rank-1 Tensor Approximation for High-Order Association in Multi-target Tracking

Online Mutual Foreground Segmentation for Multispectral Stereo Videos

Premium Partner