Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
International Journal of Computer Vision
Published in: International Journal of Computer Vision 1/2013

01-08-2013

3D Scene Reconstruction from Multiple Spherical Stereo Pairs

Authors: Hansung Kim, Adrian Hilton

Published in: International Journal of Computer Vision | Issue 1/2013

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

We propose a 3D environment modelling method using multiple pairs of high-resolution spherical images. Spherical images of a scene are captured using a rotating line scan camera. Reconstruction is based on stereo image pairs with a vertical displacement between camera views. A 3D mesh model for each pair of spherical images is reconstructed by stereo matching. For accurate surface reconstruction, we propose a PDE-based disparity estimation method which produces continuous depth fields with sharp depth discontinuities even in occluded and highly textured regions. A full environment model is constructed by fusion of partial reconstruction from spherical stereo pairs at multiple widely spaced locations. To avoid camera calibration steps for all camera locations, we calculate 3D rigid transforms between capture points using feature matching and register all meshes into a unified coordinate system. Finally a complete 3D model of the environment is generated by selecting the most reliable observations among overlapped surface measurements considering surface visibility, orientation and distance from the camera. We analyse the characteristics and behaviour of errors for spherical stereo imaging. Performance of the proposed algorithm is evaluated against ground-truth from the Middlebury stereo test bed and LIDAR scans. Results are also compared with conventional structure-from-motion algorithms. The final composite model is rendered from a wide range of viewpoints with high quality textures.
previous article Coupling Image Restoration and Segmentation: A Generalized Linear Model/Bregman Perspective

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Appendix
Available only for authorised users
Footnotes
Literature
Agarwal, S., Snavely, N., Simon, I., Seitz, S., & Szeliski, R. (2009). Building rome in a day. In Proceedings of ICCV, pp. 72–79.
Aiger, D., Mitra, N., & Cohen-Or, D. (2008). 4-points congruent sets for robust surface registration. In Proceedings of SIGGRAPH, pp. 1–10.
Akbarzadeh, A., Frahm, J.-M., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Merrell, P., Phelps, M., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewenius, H., Yang, R., Welch, G., Towles, H., Nister, D., & Pollefeys, M. (2006). Towards urban 3d reconstruction from video. In Proceedings of 3DPVT, pp. 1–8.
Alvarez, L., Deriche, R., Papadopoulo, T., & Sánchez, J. (2007). Symmetrical dense optical flow estimation with oclussions detection. International Journal of Computer Vision, 75(3), 371–385.CrossRef
Alvarez, L., Deriche, R., Sánchez, J., & Weickert, J. (2002). Dense disparity map estimation respecting image discontinuities: A pde and scale-space based approach. Journal of Visual Communication and Image Representation, 13(1), 3–21.CrossRef
Anguelov, D., Dulong, C., Filip, D., Frueh, C., Lafon, S., Lyon, R., et al. (2010). Google street view: Capturing the world at street level. IEEE Computer, 43(6), 32–38.
Asai, T., Kanbara, M., & Yokoya, N. (2005). 3d modeling of outdoor environments by integrating omnidirectional range and color images. In Proceedings of 3DIM, pp. 447–454.
Banno, A., & Ikeuchi, K. (2009). Disparity map refinement and 3d surface smoothing via directed anisotropic diffusion. In Proceedings of 3DIM.
Banno, A., & Ikeuchi, K. (2010). Omnidirectional texturing based on robust 3d registration through euclidean reconstruction from two spherical images. Computer Vision and Image Understanding, 114(4), 491–499.CrossRef
Bay, H., Ess, A., Tuytelaars, T., & Gool, L. (2008). Surf: Speeded up robust features. Computer Vision and Image Understanding, 110, 346–359.CrossRef
Ben-Ari, R., & Sochen, N. (2007). Variational stereo vision with sharp discontinuities and occlusion handling. In Proceedings of ICCV, pp. 1–7.
Benosman, R., & Devars, J. (1998). Panoramic stereovision sensor. In Proceedings of ICPR, pp. 767–769.
Besl, P., & McKay, N. (1992). A method for registration of 3-d shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256.CrossRef
Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004). High accuracy optical flow estimation based on a theory for warping. In Proceedings of ECCV, pp. 25–36.
Burt, P. J. (1981). Fast filter transforms for image processing. Computer Vision, Graphics and Image Processing, 6, 20–51.
Chen, S. (1995). Quicktime vr—An image based approach to virtual environment navigation. In Proceedings of SIGGRAPH, pp. 29–38.
Chen, Y., & Medioni, G. (1992). Object modeling by registration of multiple range images. Image and Vision Computing, 10(3), 145–155.
Cornelis, N., Leibe, B., Cornelis, K., & Gool, L. (2008). 3d urban scene modeling integrating recognition and reconstruction. International Journal of Computer Vision, 78(2), 121–141.CrossRef
Dellaert, F., Seitz, S., Thorpe, C., & Thrun, S. (2000). Structure from motion without correspondence. In Proceedings of CVPR.
Desouza, G., & Kak, A. (2002). Vision for mobile robot navigation: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(2), 237–267.CrossRef
Feldman, D., & Weinshall, D. (2005). Realtime ibr with omnidirectional crossed-slits projection. In Proceedings of ICCV, pp. 839–845.
Fischler, M., & Bolles, R. (1982). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communication of the ACM, 24, 381–395.MathSciNetCrossRef
Frahm, J.-M., Fite-Georgel, P., Gallup, D., Johnson, T., Raguram, R., Wu, C., Jen, Y.-H., Dunn, E., Clipp, B., Lazebnik, S., & Pollefeys, M. (2010). Building rome on a cloudless day. In Proceedings of ECCV, pp. 368–381.
Furukawa, Y., Curless, B., Seitz, S., & Szeliski, R. (2009). Manhattan-world stereo. In Proceedings of CVPR.
Furukawa, Y., Curless, B., Seitz, S., & Szeliski, R. (2010). Towards internet-scale multi-view stereo. In Proceedings of CVPR.
Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8), 1362–1376.CrossRef
Gargallo, P., & Sturm, P. (1988). Bayesian 3d modeling from images using multiple depth maps. In Proceedings of CVPR, pp. 885–891.
Geman, S., & McClure, D. (1985). Bayesian image analysis: An application to single photon emission tomography. In Proceedings of Statistical Computation Section, pp. 12–18.
Goesele, M., Snavely, N., Curless, B., Hoppe, H., & Seitz, S.M. (2007). Multi-view stereo for community photo collections. In Proceedings of ICCV, pp. 368–381.
Granger, S., Pennec, X., & Roche, X. (2001). Rigid point-surface registration using oriented points and an em variant of icp for computer guided oral implantology. In Proceedings of MICCAI, pp. 752–761.
Haala, N., & Kada, M. (2005). Panoramic scenes for texture mapping of 3d city models. In Proceedings of PanoPhot.
Hilton, A. (2005). Scene modelling from sparse 3d data. Image and Vision Computing, 23(10), 900–920.CrossRef
Hilton, A., Stoddart, A., Illingworth, J., & Windeatt, T. (1998). Implicit surface based geometric fusion. Computer Vision and Image Understanding, 69(3), 273–291.CrossRef
Hirschmüller, H., & Scharstein, D. (2008). Evaluation of stereo matching costs on images with radiometric differences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(9), 1582–1599.
Ince, S., & Konrad, J. (2008). Occlusion-aware optical flow estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17, 1443–1451.MathSciNet
Johnson, C. (1988). Numerical solution of partial differential equations by the finite element method. Cambridge: Cambridge University Press.
Kang, S., & Szeliski, R. (1997). 3-d scene data recovery using omnidirectional multibaseline stereo. International Journal of Computer Vision, 25(2), 167–183.CrossRef
Kazhdan, M., Bolitho, M., & Hoppe, H. (2006). Poisson surface reconstruction. In Proceedings of SGP, pp. 61–70.
Kim, H., & Hilton, A. (2009). 3d environment modelling using spherical stereo imaging. In Proceedings of 3DIM.
Kim, H., & Hilton, A. (2010). 3d modelling of static environments using multiple spherical stereo. In Proceedings of RMLE workshop in ECCV.
Kim, H., & Sohn, K. (2003a). Hierarchical depth estimation for image synthesis in mixed reality. In Proceedings of SPIE Electronic Imaging, pp. 544–553.
Kim, H., Sohn, K. (2003b). Hierarchical disparity estimation with energy-based regularization. In Proceedings of ICIP, pp. 373–376.
Klaus, A., Sormann, M., & Karner, K. (2006). Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In Proceedings of ICPR.
Kolmogorov, V., & Zabih, R. (2001). Computing visual correspondence with occlusions using graph cuts. In Proceedings of ICCV.
Lemmens, M. (2007). Airborne lidar sensor. GIM International, 21(2), 13–17.
Lhuillier, M. (2008). Automatic scene structure and camera motion using a catadioptric system. Computer Vision and Image Understanding, 109(2), 186–203.CrossRef
Li, S. (2006). Real-time spherical stereo. In Proceedings of ICPR, pp. 1046–1049.
Mathias, M., Martinovic, A., Weissenberg, J., & Gool, L. J. V. (2011). Procedural 3d building reconstruction using shape grammars and detectors. In Proceedings of 3DIMPVT, pp. 304–311.
Merrell, P., Akbarzadeh, A., Wang, L., Mordohai, P., Frahm, J.-M., Yang, R., et al. (2007). Real-time visibility-based fusion of depth maps. In Proceedings of ICCV.
Micusik, B., & Kosecka, J. (2009). Piecewise planar city 3d modeling from street view panoramic sequences. In Proceedings of CVPR, pp. 2906–2912.
Micusik, B., Martinec, D., & Pajdla, T. (2004). 3d metric reconstruction from uncalibrated omnidirectional images. In Proceedings of ACCV.
Min, D., & Sohn, K. (2008). Cost aggregation and occlusion handling with wls in stereo matching. IEEE Transactions on Image Processing, 17(8), 1431–1442.MathSciNetCrossRef
Nagel, H., & Enkelmann, W. (1986). An investigation of smoothness constraints for the estimation of displacements vector fields from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8, 565–593.CrossRef
Nayar, S.K., & Karmarkar, A. (2000). 360 x 360 mosaics. In Proceedings of CVPR, pp. 2388–2388.
Pollefeys, M., Koch, R., Vergauwen, M., & Gool, L. (2000). Automated reconstruction of 3d scenes from sequences of images. ISPRS Journal of Photogrammetry and Remote Sensing, 55(4), 251–267.CrossRef
Pollefeys, M., Nistér, D., Frahm, J., Akbarzadeh, A., Mordohai, P., Clipp, B., et al. (2008). Detailed real-time urban 3d reconstruction from video. International Journal of Computer Vision, 78(2), 143–167.
Rusinkiewicz, S., & Levoy, M. (2001). Efficient variants of the icp algorithm. In Proceedings of 3DIM, pp. 145–152.
Salman, N., & Yvinec, M. (2009). Surface reconstruction from multi-view stereo. In Proceedings of ACCV.
Scharstein, D., & Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1), 7–42.MATHCrossRef
Simon, L., Teboul, O., Koutsourakis, P., & Paragios, N. (2011). Random exploration of the procedural space for single-view 3d modeling of buildings. International Journal of Computer Vision, 93(2), 253–271.
Sizintsev, M. (2008). Hierarchical stereo with thin structures and transparency. In Proceedings of CRV, pp. 97–104.
Slesareva, N., Bruhn, A., & Weickert, J. (2005). Optic flow goes stereo: A variational method for estimating discontinuity- preserving dense disparity maps. In Proceedings of DAGM, pp. 33–40.
Snavely, N., Seitz, S., & Szeliski, R. (2006). Photo tourism: Exploring photo collections in 3d. In Proceedings of ACM SIGGRAPH, pp. 835–846.
Snavely, N., Seitz, S., & Szeliski, R. (2008). Modeling the world from internet photo collections. International Journal of Computer Vision, 80(2), 189–210.CrossRef
Soucy, M., & Laurendeau, D. (1995). A general surface approach to the integration of a set of range views. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(4), 344–358.
Strecha, C., Fransens, R., & Gool, L. J. V. (2004). Wide-baseline stereo from multiple views: A probabilistic account. In Proceedings of CVPR, pp. 552–559.
Strecha, C., Hansen, W., Gool, L., Fua, P., & Thoennessen, U. (2008). On benchmarking camera calibration and multi-view stereo for high resolution imagery. In Proceedings of CVPR, pp. 1–8.
Sun, D., Roth, S., Lewis, J., & Black, M. (2008). Learning optical flow. In Proceedings of ECCV, pp. 83–97.
Sun, J., Zheng, N., & Shum, H. (2003). Stereo matching using belief propagation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(7), 787–800.CrossRef
Szeliski, R., & Scharstein, D. (2004). Sampling the disparity space image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(3), 419–425.CrossRef
Teller, S., Antone, M., Bodnar, Z., Bosse, M., Coorg, S., Jethwa, M., et al. (2003). Calibrated, registered images of an extended urban area. International Journal of Computer Vision, 53(1), 93–107.CrossRef
Tighe, J., Feldman, J., & Lazebnik, S. (2010). SuperParsing: Scalable Nonparametric Image Parsing with Superpixels. In Proceedings of ECCV.
Turk, G., & Levoy, M. (1994). Zippered polygon meshes from range images. In Proceedings of SIGGRAPH, pp. 311–318.
Vergauwen, M., & Gool, L. (2006). Web-based 3d reconstruction service. Machine Vision Applications, 17, 411–426.CrossRef
Vu, H., Keriven, R., Labatut, P., & Pons, J. (2009). Towards high-resolution large-scale multi-view stereo. In Proceedings of CVPR, pp. 1430–1437.
Weickert, J. (1997). A review of nonlinear diffusion filtering. Lecture Notes in Computer Science, 1252, 3–28.
Williams, J., & Bennamoun, M. (2001). Simultaneous registration of multiple corresponding point sets. Computer Vision and Image Understanding, 81(1), 117–142.MATHCrossRef
Yang, Q., Wang, L., Yang, R., Stewénius, H., & Nistér, D. (2008). Stereo matching with color-weighted correlation, hierarchical belief propagation and occlusion handling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(3), 492–504.CrossRef
Yuille, A., & Poggio, T. (1984). A generalized ordering constraint for stereo correspondence. MIT A.I. Memo 777.
Zimmer, H., Bruhn, A., Valgaerts, L., Breuß, M., Weickert, J., Rosenhahn, B., & Seidel, H. (2008). Pde-based anisotropic disparity-driven stereo vision. In Proceedings of VMV, pp. 263–272.
Zomet, A., Feldman, D., Peleg, S., & Weinshall, D. (2003). Mosaicing new views: The crossed-slits projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(6), 741–754.CrossRef
Metadata
Title
3D Scene Reconstruction from Multiple Spherical Stereo Pairs
Authors
Hansung Kim
Adrian Hilton
Publication date
01-08-2013
Publisher
Springer US
Published in
International Journal of Computer Vision / Issue 1/2013
Print ISSN: 0920-5691
Electronic ISSN: 1573-1405
DOI
https://doi.org/10.1007/s11263-013-0616-1

Other articles of this Issue 1/2013

International Journal of Computer Vision 1/2013 Go to the issue

OriginalPaper

Recovering Relative Depth from Low-Level Features Without Explicit T-junction Detection and Interpretation

OriginalPaper

Coupling Image Restoration and Segmentation: A Generalized Linear Model/Bregman Perspective

OriginalPaper

Linearized Alternating Direction Method with Adaptive Penalty and Warm Starts for Fast Solving Transform Invariant Low-Rank Textures

OriginalPaper

SLEDGE: Sequential Labeling of Image Edges for Boundary Detection

Premium Partner

    Image Credits