nach oben

International Journal of Computer Vision

Erschienen in:

01.08.2013

3D Scene Reconstruction from Multiple Spherical Stereo Pairs

verfasst von: Hansung Kim, Adrian Hilton

Erschienen in: International Journal of Computer Vision | Ausgabe 1/2013

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

We propose a 3D environment modelling method using multiple pairs of high-resolution spherical images. Spherical images of a scene are captured using a rotating line scan camera. Reconstruction is based on stereo image pairs with a vertical displacement between camera views. A 3D mesh model for each pair of spherical images is reconstructed by stereo matching. For accurate surface reconstruction, we propose a PDE-based disparity estimation method which produces continuous depth fields with sharp depth discontinuities even in occluded and highly textured regions. A full environment model is constructed by fusion of partial reconstruction from spherical stereo pairs at multiple widely spaced locations. To avoid camera calibration steps for all camera locations, we calculate 3D rigid transforms between capture points using feature matching and register all meshes into a unified coordinate system. Finally a complete 3D model of the environment is generated by selecting the most reliable observations among overlapped surface measurements considering surface visibility, orientation and distance from the camera. We analyse the characteristics and behaviour of errors for spherical stereo imaging. Performance of the proposed algorithm is evaluated against ground-truth from the Middlebury stereo test bed and LIDAR scans. Results are also compared with conventional structure-from-motion algorithms. The final composite model is rendered from a wide range of viewpoints with high quality textures.

Vorheriger Artikel Coupling Image Restoration and Segmentation: A Generalized Linear Model/Bregman Perspective

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nur mit Berechtigung zugänglich

City group, http://city.csail.mit.edu/

Pointgrey, http://www.ptgrey.com/

Spheron, http://www.spheron.com/en/spheron-cgi/products/spherocam-hdr.html

Middlebury multi-view, http://vision.middlebury.edu/mview/

Middlebury stereo, http://vision.middlebury.edu/stereo/

Agarwal, S., Snavely, N., Simon, I., Seitz, S., & Szeliski, R. (2009). Building rome in a day. In Proceedings of ICCV, pp. 72–79.

Aiger, D., Mitra, N., & Cohen-Or, D. (2008). 4-points congruent sets for robust surface registration. In Proceedings of SIGGRAPH, pp. 1–10.

Akbarzadeh, A., Frahm, J.-M., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Merrell, P., Phelps, M., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewenius, H., Yang, R., Welch, G., Towles, H., Nister, D., & Pollefeys, M. (2006). Towards urban 3d reconstruction from video. In Proceedings of 3DPVT, pp. 1–8.

Alvarez, L., Deriche, R., Papadopoulo, T., & Sánchez, J. (2007). Symmetrical dense optical flow estimation with oclussions detection. International Journal of Computer Vision, 75(3), 371–385.CrossRef

Alvarez, L., Deriche, R., Sánchez, J., & Weickert, J. (2002). Dense disparity map estimation respecting image discontinuities: A pde and scale-space based approach. Journal of Visual Communication and Image Representation, 13(1), 3–21.CrossRef

Anguelov, D., Dulong, C., Filip, D., Frueh, C., Lafon, S., Lyon, R., et al. (2010). Google street view: Capturing the world at street level. IEEE Computer, 43(6), 32–38.

Asai, T., Kanbara, M., & Yokoya, N. (2005). 3d modeling of outdoor environments by integrating omnidirectional range and color images. In Proceedings of 3DIM, pp. 447–454.

Banno, A., & Ikeuchi, K. (2009). Disparity map refinement and 3d surface smoothing via directed anisotropic diffusion. In Proceedings of 3DIM.

Banno, A., & Ikeuchi, K. (2010). Omnidirectional texturing based on robust 3d registration through euclidean reconstruction from two spherical images. Computer Vision and Image Understanding, 114(4), 491–499.CrossRef

Bay, H., Ess, A., Tuytelaars, T., & Gool, L. (2008). Surf: Speeded up robust features. Computer Vision and Image Understanding, 110, 346–359.CrossRef

Ben-Ari, R., & Sochen, N. (2007). Variational stereo vision with sharp discontinuities and occlusion handling. In Proceedings of ICCV, pp. 1–7.

Benosman, R., & Devars, J. (1998). Panoramic stereovision sensor. In Proceedings of ICPR, pp. 767–769.

Besl, P., & McKay, N. (1992). A method for registration of 3-d shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256.CrossRef

Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004). High accuracy optical flow estimation based on a theory for warping. In Proceedings of ECCV, pp. 25–36.

Burt, P. J. (1981). Fast filter transforms for image processing. Computer Vision, Graphics and Image Processing, 6, 20–51.

Chen, S. (1995). Quicktime vr—An image based approach to virtual environment navigation. In Proceedings of SIGGRAPH, pp. 29–38.

Chen, Y., & Medioni, G. (1992). Object modeling by registration of multiple range images. Image and Vision Computing, 10(3), 145–155.

Cornelis, N., Leibe, B., Cornelis, K., & Gool, L. (2008). 3d urban scene modeling integrating recognition and reconstruction. International Journal of Computer Vision, 78(2), 121–141.CrossRef

Dellaert, F., Seitz, S., Thorpe, C., & Thrun, S. (2000). Structure from motion without correspondence. In Proceedings of CVPR.

Desouza, G., & Kak, A. (2002). Vision for mobile robot navigation: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(2), 237–267.CrossRef

Feldman, D., & Weinshall, D. (2005). Realtime ibr with omnidirectional crossed-slits projection. In Proceedings of ICCV, pp. 839–845.

Fischler, M., & Bolles, R. (1982). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communication of the ACM, 24, 381–395.MathSciNetCrossRef

Fisher, R. (2007). Registration and fusion of range images. Cvonline, http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FISHER/REGIS/regis.html.

Frahm, J.-M., Fite-Georgel, P., Gallup, D., Johnson, T., Raguram, R., Wu, C., Jen, Y.-H., Dunn, E., Clipp, B., Lazebnik, S., & Pollefeys, M. (2010). Building rome on a cloudless day. In Proceedings of ECCV, pp. 368–381.

Furukawa, Y., Curless, B., Seitz, S., & Szeliski, R. (2009). Manhattan-world stereo. In Proceedings of CVPR.

Furukawa, Y., Curless, B., Seitz, S., & Szeliski, R. (2010). Towards internet-scale multi-view stereo. In Proceedings of CVPR.

Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8), 1362–1376.CrossRef

Gargallo, P., & Sturm, P. (1988). Bayesian 3d modeling from images using multiple depth maps. In Proceedings of CVPR, pp. 885–891.

Geman, S., & McClure, D. (1985). Bayesian image analysis: An application to single photon emission tomography. In Proceedings of Statistical Computation Section, pp. 12–18.

Goesele, M., Snavely, N., Curless, B., Hoppe, H., & Seitz, S.M. (2007). Multi-view stereo for community photo collections. In Proceedings of ICCV, pp. 368–381.

Granger, S., Pennec, X., & Roche, X. (2001). Rigid point-surface registration using oriented points and an em variant of icp for computer guided oral implantology. In Proceedings of MICCAI, pp. 752–761.

Haala, N., & Kada, M. (2005). Panoramic scenes for texture mapping of 3d city models. In Proceedings of PanoPhot.

Hilton, A. (2005). Scene modelling from sparse 3d data. Image and Vision Computing, 23(10), 900–920.CrossRef

Hilton, A., Stoddart, A., Illingworth, J., & Windeatt, T. (1998). Implicit surface based geometric fusion. Computer Vision and Image Understanding, 69(3), 273–291.CrossRef

Hirschmüller, H., & Scharstein, D. (2008). Evaluation of stereo matching costs on images with radiometric differences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(9), 1582–1599.

Ince, S., & Konrad, J. (2008). Occlusion-aware optical flow estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17, 1443–1451.MathSciNet

Johnson, C. (1988). Numerical solution of partial differential equations by the finite element method. Cambridge: Cambridge University Press.

Kang, S., & Szeliski, R. (1997). 3-d scene data recovery using omnidirectional multibaseline stereo. International Journal of Computer Vision, 25(2), 167–183.CrossRef

Kazhdan, M., Bolitho, M., & Hoppe, H. (2006). Poisson surface reconstruction. In Proceedings of SGP, pp. 61–70.

Kim, H., & Hilton, A. (2009). 3d environment modelling using spherical stereo imaging. In Proceedings of 3DIM.

Kim, H., & Hilton, A. (2010). 3d modelling of static environments using multiple spherical stereo. In Proceedings of RMLE workshop in ECCV.

Kim, H., & Sohn, K. (2003a). Hierarchical depth estimation for image synthesis in mixed reality. In Proceedings of SPIE Electronic Imaging, pp. 544–553.

Kim, H., Sohn, K. (2003b). Hierarchical disparity estimation with energy-based regularization. In Proceedings of ICIP, pp. 373–376.

Klaus, A., Sormann, M., & Karner, K. (2006). Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In Proceedings of ICPR.

Kolmogorov, V., & Zabih, R. (2001). Computing visual correspondence with occlusions using graph cuts. In Proceedings of ICCV.

Lemmens, M. (2007). Airborne lidar sensor. GIM International, 21(2), 13–17.

Lhuillier, M. (2008). Automatic scene structure and camera motion using a catadioptric system. Computer Vision and Image Understanding, 109(2), 186–203.CrossRef

Li, S. (2006). Real-time spherical stereo. In Proceedings of ICPR, pp. 1046–1049.

Mathias, M., Martinovic, A., Weissenberg, J., & Gool, L. J. V. (2011). Procedural 3d building reconstruction using shape grammars and detectors. In Proceedings of 3DIMPVT, pp. 304–311.

Merrell, P., Akbarzadeh, A., Wang, L., Mordohai, P., Frahm, J.-M., Yang, R., et al. (2007). Real-time visibility-based fusion of depth maps. In Proceedings of ICCV.

Micusik, B., & Kosecka, J. (2009). Piecewise planar city 3d modeling from street view panoramic sequences. In Proceedings of CVPR, pp. 2906–2912.

Micusik, B., Martinec, D., & Pajdla, T. (2004). 3d metric reconstruction from uncalibrated omnidirectional images. In Proceedings of ACCV.

Min, D., & Sohn, K. (2008). Cost aggregation and occlusion handling with wls in stereo matching. IEEE Transactions on Image Processing, 17(8), 1431–1442.MathSciNetCrossRef

Nagel, H., & Enkelmann, W. (1986). An investigation of smoothness constraints for the estimation of displacements vector fields from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8, 565–593.CrossRef

Nayar, S.K., & Karmarkar, A. (2000). 360 x 360 mosaics. In Proceedings of CVPR, pp. 2388–2388.

Pollefeys, M., Koch, R., Vergauwen, M., & Gool, L. (2000). Automated reconstruction of 3d scenes from sequences of images. ISPRS Journal of Photogrammetry and Remote Sensing, 55(4), 251–267.CrossRef

Pollefeys, M., Nistér, D., Frahm, J., Akbarzadeh, A., Mordohai, P., Clipp, B., et al. (2008). Detailed real-time urban 3d reconstruction from video. International Journal of Computer Vision, 78(2), 143–167.

Rusinkiewicz, S., & Levoy, M. (2001). Efficient variants of the icp algorithm. In Proceedings of 3DIM, pp. 145–152.

Salman, N., & Yvinec, M. (2009). Surface reconstruction from multi-view stereo. In Proceedings of ACCV.

Scharstein, D., & Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1), 7–42.MATHCrossRef

Simon, L., Teboul, O., Koutsourakis, P., & Paragios, N. (2011). Random exploration of the procedural space for single-view 3d modeling of buildings. International Journal of Computer Vision, 93(2), 253–271.

Sizintsev, M. (2008). Hierarchical stereo with thin structures and transparency. In Proceedings of CRV, pp. 97–104.

Slesareva, N., Bruhn, A., & Weickert, J. (2005). Optic flow goes stereo: A variational method for estimating discontinuity- preserving dense disparity maps. In Proceedings of DAGM, pp. 33–40.

Snavely, N., Seitz, S., & Szeliski, R. (2006). Photo tourism: Exploring photo collections in 3d. In Proceedings of ACM SIGGRAPH, pp. 835–846.

Snavely, N., Seitz, S., & Szeliski, R. (2008). Modeling the world from internet photo collections. International Journal of Computer Vision, 80(2), 189–210.CrossRef

Soucy, M., & Laurendeau, D. (1995). A general surface approach to the integration of a set of range views. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(4), 344–358.

Strecha, C., Fransens, R., & Gool, L. J. V. (2004). Wide-baseline stereo from multiple views: A probabilistic account. In Proceedings of CVPR, pp. 552–559.

Strecha, C., Hansen, W., Gool, L., Fua, P., & Thoennessen, U. (2008). On benchmarking camera calibration and multi-view stereo for high resolution imagery. In Proceedings of CVPR, pp. 1–8.

Sun, D., Roth, S., Lewis, J., & Black, M. (2008). Learning optical flow. In Proceedings of ECCV, pp. 83–97.

Sun, J., Zheng, N., & Shum, H. (2003). Stereo matching using belief propagation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(7), 787–800.CrossRef

Szeliski, R., & Scharstein, D. (2004). Sampling the disparity space image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(3), 419–425.CrossRef

Teller, S., Antone, M., Bodnar, Z., Bosse, M., Coorg, S., Jethwa, M., et al. (2003). Calibrated, registered images of an extended urban area. International Journal of Computer Vision, 53(1), 93–107.CrossRef

Tighe, J., Feldman, J., & Lazebnik, S. (2010). SuperParsing: Scalable Nonparametric Image Parsing with Superpixels. In Proceedings of ECCV.

Turk, G., & Levoy, M. (1994). Zippered polygon meshes from range images. In Proceedings of SIGGRAPH, pp. 311–318.

Vergauwen, M., & Gool, L. (2006). Web-based 3d reconstruction service. Machine Vision Applications, 17, 411–426.CrossRef

Vu, H., Keriven, R., Labatut, P., & Pons, J. (2009). Towards high-resolution large-scale multi-view stereo. In Proceedings of CVPR, pp. 1430–1437.

Weickert, J. (1997). A review of nonlinear diffusion filtering. Lecture Notes in Computer Science, 1252, 3–28.

Williams, J., & Bennamoun, M. (2001). Simultaneous registration of multiple corresponding point sets. Computer Vision and Image Understanding, 81(1), 117–142.MATHCrossRef

Yang, Q., Wang, L., Yang, R., Stewénius, H., & Nistér, D. (2008). Stereo matching with color-weighted correlation, hierarchical belief propagation and occlusion handling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(3), 492–504.CrossRef

Yuille, A., & Poggio, T. (1984). A generalized ordering constraint for stereo correspondence. MIT A.I. Memo 777.

Zimmer, H., Bruhn, A., Valgaerts, L., Breuß, M., Weickert, J., Rosenhahn, B., & Seidel, H. (2008). Pde-based anisotropic disparity-driven stereo vision. In Proceedings of VMV, pp. 263–272.

Zomet, A., Feldman, D., Peleg, S., & Weinshall, D. (2003). Mosaicing new views: The crossed-slits projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(6), 741–754.CrossRef

Titel: 3D Scene Reconstruction from Multiple Spherical Stereo Pairs
verfasst von: Hansung Kim
Adrian Hilton
Publikationsdatum: 01.08.2013
Verlag: Springer US
Erschienen in: International Journal of Computer Vision / Ausgabe 1/2013
Print ISSN: 0920-5691
Elektronische ISSN: 1573-1405
DOI: https://doi.org/10.1007/s11263-013-0616-1

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Weitere Artikel der Ausgabe 1/2013

Recovering Relative Depth from Low-Level Features Without Explicit T-junction Detection and Interpretation

Linearized Alternating Direction Method with Adaptive Penalty and Warm Starts for Fast Solving Transform Invariant Low-Rank Textures

SLEDGE: Sequential Labeling of Image Edges for Boundary Detection

Coupling Image Restoration and Segmentation: A Generalized Linear Model/Bregman Perspective