Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
International Journal of Computer Vision
Published in: International Journal of Computer Vision 2/2017

23-07-2016

MultiCol Bundle Adjustment: A Generic Method for Pose Estimation, Simultaneous Self-Calibration and Reconstruction for Arbitrary Multi-Camera Systems

Authors: Steffen Urban, Sven Wursthorn, Jens Leitloff, Stefan Hinz

Published in: International Journal of Computer Vision | Issue 2/2017

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In this paper, we present a generic, modular bundle adjustment method for pose estimation, simultaneous self-calibration and reconstruction for multi-camera systems. In contrast to other approaches that use bearing vectors (camera rays) as observations, we extend the common collinearity equations with a general camera model and include the relative orientation of each camera w.r.t to the fixed multi-camera system frame yielding the extended collinearity equations that directly express all image observations as functions of all unknowns. Hence, we can either calibrate the camera system, the cameras, reconstruct the observed scene, and/or simply estimate the pose of the system by including the corresponding parameter block into the Jacobian matrix. Apart from evaluating the implementation with comprehensive simulations, we benchmark our method against recently published methods for pose estimation and bundle adjustment for multi-camera systems. Finally, all methods are evaluated using a 6 degree of freedom ground truth data set, that was recorded with a lasertracker.
previous article Minimal Basis Subspace Representation: A Unified Framework for Rigid and Non-rigid Motion Segmentation
next article Large-Scale Gaussian Process Inference with Generalized Histogram Intersection Kernels for Visual Recognition Tasks

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Literature
Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S. M., et al. (2011). Building Rome in a day. Communications of the ACM, 54(10), 105–112.CrossRef
Agarwal, S., & Mierle, K. (2012). Ceres solver: Tutorial & reference. Mountain View: Google Inc.
Baker, P., & Aloimonos, Y. (2000). Complete calibration of a multi-camera network. In Proceedings IEEE Workshop on Omnidirectional Vision (pp. 134–141). IEEE.
Barreto, J., & Daniilidis, K. (2004). Wide area multiple camera calibration and estimation of radial distortion. In Omnivis-2004, ECCV-2004 Workshop.
Blanco-Claraco, J. L., Moreno-Dueñas, F. Á., & González-Jiménez, J. (2014). The Málaga urban dataset: High-rate stereo and LiDAR in a realistic urban scenario. The International Journal of Robotics Research, 33(2), 207–214.CrossRef
Brown, D. C. (1966). Decentering distortion of lenses. Photometric Engineering, 32(3), 444–462.
Brückner, M., Bajramovic, F., & Denzler, J. (2014). Intrinsic and extrinsic active self-calibration of multi-camera systems. Machine Vision and Applications, 25(2), 389–403.CrossRef
Carrera, G., Angeli, A., & Davison, A.J. (2011). SLAM-based automatic extrinsic calibration of a multi-camera rig. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 2652–2659). IEEE.
Ceriani, S., Fontana, G., Giusti, A., Marzorati, D., Matteucci, M., Migliore, D., et al. (2009). Rawseeds ground truth collection systems for indoor self-localization and mapping. Autonomous Robots, 27(4), 353–371.CrossRef
Clipp, B., Kim, J.H., Frahm, J.M., Pollefeys, M., & Hartley, R. (2008). Robust 6dof motion estimation for non-overlapping, multi-camera systems. In IEEE Workshop on Applications of Computer Vision. WACV 2008 (pp. 1–8). IEEE.
Davison, A.J., Cid, Y.G., Kita, N. (2004). Real-time 3d SLAM with wide-angle vision. In Proceedings of the IFAC/EURON Symposium on Intelligent Autonomous Vehicles.
Duane, C. B. (1971). Close-range camera calibration. Photogrammetric Engineering, 37, 855–866.
Ess, A., Neubeck, A., Van Gool, L.J. (2007). Generalised linear pose estimation. In BMVC (pp. 1–10).
Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.MathSciNetCrossRef
Förstner, W. (2010). Minimal representations for uncertainty and estimation in projective spaces. In Computer Vision–ACCV 2010 (pp. 619–632). Springer, Berlin.
Frahm, J.M., Köser, K., & Koch, R. (2004). Pose estimation for multi-camera systems. In Pattern Recognition (pp. 286–293). Springer, Berlin.
Frank, O., Katz, R., Tisse, C.L., & Durrant-Whyte, H.F. (2007). Camera calibration for miniature, low-cost, wide-angle imaging systems. In BMVC (pp. 1–10). Citeseer.
Gao, X. S., Hou, X. R., Tang, J., & Cheng, H. F. (2003). Complete solution classification for the perspective-three-point problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(8), 930–943.CrossRef
Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012 (pp. 3354–3361). IEEE
Geyer, C., Daniilidis, K. (2000). A unifying theory for central panoramic systems and practical implications. In Computer VisionECCV 2000 (pp. 445–461). Springer, Berlin.
Hartley, R., & Zisserman, A. (2003). Multiple view geometry in computer vision. Cambridge: Cambridge university press.MATH
Hesch, J.A., Roumeliotis, S.I. (2011). A direct least-squares (DLS) method for PnP. In IEEE International Conference on Computer Vision (ICCV) (pp. 383–390). IEEE.
Kannala, J., & Brandt, S. S. (2006). A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(8), 1335–1340.CrossRef
Kneip, L., & Furgale, P. (2014). OpenGV: A unified and generalized approach to real-time calibrated geometric vision. InIEEE International Conference on Robotics and Automation (ICRA) (pp. 1–8). IEEE.
Kneip, L., Furgale, P., & Siegwart, R. (2013) Using multi-camera systems in robotics: Efficient solutions to the NPnP problem. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 3770–3776). IEEE.
Kneip, L., Li, H., & Seo, Y. (2014). UPnP: An optimal O(n) solution to the absolute pose problem with universal applicability. In Computer Vision–ECCV (pp. 127–142). Springer, Berlin.
Kneip, L., Scaramuzza, D., & Siegwart, R. (2011). A novel parametrization of the perspective-three-point problem for a direct computation of absolute camera position and orientation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2969–2976). IEEE.
Kumar, R.K., Ilie, A., Frahm. J.M., & Pollefeys, M. (2008). Simple calibration of non-overlapping cameras with a mirror. In IEEE Conference on, Computer Vision and Pattern Recognition. CVPR 2008 (pp. 1–7). IEEE
Kummerle, R., Grisetti, G., Strasdat, H., Konolige, K., & Burgard, W. (2011). \(g^2o\): A general framework for graph optimization. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 3607–3613). IEEE
Kurillo, G., Li, Z., & Bajcsy, R. (2008). Wide-area external multi-camera calibration using vision graphs and virtual calibration object. In Second ACM/IEEE International Conference on In: Distributed Smart Cameras, ICDSC 2008 (pp. 1–9). IEEE
Lepetit, V., Moreno-Noguer, F., & Fua, P. (2009). EPnP: An accurate O(n) solution to the PnP problem. International Journal of Computer Vision, 81(2), 155–166.CrossRef
Li, B., Heng, L., Koser, K., & Pollefeys M. (2013). A multiple-camera system calibration toolbox using a feature descriptor-based calibration pattern. iN IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2013 (pp. 1301–1307). IEEE
Lourakis, M. I., & Argyros, A. A. (2009). SBA: A software package for generic sparse bundle adjustment. ACM Transactions on Mathematical Software (TOMS), 36(1), 2.MathSciNetCrossRef
Ly, D. S., Demonceaux, C., Vasseur, P., & Pégard, C. (2014). Extrinsic calibration of heterogeneous cameras by line images. Machine Vision and Applications, 25(6), 1601–1614.CrossRef
Maas, H. G. (1999). Image sequence based automatic multi-camera system calibration techniques. ISPRS Journal of Photogrammetry and Remote Sensing, 54(5), 352–359.CrossRef
Mei, C., & Rives, P. (2007). Single view point omnidirectional camera calibration from planar grids. In IEEE International Conference on Robotics and Automation
Micušık, B. (2004). Two-view geometry of omnidirectional cameras. Ph.D. Thesis, Czech Technical University.
Muhle, D., Abraham, S., Heipke, C., & Wiggenhagen, M. (2011). Estimating the mutual orientation in a multi-camera system with a non overlapping field of view. In Photogrammetric Image Analysis (pp. 13–24). Springer, Berlin
Scaramuzza, D., Martinelli, A., & Siegwart, R. (2006a). A flexible technique for accurate omnidirectional camera calibration and structure from motion. In IEEE International Conference on Computer Vision Systems, 2006 ICVS’06 (pp. 45–45). IEEE
Scaramuzza, D., Martinelli, A., & Siegwart, R. (2006b). A toolbox for easily calibrating omnidirectional cameras. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 5695–5701). IEEE
Schönbein, M., & Geiger, A. (2014) Omnidirectional 3d reconstruction in augmented manhattan worlds. In International Conference on Intelligent Robots and Systems (IROS).
Schneider, J., & Förstner, W. (2013). Bundle adjustment and system calibration with points at infinity for omnidirectional camera systems. Z. f. Photogrammetrie, Fernerkundung und Geoinformation, 4, 309–321. doi:10.​1127/​1432-8364/​2013/​0179.CrossRef
Schneider, J., Schindler, F., Läbe, T., & Förstner, W. (2012). Bundle adjustment for multi-camera systems with points at infinity. In XXII ISPRS congress–technical commission III. ISPRS annals of the photogrammetry, remote sensing and spatial information sciences, vol. I-3 (pp. 75–80). Melbourne.
Schönbein, M., Straus, T., & Geiger, A. (2014). Calibrating and centering quasi-central catadioptric cameras. In IEEE International Conference on Robotics and Automation (ICRA), 2014 (pp. 4443–4450). IEEE.
Schweighofer, G., & Pinz, A. (2008). Globally optimal O(n) solution to the PnP problem for general camera models. In: BMVC (pp. 1–10).
Sturm, J., Engelhard, N., Endres, F., Burgard, W., & Cremers, D. (2012). A benchmark for the evaluation of RGB-D SLAM systems. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012 (pp. 573–580). IEEE.
Triggs, B., McLauchlan, P.F., Hartley, R.I., & Fitzgibbon, A.W. (2000). Bundle adjustment-a modern synthesis. In: Vision Algorithms: Theory and Practice (pp. 298–372). Springer, Berlin.
Urban, S., Leitloff, J., & Hinz, S. (2015). Improved wide-angle, fisheye and omnidirectional camera calibration. ISPRS Journal of Photogrammetry and Remote Sensing, 108, 72–79.CrossRef
Zhang, Z. (1999). Flexible camera calibration by viewing a plane from unknown orientations. In The Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 1 (pp. 666–673). IEEE.
Zheng, Y., Kuang, Y., Sugimoto, S., Astrom, K., & Okutomi, M. (2013). Revisiting the PnP problem: A fast, general and optimal solution. In IEEE International Conference on Computer Vision (ICCV) (pp. 2344–2351). IEEE.
Metadata
Title
MultiCol Bundle Adjustment: A Generic Method for Pose Estimation, Simultaneous Self-Calibration and Reconstruction for Arbitrary Multi-Camera Systems
Authors
Steffen Urban
Sven Wursthorn
Jens Leitloff
Stefan Hinz
Publication date
23-07-2016
Publisher
Springer US
Published in
International Journal of Computer Vision / Issue 2/2017
Print ISSN: 0920-5691
Electronic ISSN: 1573-1405
DOI
https://doi.org/10.1007/s11263-016-0935-0

Other articles of this Issue 2/2017

International Journal of Computer Vision 2/2017 Go to the issue

OriginalPaper

Behavior Discovery and Alignment of Articulated Object Classes from Unstructured Video

OriginalPaper

Weighted Nuclear Norm Minimization and Its Applications to Low Level Vision

OriginalPaper

A Portable Bio-Inspired Architecture for Efficient Robotic Vergence Control

OriginalPaper

Minimal Basis Subspace Representation: A Unified Framework for Rigid and Non-rigid Motion Segmentation

OriginalPaper

Large-Scale Gaussian Process Inference with Generalized Histogram Intersection Kernels for Visual Recognition Tasks

Premium Partner

    Image Credits