Skip to main content
Erschienen in: Soft Computing 19/2020

27.03.2020 | Methodologies and Application

Multi-sensor data fusion for accurate surface modeling

verfasst von: Mahesh K. Singh, Ashish Dutta, K. S. Venkatesh

Erschienen in: Soft Computing | Ausgabe 19/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Multi-sensor data fusion is advantageous while fusing data from heterogeneous range sensors, for scanning a scene containing both fine and coarse details. This paper presents a new multi-sensor range data fusion method with the aim to increase the descriptive contents of the entire generated surface model. First, a new training framework of the scanned range dataset to solve the relaxed Gaussian mixture model-based method by applying the convex relaxation technique is presented. The classification of the range data is based on a trained statistical model. In the data fusion experiments, a laser range sensor and Kinect (V1) are used. Based on the segmentation criterion, the range data fusion is performed by integration of the finer regions range data obtained from a laser range sensor with the coarser regions of the Kinect range data. The fused range information overcomes the weaknesses of the respective range sensors, i.e., the laser scanner is accurate but takes time while the Kinect is fast but not very accurate. The surface model of the fused range dataset generates a highly accurate, realistic surface model of the scene. The experimental results demonstrate robustness of the proposed approach.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Literatur
Zurück zum Zitat Aliakbarpour H, Ferreira JF, Prasath VS, Palaniappan K, Seetharaman G, Dias J (2017) A probabilistic fusion framework for 3-D reconstruction using heterogeneous sensors. IEEE Sens J 17(9):2640–2641CrossRef Aliakbarpour H, Ferreira JF, Prasath VS, Palaniappan K, Seetharaman G, Dias J (2017) A probabilistic fusion framework for 3-D reconstruction using heterogeneous sensors. IEEE Sens J 17(9):2640–2641CrossRef
Zurück zum Zitat An SY, Lee LK, Oh SY (2012) Fast incremental 3D plane extraction from a collection of 2d line segments for 3D mapping. In: IEEE/RSJ International conference on intelligent robots and systems (IROS). IEEE, pp 4530–4537 An SY, Lee LK, Oh SY (2012) Fast incremental 3D plane extraction from a collection of 2d line segments for 3D mapping. In: IEEE/RSJ International conference on intelligent robots and systems (IROS). IEEE, pp 4530–4537
Zurück zum Zitat Bae E, Yuan J, Tai XC (2011) Global minimization for continuous multiphase partitioning problems using a dual approach. Int J Comput Vis 92(1):112–129MathSciNetCrossRef Bae E, Yuan J, Tai XC (2011) Global minimization for continuous multiphase partitioning problems using a dual approach. Int J Comput Vis 92(1):112–129MathSciNetCrossRef
Zurück zum Zitat Bastonero P, Donadio E, Chiabrando F, Spanò A (2014) Fusion of 3D models derived from tls and image-based techniques for CH enhanced documentation. ISPRS Ann Photogramm Remote Sens Spat Inf Sci 2(5):73CrossRef Bastonero P, Donadio E, Chiabrando F, Spanò A (2014) Fusion of 3D models derived from tls and image-based techniques for CH enhanced documentation. ISPRS Ann Photogramm Remote Sens Spat Inf Sci 2(5):73CrossRef
Zurück zum Zitat Besl PJ, McKay ND (1992) Method for registration of 3-D shapes. In: Robotics-DL tentative, international society for optics and photonics, pp 586–606 Besl PJ, McKay ND (1992) Method for registration of 3-D shapes. In: Robotics-DL tentative, international society for optics and photonics, pp 586–606
Zurück zum Zitat Bishop CM (2006) Pattern recognition and machine learning. Springer, New YorkMATH Bishop CM (2006) Pattern recognition and machine learning. Springer, New YorkMATH
Zurück zum Zitat Budzan S (2014) Fusion of visual and range images for object extraction. In: International conference on computer vision and graphics. Springer, Berlin, pp 108–115 Budzan S (2014) Fusion of visual and range images for object extraction. In: International conference on computer vision and graphics. Springer, Berlin, pp 108–115
Zurück zum Zitat Budzan S, Kasprzyk J (2016) Fusion of 3D laser scanner and depth images for obstacle recognition in mobile applications. Opt Lasers Eng 77:230–240CrossRef Budzan S, Kasprzyk J (2016) Fusion of 3D laser scanner and depth images for obstacle recognition in mobile applications. Opt Lasers Eng 77:230–240CrossRef
Zurück zum Zitat Chan Y, Delmas P, Gimel’frab G, Valkenburg R (2008) On fusion of active range and passive stereo data for 3D scene modeling. In: 23rd International conference on image and vision computing New Zealand, IVCNZ Chan Y, Delmas P, Gimel’frab G, Valkenburg R (2008) On fusion of active range and passive stereo data for 3D scene modeling. In: 23rd International conference on image and vision computing New Zealand, IVCNZ
Zurück zum Zitat Chane CS, Mansouri A, Marzani FS, Boochs F (2013) Integration of 3D and multispectral data for cultural heritage applications: survey and perspectives. Image Vis Comput 31(1):91–102CrossRef Chane CS, Mansouri A, Marzani FS, Boochs F (2013) Integration of 3D and multispectral data for cultural heritage applications: survey and perspectives. Image Vis Comput 31(1):91–102CrossRef
Zurück zum Zitat Chávez A, Karstoft H (2012) Improvement of kinecttm sensor capabilities by fusion with laser sensing data using octree. Sensors 12(4):3868–3878CrossRef Chávez A, Karstoft H (2012) Improvement of kinecttm sensor capabilities by fusion with laser sensing data using octree. Sensors 12(4):3868–3878CrossRef
Zurück zum Zitat Dal Mutto C, Zanuttigh P, Mattoccia S, Cortelazzo G (2012) Locally consistent tof and stereo data fusion. In: International conference on computer vision, ECCV’12, pp 598–607 Dal Mutto C, Zanuttigh P, Mattoccia S, Cortelazzo G (2012) Locally consistent tof and stereo data fusion. In: International conference on computer vision, ECCV’12, pp 598–607
Zurück zum Zitat Dias P, Sequeira V, Gonçalves JG, Vaz F (2002) Automatic registration of laser reflectance and colour intensity images for 3D reconstruction. Robot Auton Syst 39(3):157–168CrossRef Dias P, Sequeira V, Gonçalves JG, Vaz F (2002) Automatic registration of laser reflectance and colour intensity images for 3D reconstruction. Robot Auton Syst 39(3):157–168CrossRef
Zurück zum Zitat Dias P, Sequeira V, Vaz F, Gonçalves JG (2003) Registration and fusion of intensity and range data for 3D modelling of real world scenes. In: Fourth international conference on 3-D digital imaging and modeling. IEEE, pp 418–425 Dias P, Sequeira V, Vaz F, Gonçalves JG (2003) Registration and fusion of intensity and range data for 3D modelling of real world scenes. In: Fourth international conference on 3-D digital imaging and modeling. IEEE, pp 418–425
Zurück zum Zitat Elseberg J, Magnenat S, Siegwart R, Nüchter A (2012) Comparison of nearest-neighbor-search strategies and implementations for efficient shape registration. J Softw Eng Robot 3(1):2–12 Elseberg J, Magnenat S, Siegwart R, Nüchter A (2012) Comparison of nearest-neighbor-search strategies and implementations for efficient shape registration. J Softw Eng Robot 3(1):2–12
Zurück zum Zitat Elstrom MD, Smith PW (1999) Stereo-based registration of multi-sensor imagery for enhanced visualization of remote environments. In: IEEE international conference on robotics and automation, vol 3. IEEE, pp 1948–1953 Elstrom MD, Smith PW (1999) Stereo-based registration of multi-sensor imagery for enhanced visualization of remote environments. In: IEEE international conference on robotics and automation, vol 3. IEEE, pp 1948–1953
Zurück zum Zitat Evangelidis GD, Hansard M, Horaud R (2015) Fusion of range and stereo data for high-resolution scene-modeling. IEEE Trans Pattern Anal Mach Intell 37(11):2178–2192CrossRef Evangelidis GD, Hansard M, Horaud R (2015) Fusion of range and stereo data for high-resolution scene-modeling. IEEE Trans Pattern Anal Mach Intell 37(11):2178–2192CrossRef
Zurück zum Zitat Fryskowska A, Walczykowski P, Delis P, Wojtkowska M (2015) Als and tls data fusion in cultural heritage documentation and modeling. Int Arch Photogramm Rem Sens Spat Inf Sci 40(5):147CrossRef Fryskowska A, Walczykowski P, Delis P, Wojtkowska M (2015) Als and tls data fusion in cultural heritage documentation and modeling. Int Arch Photogramm Rem Sens Spat Inf Sci 40(5):147CrossRef
Zurück zum Zitat Gerardo-Castro MP, Peynot T, Ramos F (2015) Laser-radar data fusion with Gaussian process implicit surfaces. In: Field and service robotics. Springer, Berlin, pp 289–302 Gerardo-Castro MP, Peynot T, Ramos F (2015) Laser-radar data fusion with Gaussian process implicit surfaces. In: Field and service robotics. Springer, Berlin, pp 289–302
Zurück zum Zitat Hannachi A, Kohler S, Lallement A, Hirsch E (2014) Multi-sensor data fusion for realistic and accurate 3D reconstruction. In: 2014 5th European workshop on visual information processing (EUVIP). IEEE, pp 1–6 Hannachi A, Kohler S, Lallement A, Hirsch E (2014) Multi-sensor data fusion for realistic and accurate 3D reconstruction. In: 2014 5th European workshop on visual information processing (EUVIP). IEEE, pp 1–6
Zurück zum Zitat Henry P, Krainin M, Herbst E, Ren X, Fox D (2012) RGB-D mapping: using kinect-style depth cameras for dense 3D modeling of indoor environments. Int J Robot Res 31(5):647–663CrossRef Henry P, Krainin M, Herbst E, Ren X, Fox D (2012) RGB-D mapping: using kinect-style depth cameras for dense 3D modeling of indoor environments. Int J Robot Res 31(5):647–663CrossRef
Zurück zum Zitat Hess M, Petrovic V, Meyer D, Rissolo D, Kuester F (2015) Fusion of multimodal three-dimensional data for comprehensive digital documentation of cultural heritage sites. In: Digital heritage, 2015, vol 2. IEEE, pp 595–602 Hess M, Petrovic V, Meyer D, Rissolo D, Kuester F (2015) Fusion of multimodal three-dimensional data for comprehensive digital documentation of cultural heritage sites. In: Digital heritage, 2015, vol 2. IEEE, pp 595–602
Zurück zum Zitat Himmelsbach M, Muller A, Luttel T, Wunsche HJ (2008) LIDAR based 3D object perception. In: Proceedings of 1st international workshop on cognition for technical systems Himmelsbach M, Muller A, Luttel T, Wunsche HJ (2008) LIDAR based 3D object perception. In: Proceedings of 1st international workshop on cognition for technical systems
Zurück zum Zitat Jolliffe I (2005) Principal component analysis. Wiley, New YorkMATH Jolliffe I (2005) Principal component analysis. Wiley, New YorkMATH
Zurück zum Zitat Khoshelham K, Elberink SO (2012) Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2):1437–1454CrossRef Khoshelham K, Elberink SO (2012) Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2):1437–1454CrossRef
Zurück zum Zitat Kläß J, Stückler J, Behnke S (2012) Efficient mobile robot navigation using 3D surfel grid maps. In: Conference on robotics (ROBOTIK), VDE, pp 1–4 Kläß J, Stückler J, Behnke S (2012) Efficient mobile robot navigation using 3D surfel grid maps. In: Conference on robotics (ROBOTIK), VDE, pp 1–4
Zurück zum Zitat Lalonde JF, Vandapel N, Huber DF, Hebert M (2006) Natural terrain classification using three-dimensional ladar data for ground robot mobility. J Field Robot 23(10):839–862CrossRef Lalonde JF, Vandapel N, Huber DF, Hebert M (2006) Natural terrain classification using three-dimensional ladar data for ground robot mobility. J Field Robot 23(10):839–862CrossRef
Zurück zum Zitat Li C, Kao CY, Gore JC, Ding Z (2008) Minimization of region-scalable fitting energy for image segmentation. IEEE Trans Image Process 17(10):1940–1949MathSciNetCrossRef Li C, Kao CY, Gore JC, Ding Z (2008) Minimization of region-scalable fitting energy for image segmentation. IEEE Trans Image Process 17(10):1940–1949MathSciNetCrossRef
Zurück zum Zitat McLachlan G, Krishnan T (2007) The EM algorithm and extensions, vol 382. Wiley, New YorkMATH McLachlan G, Krishnan T (2007) The EM algorithm and extensions, vol 382. Wiley, New YorkMATH
Zurück zum Zitat Papoulis A, Pillai SU (2002) Probability, random variables, and stochastic processes. Tata McGraw-Hill Education, New York City Papoulis A, Pillai SU (2002) Probability, random variables, and stochastic processes. Tata McGraw-Hill Education, New York City
Zurück zum Zitat Rockafellar RT (2015) Convex analysis. Princeton University Press, Princeton Rockafellar RT (2015) Convex analysis. Princeton University Press, Princeton
Zurück zum Zitat Rusu RB, Marton ZC, Blodow N, Holzbach A, Beetz M (2009) Model-based and learned semantic object labeling in 3D point cloud maps of kitchen environments. In: IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 3601–3608 Rusu RB, Marton ZC, Blodow N, Holzbach A, Beetz M (2009) Model-based and learned semantic object labeling in 3D point cloud maps of kitchen environments. In: IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 3601–3608
Zurück zum Zitat Singh MK, Venkatesh KS, Dutta A (2014) Accurate 3D terrain modeling by range data fusion from two heterogeneous range scanners. In: Annual IEEE India conference (INDICON), pp 1–6 Singh MK, Venkatesh KS, Dutta A (2014) Accurate 3D terrain modeling by range data fusion from two heterogeneous range scanners. In: Annual IEEE India conference (INDICON), pp 1–6
Zurück zum Zitat Singh MK, Venkatesh KS, Dutta A (2015) Range data fusion for accurate surface generation from heterogeneous range scanners. In: International conference on informatics in control, automation and robotics (ICINCO), vol 02, pp 444–449 Singh MK, Venkatesh KS, Dutta A (2015) Range data fusion for accurate surface generation from heterogeneous range scanners. In: International conference on informatics in control, automation and robotics (ICINCO), vol 02, pp 444–449
Zurück zum Zitat Stamos I, Allen P (2000) 3-D model construction using range and image data. In: IEEE conference on computer vision and pattern recognition, vol 1. IEEE, pp 531–536 Stamos I, Allen P (2000) 3-D model construction using range and image data. In: IEEE conference on computer vision and pattern recognition, vol 1. IEEE, pp 531–536
Zurück zum Zitat Trevor AJ, Rogers J, Christensen HI (2012) Planar surface slam with 3D and 2D sensors. In: IEEE international conference on robotics and automation (ICRA). IEEE, pp 3041–3048 Trevor AJ, Rogers J, Christensen HI (2012) Planar surface slam with 3D and 2D sensors. In: IEEE international conference on robotics and automation (ICRA). IEEE, pp 3041–3048
Zurück zum Zitat Tsai R (1987) A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J Robot Autom 3(4):323–344CrossRef Tsai R (1987) A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J Robot Autom 3(4):323–344CrossRef
Zurück zum Zitat Vasudevan S (2012) Data fusion with gaussian processes. Robot Auton Syst 60(12):1528–1544CrossRef Vasudevan S (2012) Data fusion with gaussian processes. Robot Auton Syst 60(12):1528–1544CrossRef
Zurück zum Zitat Wang L, Ding L, Ding X, Fang C (2011) 2D face fitting-assisted 3D face reconstruction for pose-robust face recognition. Soft Comput 15(3):417–428CrossRef Wang L, Ding L, Ding X, Fang C (2011) 2D face fitting-assisted 3D face reconstruction for pose-robust face recognition. Soft Comput 15(3):417–428CrossRef
Zurück zum Zitat Wu Y, Liu C, Lan S, Yang M (2015) Real-time 3D road scene based on virtual-real fusion method. IEEE Sens J 15(2):750–756CrossRef Wu Y, Liu C, Lan S, Yang M (2015) Real-time 3D road scene based on virtual-real fusion method. IEEE Sens J 15(2):750–756CrossRef
Zurück zum Zitat Yang Q, Tan KH, Culbertson B, Apostolopoulos J (2010) Fusion of active and passive sensors for fast 3D capture. In: IEEE international workshop on multimedia signal processing (MMSP), pp 69–74 Yang Q, Tan KH, Culbertson B, Apostolopoulos J (2010) Fusion of active and passive sensors for fast 3D capture. In: IEEE international workshop on multimedia signal processing (MMSP), pp 69–74
Zurück zum Zitat Yue H, Chen W, Wu X, Liu J (2014) Fast 3D modeling in complex environments using a single kinect sensor. Opt Lasers Eng 53:104–111CrossRef Yue H, Chen W, Wu X, Liu J (2014) Fast 3D modeling in complex environments using a single kinect sensor. Opt Lasers Eng 53:104–111CrossRef
Zurück zum Zitat Zhang Z (2012) Microsoft kinect sensor and its effect. IEEE Multimed 19(2):4–10CrossRef Zhang Z (2012) Microsoft kinect sensor and its effect. IEEE Multimed 19(2):4–10CrossRef
Zurück zum Zitat Zhu J, Wang L, Gao J, Yang R (2010) Spatial-temporal fusion for high accuracy depth maps using dynamic MRFs. IEEE Trans Pattern Anal Mach Intell 32:899–909CrossRef Zhu J, Wang L, Gao J, Yang R (2010) Spatial-temporal fusion for high accuracy depth maps using dynamic MRFs. IEEE Trans Pattern Anal Mach Intell 32:899–909CrossRef
Zurück zum Zitat Zhu J, Wang L, Yang R, Davis JE, Pan Z (2011) Reliability fusion of time-of-flight depth and stereo geometry for high quality depth maps. IEEE Trans Pattern Anal Mach Intell 33(7):1400–1414CrossRef Zhu J, Wang L, Yang R, Davis JE, Pan Z (2011) Reliability fusion of time-of-flight depth and stereo geometry for high quality depth maps. IEEE Trans Pattern Anal Mach Intell 33(7):1400–1414CrossRef
Metadaten
Titel
Multi-sensor data fusion for accurate surface modeling
verfasst von
Mahesh K. Singh
Ashish Dutta
K. S. Venkatesh
Publikationsdatum
27.03.2020
Verlag
Springer Berlin Heidelberg
Erschienen in
Soft Computing / Ausgabe 19/2020
Print ISSN: 1432-7643
Elektronische ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-020-04797-9

Weitere Artikel der Ausgabe 19/2020

Soft Computing 19/2020 Zur Ausgabe