Skip to main content
Top
Published in: Soft Computing 19/2020

27-03-2020 | Methodologies and Application

Multi-sensor data fusion for accurate surface modeling

Authors: Mahesh K. Singh, Ashish Dutta, K. S. Venkatesh

Published in: Soft Computing | Issue 19/2020

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Multi-sensor data fusion is advantageous while fusing data from heterogeneous range sensors, for scanning a scene containing both fine and coarse details. This paper presents a new multi-sensor range data fusion method with the aim to increase the descriptive contents of the entire generated surface model. First, a new training framework of the scanned range dataset to solve the relaxed Gaussian mixture model-based method by applying the convex relaxation technique is presented. The classification of the range data is based on a trained statistical model. In the data fusion experiments, a laser range sensor and Kinect (V1) are used. Based on the segmentation criterion, the range data fusion is performed by integration of the finer regions range data obtained from a laser range sensor with the coarser regions of the Kinect range data. The fused range information overcomes the weaknesses of the respective range sensors, i.e., the laser scanner is accurate but takes time while the Kinect is fast but not very accurate. The surface model of the fused range dataset generates a highly accurate, realistic surface model of the scene. The experimental results demonstrate robustness of the proposed approach.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Literature
go back to reference Aliakbarpour H, Ferreira JF, Prasath VS, Palaniappan K, Seetharaman G, Dias J (2017) A probabilistic fusion framework for 3-D reconstruction using heterogeneous sensors. IEEE Sens J 17(9):2640–2641CrossRef Aliakbarpour H, Ferreira JF, Prasath VS, Palaniappan K, Seetharaman G, Dias J (2017) A probabilistic fusion framework for 3-D reconstruction using heterogeneous sensors. IEEE Sens J 17(9):2640–2641CrossRef
go back to reference An SY, Lee LK, Oh SY (2012) Fast incremental 3D plane extraction from a collection of 2d line segments for 3D mapping. In: IEEE/RSJ International conference on intelligent robots and systems (IROS). IEEE, pp 4530–4537 An SY, Lee LK, Oh SY (2012) Fast incremental 3D plane extraction from a collection of 2d line segments for 3D mapping. In: IEEE/RSJ International conference on intelligent robots and systems (IROS). IEEE, pp 4530–4537
go back to reference Bae E, Yuan J, Tai XC (2011) Global minimization for continuous multiphase partitioning problems using a dual approach. Int J Comput Vis 92(1):112–129MathSciNetCrossRef Bae E, Yuan J, Tai XC (2011) Global minimization for continuous multiphase partitioning problems using a dual approach. Int J Comput Vis 92(1):112–129MathSciNetCrossRef
go back to reference Bastonero P, Donadio E, Chiabrando F, Spanò A (2014) Fusion of 3D models derived from tls and image-based techniques for CH enhanced documentation. ISPRS Ann Photogramm Remote Sens Spat Inf Sci 2(5):73CrossRef Bastonero P, Donadio E, Chiabrando F, Spanò A (2014) Fusion of 3D models derived from tls and image-based techniques for CH enhanced documentation. ISPRS Ann Photogramm Remote Sens Spat Inf Sci 2(5):73CrossRef
go back to reference Besl PJ, McKay ND (1992) Method for registration of 3-D shapes. In: Robotics-DL tentative, international society for optics and photonics, pp 586–606 Besl PJ, McKay ND (1992) Method for registration of 3-D shapes. In: Robotics-DL tentative, international society for optics and photonics, pp 586–606
go back to reference Bishop CM (2006) Pattern recognition and machine learning. Springer, New YorkMATH Bishop CM (2006) Pattern recognition and machine learning. Springer, New YorkMATH
go back to reference Budzan S (2014) Fusion of visual and range images for object extraction. In: International conference on computer vision and graphics. Springer, Berlin, pp 108–115 Budzan S (2014) Fusion of visual and range images for object extraction. In: International conference on computer vision and graphics. Springer, Berlin, pp 108–115
go back to reference Budzan S, Kasprzyk J (2016) Fusion of 3D laser scanner and depth images for obstacle recognition in mobile applications. Opt Lasers Eng 77:230–240CrossRef Budzan S, Kasprzyk J (2016) Fusion of 3D laser scanner and depth images for obstacle recognition in mobile applications. Opt Lasers Eng 77:230–240CrossRef
go back to reference Chan Y, Delmas P, Gimel’frab G, Valkenburg R (2008) On fusion of active range and passive stereo data for 3D scene modeling. In: 23rd International conference on image and vision computing New Zealand, IVCNZ Chan Y, Delmas P, Gimel’frab G, Valkenburg R (2008) On fusion of active range and passive stereo data for 3D scene modeling. In: 23rd International conference on image and vision computing New Zealand, IVCNZ
go back to reference Chane CS, Mansouri A, Marzani FS, Boochs F (2013) Integration of 3D and multispectral data for cultural heritage applications: survey and perspectives. Image Vis Comput 31(1):91–102CrossRef Chane CS, Mansouri A, Marzani FS, Boochs F (2013) Integration of 3D and multispectral data for cultural heritage applications: survey and perspectives. Image Vis Comput 31(1):91–102CrossRef
go back to reference Chávez A, Karstoft H (2012) Improvement of kinecttm sensor capabilities by fusion with laser sensing data using octree. Sensors 12(4):3868–3878CrossRef Chávez A, Karstoft H (2012) Improvement of kinecttm sensor capabilities by fusion with laser sensing data using octree. Sensors 12(4):3868–3878CrossRef
go back to reference Dal Mutto C, Zanuttigh P, Mattoccia S, Cortelazzo G (2012) Locally consistent tof and stereo data fusion. In: International conference on computer vision, ECCV’12, pp 598–607 Dal Mutto C, Zanuttigh P, Mattoccia S, Cortelazzo G (2012) Locally consistent tof and stereo data fusion. In: International conference on computer vision, ECCV’12, pp 598–607
go back to reference Dias P, Sequeira V, Gonçalves JG, Vaz F (2002) Automatic registration of laser reflectance and colour intensity images for 3D reconstruction. Robot Auton Syst 39(3):157–168CrossRef Dias P, Sequeira V, Gonçalves JG, Vaz F (2002) Automatic registration of laser reflectance and colour intensity images for 3D reconstruction. Robot Auton Syst 39(3):157–168CrossRef
go back to reference Dias P, Sequeira V, Vaz F, Gonçalves JG (2003) Registration and fusion of intensity and range data for 3D modelling of real world scenes. In: Fourth international conference on 3-D digital imaging and modeling. IEEE, pp 418–425 Dias P, Sequeira V, Vaz F, Gonçalves JG (2003) Registration and fusion of intensity and range data for 3D modelling of real world scenes. In: Fourth international conference on 3-D digital imaging and modeling. IEEE, pp 418–425
go back to reference Elseberg J, Magnenat S, Siegwart R, Nüchter A (2012) Comparison of nearest-neighbor-search strategies and implementations for efficient shape registration. J Softw Eng Robot 3(1):2–12 Elseberg J, Magnenat S, Siegwart R, Nüchter A (2012) Comparison of nearest-neighbor-search strategies and implementations for efficient shape registration. J Softw Eng Robot 3(1):2–12
go back to reference Elstrom MD, Smith PW (1999) Stereo-based registration of multi-sensor imagery for enhanced visualization of remote environments. In: IEEE international conference on robotics and automation, vol 3. IEEE, pp 1948–1953 Elstrom MD, Smith PW (1999) Stereo-based registration of multi-sensor imagery for enhanced visualization of remote environments. In: IEEE international conference on robotics and automation, vol 3. IEEE, pp 1948–1953
go back to reference Evangelidis GD, Hansard M, Horaud R (2015) Fusion of range and stereo data for high-resolution scene-modeling. IEEE Trans Pattern Anal Mach Intell 37(11):2178–2192CrossRef Evangelidis GD, Hansard M, Horaud R (2015) Fusion of range and stereo data for high-resolution scene-modeling. IEEE Trans Pattern Anal Mach Intell 37(11):2178–2192CrossRef
go back to reference Fryskowska A, Walczykowski P, Delis P, Wojtkowska M (2015) Als and tls data fusion in cultural heritage documentation and modeling. Int Arch Photogramm Rem Sens Spat Inf Sci 40(5):147CrossRef Fryskowska A, Walczykowski P, Delis P, Wojtkowska M (2015) Als and tls data fusion in cultural heritage documentation and modeling. Int Arch Photogramm Rem Sens Spat Inf Sci 40(5):147CrossRef
go back to reference Gerardo-Castro MP, Peynot T, Ramos F (2015) Laser-radar data fusion with Gaussian process implicit surfaces. In: Field and service robotics. Springer, Berlin, pp 289–302 Gerardo-Castro MP, Peynot T, Ramos F (2015) Laser-radar data fusion with Gaussian process implicit surfaces. In: Field and service robotics. Springer, Berlin, pp 289–302
go back to reference Hannachi A, Kohler S, Lallement A, Hirsch E (2014) Multi-sensor data fusion for realistic and accurate 3D reconstruction. In: 2014 5th European workshop on visual information processing (EUVIP). IEEE, pp 1–6 Hannachi A, Kohler S, Lallement A, Hirsch E (2014) Multi-sensor data fusion for realistic and accurate 3D reconstruction. In: 2014 5th European workshop on visual information processing (EUVIP). IEEE, pp 1–6
go back to reference Henry P, Krainin M, Herbst E, Ren X, Fox D (2012) RGB-D mapping: using kinect-style depth cameras for dense 3D modeling of indoor environments. Int J Robot Res 31(5):647–663CrossRef Henry P, Krainin M, Herbst E, Ren X, Fox D (2012) RGB-D mapping: using kinect-style depth cameras for dense 3D modeling of indoor environments. Int J Robot Res 31(5):647–663CrossRef
go back to reference Hess M, Petrovic V, Meyer D, Rissolo D, Kuester F (2015) Fusion of multimodal three-dimensional data for comprehensive digital documentation of cultural heritage sites. In: Digital heritage, 2015, vol 2. IEEE, pp 595–602 Hess M, Petrovic V, Meyer D, Rissolo D, Kuester F (2015) Fusion of multimodal three-dimensional data for comprehensive digital documentation of cultural heritage sites. In: Digital heritage, 2015, vol 2. IEEE, pp 595–602
go back to reference Himmelsbach M, Muller A, Luttel T, Wunsche HJ (2008) LIDAR based 3D object perception. In: Proceedings of 1st international workshop on cognition for technical systems Himmelsbach M, Muller A, Luttel T, Wunsche HJ (2008) LIDAR based 3D object perception. In: Proceedings of 1st international workshop on cognition for technical systems
go back to reference Jolliffe I (2005) Principal component analysis. Wiley, New YorkMATH Jolliffe I (2005) Principal component analysis. Wiley, New YorkMATH
go back to reference Khoshelham K, Elberink SO (2012) Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2):1437–1454CrossRef Khoshelham K, Elberink SO (2012) Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2):1437–1454CrossRef
go back to reference Kläß J, Stückler J, Behnke S (2012) Efficient mobile robot navigation using 3D surfel grid maps. In: Conference on robotics (ROBOTIK), VDE, pp 1–4 Kläß J, Stückler J, Behnke S (2012) Efficient mobile robot navigation using 3D surfel grid maps. In: Conference on robotics (ROBOTIK), VDE, pp 1–4
go back to reference Lalonde JF, Vandapel N, Huber DF, Hebert M (2006) Natural terrain classification using three-dimensional ladar data for ground robot mobility. J Field Robot 23(10):839–862CrossRef Lalonde JF, Vandapel N, Huber DF, Hebert M (2006) Natural terrain classification using three-dimensional ladar data for ground robot mobility. J Field Robot 23(10):839–862CrossRef
go back to reference Li C, Kao CY, Gore JC, Ding Z (2008) Minimization of region-scalable fitting energy for image segmentation. IEEE Trans Image Process 17(10):1940–1949MathSciNetCrossRef Li C, Kao CY, Gore JC, Ding Z (2008) Minimization of region-scalable fitting energy for image segmentation. IEEE Trans Image Process 17(10):1940–1949MathSciNetCrossRef
go back to reference McLachlan G, Krishnan T (2007) The EM algorithm and extensions, vol 382. Wiley, New YorkMATH McLachlan G, Krishnan T (2007) The EM algorithm and extensions, vol 382. Wiley, New YorkMATH
go back to reference Papoulis A, Pillai SU (2002) Probability, random variables, and stochastic processes. Tata McGraw-Hill Education, New York City Papoulis A, Pillai SU (2002) Probability, random variables, and stochastic processes. Tata McGraw-Hill Education, New York City
go back to reference Rockafellar RT (2015) Convex analysis. Princeton University Press, Princeton Rockafellar RT (2015) Convex analysis. Princeton University Press, Princeton
go back to reference Rusu RB, Marton ZC, Blodow N, Holzbach A, Beetz M (2009) Model-based and learned semantic object labeling in 3D point cloud maps of kitchen environments. In: IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 3601–3608 Rusu RB, Marton ZC, Blodow N, Holzbach A, Beetz M (2009) Model-based and learned semantic object labeling in 3D point cloud maps of kitchen environments. In: IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 3601–3608
go back to reference Singh MK, Venkatesh KS, Dutta A (2014) Accurate 3D terrain modeling by range data fusion from two heterogeneous range scanners. In: Annual IEEE India conference (INDICON), pp 1–6 Singh MK, Venkatesh KS, Dutta A (2014) Accurate 3D terrain modeling by range data fusion from two heterogeneous range scanners. In: Annual IEEE India conference (INDICON), pp 1–6
go back to reference Singh MK, Venkatesh KS, Dutta A (2015) Range data fusion for accurate surface generation from heterogeneous range scanners. In: International conference on informatics in control, automation and robotics (ICINCO), vol 02, pp 444–449 Singh MK, Venkatesh KS, Dutta A (2015) Range data fusion for accurate surface generation from heterogeneous range scanners. In: International conference on informatics in control, automation and robotics (ICINCO), vol 02, pp 444–449
go back to reference Stamos I, Allen P (2000) 3-D model construction using range and image data. In: IEEE conference on computer vision and pattern recognition, vol 1. IEEE, pp 531–536 Stamos I, Allen P (2000) 3-D model construction using range and image data. In: IEEE conference on computer vision and pattern recognition, vol 1. IEEE, pp 531–536
go back to reference Trevor AJ, Rogers J, Christensen HI (2012) Planar surface slam with 3D and 2D sensors. In: IEEE international conference on robotics and automation (ICRA). IEEE, pp 3041–3048 Trevor AJ, Rogers J, Christensen HI (2012) Planar surface slam with 3D and 2D sensors. In: IEEE international conference on robotics and automation (ICRA). IEEE, pp 3041–3048
go back to reference Tsai R (1987) A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J Robot Autom 3(4):323–344CrossRef Tsai R (1987) A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J Robot Autom 3(4):323–344CrossRef
go back to reference Vasudevan S (2012) Data fusion with gaussian processes. Robot Auton Syst 60(12):1528–1544CrossRef Vasudevan S (2012) Data fusion with gaussian processes. Robot Auton Syst 60(12):1528–1544CrossRef
go back to reference Wang L, Ding L, Ding X, Fang C (2011) 2D face fitting-assisted 3D face reconstruction for pose-robust face recognition. Soft Comput 15(3):417–428CrossRef Wang L, Ding L, Ding X, Fang C (2011) 2D face fitting-assisted 3D face reconstruction for pose-robust face recognition. Soft Comput 15(3):417–428CrossRef
go back to reference Wu Y, Liu C, Lan S, Yang M (2015) Real-time 3D road scene based on virtual-real fusion method. IEEE Sens J 15(2):750–756CrossRef Wu Y, Liu C, Lan S, Yang M (2015) Real-time 3D road scene based on virtual-real fusion method. IEEE Sens J 15(2):750–756CrossRef
go back to reference Yang Q, Tan KH, Culbertson B, Apostolopoulos J (2010) Fusion of active and passive sensors for fast 3D capture. In: IEEE international workshop on multimedia signal processing (MMSP), pp 69–74 Yang Q, Tan KH, Culbertson B, Apostolopoulos J (2010) Fusion of active and passive sensors for fast 3D capture. In: IEEE international workshop on multimedia signal processing (MMSP), pp 69–74
go back to reference Yue H, Chen W, Wu X, Liu J (2014) Fast 3D modeling in complex environments using a single kinect sensor. Opt Lasers Eng 53:104–111CrossRef Yue H, Chen W, Wu X, Liu J (2014) Fast 3D modeling in complex environments using a single kinect sensor. Opt Lasers Eng 53:104–111CrossRef
go back to reference Zhang Z (2012) Microsoft kinect sensor and its effect. IEEE Multimed 19(2):4–10CrossRef Zhang Z (2012) Microsoft kinect sensor and its effect. IEEE Multimed 19(2):4–10CrossRef
go back to reference Zhu J, Wang L, Gao J, Yang R (2010) Spatial-temporal fusion for high accuracy depth maps using dynamic MRFs. IEEE Trans Pattern Anal Mach Intell 32:899–909CrossRef Zhu J, Wang L, Gao J, Yang R (2010) Spatial-temporal fusion for high accuracy depth maps using dynamic MRFs. IEEE Trans Pattern Anal Mach Intell 32:899–909CrossRef
go back to reference Zhu J, Wang L, Yang R, Davis JE, Pan Z (2011) Reliability fusion of time-of-flight depth and stereo geometry for high quality depth maps. IEEE Trans Pattern Anal Mach Intell 33(7):1400–1414CrossRef Zhu J, Wang L, Yang R, Davis JE, Pan Z (2011) Reliability fusion of time-of-flight depth and stereo geometry for high quality depth maps. IEEE Trans Pattern Anal Mach Intell 33(7):1400–1414CrossRef
Metadata
Title
Multi-sensor data fusion for accurate surface modeling
Authors
Mahesh K. Singh
Ashish Dutta
K. S. Venkatesh
Publication date
27-03-2020
Publisher
Springer Berlin Heidelberg
Published in
Soft Computing / Issue 19/2020
Print ISSN: 1432-7643
Electronic ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-020-04797-9

Other articles of this Issue 19/2020

Soft Computing 19/2020 Go to the issue

Premium Partner