nach oben

Erschienen in:

2016 | OriginalPaper | Buchkapitel

Reliable Fusion of ToF and Stereo Depth Driven by Confidence Measures

verfasst von : Giulio Marin, Pietro Zanuttigh, Stefano Mattoccia

Erschienen in: Computer Vision – ECCV 2016

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

In this paper we propose a framework for the fusion of depth data produced by a Time-of-Flight (ToF) camera and stereo vision system. Initially, depth data acquired by the ToF camera are upsampled by an ad-hoc algorithm based on image segmentation and bilateral filtering. In parallel a dense disparity map is obtained using the Semi-Global Matching stereo algorithm. Reliable confidence measures are extracted for both the ToF and stereo depth data. In particular, ToF confidence also accounts for the mixed-pixel effect and the stereo confidence accounts for the relationship between the pointwise matching costs and the cost obtained by the semi-global optimization. Finally, the two depth maps are synergically fused by enforcing the local consistency of depth data accounting for the confidence of the two data sources at each location. Experimental results clearly show that the proposed method produces accurate high resolution depth maps and outperforms the compared fusion algorithms.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Graph Based Skeleton Motion Representation and Similarity Measurement for Action Recognition

Nächstes Kapitel Fast, Exact and Multi-scale Inference for Semantic Image Segmentation with Deep Gaussian CRFs

Nur mit Berechtigung zugänglich

Tippetts, B., Lee, D., Lillywhite, K., Archibald, J.: Review of stereo vision algorithms and their suitability for resource-limited systems. J. Real-Time Image Process. 11, 1–21 (2013)

Dal Mutto, C., Zanuttigh, P., Mattoccia, S., Cortelazzo, G.: Locally consistent ToF and stereo data fusion. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012. LNCS, vol. 7583, pp. 598–607. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33863-2_62 CrossRef

Mattoccia, S.: A locally global approach to stereo correspondence. In: Proceedings of 3D Digital Imaging and Modeling (3DIM), October 2009

Hansard, M., Lee, S., Choi, O., Horaud, R.: Time-of-Flight Cameras: Principles, Methods and Applications. SpringerBriefs in Computer Science. Springer, London (2013)CrossRef

Remondino, F., Stoppa, D. (eds.): TOF Range-Imaging Cameras. Springer, Heidelberg (2013)

Zanuttigh, P., Marin, G., Dal Mutto, C., Dominio, F., Minto, L., Cortelazzo, G.M.: Time-of-Flight and Structured Light Depth Cameras: Technology and Applications, 1st edn. Springer, Cham (2016)CrossRef

Piatti, D., Rinaudo, F.: SR-4000 and CamCube3.0 time of flight (TOF) cameras: tests and comparison. Remote Sens. 4(4), 1069–1089 (2012)CrossRef

Kahlmann, T., Ingensand, H.: Calibration and development for increased accuracy of 3D range imaging cameras. J. Appl. Geodesy 2, 1–11 (2008)CrossRef

Gudmundsson, S.A., Aanaes, H., Larsen, R.: Fusion of stereo vision and time of flight imaging for improved 3D estimation. Int. J. Intell. Syst. Technol. Appl. 5, 425–433 (2008)

10.

Hu, X., Mordohai, P.: A quantitative evaluation of confidence measures for stereo vision. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2121–2133 (2012)CrossRef

11.

Nair, R., Ruhl, K., Lenzen, F., Meister, S., Schäfer, H., Garbe, C.S., Eisemann, M., Magnor, M., Kondermann, D.: A survey on time-of-flight stereo fusion. In: Grzegorzek, M., Theobalt, C., Koch, R., Kolb, A. (eds.) Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications. LNCS, vol. 8200, pp. 105–127. Springer, Heidelberg (2013). doi:10.1007/978-3-642-44964-2_6 CrossRef

12.

Diebel, J., Thrun, S.: An application of markov random fields to range sensing. In: Proceedings of NIPS, pp. 291–298. MIT Press (2005)

13.

Yang, Q., Yang, R., Davis, J., Nister, D.: Spatial-depth super resolution for range images. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2007)

14.

Yang, Q., Ahuja, N., Yang, R., Tan, K., Davis, J., Culbertson, B., Apostolopoulos, J., Wang, G.: Fusion of median and bilateral filtering for range image upsampling. IEEE Trans. Image Process. 22, 4841–4852 (2013)MathSciNetCrossRef

15.

Schwarz, S., Sjostrom, M., Olsson, R.: Time-of-flight sensor fusion with depth measurement reliability weighting. In: 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), pp. 1–4 (2014)

16.

Garro, V., Dal Mutto, C., Zanuttigh, P., M. Cortelazzo, G.: A novel interpolation scheme for range data with side information. In: Proceedings of CVMP (2009)

17.

Dolson, J., Baek, J., Plagemann, C., Thrun, S.: Upsampling range data in dynamic environments. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1141–1148 (2010)

18.

Kuhnert, K.D., Stommel, M.: Fusion of stereo-camera and pmd-camera data for real-time suited precise 3D environment reconstruction. In: Proceedings of International Conference on Intelligent Robots and Systems, pp. 4780–4785 (2006)

19.

Frick, A., Kellner, F., Bartczak, B., Koch, R.: Generation of 3D-TV LDV-content with time-of-flight camera. In: Proceedings of 3DTV Conference (2009)

20.

Kim, Y.M., Theobald, C., Diebel, J., Kosecka, J., Miscusik, B., Thrun, S.: Multi-view image and tof sensor fusion for dense 3D reconstruction. In: Proceedings of 3D Digital Imaging and Modeling (3DIM), October 2009

21.

Dal Mutto, C., Zanuttigh, P., Cortelazzo, G.: A probabilistic approach to ToF and stereo data fusion. In: Proceedings of 3DPVT, Paris, France (2010)

22.

Dal Mutto, C., Zanuttigh, P., Cortelazzo, G.: Probabilistic tof and stereo data fusion based on mixed pixels measurement models. IEEE Trans. Pattern Anal. Mach. Intell. 37(11), 2260–2272 (2015)CrossRef

23.

Zhu, J., Wang, L., Yang, R., Davis, J.: Fusion of time-of-flight depth and stereo for high accuracy depth maps. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)

24.

Zhu, J., Wang, L., Gao, J., Yang, R.: Spatial-temporal fusion for high accuracy depth maps using dynamic MRFs. IEEE Trans. Pattern Anal. Mach. Intell. 32, 899–909 (2010)CrossRef

25.

Zhu, J., Wang, L., Yang, R., Davis, J.E., Pan, Z.: Reliability fusion of time-of-flight depth and stereo geometry for high quality depth maps. IEEE Trans. Pattern Anal. Mach. Intell. 33(7), 1400–1414 (2011)CrossRef

26.

Nair, R., Lenzen, F., Meister, S., Schaefer, H., Garbe, C., Kondermann, D.: High accuracy tof and stereo sensor fusion at interactive rates. In: Proceedings of European Conference on Computer Vision Workshops (ECCVW) (2012)

27.

Evangelidis, G., Hansard, M., Horaud, R.: Fusion of range and stereo data for high-resolution scene-modeling. IEEE Trans. Pattern Anal. Mach. Intell. 37(11), 2178–2192 (2015)CrossRef

28.

Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (1998)CrossRef

29.

Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)CrossRef

30.

Hirschmuller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30, 328–341 (2008)CrossRef

31.

Mattoccia, S.: Fast locally consistent dense stereo on multicore. In: 6th IEEE Embedded Computer Vision Workshop (CVPR Workshop), June 2010

Titel: Reliable Fusion of ToF and Stereo Depth Driven by Confidence Measures
verfasst von: Giulio Marin
Pietro Zanuttigh
Stefano Mattoccia
Verlag: Springer International Publishing
Buch: Computer Vision – ECCV 2016
Print ISBN: 978-3-319-46477-0

Electronic ISBN: 978-3-319-46478-7

Copyright-Jahr: 2016
DOI: https://doi.org/10.1007/978-3-319-46478-7_24

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"