Top

Published in:

2020 | OriginalPaper | Chapter

Elimination of Incorrect Depth Points for Depth Completion

Authors : Chuhua Xian, Kun Qian, Guoliang Luo, Guiqing Li, Jianming Lv

Published in: Advances in Computer Graphics

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Commodity-level scan cameras generally capture RGB-D image with depth missing or incorrect depth points if the surface of the object is transparent, bright, or black. These incorrect depth points are generated randomly and limit the downstream applications of raw RGB-D images. In this paper, we propose a coarse-to-fine method to detect and eliminate the incorrect depth points via RGB semantics. In our flowchart, deep learning-based networks are applied to predict the potential regions with incorrect depth points and the normals of the point cloud. Then we develop a three-step elimination method to remove the incorrect depth points in the regions. Experimental results show that our method leads to great improvements for downstream applications of RGB-D images, especially in depth completion application.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Physics-Guided Sound Synthesis for Rotating Blades

next chapter Pose Transfer of 2D Human Cartoon Characters

C., D.H., Kannala, J., Heikkila, J.: Joint depth and color camera calibrationwith distortion correction. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 2058–2064 (2012).https://doi.org/10.1109/tpami.2012.125

Chan, D., Buisman, H., Theobalt, C., Thrun, S.: A noise-aware filter for real-time depth upsampling (2008)

Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 833–851. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_49CrossRef

Diebel, J., Thrun, S.: An application of Markov random fields to range sensing. In: Weiss, Y., Schölkopf, B., Platt, J.C. (eds.) Advances in Neural Information Processing Systems, vol. 18, pp. 291–298. MIT Press (2006)

Huang, Z., Fan, J., Cheng, S., Yi, S., Wang, X., Li, H.: HMS-net: hierarchical multi-scale sparsity-invariant network for sparse depth completion. IEEE Trans. Image Process. 29, 3429–3441 (2020). https://doi.org/10.1109/tpami.2012.125

Jaritz, M., Charette, R.D., Wirbel, E., Perrotton, X., Nashashibi, F.: Sparse and dense data with CNNs: depth completion and semantic segmentation. In: 2018 International Conference on 3D Vision (3DV). IEEE, September 2018. https://doi.org/10.1109/3dv.2018.00017

Kopf, J., Cohen, M.F., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. In: ACM SIGGRAPH 2007 Papers on - SIGGRAPH 2007. ACM Press (2007). https://doi.org/10.1145/1275808.1276497

Lachat, E., Macher, H., Landes, T., Grussenmeyer, P.: Assessment and calibration of a RGB-d camera (Kinect v2 sensor) towards a potential use for close-range 3D modeling. Remote Sens. 7(10), 13070–13097 (2015). https://doi.org/10.3390/rs71013070

Liu, J., Gong, X.: Guided depth enhancement via anisotropic diffusion. In: Huet, B., Ngo, C.-W., Tang, J., Zhou, Z.-H., Hauptmann, A.G., Yan, S. (eds.) PCM 2013. LNCS, vol. 8294, pp. 408–417. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-03731-8_38CrossRef

10.

Pagliari, D., Pinto, L.: Calibration of Kinect for XBOX one and comparison between the two generations of microsoft sensors. Sensors 15(11), 27569–27589 (2015). https://doi.org/10.3390/Pagliari_2015

11.

Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. In: ACM SIGGRAPH 2004 Papers on - SIGGRAPH 2004. ACM Press (2004). https://doi.org/10.1145/1186562.1015777

12.

Schall, O., Belyaev, A., Seidel, H.P.: Feature-preserving non-local denoising of static and time-varying range data. In: Proceedings of the 2007 ACM Symposium on Solid and Physical Modeling - SPM 2007. ACM Press (2007). https://doi.org/10.1145/1236246.1236277

13.

Shan, D.R., Ke, Y.L.: Surface reconstruction from unorganized points based on 2D delaunay neighbors. In: Sui, W. (ed.) Second International Conference on Image and Graphics. SPIE, July 2002. https://doi.org/10.1117/12.477093

14.

Uhrig, J., Schneider, N., Schneider, L., Franke, U., Brox, T., Geiger, A.: Sparsity invariant CNNs. In: 2017 International Conference on 3D Vision (3DV). IEEE, October 2017. https://doi.org/10.1109/3dv.2017.00012

15.

Wang, X., Zhang, P., Zhang, Y., Ma, L., Kwong, S., Jiang, J.: Deep intensity guidance based compression artifacts reduction for depth map. J. Vis. Commun. Image Represent. 57, 234–242 (2018). https://doi.org/10.1016/j.jvcir.2018.11.008

16.

Yan, S., et al.: DDRNet: depth map denoising and refinement for consumer depth cameras using cascaded CNNs. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 155–171. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01249-6_10CrossRef

17.

Zhang, Y., Funkhouser, T.: Deep depth completion of a single RGB-D image. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, June 2018. https://doi.org/10.1109/cvpr.2018.00026

18.

Zhang, Y., et al.: Physically-based rendering for indoor scene understanding using convolutional neural networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, July 2017. https://doi.org/10.1109/cvpr.2017.537

19.

Zhou, B., et al.: Semantic understanding of scenes through the ADE20k dataset. Int. J. Comput. Vis. 127(3), 302–321, December 2018. https://doi.org/10.1007/s11263-018-1140-0

Title: Elimination of Incorrect Depth Points for Depth Completion
Authors: Chuhua Xian
Kun Qian
Guoliang Luo
Guiqing Li
Jianming Lv
Publisher: Springer International Publishing
Book: Advances in Computer Graphics
Print ISBN: 978-3-030-61863-6

Electronic ISBN: 978-3-030-61864-3

Copyright Year: 2020
DOI: https://doi.org/10.1007/978-3-030-61864-3_21

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner