Top

Neural Processing Letters

Published in:

31-05-2022

Triaxial Squeeze Attention Module and Mutual-Exclusion Loss Based Unsupervised Monocular Depth Estimation

Authors: Jiansheng Wei, Shuguo Pan, Wang Gao, Tao Zhao

Published in: Neural Processing Letters | Issue 5/2022

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Monocular depth estimation plays a crucial role in scene perception and 3D reconstruction. Supervised learning based depth estimation needs vast amounts of ground-truth depth data for training, which seriously restricts its generalization. In recent years, the unsupervised learning methods without LiDAR points cloud have attracted more and more attention. In this paper, an unsupervised monocular depth estimation method using stereo pairs for training is designed. We present a triaxial squeeze attention module and introduce it into our unsupervised framework to augment the representations of the depth map in detail. We also propose a novel training loss that enforces mutual-exclusion in image reconstruction to improve the performance and robustness in unsupervised learning. Experimental results on KITTI show that our method not only outperforms existing unsupervised methods but also achieves better results comparable with several supervised approaches trained with ground-truth data. The improvements in our method can better preserve the details of the depth map and allow the shape of objects to be maintained more smoothly.

previous article DMADRL: A Distributed Multi-agent Deep Reinforcement Learning Algorithm for Cognitive Offloading in Dynamic MEC Networks

next article P &GGD: A Joint-Way Model Optimization Strategy Based on Filter Pruning and Filter Grafting For Tea Leaves Classification

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Geng M, Shang S, Ding B, Wang H, Zhang P (2020) Unsupervised learning-based depth estimation-aided visual slam approach. Circ Syst Signal Process 39(2):543–570. https://doi.org/10.1007/s00034-019-01173-3CrossRef

Lee SJ, Choi H, Hwang SS (2020) Real-time depth estimation using recurrent CNN with sparse depth cues for SLAM system. Int J Control Autom 18(1):206–216. https://doi.org/10.1007/s12555-019-0350-8CrossRef

Wang Y, Chao W, Garg D, Hariharan B, Campbell M, Weinberger KQ (2019) Pseudo-LiDAR from visual depth estimation: bridging the gap in 3D object detection for autonomous driving. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR 2019). pp 8437–8445. https://doi.org/10.1109/CVPR.2019.00864

Chen C, Seff A, Kornhauser A, Xiao J (2015) Deepdriving: learning affordance for direct perception in autonomous driving. In: 2015 IEEE international conference on computer vision (ICCV). pp 2722–2730. https://doi.org/10.1109/ICCV.2015.312

Jin Y, Lee M (2019) Enhancing binocular depth estimation based on proactive perception and action cyclic learning for an autonomous developmental robot. IEEE Trans Syst Man Cybern Ssyat 49(1):169–180. https://doi.org/10.1109/TSMC.2017.2779474CrossRef

Ding Y, Lin L, Wang L, Zhang M, Li D (2020) Digging into the multi-scale structure for a more refined depth map and 3D reconstruction. Neural Comput Appl 32(15):11217–11228. https://doi.org/10.1007/s00521-020-04702-3CrossRef

Wang B, Feng Y, Liu H (2018) Multi-scale features fusion from sparse LiDAR data and single image for depth completion. Electron Lett 54(24):1375–1376. https://doi.org/10.1049/el.2018.6149CrossRef

Willis AR, Papadakis J, Brink KM (2017) Linear depth reconstruction for RGBD sensors, Southeastcon 2017

Guo Y, Chen T (2018) Semantic segmentation of RGBD images based on deep depth regression. Pattern Recogn Lett 109:55–64. https://doi.org/10.1016/j.patrec.2017.08.026CrossRef

10.

Wang Y, Gao Y, Achim A, Dahnoun N (2014) Robust obstacle detection based on a novel disparity calculation method and G-disparity. Comput Vis Image Underst 123:23–40. https://doi.org/10.1016/j.cviu.2014.02.014CrossRef

11.

Zhou C, Liu Y, Sun Q, Lasang P (2021) Vehicle detection and disparity estimation using blended stereo images. IEEE Trans Intell Veh 6(4):690–698. https://doi.org/10.1109/TIV.2020.3049008CrossRef

12.

Wu G, Li Y, Huang Y, Liu Y (2019) Joint view synthesis and disparity refinement for stereo matching. Front Comput Sci 13(6):1337–1352. https://doi.org/10.1007/s11704-018-8099-4CrossRef

13.

Hu P, Yang S, Zhang G, Deng H (2021) High-speed and accurate 3D shape measurement using DIC-assisted phase matching and triple-scanning. Opt Lasers Eng. https://doi.org/10.1016/j.optlaseng.2021.106725CrossRef

14.

Bao Z, Li B, Zhang W (2019) Robustness of ToF and stereo fusion for high-accuracy depth map. IET Comput Vis 13(7):676–681. https://doi.org/10.1049/iet-cvi.2018.5476CrossRef

15.

Wang C et al (2021) Self-supervised multiscale adversarial regression network for stereo disparity estimation. IEEE T Cybern 51(10):4770–4783. https://doi.org/10.1109/TCYB.2020.2999492CrossRef

16.

Dong Q, Feng J (2018) Adaptive disparity computation using local and non-local cost aggregations. Multimed Tools Appl 77(24):31647–31663. https://doi.org/10.1007/s11042-018-6236-6CrossRef

17.

Wang C, Miguel Buenaposada J, Zhu R, Lucey S (2018) Learning depth from monocular videos using direct methods. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR). pp 2022–2030. https://doi.org/10.1109/CVPR.2018.00216

18.

Mayer N et al (2016) A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR). pp 4040–4048. https://doi.org/10.1109/CVPR.2016.438

19.

Laina I, Rupprecht C, Belagiannis V, Tombari F, Navab N (2016) Deeper depth prediction with fully convolutional residual networks. In: Proceedings of 2016 fourth international conference on 3D vision (3DV). pp 239–248. https://doi.org/10.1109/3DV.2016.32

20.

Saxena A, Sun M, Ng AY (2009) Make3D: learning 3D scene structure from a single still image. IEEE Trans Pattern Anal 31(5):824–840. https://doi.org/10.1109/TPAMI.2008.132CrossRef

21.

Wu D, Luo X, Shang M, He Y, Wang G, Zhou M (2021) A deep latent factor model for high-dimensional and sparse matrices in recommender systems. IEEE Trans Syst Man Cybern Syst 51(7):4285–4296. https://doi.org/10.1109/TSMC.2019.2931393CrossRef

22.

Luo X, Zhou M, Li S, Wu D, Liu Z, Shang M (2021) Algorithms of unconstrained non-negative latent factor analysis for recommender systems. IEEE Trans Big Data 7(1):227–240. https://doi.org/10.1109/TBDATA.2019.2916868CrossRef

23.

Tan N, Zhong Z, Yu P, Li Z, Ni F (2022) A discrete model-free scheme for fault tolerant tracking control of redundant manipulators. IEEE Trans Ind Inform. https://doi.org/10.1109/TII.2022.3149919CrossRef

24.

Ye X, Ji X, Sun B, Chen S, Wang Z, Li H (2020) DRM-SLAM: towards dense reconstruction of monocular SLAM with scene depth fusion. Neurocomputing 396:76–91. https://doi.org/10.1016/j.neucom.2020.02.044CrossRef

25.

Luo H, Gao Y, Wu Y, Liao C, Yang X, Cheng K (2019) Real-time dense monocular SLAM with online adapted depth prediction network. IEEE Trans Multimed 21(2):470–483. https://doi.org/10.1109/TMM.2018.2859034CrossRef

26.

Eigen D, Puhrsch C, Fergus R (2014) Depth map prediction from a single image using a multi-scale deep network. Adv Neural Inf Process Syst 27

27.

Liu F, Shen C, Lin G (2015) Deep convolutional neural fields for depth estimation from a single image. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR). pp 5162–5170

28.

Fu H, Gong M, Wang C, Batmanghelich K, Tao D (2018) Deep ordinal regression network for monocular depth estimation. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR). pp 2002–2011. https://doi.org/10.1109/CVPR.2018.00214

29.

Ma F, Karaman S (2018) Sparse-to-dense: depth prediction from sparse depth samples and a single image. In: 2018 IEEE international conference on robotics and automation (ICRA), pp 4796–4803

30.

Uhrig J, Schneider N, Schneider L, Franke U, Brox T, Geiger A (2017) Sparsity invariant CNNs. In: Proceedings 2017 international conference on 3D vision (3DV), pp 11–20, https://doi.org/10.1109/3DV.2017.00012

31.

Garg R, VijayKumar BG, Carneiro G, Reid I (2016) Unsupervised CNN for single view depth estimation: geometry to the rescue. Comput Vis - ECCV 2016 PT VIII 9912:740–756. https://doi.org/10.1007/978-3-319-46484-8_45CrossRef

32.

Yu JJ, Harley AW, Derpanis KG (2016) Back to basics: unsupervised learning of optical flow via brightness constancy and motion smoothness. Comput Vis- ECCV 2016 Workshops Pt III 9915:3–10. https://doi.org/10.1007/978-3-319-49409-8_1MathSciNetCrossRef

33.

Godard C, Mac Aodha O, Brostow GJ (2017) Unsupervised monocular depth estimation with left-right consistency. In: 30th IEEE conference on computer vision and pattern recognition (CVPR 2017), pp 6602–6611. https://doi.org/10.1109/CVPR.2017.699

34.

Feng X, Fang B (2021) Algorithm for epipolar geometry and correcting monocular stereo vision based on a plane mirror. Optik. https://doi.org/10.1016/j.ijleo.2020.165890CrossRef

35.

Chen J, Yang X, Jia Q, Liao C (2021) DENAO: monocular depth estimation network with auxiliary optical flow. IEEE Trans Pattern Anal 43(8):2598–2610. https://doi.org/10.1109/TPAMI.2020.2977021CrossRef

36.

Gomaa MAK, de Silva O, Mann GKI, Gosine RG (2021) Observability-constrained VINS for MAVs using interacting multiple model algorithm. IEEE Trans Aerosp Electron Syst 57(3):1423–1442. https://doi.org/10.1109/TAES.2020.3043534CrossRef

37.

Dai R et al (2019) Unsupervised learning of depth estimation based on attention model and global pose optimization. Signal Process-Image Commnun 78:284–292. https://doi.org/10.1016/j.image.2019.07.007CrossRef

38.

Song X et al (2021) MLDA-Net: multi-level dual attention-based network for self-supervised monocular depth estimation. IEEE Trans Image Process 30:4691–4705. https://doi.org/10.1109/TIP.2021.3074306CrossRef

39.

Xu X, Chen Z, Yin F (2021) Multi-scale spatial attention-guided monocular depth estimation with semantic enhancement. IEEE Trans Image Process 30:8811–8822. https://doi.org/10.1109/TIP.2021.3120670CrossRef

40.

Yang D, Zhong X, Gu D, Peng X, Hu H (2020) Unsupervised framework for depth estimation and camera motion prediction from video. Neurocomputing 385:169–185. https://doi.org/10.1016/j.neucom.2019.12.049CrossRef

41.

Zhou T, Brown M, Snavely N, Lowe DG (2017) Unsupervised learning of depth and ego-motion from video. In: 30th IEEE conference on computer vision and pattern recognition (CVPR 2017), pp 6612+. https://doi.org/10.1109/CVPR.2017.700

42.

Pilzer A, Xu D, Puscas MM, Ricci E, Sebe N (2018) Unsupervised adversarial depth estimation using cycled generative networks. In: 2018 international conference on 3D vision (3DV), pp 587–595. https://doi.org/10.1109/3DV.2018.00073

43.

Ji Z, Song X, Song H, Yang H, Guo X (2021) RDRF-Net: a pyramid architecture network with residual-based dynamic receptive fields for unsupervised depth estimation. Neurocomputing 457:1–12. https://doi.org/10.1016/j.neucom.2021.05.089CrossRef

44.

Zhao S, Fu H, Gong M, Tao D (2019) Geometry-aware symmetric domain adaptation for monocular depth estimation. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR 2019), pp 9780–9790. https://doi.org/10.1109/CVPR.2019.01002

45.

Chen Y, Zhao H, Hu Z, Peng J (2021) Attention-based context aggregation network for monocular depth estimation. Int J Mach Learn Cybern 12(6):1583–1596. https://doi.org/10.1007/s13042-020-01251-yCrossRef

46.

Song M, Lim S, Kim W (2021) Monocular depth estimation using laplacian pyramid-based depth residuals. IEEE Trans Circ Syst Video 31(11):4381–4393. https://doi.org/10.1109/TCSVT.2021.3049869CrossRef

Title: Triaxial Squeeze Attention Module and Mutual-Exclusion Loss Based Unsupervised Monocular Depth Estimation
Authors: Jiansheng Wei
Shuguo Pan
Wang Gao
Tao Zhao
Publication date: 31-05-2022
Publisher: Springer US
Published in: Neural Processing Letters / Issue 5/2022
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI: https://doi.org/10.1007/s11063-022-10812-x

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 5/2022

Res-CapsNet: Residual Capsule Network for Data Classification

Contextualized Multidimensional Personality Recognition using Combination of Deep Neural Network and Ensemble Learning

Sequence in Hybridization of Statistical and Intelligent Models in Time Series Forecasting

A Novel Quasi-oppositional Chaotic Harris Hawk’s Optimization Algorithm for Optimal Siting and Sizing of Distributed Generation in Radial Distribution System

MTPose: Human Pose Estimation with High-Resolution Multi-scale Transformers

BARLAT: A Nearly Unsupervised Approach for Aspect Category Detection