Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
International Journal of Computer Assisted Radiology and Surgery
Published in: International Journal of Computer Assisted Radiology and Surgery 6/2021

17-05-2021 | Original Article

Unsupervised colonoscopic depth estimation by domain translations with a Lambertian-reflection keeping auxiliary task

Authors: Hayato Itoh, Masahiro Oda, Yuichi Mori, Masashi Misawa, Shin-Ei Kudo, Kenichiro Imai, Sayo Ito, Kinichi Hotta, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori, Kensaku Mori

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 6/2021

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Purpose

A three-dimensional (3D) structure extraction technique viewed from a two-dimensional image is essential for the development of a computer-aided diagnosis (CAD) system for colonoscopy. However, a straightforward application of existing depth-estimation methods to colonoscopic images is impossible or inappropriate due to several limitations of colonoscopes. In particular, the absence of ground-truth depth for colonoscopic images hinders the application of supervised machine learning methods. To circumvent these difficulties, we developed an unsupervised and accurate depth-estimation method.

Method

We propose a novel unsupervised depth-estimation method by introducing a Lambertian-reflection model as an auxiliary task to domain translation between real and virtual colonoscopic images. This auxiliary task contributes to accurate depth estimation by maintaining the Lambertian-reflection assumption. In our experiments, we qualitatively evaluate the proposed method by comparing it with state-of-the-art unsupervised methods. Furthermore, we present two quantitative evaluations of the proposed method using a measuring device, as well as a new 3D reconstruction technique and measured polyp sizes.

Results

Our proposed method achieved accurate depth estimation with an average estimation error of less than 1 mm for regions close to the colonoscope in both of two types of quantitative evaluations. Qualitative evaluation showed that the introduced auxiliary task reduces the effects of specular reflections and colon wall textures on depth estimation and our proposed method achieved smooth depth estimation without noise, thus validating the proposed method.

Conclusions

We developed an accurate depth-estimation method with a new type of unsupervised domain translation with the auxiliary task. This method is useful for analysis of colonoscopic images and for the development of a CAD system since it can extract accurate 3D information.
previous article Multiview multimodal network for breast cancer diagnosis in contrast-enhanced spectral mammography images
next article A novel multi-DoF surgical robotic system for brachytherapy on liver tumor: Design and control

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
Nadeem S, Kaufman A (2016) Computer-aided detection of polyps in optical colonoscopy images. In: Proceedings of SPIE 9785, medical imaging 2016: computer-aided diagnosis, pp 549–560
2.
Itoh H, Roth HR, Lu L, Oda M, Misawa M, Mori Y, Kudo S-E, Mori K (2018) Towards automated colonoscopy diagnosis: binary polyp size estimation via unsupervised depth learning. In: Proceedings of medical image computing and computer assisted intervention, pp 611–619
3.
Ma R, Wang R, Pizer S, Rosenman J, McGill SK, Frahm J-H (2019) Real-time 3D reconstruction of colonoscopic surfaces for determining missing regions. In: Proceedings of medical image computing and computer assisted intervention, pp 573–582
4.
Chen, RJ, Bobrow TL, Athey T, Mahmood F, Durr NJ (2019) slam endoscopy enhanced by adversarial depth prediction. In: Proceedings of KDD’19 workshop on applied data science for healthcare
5.
Saxena A, Sung HC, Andrew YN (2006) Learning depth from single monocular images. Adv Neural Inf Process Syst 18:1161–1168
6.
Eigen D, Fergus R (2015) Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. In: Proceedings of IEEE international conference on computer vision, pp 2650–2658
7.
Ma F, Karaman S (2018) Sparse-to-dense: depth prediction from sparse depth samples and a single image. In: Proceedings of IEEE international conference on robotics and automation, pp 4796–4803
8.
Prados E, Faugeras O (2006) Shape from shading. Handbook of mathematical models in computer vision. Springer, Berlin, pp 375–388
9.
Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press, Cambridge
10.
Garg R, Vijay Kumar BG, Carneiro G, Reid I (2016) Unsupervised CNN for single view depth estimation: geometry to the rescue. In: Proceedings of European conference on computer vision, pp 740–756
11.
Zhou T, Brown M, Snavely N, Lowe DG (2017) Unsupervised learning of depth and ego-motion from video. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 6612–6619
12.
Wang C, Buenaposada JM, Zhu R, Lucey S (2018) Learning depth from monocular videos using direct methods. In: Proceedings of IEEE conference on computer vision and pattern, pp 2022–2030
13.
Godard C, Aodha OM, Firman M, Brostow G (2019) Digging into self-supervised monocular depth estimation. In: Proceedings of IEEE international conference on computer vision, pp 3827–3837
14.
Mori K, Suenaga Y, Toriwaki J (2003) Fast software-based volume rendering using multimedia instructions on PC platforms and its application to virtual endoscopy. In: Proceedings of SPIE Med Imaging 5031:111–122
15.
Belhumeur PN, Kriegman DJ, Yuille AL (1999) The bas-relief ambiguity. Int J Comput Vis 35(1):33–44CrossRef
16.
Faisal M, Nicholas JD (2018) Deep learning and conditional random fields-based depth estimation and topographical reconstruction from conventional endoscopy. Med Image Anal 48:230–243CrossRef
17.
Rau A, Edwards PJE, Ahmad OF, Riordan P, Janatka M, Lovat LB, Stoyanov D (2019) Implicit domain adaptation with conditional generative adversarial networks for depth prediction in endoscopy. Int J Comput Assist Radiol Surg 14:1167–1176
18.
Oda M, Tanaka K, Takabatake H, Mori M, Natori H, Mori K (2019) Realistic endoscopic image generation method using virtual-to-real image-domain translation. IET Healthcare Technol Lett 6(6):214–219
19.
Mathew S, Nadeem S, Kumari S, Kaufman A (2020) Augmenting colonoscopy using extended and directional CycleGAN for lossy image translation. In: Proceedings of IEEE international conference on computer vision, pp 4695–4704
20.
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680
21.
Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of IEEE international conference on computer vision, pp 2242–2251
22.
Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334CrossRef
23.
Cignoni P, Callieri M, Corsini M, Dellepiane M, Ganovelli F, Ranzuglia G (2008) MeshLab: an open-source mesh processing tool. In: Proceedings of eurographics Italian chapter conference
24.
Chu C, Zhmoginov A, Sandler M (2017) CycleGAN, a master of steganography. In: Proceedings of NIPS 2017 workshop “machine deception”
Metadata
Title
Unsupervised colonoscopic depth estimation by domain translations with a Lambertian-reflection keeping auxiliary task
Authors
Hayato Itoh
Masahiro Oda
Yuichi Mori
Masashi Misawa
Shin-Ei Kudo
Kenichiro Imai
Sayo Ito
Kinichi Hotta
Hirotsugu Takabatake
Masaki Mori
Hiroshi Natori
Kensaku Mori
Publication date
17-05-2021
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 6/2021
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-021-02398-x

Other articles of this Issue 6/2021

International Journal of Computer Assisted Radiology and Surgery 6/2021 Go to the issue

Original Article

Effect of marker position and size on the registration accuracy of HoloLens in a non-clinical setting with implications for high-precision surgical tasks

Original Article

Automatic ultrasound scanning robotic system with optical waveguide-based force measurement

Original Article

Multiview multimodal network for breast cancer diagnosis in contrast-enhanced spectral mammography images

Original Article

Evaluation of in-house, haptic assisted surgical planning for virtual reduction of complex mandibular fractures

Original Article

A level-wise spine registration framework to account for large pose changes

Original Article

Individual 3D-printed fixation masks for radiotherapy: first clinical experiences

Premium Partner

    Image Credits