Skip to main content
Top

2021 | OriginalPaper | Chapter

FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos

Authors : Shawn Mathew, Saad Nadeem, Arie Kaufman

Published in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2021

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Haustral folds are colon wall protrusions implicated for high polyp miss rate during optical colonoscopy procedures. If segmented accurately, haustral folds can allow for better estimation of missed surface and can also serve as valuable landmarks for registering pre-treatment virtual (CT) and optical colonoscopies, to guide navigation towards the anomalies found in pre-treatment scans. We present a novel generative adversarial network, FoldIt, for feature-consistent image translation of optical colonoscopy videos to virtual colonoscopy renderings with haustral fold overlays. A new transitive loss is introduced in order to leverage ground truth information between haustral fold annotations and virtual colonoscopy renderings. We demonstrate the effectiveness of our model on real challenging optical colonoscopy videos as well as on textured virtual colonoscopy videos with clinician-verified haustral fold annotations. All code and scripts to reproduce the experiments of this paper will be made available via our Computational Endoscopy Platform at https://​github.​com/​nadeemlab/​CEP.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Footnotes
Literature
1.
go back to reference Amodio, M., Krishnaswamy, S.: Travelgan: image-to-image translation by transformation vector learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8983–8992 (2019) Amodio, M., Krishnaswamy, S.: Travelgan: image-to-image translation by transformation vector learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8983–8992 (2019)
2.
go back to reference Bae, G., Budvytis, I., Yeung, C.K., Cipolla, R.: Deep multi-view stereo for dense 3D reconstruction from monocular endoscopic video. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 774–783 (2020) Bae, G., Budvytis, I., Yeung, C.K., Cipolla, R.: Deep multi-view stereo for dense 3D reconstruction from monocular endoscopic video. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 774–783 (2020)
3.
go back to reference Chen, R.J., Bobrow, T.L., Athey, T., Mahmood, F., Durr, N.J.: Slam endoscopy enhanced by adversarial depth prediction. arXiv preprint arXiv:1907.00283 (2019) Chen, R.J., Bobrow, T.L., Athey, T., Mahmood, F., Durr, N.J.: Slam endoscopy enhanced by adversarial depth prediction. arXiv preprint arXiv:​1907.​00283 (2019)
4.
go back to reference Choi, Y., Choi, M., Kim, M., Ha, J.W., Kim, S., Choo, J.: Stargan: unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8789–8797 (2018) Choi, Y., Choi, M., Kim, M., Ha, J.W., Kim, S., Choo, J.: Stargan: unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8789–8797 (2018)
5.
go back to reference Fang, H., Deng, W., Zhong, Y., Hu, J.: Triple-GAN: progressive face aging with triple translation loss. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 804–805 (2020) Fang, H., Deng, W., Zhong, Y., Hu, J.: Triple-GAN: progressive face aging with triple translation loss. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 804–805 (2020)
7.
go back to reference Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014) Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
8.
go back to reference He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016) He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
9.
go back to reference İncetan, K., et al.: VR-Caps: a virtual environment for capsule endoscopy.Med. Image Anal. 70, 101990 (2021) İncetan, K., et al.: VR-Caps: a virtual environment for capsule endoscopy.Med. Image Anal. 70, 101990 (2021)
10.
go back to reference Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017) Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
11.
go back to reference Liu, X., et al.: Reconstructing sinus anatomy from endoscopic video-towards a radiation-free approach for quantitative longitudinal assessment. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 3–13 (2020) Liu, X., et al.: Reconstructing sinus anatomy from endoscopic video-towards a radiation-free approach for quantitative longitudinal assessment. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 3–13 (2020)
12.
go back to reference Ma, R., Wang, R., Pizer, S., Rosenman, J., McGill, S.K., Frahm, J.M.: Real-time 3D reconstruction of colonoscopic surfaces for determining missing regions. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 573–582 (2019) Ma, R., Wang, R., Pizer, S., Rosenman, J., McGill, S.K., Frahm, J.M.: Real-time 3D reconstruction of colonoscopic surfaces for determining missing regions. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 573–582 (2019)
13.
go back to reference Mahmood, F., Chen, R., Durr, N.J.: Unsupervised reverse domain adaptation for synthetic medical images via adversarial training. IEEE Trans. Med. Imaging 37(12), 2572–2581 (2018)CrossRef Mahmood, F., Chen, R., Durr, N.J.: Unsupervised reverse domain adaptation for synthetic medical images via adversarial training. IEEE Trans. Med. Imaging 37(12), 2572–2581 (2018)CrossRef
14.
go back to reference Mathew, S., Nadeem, S., Kaufman, A.: Visualizing missing surfaces in colonoscopy videos using shared latent space representations. In: IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 329–333 (2021) Mathew, S., Nadeem, S., Kaufman, A.: Visualizing missing surfaces in colonoscopy videos using shared latent space representations. In: IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 329–333 (2021)
15.
go back to reference Mathew, S., Nadeem, S., Kumari, S., Kaufman, A.: Augmenting colonoscopy using extended and directional cyclegan for lossy image translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4696–4705 (2020) Mathew, S., Nadeem, S., Kumari, S., Kaufman, A.: Augmenting colonoscopy using extended and directional cyclegan for lossy image translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4696–4705 (2020)
16.
go back to reference Nadeem, S., Kaufman, A.: Computer-aided detection of polyps in optical colonoscopy images. SPIE Med. Imaging 9785, 978525 (2016) Nadeem, S., Kaufman, A.: Computer-aided detection of polyps in optical colonoscopy images. SPIE Med. Imaging 9785, 978525 (2016)
17.
go back to reference Nadeem, S., Marino, J., Gu, X., Kaufman, A.: Corresponding supine and prone colon visualization using eigenfunction analysis and fold modeling. IEEE Trans. Vis. Comput. Gr. 23(1), 751–760 (2016)CrossRef Nadeem, S., Marino, J., Gu, X., Kaufman, A.: Corresponding supine and prone colon visualization using eigenfunction analysis and fold modeling. IEEE Trans. Vis. Comput. Gr. 23(1), 751–760 (2016)CrossRef
19.
go back to reference Xu, J., et al.: Ofgan: realistic rendition of synthetic colonoscopy videos. in: international Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 732–741 (2020) Xu, J., et al.: Ofgan: realistic rendition of synthetic colonoscopy videos. in: international Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 732–741 (2020)
20.
go back to reference Zhu, J.Y., Park, T., Isola, P., Efros, A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International Conference on Computer Vision, pp. 2223–2232 (2017) Zhu, J.Y., Park, T., Isola, P., Efros, A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)
Metadata
Title
FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos
Authors
Shawn Mathew
Saad Nadeem
Arie Kaufman
Copyright Year
2021
DOI
https://doi.org/10.1007/978-3-030-87199-4_21

Premium Partner