Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

07-08-2019 | Original Article

Improving realism in patient-specific abdominal ultrasound simulation using CycleGANs

Authors: Santiago Vitale, José Ignacio Orlando, Emmanuel Iarussi, Ignacio Larrabide

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 2/2020

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Purpose

In this paper, we propose to apply generative adversarial neural networks trained with a cycle consistency loss, or CycleGANs, to improve realism in ultrasound (US) simulation from computed tomography (CT) scans.

Methods

A ray-casting US simulation approach is used to generate intermediate synthetic images from abdominal CT scans. Then, an unpaired set of these synthetic and real US images is used to train CycleGANs with two alternative architectures for the generator, a U-Net and a ResNet. These networks are finally used to translate ray-casting based simulations into more realistic synthetic US images.

Results

Our approach was evaluated both qualitatively and quantitatively. A user study performed by 21 experts in US imaging shows that both networks significantly improve realism with respect to the original ray-casting algorithm (\(p \ll 0.0001\)), with the ResNet model performing better than the U-Net (\(p \ll 0.0001\)).

Conclusion

Applying CycleGANs allows to obtain better synthetic US images of the abdomen. These results can contribute to reduce the gap between artificially generated and real US scans, which might positively impact in applications such as semi-supervised training of machine learning algorithms and low-cost training of medical doctors and radiologists in US image interpretation.

next article Temporally downsampled cerebral CT perfusion image restoration using deep residual learning

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

http://www.vision.ee.ethz.ch/datasets_extra/usliverseq.zip.

http://deepultrasound.ai/.

https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix.

American College of Emergency Physicians (2001) Use of ultrasound imaging by emergency physicians. Ann Emerg Med 38(4):469

Behboodi B, Rivaz H (2019) Ultrasound segmentation using u-net: learning from simulated data and testing on real data. arXiv:1904.11031

Cohen JP, Luck M, Honari S (2018) Distribution matching losses can hallucinate features in medical image translation. arXiv:1805.08841

D’Amato JP, Lo Vercio L, Rubí P, Fernández Vera E, Barbuzza R, del Fresno M, Larrabide I (2015) Efficient scatter model for simulation of ultrasound images from computed tomography data. In: 11th International symposium on medical information processing and analysis, vol 9681. International Society for Optics and Photonics, p 968105

De Leeuw JR (2015) jspsych: a JavaScript library for creating behavioral experiments in a web browser. Behav Res Methods 47(1):1–12CrossRef

De Luca V, Tschannen M, Székely G, Tanner C (2013) A learning-based approach for fast and robust vessel tracking in long ultrasound sequences. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 518–525

Dinh VA, Fu JY, Lu S, Chiem A, Fox JC, Blaivas M (2016) Integration of ultrasound in medical education at United States medical schools: a national survey of directors’ experiences. J Ultrasound Med 35(2):413–419CrossRef

Engelhardt S, De Simone R, Full PM, Karck M, Wolf I (2018) Improving surgical training phantoms by hyperrealism: Deep unpaired image-to-image translation from real surgeries. arXiv:1806.03627

Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ (eds) Proceedings of the 27th international conference on neural information processing systems, vol 2. MIT Press, Cambridge, pp 2672–2680

10.

He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

11.

Heer I, Middendorf K, Müller-Egloff S, Dugas M, Strauss A (2004) Ultrasound training: the virtual patient. Ultrasound Obstet Gynecol 24(4):440–444CrossRef

12.

Hiasa Y, Otake Y, Takao M, Matsuoka T, Takashima K, Carass A, Prince JL, Sugano N, Sato Y (2018) Cross-modality image synthesis from unpaired data using CycleGAN. In: International workshop on simulation and synthesis in medical imaging. Springer, pp 31–41

13.

Ircad data set. https://www.ircad.fr/research/3dircadb/. Accessed 2018-12-26

14.

Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: CVPR

15.

Kutter O, Shams R, Navab N (2009) Visualization and GPU-accelerated simulation of medical ultrasound from CT images. Comput Methods Progr Biomed 94(3):250–266CrossRef

16.

Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Van Der Laak JA, Van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88CrossRef

17.

Noble JA (2016) Reflections on ultrasound image analysis. Med Image Anal 33:33–37. https://doi.org/10.1016/j.media.2016.06.015 CrossRef

18.

Odena A, Dumoulin V, Olah C (2016) Deconvolution and checkerboard artifacts. Distill 1(10):e3CrossRef

19.

Østergaard ML, Ewertsen C, Konge L, Albrecht-Beste E, Nielsen MB (2016) Simulation-based abdominal ultrasound training—a systematic review. Ultraschall in der Medizin Eur J Ultrasound 37(03):253–261CrossRef

20.

Petrusca L, Cattin P, De Luca V, Preiswerk F, Celicanin Z, Auboiroux V, Viallon M, Arnold P, Santini F, Terraz S, Scheffler K, Becker CD, Salomir R (2013) Hybrid ultrasound/magnetic resonance simultaneous acquisition and image fusion for motion monitoring in the upper abdomen. Investig Radiol 48(5):333–340CrossRef

21.

Pham AH, Stage B, Hemmsen MC, Lundgren B, Pedersen MM, Jensen JA (2011) Simulation of shadowing effects in ultrasound imaging from computed tomography images. In: 2011 IEEE international ultrasonics symposium, pp 1411–1414. https://doi.org/10.1109/ULTSYM.2011.0349

22.

Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

23.

Rubi P, Vera EF, Larrabide I, Calvo M, D’Amato J, Larrabide I (2017) Comparison of real-time ultrasound simulation models using abdominal CT images. In: 12th international symposium on medical information processing and analysis, vol 10160. International Society for Optics and Photonics, p 1016009

24.

Shams R, Hartley R, Navab N (2008) Real-time simulation of medical ultrasound from ct images. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 734–741

25.

Taigman Y, Polyak A, Wolf L (2017) Unsupervised cross-domain image generation. In: International conference on learning representations, ICLR 2017

26.

Terkamp C, Kirchner G, Wedemeyer J, Dettmer A, Kielstein J, Reindell H, Bleck J, Manns M, Gebel M (2003) Simulation of abdomen sonography. evaluation of a new ultrasound simulator. Ultraschall in der Medizin 24(04):239–244CrossRef

27.

Walcher F, Weinlich M, Conrad G, Schweigkofler U, Breitkreutz R, Kirschning T, Marzi I (2006) Prehospital ultrasound imaging improves management of abdominal trauma. Br J Surg Inc Eur J Surg Swiss Surg 93(2):238–242

28.

Wang C, Macnaught G, Papanastasiou G, MacGillivray T, Newby D (2018) Unsupervised learning for cross-domain medical image synthesis using deformation invariant cycle consistency networks. In: International workshop on simulation and synthesis in medical imaging. Springer, pp 52–60

29.

Wein W, Kamen A, Clevert DA, Kutter O, Navab N (2007) Simulation and fully automatic multimodal registration of medical ultrasound. pp 136–43. https://doi.org/10.1007/978-3-540-75757-3_17 CrossRef

30.

Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: ICCV

Title: Improving realism in patient-specific abdominal ultrasound simulation using CycleGANs
Authors: Santiago Vitale
José Ignacio Orlando
Emmanuel Iarussi
Ignacio Larrabide
Publication date: 07-08-2019
Publisher: Springer International Publishing
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 2/2020
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-019-02046-5

Springer Professional

Abstract

Purpose

Methods

Results

Conclusion

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 2/2020

Multimodal 3D medical image registration guided by shape encoder–decoder networks

Automatic atlas-based liver segmental anatomy identification for hepatic surgical planning

Body-mounted robotic assistant for MRI-guided low back pain injection

Accuracy assessment of target tracking using two 5-degrees-of-freedom wireless transponders

Soft tissue deformation tracking by means of an optimized fiducial marker layout with application to cancer tumors

Automatic cancer tissue detection using multispectral photoacoustic imaging

Premium Partner