Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

06-01-2023 | Original Article

Lesion-preserving unpaired image-to-image translation between MRI and CT from ischemic stroke patients

Authors: Alejandro Gutierrez, Anup Tuladhar, Matthias Wilms, Deepthi Rajashekar, Michael D. Hill, Andrew Demchuk, Mayank Goyal, Jens Fiehler, Nils D. Forkert

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 5/2023

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Purpose

Multiple medical imaging modalities are used for clinical follow-up ischemic stroke analysis. Mixed-modality datasets are challenging, both for clinical rating purposes and for training machine learning models. While image-to-image translation methods have been applied to harmonize stroke patient images to a single modality, they have only been used for paired data so far. In the more common unpaired scenario, the standard cycle-consistent generative adversarial network (CycleGAN) method is not able to translate the stroke lesions properly. Thus, the aim of this work was to develop and evaluate a novel image-to-image translation regularization approach for unpaired 3D follow-up stroke patient datasets.

Methods

A modified CycleGAN was used to translate images between 238 non-contrast computed tomography (NCCT) and 244 fluid-attenuated inversion recovery (FLAIR) MRI datasets, two of the most relevant follow-up modalities in clinical practice. We introduced an additional attention-guided mechanism to encourage an improved translation of the lesion and a gradient-consistency loss to preserve structural brain morphology.

Results

The proposed modifications were able to preserve the overall quality provided by the CycleGAN translation. This was confirmed by the FID score and gradient correlation results. Furthermore, the lesion preservation was significantly improved compared to a standard CycleGAN. This was evaluated for location and volume with segmentation models, which were trained on real datasets and applied to the translated test images. Here, the Dice score coefficient resulted in 0.81 and 0.62 for datasets translated to FLAIR and NCCT, respectively, compared to 0.57 and 0.50 for the corresponding datasets translated using a standard CycleGAN. Finally, an analysis of the distribution of mean lesion intensities showed substantial improvements.

Conclusion

The results of this work show that the proposed image-to-image translation method is effective at preserving stroke lesions in unpaired modality translation, supporting its potential as a tool for stroke image analysis in real-life scenarios.

previous article Multiclass datasets expand neural network utility: an example on ankle radiographs

next article A hybrid hierarchical strategy for registration of 7T TOF-MRI to 7T PC-MRI intracranial vessel data

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Fiehler J, Thomalla G, Bernhardt M, Kniep H, Berlis A, Dorn F, Eckert B, Kemmling A, Langner S, Remonda L, Reith W, Rohde S, Möhlenbruch M, Bendszus M, Forkert ND, Gellissen S (2019) ERASER: a thrombectomy study with predictive analytics end point. Stroke 50(5):1275–1278. https://doi.org/10.1161/STROKEAHA.119.024858CrossRefPubMed

Makkat S, Vandevenne JE, Verswijvel G, Ijsewijn T, Grieten M, Palmers Y, De Schepper AM, Parizel PM (2002) Signs of acute stroke seen on fluid-attenuated inversion recovery MR imaging. Am J Roentgenol 179(1):237–243CrossRef

Peultier A-C, Redekop WK, Dippel DW, Bereczki D, Si-Mohamed S, Douek PC, Severens JL (2019) What stroke image do we want? European survey on acute stroke imaging and revascularisation treatment. Health Policy Technol 8(3):261–267. https://doi.org/10.1016/j.hlpt.2019.08.005CrossRef

Clèrigues A, Valverde S, Bernal J, Freixenet J, Oliver A, Lladó X (2020) Acute and sub-acute stroke lesion segmentation from multimodal MRI. Comput Methods Progr Biomed 194:105521. https://doi.org/10.1016/j.cmpb.2020.105521CrossRef

Gillmann C, Peter L, Schmidt C, Saur D, Scheuermann G (2021) Visualizing multimodal deep learning for lesion prediction. IEEE Computer springer nature 2021 LATEX template 14 Lesion-preserving image-to-image translation graphics and applications 41(5), 90–98. https://doi.org/10.1109/MCG.2021.3099881

Lo Vercio L, Amador K, Bannister J, Crites S, Gutierrez A, MacDonald ME, Moore J, Mouches P, Rajasheka D, Schimert S, Subbanna N, Tuladhar A, Wang N, Wilms M, Winder A, Forkert ND (2020) Supervised machine learning tools: a tutorial for clinicians. J Neural Eng 17(6):062001. https://doi.org/10.1088/1741-2552/abbff2CrossRef

MacEachern SJ, Forkert ND (2021) Machine learning for precision medicine. Genome 64(4):416–425. https://doi.org/10.1139/gen-2020-0131CrossRefPubMed

Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision. pp. 2223–2232

Hiasa Y, Otak Y, Takao M, Matsuoka T, Takashima K, Carass A, Prince J, Sugano N, Sato Y (2018) Cross-modality image synthesis from unpaired data using CycleGAN. In: International workshop on simulation and synthesis in medical imaging. Springer, pp. 31–41. https://doi.org/10.1007/978-3-030-00536-84

10.

Cohen JP, Luck M, Honari S (2018) Distribution matching losses can hallucinate features in medical image translation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp. 529–536. https://doi.org/10.1007/978-3-030-00928-160

11.

Alami Mejjati Y, Richardt C, Tompkin J, Cosker D, Kim KI (2018) Unsupervised attention–guided image-to-image translation. Advances in neural information processing systems, 31

12.

Emami H, DongM, Glide–Hurst, CK. (2020) Attention-guided generative adversarial network to address atypical anatomy in synthetic CT generation. In: 2020 IEEE 21st International conference on information reuse and integration for data science (IRI), pp. 188–193 IEEE https://doi.org/10.1109/IRI49571.2020.00034

13.

Abu-Srhan A, Almallahi I, Abushariah MA, Mahafza W, Al-Kadi OS (2021) Paired-unpaired unsupervised attention guided GAN with transfer learning for bidirectional brain MR-CT synthesis. Comput Biol Med 136:104763. https://doi.org/10.1016/j.compbiomed.2021.104763CrossRefPubMed

14.

Gutierrez A, Tuladhar A, Rajashekar D, Forkert ND (2022) Lesion-preserving unpaired image-to-image translation between MRI and CT from ischemic stroke patients. In: Medical imaging 2022: computer-aided diagnosis, vol. 12033, pp. 308–314 https://doi.org/10.1117/12.2613203

15.

Demchuk A, Goyal M, Menon B, Eesa M, Ryckborst K, Kamal N, Patil S, Mishra S, Almekhlafi M, Randhawa P, Roy D, Willinsky R, Montanera W, Silver F, Shuaib A, Rempel J, Jovin T, Frei D, Sapkota B, Hill M (2015) Endovascular treatment for small core and anterior circulation proximal occlusion with emphasis on minimizing CT to recanalization times (ESCAPE) trial: methodology. Int J Stroke 10(3):429–438. https://doi.org/10.1111/ijs.12424CrossRefPubMed

16.

Cheng B, Forkert ND, Zavaglia M, Hilgetag C, Golsari A, Siemonsen S, Fiehler J, Pedraza S, Puig J, Cho T-H, Alawneh J, Baron J-C, Ostergaard L, Gerloff C, Thomalla G (2014) Influence of stroke infarct location on functional outcome measured by the modified rankin scale. Stroke 45(6):1695–1702. https://doi.org/10.1161/STROKEAHA.114.005152CrossRefPubMed

17.

Menon B, Al-Ajlan F, Najm M, Puig J, Castellanos M, Dowlatshahi D, Calleja A, Sohn S-I, Ahn SH, Poppe A, Mikulík R, Asdaghi N, Field T, Jin A, Asil T, Boulanger J-M, Smith E, Coutts S, Barber P, Demchuk A (2018) Association of clinical, imaging, and thrombus characteristics with recanalization of visible intracranial occlusion in patients with acute ischemic stroke. JAMA 320(10):1017–1026. https://doi.org/10.1001/jama.2018.12498CrossRefPubMedPubMedCentral

18.

Avants BB, Tustison NJ, Song G, Cook PA, Klein A, Gee JC (2011) A reproducible evaluation of ANTs similarity metric performance in brain image registration. Neuroimage 54(3):2033–2044. https://doi.org/10.1016/j.neuroimage.2010.09.025CrossRefPubMed

19.

Muschelli J, Ullman NL, Mould WA, Vespa P, Hanley DF, Crainiceanu CM (2015) Validated automatic brain extraction of head CT images. Neuroimage 114:379–385. https://doi.org/10.1016/j.neuroimage.2015.03.074CrossRefPubMed

20.

Rajashekar D, Wilms M, MacDonald ME, Ehrhardt J, Mouches P, Frayne R, Hill MD, Forkert ND (2020) High-resolution T2-FLAIR and non-contrast CT brain atlas of the elderly. Sci Data 7(1):1–7. https://doi.org/10.1038/s41597-020-0379-9CrossRef

21.

Ehrhardt J, Säring D, Handels H (2007) Structure-preserving interpolation of temporal and spatial image sequences using an optical flow-based method. Methods Inf Med 46(03):300–307. https://doi.org/10.1160/ME9047CrossRefPubMed

22.

Heusel M, Ramsauer H, Unterthiner T, Nessler B, & Hochreiter S (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30

23.

Zhang Y, Liu S, Li C, Wang J (2021) Rethinking the dice loss for deep learning lesion segmentation in medical images. J Shanghai Jiaotong Univ (Sci) 26:93–102. https://doi.org/10.1007/s12204-021-2264-xCrossRef

24.

Thiyagarajan SK, Murugan K (2021) A systematic review on techniques adapted for segmentation and classification of ischemic stroke lesions from brain MR images. Wireless Pers Commun 118(2):1225–1244. https://doi.org/10.1007/s11277-021-08069-zCrossRef

25.

Broocks G, Leischner H, Hanning U, Flottmann F, Faizy T, Schön G, Sporns P, Thomalla G, Kamalian S, Lev M, Fiehler J, Kemmling A (2020) Lesion age imaging in acute stroke: water uptake in CT versus DWI-FLAIR mismatch. Ann Neurol 88(6):1144–1152CrossRefPubMed

26.

Qazi E, Al-Ajlan F, Mahajan A, Sohn S-I, Mishra SChang H, Najm M, d’Esterre C, Demchuk A, Goyal M, Lee T, Hill M, Menon B (2016) Non-contrast CT in place of MRI mismatch in the imaging triage of acute ischemic stroke patients. Med Res Arch 4(6)

Title: Lesion-preserving unpaired image-to-image translation between MRI and CT from ischemic stroke patients
Authors: Alejandro Gutierrez
Anup Tuladhar
Matthias Wilms
Deepthi Rajashekar
Michael D. Hill
Andrew Demchuk
Mayank Goyal
Jens Fiehler
Nils D. Forkert
Publication date: 06-01-2023
Publisher: Springer International Publishing
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 5/2023
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-022-02828-4

Springer Professional

Abstract

Purpose

Methods

Results

Conclusion

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 5/2023

Diagnostic posture control system for seated-style echocardiography robot

A virtual surgical prototype system based on gesture recognition for virtual surgical training in maxillofacial surgery

A hybrid hierarchical strategy for registration of 7T TOF-MRI to 7T PC-MRI intracranial vessel data

Tissues margin-based analytical anisotropic algorithm boosting method via deep learning attention mechanism with cervical cancer

Correction to: Gaussian affinity and GIoU-based loss for perforation detection and localization from colonoscopy videos

Domain generalization improves end-to-end object detection for real-time surgical tool detection

Premium Partner