Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
International Journal of Machine Learning and Cybernetics
Published in: International Journal of Machine Learning and Cybernetics 11/2023

07-06-2023 | Original Article

An attentive-based generative model for medical image synthesis

Authors: Jiayuan Wang, Q. M. Jonathan Wu, Farhad Pourpanah

Published in: International Journal of Machine Learning and Cybernetics | Issue 11/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Magnetic resonance (MR) and computer tomography (CT) imaging are valuable tools for diagnosing diseases and planning treatment. However, limitations such as radiation exposure and cost can restrict access to certain imaging modalities. To address this issue, medical image synthesis can generate one modality from another, but many existing models struggle with high-quality image synthesis when multiple slices are present in the dataset. This study proposes an attention-based dual contrast generative model, called ADC-cycleGAN, which can synthesize medical images from unpaired data with multiple slices. The model integrates a dual contrast loss term with the CycleGAN loss to ensure that the synthesized images are distinguishable from the source domain. Additionally, an attention mechanism is incorporated into the generators to extract informative features from both channel and spatial domains. To improve performance when dealing with multiple slices, the K-means algorithm is used to cluster the dataset into K groups, and each group is used to train a separate ADC-cycleGAN. Experimental results demonstrate that the proposed ADC-cycleGAN model produces comparable samples to other state-of-the-art generative models, achieving the highest PSNR and SSIM values of 19.04385 and 0.68551, respectively. We publish the code at https://​github.​com/​JiayuanWang-JW/​ADC-cycleGAN.
previous article A novel fairness-aware ensemble model based on hybrid sampling and modified two-layer stacking for fair classification
next article Cervical cancerous cell classification: opposition-based harmony search for deep feature selection

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
Show more products
Footnotes
Literature
1.
Xu L, Zeng X, Zhang H, Li W, Lei J, Huang Z (2020) Bpgan: Bidirectional ct-to-mri prediction using multi-generative multi-adversarial nets with spectral normalization and localization. Neural Netw 128:82–96CrossRef
2.
Yang H, Lu X, Wang S-H, Lu Z, Yao J, Jiang Y, Qian P (2021) Synthesizing multi-contrast mr images via novel 3d conditional variational auto-encoding gan. Mobile Netw Appl 26(1):415–424CrossRef
3.
Chen X, Lian C, Wang L, Deng H, Fung SH, Nie D, Thung K-H, Yap P-T, Gateno J, Xia JJ et al (2019) One-shot generative adversarial learning for mri segmentation of craniomaxillofacial bony structures. IEEE Trans Med Imaging 39(3):787–796CrossRef
4.
Lee JH, Han IH, Kim DH, Yu S, Lee IS, Song YS, Joo S, Jin C-B, Kim H (2020) Spine computed tomography to magnetic resonance image synthesis using generative adversarial networks: a preliminary study. J Korean Neurosurg Soc 63(3):386–396CrossRef
5.
Tomar D, Lortkipanidze M, Vray G, Bozorgtabar B, Thiran J-P (2021) Self-attentive spatial adaptive normalization for cross-modality domain adaptation. IEEE Trans Med Imaging 40(10):2926–2938CrossRef
6.
Mérida I, Costes N, Heckemann RA, Drzezga A, Förster S, Hammers A (2015) Evaluation of several multi-atlas methods for pseudo-ct generation in brain mri-pet attenuation correction. In: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), pp. 1431–1434. IEEE
7.
Lian C, Li X, Kong L, Wang J, Zhang W, Huang X, Wang L (2022) Cocyclereg: collaborative cycle-consistency method for multi-modal medical image registration. Neurocomputing
8.
Li X, Jia M, Islam MT, Yu L, Xing L (2020) Self-supervised feature learning via exploiting multi-modal data for retinal disease diagnosis. IEEE Trans Med Imaging 39(12):4023–4033CrossRef
9.
Jiao J, Namburete AI, Papageorghiou AT, Noble JA (2020) Self-supervised ultrasound to mri fetal brain image synthesis. IEEE Trans Med Imaging 39(12):4413–4424CrossRef
10.
Berker Y, Franke J, Salomon A, Palmowski M, Donker HC, Temur Y, Mottaghy FM, Kuhl C, Izquierdo-Garcia D, Fayad ZA et al (2012) Mri-based attenuation correction for hybrid pet/mri systems: a 4-class tissue segmentation technique using a combined ultrashort-echo-time/dixon mri sequence. J Nucl Med 53(5):796–804CrossRef
11.
Sjölund J, Forsberg D, Andersson M, Knutsson H (2015) Generating patient specific pseudo-ct of the head from mr using atlas-based regression. Phys Med Biol 60(2):825CrossRef
12.
Bhosale YH, Patnaik KS (2022) Application of deep learning techniques in diagnosis of covid-19 (coronavirus): a systematic review. Neural Process Lett: 1–53
13.
Bhosale YH, Patnaik KS (2023) Puldi-covid: chronic obstructive pulmonary (lung) diseases with covid-19 classification using ensemble deep convolutional neural network from chest x-ray images to minimize severity and mortality rates. Biomed Signal Process Control 81:104445CrossRef
14.
Li R, Zhang W, Suk H-I, Wang L, Li J, Shen D, Ji S (2014) Deep learning based imaging data completion for improved brain disease diagnosis. In: International Conference on Medical Image Computing and Computer-assisted Intervention, pp. 305–312. Springer
15.
Huang Y, Shao L, Frangi AF (2017) Simultaneous super-resolution and cross-modality synthesis of 3d medical images using weakly-supervised joint convolutional sparse coding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6070–6079
16.
Zhao Y, Liao S, Guo Y, Zhao L, Yan Z, Hong S, Hermosillo G, Liu T, Zhou XS, Zhan Y (2018) Towards mr-only radiotherapy treatment planning: synthetic ct generation using multi-view deep convolutional neural networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 286–294
17.
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680
18.
Nie D, Trullo R, Lian J, Wang L, Petitjean C, Ruan S, Wang Q, Shen D (2018) Medical image synthesis with deep convolutional adversarial networks. IEEE Trans Biomed Eng 65(12):2720–2730CrossRef
19.
Dalmaz O, Yurt M, Çukur T (2022) Resvit: residual vision transformers for multi-modal medical image synthesis. IEEE Trans Med Imaging
20.
Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232
21.
Liu S, Zhang B, Liu Y, Han A, Shi H, Guan T, He Y (2021) Unpaired stain transfer using pathology-consistent constrained generative adversarial networks. IEEE Trans Med Imaging 40(8):1977–1989CrossRef
22.
Huo Y, Xu Z, Moon H, Bao S, Assad A, Moyo TK, Savona MR, Abramson RG, Landman BA (2018) Synseg-net: synthetic segmentation without target modality ground truth. IEEE Trans Med Imaging 38(4):1016–1025CrossRef
23.
Liu Y, Lei Y, Wang T, Fu Y, Tang X, Curran WJ, Liu T, Patel P, Yang X (2020) Cbct-based synthetic ct generation using deep-attention cyclegan for pancreatic adaptive radiotherapy. Med Phys 47(6):2472–2483CrossRef
24.
Huang Z, Chen Z, Zhang Q, Quan G, Ji M, Zhang C, Yang Y, Liu X, Liang D, Zheng H et al (2020) Cagan: a cycle-consistent generative adversarial network with attention for low-dose ct imaging. IEEE Trans Comput Imaging 6:1203–1218MathSciNetCrossRef
25.
Xu Z, Qi C, Xu G (2019) Semi-supervised attention-guided cyclegan for data augmentation on medical images. In: IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 563–568
26.
Nie D, Shen D (2020) Adversarial confidence learning for medical image segmentation and synthesis. Int J Comput Vis 128(10):2494–2513MathSciNetCrossRefMATH
27.
Yang H, Sun J, Carass A, Zhao C, Lee J, Prince JL, Xu Z (2020) Unsupervised mr-to-ct synthesis using structure-constrained cyclegan. IEEE Trans Med Imaging 39(12):4249–4261CrossRef
28.
29.
Han X (2017) Mr-based synthetic ct generation using a deep convolutional neural network method. Med Phys 44(4):1408–1419CrossRef
30.
Abu-Srhan A, Almallahi I, Abushariah MA, Mahafza W, Al-Kadi OS (2021) Paired-unpaired unsupervised attention guided gan with transfer learning for bidirectional brain mr-ct synthesis. Comput Biol Med 136:104763CrossRef
31.
Woo S, Park J, Lee J-Y, Kweon IS (2018) CBAM: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19
32.
Wang T, Lei Y, Fu Y, Wynne JF, Curran WJ, Liu T, Yang X (2021) A review on medical imaging synthesis using deep learning and its clinical applications. J Appl Clin Med Phys 22(1):11–36CrossRef
33.
Hofmann M, Steinke F, Scheel V, Charpiat G, Farquhar J, Aschoff P, Brady M, Schölkopf B, Pichler BJ (2008) Mri-based attenuation correction for pet/mri: a novel approach combining pattern recognition and atlas registration. J Nucl Med 49(11):1875–1883CrossRef
34.
Chen M, Jog A, Carass A, Prince JL (2015) Using image synthesis for multi-channel registration of different image modalities. In: Medical Imaging 2015: Image Processing, vol. 9413, pp. 462–468. SPIE
35.
Dowling JA, Lambert J, Parker J, Salvado O, Fripp J, Capp A, Wratten C, Denham JW, Greer PB (2012) An atlas-based electron density mapping method for magnetic resonance imaging (mri)-alone treatment planning and adaptive mri-based prostate radiation therapy. Int J Radiat Oncol Biol Phys 83(1):5–11CrossRef
36.
Izquierdo-Garcia D, Hansen AE, Förster S, Benoit D, Schachoff S, Fürst S, Chen KT, Chonde DB, Catana C (2014) An spm8-based approach for attenuation correction combining segmentation and nonrigid template formation: application to simultaneous pet/mr brain imaging. J Nucl Med 55(11):1825–1830CrossRef
37.
Delpon G, Escande A, Ruef T, Darréon J, Fontaine J, Noblet C, Supiot S, Lacornerie T, Pasquier D (2016) Comparison of automated atlas-based segmentation software for postoperative prostate cancer radiotherapy. Front Oncology 6:178CrossRef
38.
Hsu S-H, Cao Y, Huang K, Feng M, Balter JM (2013) Investigation of a method for generating synthetic ct models from mri scans of the head and neck for radiation therapy. Physics Med Biol 58(23):8419CrossRef
39.
Burgos N, Cardoso MJ, Thielemans K, Modat M, Pedemonte S, Dickson J, Barnes A, Ahmed R, Mahoney CJ, Schott JM et al (2014) Attenuation correction synthesis for hybrid pet-mr scanners: application to brain studies. IEEE Trans Med Imaging 33(12):2332–2341CrossRef
40.
Sevetlidis V, Giuffrida MV, Tsaftaris SA (2016) Whole image synthesis using a deep encoder-decoder network. In: International Workshop on Simulation and Synthesis in Medical Imaging, pp. 127–137. Springer
41.
Zhou T, Fu H, Chen G, Shen J, Shao L (2020) Hi-net: hybrid-fusion network for multi-modal mr image synthesis. IEEE Trans Med Imaging 39(9):2772–2781CrossRef
42.
Cao B, Zhang H, Wang N, Gao X, Shen D (2020) Auto-gan: self-supervised collaborative learning for medical image synthesis. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 10486–10493
43.
Zhang T, Fu H, Zhao Y, Cheng J, Guo M, Gu Z, Yang B, Xiao Y, Gao S, Liu J (2019) Skrgan: sketching-rendering unconditional generative adversarial networks for medical image synthesis. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 777–785. Springer
44.
Hu S, Yuan J, Wang S (2019) Cross-modality synthesis from mri to pet using adversarial u-net with different normalization. In: 2019 International Conference on Medical Imaging Physics and Engineering (ICMIPE), pp. 1–5. IEEE
45.
Wu H, Jiang X, Jia F (2019) Uc-gan for mr to ct image synthesis. In: Workshop on Artificial Intelligence in Radiation Therapy, pp. 146–153. Springer
46.
Yang H, Sun J, Carass A, Zhao C, Lee J, Prince JL, Xu Z (2020) Unsupervised mr-to-ct synthesis using structure-constrained cyclegan. IEEE Trans Med Imaging 39(12):4249–4261CrossRef
47.
Chen R, Huang W, Huang B, Sun F, Fang B (2020) Reusing discriminators for encoding: Towards unsupervised image-to-image translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8168–8177
48.
Lee J, Gu J, Ye JC (2021) Unsupervised ct metal artifact learning using attention-guided \(\beta\)-cyclegan. IEEE Trans Med Imaging 40(12):3932–3944CrossRef
49.
Kong L, Lian C, Huang D, Hu Y, Zhou Q et al (2021) Breaking the dilemma of medical image-to-image translation. Adv Neural Inform Process Syst 34
50.
Kim J, Kim M, Kang H, Lee KH U-gat-it: Unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation. In: International Conference on Learning Representations
51.
Larochelle H, Hinton GE (2010) Learning to combine foveal glimpses with a third-order boltzmann machine. Adv Neural Inform Process Syst 23
52.
Fukui H, Hirakawa T, Yamashita T, Fujiyoshi H (2019) Attention branch network: Learning of attention mechanism for visual explanation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10705–10714
53.
Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3146–3154
54.
Liu G, Guo J (2019) Bidirectional lstm with attention mechanism and convolutional layer for text classification. Neurocomputing 337:325–338CrossRef
55.
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inform Process Syst:30
56.
Chen C-FR, Fan Q, Panda R (2021) Crossvit: cross-attention multi-scale vision transformer for image classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 357–366
57.
Guo M-H, Xu T-X, Liu J-J, Liu Z-N, Jiang P-T, Mu T-J, Zhang S-H, Martin RR, Cheng M-M, Hu S-M (2022) Attention mechanisms in computer vision: a survey. Comput Vis Med:1–38
58.
Misra D, Nalamada T, Arasanipalai AU, Hou Q (2021) Rotate to attend: convolutional triplet attention module. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3139–3148
59.
Wang S-H, Fernandes SL, Zhu Z, Zhang Y-D (2021) Avnc: attention-based vgg-style network for covid-19 diagnosis by cbam. IEEE Sens J 22(18):17431–17438CrossRef
60.
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778
61.
Snell J, Ridgeway K, Liao R, Roads BD, Mozer MC, Zemel RS (2017) Learning to generate images with perceptual similarity metrics. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 4277–4281. IEEE
62.
Zhou Y, Wang X, Zhang M, Zhu J, Zheng R, Wu Q (2019) Mpce: a maximum probability based cross entropy loss function for neural network classification. IEEE Access 7:146331–146341CrossRef
63.
Zhong Y, Liu L, Zhao D, Li H (2020) A generative adversarial network for image denoising. Multimed Tools Appl 79(23):16517–16529CrossRef
64.
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612CrossRef
Metadata
Title
An attentive-based generative model for medical image synthesis
Authors
Jiayuan Wang
Q. M. Jonathan Wu
Farhad Pourpanah
Publication date
07-06-2023
Publisher
Springer Berlin Heidelberg
Published in
International Journal of Machine Learning and Cybernetics / Issue 11/2023
Print ISSN: 1868-8071
Electronic ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-023-01871-0

Other articles of this Issue 11/2023

International Journal of Machine Learning and Cybernetics 11/2023 Go to the issue

Original Article

Optimal scale selection based on three-way decisions with decision-theoretic rough sets in multi-scale set-valued decision tables

Original Article

Consensus latent incomplete multi-view clustering with low-rank tensor constraint

Original Article

Brain-inspired learning to deeper inductive reasoning for video captioning

Original Article

Reinforced feature selection using Q-learning based on collaborative agents

Original Article

Manifold-based denoising, outlier detection, and dimension reduction algorithm for high-dimensional data

Original Article

Personalized federated learning based on multi-head attention algorithm