Skip to main content
Top

Hint

Swipe to navigate through the chapters of this book

2021 | OriginalPaper | Chapter

Non-isomorphic Inter-modality Graph Alignment and Synthesis for Holistic Brain Mapping

Authors : Islem Mhiri, Ahmed Nebli, Mohamed Ali Mahjoub, Islem Rekik

Published in: Information Processing in Medical Imaging

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Brain graph synthesis marked a new era for predicting a target brain graph from a source one without incurring the high acquisition cost and processing time of neuroimaging data. However, works on recovering a brain graph in one modality (e.g., functional brain imaging) from a brain graph in another (e.g., structural brain imaging) remain largely scarce. Besides, existing multimodal graph synthesis frameworks have several limitations. First, they mainly focus on generating graphs from the same domain (intra-modality), overlooking the rich multimodal representations of brain connectivity (inter-modality). Second, they can only handle isomorphic graph generation tasks, limiting their generalizability to synthesizing target graphs with a different node size and topological structure from those of the source one. More importantly, both target and source domains might have different distributions, which causes a domain fracture between them (i.e., distribution misalignment). To address such challenges, we propose an inter-modality aligner of non-isomorphic graphs (IMANGraphNet) framework to infer a target graph modality based on a given modality. Our three core contributions lie in (i) predicting a target graph (e.g., functional) from a source graph (e.g., morphological) based on a novel graph generative adversarial network (gGAN); (ii) using non-isomorphic graphs for both source and target domains with a different number of nodes, edges and structure; and (iii) enforcing the source distribution to match that of the ground truth graphs using a graph aligner to relax the loss function to optimize. Furthermore, to handle the unstable behavior of gGAN, we design a new Ground Truth-Preserving (GT-P) loss function to guide the non-isomorphic generator in learning the topological structure of ground truth brain graphs more effectively. Our comprehensive experiments on predicting target functional brain graphs from source morphological graphs demonstrate the outperformance of IMANGraphNet in comparison with its variants. IMANGraphNet presents the first framework for brain graph synthesis based on aligned non-isomorphic inter-modality brain graphs which handles variations in graph size, distribution and structure. This can be further leveraged for integrative and holistic brain mapping as well as developing multimodal neurological disorder diagnosis frameworks.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Footnotes
1
http://fcon_1000.projects.nitrc.org/.
 
Literature
3.
go back to reference Yu, B., Wang, Y., Wang, L., Shen, D., Zhou, L.: Medical image synthesis via deep learning. In: Deep Learning in Medical Image Analysis, pp. 23–44 (2020) Yu, B., Wang, Y., Wang, L., Shen, D., Zhou, L.: Medical image synthesis via deep learning. In: Deep Learning in Medical Image Analysis, pp. 23–44 (2020)
4.
go back to reference Zhou, T., Fu, H., Chen, G., Shen, J., Shao, L.: Hi-Net: hybrid-fusion network for multi-modal MR image synthesis. IEEE Trans. Med. Imaging 39, 2772–2781 (2020) CrossRef Zhou, T., Fu, H., Chen, G., Shen, J., Shao, L.: Hi-Net: hybrid-fusion network for multi-modal MR image synthesis. IEEE Trans. Med. Imaging 39, 2772–2781 (2020) CrossRef
5.
go back to reference Liu, Y., et al.: Joint neuroimage synthesis and representation learning for conversion prediction of subjective cognitive decline. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 583–592 (2020) Liu, Y., et al.: Joint neuroimage synthesis and representation learning for conversion prediction of subjective cognitive decline. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 583–592 (2020)
6.
go back to reference Dai, X., et al.: Multimodal MRI synthesis using unified generative adversarial networks. Med. Phys. 47, 6343–6354 (2020) CrossRef Dai, X., et al.: Multimodal MRI synthesis using unified generative adversarial networks. Med. Phys. 47, 6343–6354 (2020) CrossRef
7.
go back to reference Yang, Q., et al.: MRI cross-modality image-to-image translation. Sci. Rep. 10, 1–18 (2020) CrossRef Yang, Q., et al.: MRI cross-modality image-to-image translation. Sci. Rep. 10, 1–18 (2020) CrossRef
8.
go back to reference Bassett, D.S., Sporns, O.: Network neuroscience. Nat. Neurosci. 20, 353–364 (2017) CrossRef Bassett, D.S., Sporns, O.: Network neuroscience. Nat. Neurosci. 20, 353–364 (2017) CrossRef
9.
go back to reference van den Heuvel, M.P., Sporns, O.: A cross-disorder connectome landscape of brain dysconnectivity. Nat. Rev. Neurosci. 20, 435–446 (2019) CrossRef van den Heuvel, M.P., Sporns, O.: A cross-disorder connectome landscape of brain dysconnectivity. Nat. Rev. Neurosci. 20, 435–446 (2019) CrossRef
10.
go back to reference Bronstein, M.M., Bruna, J., LeCun, Y., Szlam, A., Vandergheynst, P.: Geometric deep learning: going beyond Euclidean data. IEEE Sig. Process. Mag. 34, 18–42 (2017) CrossRef Bronstein, M.M., Bruna, J., LeCun, Y., Szlam, A., Vandergheynst, P.: Geometric deep learning: going beyond Euclidean data. IEEE Sig. Process. Mag. 34, 18–42 (2017) CrossRef
13.
go back to reference Bessadok, A., Mahjoub, M.A., Rekik, I.: Brain graph synthesis by dual adversarial domain alignment and target graph prediction from a source graph. Med. Image Anal. 68, 101902 (2021) CrossRef Bessadok, A., Mahjoub, M.A., Rekik, I.: Brain graph synthesis by dual adversarial domain alignment and target graph prediction from a source graph. Med. Image Anal. 68, 101902 (2021) CrossRef
14.
go back to reference Zhang, L., Wang, L., Zhu, D.: Recovering brain structural connectivity from functional connectivity via multi-GCN based generative adversarial network. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 53–61 (2020) Zhang, L., Wang, L., Zhu, D.: Recovering brain structural connectivity from functional connectivity via multi-GCN based generative adversarial network. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 53–61 (2020)
15.
go back to reference Bessadok, A., Mahjoub, M.A., Rekik, I.: Topology-aware generative adversarial network for joint prediction of multiple brain graphs from a single brain graph. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 551–561 (2020) Bessadok, A., Mahjoub, M.A., Rekik, I.: Topology-aware generative adversarial network for joint prediction of multiple brain graphs from a single brain graph. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 551–561 (2020)
16.
go back to reference Simonovsky, M., Komodakis, N.: Dynamic edge-conditioned filters in convolutional neural networks on graphs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3693–3702 (2017) Simonovsky, M., Komodakis, N.: Dynamic edge-conditioned filters in convolutional neural networks on graphs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3693–3702 (2017)
17.
go back to reference Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241 (2015) Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241 (2015)
20.
go back to reference Dosenbach, N.U., et al.: Prediction of individual brain maturity using fMRI. Science 329, 1358–1361 (2010) CrossRef Dosenbach, N.U., et al.: Prediction of individual brain maturity using fMRI. Science 329, 1358–1361 (2010) CrossRef
21.
go back to reference Loshchilov, I., Hutter, F.: Fixing weight decay regularization in Adam (2018) Loshchilov, I., Hutter, F.: Fixing weight decay regularization in Adam (2018)
22.
go back to reference Glasser, M., Coalson, T., Robinson, E., Hacker, C.D., et al.: A multi-modal parcellation of human cerebral cortex. Nature 536, 171–178 (2016) CrossRef Glasser, M., Coalson, T., Robinson, E., Hacker, C.D., et al.: A multi-modal parcellation of human cerebral cortex. Nature 536, 171–178 (2016) CrossRef
Metadata
Title
Non-isomorphic Inter-modality Graph Alignment and Synthesis for Holistic Brain Mapping
Authors
Islem Mhiri
Ahmed Nebli
Mohamed Ali Mahjoub
Islem Rekik
Copyright Year
2021
DOI
https://doi.org/10.1007/978-3-030-78191-0_16

Premium Partner