Skip to main content

2021 | OriginalPaper | Buchkapitel

StairwayGraphNet for Inter- and Intra-modality Multi-resolution Brain Graph Alignment and Synthesis

verfasst von : Islem Mhiri, Mohamed Ali Mahjoub, Islem Rekik

Erschienen in: Machine Learning in Medical Imaging

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Synthesizing multimodality medical data provides complementary knowledge and helps doctors make precise clinical decisions. Although promising, existing multimodal brain graph synthesis frameworks have several limitations. First, they mainly tackle only one problem (intra- or inter-modality), limiting their generalizability to synthesizing inter- and intra-modality simultaneously. Second, while few techniques work on super-resolving low-resolution brain graphs within a single modality (i.e., intra), inter-modality graph super-resolution remains unexplored though this would avoid the need for costly data collection and processing. More importantly, both target and source domains might have different distributions, which causes a domain fracture between them. To fill these gaps, we propose a multi-resolution StairwayGraphNet (SG-Net) framework to jointly infer a target graph modality based on a given modality and super-resolve brain graphs in both inter and intra domains. Our SG-Net is grounded in three main contributions: (i) predicting a target graph from a source one based on a novel graph generative adversarial network in both inter (e.g., morphological-functional) and intra (e.g., functional-functional) domains, (ii) generating high-resolution brain graphs without resorting to the time consuming and expensive MRI processing steps, and (iii) enforcing the source distribution to match that of the ground truth graphs using an inter-modality aligner to relax the loss function to optimize. Moreover, we design a new Ground Truth-Preserving loss function to guide both generators in learning the topological structure of ground truth brain graphs more accurately. Our comprehensive experiments on predicting target brain graphs from source graphs using a multi-resolution stairway showed the outperformance of our method in comparison with its variants and state-of-the-art method. SG-Net presents the first work for graph alignment and synthesis across varying modalities and resolutions, which handles graph size, distribution, and structure variations. Our Python TIS-Net code is available on BASIRA GitHub at https://​github.​com/​basiralab/​SG-Net.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Cao, B., Zhang, H., Wang, N., Gao, X., Shen, D.: Auto-gan: self-supervised collaborative learning for medical image synthesis. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 10486–10493 (2020) Cao, B., Zhang, H., Wang, N., Gao, X., Shen, D.: Auto-gan: self-supervised collaborative learning for medical image synthesis. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 10486–10493 (2020)
3.
Zurück zum Zitat Singh, N.K., Raza, K.: Medical image generation using generative adversarial networks: A review, pp. 77–96. A Computational Perspective in Healthcare, Health Informatics (2021) Singh, N.K., Raza, K.: Medical image generation using generative adversarial networks: A review, pp. 77–96. A Computational Perspective in Healthcare, Health Informatics (2021)
4.
Zurück zum Zitat Wang, C.: Dicyc: gan-based deformation invariant cross-domain information fusion for medical image synthesis. Inf. Fus. 67, 147–160 (2021) CrossRef Wang, C.: Dicyc: gan-based deformation invariant cross-domain information fusion for medical image synthesis. Inf. Fus. 67, 147–160 (2021) CrossRef
5.
Zurück zum Zitat Liu, Y., Pan, Y., Yang, W., Ning, Z., Yue, L., Liu, M., Shen, D.: Joint neuroimage synthesis and representation learning for conversion prediction of subjective cognitive decline. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 583–592 (2020) Liu, Y., Pan, Y., Yang, W., Ning, Z., Yue, L., Liu, M., Shen, D.: Joint neuroimage synthesis and representation learning for conversion prediction of subjective cognitive decline. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 583–592 (2020)
6.
Zurück zum Zitat Zhan, B., et al.: LR-CGAN: latent representation based conditional generative adversarial network for multi-modality MRI synthesis. Biomedical Signal Processing and Control 66, 102457 (2021) CrossRef Zhan, B., et al.: LR-CGAN: latent representation based conditional generative adversarial network for multi-modality MRI synthesis. Biomedical Signal Processing and Control 66, 102457 (2021) CrossRef
7.
Zurück zum Zitat van den Heuvel, M.P., Sporns, O.: A cross-disorder connectome landscape of brain dysconnectivity. Nature Rev. Neurosci. 20, 435–446 (2019) CrossRef van den Heuvel, M.P., Sporns, O.: A cross-disorder connectome landscape of brain dysconnectivity. Nature Rev. Neurosci. 20, 435–446 (2019) CrossRef
8.
Zurück zum Zitat Bronstein, M.M., Bruna, J., LeCun, Y., Szlam, A., Vandergheynst, P.: Geometric deep learning: going beyond euclidean data. IEEE Sig. Process. Mag. 34, 18–42 (2017) CrossRef Bronstein, M.M., Bruna, J., LeCun, Y., Szlam, A., Vandergheynst, P.: Geometric deep learning: going beyond euclidean data. IEEE Sig. Process. Mag. 34, 18–42 (2017) CrossRef
10.
Zurück zum Zitat Bessadok, A., Mahjoub, M.A., Rekik, I.: Topology-aware generative adversarial network for joint prediction of multiple brain graphs from a single brain graph. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 551–561 (2020) Bessadok, A., Mahjoub, M.A., Rekik, I.: Topology-aware generative adversarial network for joint prediction of multiple brain graphs from a single brain graph. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 551–561 (2020)
11.
Zurück zum Zitat Zhang, L., Wang, L., Zhu, D.: Recovering brain structural connectivity from functional connectivity via multi-gcn based generative adversarial network. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 53–61 (2020) Zhang, L., Wang, L., Zhu, D.: Recovering brain structural connectivity from functional connectivity via multi-gcn based generative adversarial network. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 53–61 (2020)
12.
Zurück zum Zitat Isallari, M., Rekik, I.: GSR-Net: Graph super-resolution network for predicting high-resolution from low-resolution functional brain connectomes. In: International Workshop on Machine Learning in Medical Imaging, pp. 139–149 (2020) Isallari, M., Rekik, I.: GSR-Net: Graph super-resolution network for predicting high-resolution from low-resolution functional brain connectomes. In: International Workshop on Machine Learning in Medical Imaging, pp. 139–149 (2020)
13.
Zurück zum Zitat Pan, S., Hu, R., Long, G., Jiang, J., Yao, L., Zhang, C.: Adversarially regularized graph autoencoder for graph embedding. arXiv preprint arXiv:​1802.​04407 (2018) Pan, S., Hu, R., Long, G., Jiang, J., Yao, L., Zhang, C.: Adversarially regularized graph autoencoder for graph embedding. arXiv preprint arXiv:​1802.​04407 (2018)
14.
Zurück zum Zitat Simonovsky, M., Komodakis, N.: Dynamic edge-conditioned filters in convolutional neural networks on graphs. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3693–3702 (2017) Simonovsky, M., Komodakis, N.: Dynamic edge-conditioned filters in convolutional neural networks on graphs. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3693–3702 (2017)
15.
Zurück zum Zitat Zemouri, R.: Semi-supervised adversarial variational autoencoder. Mach. Learn. Know. Extr. 2, 361–378 (2020) CrossRef Zemouri, R.: Semi-supervised adversarial variational autoencoder. Mach. Learn. Know. Extr. 2, 361–378 (2020) CrossRef
17.
Zurück zum Zitat Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241 (2015) Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241 (2015)
18.
19.
Zurück zum Zitat Gürler, Z., Nebli, A., Rekik, I.: Foreseeing brain graph evolution over time using deep adversarial network normalizer. In: International Workshop on Predictive Intelligence In Medicine, pp. 111–122 (2020) Gürler, Z., Nebli, A., Rekik, I.: Foreseeing brain graph evolution over time using deep adversarial network normalizer. In: International Workshop on Predictive Intelligence In Medicine, pp. 111–122 (2020)
21.
Zurück zum Zitat Fischl, B.: Neuroimage. Freesurfer 62, 774–781 (2012) Fischl, B.: Neuroimage. Freesurfer 62, 774–781 (2012)
22.
Zurück zum Zitat Dosenbach, N.U., et al.: Prediction of individual brain maturity using FMRI. Science 329, 1358–1361 (2010) CrossRef Dosenbach, N.U., et al.: Prediction of individual brain maturity using FMRI. Science 329, 1358–1361 (2010) CrossRef
23.
Zurück zum Zitat Shen, X., Tokoglu, F., Papademetris, X., Constable, R.T.: Groupwise whole-brain parcellation from resting-state FMRI data for network node identification. Neuroimage 82, 403–415 (2013) CrossRef Shen, X., Tokoglu, F., Papademetris, X., Constable, R.T.: Groupwise whole-brain parcellation from resting-state FMRI data for network node identification. Neuroimage 82, 403–415 (2013) CrossRef
24.
Zurück zum Zitat Loshchilov, I., Hutter, F.: Fixing weight decay regularization in adam (2018) Loshchilov, I., Hutter, F.: Fixing weight decay regularization in adam (2018)
25.
Zurück zum Zitat Glasser, M.F., et al.: A multi-modal parcellation of human cerebral cortex. Nature 536, 171 (2016) CrossRef Glasser, M.F., et al.: A multi-modal parcellation of human cerebral cortex. Nature 536, 171 (2016) CrossRef
Metadaten
Titel
StairwayGraphNet for Inter- and Intra-modality Multi-resolution Brain Graph Alignment and Synthesis
verfasst von
Islem Mhiri
Mohamed Ali Mahjoub
Islem Rekik
Copyright-Jahr
2021
DOI
https://doi.org/10.1007/978-3-030-87589-3_15