Skip to main content
main-content
Top

Hint

Swipe to navigate through the chapters of this book

2019 | OriginalPaper | Chapter

Multi-modal Segmentation with Missing MR Sequences Using Pre-trained Fusion Networks

share
SHARE

Abstract

Missing data is a common problem in machine learning and in retrospective imaging research it is often encountered in the form of missing imaging modalities. We propose to take into account missing modalities in the design and training of neural networks, to ensure that they are capable of providing the best possible prediction even when multiple images are not available. The proposed network combines three modifications to the standard 3D UNet architecture: a training scheme with dropout of modalities, a multi-pathway architecture with fusion layer in the final stage, and the separate pre-training of these pathways. These modifications are evaluated incrementally in terms of performance on full and missing data, using the BraTS multi-modal segmentation challenge. The final model shows significant improvement with respect to the state of the art on missing data and requires less memory during training.
Literature
1.
go back to reference Bakas, S., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. Data 4, 170117 (2017) CrossRef Bakas, S., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. Data 4, 170117 (2017) CrossRef
4.
go back to reference Isensee, F., Kickingereder, P., Wick, W., Bendszus, M., Maier-Hein, K.H.: No New-Net. ArXiv e-prints, September 2018 Isensee, F., Kickingereder, P., Wick, W., Bendszus, M., Maier-Hein, K.H.: No New-Net. ArXiv e-prints, September 2018
5.
go back to reference Jerez, J., et al.: Missing data imputation using statistical and machine learning methods in a real breast cancer problem. Artif. Intell. Med. 50(2), 105–115 (2010) CrossRef Jerez, J., et al.: Missing data imputation using statistical and machine learning methods in a real breast cancer problem. Artif. Intell. Med. 50(2), 105–115 (2010) CrossRef
6.
go back to reference Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2014) Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2014)
7.
go back to reference Maaten, L.V.D., Hinton, G., Visualizing data using t-SNE: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008) MATH Maaten, L.V.D., Hinton, G., Visualizing data using t-SNE: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008) MATH
8.
go back to reference Menze, B.H., Jakab, A., Bauer, S., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015) CrossRef Menze, B.H., Jakab, A., Bauer, S., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015) CrossRef
Metadata
Title
Multi-modal Segmentation with Missing MR Sequences Using Pre-trained Fusion Networks
Authors
Karin van Garderen
Marion Smits
Stefan Klein
Copyright Year
2019
DOI
https://doi.org/10.1007/978-3-030-33391-1_19

Premium Partner