RobustSsF: Robust Missing Modality Brain Tumor Segmentation with Self-supervised Learning-Based Scenario-Specific Fusion

All modalities of Magnetic Resonance Imaging (MRI) have an essential role in diagnosing brain tumors, but there are some challenges posed by missing or incomplete modalities in multimodal MRI. Existing models have failed to achieve robust performance across all scenarios. To address this issue, this paper proposes a novel 4encoder-4decoder architecture that incorporates both “dedicated” and “single” models. Our model includes multiple Scenario-specific Fusion (SsF) decoders that construct different features depending on the missing modality scenarios. To train our model, we introduce a novel self-supervised learning-based loss function called Couple Regularization (CReg) to achieve robust learning and the Lifelong Learning Strategy (LLS) to enhance model performance. The experimental results on BraTS2018 demonstrate that RobustSsF has successfully improved robustness by reducing standard deviations from 12 times to 76 times lower, also achieving state-of-the-art results in all scenarios when the T1ce modality is missing.