Predictive Intelligence in Medicine

Traditional brain lesion segmentation models for multi-modal MRI are typically tailored to specific pathologies, relying on datasets with predefined modalities. Adapting to new MRI modalities or pathologies often requires training separate models, which contrasts with how medical professionals incrementally expand their expertise by learning from diverse datasets over time. Inspired by this human learning process, we propose a unified segmentation model capable of sequentially learning from multiple datasets with varying modalities and pathologies. Our approach leverages a privacy-aware continual learning framework that integrates a mixture-of-experts mechanism and dual knowledge distillation to mitigate catastrophic forgetting while not compromising performance on newly encountered datasets. Extensive experiments across five diverse brain MRI datasets and four dataset sequences demonstrate the effectiveness of our framework in maintaining a single adaptable model, capable of handling varying hospital protocols, imaging modalities, and disease types. Compared to widely used privacy-aware continual learning methods such as LwF, SI, EWC, MiB, and TED, our method achieves an average Dice score improvement of approximately 14%. Our framework represents a significant step toward more versatile and practical brain lesion segmentation models, with implementation available on github.​com/​xmindflow/​BrainCL.

Interstitial lung disease (ILD) is a leading cause of morbidity and mortality in systemic sclerosis (SSc). Chest computed tomography (CT) is the primary imaging modality for diagnosing and monitoring lung complications in SSc patients. However, its role in disease progression and mortality prediction has not yet been fully clarified. This study introduces a novel, large-scale longitudinal chest CT analysis framework that utilizes radiomics and deep learning to predict mortality associated with lung complications of SSc. We collected and analyzed 2,125 CT scans from SSc patients enrolled in the Northwestern Scleroderma Registry, conducting mortality analyses at one, three, and five years using advanced imaging analysis techniques. Death labels were assigned based on recorded deaths over the one-, three-, and five-year intervals, confirmed by expert physicians. In our dataset, 181, 326, and 428 of the 2,125 CT scans were from patients who died within one, three, and five years, respectively. Using ResNet-18, DenseNet-121, and Swin Transformer we use pre-trained models, and fine-tuned on 2,125 images of SSc patients. Models achieved an AUC of 0.769, 0.801, 0.709 for predicting mortality within one-, three-, and five-years, respectively. Our findings highlight the potential of both radiomics and deep learning computational methods to improve early detection and risk assessment of SSc-related interstitial lung disease, marking a significant advancement in the literature.

Brain tumors represent a significant proportion of cancers in humans, with an incidence that continues to rise. Glioblastoma, the most aggressive tumor, demonstrates a variable response to treatment. Patients diagnosed with glioblastoma have a median survival of 15 months. A major challenge is that treatment efficacy, evaluated by anatomical MRI, becomes apparent more than two months after initiation. Given the limited survival time, early identification of non-responders before treatment onset is crucial. A binary classification model was performed on a cohort diagnosed with glioblastoma and treated between 2018 and 2023 at our center. Initially, treatment efficacy prediction was assessed using only the surgical criterion. The obtained sensitivity, specificity, and accuracy were 79.78%, 59.30% and 69.71%, respectively. Subsequently, a classifier was pre-trained using transfer learning on the ResNet-51Q model. This model takes as input nine central slices of pre-treatment MRI per patient. The results obtained on the test set were 79.10%, 90.74%, and 81.68% for sensitivity, specificity, and accuracy respectively. Deep hybrid learning (DHL) models were trained to include clinical data, with 84.38%, 94.74% and 90.00% for sensitivity, specificity, and accuracy respectively. Compared with the criterion of surgery alone, the deep learning approach improves the prediction of treatment efficacy prior to its administration. We enhanced performance by incorporating clinical data. Using models to predict treatment efficacy in GBM patients from pre-treatment data has considerable potential for personalising treatment regimens.

Deep learning models often struggle to maintain robust generalizability in medical imaging, particularly under domain-fracture scenarios where distributional shifts arise due to varying imaging techniques, acquisition protocols, patient populations, demographics, and equipment. In practice, each hospital may need to develop and train distinct models—differing in functionality (i.e., learning task) and morphology including width and depth—to handle their local data distributions. For example, while one hospital may utilize Euclidean architectures such as MLPs and CNNs to process structured tabular data or regular grid-like image data, another hospital may need to deploy non-Euclidean architectures such as graph neural networks (GNNs) to process inherently irregular data like brain connectomes or other graph-structured biomedical information. However, how to train such heterogeneous models coherently across different datasets, in a manner that enhances the generalizability of each model, remains an open and challenging problem. In this paper, we address this issue by introducing a new learning paradigm, namely unified learning. To address the topological differences between these heterogeneous architectures, we first encode each model into a graph representation, enabling us to unify these diverse models within a shared graph learning space. Once represented in this space, a GNN guides the optimization of the unified models. By decoupling the parameters of individual deep learning models and controlling them through the unified GNN (uGNN), our approach enables parameter-sharing and knowledge-transfer across varying architectures (MLPs, CNNs and GNNs) and distributions, ultimately improving its generalizability. We evaluate our framework on MorphoMNIST and two MedMNIST benchmarks—PneumoniaMNIST and BreastMNIST—and find that our unified learning improves the performance of individual models when trained on unique distributions and tested on mixed ones, thereby demonstrating generalizability to unseen data with strong distributional shifts. Our source code including benchmarks and evaluation datasets is available at https://​github.​com/​basiralab/​uGNN.

Accurate prediction of stable and progressive mild cognitive impairment is crucial for early intervention in Alzheimer’s disease. Multimodal data, including clinical tabular data and MRI scans, provide complementary information, yet effective integration remains a challenge. Existing fusion methods, such as cross-attention, emphasize shared modality information while often overlooking critical modality-specific information. In this paper, we propose a novel fusion model that incorporates a hybrid attention module and a latent similarity divergence loss to effectively integrate shared and modality-specific features in a balanced manner. Within the hybrid attention module, self-attention is employed for modality-specific feature learning, while bidirectional cross-modal attention is introduced to extract shared features from clinical tabular and MRI image data. To model clinical tabular data effectively, we propose a column embedding block pre-trained on a large NC-MCI-AD dataset. It captures disease-relevant features while also modeling missing data patterns, making it a robust and essential component for downstream tasks. To model anatomical structure and spatial relationships, image-level features are extracted from MRI data using FastSurferCNN and converted into graph-level representations to capture interactions between brain regions, enabling a richer understanding of disease-relevant patterns. Extensive experiments on ADNI dataset demonstrate that our method outperforms state-of-the-art methods, achieving 0.9042 balanced accuracy and 0.9403 AUC for sMCI and pMCI classification.

The application of deep learning–based generative models has brought a paradigm shift to medical imaging, particularly in MRI-based brain studies involving modality translation and multimodal fusion. This work introduces GM-LDM, a novel latent diffusion framework designed to improve the efficiency and accuracy of MRI generation. To ensure statistical consistency, GM-LDM incorporates a KL-regularized 3D autoencoder pre-trained on large-scale datasets from the ABCD and UK Biobank studies. A Vision Transformer (ViT)-based encoder-decoder serves as the denoising network, enhancing the fidelity of generated images. Crucially, GM-LDM integrates conditional information—specifically, functional network connectivity (FNC)—to enable personalized brain image synthesis, functional-guided structural synthesis, and biomarker discovery in neurological disorders such as schizophrenia. Experimental results demonstrate that GM-LDM generates subject-specific 3D gray matter volumes with high accuracy, showing strong potential for clinical neuroscience applications, including precise disease diagnosis and group-level biomarker localization.

Accurate synthesis of patient-specific chest X-ray (CXR) images that reflect temporal disease progression remains challenging due to the complex interplay of multimodal longitudinal data. We introduce PAX-Diff, a multimodal progression-aware chest X-ray generation framework built on a diffusion model, which synthesizes future medical images by tracking temporal changes within patients’ historical sequences of X-rays and radiology reports. The key is the proposed cross-modal progression-aware conditioning net, consisting of two core components: an intra-visit multimodal learner to align image-texts, and a cross-visit causal attention module to connect the underlying info globally across all image-text pairs from historical visits. Benefiting from this structure, it provides a conditioning signal that effectively integrates historical information, enabling the controlled generation of future images. By extending next-token prediction to cross-visit feature blocks, PAX-Diff explicitly models temporal dependencies across clinical visits, thus provides a predicted multimodal representation of the next visit with causality. Additionally, we propose a hierarchical condition alignment through global cosine similarity and local-level perceptual alignment to refine the model’s training process, enhancing its ability to produce accurate images with clinical semantic consistency.

Knee osteoarthritis (KOA) is a widespread degenerative joint disorder that significantly impacts patient mobility and quality of life. Early and accurate assessment of KOA severity is crucial for effective clinical decision-making and treatment planning. In this study, we propose the Swin Transformer Based Bidirectional Feature Pyramid Network (ST-BiFPN) that combines the multi-scale feature extraction capabilities of a Bidirectional Feature Pyramid Network (BiFPN) with the global contextual modeling strengths of the Swin Transformer. This integrated approach is designed to capture both the subtle local changes and broader structural variations present in knee X-ray images. Extensive evaluations on a comprehensive knee osteoarthritis dataset demonstrate that our method achieves superior performance with 71.14% accuracy, outperforming traditional convolutional neural network architectures by 7-9%. These promising results suggest that the ST-BiFPN offers a robust tool for the automated classification of KOA severity, paving the way for improved diagnostic support and personalized treatment strategies in clinical settings.

Resting-state functional magnetic resonance imaging (rs-fMRI) serves as a powerful tool for studying brain function, yet deriving the optimal representation of brain function remains a challenge. Traditional methods often rely on region of interest (ROI)-based analyses, which simplify complexity and reduce noise but are constrained by fixed ROI partitions and averaged voxel signals, potentially missing valuable information. In our study, we introduce a novel framework for autonomously learning representations for any brain region directly from voxel-level fMRI data. This approach is designed to manage arbitrary ROIs. It incorporates two primary stages: During the pre-training stage, a global-adapt encoder captures whole-brain feature representations from 4D fMRI data, while a mask encoder processes brain region masks to extract geometric features. These features merge with their corresponding fMRI representations to reconstruct the mean BOLD signal of the region, facilitating self-supervised training. By providing a range of brain region masks, our framework enables the learning of representations for a set of arbitrary ROIs, whether derived from established brain atlases or crafted manually. In the fine-tuning stage, the pre-trained model adapts to downstream tasks like gender classification, age prediction, and intelligence prediction. Experiments conducted with the HCP and UK Biobank datasets reveal that our method surpasses competing approaches, delivering highly interpretable and neurofunctionally relevant brain region representations.

Brain connectomes, which represent neural connectivity as graphs, are crucial for understanding brain organization but are costly and time-consuming to acquire, motivating the development of generative methods. Recent progress in graph generative modeling offers a convenient data-driven alternative, enabling synthetic generation of connectomes and alleviating the dependence on extensive neuroimaging acquisitions. However, existing generative models still have the following key limitations: (i) they compress the entire graph into a single latent code, such as variational graph autoencoders (VGAEs), thereby blurring fine-grained local motifs; (ii) they rely on rich node attributes that brain connectomes typically do not provide, which diminishes reconstruction quality; (iii) edge-centric models like dual-graph frameworks emphasize topology but often overlook the accurate prediction of edge weights, sacrificing quantitative fidelity; and (iv) many state-of-the-art architectures, such as those using edge-conditioned convolutions, employ computationally expensive architectures, which results in significant memory demands and limits their scalability to larger connectomes. To address these limitations, we propose GraphTreeGen (GTG), a novel subtree-centric generative framework explicitly designed for efficient and accurate brain connectome generation. GTG decomposes each connectome into entropy-guided k-hop trees that capture informative local structure, which are then encoded by a shared GCN. A bipartite message-passing layer merges subtree embeddings with global node features, and a dual-branch decoder jointly predicts edge existence and weights to rebuild the full adjacency matrix. Experiments demonstrate that GTG significantly outperforms state-of-the-art baselines in self-supervised task, and remains competitive in supervised settings, delivering higher structural fidelity and more precise edge weights while using far less memory. Its modular, resource-efficient design also lays the groundwork for extensions to connectome super-resolution and cross-modality synthesis.

We present the first federated learning (FL) approach for pancreas part (head, body, tail) segmentation in MRI, addressing a critical clinical challenge as a significant innovation. Pancreatic diseases exhibit marked regional heterogeneity—cancers predominantly occur in the head region while chronic pancreatitis causes tissue loss in the tail—making accurate segmentation of the organ into head, body, and tail regions essential for precise diagnosis and treatment planning. This segmentation task remains exceptionally challenging in MRI due to variable morphology, poor soft-tissue contrast, and anatomical variations across patients. Our novel contribution tackles two fundamental challenges: first, the technical complexity of pancreas part delineation in MRI, and second the data scarcity problem that has hindered prior approaches. We introduce a privacy-preserving FL framework that enables collaborative model training across seven medical institutions without direct data sharing, leveraging a diverse dataset of 711 T1W and 726 T2W MRI scans. Our key innovations include: (1) a systematic evaluation of three state-of-the-art segmentation architectures (U-Net, Attention U-Net, Swin UNETR) paired with two FL algorithms (FedAvg, FedProx), revealing Attention U-Net with FedAvg as optimal for pancreatic heterogeneity, which was never been done before; (2) a novel anatomically-informed loss function prioritizing region-specific texture contrasts in MRI. Comprehensive evaluation demonstrates that our approach achieves clinically viable performance despite training on distributed, heterogeneous datasets.

Multicenter imaging data offer a valuable opportunity to develop robust deep learning models, particularly in rare diseases such as Primary Central Nervous System Lymphoma (PCNSL), where single-center data are often insufficient. However, heterogeneity in imaging protocols, scanner types, and acquisition settings introduces domain shifts that hinder model generalization. Synthesizing a unified target modality is a promising solution to harmonize such data. While previous studies have explored one-to-one or paired translation methods, these are either non-scalable or require unavailable data pairs. Other approaches generate synthetic latent domains with limited clinical interpretability and high computational cost. To address these limitations, we propose InStyleGAN, an unpaired harmonization framework using a style-guided 3D CycleGAN tailored for PCNSL. Our key contributions include: (i) synthesizing a target modality from heterogeneous PET data using a 3D CycleGAN backbone; (ii) leveraging unpaired PET data with significant inter-site variations in distribution, structure and style; (iii) enforcing target-style distribution alignment via Adaptive Instance Normalization to relax the optimization. Furthermore, we introduce a novel Style-Consistency loss to better preserve content while learning the style of the target modality. Experiments demonstrate that InStyleGAN outperforms existing variants in harmonizing PET data across centers. To our knowledge, this is the first dedicated framework for multicenter PET harmonization in PCNSL.

High-dimensional medical data presents unique challenges for multimodal systems, exhibiting intricate spatial and temporal patterns that resist conventional modeling approaches. Traditional approaches typically process these modalities that create significant barriers to clinical deployment and cross-modal knowledge transfer. Motivated by recent successes in in-context learning (ICL) for biomedical applications, we investigate whether large language models (LLMs) can perform medical prediction tasks through structured graph-based prompts in a fully training-free manner. We propose a unified framework that introduces a novel dual-level representation approach: mathematical dimensionality reduction techniques such as PCA construct text-level inputs, while a training-free Graph Transformer generates embedding-level representations from multi-modal graph structural information. Through experiment, we establish that the combination of Universal Graph Transformer with additional text-level representations achieves optimal performance with comparable performance to the pretrained method, our training-free paradigm eliminates the need for complex cross-modal alignment modules and extensive parameter optimization, demonstrating the feasibility of leveraging LLMs for medical graph analysis without task-specific training.

Accurate preoperative prediction of erectile dysfunction (ED) is important for counseling patients undergoing radical prostatectomy. While clinical features are well-established predictors, the added value of preoperative MRI remains underexplored. We assess whether MRI provides additional predictive value for ED at 12 months post-surgery, evaluating four strategies: (1) a clinical-only baseline; (2) classical models using handcrafted MRI-based features; (3) deep learning models trained on MRI slices; and (4) fusion of imaging and clinical inputs. Imaging-based models (AUC 0.569) slightly outperformed handcrafted approaches (AUC 0.554) but fell short of the clinical baseline (AUC 0.663). Fusion models offered marginal gains (AUC 0.586) but did not surpass clinical-only performance. SHAP analysis on the fusion method confirmed that clinical features were the primary drivers of predictive accuracy, while saliency maps suggested that imaging models focused on anatomically relevant regions. However, MRI offered limited added value beyond clinical data in prediction, likely due to dataset size, imaging-related variability, and missing clinical confounders. Larger, standardized datasets and improved integration strategies are needed to better evaluate MRI’s potential in ED prediction.

Cine cardiac magnetic resonance (cine-CMR) imaging is the gold-standard modality for assessing myocardial structure and function, including Left Ventricular Ejection Fraction (LVEF) measurement. However, cine-CMR presents challenges due to its high-dimensional 3D + time nature. To address this, we propose cine-CLIP, which employs standard deviation (std) mapping across time to reduce 4D data to 3D while preserving dynamic and spatial information. Through extensive experiments on publicly available UK Biobank and ACDC datasets, our method, cine-CLIP, achieves state-of-the-art (SOTA) LVEF prediction with a mean absolute error (MAE) of 2.523, outperforming other techniques. To assess generalizability, we further validate our method on the external Kaggle Data Science Bowl dataset, which follows a slightly different CMR acquisition protocol. Despite domain shifts, we achieved an MAE of 5.091, surpassing prior methods. These findings highlight Cine-CLIP’s ability to capture the high-dimensional complexity of cardiovascular disease from cine-CMR data. LVEF prediction serves as a proof of concept, demonstrating the model’s effectiveness in this task. However, this framework has the potential to be extended to other clinical metrics. The code is available at https://​github.​com/​enriquealmar9/​Cine-CLIP.

Deep generative models have demonstrated remarkable success in medical image synthesis. However, ensuring conditioning faithfulness and high-quality synthetic images for direct or counterfactual generation remains a challenge. In this work, we introduce a cycle training framework to fine-tune diffusion models for improved conditioning adherence and enhanced synthetic image realism. Our approach, Cycle Diffusion Model (CDM), enforces consistency between generated and original images by incorporating cycle constraints, enabling more reliable direct and counterfactual generation. Experiments on a combined 3D brain MRI dataset (from ABCD, HCP aging & young adults, ADNI, and PPMI) show that our method improves conditioning accuracy and enhances image quality as measured by FID and SSIM. The results suggest that the cycle strategy used in CDM can be an effective method for refining diffusion-based medical image generation, with applications in data augmentation, counterfactual, and disease progression modeling.

Kinetic modeling enables in vivo quantification of tracer uptake and glucose metabolism in [\(^{18}\)F]Fluorodeoxyglucose (\(\mathrm {[^{18}F]FDG}\)) dynamic positron emission tomography (dPET) imaging of mice. However, kinetic modeling requires the accurate determination of the arterial input function (AIF) during imaging, which is time-consuming and invasive. Recent studies have shown the efficacy of using deep learning to directly predict the input function, surpassing established methods such as the image-derived input function (IDIF). In this work, we trained a physics-informed deep learning-based input function prediction model (PIDLIF) to estimate the AIF directly from the PET images, incorporating a kinetic modeling loss during training. The proposed method uses a two-tissue compartment model over two regions, the myocardium and brain of the mice, and is trained on a dataset of 70 \(\mathrm {[^{18}F]FDG}\) dPET images of mice accompanied by the measured AIF during imaging. The proposed method had comparable performance to the network without a physics-informed loss, and when sudden movement causing blurring in the images was simulated, the PIDLIF model maintained high performance in severe cases of image degradation. The proposed physics-informed method exhibits an improved robustness that is promoted by physically constraining the problem, enforcing consistency for out-of-distribution samples. In conclusion, the PIDLIF model offers insight into the effects of leveraging physiological distribution mechanics in mice to guide a deep learning-based AIF prediction network in images with severe degradation as a result of blurring due to movement during imaging.

Percutaneous coronary intervention (PCI) is a standard treatment for significant coronary artery disease (CAD); however, predicting the need for repeat PCI remains clinically challenging. Traditional survival models, such as the Cox proportional hazards model, rely on static baseline features and fail to capture the dynamic nature of patient trajectories following initial PCI. In this study, we propose DA-RNN-Surv, a novel interpretable survival analysis framework leveraging dual attention recurrent neural networks to incorporate longitudinal electronic health record (EHR) data. Using a real-world cohort of 6,252 PCI patients collected over a 10-year period, DA-RNN-Surv achieved significantly higher predictive performance, with concordance indices ranging from 0.722 at month 3 to 0.894 at month 24, outperforming traditional Cox (0.525 to 0.635) and DeepSurv (0.506 to 0.616) models. Importantly, DA-RNN-Surv provides interpretability through visit-level and feature-level attention weights, identifying key clinical timepoints (e.g., baseline and follow-up months) and influential variables such as medication adherence and comorbidities. These findings demonstrate the clinical value of explicitly modeling temporal information, highlighting DA-RNN-Surv’s potential for enhancing personalized patient management and preventive intervention strategies post-PCI.

One goal of disease progression modeling is to infer biomarker trajectories in patients as the disease unfolds. Mapping the disease trajectory is critical in early diagnosis, patient staging, and clinical trial design. However, accurately capturing biomarker trajectories over time is challenging due to limited longitudinal data and substantial inter-individual variability. Discrete disease progression models were developed to overcome these challenges by using cross-sectional or short-term longitudinal data to map disease progression. These models often view disease progression as a latent permutation of events, using maximum likelihood estimation to predict event sequence. Using these models can be restrictive due to their combinatorial growth in runtime and vulnerability to noise, making them infeasible to use on high-dimensional or noisy data. Here, we introduce FastEBM which leverages manifold learning and Markov chains to model disease progression as a subject-level ordering along the disease continuum, enabling us to achieve fast and noise-robust event sequence estimation. Using this approach we were able to achieve 3,500 times faster inference compared to the state-of-the-art models while maintaining high accuracy. Using simulated data, FastEBM outperforms other existing EBM models by up to 80% in terms of accuracy in high-dimensional noisy settings. Furthermore, when applied to real-world data from the Alzheimer’s Disease Neuroimaging Initiative, only FastEBM revealed a sequence of disease progression that aligns with established clinical understanding and literature: increased amyloid-\(\beta \) levels preceded tau accumulation, which were followed by structural brain atrophy and subsequently, cognitive decline. Overall, FastEBM provides an efficient, interpretable, and accurate way to model disease progression that can be applied to data from different modalities with potential for clinical application.

Liver cancer remains a major global health challenge, and hepatocellular carcinoma (HCC) accounts for approximately 80% of all primary malignant liver tumors. Among various risk factors for HCC, chronic hepatitis B (CHB) plays a particularly significant role in Asian populations, where infected individuals face a substantially higher risk than the general population. This study focuses on a Korean cohort of patients with CHB and aims to develop precise risk prediction models for HCC. Traditional clinical risk scoring systems are widely used to estimate HCC risk, but these methods have inherent limitations due to their linear structure and limited capacity to capture complex patient characteristics. To address these limitations, recent advances in predictive models and artificial intelligence have enabled more accurate and personalized disease risk predictions. This study applies a transformer-based predictive model, specifically ExcelFormer, to estimate the 5-year risk of HCC development in patients with CHB in this cohort. ExcelFormer incorporates various clinical variables and captures nonlinear interactions among features, which leads to improved predictive performance. The model achieves an AUROC of 0.8524, and its performance further improves to 0.8839 when additional laboratory variables are included. These results demonstrate clear advantages over conventional scoring systems in predictive accuracy and adaptability.

Drug recommendation is an essential task in machine learning-based clinical decision support systems. However, the risk of drug-drug interactions (DDI) between co-prescribed medications remains a significant challenge. Previous studies have used graph neural networks (GNNs) to represent drug structures. Regardless, their simplified discrete forms cannot fully capture the molecular binding affinity and reactivity. Therefore, we propose Multimodal DDI Prediction with Molecular Electron Localization Function (ELF) Maps (MMM), a novel framework that integrates three-dimensional (3D) quantum-chemical information into drug representation learning. It generates 3D electron density maps using the ELF. To capture both therapeutic relevance and interaction risks, MMM combines ELF-derived features that encode global electronic properties with a bipartite graph encoder that models local substructure interactions. This design enables learning complementary characteristics of drug molecules. We evaluate MMM in the MIMIC-III dataset (250 drugs, 442 substructures), comparing it with several baseline models. In particular, a comparison with the GNN-based SafeDrug model demonstrates statistically significant improvements in the F1-score (p = 0.0387), Jaccard (p = 0.0112), and the DDI rate (p = 0.0386). These results demonstrate the potential of ELF-based 3D representations to enhance prediction accuracy and support safer combinatorial drug prescribing in clinical practice.