Cancer Prevention, Detection, and Intervention

Malignant ovarian tumors (OTs) are a leading cause of gynecological cancer deaths, and often remain asymptomatic until advanced stages, making early and accurate diagnosis crucial for effective treatment and good patient outcome. Current diagnostic methods often fall short due to the heterogeneous nature of OTs and the complexities in distinguishing benign from malignant forms. To overcome these limitations, this study proposes a novel framework leveraging transformer-based multiple-instance learning (MIL) and hierarchical self-supervised pre-training. To validate the model, a comprehensive multi-center dataset has been compiled, encompassing diverse patient demographics and imaging protocols. Benchmarking against conventional radiomics methods and other deep learning approaches, the hierarchical MIL model demonstrates superior performance with a median AUROC of 0.84 and high recall of 0.91. These results highlight significant improvements in sensitivity, essential for minimizing false negatives in clinical settings. The performed study emphasizes the importance of multi-center validation and external dataset testing to ensure generalization of the proposed model and obtain a higher robustness. The encountered complexity of multi-center data is found significant, since various clinical factors play an influential role. This makes baseline comparisons virtually impossible and the need for more multi-center research increasingly compelling and encouraging.

Glioblastoma (GBM), primary central nervous system lymphoma (PCNSL), and brain metastases (BM) are three common malignant central nervous system tumors. Accurate preoperative differentiation is essential for appropriate treatment planning and prognosis, however, it’s challenging to differentiate these tumors using MRI due to their similar anatomical structures and imaging characteristics. In this paper, we first construct a new multi-center brain MRI dataset, including 315 training cases (GBM 64, PCNSL 59, BM 192) and 124 external test cases (24:23:77). Moreover, we propose a novel framework FoTNet for accurate diagnosis of the three tumors. Our model achieves a classification accuracy of 92.5% and an average AUC of 0.9754, outperforming previous methods. Our results demonstrates the great potential of AI in assisting physicians in differentiating between GBM, PCNSL, and BM, particularly in resource-limited clinical settings.

Gastrointestinal cancer is the leading cause of cancer-related incidence and death. Therefore, it is important to develop a novel computer-aided diagnosis system for early detection and enhanced treatment. Traditional approaches rely on the expertise of gastroenterologists to identify diseases. However, it is a subjective process, and the interpretation can vary even between expert clinicians. Considering recent progress in classifying gastrointestinal anomalies and landmarks in endoscopic and video capsule endoscopy images, this study proposes a hybrid model incorporating the advantages of Transformers and Convolutional Neural Networks (CNNs) for enhanced classification performance. Our model utilizes DenseNet201 as a CNN branch to extract local features and integrates the Swin Transformer branch for global feature understanding. Both of their features are combined to perform the classification task. For the GastroVision dataset, our proposed model demonstrates excellent performance with Precision, Recall, F1 score, Accuracy, and Matthews Correlation Coefficient (MCC) of 0.8320, 0.8386, 0.8324, 0.8386, and 0.8191, respectively, showcasing its robustness against class imbalance dataset and surpassing other CNNs as well as Swin Transformer model. Similarly, for the Kvasir-Capsule, a large video capsule endoscopy dataset, our model surpassed all other models, thereby achieving overall Precision, Recall, F1 score, Accuracy, and MCC of 0.7007, 0.7239, 0.6900, 0.7239, and 0.3871. Moreover, we generated saliency maps to explain our model’s focus areas, showing its reliable decision-making process. The results underscore the potential of our hybrid CNN-Transformer model in aiding the early and accurate detection of gastrointestinal (GI) anomalies.

Recent studies demonstrate promising efficacy with immune checkpoint inhibitors (ICI) for brain metastases (BM), an unmet need in modern oncology. However, a predictive biomarker for ICI efficacy is needed to inform precision-based use of ICI given its high toxicity rate. Here, we present several multimodal deep learning (DL) approaches that integrate pre-treatment magnetic resonance imaging (MRI) and clinical metadata to predict ICI efficacy for BM. Using a multi-institutional dataset of 548 patients, our best-performing models achieve an AUROC of 0.674 (±0.041). In future work, we will accrue additional clinical and radiologic data to improve performance. Furthermore, our work thus far will serve as a baseline by which to trial alternate fusion strategies to improve and refine multimodal biomarker discovery for precision oncology.

While deep learning excels in many areas, its application in medicine is hindered by limited data, which restricts model generalizability. Few-shot learning has emerged as a potential solution to this problem. In this work, we leverage the strengths of meta-learning, the primary framework for few-shot learning, along with diffusion-based generative models to enhance few-shot learning capabilities. We propose a novel method that jointly trains a diffusion model and a feature extractor in an episodic-based manner. The diffusion model learns conditional generation based on each episode’s support samples. After updating its parameters, it generates additional support samples for each class. The augmented support set is used to train a feature extractor within a prototypical meta-learning framework. Notably, we propose a weighted prototype computation based on the distance between each generated sample and the original class prototype, i.e., derived solely from the original support samples. Evaluations on two tumor characterization tasks (prostate cancer aggressiveness and breast cancer malignity assessment) demonstrate our approach’s effectiveness in improving prototype representation and boosting classification performance. Find our code at: https://​github.​com/​evapachetti/​meta_​diffusion.

Rising breast cancer (BC) occurrence and mortality are major global concerns for women. Deep learning (DL) has demonstrated superior diagnostic performance in BC classification compared to human expert readers. However, the predominant use of unimodal (digital mammography) features may limit the current performance of diagnostic models. To address this, we collected a novel multimodal dataset comprising both imaging and textual data. This study proposes a multimodal DL architecture for BC classification, utilizing images (mammograms; four views) and textual data (radiological reports) from our new in-house dataset. Various augmentation techniques were applied to enhance the training data size for both imaging and textual data. We explored the performance of eleven SOTA DL architectures (VGG16, VGG19, ResNet34, ResNet50, MobileNet-v3, EffNet-b0, EffNet-b1, EffNet-b2, EffNet-b3, EffNet-b7, and Vision Transformer (ViT)) as imaging feature extractors. For textual feature extraction, we utilized either artificial neural networks (ANNs) or long short-term memory (LSTM) networks. The combined imaging and textual features were then inputted into an ANN classifier for BC classification, using the late fusion technique. We evaluated different feature extractor and classifier arrangements. The VGG19 and ANN combinations achieved the highest accuracy of 0.951. For precision, the VGG19 and ANN combination again surpassed other CNN and LSTM, ANN based architectures by achieving a score of 0.95. The best sensitivity score of 0.903 was achieved by the VGG16+LSTM. The highest F1 score of 0.931 was achieved by VGG19+LSTM. Only the VGG16+LSTM achieved the best area under the curve (AUC) of 0.937, with VGG16+LSTM closely following with a 0.929 AUC score.

Recognition of early neoplasia in Barrett’s Esophagus (BE) is challenging, despite advances in endoscopic technology. Even with correct identification, the subtle nature of lesions leads to significant inter-observer variability in placing targeted biopsy markers and delineation of lesions. Computer-Aided Detection (CADe) systems may assist endoscopists, however, compliance of endoscopists with CADe is often suboptimal, reducing joint performance below CADe stand-alone performance. Improved localization performance of CADe could enhance compliance. These systems often use fused consensus ground-truths (GT), which may not capture subtle neoplasia gradations, affecting classification and localization. This study evaluates five consensus GT strategies from multi-expert segmentation labels and four loss functions for their impact on classification and localization performance. The dataset includes 7,995 non-dysplastic BE images (1,256 patients) and 2,947 neoplastic images (823 patients), with each neoplastic image annotated by two experts. Classification, localization for true positives, and combined detection performance are assessed and compared with 14 independent Barrett’s experts. Results show that using multiple consensus GT masks with a compound Binary Cross-Entropy and Dice loss achieves the best classification sensitivity and near-expert level localization, making it the most effective training strategy. The code is made publicly available at: https://​github.​com/​BONS-AI-VCA-AMC/​BE-CADe-GT.

Hepatocellular carcinoma (HCC) is a common type of liver cancer. Its effective diagnosis and monitoring require analyzing computed tomography (CT) scans with intravenous contrast in multiple phases, taken at different intervals post-injection. Organ movement during these intervals, caused by factors like breathing, heartbeat, or patient motion, can affect the accuracy of HCC detection. Aligning two or more scans precisely, especially ensuring the liver’s alignment, is crucial for reconstructing small lesions effectively. Additionally, the presence of various liver lesions, such as active HCC tumors, chemoembolizations, necrosis, portal vein thrombosis, cysts, or other lesions, complicates the diagnosis process. In this paper, we tackle these challenges and propose a deep learning pipeline for detecting, segmenting and ultimately quantifying HCC in multi-phase CT scans. Our rigorous experiments, conducted on a carefully curated dataset from a clinical trial involving HCC patients, demonstrate that our approach not only achieves high-quality detection and segmentation of HCC but also enables fully-automatic, objective, reproducible and accurate response assessment in HCC patients.

Liver fibrosis is a chronic disease that must be treated to prevent further complications, including liver cancer. The diagnosis of liver fibrosis in CT imaging can be challenging and is often subject to disagreements between radiologists. The nodularity of the liver surface is a well-known feature of fibrosis, which can be quantified in clinical practice with specialized software applications that rely on semi-automatic delineation of the liver contours. This approach, however, requires a high degree of expertise and is time-consuming. While deep learning methods have recently shown excellent performance for liver segmentation, the predicted contours are typically insufficiently accurate for nodularity quantification. In this work, we propose a local thresholding approach to refine the predictions of a deep network trained to segment the liver in CT images. We show that our refinement method improves the estimation of the liver surface nodularity compared to a baseline deep network, with Spearman’s correlation coefficients of 0.60 and 0.47, respectively. This new estimator predicts advanced fibrosis better than the reference clinical approach, with areas under the curve of 74.6% and 67.9%, respectively.

Early and precise detection of prostate cancer using Magnetic Resonance Imaging (MRI) remains a significant challenge in medical research. Despite the promising potential of Deep Neural Networks (DNNs) for prostate cancer screening, ensuring their reliability is crucial. Accurately quantifying prediction uncertainty in diagnoses is imperative in clinical settings. In this study, we introduce a deep learning model designed not only to detect prostate cancer but also to quantify prediction uncertainty, thus distinguishing between confident and uncertain predictions. Our approach uses a 3D DenseNet-121 backbone for feature extraction and Monte Carlo Dropout (MCD) to approximate Bayesian inference, allowing us to estimate the uncertainty in the model’s predictions. We evaluated the model on data from 157 patients, analyzing its reliability and performing an ablation study across different MRI sequences. The model achieved an Area Under the Curve (AUC) of 0.79 across all MRI sequences. In the optimal setup, it classified \(75\%\) of predictions as certain and \(25\%\) as uncertain, with an AUC of 0.9 for certain predictions. These results clearly demonstrate the model’s efficacy in accurately quantifying the reliability of its classifications. By automatically identifying uncertain cases, our approach enables radiologists to focus their attention on these, potentially reducing their workload while enhancing diagnostic accuracy.

Colon cancer is among the leading causes of cancer-related death worldwide for both men and women, with colorectal polyps serving as a significant predisposing factor. Early polyp identification and removal-the precursors to colorectal cancer-is essential to its prevention. Colonoscopy is considered the gold standard for colorectal cancer screening because it allows for the immediate removal of polyps, preventing them from developing into cancer. Despite its effectiveness, conventional colonoscopy is time-consuming, highly labor-intensive, and prone to human mistakes. Therefore, we modified the efficient object detection model, YOLO-V8, to develop our novel approach, EDF-YOLO8, for automating polyp identification. Our model employs deformable convolution in the bottleneck as a robust solution for effectively detecting polyps of various sizes. We enhance the effectiveness of our model by incorporating the Exponential Linear Unit (ELU), which further increases the detection accuracy and tends to accelerate the model learning process. We trained and tested the suggested model on two distinct datasets from publicly accessible sources and conducted thorough assessments to ensure its robustness and generalizability. The proposed model achieved an outstanding performance, attaining a mAP50 score of 0.931 and 0.894 for the Kvasir and Polypgen datasets, respectively. Performance analysis demonstrates the efficiency and robustness of our model in accurately detecting polyps from colonoscopic frames from different datasets.

Laryngeal cancer (LC) and other benign conditions are major concerns in modern ear, nose, and throat medicine. A comprehensive evaluation of the larynx should employ flexible or rigid endoscopes to identify early-stage lesions, possibly enhanced with advanced imaging techniques such as Narrow Band Imaging (NBI) to empower tissue visualization. Factors that make the detection, diagnosis, and treatment of LC challenging include the huge amount of uninformative frames and the expertise-dependent nature of the assessment, leading to time-consuming procedures with high cognitive loads and the possibility of missed detections and misdiagnoses, especially for less-experienced clinicians. Deep Learning (DL) approaches have recently been studied regarding frame quality assessment, abnormal mass identification, and their margins definition to improve diagnostic accuracy and surgical outcomes. In this work, we proposed the integration of several Convolutional Neural Networks (CNNs) into a single computer-aided system for the assistance of less-experienced otolaryngologists, by directing their attention toward good-quality frames from which lesions can be automatically detected and characterized. We addressed the following challenging tasks: informative frame selection, lesion detection, lesion classification, and lesion segmentation. The developed system demonstrated a good trade-off between efficacy metrics and real-time performance, and the potential for clinical applications.

In breast ultrasound imaging, the scarcity of detailed annotations poses a major barrier to developing robust object detection models. This challenge is compounded by the high intra-class variability, where different slices of a 3D object can appear drastically different in 2D images, and low inter-class variance, where pathological features are often small and subtle compared to the rest of the image. These factors make it difficult to train models that require precise bounding box annotations or extensive labeled datasets. Addressing these issues, this study introduces a novel weakly supervised object detection (WSOD) model that capitalizes on image-level labels, which are more readily available and require significantly less effort from medical professionals. Our approach integrates Multi-Instance Learning (MIL) and Self-Supervised Learning (SSL) within a Deformable DETR framework, aiming to focus the model’s attention on relevant regions without detailed annotations. Tested on the two publicly available datasets, our model demonstrates significant improvements in mean average precision (mAP) and recall, surpassing existing state-of-the-art methods. Ablation studies confirm the essential importance of MIL and SSL in enhancing detection accuracy, validating our model as a potent solution for overcoming data scarcity in medical imaging.

When developing Computer Aided Detection (CAD) systems for Digital Breast Tomosynthesis (DBT), the complexity arising from the volumetric nature of the modality poses significant technical challenges for obtaining large-scale accurate annotations. Without access to large-scale annotations, the resulting model may not generalize to different domains. Given the costly nature of obtaining DBT annotations, how to effectively increase the amount of data used for training DBT CAD systems remains an open challenge.

In this paper, we present SelectiveKD, a semi-supervised learning framework for building cancer detection models for DBT, which only requires a limited number of annotated slices to reach high performance. We achieve this by utilizing unlabeled slices available in a DBT stack through a knowledge distillation framework in which the teacher model provides a supervisory signal to the student model for all slices in the DBT volume. Our framework mitigates the potential noise in the supervisory signal from a sub-optimal teacher by implementing a selective dataset expansion strategy using pseudo labels.

We evaluate our approach with a large-scale real-world dataset of over 10,000 DBT exams collected from multiple device manufacturers and locations. The resulting SelectiveKD process effectively utilizes unannotated slices from a DBT stack, leading to significantly improved cancer classification performance (AUC) and generalization performance.

Kidney cancer is a global health concern, and accurate assessment of patient frailty is crucial for optimizing surgical outcomes. This paper introduces AI Age Discrepancy, a novel metric derived from machine learning analysis of preoperative abdominal CT scans, as a potential indicator of frailty and postoperative risk in kidney cancer patients. This retrospective study of 599 patients from the 2023 Kidney Tumor Segmentation (KiTS) challenge dataset found that a higher AI Age Discrepancy is significantly associated with longer hospital stays and lower overall survival rates, independent of established factors. This suggests that AI Age Discrepancy may provide valuable insights into patient frailty and could thus inform clinical decision-making in kidney cancer treatment.

Esophageal cancer is a major cause of cancer-related mortality internationally, with high recurrence rates and poor survival even among patients treated with curative-intent surgery. Investigating relevant prognostic factors and predicting prognosis can enhance post-operative clinical decision-making and potentially improve patients’ outcomes. In this work, we assessed prognostic factor identification and discriminative performances of three models for Disease-Free Survival (DFS) and Overall Survival (OS) using a large multicenter international dataset from ENSURE study. We first employed Cox Proportional Hazards (CoxPH) model to assess the impact of each feature on outcomes. Subsequently, we utilised CoxPH and two deep neural network (DNN)-based models, DeepSurv and DeepHit, to predict DFS and OS. The significant prognostic factors identified by our models were consistent with clinical literature, with post-operative pathologic features showing higher significance than clinical stage features. DeepSurv and DeepHit demonstrated comparable discriminative accuracy to CoxPH, with DeepSurv slightly outperforming in both DFS and OS prediction tasks, achieving C-index of 0.735 and 0.74, respectively. While these results suggested the potential of DNNs as prognostic tools for improving predictive accuracy and providing personalised guidance with respect to risk stratification, CoxPH still remains an adequately good prediction model, with the data used in this study.

Magnetic resonance guided focused ultrasound (MRgFUS) is one of the most attractive emerging minimally invasive procedures for breast cancer, which induces localized hyperthermia, resulting in tumor cell death. Accurately assessing the post-ablation viability of all treated tumor tissue and surrounding margins immediately after MRgFUS thermal therapy residual tumor tissue is essential for evaluating treatment efficacy. While both thermal and vascular MRI-derived biomarkers are currently used to assess treatment efficacy, currently, no adequately accurate methods exist for the in vivo determination of tissue viability during treatment. The non-perfused volume (NPV) acquired three or more days following MRgFUS thermal ablation treatment is most correlated with the gold standard of histology. However, its delayed timing impedes real-time guidance for the treating clinician during the procedure. We present a robust deep-learning framework that leverages multiparametric MR imaging acquired during treatment to predict treatment efficacy. The network uses qualtitative T1, T2 weighted images and MR temperature image derived metrics to predict the three day post-ablation NPV. To validate the proposed approach, an ablation study was conducted on a dataset (N=6) of VX2 tumor model rabbits that had undergone MRgFUS ablation. Using a deep learning framework, we evaluated which of the acquired MRI inputs were most predictive of treatment efficacy as compared to the expert radiologist annotated 3 day post-treatment images.

Survival analysis is paramount for cancer patients as it offers crucial prognostic insights for treatment planning. The performance of existing survival analysis methods is mainly limited by two factors: 1) inefficient extraction of features from multi-modal medical data, e.g., MR images and clinical diagnostic descriptions; and 2) inadequate focus on disease-relevant regions, e.g., primary tumor. To deal with these challenges, in this study, we propose a rectal cancer survival analysis model, dubbed as SurRecNet, which effectively fuse MR images and diagnostic descriptions and takes advantage of multi-task learning. Specifically, we introduce a cross-modality alignment module, aiming to precisely align diagnostic descriptions with MR images at a granular level and facilitate accurate survival analysis. Furthermore, SurRecNet simultaneously predicts tumor masks, relapse states, and survival outcomes by leveraging multi-task learning strategy, imitating the diagnostic process of radiologists to enhance prediction performance. Experimental results on a real clinical rectal multi-modal dataset demonstrate that our SurRecNet significantly outperforms the state-of-the-art methods.

Prediction of biochemical recurrence (BCR) after prostate cancer radiotherapy is crucial for devising personalised treatments. BCR has been traditionally predicted using clinical data or in vivo imaging within AI frameworks such as radiomics approaches, but with limited results and reduced interpretability. These analysis are additionally hindered by the imbalanced and heterogeneous nature of data. In this paper, we present a novel approach to predict BCR at 5 years, based not only on clinical and image features, but also on a patient specific radiobiological mechanistic in silico model simulating tumour growth and radiation response. By combining all these data, we aim at i) improving the prediction of BCR after prostate cancer radiotherapy (RT), and ii) bringing interpretability to this prediction. A cohort of 254 patients was used. Pre-treatment T2-w MRIs, ADC maps and 7 clinicopathological characteristics were available. Patient specific digital twins of tumours were created from MRIs. The prescribed treatment was simulated with the mechanistic model yielding 414 features characterising the response of the tumour to RT. A first univariate feature selection analysis was conducted to select the most predictive features. Then, a machine learning algorithm was trained using selected features and compared with a deep learning (DL) approach based on clinicopathological characteristics and MRIs. Our approach achieved an AUC of 0.74 by training a random forest classifier combining most predictive features. The DL model achieved an AUC of 0.69. This methodology opens the road to interpretability of the response to radiotherapy and tailored treatments for prostate cancer patients.

AutoDoseRank (Automated Dosimetry-informed Segmentation Ranking) is a novel methodology for ranking segmentations of glioblastoma, a highly aggressive brain tumor, by dosimetric quality. AutoDoseRank uses a deep learning-based dose predictor along with a dosimetric ranking scheme capable of sorting a set of candidate segmentations by quality. With the advent of auto-segmentation for radiotherapy, we expect that radiation oncologists will spend more time triaging and evaluating the quality of generated segmentation proposals rather than manually drawing them. It is known that changes in segmentation evaluated purely through a geometric lens like Dice, do not correlate with eventual clinical outcomes. Our approach therefore aims to incorporate organ-specific dosimetric constraints used in clinical radiotherapy planning into a patient-level ranking. The effectiveness of AutoDoseRank is measured by comparing its ability to rank segmentations against that of four expert radiation oncologists. We show that AutoDoseRank is better than three out of four experts while being only slightly outperformed by the most experienced and meticulous one. These results highlight AutoDoseRank’s capability to monitor the quality of auto-segmentation dosimetrically, something that is ever increasing in importance in the radiation oncology workflow. Code to reproduce this analysis is available at https://​github.​com/​ubern-mia/​autodoserank.​

Survival analysis plays a crucial role in estimating the likelihood of future events for patients by modeling time-to-event data, particularly in healthcare settings where predictions about outcomes such as death and disease recurrence are essential. However, this analysis poses challenges due to the presence of censored data, where time-to-event information is missing for certain data points. Yet, censored data can offer valuable insights, provided we appropriately incorporate the censoring time during modeling. In this paper, we propose SurvCORN, a novel method utilizing conditional ordinal ranking networks to predict survival curves directly. Additionally, we introduce SurvMAE, a metric designed to evaluate the accuracy of model predictions in estimating time-to-event outcomes. Through empirical evaluation on two real-world cancer datasets, we demonstrate SurvCORN’s ability to maintain accurate ordering between patient outcomes while improving individual time-to-event predictions. Our contributions extend recent advancements in ordinal regression to survival analysis, offering valuable insights into accurate prognosis in healthcare settings. Our code is available at https://​github.​com/​BioMedIA-MBZUAI/​SurvCORN.