Uncertainty for Safe Utilization of Machine Learning in Medical Imaging

Scribbles are a popular form of weak annotation for the segmentation of three-dimensional medical images, but typically require iterative refinement to achieve the desired segmentation map. The complexity of diffusion MRI (dMRI) poses additional challenges. Previous work addressed the high dimensionality of dMRI via unsupervised representation learning, and combined it with a random forest classifier that can be re-trained quickly enough to provide interactive feedback to the human annotator. Our work extends that framework in multiple ways. Our main contribution is to add an active learning component that suggests locations in which additional scribbles should be placed. It relies on uncertainty quantification via test time augmentation (TTA). Second, we observe that TTA increases segmentation accuracy even by itself. Moreover, we demonstrate that anomaly detection via isolation forests effectively suppresses false positives that arise when generalizing from sparse scribbles. Taken together, these contributions substantially improve the accuracy that can be achieved with various annotation budgets.

Detecting genetic aberrations is crucial in cancer diagnosis, typically through fluorescence in situ hybridization (FISH). However, existing FISH image classification methods face challenges due to signal variability, the need for costly manual annotations and fail to adequately address the intrinsic uncertainty. We introduce a novel approach that leverages synthetic images to eliminate the requirement for manual annotations and utilizes a joint contrastive and classification objective for training to account for inter-class variation effectively. We demonstrate the superior generalization capabilities and uncertainty calibration of our method, which is trained on synthetic data, by testing it on a manually annotated dataset of real-world FISH images. Our model offers superior calibration in terms of classification accuracy and uncertainty quantification with a classification accuracy of 96.7% among the 50% most certain cases. The presented end-to-end method reduces the demands on personnel and time and improves the diagnostic workflow due to its accuracy and adaptability. All code and data is publicly accessible at: https://​github.​com/​SimonBon/​FISHing.

Automated generation of radiology reports from X-ray images serves as a crucial task to streamline the diagnostic workflow for medical imaging and enhance the efficiency of radiologist decision-making. For clinical accuracy, most existing approaches focus on achieving accurate predictions of the existence of abnormalities, despite the inherent uncertainty impacting the reliability of the generated report, which is often clarified by radiologists simultaneously. In this paper, we present a unified report generation framework featuring a novel diagnostic uncertainty estimation model, named Diagnostic Uncertainty Encoding framework (DiagUE). Inspired by the clinician’s uncertainty-aware radiology decision-making behavior, DiagUE first formulates belief-based diagnostic uncertainty metrics that effectively capture the variability of radiology abnormalities. Then, the estimated uncertainty-aware abnormality prediction is integrated with a report generation model under a novel visual-language encoding mechanism. Extensive experiments on two public benchmark datasets demonstrate that DiagUE could outperform SOTA baselines in ensuring the clinical accuracy of both abnormality description and diagnostic uncertainty of the report generation.

Deep Learning (DL), which involves powerful “black box” predictors, has achieved state-of-the-art performance in medical image analysis. However, these methods lack transparency and interpretability of point predictions without assessing the quality of their outputs. Knowing how much confidence there is in a prediction is essential for gaining clinicians’ trust in the technology and its use in medical decision-making. In this paper, we explore the use of Conformal Prediction (CP) methods to recommend statistically rigorous reliable prediction sets to a clinician, using multi-modal imaging for the genetic diagnosis of the 36 most common molecular causes of inherited retinal diseases (IRDs). These are monogenic conditions that represent a leading cause of blindness in children and working-age adults and require a costly and time-consuming genetic test for diagnosis. Three methods of CP were assessed: Least Ambiguous Adaptive Prediction Sets (LAPS), Adaptative Prediction Sets (APS), and Regularized Adaptive Prediction Sets (RAPS). Our IRD classifier (Eye2Gene), in combination with the three conformal predictors, was evaluated on an internal holdout subset and datasets from four external clinical centres. RAPS proved to be the best-performing method with single-digit set sizes and coverage above 90% at a confidence level of 80%. Implementing adaptive CP methods has the potential to reduce waiting time and costs of genetic diagnosis of IRDs by improving upon the current gene prioritisation systems, while simultaneously enabling safety-critical clinical environments by flagging clinicians for a second opinion.

In the realm of robotic-assisted surgeries, like laparoscopic cholecystectomy, the integration of deep learning (DL) models marks a significant advancement in achieving surgical precision and minimal invasiveness, which in turn, elevates patient outcomes and reduces recovery times. However, the vulnerability of these DL models to adversarial attacks introduces a critical risk, emphasizing the need for enhanced model robustness. Our study addresses this challenge by proposing a comprehensive framework that not only fortifies surgical action recognition models against adversarial threats through adversarial training and pre-processing strategies but also incorporates uncertainty estimation to enhance prediction confidence and trustworthiness. Our framework demonstrates superior resilience against a wide spectrum of adversarial attacks and showcases improved reliability in surgical tool detection under adversarial conditions. It achieves an improvement from 8% to 23.58% in terms of triplet (instrument, verb, triplet) predictions. These contributions significantly enhance the security and reliability of deep learning applications in the critical domain of robotic surgery, offering an approach that safeguards advanced surgical technologies against malicious threats, thereby promising enhanced patient care and surgical precision. Code is available at https://​github.​com/​umair1221/​guardian.

Automation of medical image interpretation could alleviate bottlenecks in diagnostic workflows, and has become of particular interest in recent years due to advancements in natural language processing. Great strides have been made towards automated radiology report generation via AI, yet ensuring clinical accuracy in generated reports is a significant challenge, hindering deployment of such methods in clinical practice. In this work we propose a quality control framework for assessing the reliability of AI-generated radiology reports with respect to semantics of diagnostic importance using modular auxiliary auditing components (ACs). Evaluating our pipeline on the MIMIC-CXR dataset, our findings show that incorporating ACs in the form of disease-classifiers can enable auditing that identifies more reliable reports, resulting in higher F1 scores compared to unfiltered generated reports. Additionally, leveraging the confidence of the AC labels further improves the audit’s effectiveness. Code will be made available at: https://​github.​com/​hermionewarr/​GenX_​Report_​Audit.

Recent works have introduced methods to estimate segmentation performance without ground truth, relying solely on neural network softmax outputs. These techniques hold potential for intuitive output quality control. However, such performance estimates rely on calibrated softmax outputs, which is often not the case in modern neural networks. Moreover, the estimates do not take into account inherent uncertainty in segmentation tasks. These limitations may render precise performance predictions unattainable, restricting the practical applicability of performance estimation methods. To address these challenges, we develop a novel approach for predicting performance ranges with statistical guarantees of containing the ground truth with a user specified probability. Our method leverages sampling-based segmentation uncertainty estimation to derive heuristic performance ranges, and applies split conformal prediction to transform these estimates into rigorous prediction ranges that meet the desired guarantees. We demonstrate our approach on the FIVES retinal vessel segmentation dataset and compare five commonly used sampling-based uncertainty estimation techniques. Our results show that it is possible to achieve the desired coverage with small prediction ranges, highlighting the potential of performance range prediction as a valuable tool for output quality control (Code available at https://​github.​com/​annawundram/​PerformanceRange​Prediction).

A reliable/trustworthy image segmentation pipeline plays a central role in deploying AI medical image analysis systems in clinical practice. Given a segmentation map produced by a segmentation model, it is desired to have an automatic, accurate, and reliable method in the pipeline for segmentation quality assessment (SQA) when the ground truth is absent. In this paper, we present a novel holistic consistency based method for assessing at the subject-level the quality of segmentation produced by state-of-the-art segmentation models. Our method does not train a dedicated model using labeled samples to assess segmentation quality; instead, it systematically explores the segmentation consistency in an unsupervised manner. Our approach examines the consistency of segmentation results across three major aspects: (1) consistency across sub-models; (2) consistency across models; (3) consistency across different runs with random dropouts. For a given test image, combining consistency scores from the above mentioned aspects, we can generate an overall consistency score that is highly correlated with the true segmentation quality score (e.g., Dice score) in both linear correlation and rank correlation. Empirical results on two public datasets demonstrate that our proposed method outperforms previous unsupervised methods for subject-level SQA.

Deep learning image classifiers often struggle with domain shift, leading to significant performance degradation in real-world applications. In this paper, we introduce our CROCODILE framework, showing how tools from causality can foster a model’s robustness to domain shift via feature disentanglement, contrastive learning losses, and the injection of prior knowledge. This way, the model relies less on spurious correlations, learns the mechanism bringing from images to prediction better, and outperforms baselines on out-of-distribution (OOD) data. We apply our method to multi-label lung disease classification from chest X-rays (CXRs), utilizing over 750000 images from four datasets. Our bias-mitigation method improves domain generalization, broadening the applicability and reliability of deep learning models for a safer medical image analysis. Find our code at: https://​github.​com/​gianlucarloni/​crocodile.

Generating pseudo-healthy reconstructions of images is an effective way to detect anomalies, as identifying the differences between the reconstruction and the original can localise arbitrary anomalies whilst also providing interpretability for an observer by displaying what the image ‘should’ look like. All existing reconstruction-based methods have a common shortcoming; they assume that models trained on purely normal data are incapable of reproducing pathologies yet also able to fully maintain healthy tissue. These implicit assumptions often fail, with models either not recovering normal regions or reproducing both the normal and abnormal features. We rectify this issue using image-conditioned diffusion models. Our model takes the input image as conditioning and is explicitly trained to correct synthetic anomalies introduced into healthy images, ensuring that it removes anomalies at test time. This conditioning allows the model to attend to the entire image without any loss of information, enabling it to replicate healthy regions with high fidelity. We evaluate our method across four datasets and define a new state-of-the-art performance for residual-based anomaly detection. Code is available at https://​github.​com/​matt-baugh/​img-cond-diffusion-model-ad.

Deep learning models are subject to failure when inferring upon out-of-distribution (OOD) data, i.e., data that differs from the models’ train data. Within medical image settings, OOD data can be subtle and non-obvious to the human observer. Thus, developing highly sensitive algorithms is critical to automatically detect medical image OOD data. Previous works have demonstrated the utility of using the distance between embedded train and test features as an OOD measure. These methods, however, do not consider variations in feature importance to the prediction task, treating all features equally. In this work, we propose a method to enhance distance-based OOD measures via feature importance weighting, which is determined through an information bottleneck optimization process. We demonstrate the utility of the weighted OOD measure within the metastatic liver tumor segmentation task and compare its performance to its non-weighted counterpart in two assessments. The weighted OOD measure enhanced the detection of artificially perturbed data, where greater benefit was observed for smaller perturbations (e.g., AUC = 0.8 vs. AUC = 0.72). In addition, the weighted OOD measure achieved better correlation to liver tumor segmentation Dice coefficient (e.g., ρ = −0.76 vs ρ = −0.21). In summary, this work demonstrates the benefit of feature importance weighting for distance-based OOD detection.

An assumption-free, disease-agnostic pathology detector and segmentor is often regarded as one of the holy grails in medical image analysis. Building on this concept, un- or weakly supervised anomaly localization approaches have gained popularity. These methods aim to model normal or healthy samples using data and then detect deviations (i.e., abnormalities). However, as this is an emerging field situated between image segmentation and out-of-distribution detection, most approaches have adapted their evaluation setups and metrics from either of these areas. Consequently, they may have overlooked peculiarities inherent to anomaly localization. In this paper, we revisit the anomaly localization setup, analyze commonly used metrics, introduce alternative metrics inspired by instance segmentation, and compare these metrics across various settings and algorithms. We contend that the choice of metric is use-case dependent, but the SoftInstanceIoU and other object-based metrics show significant promise for future applications.

Detecting pathological abnormalities in medical images in an unsupervised manner holds potential for advancing modern medical diagnostics. However, supervised methods encounter challenges with exceedingly unbalanced training distributions due to limited clinical incidence rates. Likelihood-based unsupervised Out-of-Distribution (OOD) detection with generative models, especially Normalizing Flows, in which pathological abnormalities are considered OOD, could offer a promising solution. However, research in this direction has shown limited success as prior work has revealed that the likelihood does not accurately reflect the degree of anomaly for OOD samples, where in many instances higher likelihoods are assigned to anomalous samples compared to training samples. In this study, we present the first exploration of typicality (i.e. determining if samples belong to the typical set) for OOD detection in medical imaging, where test samples are juxtaposed against the probability mass rather than the density. The obtained findings demonstrate the superiority of evaluating typicality against likelihood for finding pathological abnormalities. We achieve state-of-the-art performance on the ISIC, COVID-19, and RSNA Pneumonia datasets, while being robust against significant data imbalances.

Reliable use of deep neural networks (DNNs) for medical image analysis requires methods to identify inputs that differ significantly from the training data, called out-of-distribution (OOD), to prevent erroneous predictions. OOD detection methods can be categorised as either confidence-based (using the model’s output layer for OOD detection) or feature-based (not using the output layer). We created two new OOD benchmarks by dividing the D7P (dermatology) and BreastMNIST (ultrasound) datasets into subsets which either contain or don’t contain an artefact (rulers or annotations respectively). Models were trained with artefact-free images, and images with the artefacts were used as OOD test sets. For each OOD image, we created a counterfactual by manually removing the artefact via image processing, to assess the artefact’s impact on the model’s predictions. We show that OOD artefacts can boost a model’s softmax confidence in its predictions, due to correlations in training data among other factors. This contradicts the common assumption that OOD artefacts should lead to more uncertain outputs, an assumption on which most confidence-based methods rely. We use this to explain why feature-based methods (e.g. Mahalanobis score) typically have greater OOD detection performance than confidence-based methods (e.g. MCP). However, we also show that feature-based methods typically perform worse at distinguishing between inputs that lead to correct and incorrect predictions (for both OOD and ID data). Following from these insights, we argue that a combination of feature-based and confidence-based methods should be used within DNN pipelines to mitigate their respective weaknesses. These project’s code and OOD benchmarks are available at: https://​github.​com/​HarryAnthony/​Evaluating_​OOD_​detection.

Vision transformers (ViTs) have emerged as strong alternatives to conventional convolutional neural networks (CNNs), owing to their scalability, enhanced generalization, and superior performance in out-of-distribution (OOD) scenarios. Despite their strengths, ViTs are prone to significant overfitting with scarce training data. This issue severely limits their reliability in critical applications, such as biomedical image analysis, where accurate uncertainty estimation is crucial. The challenge lies in the inherent lack of insight into the transformer network’s confidence and uncertainty levels. To tackle this issue, we propose a novel stochastic vision transformer characterized by three components: 1) Stochastic elliptical Gaussian embedding which encodes uncertainty into the embedding of image patches, 2) a Fréchet Inception Distance (FID)-based attention mechanism for the Gaussian embeddings and 3) a FID-based regularization term, which imposes distance and uncertainty awareness into the learning of stochastic representations. We demonstrate the effectiveness of our method for in-distribution calibration and OOD detection experiments on the skin cancer dataset ISIC2019.

Image analysis is a key tool for describing the detailed mechanisms of folliculogenesis, such as evaluating the quantity of mouse Primordial ovarian Follicles (PMF) in the ovarian reserve. The development of high-resolution virtual slide scanners offers the possibility of quantifying, robustifying and accelerating the histopathological procedure. A major challenge for machine learning is to control the precision of predictions while enabling a high recall, in order to provide reproducibility. We use a multiple testing procedure that gives an overperforming way to solve the standard Precision-Recall trade-off that gives probabilistic guarantees on the precision. In addition, we significantly improve the overall performance of the models (increase of F1-score) by selecting the decision threshold using contextual biological information or using an auxiliary model. As it is model-agnostic, this contextual selection procedure paves the way to the development of a strategy that can improve the performance of any model without the need of retraining it.

Chest X-ray imaging is essential for diagnosing thoracic diseases, with multi-label classification playing a critical role in identifying multiple conditions from a single image. Despite deep neural networks significantly advancing this field, noisy labels extracted from clinical reports pose a significant challenge, undermining the performance of deep models. Several research attempts have been made to address this issue but fail to consider the critical inter-class correlations prevalent in chest X-ray diagnostics. To this end, we propose a Global and Local Noise Correction framework. Our framework comprises a classification backbone and two primary components: a global noise correction module and a local noise correction module. The global noise correction module calculates the noise transition matrix based on the label co-occurrence frequencies and uses the estimated noise transition matrix to reduce the impact of the noisy labels. The local noise correction module treats the temporal ensembling of samples’ historical predictions as the instance-specific pseudo labels, which also serve as the supervision. The proposed framework addresses the shortcomings of existing techniques, i.e., the unreliability of noise transition matrices in the presence of class imbalances and zero co-occurrence frequencies. Comprehensive experimental results demonstrate that our framework surpasses competing methods, showcasing its superior ability to combat label noise and improve multi-label chest X-ray classification accuracy.

In the medical domain, where a misdiagnosis can have life-altering ramifications, understanding the certainty of model predictions is an important part of the model development process. However, deep learning approaches suffer from a lack of a native uncertainty metric found in other statistical learning methods. One common technique for uncertainty estimation is the use of Monte-Carlo (MC) dropout at training and inference. Another approach is Conformal Prediction for Uncertainty Quantification (CUQ). This paper will explore these two methods as applied to a cervical cancer screening algorithm currently under development for use in low-resource settings. We find that overall, CUQ and MC inference produce similar uncertainty patterns, that CUQ can aid in model development through class delineation, and that CUQ uncertainty is higher when the model is incorrect, providing further fine-grained information for clinical decisions. Code available here

Deep learning models have gained significant attention due to their promising performance in medical image tasks. However, a gap remains between experimental accuracy and real-world applications. The inherited black-box nature of the deep learning model introduces uncertainty, trustworthy issues, and difficulties in performing quality control of deployed deep learning models. While quality control methods focusing on uncertainty estimation for segmentation tasks exist, there are comparatively fewer approaches for classification, particularly in multi-label datasets. This paper addresses this gap by proposing a quality control method that bridges interpretability and uncertainty estimation through a graph-based class distinctiveness calculation. Using the CheXpert dataset, the proposed approach achieved a higher \(F_1\) score on the bootstrapped test set compared to baselines quality control approaches based on predictive entropy and test-time augmentation.