Graphs in Biomedical Image Analysis and Integrating Medical Imaging and Non-Imaging Modalities

Graph convolutional networks (GCNs) allow to apply traditional convolution operations in non-Euclidean domains, where data are commonly modelled as irregular graphs. Medical imaging and, in particular, neuroscience studies often rely on such graph representations, with brain connectivity networks being a characteristic example, while ultimately seeking the locus of phenotypic or disease-related differences in the brain. These regions of interest (ROIs) are, then, considered to be closely associated with function and/or behaviour. Driven by this, we explore GCNs for the task of ROI identification and propose a visual attribution method based on class activation mapping. By undertaking a sex classification task as proof of concept, we show that this method can be used to identify salient nodes (brain regions) without prior node labels. Based on experiments conducted on neuroimaging data of more than 5000 participants from UK Biobank, we demonstrate the robustness of the proposed method in highlighting reproducible regions across individuals. We further evaluate the neurobiological relevance of the identified regions based on evidence from large-scale UK Biobank studies.

The human brain demonstrates a network structure that is commonly represented using graphs with pseudonym connectome. Traditionally, connectomes encode only inter-regional connectivity as edges, while regional information, such as centrality of a node that may be crucial to the analysis, is usually handled as statistical covariates. This results in an incomplete encoding of valuable information. In order to alleviate such problems, we propose an enriched connectome encoding regional properties of the brain network, such as structural node degree, strength, and centrality, as node features in addition to representing structural connectivity between regions as weighted edges. We further present an efficient graph matching algorithm, providing two measures to quantify similarity between enriched connectomes. We demonstrate the utility of our graph representation and similarity measures on classifying a traumatic brain injury dataset. Our results show that the enriched representation combining nodal features and structural connectivity information with the graph matching based similarity measures is able to differentiate the groups better than the traditional connectome representation.

In large population-based studies and in clinical routine, tasks like disease diagnosis and progression prediction are inherently based on a rich set of multi-modal data, including imaging and other sensor data, clinical scores, phenotypes, labels and demographics. However, missing features, rater bias and inaccurate measurements are typical ailments of real-life medical datasets. Recently, it has been shown that deep learning with graph convolution neural networks (GCN) can outperform traditional machine learning in disease classification, but missing features remain an open problem. In this work, we follow up on the idea of modeling multi-modal disease classification as a matrix completion problem, with simultaneous classification and non-linear imputation of features. Compared to methods before, we arrange subjects in a graph-structure and solve classification through geometric matrix completion, which simulates a heat diffusion process that is learned and solved with a recurrent neural network. We demonstrate the potential of this method on the ADNI-based TADPOLE dataset and on the task of predicting the transition from MCI to Alzheimer’s disease. With an AUC of 0.950 and classification accuracy of 87%, our approach outperforms standard linear and non-linear classifiers, as well as several state-of-the-art results in related literature, including a recently proposed GCN-based approach.

In this study, we propose a novel brain parcellation algorithm, called BrainParcel. BrainParcel defines a set of supervoxels by partitioning a voxel level brain graph into a number of subgraphs, which are assumed to represent “homogeneous” brain regions with respect to a predefined criteria. Aforementioned brain graph is constructed by a set of local meshes, called mesh networks. Then, the supervoxels are obtained using a graph partitioning algorithm. The supervoxels form partitions of brain as an alternative to anatomical regions (AAL). Compared to AAL, supervoxels gather functionally and spatially close voxels. This study shows that BrainParcel can achieve higher accuracies in cognitive state classification compared to AAL. It has a better representation power compared to similar brain segmentation methods, reported the literature.

In this study, we propose a neural network approach to capture the functional connectivities among anatomic brain regions. The suggested approach estimates a set of brain networks, each of which represents the connectivity patterns of a cognitive process. We employ two different architectures of neural networks to extract directed and undirected brain networks from functional Magnetic Resonance Imaging (fMRI) data. Then, we use the edge weights of the estimated brain networks to train a classifier, namely, Support Vector Machines (SVM) to label the underlying cognitive process. We compare our brain network models with popular models, which generate similar functional brain networks. We observe that both undirected and directed brain networks surpass the performances of the network models used in the fMRI literature. We also observe that directed brain networks offer more discriminative features compared to the undirected ones for recognizing the cognitive processes. The representation power of the suggested brain networks are tested in a task-fMRI dataset of Human Connectome Project and a Complex Problem Solving dataset.

In this work, we consider the problem of predicting the course of a progressive disease, such as cancer or Alzheimer’s. Progressive diseases often start with mild symptoms that might precede a diagnosis, and each patient follows their own trajectory. Patient trajectories exhibit wild variability, which can be associated with many factors such as genotype, age, or sex. An additional layer of complexity is that, in real life, the amount and type of data available for each patient can differ significantly. For example, for one patient we might have no prior history, whereas for another patient we might have detailed clinical assessments obtained at multiple prior time-points. This paper presents a probabilistic model that can handle multiple modalities (including images and clinical assessments) and variable patient histories with irregular timings and missing entries, to predict clinical scores at future time-points. We use a sigmoidal function to model latent disease progression, which gives rise to clinical observations in our generative model. We implemented an approximate Bayesian inference strategy on the proposed model to estimate the parameters on data from a large population of subjects. Furthermore, the Bayesian framework enables the model to automatically fine-tune its predictions based on historical observations that might be available on the test subject. We applied our method to a longitudinal Alzheimer’s disease dataset with more than 3,000 subjects [1] with comparisons against several benchmarks.

Functional connectivities in the brain explain how different brain regions interact with each other when conducting a specific activity. Canonical correlation analysis (CCA) based models, have been used to detect correlations and to analyze brain connectivities which further help explore how the brain works. However, the data representation of CCA lacks label related information and may be limited when applied to functional connectivity study. Collaborative regression was proposed to address the limitation of CCA by combining correlation analysis and regression. However, both prediction and correlation are sacrificed as linear collaborative regression use the same set of projections on both correlation and regression. We propose a novel method, deep collaborative learning (DCL), to address the limitations of CCA and collaborative regression. DCL improves collaborative regression by combining correlation analysis and label information using deep networks, which may lead to better performance both for classification/prediction and for correlation detection. Results demonstrated the out-performance of DCL over other conventional models in terms of classification accuracy. Experiments showed the difference of brain connectivities between different age groups may be more significant than that between different cognition groups.

Alzheimer’s disease (AD) is the only major cause of mortality in the world without an effective disease modifying treatment. Evidence supporting the so called “disconnection hypothesis” suggests that functional connectivity biomarkers may have clinical potential for early detection of AD. However, known issues with low test-retest reliability and signal to noise in functional connectivity may prevent accuracy and subsequent predictive capacity. We validate the utility of a novel principal component based diagnostic identifiability framework to increase separation in functional connectivity across the Alzheimer’s spectrum by identifying and reconstructing FC using only AD sensitive components or connectivity modes. We show that this framework (1) increases test-retest correspondence and (2) allows for better separation, in functional connectivity, of diagnostic groups both at the whole brain and individual resting state network level. Finally, we evaluate a posteriori the association between connectivity mode weights with longitudinal neurocognitive outcomes.

Nowadays, a lot of scientific efforts are concentrated on the diagnosis of Alzheimers Disease (AD) applying deep learning methods to neuroimaging data. Even for 2017, there were published more than hundred papers dedicated to AD diagnosis, whereas only a few works considered a problem of mild cognitive impairments (MCI) conversion to AD. However, the conversion prediction is an important problem since approximately 15% of patients with MCI converges to AD every year. In the current work, we are focusing on the conversion prediction using brain Magnetic Resonance Imaging and clinical data, such as demographics, cognitive assessments, genetic, and biochemical markers. First of all, we applied state-of-the-art deep learning algorithms on the neuroimaging data and compared these results with two machine learning algorithms that we fit on the clinical data. As a result, the models trained on the clinical data outperform the deep learning algorithms applied to the MR images. To explore the impact of neuroimaging further, we trained a deep feed-forward embedding using similarity learning with Histogram loss on all available MRIs and obtained 64-dimensional vector representation of neuroimaging data. The use of learned representation from the deep embedding allowed to increase the quality of prediction based on the neuroimaging. Finally, the current results on this dataset show that the neuroimaging does have an effect on conversion prediction, however cannot noticeably increase the quality of the prediction. The best results of predicting MCI-to-AD conversion are provided by XGBoost algorithm trained on the clinical and embedding data. The resulting accuracy is \(\text {ACC} = 0.76 \pm 0.01\) and the area under the ROC curve – \(\text {ROC AUC} = 0.86 \pm 0.01\).

Anorexia nervosa (AN) and body dysmorphic disorder (BDD) share several phenomenological features including distorted perception of appearance, obsessions/compulsions, and limited insight. They also show partially overlapping patterns of brain activation, white matter connectivity, and electrophysiological responses. These markers have also shown associations with symptom severity within each disorder. We aimed to determine: (a) if, cross-diagnostically, neural activity and connectivity predict dimensional clinical phenotypes, and (b) the relative contribution of multimodal markers to these predictions beyond demographics and psychometrics, in a multivariate context. We used functional magnetic resonance imaging (fMRI) data from a visual task, graph theory metrics of white matter connectivity from diffusor tensor imaging, anxiety and depression psychometric scores, and demographics to predict dimensional phenotypes of insight and obsession/compulsions across a sample of unmedicated adults with BDD (n = 29) and weight-restored AN (n = 24). The multivariate model that included fMRI and white matter connectivity data performed significantly better in predicting both insight and obsessions/compulsions than a model only including demographics and psychometrics. These results demonstrate the utility of neurobiologically-based markers to predict important clinical phenotypes. The findings also contribute to understanding potential cross-diagnostic substrates for these phenotypes in these related but nosologically discrete disorders.