Information Processing in Medical Imaging

In this paper, we present novel algorithms to compute robust statistics from manifold-valued data. Specifically, we present algorithms for estimating the robust Fréchet Mean (FM) and performing a robust exact-principal geodesic analysis (ePGA) for data lying on known Riemannian manifolds. We formulate the minimization problems involved in both these problems using the minimum distance estimator called the L$$_2$$E. This leads to a nonlinear optimization which is solved efficiently using a Riemannian accelerated gradient descent technique. We present competitive performance results of our algorithms applied to synthetic data with outliers, the corpus callosum shapes extracted from OASIS MRI database, and diffusion MRI scans from movement disorder patients respectively.

We tackle the problem of template estimation when data have been randomly transformed under an isometric group action in the presence of noise. In order to estimate the template, one often minimizes the variance when the influence of the transformations have been removed (computation of the Fréchet mean in quotient space). The consistency bias is defined as the distance (possibly zero) between the orbit of the template and the orbit of one element which minimizes the variance. In this article we establish an asymptotic behavior of the consistency bias with respect to the noise level. This behavior is linear with respect to the noise level. As a result the inconsistency is unavoidable as soon as the noise is large enough. In practice, the template estimation with a finite sample is often done with an algorithm called max-max. We show the convergence of this algorithm to an empirical Karcher mean. Finally, our numerical experiments show that the bias observed in practice cannot be attributed to the small sample size or to a convergence problem but is indeed due to the previously studied inconsistency.

Typical cerebral cortical analyses rely on spatial normalization and are sensitive to misregistration arising from partial homologies between subject brains and local optima in nonlinear registration. In contrast, we use a descriptor of the 3D cortical sheet (jointly modeling folding and thickness) that is robust to misregistration. Our histogram-based descriptor lies on a Riemannian manifold. We propose new regularized nonlinear methods for (i) detecting group differences, using a Mercer kernel with an implicit lifting map to a reproducing kernel Hilbert space, and (ii) regression against clinical variables, using kernel density estimation. For both methods, we employ kernels that exploit the Riemannian structure. Results on simulated and clinical data shows the improved accuracy and stability of our approach in cortical-sheet analysis.

Manifold-valued data arises frequently in medical imaging, surface modeling, computational biology, and computer vision, among many others. The aim of this paper is to introduce a conditional local distance correlation measure for characterizing a nonlinear association between manifold-valued data, denoted by X, and a set of variables (e.g., diagnosis), denoted by Y, conditional on the other set of variables (e.g., gender and age), denoted by Z. Our nonlinear association measure is solely based on the distance of the space that X, Y, and Z are resided, avoiding both specifying any parametric distribution and link function and projecting data to local tangent planes. It can be easily extended to the case when both X and Y are manifold-valued data. We develop a computationally fast estimation procedure to calculate such nonlinear association measure. Moreover, we use a bootstrap method to determine its asymptotic distribution and p-value in order to test a key hypothesis of conditional independence. Simulation studies and a real data analysis are used to evaluate the finite sample properties of our methods.

We introduce a regression model for data on non-linear manifolds. The model describes the relation between a set of manifold valued observations, such as shapes of anatomical objects, and Euclidean explanatory variables. The approach is based on stochastic development of Euclidean diffusion processes to the manifold. Defining the data distribution as the transition distribution of the mapped stochastic process, parameters of the model, the non-linear analogue of design matrix and intercept, are found via maximum likelihood. The model is intrinsically related to the geometry encoded in the connection of the manifold. We propose an estimation procedure which applies the Laplace approximation of the likelihood function. A simulation study of the performance of the model is performed and the model is applied to a real dataset of Corpus Callosum shapes.

We propose a framework for probabilistic shape clustering based on kernel-space embeddings derived from spectral signatures. Our root motivation is to investigate practical yet principled clustering schemes that rely on geometrical invariants of shapes rather than explicit registration. To that end we revisit the use of the Laplacian spectrum and introduce a parametric family of reproducing kernels for shapes, extending WESD [12] and shape DNA [20] like metrics. Parameters provide control over the relative importance of local and global shape features and can be adjusted to emphasize a scale of interest. As a result of kernelization, shapes are embedded in an infinite-dimensional inner product space. We leverage this structure to formulate shape clustering via a Bayesian mixture of kernel-space Principal Component Analysers. We derive simple variational Bayes inference schemes in Hilbert space, addressing technicalities stemming from the infinite dimensionality. The proposed approach is validated on tasks of unsupervised clustering of sub-cortical structures, as well as classification of cardiac left ventricles w.r.t. pathological groups.

In cardiac image analysis, it is important yet challenging to reconstruct the trabeculae, namely, fine muscle columns whose ends are attached to the ventricular walls. To extract these fine structures, traditional image segmentation methods are insufficient. In this paper, we propose a novel method to jointly detect salient topological handles and compute the optimal representations of them. The detected handles are considered hypothetical trabeculae structures. They are further screened using a classifier and are then included in the final segmentation. We show in experiments the significance of our contribution compared with previous standard segmentation methods without topological priors, as well as with previous topological method in which non-optimal representations of topological handles are used.

This paper proposes a Bayesian treatment of a latent variable model for learning generative shape models of grid-structured representations, aka label maps, that relies on direct probabilistic formulation with a variational approach for deterministic model learning. Spatial coherency and sparsity priors are incorporated to lend stability to the optimization problem, thereby regularizing the solution space while avoiding overfitting in this high-dimensional, low-sample-size scenario. Hyperparameters are estimated in closed-form using type-II maximum likelihood to avoid grid searches. Further, a mixture formulation is proposed to capture nonlinear shape variations in a way that balances the model expressiveness with the efficiency of learning and inference. Experiments show that the proposed model outperforms state-of-the-art representations on real datasets w.r.t. generalization to unseen samples.

We consider the task of constructing (metric) shape space(s) from a topological perspective. In particular, we present a generic construction scheme and demonstrate how to apply this scheme when shape is interpreted as the differences that remain after factoring out translation, scaling and rotation. This is achieved by leveraging a recently proposed injective functional transform of 2D/3D (binary) objects, based on persistent homology. The resulting shape space is then equipped with a similarity measure that is (1) by design robust to noise and (2) fulfills all metric axioms. From a practical point of view, analyses of object shape can then be carried out directly on segmented objects obtained from some imaging modality without any preprocessing, such as alignment, smoothing, or landmark selection. We demonstrate the utility of the approach on the problem of distinguishing segmented hippocampi from normal controls vs. patients with Alzheimer’s disease in a challenging setup where volume changes are no longer discriminative.

The event-based model (EBM) for data-driven disease progression modeling estimates the sequence in which biomarkers for a disease become abnormal. This helps in understanding the dynamics of disease progression and facilitates early diagnosis by staging patients on a disease progression timeline. Existing EBM methods are all generative in nature. In this work we propose a novel discriminative approach to EBM, which is shown to be more accurate as well as computationally more efficient than existing state-of-the art EBM methods. The method first estimates for each subject an approximate ordering of events, by ranking the posterior probabilities of individual biomarkers being abnormal. Subsequently, the central ordering over all subjects is estimated by fitting a generalized Mallows model to these approximate subject-specific orderings based on a novel probabilistic Kendall’s Tau distance. To evaluate the accuracy, we performed extensive experiments on synthetic data simulating the progression of Alzheimer’s disease. Subsequently, the method was applied to the Alzheimer’s Disease Neuroimaging Initiative (ADNI) data to estimate the central event ordering in the dataset. The experiments benchmark the accuracy of the new model under various conditions and compare it with existing state-of-the-art EBM methods. The results indicate that discriminative EBM could be a simple and elegant approach to disease progression modeling.

We present a disease progression model with single vertex resolution that we apply to cortical thickness data. Our model works by clustering together vertices on the cortex that have similar temporal dynamics and building a common trajectory for vertices in the same cluster. The model estimates optimal stages and progression speeds for every subject. Simulated data show that it is able to accurately recover the vertex clusters and the underlying parameters. Moreover, our clustering model finds similar patterns of atrophy for typical Alzheimer’s disease (tAD) subjects on two independent datasets: the Alzheimer’s Disease Neuroimaging Initiative (ADNI) and a cohort from the Dementia Research Centre (DRC), UK. Using a separate set of subjects with Posterior Cortical Atrophy (PCA) from the DRC dataset, we also show that the model finds different patterns of atrophy in PCA compared to tAD. Finally, our model provides a novel way to parcellate the brain based on disease dynamics.

Obtaining models that capture imaging markers relevant for disease progression and treatment monitoring is challenging. Models are typically based on large amounts of data with annotated examples of known markers aiming at automating detection. High annotation effort and the limitation to a vocabulary of known markers limit the power of such approaches. Here, we perform unsupervised learning to identify anomalies in imaging data as candidates for markers. We propose AnoGAN, a deep convolutional generative adversarial network to learn a manifold of normal anatomical variability, accompanying a novel anomaly scoring scheme based on the mapping from image space to a latent space. Applied to new data, the model labels anomalies, and scores image patches indicating their fit into the learned distribution. Results on optical coherence tomography images of the retina demonstrate that the approach correctly identifies anomalous images, such as images containing retinal fluid or hyperreflective foci.

Recently hyper-graph learning gains increasing attention in medical imaging area since the hyper-graph, a generalization of a graph, opts to characterize the complex subject-wise relationship behind multi-modal neuroimaging data. However, current hyper-graph methods mainly have two limitations: (1) The data representation encoded in the hyper-graph is learned only from the observed imaging features for each modality separately. Therefore, the learned subject-wise relationships are neither consistent across modalities nor fully consensus with the clinical labels or clinical scores. (2) The learning procedure of data representation is completely independent to the subsequent classification step. Since the data representation optimized in the feature domain is not exactly aligned with the clinical labels, such independent step-by-step workflow might result in sub-optimal classification. To address these limitations, we propose a novel dynamic hyper-graph inference framework, working in a semi-supervised manner, which iteratively estimates and adjusts the subject-wise relationship from multi-modal neuroimaging data until the learned data representation (encoded in the hyper-graph) achieves largest consensus with the observed clinical labels and scores. It is worth noting our inference framework is also flexible to integrate classification (identifying individuals with neuro-disease) and regression (predicting the clinical scores). We have demonstrated the performance of our proposed dynamic hyper-graph inference framework in identifying MCI (Mild Cognition Impairment) subjects and the fine-grained recognition of different progression stage of MCI, where we achieve more accurate diagnosis result than conventional counterpart methods.

We propose a non-parametric approach for characterizing heterogeneous diseases in large-scale studies. We target diseases where multiple types of pathology present simultaneously in each subject and a more severe disease manifests as a higher level of tissue destruction. For each subject, we model the collection of local image descriptors as samples generated by an unknown subject-specific probability density. Instead of approximating the probability density via a parametric family, we propose to side step the parametric inference by directly estimating the divergence between subject densities. Our method maps the collection of local image descriptors to a signature vector that is used to predict a clinical measurement. We are able to interpret the prediction of the clinical variable in the population and individual levels by carefully studying the divergences. We illustrate an application this method on simulated data as well as on a large-scale lung CT study of Chronic Obstructive Pulmonary Disease (COPD). Our approach outperforms classical methods on both simulated and COPD data and demonstrates the state-of-the-art prediction on an important physiologic measure of airflow (the forced respiratory volume in one second, FEV1).

Alzheimer’s Disease (AD) is the most common type of dementia. Identifying correct biomarkers may determine pre-symptomatic AD subjects and enable early intervention. Recently, Multi-task sparse feature learning has been successfully applied to many computer vision and biomedical informatics researches. It aims to improve the generalization performance by exploiting the shared features among different tasks. However, most of the existing algorithms are formulated as a supervised learning scheme. Its drawback is with either insufficient feature numbers or missing label information. To address these challenges, we formulate an unsupervised framework for multi-task sparse feature learning based on a novel dictionary learning algorithm. To solve the unsupervised learning problem, we propose a two-stage Multi-Source Multi-Target Dictionary Learning (MMDL) algorithm. In stage 1, we propose a multi-source dictionary learning method to utilize the common and individual sparse features in different time slots. In stage 2, supported by a rigorous theoretical analysis, we develop a multi-task learning method to solve the missing label problem. Empirical studies on an $$N = 3970$$ longitudinal brain image data set, which involves 2 sources and 5 targets, demonstrate the improved prediction accuracy and speed efficiency of MMDL in comparison with other state-of-the-art algorithms.

Alzheimer’s disease (AD) is a progressive neurodegenerative disorder. As the prodromal stage of AD, Mild Cognitive Impairment (MCI) maintains a good chance of converting to AD. How to efficaciously detect this conversion from MCI to AD is significant in AD diagnosis. Different from standard classification problems where the distributions of classes are independent, the AD outcomes are usually interrelated (their distributions have certain overlaps). Most of existing methods failed to examine the interrelations among different classes, such as AD, MCI conversion and MCI non-conversion. In this paper, we proposed a novel self-learned low-rank structured learning model to automatically uncover the interrelations among classes and utilized such interrelated structures to enhance classification. We conducted experiments on the ADNI cohort data. Empirical results demonstrated advantages of our model.

We propose a learning method to identify which specific regions and features of images contribute to a certain classification. In the medical imaging context, they can be the evidence regions where the abnormalities are most likely to appear, and the discriminative features of these regions supporting the pathology classification. The learning is weakly-supervised requiring only the pathological labels and no other prior knowledge. The method can also be applied to learn the salient description of an anatomy discriminative from its background, in order to localise the anatomy before a classification step. We formulate evidence pinpointing as a sparse descriptor learning problem. Because of the large computational complexity, the objective function is composed in a stochastic way and is optimised by the Regularised Dual Averaging algorithm. We demonstrate that the learnt feature descriptors contain more specific and better discriminative information than hand-crafted descriptors contributing to superior performance for the tasks of anatomy localisation and pathology classification respectively. We apply our method on the problem of lumbar spinal stenosis for localising and classifying vertebrae in MRI images. Experimental results show that our method when trained with only target labels achieves better or competitive performance on both tasks compared with strongly-supervised methods requiring labels and multiple landmarks. A further improvement is achieved with training on additional weakly annotated data, which gives robust localisation with average error within 2 mm and classification accuracies close to human performance.

Estimation of patient-specific model parameters is important for personalized modeling, although sparse and noisy clinical data can introduce significant uncertainty in the estimated parameter values. This importance source of uncertainty, if left unquantified, will lead to unknown variability in model outputs that hinder their reliable adoptions. Probabilistic estimation model parameters, however, remains an unresolved challenge because standard Markov Chain Monte Carlo sampling requires repeated model simulations that are computationally infeasible. A common solution is to replace the simulation model with a computationally-efficient surrogate for a faster sampling. However, by sampling from an approximation of the exact posterior probability density function (pdf) of the parameters, the efficiency is gained at the expense of sampling accuracy. In this paper, we address this issue by integrating surrogate modeling into Metropolis Hasting (MH) sampling of the exact posterior pdfs to improve its acceptance rate. It is done by first quickly constructing a Gaussian process (GP) surrogate of the exact posterior pdfs using deterministic optimization. This efficient surrogate is then used to modify commonly-used proposal distributions in MH sampling such that only proposals accepted by the surrogate will be tested by the exact posterior pdf for acceptance/rejection, reducing unnecessary model simulations at unlikely candidates. Synthetic and real-data experiments using the presented method show a significant gain in computational efficiency without compromising the accuracy. In addition, insights into the non-identifiability and heterogeneity of tissue properties can be gained from the obtained posterior distributions.

Metastasis detection of lymph nodes in Whole-slide Images (WSIs) plays a critical role in the diagnosis of breast cancer. Automatic metastasis detection is a challenging issue due to the large variance of their appearances and the size of WSIs. Recently, deep neural networks have been employed to detect cancer metastases by dividing the WSIs into small image patches. However, most existing works simply treat these patches independently and do not consider the structural information among them. In this paper, we propose a novel deep neural network, namely Spatially Structured Network (Spatio-Net) to tackle the metastasis detection problem in WSIs. By integrating the Convolutional Neural Network (CNN) with the 2D Long-Short Term Memory (2D-LSTM), our Spatio-Net is able to learn the appearances and spatial dependencies of image patches effectively. Specifically, the CNN encodes each image patch into a compact feature vector, and the 2D-LSTM layers provide the classification results (i.e., normal or tumor), considering its dependencies on other relevant image patches. Moreover, a new loss function is designed to constrain the structure of the output labels, which further improves the performance. Finally, the metastasis positions are obtained by locating the regions with high tumor probabilities in the resulting accurate probability map. The proposed method is validated on hundreds of WSIs, and the accuracy is significantly improved, in comparison with a state-of-the-art baseline that does not have the spatial dependency constraint.

Risk stratification of lung nodules is a task of primary importance in lung cancer diagnosis. Any improvement in robust and accurate nodule characterization can assist in identifying cancer stage, prognosis, and improving treatment planning. In this study, we propose a 3D Convolutional Neural Network (CNN) based nodule characterization strategy. With a completely 3D approach, we utilize the volumetric information from a CT scan which would be otherwise lost in the conventional 2D CNN based approaches. In order to address the need for a large amount of training data for CNN, we resort to transfer learning to obtain highly discriminative features. Moreover, we also acquire the task dependent feature representation for six high-level nodule attributes and fuse this complementary information via a Multi-task learning (MTL) framework. Finally, we propose to incorporate potential disagreement among radiologists while scoring different nodule attributes in a graph regularized sparse multi-task learning. We evaluated our proposed approach on one of the largest publicly available lung nodule datasets comprising 1018 scans and obtained state-of-the-art results in regressing the malignancy scores.

The preservation of the spatial relationships among axonal pathways has long been studied and known to be critical for many functions of the brain. Being a fundamental property of the brain connections, there is an intuitive understanding of topographic regularity in neuroscience but yet to be systematically explored in connectome imaging research. In this work, we propose a general mathematical model for topographic regularity of fiber bundles that is consistent with its neuroanatomical understanding. Our model is based on a novel group spectral graph analysis (GSGA) framework motivated by spectral graph theory and tensor decomposition. GSGA provides a common set of eigenvectors for the graphs formed by topographic proximity measures whose preservation along individual tracts in return is modeled as topographic regularity. To demonstrate the application of this novel measure of topographic regularity, we apply it to filter fiber tracts from connectome imaging. Using large-scale data from the Human Connectome Project (HCP), we show that our novel algorithm can achieve better performance than existing methods on the filtering of both individual bundles and whole brain tractograms.

The introduction of graph theory in neuroimaging has provided invaluable tools for the study of brain connectivity. These methods require the definition of a graph, which is typically derived by estimating the effective connectivity between brain regions through the optimization of an ill-posed inverse problem. Considerable efforts have been devoted to the development of methods extracting sparse connectivity graphs.The present paper aims at highlighting the benefits of an alternative approach. We investigate low-rank L2 regularized matrices recently introduced under the denomination of Riccati regularized precision matrices. We demonstrate their benefits for the analysis of cortical thickness map and the extraction of functional biomarkers from resting state fMRI scans. In addition, we explain how speed and result quality can be further improved with random projections. The promising results obtained using the Human Connectome Project dataset, as well as, the numerous possible extensions and applications suggest that Riccati precision matrices might usefully complement current sparse approaches.

Community detection methods have been widely used for studying the modular structure of the brain. However, few of these methods exploit the intrinsic properties of brain networks other than modularity to tackle the pronounced noise in neuroimaging data. We propose a random walker (RW) based approach that reflects how regions of a brain subnetwork tend to be inter-linked by a provincial hub. By using provincial hubs to guide seed setting, RW provides the exact posterior probability of a brain region belonging to each given subnetwork, which mitigates forced hard assignments of brain regions to subnetworks as is the case in most existing methods. We further present an extension that enables multimodal integration for exploiting complementary information from functional Magnetic Resonance Imaging (fMRI) and diffusion MRI (dMRI) data. On synthetic data, our approach achieves higher accuracy in subnetwork extraction than unimodal and existing multimodal approaches. On real data from the Human Connectome Project (HCP), our estimated subnetworks match well with established brain systems and attain higher inter-subject reproducibility.

We present a novel framework for characterizing paired brain networks using techniques in hyper-networks, sparse learning and persistent homology. The framework is general enough for dealing with any type of paired images such as twins, multimodal and longitudinal images. The exact nonparametric statistical inference procedure is derived on testing monotonic graph theory features that do not rely on time consuming permutation tests. The proposed method computes the exact probability in quadratic time while the permutation tests require exponential time. As illustrations, we apply the method to simulated networks and a twin fMRI study. In case of the latter, we determine the statistical significance of the heritability index of the large-scale reward network where every voxel is a network node.

There is an increasing consensus in the scientific and medical communtities that functional brain analysis should be conducted from a connectionist standpoint. Most connectivity studies to date rely on derived measures of graph properties. In this paper, we show that brain networks can be analyzed effectively by considering them as elements of the Riemannian manifold of symmetric positive definite matrices $$\text {Sym}^+$$. Using recently proposed methods for manifold multivariate linear modelling, we analyze the developing functional connectivity of a small cohort of children aged 6 to 13 of both genders with strongly varying handedness indices at both rest and task simultaneously. We show that even with small sample sizes we can obtain results that reflect findings on large cohorts, and that $$\text {Sym}^+$$ is a better framework for analyzing functional brain connectivity compared to Euclidean space.

Structured sparsity penalization has recently improved statistical models applied to high-dimensional data in various domains. As an extension to medical imaging, the present work incorporates priors on network hierarchies of brain regions into logistic-regression to distinguish neural activity effects. These priors bridge two separately studied levels of brain architecture: functional segregation into regions and functional integration by networks. Hierarchical region-network priors are shown to better classify and recover 18 psychological tasks than other sparse estimators. Varying the relative importance of region and network structure within the hierarchical tree penalty captured complementary aspects of the neural activity patterns. Local and global priors of neurobiological knowledge are thus demonstrated to offer advantages in generalization performance, sample complexity, and domain interpretability.

One of the primary objectives of human brain mapping is the division of the cortical surface into functionally distinct regions, i.e. parcellation. While it is generally agreed that at macro-scale different regions of the cortex have different functions, the exact number and configuration of these regions is not known. Methods for the discovery of these regions are thus important, particularly as the volume of available information grows. Towards this end, we present a parcellation method based on a Bayesian non-parametric mixture model of cortical connectivity.

Deep convolutional neural networks are powerful tools for learning visual representations from images. However, designing efficient deep architectures to analyse volumetric medical images remains challenging. This work investigates efficient and flexible elements of modern convolutional networks such as dilated convolution and residual connection. With these essential building blocks, we propose a high-resolution, compact convolutional network for volumetric image segmentation. To illustrate its efficiency of learning 3D representation from large-scale image data, the proposed network is validated with the challenging task of parcellating 155 neuroanatomical structures from brain MR images. Our experiments show that the proposed network architecture compares favourably with state-of-the-art volumetric segmentation networks while being an order of magnitude more compact. We consider the brain parcellation task as a pretext task for volumetric image segmentation; our trained network potentially provides a good starting point for transfer learning. Additionally, we show the feasibility of voxel-level uncertainty estimation using a sampling approximation through dropout.

This paper seeks to discover common change-point patterns, associated with functional connectivity (FC) in human brain, across multiple subjects. FC, represented as a covariance or a correlation matrix, relates to the similarity of fMRI responses across different brain regions, when a brain is simply resting or performing a task under an external stimulus. While the dynamical nature of FC is well accepted, this paper develops a formal statistical test for finding change-points in times series associated with FC observed over time. It represents instantaneous connectivity by a symmetric positive-definite matrix, and uses a Riemannian metric on this space to develop a graphical method for detecting change-points in a time series of such matrices. It also provides a graphical representation of estimated FC for stationary subintervals in between detected change-points. Furthermore, it uses a temporal alignment of the test statistic, viewed as a real-valued function over time, to remove temporal variability and to discover common change-point patterns across subjects, tasks, and regions. This method is illustrated using HCP database for multiple subjects and tasks.

Finding the common structural brain connectivity network for a given population is an open problem, crucial for current neuroscience. Recent evidence suggests there’s a tightly connected network shared between humans. Obtaining this network will, among many advantages, allow us to focus cognitive and clinical analyses on common connections, thus increasing their statistical power. In turn, knowledge about the common network will facilitate novel analyses to understand the structure-function relationship in the brain.In this work, we present a new algorithm for computing the core structural connectivity network of a subject sample combining graph theory and statistics. Our algorithm works in accordance with novel evidence on brain topology. We analyze the problem theoretically and prove its complexity. Using 309 subjects, we show its advantages when used as a feature selection for connectivity analysis on populations, outperforming the current approaches.

Many methods have been developed to spatially normalize a population of brain images for estimating a mean image as a population- average atlas. However, methods for deriving a network atlas from a set of brain networks sitting on a complex manifold are still absent. Learning how to average brain networks across subjects constitutes a key step in creating a reliable mean representation of a population of brain networks, which can be used to spot abnormal deviations from the healthy network atlas. In this work, we propose a novel network atlas estimation framework, which guarantees that the produced network atlas is clean (for tuning down noisy measurements) and well-centered (for being optimally close to all subjects and representing the individual traits of each subject in the population). Specifically, for a population of brain networks, we first build a tensor, where each of its frontal-views (i.e., frontal matrices) represents a connectivity network matrix of a single subject in the population. Then, we use tensor robust principal component analysis for jointly denoising all subjects’ networks through cleaving a sparse noisy network population tensor from a clean low-rank network tensor. Second, we build a graph where each node represents a frontal-view of the unfolded clean tensor (network), to leverage the local manifold structure of these networks when fusing them. Specifically, we progressively shrink the graph of networks towards the centered mean network atlas through non-linear diffusion along the local neighbors of each of its nodes. Our evaluation on the developing functional and morphological brain networks at 1, 3, 6, 9 and 12 months of age has showed a better centeredness of our network atlases, in comparison with the baseline network fusion method. Further cleaning of the population of networks produces even more centered atlases, especially for the noisy functional connectivity networks.

Functional connectivity (FC) has been widely investigated in many imaging-based neuroscience and clinical studies. Since functional Magnetic Resonance Image (MRI) signal is just an indirect reflection of brain activity, it is difficult to accurately quantify the FC strength only based on signal correlation. To address this limitation, we propose a learning-based tensor model to derive high sensitivity and specificity connectome biomarkers at the individual level from resting-state fMRI images. First, we propose a learning-based approach to estimate the intrinsic functional connectivity. In addition to the low level region-to-region signal correlation, latent module-to-module connection is also estimated and used to provide high level heuristics for measuring connectivity strength. Furthermore, sparsity constraint is employed to automatically remove the spurious connections, thus alleviating the issue of searching for optimal threshold. Second, we integrate our learning-based approach with the sliding-window technique to further reveal the dynamics of functional connectivity. Specifically, we stack the functional connectivity matrix within each sliding window and form a 3D tensor where the third dimension denotes for time. Then we obtain dynamic functional connectivity (dFC) for each individual subject by simultaneously estimating the within-sliding-window functional connectivity and characterizing the across-sliding-window temporal dynamics. Third, in order to enhance the robustness of the connectome patterns extracted from dFC, we extend the individual-based 3D tensors to a population-based 4D tensor (with the fourth dimension stands for the training subjects) and learn the statistics of connectome patterns via 4D tensor analysis. Since our 4D tensor model jointly (1) optimizes dFC for each training subject and (2) captures the principle connectome patterns, our statistical model gains more statistical power of representing new subject than current state-of-the-art methods which in contrast perform above two steps separately. We have applied our tensor statistical model to identify ASD (Autism Spectrum Disorder) by using the learned dFC patterns. Promising classification results have been achieved demonstrating high discrimination power and great potentials in computer assisted diagnosis of neuro-disorders.

Task-based fMRI (tfMRI) has been widely used to study functional brain networks. Modeling tfMRI data is challenging due to at least two problems: the lack of the ground truth of underlying neural activity and the intrinsic structure of tfMRI data is highly complex. To better understand brain networks based on fMRI data, data-driven approaches were proposed, for instance, Independent Component Analysis (ICA) and Sparse Dictionary Learning (SDL). However, both ICA and SDL only build shallow models, and they are under the strong assumption that original fMRI signal could be linearly decomposed into time series components with their corresponding spatial maps. As growing evidence shows that human brain function is hierarchically organized, new approaches that can infer and model the hierarchical structure of brain networks are widely called for. Recently, deep convolutional neural network (CNN) has drawn much attention, in that deep CNN has been proven to be a powerful method for learning high-level and mid-level abstractions from low-level raw data. Inspired by the power of deep CNN, in this study, we developed a new neural network structure based on CNN, called Deep Convolutional Auto-Encoder (DCAE), in order to take the advantages of both data-driven approach and CNN’s hierarchical feature abstraction ability for the purpose of learning mid-level and high-level features from complex tfMRI time series in an unsupervised manner. The DCAE has been applied and tested on the publicly available human connectome project (HCP) tfMRI datasets, and promising results are achieved.

In diffusion MRI, a tensor field or a spherical function field, e.g., an Orientation Distribution Function (ODF) field, are estimated from measured diffusion weighted images. In this paper, inspired by microscopic theoretical treatment of phases in liquid crystals, we introduce a novel mathematical framework, called Director Field Analysis (DFA), to study local geometric structural information of white matter from the estimated tensor field or spherical function field. (1) We propose Orientational Order (OO) and Orientational Dispersion (OD) indices to describe the degree of alignment and dispersion of a spherical function in each voxel; (2) We estimate a local orthogonal coordinate frame in each voxel with anisotropic diffusion; (3) Finally, we define three indices to describe three types of orientational distortion (splay, bend, and twist) in a local spatial neighborhood, and a total distortion index to describe distortions of all three types. To our knowledge, this is the first work to quantitatively describe orientational distortion (splay, bend, and twist) in diffusion MRI. The proposed scalar indices are useful to detect local geometric changes of white matter for voxel-based or tract-based analysis in both DTI and HARDI acquisitions.

Diffusion compartment imaging (DCI) characterizes tissues in vivo by separately modeling the diffusion signal arising from a finite number of large scale microstructural environments in each voxel, also referred to as compartments. The DIAMOND model has recently been proposed to characterize the 3-D diffusivity of each compartment using a statistical distribution of diffusion tensors. It enabled the evaluation of compartment-specific diffusion characteristics while also accounting for the intra-compartment heterogeneity. In its original formulation, however, DIAMOND could only describe symmetric heterogeneity, while tissue heterogeneity likely differs along and perpendicular to the orientation of the fascicles. In this work we propose a new statistical distribution model able to decouple axial and radial heterogeneity of each compartment in each voxel. We derive the corresponding analytical expression of the diffusion attenuated signal and evaluate this new approach with both numerical simulations and in vivo data. We show that the heterogeneity arising from white matter fascicles is anisotropic and that the shape of the distribution is sensitive to changes in axonal dispersion and axonal radius heterogeneity. We demonstrate that decoupling the modeling of axial and radial heterogeneity has a substantial impact of the estimated heterogeneity, enables improved estimation of other model parameters and enables improved signal prediction. Our distribution model characterizes not only the orientation of each white matter fascicle but also their diffusivities; it may enable unprecedented characterization of the brain development and of brain disease and injury.

High angular resolution diffusion imaging (HARDI) at higher b values leads to signal measurements having (exponentially) lower magnitudes, a strong Rician bias, and more corruptions from artifacts. Typical undersampled-HARDI reconstruction methods assume Gaussian noise models and limited/no regularization, leading to underestimated tract anisotropy and reduced ability to detect crossings. We propose novel Bayesian frameworks to model Rician statistics during dictionary learning and reconstruction. For dictionary learning, we propose edge-preserving smoothness priors on dictionary atoms. For reconstruction, we employ sparsity-based multiscale smoothness priors on the reconstructed image. In both frameworks, we propose kernel-based non-local regularization on dictionary coefficients and stronger sparsity via quasi norms. The results show improved dictionaries and reconstructions, over the state of the art.

Diffusion magnetic resonance imaging (dMRI) provides a unique tool for noninvasively probing the microstructure of the neuronal tissue. The NODDI model has been a popular approach to the estimation of tissue microstructure in many neuroscience studies. It represents the diffusion signals with three types of diffusion in tissue: intra-cellular, extra-cellular, and cerebrospinal fluid compartments. However, the original NODDI method uses a computationally expensive procedure to fit the model and could require a large number of diffusion gradients for accurate microstructure estimation, which may be impractical for clinical use. Therefore, efforts have been devoted to efficient and accurate NODDI microstructure estimation with a reduced number of diffusion gradients. In this work, we propose a deep network based approach to the NODDI microstructure estimation, which is named Microstructure Estimation using a Deep Network (MEDN). Motivated by the AMICO algorithm which accelerates the computation of NODDI parameters, we formulate the microstructure estimation problem in a dictionary-based framework. The proposed network comprises two cascaded stages. The first stage resembles the solution to a dictionary-based sparse reconstruction problem and the second stage computes the final microstructure using the output of the first stage. The weights in the two stages are jointly learned from training data, which is obtained from training dMRI scans with diffusion gradients that densely sample the q-space. The proposed method was applied to brain dMRI scans, where two shells each with 30 gradient directions (60 diffusion gradients in total) were used. Estimation accuracy with respect to the gold standard was measured and the results demonstrate that MEDN outperforms the competing algorithms.

Diffusion-weighted magnetic resonance imaging (MRI) provides a unique approach to understand the geometric structure of brain fiber bundles and to delineate the diffusion properties across subjects and time. It can be used to identify structural connectivity abnormalities and helps to diagnose brain-related disorders. The aim of this paper is to develop a novel, robust, and efficient dimensional reduction and regression framework, called hierarchical functional principal regression model (HFPRM), to effectively correlate high-dimensional fiber bundle statistics with a set of predictors of interest, such as age, diagnosis status, and genetic markers. The three key novelties of HFPRM include the simultaneous analysis of a large number of fiber bundles, the disentanglement of global and individual latent factors that characterizes between-tract correlation patterns, and a bi-level analysis on the predictor effects. Simulations are conducted to evaluate the finite sample performance of HFPRM. We have also applied HFPRM to a genome-wide association study to explore important genetic variants in neonatal white matter development.

Elastic physical models have been widely used to regularize deformations in different medical object registration tasks. Traditional approaches usually assume uniform isotropic tissue elasticity (a constant regularization weight) across the whole domain, which contradicts human tissue elasticity being not only inhomogeneous but also anisotropic. We focus on producing more physically realistic deformations for the task of surface registration. We model the surface as an orthotropic elastic thin shell, and we propose a novel statistical framework to estimate inhomogeneous and anisotropic shell elasticity parameters only from a group of known surface deformations. With this framework we show that a joint estimation of within-patient surface deformations and the shell elasticity parameters can improve groupwise registration accuracy. The method is tested in the context of endoscopic reconstruction-surface registration.

Accurate estimation of regional wall thicknesses (RWT) of left ventricular (LV) myocardium from cardiac MR sequences is of significant importance for identification and diagnosis of cardiac disease. Existing RWT estimation still relies on segmentation of LV myocardium, which requires strong prior information and user interaction. No work has been devoted into direct estimation of RWT from cardiac MR images due to the diverse shapes and structures for various subjects and cardiac diseases, as well as the complex regional deformation of LV myocardium during the systole and diastole phases of the cardiac cycle. In this paper, we present a newly proposed Residual Recurrent Neural Network (ResRNN) that fully leverages the spatial and temporal dynamics of LV myocardium to achieve accurate frame-wise RWT estimation. Our ResRNN comprises two paths: (1) a feed forward convolution neural network (CNN) for effective and robust CNN embedding learning of various cardiac images and preliminary estimation of RWT from each frame itself independently, and (2) a recurrent neural network (RNN) for further improving the estimation by modeling spatial and temporal dynamics of LV myocardium. For the RNN path, we design for cardiac sequences a Circle-RNN to eliminate the effect of null hidden input for the first time-step. Our ResRNN is capable of obtaining accurate estimation of cardiac RWT with Mean Absolute Error of 1.44 mm (less than 1-pixel error) when validated on cardiac MR sequences of 145 subjects, evidencing its great potential in clinical cardiac function assessment.

Multi-class segmentation is a crucial step in cell image analysis. This process becomes challenging when little information is available for recognising cells from the background, due to their poor discriminative features. To alleviate this, directed acyclic graphs such as trees have been proposed to model top-down statistical dependencies as a prior for improved image segmentation. However, using trees, modelling the relations between labels of multiple classes becomes difficult. To overcome this limitation, we propose a polytree graphical model that captures label proximity relations more naturally compared to tree based approaches. A novel recursive mechanism based on two-pass message passing is developed to efficiently calculate closed form posteriors of graph nodes on the polytree. The algorithm is evaluated using simulated data, synthetic images and real fluorescence microscopy images. Our method achieves Dice scores of 94.5% and 98% on macrophage and seed classes, respectively, outperforming GMM based classifiers.

The Cobb angle that quantitatively evaluates the spinal curvature plays an important role in the scoliosis diagnosis and treatment. Conventional measurement of these angles suffers from huge variability and low reliability due to intensive manual intervention. However, since there exist high ambiguity and variability around boundaries of vertebrae, it is challenging to obtain Cobb angles automatically. In this paper, we formulate the estimation of the Cobb angles from spinal X-rays as a multi-output regression task. We propose structured support vector regression (S$$^2$$VR) to jointly estimate Cobb angles and landmarks of the spine in X-rays in one single framework. The proposed S$$^2$$VR can faithfully handle the nonlinear relationship between input images and quantitative outputs, while explicitly capturing the intrinsic correlation of outputs. We introduce the manifold regularization to exploit the geometry of the output space. We propose learning the kernel in S$$^2$$VR by kernel alignment to enhance its discriminative ability. The proposed method is evaluated on the spinal X-rays dataset of 439 scoliosis subjects, which achieves the inspiring correlation coefficient of $$92.76\%$$ with ground truth obtained manually by human experts and outperforms two baseline methods. Our method achieves the direct estimation of Cobb angles with high accuracy, indicating its great potential in clinical use.

Brain imaging genetics attracts more and more attention since it can reveal associations between genetic factors and the structures or functions of human brain. Sparse canonical correlation analysis (SCCA) is a powerful bi-multivariate association identification technique in imaging genetics. There have been many SCCA methods which could capture different types of structured imaging genetic relationships. These methods either use the group lasso to recover the group structure, or employ the graph/network guided fused lasso to find out the network structure. However, the group lasso methods have limitation in generalization because of the incomplete or unavailable prior knowledge in real world. The graph/network guided methods are sensitive to the sign of the sample correlation which may be incorrectly estimated. We introduce a new SCCA model using a novel graph guided pairwise group lasso penalty, and propose an efficient optimization algorithm. The proposed method has a strong upper bound for the grouping effect for both positively and negatively correlated variables. We show that our method performs better than or equally to two state-of-the-art SCCA methods on both synthetic and real neuroimaging genetics data. In particular, our method identifies stronger canonical correlations and captures better canonical loading profiles, showing its promise for revealing biologically meaningful imaging genetic associations.

This paper presents an efficient algorithm for large deformation diffeomorphic metric mapping (LDDMM) with geodesic shooting for image registration. We introduce a novel finite dimensional Fourier representation of diffeomorphic deformations based on the key fact that the high frequency components of a diffeomorphism remain stationary throughout the integration process when computing the deformation associated with smooth velocity fields. We show that manipulating high dimensional diffeomorphisms can be carried out entirely in the bandlimited space by integrating the nonstationary low frequency components of the displacement field. This insight substantially reduces the computational cost of the registration problem. Experimental results show that our method is significantly faster than the state-of-the-art diffeomorphic image registration methods while producing equally accurate alignment. We demonstrate our algorithm in two different applications of image registration: neuroimaging and in-utero imaging.

In the study of shapes of human organs using computational anatomy, variations are found to arise from inter-subject anatomical differences, disease-specific effects, and measurement noise. This paper introduces a stochastic model for incorporating random variations into the Large Deformation Diffeomorphic Metric Mapping (LDDMM) framework. By accounting for randomness in a particular setup which is crafted to fit the geometrical properties of LDDMM, we formulate the template estimation problem for landmarks with noise and give two methods for efficiently estimating the parameters of the noise fields from a prescribed data set. One method directly approximates the time evolution of the variance of each landmark by a finite set of differential equations, and the other is based on an Expectation-Maximisation algorithm. In the second method, the evaluation of the data likelihood is achieved without registering the landmarks, by applying bridge sampling using a stochastically perturbed version of the large deformation gradient flow algorithm. The method and the estimation algorithms are experimentally validated on synthetic examples and shape data of human corpora callosa.

Many nonlinear registration algorithms are subject to an asymmetry with respect to the order of image inputs. Often, one image is considered the moving image while the other is fixed. Hence, the moving image is subject to additional interpolation relative to the fixed image. Further, the fixed image is in a way represented by the deformed moving image; any noise or artifacts present in the moving image are thus retained in this representation. This asymmetry has even been shown to result in bias in various forms of registration derived measurements. These problems are particularly evident in the geodesic shooting in diffeomorphisms context, where a continuous time geodesic model of image deformation is on the orbit of the moving image. Were the images input in the opposite order, the model would lie on the orbit of the other image. This paper presents a symmetrical formulation of the geodesic shooting in diffeomorphisms model with an accompanying algorithm that treats the intensity and gradient information in both images in nearly an equal way. After formulating the algorithm, we validate in a set of longitudinal 3D brain MRI pairs that the transformations the symmetrical algorithm produces are indeed significantly more robust to switching the order of image inputs than traditional geodesic shooting.

Significant advances have been made towards building accurate automatic segmentation systems for a variety of biomedical applications using machine learning. However, the performance of these systems often degrades when they are applied on new data that differ from the training data, for example, due to variations in imaging protocols. Manually annotating new data for each test domain is not a feasible solution. In this work we investigate unsupervised domain adaptation using adversarial neural networks to train a segmentation method which is more robust to differences in the input data, and which does not require any annotations on the test domain. Specifically, we derive domain-invariant features by learning to counter an adversarial network, which attempts to classify the domain of the input data by observing the activations of the segmentation network. Furthermore, we propose a multi-connected domain discriminator for improved adversarial training. Our system is evaluated using two MR databases of subjects with traumatic brain injuries, acquired using different scanners and imaging protocols. Using our unsupervised approach, we obtain segmentation accuracies which are close to the upper bound of supervised domain adaptation.

Manual delineations are of paramount importance in medical imaging, for instance to train supervised methods and evaluate automatic segmentation algorithms. In volumetric images, manually tracing regions of interest is an excruciating process in which much time is wasted labeling neighboring 2D slices that are similar to each other. Here we present a method to compute a set of discrete minimal surfaces whose boundaries are specified by user-provided segmentations on one or more planes. Using this method, the user can for example manually delineate one slice every n and let the algorithm complete the segmentation for the slices in between. Using a discrete framework, this method globally minimizes a cost function that combines a regularizer with a data term based on image intensities, while ensuring that the surfaces do not intersect each other or leave holes in between. While the resulting optimization problem is an integer program and thus NP-hard, we show that the equality constraint matrix is totally unimodular, which enables us to solve the linear program (LP) relaxation instead. We can then capitalize on the existence of efficient LP solvers to compute a globally optimal solution in practical times. Experiments on two different datasets illustrate the superiority of the proposed method over the use of independent, label-wise optimal surfaces ($$\sim $$5% mean increase in Dice when one every six slices is labeled, with some structures improving up to $$\sim $$10% in Dice).

Encouraged by the success of CNNs in classification problems, CNNs are being actively applied to image-wide prediction problems such as segmentation, optic flow, reconstruction, restoration etc. These approaches fall under the category of fully convolutional networks [FCN] and have been very successful in bringing contexts into learning for image analysis. In this work, we address the problem of segmentation from medical images. Segmentation or object delineation from medical images/volumes is a fundamental step for subsequent quantification tasks key to diagnosis. Semantic segmentation has been popularly addressed using FCN (e.g. U-NET) with impressive results and has been the fore runner in recent segmentation challenges. However, there are a few drawbacks of FCN approaches which recent works have tried to address. Firstly, local geometry such as smoothness and shape are not reliably captured. Secondly, spatial context captured by FCNs while giving the advantage of a richer representation carries the intrinsic drawback of overfitting, and is quite sensitive to appearance and shape changes. To handle above issues, in this work, we propose a hybrid of generative modeling of image formation to jointly learn the triad of foreground (F), background (B) and shape (S). Such generative modeling of F, B, S would carry the advantages of FCN in capturing contexts. Further we expect the approach to be useful under limited training data, results easy to interpret, and enable easy transfer of learning across segmentation problems. We present $${\sim }8\%$$ improvement over state of art FCN approaches for US kidney segmentation and while achieving comparable results on CT lung nodule segmentation.

Automatic localization and labeling of vertebra in 3D medical images plays an important role in many clinical tasks, including pathological diagnosis, surgical planning and postoperative assessment. However, the unusual conditions of pathological cases, such as the abnormal spine curvature, bright visual imaging artifacts caused by metal implants, and the limited field of view, increase the difficulties of accurate localization. In this paper, we propose an automatic and fast algorithm to localize and label the vertebra centroids in 3D CT volumes. First, we deploy a deep image-to-image network (DI2IN) to initialize vertebra locations, employing the convolutional encoder-decoder architecture together with multi-level feature concatenation and deep supervision. Next, the centroid probability maps from DI2IN are iteratively evolved with the message passing schemes based on the mutual relation of vertebra centroids. Finally, the localization results are refined with sparsity regularization. The proposed method is evaluated on a public dataset of 302 spine CT volumes with various pathologies. Our method outperforms other state-of-the-art methods in terms of localization accuracy. The run time is around 3 seconds on average per case. To further boost the performance, we retrain the DI2IN on additional 1000+ 3D CT volumes from different patients. To the best of our knowledge, this is the first time more than 1000 3D CT volumes with expert annotation are adopted in experiments for the anatomic landmark detection tasks. Our experimental results show that training with such a large dataset significantly improves the performance and the overall identification rate, for the first time by our knowledge, reaches 90%.

The acquisition of Magnetic Resonance Imaging (MRI) is inherently slow. Inspired by recent advances in deep learning, we propose a framework for reconstructing MR images from undersampled data using a deep cascade of convolutional neural networks to accelerate the data acquisition process. We show that for Cartesian undersampling of 2D cardiac MR images, the proposed method outperforms the state-of-the-art compressed sensing approaches, such as dictionary learning-based MRI (DLMRI) reconstruction, in terms of reconstruction error, perceptual quality and reconstruction speed for both 3-fold and 6-fold undersampling. Compared to DLMRI, the error produced by the method proposed is approximately twice as small, allowing to preserve anatomical structures more faithfully. Using our method, each image can be reconstructed in 23 ms, which is fast enough to enable real-time applications.

We present an algorithm for creating high resolution anatomically plausible images consistent with acquired clinical brain MRI scans with large inter-slice spacing. Although large databases of clinical images contain a wealth of information, medical acquisition constraints result in sparse scans that miss much of the anatomy. These characteristics often render computational analysis impractical as standard processing algorithms tend to fail when applied to such images. Highly specialized or application-specific algorithms that explicitly handle sparse slice spacing do not generalize well across problem domains. In contrast, our goal is to enable application of existing algorithms that were originally developed for high resolution research scans to significantly undersampled scans. We introduce a model that captures fine-scale anatomical similarity across subjects in clinical image collections and use it to fill in the missing data in scans with large slice spacing. Our experimental results demonstrate that the proposed method outperforms current upsampling methods and promises to facilitate subsequent analysis not previously possible with scans of this quality.

The analysis of vessel morphology and connectivity has an impact on a number of cardiovascular and neurovascular applications by providing patient-specific high-level quantitative features such as spatial location, direction and scale. In this paper we present an end-to-end approach to extract an acyclic vascular tree from angiographic data by solving a connectivity-enforcing anisotropic fast marching over a voxel-wise tensor field representing the orientation of the underlying vascular tree. The method is validated using synthetic and real vascular images. We compare VTrails against classical and state-of-the-art ridge detectors for tubular structures by assessing the connectedness of the vesselness map and inspecting the synthesized tensor field as proof of concept. VTrails performance is evaluated on images with different levels of degradation: we verify that the extracted vascular network is an acyclic graph (i.e. a tree), and we report the extraction accuracy, precision and recall.