Medical Image Computing and Computer-Assisted Intervention – MICCAI 2013

Minimally invasive laparoscopic surgery is widely used for the treatment of cancer and other diseases. During the procedure, gas insufflation is used to create space for laparoscopic tools and operation. Insufflation causes the organs and abdominal wall to deform significantly. Due to this large deformation, the benefit of surgical plans, which are typically based on pre-operative images, is limited for real time navigation. In some recent work, intra-operative images, such as cone-beam CT or interventional CT, are introduced to provide updated volumetric information after insufflation. Other works in this area have focused on simulation of gas insufflation and exploited only the pre-operative images to estimate deformation. This paper proposes a novel registration method for pre- and intra-operative 3D image fusion for laparoscopic surgery. In this approach, the deformation of pre-operative images is driven by a biomechanical model of the insufflation process. The proposed method was validated by five synthetic data sets generated from clinical images and three pairs of in vivo CT scans acquired from two pigs, before and after insufflation. The results show the proposed method achieved high accuracy for both the synthetic and real insufflation data.

This paper introduces a novel method for inferring spatially varying regularisation in non-rigid registration. This is achieved through full Bayesian inference on a probabilistic registration model, where the prior on transformations is parametrised as a weighted mixture of spatially localised components. Such an approach has the advantage of allowing the registration to be more flexibly driven by the data than a more traditional global regularisation scheme, such as bending energy. The proposed method adaptively determines the influence of the prior in a local region. The importance of the prior may be reduced in areas where the data better supports deformations, or can enforce a stronger constraint in less informative areas. Consequently, the use of such a spatially adaptive prior may reduce the unwanted impact of regularisation on the inferred deformation field. This is especially important for applications such as tensor based morphometry, where the features of interest are directly derived from the deformation field. The proposed approach is demonstrated with application to tensor based morphometry analysis of subjects with Alzheimer’s disease and healthy controls. The results show that using the proposed spatially adaptive prior leads to deformation fields that have a substantially lower average complexity, but which also provide more accurate localisation of statistical group differences.

Establishing correspondence points across a set of biomedical shapes is an important technology for a variety of applications that rely on statistical analysis of individual subjects and populations. The inherent complexity (e.g. cortical surface shapes) and variability (e.g. cardiac chambers) evident in many biomedical shapes introduce significant challenges in finding a useful set of dense correspondences. Application specific strategies, such as registration of simplified (e.g. inflated or smoothed) surfaces or relying on manually placed landmarks, provide some improvement but suffer from limitations including increased computational complexity and ambiguity in landmark placement. This paper proposes a method for dense point correspondence on shape ensembles using geodesic distances to a priori landmarks as features. A novel set of numerical techniques for fast computation of geodesic distances to point sets is used to extract these features. The proposed method minimizes the ensemble entropy based on these features, resulting in isometry invariant correspondences in a very general, flexible framework.

We seek to compute a diffeomorphic map between a pair of diffusion-weighted images under large deformation. Unlike existing techniques, our method allows

any

diffusion model to be fitted after registration for subsequent multifaceted analysis. This is achieved by directly aligning the diffusion-weighted images using a large deformation diffeomorphic registration framework formulated from an optimal control perspective. Our algorithm seeks the optimal coordinate mapping by simultaneously considering structural alignment, local fiber reorientation, and deformation regularization. Our algorithm also incorporates a multi-kernel strategy to concurrently register anatomical structures of different scales. We demonstrate the efficacy of our approach using

in vivo

data and report on detailed qualitative and quantitative results in comparison with several different registration strategies.

A novel dynamic (4D) PET to PET image registration procedure is proposed and applied to multiple PET scans acquired with the high resolution research tomograph (HRRT), the highest resolution human brain PET scanner available in the world. By extending the recent diffeomorphic log-demons (DLD) method and applying it to multiple dynamic [

11

C]raclopride scans from the HRRT, an important step towards construction of a PET atlas of unprecedented quality for [

11

C]raclopride imaging of the human brain has been achieved. Accounting for the temporal dimension in PET data improves registration accuracy when compared to registration of 3D to 3D time-averaged PET images. The DLD approach was chosen for its ease in providing both an intensity and shape template, through iterative sequential pair-wise registrations with fast convergence. The proposed method is applicable to any PET radiotracer, providing 4D atlases with useful applications in high accuracy PET data simulations and automated PET image analysis.

We develop a random walk-based image registration method [1] that incorporates two novelties: 1) a progressive optimization scheme that conducts the solution search efficiently via a novel use of information derived from the obtained probabilistic solution, and 2) a data-likelihood re-weighting step that contextually performs feature selection in a spatially adaptive manner so that the data costs are based primarily on trusted information sources. Synthetic experiments on three public datasets of different anatomical regions and modalities showed that our method performed efficient search without sacrificing registration accuracy. Experiments performed on 60 real brain image pairs from a public dataset also demonstrated our method’s better performance over existing non-probabilistic image registration methods.

A conclusive mapping of myeloarchitecture (myelin patterns) onto the cortical sheet and, thus, a corresponding mapping to cytoarchitecture (cell configuration) does not exist today. In this paper we present a generative model which can predict, on the basis of known cytoarchitecture, myeloarchitecture in different primary and non-primary cortical areas, resulting in simulated in-vivo quantitative T1 maps. The predicted patterns can be used in brain parcellation. Our model is validated using a similarity distance metric which enables quantitative comparison of the results with empirical data measured using MRI. The work presented may provide new perspectives for this line of research, both in imaging and in modelling the relationship with myelo- and cytoarchitecture, thus leading the way towards in-vivo histology using MRI.

This paper proposes a vision-based method for detecting apoptosis (programmed cell death), which is essential for non-perturbative monitoring of cell expansion. Our method targets non-adherent cells, which float or are suspended freely in the culture medium—in contrast to adherent cells, which are attached to a petri dish. The method first detects cell regions and tracks them over time, resulting in the construction of cell tracklets. For each of the tracklets, visual properties of the cell are then examined to know whether and when the tracklet shows a transition from a live cell to a dead cell, in order to determine the occurrence and timing of a cell death event. For the validation, a transductive learning framework is adopted to utilize unlabeled data in addition to labeled data. Our method achieved promising performance in the experiments with hematopoietic stem cell (HSC) populations, which are currently in clinical use for rescuing hematopoietic function during bone marrow transplants.

Recent advances in microscopic detection techniques include fluorescence spectroscopy, fibred confocal microscopy and optical coherence tomography. These methods can be integrated with miniaturised probes to assist endoscopy, thus enabling diseases to be detected at an early and pre-invasive stage, forgoing the need for histopathological samples and off-line analysis. Since optical-based biopsy does not leave visible marks after sampling, it is important to track the biopsy sites to enable accurate retargeting and subsequent serial examination. In this paper, a novel approach is proposed for pathological site retargeting in gastroscopic examinations. The proposed method is based on affine deformation modelling with geometrical association combined with cascaded online learning and tracking. It provides online

in vivo

retargeting, and is able to track pathological sites in the presence of tissue deformation. It is also robust to partial occlusions and can be applied to a range of imaging probes including confocal laser endomicroscopy.

For automatic segmentation of optic disc and cup from color fundus photograph, we describe a fairly general energy function that can naturally fit into a global optimization framework with graph cut. Distinguished from most previous work, our energy function includes priors on the shape & location of disc & cup, the rim thickness and the geometric interaction of “disc contains cup”. These priors together with the effective optimization of graph cut enable our algorithm to generate reliable and robust solutions. Our approach is able to outperform several state-of-the-art segmentation methods, as shown by a set of experimental comparisons with manual delineations and a series of results of correlations with the assessments of a merchant-provided software from Optical Coherence Tomography (OCT) regarding several cup and disc parameters.

Detection and segmentation of ovarian cancer metastases have great clinical impacts on women’s health. However, the random distribution and weak boundaries of metastases significantly complicate this task. This paper presents a variational framework that combines region competition based level set propagation and image matching flow computation to jointly detect and segment metastases. Image matching flow not only detects metastases, but also creates shape priors to reduce over-segmentation. Accordingly, accurate segmentation helps to improve the detection accuracy by separating flow computation in metastasis and non-metastasis regions. Since all components in the image processing pipeline benefit from each other, our joint framework can achieve accurate metastasis detection and segmentation. Validation on 50 patient datasets demonstrated that our joint approach was superior to a sequential method with sensitivity 89.2% vs. 81.4% (Fisher exact test

p

 = 0.046) and false positive per patient 1.04 vs. 2.04. The Dice coefficient of metastasis segmentation was 92±5.2% vs. 72±8% (paired t-test

p

 = 0.022), and the average surface distance was 1.9±1.5mm vs. 4.5±2.2mm (paired t-test

p

 = 0.004).

Image-based classification of tissue histology, in terms of different components (e.g., subtypes of aberrant phenotypic signatures), provides a set of indices for tumor composition. Subsequently, integration of these indices in whole slide images (WSI), from a large cohort, can provide predictive models of the clinical outcome. However, the performance of the existing histology-based classification techniques is hindered as a result of large technical and biological variations that are always present in a large cohort. In this paper, we propose an algorithm for classification of tissue histology based on predictive sparse decomposition (PSD) and spatial pyramid matching (SPM), which utilize sparse tissue morphometric signatures at various locations and scales. The method has been evaluated on two distinct datasets of different tumor types collected from The Cancer Genome Atlas (TCGA). The novelties of our approach are: (i) extensibility to different tumor types; (ii) robustness in the presence of wide technical and biological variations; and (iii) scalability with varying training sample size.

Dynamic contrast-enhanced computed tomography (DCE-CT) is a valuable imaging modality to assess tissues properties, particularly in tumours, by estimating pharmacokinetic parameters from the evolution of pixels intensities in 3D+t acquisitions. However, this requires a registration of the whole sequence of volumes, which is challenging especially when the patient breathes freely. In this paper, we propose a generic, fast and automatic method to address this problem. As standard iconic registration methods are not robust to contrast intake, we rather rely on the segmentation of the organ of interest. This segmentation is performed jointly with the registration of the sequence within a novel co-segmentation framework. Our approach is based on implicit template deformation, that we extend to a co-segmentation algorithm which provides as outputs both a segmentation of the organ of interest in every image and stabilising transformations for the whole sequence. The proposed method is validated on 15 datasets acquired from patients with renal lesions and shows improvement in terms of registration and estimation of pharmacokinetic parameters over the state-of-the-art method.

In modern robotic radiation therapy, tumor movements due to respiration can be compensated. The accuracy of these methods can be increased by time series prediction of external optical surrogates. An algorithm based on relevance vector machines (RVM) is introduced. We evaluate RVM with linear and nonlinear basis functions on a real patient data set containing 304 motion traces and compare it with a wavelet based least mean square algorithm (wLMS), the best algorithm for this data set so far. Linear RVM outperforms wLMS significantly and increases the prediction accuracy for 80.3 % of the data. We show that real time prediction is possible in case of linear RVM and discuss how the predicted variance can be used to construct promising hybrid algorithms, which further reduce the prediction error.

A novel real-time multi-sensor framework for range imaging (RI) based respiratory motion analysis in image guided interventions such as fractionated radiation therapy is presented. We constitute our method based upon real-time constraints in clinical practice and an analytic analysis of RI based elastic body surface deformation fields. For the latter, we show that the underlying joint rigid and non-rigid registration problem is ill-conditioned and identify insufficient body coverage as an error source. Facing these issues, we propose a novel manifold ray casting technique enabling the reconstruction of an 180° coverage body surface model composed of ~3·10

5

points from volumetrically fused multi-view range data in ~25 ms. Exploiting the wide field of view surface model enabled by our method, we reduce the error in motion compensated patient alignment by a factor of 2.7 in the translational and 2.4 in the rotational component compared to conventional single sensor surface coverage.

Prostate motion due to transrectal ultrasound (TRUS) probe pressure and patient movement causes target misalignments during 3D TRUS-guided biopsy. Several solutions have been proposed to perform 2D-3D registration for motion compensation. To improve registration accuracy and robustness, we developed and evaluated a registration algorithm whose optimization is based on learned prostate motion characteristics relative to different tracked probe positions and prostate sizes. We performed a principal component analysis of previously observed motions and utilized the principal directions to initialize Powell’s direction set method during optimization. Compared with the standard initialization, our approach improved target registration error to 2.53±1.25 mm after registration. Multiple initializations along the major principal directions improved the robustness of the method at the cost of additional execution time of 1.5 s. With a total execution time of 3.2 s to perform motion compensation, this method is amenable to useful integration into a clinical 3D guided prostate biopsy workflow.

This study introduces a combination of two registration techniques for respiratory motion removal and the quantification of small bowel motility from free breathing cine MRI. The use of robust data decomposition registration (RDDR) allows for exclusive correction of respiratory motion in order to avoid errors in further analysis of motility due to the effects of breathing. The proposed method is assessed using regions of interest (ROIs) contoured in dynamic MRI of six healthy volunteers. The use of RDDR prior to motility quantification results in reduced errors on motility scores in ROIs, with respect to breath-holds.

The correct visualization of anatomical structures is a critical component of neurosurgical navigation systems, to guide the surgeon to the areas of interest as well as to avoid brain damage. A major challenge for neuronavigation systems is the brain shift, or deformation of the exposed brain in comparison to preoperative Magnetic Resonance (MR) image sets. In this work paper, a non-rigid deformation pipeline is proposed for brain shift compensation of preoperative imaging datasets using superficial blood vessels as landmarks. The input was preoperative and intraoperative 3D image sets of superficial vessel centerlines. The intraoperative vessels (obtained using 3 Near-Infrared cameras) were registered and aligned with preoperative Magnetic Resonance Angiography vessel centerlines using manual interaction for the rigid transformation and, for the non-rigid transformation, the non-rigid point set registration method Coherent Point Drift. The rigid registration transforms the intraoperative points from the camera coordinate system to the preoperative MR coordinate system, and the non-rigid registration deals with local transformations in the MR coordinate system. Finally, the generation of a new deformed volume is achieved with the Thin-Plate Spline (TPS) method using as control points the matches in the MR coordinate system found in the previous step. The method was tested in a rabbit brain exposed via craniotomy, where deformations were produced by a balloon inserted into the brain. There was a good correlation between the real state of the brain and the deformed volume obtained using the pipeline. Maximum displacements were approximately 4.0 mm for the exposed brain alone, and 6.7 mm after balloon inflation.

We propose an effective 4D image registration algorithm for dynamic volumetric lung images. The registration will construct a deforming 3D model with continuous trajectory and smooth spatial deformation, and the model interpolates the interested region in the 4D (3D+T) CT images. The resultant non-rigid transformation is represented using two 4D B-spline functions, indicating a forward and an inverse 4D parameterization respectively. The registration process solves these two functions by minimizing an objective function that penalizes intensity matching error, feature alignment error, spatial and temporal non-smoothness, and inverse inconsistency. We test our algorithm for respiratory motion estimation on public benchmarks and on clinic lung CT data. The experimental results demonstrate the efficacy of our algorithm.

Organ segmentation is a challenging problem on which significant progress has been made. Deformable models (DM) and graphical models (GM) are two important categories of optimization based image segmentation methods. Efforts have been made on integrating two types of models into one framework. However, previous methods are not designed for segmenting multiple organs simultaneously and accurately. In this paper, we propose a hybrid

multi organ

segmentation approach by integrating DM and GM in a coupled optimization framework. Specifically, we show that region-based deformable models can be integrated with Markov Random Fields (MRF), such that multiple models’ evolutions are driven by a maximum a posteriori (MAP) inference. It brings global and local deformation constraints into a unified framework for simultaneous segmentation of multiple objects in an image. We validate this proposed method on two challenging problems of multi organ segmentation, and the results are promising.

This paper presents an automated multi-organ segmentation method for 3D abdominal CT images based on a spatially-divided probabilistic atlases. Most previous abdominal organ segmentation methods are ineffective to deal with the large differences among patients in organ shape and position in local areas. In this paper, we propose an automated multi-organ segmentation method based on a spatially-divided probabilistic atlas, and solve this problem by introducing a scale hierarchical probabilistic atlas. The algorithm consists of image-space division and a multi-scale weighting scheme. The generated spatial-divided probabilistic atlas efficiently reduces the inter-subject variance in organ shape and position either in global or local regions. Our proposed method was evaluated using 100 abdominal CT volumes with manually traced ground truth data. Experimental results showed that it can segment the liver, spleen, pancreas, and kidneys with Dice similarity indices of 95.1%, 91.4%, 69.1%, and 90.1%, respectively.

Delineation of the prostate from transrectal ultrasound images is a necessary step in several computer-assisted clinical interventions, such as low dose rate brachytherapy. Current approaches to user segmentation require user intervention and therefore it is subject to user errors. It is desirable to have a fully automatic segmentation for improved segmentation consistency and speed. In this paper, we propose a multi-atlas fusion framework to automatically segment prostate transrectal ultrasound images. The framework initially registers a dataset of a priori segmented ultrasound images to a target image. Subsequently, it uses the pairwise similarity of registered prostate shapes, which is independent of the image-similarity metric optimized during the registration process, to prune the dataset prior to the fusion and consensus segmentation step. A leave-one-out cross-validation of the proposed framework on a dataset of 50 transrectal ultrasound volumes obtained from patients undergoing brachytherapy treatment shows that the proposed is clinically robust, accurate and reproducible.

Recent work has shown that using Random Forests (RFs) to vote for the optimal position of model feature points leads to robust and accurate shape model matching. This paper applies RF regression-voting as part of a fully automatic shape model matching (FASMM) system to three different radiograph segmentation problems: the proximal femur, the bones of the knee joint and the joints of the hand. We investigate why this approach works so well and demonstrate that the performance comes from a combination of three properties:

(i)

The integration of votes from multiple regions around the model point.

(ii)

The combination of multiple independent votes from each tree.

(iii)

The use of a coarse to fine strategy. We show that each property can improve performance, and that the best performance comes from using all three. We demonstrate that FASMM based on RF regression-voting generalises well across application areas, achieving state of the art performance in each of the three segmentation problems. This FASMM system provides an accurate and time-efficient way for the segmentation of bony structures in radiographs.

Segmentation of the femur and pelvis is a prerequisite for patient-specific planning and simulation for hip surgery. Accurate boundary determination of the femoral head and acetabulum is the primary challenge in diseased hip joints because of deformed shapes and extreme narrowness of the joint space. To overcome this difficulty, we investigated a multi-stage method in which the hierarchical hip statistical shape model (SSM) is initially utilized to complete segmentation of the pelvis and distal femur, and then the conditional femoral head SSM is used under the condition that the regions segmented during the previous stage are known. CT data from 100 diseased patients categorized on the basis of their disease type and severity, which included 200 hemi-hips, were used to validate the method, which delivered significantly increased segmentation accuracy for the femoral head.

We propose a novel global optimization approach to segmenting a given 3D prostate T2w magnetic resonance (MR) image, which enforces the inherent axial symmetry of the prostate shape and simultaneously performs a sequence of 2D axial slice-wise segmentations with a global 3D coherence prior. We show that the proposed challenging combinatorial optimization problem can be solved globally and exactly by means of convex relaxation. With this regard, we introduce a novel coupled continuous max-flow model, which is dual to the studied convex relaxed optimization formulation and leads to an efficient multiplier augmented algorithm based on the modern convex optimization theory. Moreover, the new continuous max-flow based algorithm was implemented on GPUs to achieve a substantial improvement in computation. Experimental results using public and in-house datasets demonstrate great advantages of the proposed method in terms of both accuracy and efficiency.

Volumetric image segmentation methods often produce delineations of anatomical structures and pathologies that require user modifications. We present a new method for the correction of segmentation errors. Given an initial geometrical mesh, our method semi automatically identifies the mesh vertices in erroneous regions with min-cut segmentation. It then deforms the mesh by correcting its vertex coordinates with Laplace deformation based on local geometrical properties. The key advantages of our method are that: 1) it supports fast user interaction on a single surface rendered 2D view; 2) its parameters values are fixed to the same value for all cases; 3) it is independent of the initial segmentation method, and; 4) it is applicable to a variety of anatomical structures and pathologies. Experimental evaluation on 44 initial segmentations of kidney and kidney vessels from CT scans show an improvement of 83% and 75% in the average surface distance and the volume overlap error between the initial and the corrected segmentations with respect to the ground-truth.

Our proposed method combines semi supervised learning (SSL) and active learning (AL) for automatic detection and segmentation of Crohn’s disease (CD) from abdominal magnetic resonance (MR) images. Random forest (RF) classifiers are used due to fast SSL classification and capacity to interpret learned knowledge. Query samples for AL are selected by a novel information density weighted approach using context information, semantic knowledge and labeling uncertainty. Experimental results show that our proposed method combines the advantages of SSL and AL, and with fewer samples achieves higher classification and segmentation accuracy over fully supervised methods.

Conventional statistical shape models use Principal Component Analysis (PCA) to describe shape variations. However, such a PCA-based model assumes a Gaussian distribution of data. A model with Independent Component Analysis (ICA) does not require the Gaussian assumption and can additionally describe the local shape variation. In this paper, we propose a Hierarchical Constrained Local Model (HCLM) using ICA. The first or coarse level of HCLM locates the full landmark set, while the second level refines a relevant landmark subset. We then apply the HCLM to Down syndrome detection from photographs of young pediatric patients. Down syndrome is the most common chromosomal condition and its early detection is crucial. After locating facial anatomical landmarks using HCLM, geometric and local texture features are extracted and selected. A variety of classifiers are evaluated to identify Down syndrome from a healthy population. The best performance achieved 95.6% accuracy using support vector machine with radial basis function kernel. The results show that the ICA-based HCLM outperformed both PCA-based CLM and ICA-based CLM.

In the field of computer aided medical image analysis, it is often difficult to obtain reliable ground truth for evaluating algorithms or supervising statistical learning procedures. In this paper we present a new method for training a classification forest from images labelled by variably performing experts, while simultaneously evaluating the performance of each expert. Our approach builds upon state-of-the-art randomized classification forest techniques for medical image segmentation and recent methods for the fusion of multiple expert decisions. By incorporating the performance evaluation within the training phase, we obtain a novel forest framework for learning from conflicting expert decisions, accounting for both inter- and intra-expert variability. We demonstrate on a synthetic example that our method allows to retrieve the correct segmentation among other incorrectly labelled images, and we present an application to the automatic segmentation of the midbrain in 3D transcranial ultrasound images.

Quantitative histomorphometry is the process of modeling appearance of disease morphology on digitized histopathology images via image–based features (e.g., texture, graphs). Due to the

curse of dimensionality

, building classifiers with large numbers of features requires feature selection (which may require a large training set) or dimensionality reduction (DR). DR methods map the original high–dimensional features in terms of eigenvectors and eigenvalues, which limits the potential for feature transparency or interpretability. Although methods exist for variable selection and ranking on embeddings obtained via linear DR schemes (e.g., principal components analysis (PCA)), similar methods do not yet exist for nonlinear DR (NLDR) methods. In this work we present a simple yet elegant method for approximating the mapping between the data in the original feature space and the transformed data in the kernel PCA (KPCA) embedding space; this mapping provides the basis for quantification of variable importance in nonlinear kernels (VINK). We show how VINK can be implemented in conjunction with the popular Isomap and Laplacian eigenmap algorithms. VINK is evaluated in the contexts of three different problems in digital pathology: (1) predicting five year PSA failure following radical prostatectomy, (2) predicting Oncotype DX recurrence risk scores for ER+ breast cancers, and (3) distinguishing good and poor outcome p16+ oropharyngeal tumors. We demonstrate that subsets of features identified by VINK provide similar or better classification or regression performance compared to the original high dimensional feature sets.

Segmentation of anatomical structures in medical images is often based on a voxel/pixel classification approach. Deep learning systems, such as convolutional neural networks (CNNs), can infer a hierarchical representation of images that fosters categorization. We propose a novel system for voxel classification integrating three 2D CNNs, which have a one-to-one association with the

xy

,

yz

and

zx

planes of 3D image, respectively. We applied our method to the segmentation of tibial cartilage in low field knee MRI scans and tested it on 114 unseen scans. Although our method uses only 2D features at a single scale, it performs better than a state-of-the-art method using 3D multi-scale features. In the latter approach, the features and the classifier have been carefully adapted to the problem at hand. That we were able to get better results by a deep learning architecture that autonomously learns the features from the images is the main insight of this study.

Image representation plays an important role in medical image analysis. The key to the success of different medical image analysis algorithms is heavily dependent on how we represent the input data, namely features used to characterize the input image. In the literature, feature engineering remains as an active research topic, and many novel hand-crafted features are designed such as Haar wavelet, histogram of oriented gradient, and local binary patterns. However, such features are not designed with the guidance of the underlying dataset at hand. To this end, we argue that the most effective features should be designed in a learning based manner, namely representation learning, which can be adapted to different patient datasets at hand. In this paper, we introduce a deep learning framework to achieve this goal. Specifically, a stacked independent subspace analysis (ISA) network is adopted to learn the most effective features in a hierarchical and unsupervised manner. The learnt features are adapted to the dataset at hand and encode high level semantic anatomical information. The proposed method is evaluated on the application of automatic prostate MR segmentation. Experimental results show that significant segmentation accuracy improvement can be achieved by the proposed deep learning method compared to other state-of-the-art segmentation approaches.

Accurate localization and identification of vertebrae in spinal imaging is crucial for the clinical tasks of diagnosis, surgical planning, and post-operative assessment. The main difficulties for automatic methods arise from the frequent presence of abnormal spine curvature, small field of view, and image artifacts caused by surgical implants. Many previous methods rely on parametric models of appearance and shape whose performance can substantially degrade for pathological cases.

We propose a robust localization and identification algorithm which builds upon supervised classification forests and avoids an explicit parametric model of appearance. We overcome the tedious requirement for dense annotations by a semi-automatic labeling strategy. Sparse centroid annotations are transformed into dense probabilistic labels which capture the inherent identification uncertainty. Using the dense labels, we learn a discriminative centroid classifier based on local and contextual intensity features which is robust to typical characteristics of spinal pathologies and image artifacts. Extensive evaluation is performed on a challenging dataset of 224 spine CT scans of patients with varying pathologies including high-grade scoliosis, kyphosis, and presence of surgical implants. Additionally, we test our method on a heterogeneous dataset of another 200, mostly abdominal, CTs. Quantitative evaluation is carried out with respect to localization errors and identification rates, and compared to a recently proposed method. Our approach is efficient and outperforms state-of-the-art on pathological cases.

Today’s follow-up of patients presenting abdominal tumors is generally performed through acquisition of dynamic sequences of contrast-enhanced CT. Estimating parameters of appropriate models of contrast intake diffusion through tissues should help characterizing the tumor physiology, but is impeded by the high level of noise inherent to the acquisition conditions. To improve the quality of estimation, we consider parameter estimation in voxels as a multi-task learning problem (one task per voxel) that takes advantage from the similarity between two tasks. We introduce a temporal similarity between tasks based on a robust distance between observed contrast-intake profiles of intensity. Using synthetic images, we compare multi-task learning using this temporal similarity, a spatial similarity and a single-task learning. The similarities based on temporal profiles are shown to bring significant improvements compared to the spatial one. Results on real CT sequences also confirm the relevance of the approach.

This paper presents the results of an

in vivo

clinical study to accurately characterize prostate cancer using new features of ultrasound RF time series.

Methods:

The mean central frequency and wavelet features of ultrasound RF time series from seven patients are used along with an elaborate framework of ultrasound to histology registration to identify and verify cancer in prostate tissue regions as small as 1.7 mm × 1.7 mm.

Results:

In a leave-one-patient-out cross-validation strategy, an average classification accuracy of 76% and the area under ROC curve of 0.83 are achieved using two proposed RF time series features. The results statistically significantly outperform those achieved by previously reported features in the literature. The proposed features show the clinical relevance of RF time series for

in vivo

characterization of cancer.

Quantifying local changes to the airway wall surfaces from computed tomography images is important in the study of diseases such as chronic obstructive pulmonary disease. Current approaches segment the airways in the individual time point images and subsequently aggregate per airway generation or perform branch matching to assess regional changes. In contrast, we propose an integrated approach analysing the time points simultaneously using a subject-specific groupwise space and 4D optimal surface segmentation. The method combines information from all time points and measurements are matched locally at any position on the resulting surfaces.

Visual inspection of the scans of 10 subjects showed increased tree length compared to the state of the art with little change in the amount of false positives. A large scale analysis of the airways of 374 subjects including a total of 1870 images showed significant correlation with lung function and high reproducibility of the measurements.

Breast tumors are heterogeneous lesions. Intra-tumor heterogeneity presents a major challenge for cancer diagnosis and treatment. Few studies have worked on capturing tumor heterogeneity from imaging. Most studies to date consider aggregate measures for tumor characterization. In this work we capture tumor heterogeneity by partitioning tumor pixels into subregions and extracting heterogeneity wavelet kinetic (HetWave) features from breast dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) to obtain the spatiotemporal patterns of the wavelet coefficients and contrast agent uptake from each partition. Using a genetic algorithm for feature selection, and a logistic regression classifier with leave one-out cross validation, we tested our proposed HetWave features for the task of classifying breast cancer recurrence risk. The classifier based on our features gave an ROC AUC of 0.78, outperforming previously proposed kinetic, texture, and spatial enhancement variance features which give AUCs of 0.69, 0.64, and 0.65, respectively.

The accurate diagnosis of Alzheimer’s Disease (AD) and Mild Cognitive Impairment (MCI) is important in early dementia detection and treatment planning. Most of current studies formulate the AD diagnosis scenario as a classification problem and solve it using various machine learners trained with multi-modal biomarkers. However, the diagnosis accuracy is usually constrained by the performance of the machine learners as well as the methods of integrating the multi-modal data. In this study, we propose a novel diagnosis algorithm, the Multifold Bayesian Kernelization (MBK), which models the diagnosis process as a synthesis analysis of multi-modal biomarkers. MBK constructs a kernel for each biomarker that maximizes the local neighborhood affinity, and further evaluates the contribution of each biomarker based on a Bayesian framework. MBK adopts a novel diagnosis scheme that could infer the subject’s diagnosis by synthesizing the output diagnosis probabilities of individual biomarkers. The proposed algorithm, validated using multi-modal neuroimaging data from the ADNI baseline cohort with 85 AD, 169 MCI and 77 cognitive normal subjects, achieves significant improvements on all diagnosis groups compared to the state-of-the-art methods.

One of the main limitations of

l

1

-norm feature selection is that it focuses on estimating the target vector for each sample individually without considering relations with other samples. However, it’s believed that the geometrical relation among target vectors in the training set may provide useful information, and it would be natural to expect that the predicted vectors have similar geometric relations as the target vectors. To overcome these limitations, we formulate this as a graph-matching feature selection problem between a predicted graph and a target graph. In the predicted graph a node is represented by predicted vector that may describe regional gray matter volume or cortical thickness features, and in the target graph a node is represented by target vector that include class label and clinical scores. In particular, we devise new regularization terms in sparse representation to impose high-order graph matching between the target vectors and the predicted ones. Finally, the selected regional gray matter volume and cortical thickness features are fused in kernel space for classification. Using the ADNI dataset, we evaluate the effectiveness of the proposed method and obtain the accuracies of 92.17% and 81.57% in AD and MCI classification, respectively.

Capability of detecting causal or effective connectivity from resting-state functional magnetic resonance imaging (R-fMRI) is highly desirable for better understanding the cooperative nature of the brain. Effective connectivity provides specific dynamic temporal information of R-fMRI time series and reflects the directional causal influence of one brain region over another. These causal influences among brain regions are normally extracted based on the concept of Granger causality. Conventionally, the effective connectivity is inferred using multivariate autoregressive (MAR) modeling with default model order

q

= 1, considering low frequency fluctuation of R-fMRI time series. This assumption, although reduces the modeling complexity, does not guarantee the best fitting of R-fMRI time series at different brain regions. Instead of using the default model order, we propose to estimate the optimal model order based upon MAR order distribution to better characterize these causal influences at each brain region. Due to sparse nature of brain connectivity networks, an orthogonal least square (OLS) regression algorithm is incorporated to MAR modeling to minimize spurious effective connectivity. Effective connectivity networks inferred using the proposed optimal sparse MAR modeling are applied to Mild Cognitive Impairment (MCI) identification and obtained promising results, demonstrating the importance of using optimal causal relationships between brain regions for neurodegeneration disorder identification.

Anatomical changes like brain atrophy or growth are usually not homogeneous in space and across spatial scales, since they map differently depending on the anatomical structures. Thus, the accurate analysis of volume changes from medical images requires to reliably localize and distinguish the spatial changes occurring at different scales, from voxel to regional level. We propose here a framework for the sparse probabilistic scale-space analysis of volume changes encoded by deformations. Our framework is based on the Helmoltz decomposition of vector fields. By scale-space analysis of the scalar pressure map associated to the irrotational component of the deformation, we robustly identify the areas of maximal volume changes, and we define a consistent sparse decomposition of the irrotational component. We show the effectiveness of our framework in the challenging problem of detecting the progression of tumor growth, and in the group-wise analysis of the longitudinal atrophy in Alzheimer’s disease.

Measurements of myelination and indicators of myelination status in the preterm brain could be predictive of later neurological outcome. Quantitative imaging of myelin could thus serve to develop predictive biomarkers; however, accurate estimation of myelin content is difficult. In this work we show that measurement of the myelin water fraction (MWF) is achievable using widely available pulse sequences and state-of-the-art algorithmic modelling of the MR imaging. We show results of myelin water fraction measurement at both 30 (4 infants) and 40 (2 infants) weeks equivalent gestational age (EGA) and show that the spatial pattern of myelin is different between these ages. Furthermore we apply a multi-component fitting routine to multi-shell diffusion weighted data to show differences in neurite density and local spatial arrangement in grey and white matter. Finally we combine these results to investigate the relationships between the diffusion and myelin measurements to show that MWF in the preterm brain may be measured alongside multi-component diffusion characteristics using clinically feasible MR sequences.

We present a method for automatic struts detection and stent shape estimation in cross-sectional intravascular ultrasound images. A stent shape is first estimated through a comprehensive interpretation of the vessel morphology, performed using a supervised context-aware multi-class classification scheme. Then, the successive strut identification exploits both local appearance and the defined stent shape. The method is tested on 589 images obtained from 80 patients, achieving a F-measure of 74.1% and an averaged distance between manual and automatic struts of 0.10

mm

.

Epilepsy is a disorder of the brain that can lead to acute crisis and temporary loss of brain functions. Surgery is used to remove focal lesions that remain resistant to treatment. An accurate localization of epileptogenic lesions has a strong influence on the outcome of epilepsy surgery. Magnetic resonance imaging (MRI) is clinically used for lesion detection and treatment planning, mainly through simple visual analysis. However, visual inspection in MRI can be highly subjective and subtle 3D structural abnormalities are not always entirely removed during surgery. In this paper, we introduce a lesion abnormality score based on computerized comparison of the 3D texture properties between brain hemispheres in T1 MRI. Overlapping cubic texture blocks extracted from user–defined 3D regions of interest (ROI) are expressed in terms of energies of 3D steerable Riesz wavelets. The abnormality score is defined as the Hausdorff distance between the ROI and its corresponding contralateral region in the brain, both expressed as ensembles of blocks in the feature space. A classification based on the proposed score allowed an accuracy of 85% with 10 control subjects and 8 patients with epileptogenic lesions. The approach therefore constitutes a valuable tool for the objective pre–surgical evaluation of patients undergoing epilepsy surgery.

Accurate registration of ultrasound images to statistical shape models is a challenging problem in percutaneous spine injection procedures due to the typical imaging artifacts inherent to ultrasound. In this paper we propose a robust and accurate registration method that matches local phase bone features extracted from ultrasound images to a statistical shape model. The local phase information for enhancing the bone surfaces is obtained using a gradient energy tensor filter, which combines advantages of the monogenic scale-space and Gaussian scale-space filters, resulting in an improved simultaneous estimation of phase and orientation information. A novel statistical shape model was built by separating the pose statistics from the shape statistics. This model is then registered to the local phase bone surfaces using an iterative expectation maximization registration technique. Validation on 96

in vivo

clinical scans obtained from eight patients resulted in a root mean square registration error of 2 mm (SD: 0.4 mm), which is below the clinically acceptable threshold of 3.5 mm. The improvement achieved in registration accuracy using the new features was also significant (p < 0.05) compared to state of the art local phase image processing methods.

Despite rapid growth of robot assisted catheterization in recent years, most current platforms are based on master-slave designs with limited operator-robot collaborative control and automation. Under this setup, information concerning subject specific behavior and context-driven manoeuvre is not re-utilized for subsequent intervention. For endovascular catheterization, the robot itself is designed with little consideration of underlying skills and associated motion patterns. This paper proposes a learning-based approach for generating optimum motion trajectories from multiple demonstrations of a catheterization task such that it can be used for automating catheter motion within a collaborative setting. Motion models are generated from experienced manipulation of a catheterization procedure and replicated using a robotic catheter driver to assist inexperienced operators. Catheter tip motions of the automated approach are compared against the manual training sets for validating the proposed framework. The results show significant improvements in the quality of catheterization, which facilitate the design of hands-on collaborative robots that make full use of the natural skills of the operators.

Image-guided radiotherapy (IGRT) requires fast and accurate localization of prostate in treatment CTs, which is challenging due to low tissue contrast and large anatomical variations across patients. On the other hand, in IGRT workflow, a series of CT images is acquired from the same patient under treatment, which contains valuable patient-specific information yet is often neglected by previous works. In this paper, we propose a novel learning framework, namely incremental learning with selective memory (ILSM), to effectively learn the patient-specific appearance characteristics from these patient-specific images. Specifically, starting with a population-based discriminative appearance model, ILSM aims to “personalize” the model to fit patient-specific appearance characteristics. Particularly, the model is personalized with two steps,

backward pruning

that discards obsolete population-based knowledge, and

forward learning

that incorporates patient-specific characteristics. By effectively combining the patient-specific characteristics with the general population statistics, the incrementally learned appearance model can localize the prostate of the specific patient much more accurately. Validated on a large dataset (349 CT scans), our method achieved high localization accuracy (DSC ~0.87) in 4 seconds.

In prostate cancer radiotherapy the association between the dose distribution and the occurrence of undesirable side-effects is yet to be revealed. In this work a method to perform population analysis by comparing the dose distributions is proposed. The method is a tensor-based approach that generalises an existing method for 2D images and allows for the highlighting of over irradiated zones correlated with rectal bleeding after prostate cancer radiotherapy. Thus, the aim is to contribute to the elucidation of the dose patterns correlated with rectal toxicity. The method was applied to a cohort of 63 patients and it was able to build up a dose pattern characterizing the difference between patients presenting rectal bleeding after prostate cancer radiotherapy and those who did not.

Transcatheter aortic valve implantation (TAVI) is becoming the standard choice of care for non-operable patients suffering from severe aortic valve stenosis. As there is no direct view or access to the affected anatomy, accurate preoperative planning is crucial for a successful outcome. The most important decision during planning is selecting the proper implant type and size. Due to the wide variety in device sizes and types and non-circular annulus shapes, there is often no obvious choice for the specific patient. Most clinicians base their final decision on their previous experience. As a first step towards a more predictive planning, we propose an integrated method to estimate the aortic apparatus from CT images and compute implant deployment. Aortic anatomy, which includes aortic root, leaflets and calcifications, is automatically extracted using robust modeling and machine learning algorithms. Then, the finite element method is employed to calculate the deployment of a TAVI implant inside the patient-specific aortic anatomy. The anatomical model was evaluated on 198 CT images, yielding an accuracy of 1.30±0.23

mm

. In eleven subjects, pre- and post-TAVI CT images were available. Errors in predicted implant deployment were of 1.74±0.40

mm

in average and 1.32

mm

in the aortic valve annulus region, which is almost three times lower than the average gap of 3

mm

between consecutive implant sizes. Our framework may thus constitute a surrogate tool for TAVI planning.

This paper presents and evaluates a deep learning architecture for automated basal cell carcinoma cancer detection that integrates (1) image representation learning, (2) image classification and (3) result interpretability. A novel characteristic of this approach is that it extends the deep learning architecture to also include an interpretable layer that highlights the visual patterns that contribute to discriminate between cancerous and normal tissues patterns, working akin to a digital staining which spotlights image regions important for diagnostic decisions. Experimental evaluation was performed on set of 1,417 images from 308 regions of interest of skin histopathology slides, where the presence of absence of basal cell carcinoma needs to be determined. Different image representation strategies, including bag of features (BOF), canonical (discrete cosine transform (DCT) and Haar-based wavelet transform (Haar)) and proposed learned-from-data representations, were evaluated for comparison. Experimental results show that the representation learned from a large histology image data set has the best overall performance (89.4% in F-measure and 91.4% in balanced accuracy), which represents an improvement of around 7% over canonical representations and 3% over the best equivalent BOF representation.

We use deep max-pooling convolutional neural networks to detect mitosis in breast histology images. The networks are trained to classify each pixel in the images, using as context a patch centered on the pixel. Simple postprocessing is then applied to the network output. Our approach won the ICPR 2012 mitosis detection competition, outperforming other contestants by a significant margin.

Segmentation schemes such as hierarchical region merging or correllation clustering rely on edge weights between adjacent (super-)voxels. The quality of these edge weights directly affects the quality of the resulting segmentations.

Unstructured

learning methods seek to minimize the classification error on

individual

edges. This ignores that a few local mistakes (tiny boundary gaps) can cause catastrophic global segmentation errors. Boundary evidence learning should therefore optimize

structured

quality criteria such as Rand Error or Variation of Information. We present the first structured learning scheme using a structured loss function; and we introduce a new hierarchical scheme that allows to approximately solve the NP hard prediction problem even for huge volume images. The value of these contributions is demonstrated on two challenging neural circuit reconstruction problems in serial sectioning electron microscopic images with billions of voxels. Our contributions lead to a partitioning quality that improves over the current state of the art.

Millions of people worldwide suffer from fragility fractures, which cause significant morbidity, financial costs and even mortality. The gold standard to quantify structural properties of trabecular bone is based on the morphometric parameters obtained from

μ

CT images of clinical bone biopsy specimens. The currently used image processing approaches are not able to fully explain the variation in bone strength. In this study, we introduce the contour tree connectivity (CTC) as a novel morphometric parameter to study trabecular bone quality. With CTC, we calculate a new connectivity measure for trabecular bone by using contour tree representation of binary images and algebraic graph theory. To test our approach, we use trabecular bone biopsies obtained from 55 female patients. We study the correlation of CTC with biomechanical test results as well as other morphometric parameters obtained from

μ

CT. The results based on our dataset show that CTC is the 3

rd

best predictive feature of ultimate bone strength after bone volume fraction and degree of anisotropy.

While many approaches exist for the automated segmentation of retinal vessels in fundus photographs, limited work has focused on the problem of separating the arterial from the venous trees. The few existing approaches that do exist for separating arteries from veins are local and/or greedy in nature, making them susceptible to errors or limiting their applicability to only the very largest vessels. In this work, we propose a new, more global, optimization framework for separating two overlapping trees within medical images and apply this approach for the separation of arteriovenous trees in low-contrast color fundus images. In particular, our approach has two stages. The first stage is to generate a vessel potential connectivity map (VPCM) consisting of vessel segments and the potential connectivity between them. The second stage is to separate the VPCM into multiple anatomical trees using a graph-based meta-heuristic algorithm. Based on a graph model, the algorithm first uses local knowledge and global constraints of the vasculature to generate near-optimal candidate solutions, and then obtains the final solution based on global costs. We test the algorithm on 48 low-contrast fundus images and the promising results suggest its applicability and robustness.

We propose a 2D continuous-time Hidden Markov Model (2D CT-HMM) for glaucoma progression modeling given longitudinal structural and functional measurements. CT-HMM is suitable for modeling longitudinal medical data consisting of visits at arbitrary times, and 2D state structure is more appropriate for glaucoma since the time courses of functional and structural degeneration are usually different. The learned model not only corroborates the clinical findings that structural degeneration is more evident than functional degeneration in early glaucoma and the opposite is observed in more advanced stages, but also reveals the exact stages where the trend reverses. A method to detect time segments of fast progression is also proposed. Our results show that this detector can effectively identify patients with rapid degeneration. The model and the derived detector can be of clinical value for glaucoma monitoring.

Good feature design is important to achieve effective image classification. This paper presents a novel feature design with two main contributions. First, prior to computing the feature descriptors, we propose to transform the images with learning-based filters to obtain more representative feature descriptors. Second, we propose to transform the computed descriptors with another set of learning-based filters to further improve the classification accuracy. In this way, while generic feature descriptors are used, data-adaptive information is integrated into the feature extraction process based on the optimization objective to enhance the discriminative power of feature descriptors. The feature design is applicable to different application domains, and is evaluated on both lung tissue classification in high-resolution computed tomography (HRCT) images and apoptosis detection in time-lapse phase contrast microscopy image sequences. Both experiments show promising performance improvements over the state-of-the-art.

The accurate and automated measuring of durations of certain human embryo stages is important to assess embryo viability and predict its clinical outcomes in

in vitro

fertilization (IVF). In this work, we present a multi-level embryo stage classification method to identify the number of cells at every time point of a time-lapse microscopy video of early human embryo development. The proposed method employs a rich set of hand-crafted and automatically learned embryo features for classification and avoids explicit segmentation or tracking of individual embryo cells. It was quantitatively evaluated using a total of 389 human embryo videos, resulting in a 87.92% overall embryo stage classification accuracy.

Cataracts, which result from lens opacification, are the leading cause of blindness worldwide. Current methods for determining the severity of cataracts are based on manual assessments that may be weakened by subjectivity. In this work, we propose a system to automatically grade the severity of nuclear cataracts from slit-lamp images. We introduce a new feature for cataract grading together with a group sparsity-based constraint for linear regression, which performs feature selection, parameter selection and regression model training simultaneously. In experiments on a large database of 5378 images, our system outperforms the state-of-the-art by yielding with respect to clinical grading a mean absolute error (

ε

) of 0.336, a 69.0% exact integral agreement ratio (

R

0

), a 85.2% decimal grading error ≤ 0.5 (

R

e

0.5

), and a 98.9% decimal grading error ≤ 1.0 (

R

e

1.0

). Through a more objective grading of cataracts using our proposed system, there is potential for better clinical management of the disease.

We propose a new method to recover 3D time-resolved velocity vectors within the left ventricle (LV) using a combination of multiple registered 3D colour Doppler images and LV wall motion. Incorporation of wall motion, calculated from 3D B-Mode images, and the use of a multi-scale reconstruction framework allow recovery of 3D velocity over the entire ventricle, even in regions where there is little or no Doppler data.

Our method is tested on the LV of a paediatric patient and is compared to 2D and 3D flow Magnetic Resonance Imaging (MRI). Use of wall motion information increased stroke volume accuracy by 14%, and enabled full 3D velocity mapping within the ventricle. Velocity distribution showed good agreement with respect to MRI, and vortex formation during diastole was successfully reconstructed.

We present a registration framework that combines both tissue Doppler and B-mode echocardiographic sequences. The estimated spatiotemporal transform is diffeomorphic, and calculated by modeling its corresponding velocity field using continuous B-splines. A new cost function using both B-mode image voxel intensities and Doppler velocities is also proposed. Registration accuracy was evaluated on synthetic data with known ground truth. Results showed that our method allows quantifying wall motion with higher accuracy than when using a single modality. On patient data, both displacement and velocity curves were compared with the ones obtained from widely used commercial software using either B-mode images or TDI. Our method demonstrated to be more robust to image noise while being independent from the beam angle.

Cardiac fiber architecture plays an important role in electrophysiological and mechanical functions of the heart. Yet, its inter-subject variability and more particularly, its relationship to the shape of the myocardium, is not fully understood. In this paper, we extend the statistical analysis of cardiac fiber architecture beyond its description with a fixed average geometry. We study the co-variation of fiber architecture with either shape or strain-based information by exploring their principal modes of

joint variations

. We apply our general framework to a dataset of 8

ex vivo

canine hearts, and find that strain-based information appears to correlate best with the fiber architecture. Furthermore, compared to current approaches that warp an average atlas to the patient geometry, our preliminary results show that joint statistics improves fiber synthesis from shape by 8.0%, with cases up to 25.9%. Our experiments also reveal evidence on a possible relation between architectural variability and myocardial thickness.

Given the observed abnormal motion dynamics of patients with heart conditions, quantifying cardiac motion in both normal and pathological cases can provide useful insights for therapy planning. In order to be able to analyse the motion over multiple subjects in a robust manner, it is desirable to represent the motion by a low number of parameters. We propose a reduced order cardiac motion model, reduced in space through a polyaffine model, and reduced in time by statistical model order reduction. The method is applied to a data-set of synthetic cases with known ground truth to validate the accuracy of the left ventricular motion tracking, and to validate a patient-specific reduced-order motion model. Population-based statistics are computed on a set of 15 healthy volunteers to obtain separate spatial and temporal bases. Results demonstrate that the reduced model can efficiently detect abnormal motion patterns and even allowed to retrospectively reveal abnormal unnoticed motion within the control subjects.

Recent progress in diffusion imaging has lead to in-vivo acquisitions of fiber orientation data in the beating heart. Current methods are however limited in resolution to a few short-axis slices. For this particular application and others where the diffusion volume is subsampled, partial or even damaged, the reconstruction of a complete volume can be challenging. To address this problem, we present two complementary methods for fiber reconstruction from sparse orientation measurements, both of which derive from second-order properties related to fiber curvature as described by Maurer-Cartan connection forms. The first is an extrinsic partial volume reconstruction method based on principal component analysis of the connection forms and is best put to use when dealing with highly damaged or sparse data. The second is an intrinsic method based on curvilinear interpolation of the connection forms on ellipsoidal shells and is advantageous when more slice data becomes available. Using a database of 8 cardiac rat diffusion tensor images we demonstrate that both methods are able to reconstruct complete volumes to good accuracy and lead to low reconstruction errors.

In this work we formulate vessel segmentation on contrast-enhanced CT angiogram images as a Bayesian tracking problem. To obtain posterior probability estimation of vessel location, we employ sequential Monte Carlo tracking and propose a new vessel segmentation method by fusing multiple cues extracted from CT images. These cues include intensity, vesselness, organ detection, and bridge information for poorly enhanced segments from global path minimization. By fusing local and global information for vessel tracking, we achieved high accuracy and robustness, with significantly improved precision compared to a traditional segmentation method (

p

=0.0002). Our method was applied to the segmentation of the marginal artery of the colon, a small bore vessel of potential importance for colon segmentation and CT colonography. Experimental results indicate the effectiveness of the proposed method.

We introduce an automated method for the 3D tracking of carotids acquired as a sequence of 2D ultrasound images. The method includes an image stabilization step that compensates for the cardiac and respiratory motion of the carotid, and tracks the carotid wall via a shape and appearance model trained from representative images. Envisaging an application in automatic detection of plaques, the algorithm was tested on ultrasound volumes from 4,000 patients and its accuracy was evaluated by measuring the distance between the location of more than 4,000 carotid plaques and the location of the carotid wall as estimated by the proposed algorithm. Results show that the centroids of over 95% of the carotid plaques in the dataset were located within 3 mm of the estimated carotid wall, indicating the accuracy of the tracking algorithm.

An approach to study population differences in cerebral vasculature is proposed. This is done by 1) extending the concept of encoding cerebral blood vessel networks as spatial graphs and 2) quantifying graph similarity in a kernel-based discriminant classifier setup. We argue that augmenting graph vertices with information about their proximity to selected brain structures adds discriminative information and consequently leads to a more expressive encoding. Using graph-kernels then allows us to quantify graph similarity in a principled way. To demonstrate our approach, we assess the hypothesis that gender differences manifest as variations in the architecture of cerebral blood vessels, an observation that previously had only been tested and confirmed for the Circle of Willis. Our results strongly support this hypothesis, i.e, we can demonstrate non-trivial, statistically significant deviations from random gender classification in a cross-validation setup on 40 healthy patients.

We present a new approach for automated segmentation of the carotid lumen bifurcation from 3D free-hand ultrasound using a 3D surface graph cut method. The method requires only the manual selection of single seed points in the internal, external, and common carotid arteries. Subsequently, the centerline between these points is automatically traced, and the optimal lumen surface is found around the centerline using graph cuts. To refine the result, the latter process was iterated. The method was tested on twelve carotid arteries from six subjects including three patients with a moderate carotid artery stenosis. Our method successfully segmented the lumen in all cases. We obtained an average dice overlap with respect to a manual segmentation of 84% for healthy volunteers. For the patient data, we obtained a dice overlap of 66.7%.

In this paper, we present a novel graph-based method for segmenting the whole 3D vessel tree structures. Our method exploits a new adaptive cylinder flux (

ACF

) based connectivity framework, which is formulated based on random walks [8]. To avoid the shrinking problem of elongated structure, all existing graph-based energy optimization methods for vessel segmentation rely on skeleton or ROI extraction. As a result, the performance of these vessel segmentation methods then depends heavily on the skeleton extraction results. In this paper, with the help of

ACF

based connectivity framework, a global optimal segmentation result can be obtained without extracting skeleton or ROI. The classical issues of the graph-based methods, such as shrinking bias and sensitivity to seed point location, can be solved effectively with the proposed method thanks to the connectivity framework.

Assessing airway wall surfaces and the lumen from high resolution computed tomography (CT) scans are of great importance for diagnosing pulmonary diseases. However, accurately determining inner and outer airway wall surfaces of a complete 3-D tree structure can be quite challenging because of its complex nature. In this paper, we introduce a computational framework to accurately quantify airways through (i) a precise segmentation of the lumen, and (ii) a spatially constrained Markov random walk method to estimate the airway walls. Our results demonstrate that the proposed airway analysis platform identified the inner and outer airway surfaces better than methods commonly used in clinics, such as full width at half maximum and phase congruency.

Despite recent advances, automatic blood vessel extraction from low quality retina images remains difficult. We propose an interactive approach that enables a user to efficiently obtain near perfect vessel segmentation with a few mouse clicks. Given two seed points, the approach seeks an optimal path between them by minimizing a cost function. In contrast to the Live-Vessel approach, the graph in our approach is based on the curve fragments generated with vessel tracing instead of individual pixels. This enables our approach to overcome the shortcut problem in extracting tortuous vessels and the problem of vessel interference in extracting neighboring vessels in minimal-cost path techniques, resulting in less user interaction for extracting thin and tortuous vessels from low contrast images. It also makes the approach much faster.

Respiratory motion remains a major challenge for whole-heart coronary magnetic resonance angiography (CMRA). Recently, iterative reconstruction has been augmented with non-rigid motion compensation to correct for the effects of respiratory motion. The major challenge of this approach is the estimation of dense deformation fields. In this work, the application of such a motion-compensated reconstruction is proposed for accelerated 3D Cartesian whole-heart CMRA. Without the need for extra calibration data or user interaction, the non-rigid deformations due to respiratory motion are directly estimated on the acquired image data. In-vivo experiments on 14 healthy volunteers were performed to compare the proposed method with the result of a navigator-gated reference scan. While reducing the acquisition time by one third, the reconstructed images resulted in equivalent vessel sharpness of 0.44 ±0.06 mm

− 1

and 0.45 ±0.05 mm

− 1

, respectively.

In recent years, there has been a great interest in computer-aided diagnosis of Alzheimer’s Disease (AD) and its prodromal stage, Mild Cognitive Impairment (MCI). Unlike the previous methods that consider simple low-level features such as gray matter tissue volumes from MRI, mean signal intensities from PET, in this paper, we propose a deep learning-based feature representation with a stacked auto-encoder. We believe that there exist latent complicated patterns,

e.g.

, non-linear relations, inherent in the low-level features. Combining latent information with the original low-level features helps build a robust model for AD/MCI classification with high diagnostic accuracy. Using the ADNI dataset, we conducted experiments showing that the proposed method is 95.9%, 85.0%, and 75.8% accurate for AD, MCI, and MCI-converter diagnosis, respectively.

Neuroimaging group analyses are used to compare the inter-subject variability observed in brain organization with behavioural or genetic variables and to assess risks factors of brain diseases. The lack of stability and of sensitivity of current voxel-based analysis schemes may however lead to non-reproducible results. A new approach is introduced to overcome the limitations of standard methods, in which active voxels are detected according to a consensus on several random parcellations of the brain images, while a permutation test controls the false positive risk. Both on syntetic and real data, this approach shows higher sensitivity, better recovery and higher reproducibility than standard methods and succeeds in detecting a significant association in an imaging-genetic study between a genetic variant next to the COMT gene and a region in the left thalamus on a functional Magnetic Resonance Imaging contrast.

Machine learning techniques have been widely used to support the diagnosis of neurological diseases such as dementia. Recent approaches utilize local intensity patterns within patches to derive voxelwise grading measures of disease. However, the relationships among these patches are usually ignored. In addition, there is some ambiguity in assigning disease labels to the extracted patches. Not all of the patches extracted from patients with dementia are characteristic of morphology associated with disease. In this paper, we propose to use a multiple instance learning method to address the problem of assigning training labels to the patches. In addition, a graph is built for each image to exploit the relationships among these patches, which aids the classification work. We illustrate the proposed approach in an application for the detection of Alzheimer’s disease (AD): Using the baseline MR images of 834 subjects from the ADNI study, the proposed method can achieve a classification accuracy of 88.8% between AD patients and healthy controls, and 69.6% between patients with stable Mild Cognitive Impairment (MCI) and progressive MCI. These results compare favourably with state-of-the-art classification methods.

Spontaneous brain activity reveals mechanisms of brain function and dysfunction. Its population-level statistical analysis based on functional images often relies on the definition of brain regions that must summarize efficiently the covariance structure between the multiple brain networks. In this paper, we extend a network-discovery approach, namely dictionary learning, to readily extract brain regions. To do so, we introduce a new tool drawing from clustering and linear decomposition methods by carefully crafting a penalty. Our approach automatically extracts regions from rest fMRI that better explain the data and are more stable across subjects than reference decomposition or clustering methods.

Although the study of cerebral vasoreactivity using fMRI is mainly conducted through the BOLD fMRI modality, owing to its relatively high signal-to-noise ratio (SNR), ASL fMRI provides a more interpretable measure of cerebral vasoreactivity than BOLD fMRI. Still, ASL suffers from a low SNR and is hampered by a large amount of physiological noise. The current contribution aims at improving the recovery of the vasoreactive component from the ASL signal. To this end, a Bayesian hierarchical model is proposed, enabling the recovery of perfusion levels as well as fitting their dynamics. On a single-subject ASL real data set involving perfusion changes induced by hypercapnia, the approach is compared with a classical GLM-based analysis. A better goodness-of-fit is achieved, especially in the transitions between baseline and hypercapnia periods. Also, perfusion levels are recovered with higher sensitivity and show a better contrast between gray- and white matter.

The Allen Brain Atlas (ABA) database provides comprehensive 3D atlas of gene expression in the adult mouse brain for studying the spatial expression patterns in the mammalian central nervous system. It is computationally challenging to construct the accurate anatomical and genetic networks using the ABA 4D data. In this paper, we propose a novel sparse simplex model to accurately construct the brain anatomical and genetic networks, which are important to reveal the brain spatial expression patterns. Our new approach addresses the shift-invariant and parameter tuning problems, which are notorious in the existing network analysis methods, such that the proposed model is more suitable for solving practical biomedical problems. We validate our new model using the 4D ABA data, and the network construction results show the superior performance of the proposed sparse simplex model.

Manifold learning of medical images plays a potentially important role for modeling anatomical variability within a population with applications that include segmentation, registration, and prediction of clinical parameters. This paper describes a novel method for learning the manifold of 3D brain images that, unlike most existing manifold learning methods, does not require the manifold space to be locally linear, and does not require a predefined similarity measure or a prebuilt proximity graph. Our manifold learning method is based on deep learning, a machine learning approach that uses layered networks (called deep belief networks, or DBNs) and has received much attention recently in the computer vision field due to their success in object recognition tasks. DBNs have traditionally been too computationally expensive for application to 3D images due to the large number of trainable parameters. Our primary contributions are 1) a much more computationally efficient training method for DBNs that makes training on 3D medical images with a resolution of up to 128 × 128 × 128 practical, and 2) the demonstration that DBNs can learn a low-dimensional manifold of brain volumes that detects modes of variations that correlate to demographic and disease parameters.

Point Distribution Models (PDM) are one of the most extended methods to characterize the underlying population of set of samples, whose usefulness has been demonstrated in a wide variety of applications, including medical imaging. However, one important issue remains unsolved: the large number of training samples required. This problem becomes critical as the complexity of the problem increases, and the modeling of 3

D

multiobjects/organs represents one of the most challenging cases. Based on the 3

D

wavelet transform, this paper introduces a multiresolution hierarchical variant of PDM (MRH-PDM) able to efficiently characterize the different inter-object relationships, as well as the particular locality of each element separately. The significant advantage of this new method over two previous approaches in terms of accuracy has been successfully verified for the particular case of 3

D

subcortical brain structures.

Establishing accurate anatomical correspondences is critical for medical image registration. Although many hand-engineered features have been proposed for correspondence detection in various registration applications, no features are general enough to work well for all image data. Although many learning-based methods have been developed to help selection of best features for guiding correspondence detection across subjects with large anatomical variations, they are often limited by requiring the known correspondences (often presumably estimated by certain registration methods) as the ground truth for training. To address this limitation, we propose using an unsupervised deep learning approach to directly learn the basis filters that can effectively represent all observed image patches. Then, the coefficients by these learnt basis filters in representing the particular image patch can be regarded as the morphological signature for correspondence detection during image registration. Specifically, a stacked two-layer convolutional network is constructed to seek for the hierarchical representations for each image patch, where the high-level features are inferred from the responses of the low-level network. By replacing the hand-engineered features with our learnt data-adaptive features for image registration, we achieve promising registration results, which demonstrates that a general approach can be built to improve image registration by using data-adaptive features through unsupervised deep learning.

Previous works investigated a range of spatio-temporal models for fMRI data analysis to provide robust determination of functional region-of-interest (ROI). We present a novel spatio-temporal fMRI model that is suitable for identifying a number of distinct temporal patterns and their spatial support in the voxel space. Accordingly, fMRI signals on a single voxel are modeled as a probabilistic superposition of those temporal patterns. The spatially varying influence of individual patterns is defined in terms of a parameterised function. The temporal pattern is characterised by both the underlying hemodynamic response function (HRF) and a time series of the individual stimulus-response magnitudes, which makes the proposed model particularly suitable for modeling rapid event-related fMRI data. Moreover, a parametric approach is adopted to represent the HRFs. The resulting methodology is conceptually principled and computationally efficient. We first verify the proposed model in a controlled experimental setting using synthetic data. The model is further applied to analyzing real fMRI data, with focus on functional homogeneity within individual ROIs.

Group-wise activation detection in task-based fMRI has been widely used because of its robustness to noises and statistical power to deal with variability of individual brains. However, current group-wise fMRI activation detection methods typically rely on the spatial alignment established by co-registration of individual brains’ fMRI images into the same template space, which has difficulty in dealing with the remarkable anatomic variation of different brains. As a consequence, the resulted misalignment among multiple brains could substantially degrade the accuracy and specificity of group-wise fMRI activation detection. To address these challenges, this paper presents a novel methodology to detect group-wise fMRI activation based on a publicly released dense map of DTI-derived structural cortical landmarks, which possess intrinsic correspondences across individuals and populations. The basic idea here is that a first-level general linear model (GLM) analysis is performed on fMRI signals of each corresponding cortical landmark in each individual brain’s own space, and then the single-subject effect size of the same landmark from a group of subjects are statistically integrated and assessed at the group level using the mixed-effects model. As a result, the consistently activated cortical landmarks are determined and declared group-wisely in response to external block-based stimuli. Our experimental results demonstrated that the proposed approach can map meaningful group-wise activation patterns on the atlas of cortical landmarks without image registration between subjects and spatial smoothing.

Due to the difficulties in establishing accurate correspondences of brain network nodes across individual subjects, systematic elucidation of possible functional connectivity (FC) alterations in mild cognitive impairment (MCI) compared with normal controls (NC) is a challenging problem. To address this challenge, in this paper, we develop and apply novel predictive models of resting state networks (RSNs) learned from multimodal resting state fMRI (R-fMRI) and DTI data to assess large-scale FC alterations in MCI. Our rationale is that some RSNs in MCI are substantially altered and can hardly be directly compared with those in NC. Instead, structural landmarks derived from DTI data are much more consistent and correspondent across MCI/NC brains, and therefore can be employed to encode RSNs in NC and serve as the predictive models of RSNs for MCI. To derive these predictive models, RSNs in NC are constructed by group-wise ICA clustering and employed to functionally annotate corresponding structural landmarks. Afterwards, these functionally-annotated structural landmarks are predicted in MCI based on DTI data and used to assess FC alterations in MCI. Experimental results demonstrated that the predictive models of RSNs are effective and can comprehensively reveal widespread FC alterations in MCI.

Functional magnetic resonance imaging (fMRI) has been widely used for inferring brain regions that tend to work in tandem and grouping them into subnetworks. Despite that certain brain regions are known to interact with multiple subnetworks, few existing techniques support identification of subnetworks with overlaps. To address this limitation, we propose a novel approach based on replicator dynamics that facilitates detection of sparse overlapping subnetworks. We refer to our approach as overlapping replicator dynamics (RDOL). On synthetic data, we show that RDOL achieves higher accuracy in subnetwork identification than state-of-the-art methods. On real data, we demonstrate that RDOL is able to identify major functional hubs that are known to serve as communication channels between brain regions, in addition to detecting commonly observed functional subnetworks. Moreover, we illustrate that knowing the subnetwork overlaps enables inference of functional pathways, e.g. from primary sensory areas to the integration hubs.

Imaging genetics aims to discover how variants in the human genome influence brain measures derived from images. Genome-wide association scans (GWAS) can screen the genome for common differences in our DNA that relate to brain measures. In small samples, GWAS has low power as individual gene effects are weak and one must also correct for multiple comparisons across the genome and the image. Here we extend recent work on genetic clustering of images, to analyze surface-based models of anatomy using GWAS. We performed spherical harmonic analysis of hippocampal surfaces, automatically extracted from brain MRI scans of 1254 subjects. We clustered hippocampal surface regions with common genetic influences by examining genetic correlations (

r

g

) between the normalized deformation values at all pairs of surface points. Using genetic correlations to cluster surface measures, we were able to boost effect sizes for genetic associations, compared to clustering with traditional phenotypic correlations using Pearson’s

r

.

Growing evidence from the functional neuroimaging field suggests that human brain functions are realized via dynamic functional interactions on large-scale structural networks. Even in resting state, functional brain networks exhibit remarkable temporal dynamics. However, it has been rarely explored to computationally model such dynamic functional information flows on large-scale brain networks. In this paper, we present a novel computational framework to explore this problem using multimodal resting state fMRI (R-fMRI) and diffusion tensor imaging (DTI) data. Basically, recent literature reports including our own studies have demonstrated that the resting state brain networks dynamically undergo a set of distinct brain states. Within each quasi-stable state, functional information flows from one set of structural brain nodes to other sets of nodes, which is analogous to the message package routing on the Internet from the source node to the destination. Therefore, based on the large-scale structural brain networks constructed from DTI data, we employ a dynamic programming strategy to infer functional information transition routines on structural networks, based on which hub routers that most frequently participate in these routines are identified. It is interesting that a majority of those hub routers are located within the default mode network (DMN), revealing a possible mechanism of the critical functional hub roles played by the DMN in resting state. Also, application of this framework on a post trauma stress disorder (PTSD) dataset demonstrated interesting difference in hub router distributions between PTSD patients and healthy controls.