Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2015

Early detection, assessment and treatment of intracranial aneurysms is important to prevent rupture, which may cause death. We propose a framework for detection and quantification of morphology of the aneurysms. A novel detector using decision forests, which employ responses of blobness and vesselness filters encoded in rotation invariant and scale normalized frequency components of spherical harmonics representation is proposed. Aneurysm location is used to seed growcut segmentation, followed by improved neck extraction based on intravascular ray-casting and robust closed-curve fit to the segmentation. Aneurysm segmentation and neck curve are used to compute three morphologic metrics: neck width, dome height and aspect ratio. The proposed framework was evaluated on ten cerebral 3D-DSA images containing saccular aneurysms. Sensitivity of aneurysm detection was 100% at 0.4 false positives per image. Compared to measurements of two expert raters, the values of metrics obtained by the proposed framework were accurate and, thus, suitable for assessing the risk of rupture.

Glaucoma, Pathological Myopia (PM), and Age-related Macular Degeneration (AMD) are three leading ocular diseases worldwide. Visual features extracted from retinal fundus images have been increasingly used for detecting these three diseases. In this paper, we present a discriminative feature selection model based on multi-task learning, which imposes the exclusive group lasso regularization for competitive sparse feature selection and the graph Laplacian regularization to embed the correlations among multiple diseases. Moreover, this multi-task linear discriminative model is able to simultaneously select sparse features and detect multiple ocular diseases. Extensive experiments are conducted to validate the proposed framework on the

SiMES

dataset. From the Area Under Curve (AUC) results in multiple ocular diseases classification, our method is shown to outperform the state-of-the-art algorithms.

Architecture distortions of glands and villi are indication of chronic inflammation. However, the “duality” nature of these two structures causes lots of ambiguity for their detection in H&E histology tissue images, especially when multiple instances are clustered together. Based on the observation that once such an object is detected for certain, the ambiguity in the neighborhood of the detected object can be reduced considerably, we propose to combine deep learning and domain knowledge in a unified framework, to simultaneously detect (the closely related) glands and villi in H&E histology tissue images. Our method iterates between exploring domain knowledge and performing deep learning classification, and the two components benefit from each other. (1) By exploring domain knowledge, the generated object proposals (to be fed to deep learning) form a more complete coverage of the true objects and the segmentation of object proposals can be more accurate, thus improving deep learning’s performance on classification. (2) Deep learning can help verify the class of each object proposal, and provide feedback to repeatedly “refresh” and enhance domain knowledge so that more reliable object proposals can be generated later on. Experiments on clinical data validate our ideas and show that our method improves the state-of-the-art for gland detection in H&E histology tissue images (to our best knowledge, we are not aware of any method for villi detection).

We present a novel HMM based approach to simultaneous segmentation of vessel walls in Lymphatic confocal images. The vessel borders are parameterized using RBFs to minimize the number of tracking points. The proposed method tracks the hidden states that indicate border locations for both the inner and outer walls. The observation for both borders is obtained using edge-based features from steerable filters. Two separate Gaussian probability distributions for the vessel borders and background are used to infer the emission probability, and the transmission probability is learned using a Baum-Welch algorithm. We transform the segmentation problem into a minimization of an s-excess graph cost, with each node in the graph corresponding to a hidden state and the weight for each node being defined by its emission probability. We define the inter-relations between neighboring nodes based on the transmission probability. We present both qualitative and quantitative analysis in comparison to the popular Viterbi algorithm.

Automated computer-aided analysis of lung vessels has shown to yield promising results for non-invasive diagnosis of lung diseases. In order to detect vascular changes affecting arteries and veins differently, an algorithm capable of identifying these two compartments is needed. We propose a fully automatic algorithm that separates arteries and veins in thoracic computed tomography (CT) images based on two integer programs. The first extracts multiple subtrees inside a graph of vessel paths. The second labels each tree as either artery or vein by maximizing both, the contact surface in their Voronoi diagram, and a measure based on closeness to accompanying bronchi. We evaluate the performance of our automatic algorithm on 10 manual segmentations of arterial and venous trees from patients with and without pulmonary vascular disease, achieving an average voxel based overlap of 94.1% (range: 85.0% – 98.7%), outperforming a recent state-of-the-art interactive method.

Mammography is the first-line modality for screening and diagnosis of breast cancer. Following the common practice of radiologists to examine two mammography views, we propose a fully automated dual-view analysis framework for breast mass detection in mammograms. The framework combines unsupervised segmentation and random-forest classification to detect and rank candidate masses in cranial-caudal (CC) and mediolateral-oblique (MLO) views. Subsequently, it estimates correspondences between pairs of candidates in the two views. The performance of the method was evaluated using a publicly available full-field digital mammography database (INbreast). Dual-view analysis provided area under the ROC curve of 0.94, with detection sensitivity of 87% at specificity of 90%, which significantly improved single-view performance (72% sensitivity at 90% specificity, 78% specificity at 87% sensitivity, P<0.05). One-to-one mapping of candidate masses from two views facilitated correct estimation of the breast quadrant in 77% of the cases. The proposed method may assist radiologists to efficiently identify and classify breast masses.

Histograms of oriented gradients (HOG) are widely employed image descriptors in modern computer-aided diagnosis systems. Built upon a set of local, robust statistics of low-level image gradients, HOG features are usually computed on raw intensity images. In this paper, we explore a learned image transformation scheme for producing higher-level inputs to HOG. Leveraging semantic object boundary cues, our methods compute data-driven image feature maps via a supervised boundary detector. Compared with the raw image map, boundary cues offer mid-level, more object-specific visual responses that can be suited for subsequent HOG encoding. We validate integrations of several image transformation maps with an application of computer-aided detection of lymph nodes on thoracoabdominal CT images. Our experiments demonstrate that semantic boundary cues based HOG descriptors complement and enrich the raw intensity alone. We observe an overall system with substantially improved results (~78% versus 60% recall at 3 FP/volume for two target regions). The proposed system also moderately outperforms the state-of-the-art deep convolutional neural network (CNN) system in the mediastinum region, without relying on data augmentation and requiring significantly fewer training samples.

Computer-aided detection (CAD) can play a major role in diagnosing pulmonary embolism (PE) at CT pulmonary angiography (CTPA). However, despite their demonstrated utility, to achieve a clinically acceptable sensitivity, existing PE CAD systems generate a high number of false positives, imposing extra burdens on radiologists to adjudicate these superfluous CAD findings. In this study, we investigate the feasibility of convolutional neural networks (CNNs) as an effective mechanism for eliminating false positives. A critical issue in successfully utilizing CNNs for detecting an object in 3D images is to develop a “right” image representation for the object. Toward this end, we have developed a vessel-aligned multi-planar image representation of emboli. Our image representation offers three advantages: (1) efficiency and compactness—concisely summarizing the 3D contextual information around an embolus in only 2 image channels, (2) consistency—automatically aligning the embolus in the 2-channel images according to the orientation of the affected vessel, and (3) expandability—naturally supporting data augmentation for training CNNs. We have evaluated our CAD approach using 121 CTPA datasets with a total of 326 emboli, achieving a sensitivity of 83% at 2 false positives per volume. This performance is superior to the best performing CAD system in the literature, which achieves a sensitivity of 71% at the same level of false positives. We have further evaluated our system using the entire 20 CTPA test datasets from the PE challenge. Our system outperforms the winning system from the challenge at 0mm localization error but is outperformed by it at 2mm and 5mm localization errors. In our view, the performance at 0mm localization error is more important than those at 2mm and 5mm localization errors.

We propose an automatic feature selection framework for analyzing temporal ultrasound signals of prostate tissue. The framework consists of: 1) an unsupervised feature reduction step that uses Deep Belief Network (DBN) on spectral components of the temporal ultrasound data; 2) a supervised fine-tuning step that uses the histopathology of the tissue samples to further optimize the DBN; 3) a Support Vector Machine (SVM) classifier that uses the activation of the DBN as input and outputs a likelihood for the cancer. In leave-one-core-out cross-validation experiments using 35 biopsy cores, an area under the curve of 0.91 is obtained for cancer prediction. Subsequently, an independent group of 36 biopsy cores was used for validation of the model. The results show that the framework can predict 22 out of 23 benign, and all of cancerous cores correctly. We conclude that temporal analysis of ultrasound data can potentially complement multi-parametric Magnetic Resonance Imaging (mp-MRI) by improving the differentiation of benign and cancerous prostate tissue.

The identification of subtle brain changes that are associated with mild cognitive impairment (MCI), the at-risk stage of Alzheimer’s disease, is still a challenging task. Different from existing works, which employ multimodal data (e.g., MRI, PET or CSF) to identify MCI subjects from normal elderly controls, we use four MRI sequences, including T1-weighted MRI (T1), Diffusion Tensor Imaging (DTI), Resting-State functional MRI (RS-fMRI) and Arterial Spin Labeling (ASL) perfusion imaging. Since these MRI sequences simultaneously capture various aspects of brain structure and function during clinical routine scan, it simplifies finding the relationship between subjects by incorporating the mutual information among them. To this end, we devise a hypergraph-based semi-supervised learning algorithm. In particular, we first construct a hypergraph for each of MRI sequences separately using a star expansion method with both the training and testing data. A centralized learning is then performed to model the optimal relevance between subjects by incorporating mutual information between different MRI sequences. We then combine all centralized hypergraphs by learning the optimal weight of each hypergraph based on the minimum Laplacian. We apply our proposed method on a cohort of 41 consecutive MCI subjects and 63 age-and-gender matched controls with four MRI sequences. Our method achieves at least a 7.61% improvement in classification accuracy compared to state-of-the-art methods using multiple MRI data.

Alzheimer’s disease (AD) is an irreversible neurodegenerative disorder that can lead to progressive memory loss and cognition impairment. Therefore, diagnosing AD during the risk stage, a.k.a. Mild Cognitive Impairment (MCI), has attracted ever increasing interest. Besides the automated diagnosis of MCI, it is important to provide physicians with related MCI cases with visually similar imaging data for case-based reasoning or evidence-based medicine in clinical practices. To this end, we propose a multi-graph learning based medical image retrieval technique for MCI diagnostic assistance. Our method is comprised of two stages, the query category prediction and ranking. In the first stage, the query is formulated into a multi-graph structure with a set of selected subjects in the database to learn the relevance between the query subject and the existing subject categories through learning the multi-graph combination weights. This predicts the category that the query belongs to, based on which a set of subjects in the database are selected as candidate retrieval results. In the second stage, the relationship between these candidates and the query is further learned with a new multi-graph, which is used to rank the candidates. The returned subjects can be demonstrated to physicians as reference cases for MCI diagnosing. We evaluated the proposed method on a cohort of 60 consecutive MCI subjects and 350 normal controls with MRI data under three imaging parameters: T1 weighted imaging (T1), Diffusion Tensor Imaging (DTI) and Arterial Spin Labeling (ASL). The proposed method can achieve average 3.45 relevant samples in top 5 returned results, which significantly outperforms the baseline methods compared.

We propose a fast and accurate method for counting the mitotic figures from histopathological slides using regenerative random forest. Our method performs automatic feature selection in an integrated manner with classification. The proposed random forest assigns a weight to each feature (dimension) of the feature vector in a novel manner based on the importance of the feature (dimension). The forest also assigns a misclassification-based penalty term to each tree in the forest. The trees are then regenerated to make a new population of trees (new forest) and only the more important features survive in the new forest. The feature vector is constructed from domain knowledge using the intensity features of nucleus, features of nuclear membrane and features of the possible stroma region surrounding the cell. The use of domain knowledge improves the classification performance. Experiments show at least 4% improvement in F-measure with an improvement in time complexity on the MITOS dataset from ICPR 2012 grand challenge.

A novel feature set for medical image analysis, named HoTPiG (Histogram of Triangular Paths in Graph), is presented. The feature set is designed to detect morphologically abnormal lesions in branching tree-like structures such as vessels. Given a graph structure extracted from a binarized volume, the proposed feature extraction algorithm can effectively encode both the morphological characteristics and the local branching pattern of the structure around each graph node (e.g., each voxel in the vessel). The features are derived from a 3-D histogram whose bins represent a triplet of shortest path distances between the target node and all possible node pairs near the target node. The extracted feature set is a vector with a fixed length and is readily applicable to state-of-the-art machine learning methods. Furthermore, since our method can handle vessel-like structures without thinning or centerline extraction processes, it is free from the “short-hair” problem and local features of vessels such as caliper changes and bumps are also encoded as a whole. Using the proposed feature set, a cerebral aneurysm detection application for clinical magnetic resonance angiography (MRA) images was implemented. In an evaluation with 300 datasets, the sensitivities of aneurysm detection were 81.8% and 89.2% when the numbers of false positives were 3 and 10 per case, respectively, thus validating the effectiveness of the proposed feature set.

3D ultrasound segmentation is a challenging task due to image artefacts, low signal-to-noise ratio and lack of contrast at anatomical boundaries. Current solutions usually rely on complex, anatomy-specific regularization methods to improve segmentation accuracy. In this work, we propose a highly adaptive learning-based method for fully automatic segmentation of ultrasound volumes. During training, anatomy-specific features are obtained through a sparse auto-encoder. The extracted features are employed in a Hough Forest based framework to retrieve the position of the target anatomy and its segmentation contour. The resulting method is fully automatic, i.e. it does not require any human interaction, and can robustly and automatically adapt to different anatomies yet enforcing appearance and shape constraints. We demonstrate the performance of the method for three different applications: segmentation of midbrain, left ventricle of the heart and prostate.

Due to the complex clinical picture of Parkinson’s disease (PD), the reliable diagnosis of patients is still challenging. A promising approach is the structural characterization of brain areas affected in PD by diffusion magnetic resonance imaging (dMRI). Standard classification methods depend on an accurate non-linear alignment of all images to a common reference template, and are challenged by the resulting huge dimensionality of the extracted feature space. Here, we propose a novel diagnosis pipeline based on the Fisher vector algorithm. This technique allows for a precise encoding into a high-level descriptor of standard diffusion measures like the fractional anisotropy and the mean diffusivity, extracted from the regions of interest (ROIs) typically involved in PD. The obtained low dimensional, fixed-length descriptors are independent of the image alignment and boost the linear separability of the problem in the description space, leading to more efficient and accurate diagnosis. In a test cohort of 50 PD patients and 50 controls, the implemented methodology outperforms previous methods when using a logistic linear regressor for classification of each ROI independently, which are subsequently combined into a single classification decision.

Osteoarthritis (OA) is considered to be one of the leading causes of disability, however clinical detection relies heavily on subjective experience to condense the continuous features into discrete grades. We present a fully automated method to standardise the measurement of OA features in the knee used to diagnose disease grade. Our approach combines features derived from both bone shape (obtained from an automated bone segmentation system) and image texture in the tibia. A simple weighted sum of the outputs of two Random Forest classifiers (one trained on shape features, the other on texture features) is sufficient to improve performance over either method on its own. We also demonstrate that Random Forests trained on simple pixel ratio features are as effective as the best previously reported texture measures on this task. We demonstrate the performance of the system on 500 knee radiographs from the OAI study.

In this paper, for the first time, we propose a data-driven search and retrieval (hashing) technique for large neuron image databases. The presented method is established upon hashing forests, where multiple unsupervised random trees are used to encode neurons by parsing the neuromorphological feature space into balanced subspaces. We introduce an inverse coding formulation for retrieval of relevant neurons to effectively mitigate the need for pairwise comparisons across the database. Experimental validations show the superiority of our proposed technique over the state-of-the art methods, in terms of precision-recall trade off for a particular code size. This demonstrates the potential of this approach for effective morphology preserving encoding and retrieval in large neuron databases.

Compared with global measurements such as ejection fraction, regional myocardial deformation can better aid detection of cardiac dysfunction. Although tagged and strain-encoded MR images can provide such regional information, they are uncommon in clinical routine. In contrast, cardiac CT images are more common with lower cost, but only provide motion of cardiac boundaries and additional constraints are required to obtain the myocardial strains. To verify the potential of contrast-enhanced CT images on computer-aided infarction identification, we propose a biomechanical approach combined with the support vector machine (SVM). A biomechanical model is used with deformable image registration to estimate 3D myocardial strains from CT images, and the regional strains and CT image intensities are input to the SVM classifier for regional infarction identification. Cross-validations on ten canine image sequences with artificially induced infarctions showed that the normalized radial and first principal strains were the most discriminative features, with respective classification accuracies of 87±13% and 84±10% when used with the normalized CT image intensity.

Cochlear Implants (CIs) restore hearing using an electrode array that is surgically implanted into the cochlea. Research has indicated there is a link between electrode location within the cochlea and hearing outcomes, however, comprehensive analysis of this phenomenon has not been possible because techniques proposed for locating electrodes only work for specific implant models or are too labor intensive to be applied on large datasets. We present a general and automatic graph-based method for localizing electrode arrays in CTs that is effective for various implant models. It relies on a novel algorithm for finding an optimal path of fixed length in a graph and achieves maximum localization errors that are sub-voxel. These results indicate that our methods could be used on a large scale to study the link between electrode placement and outcome across electrode array types, which could lead to advances that improve hearing outcomes for CI users.

The ultimate goal of this paper is to develop a computer-aided diagnostic (CAD) system for the accurate and early diagnosis of autism spectrum disorders (ASDs) using diffusion tensor imaging (DTI). This CAD system consists of three main steps. First, the brain tissues are segmented based on three image descriptors: a visual appearance model that has the ability to model a large dimensional feature space, a shape model that is adapted during the segmentation process using first- and second-order visual appearance features, and a spatially invariant second-order homogeneity descriptor. Secondly, discriminatory features are extracted from the segmented brains. Cortex shape variability is assessed using shape construction methods, and white matter integrity is further examined through connectivity analysis. Finally, the diagnostic capabilities of these extracted features are investigated. The accuracy of the presented CAD system has been tested on 38 infants with a high risk of developing ASDs. The statistical analysis and the diagnostic results (87% accuracy and AUC of 0.96 using random forest classifier) confirm the high performance and the efficiency of the proposed CAD system.

Computational tools for the analysis of complex biological networks are lacking in human connectome research. Especially, how to discover the brain network patterns shared by a group of subjects is a challenging computational neuroscience problem. Although some single graph clustering methods can be extended to solve the multi-graph cases, the discovered network patterns are often imbalanced,

e.g.

isolated points. To address these problems, we propose a novel indicator constrained and balanced multi-graph normalized cut method to identify the connectome module patterns from the connectivity brain networks of the targeted subject group. We evaluated our method by analyzing the weighted fiber connectivity networks.

In this work we consider the problem of sematic part-labeling of 3-D meshesof ear implants. This is a challenging problem and automatic solutions are of high practical relevance, since they help to automate the design of hearing aids. The contribution of this work is a new framework which outperforms existing approaches for this task. To achieve the boost in performance we introduce the new concept of a global parametric transition prior. To our knowledge, this is the first time that such a generic prior is used for 3-D mesh processing, and it may be found useful for a large class of 3-D meshes. To foster more research on the important topic of ear implant labeling, we collected a large data set of 3-D meshes, with associated ground truth labels, which we will make publicly available.

In this paper, we present a novel approach to candidate ground truth label generation for large-scale medical image collections by combining clinically-relevant textual and visual analysis through the framework of ambiguous label learning. In particular, we present a novel string matching algorithm for extracting disease labels from patient reports associated with imaging studies. These are assigned as ambiguous labels to the images of the study. Visual analysis is then performed on the images of the study and diagnostically relevant features are extracted from relevant regions within images. Finally, we learn the correlation between the ambiguous disease labels and visual features through an ambiguous SVM learning framework. The approach was validated in a large Doppler image collection of over 7000 images showing a scalable way to semi-automatically ground truth large image collections.

Traumatic brain injury (TBI) affects over 1.5 million Americans each year, and more than 75% of TBI cases are classified as mild (mTBI). Several functional network alternations have been reported after mTBI; however, the network alterations on a large scale, particularly on connectome scale, are still unknown. To analyze brain network, in a previous work, 358 landmarks named dense individualized common connectivity based cortical landmarks (DICCCOL) were identified on cortical surface. These landmarks preserve structural connection consistency and maintain functional correspondence across subjects. Hence DICCCOLs have been shown powerful in identifying connectivity signatures in affected brains. However, on such fine scales, the longitudinal changes in brain network of mTBI patients were complicated by the noise embedded in the systems as well as the normal variability of individuals at different times. Faced with such problems, we proposed a novel framework to analyze longitudinal changes from the perspective of network clusters. Specifically, multiview spectral clustering algorithm was applied to cluster brain networks based on DICCCOLs. And both structural and functional networks were analyzed. Our results showed that significant longitudinal changes were identified from mTBI patients that can be related to the neurocognitive recovery and the brain’s effort to compensate the effect of injury.

Cardiac Phase-resolved Blood Oxygen-Level-Dependent (CP-BOLD) MRI is a new contrast agent- and stress-free imaging technique for the assessment of myocardial ischemia at rest. The precise registration among the cardiac phases in this cine type acquisition is essential for automating the analysis of images of this technique, since it can potentially lead to better specificity of ischemia detection. However, inconsistency in myocardial intensity patterns and the changes in myocardial shape due to the heart’s motion lead to low registration performance for state-of-the-art methods. This low accuracy can be explained by the lack of distinguishable features in CP-BOLD and inappropriate metric definitions in current intensity-based registration frameworks. In this paper, the sparse representations, which are defined by a discriminative dictionary learning approach for source and target images, are used to improve myocardial registration. This method combines appearance with Gabor and HOG features in a dictionary learning framework to sparsely represent features in a low dimensional space. The sum of absolute differences of these distinctive sparse representations are used to define a similarity term in the registration framework. The proposed approach is validated on a dataset of CP-BOLD MR and standard CINE MR acquired in baseline and ischemic condition across 10 canines.

Existing feature descriptor-based methods on retinal image registration are mainly based on scale-invariant feature transform (SIFT) or partial intensity invariant feature descriptor (PIIFD). While these descriptors are many times being exploited, they have not been applied to color fundus and optical coherence tomography (OCT) fundus image pairs. OCT fundus images are challenging to register as they are often degraded by speckle noise. The descriptors also demand high dimensionality to adequately represent the features of interest. To this end, this paper presents a registration algorithm coined low-dimensional step pattern analysis (LoSPA), tailored to achieve low dimensionality while providing sufficient distinctiveness to effectively register OCT fundus images with color fundus photographs. The algorithm locates hypotheses of robust corner features based on connecting edges from the edge maps, mainly formed by vascular junctions. It continues with describing the corner features in a rotation invariant manner using step patterns. These customized step patterns are insensitive to intensity changes. We conduct comparative evaluation and LoSPA achieves a higher success rate in registration when compared to the state-of-the-art algorithms.

Deformable registration between pre-operative and follow-up scans of glioma patients is important since it allows us to map post-operative longitudinal progression of the tumor onto baseline scans, thus, to develop predictive models of tumor infiltration and recurrence. This task is very challenging due to large deformations, missing correspondences, and inconsistent intensity profiles between the scans. Here, we propose a new method that combines registration with estimation of patient specific templates. These templates, built from pre-operative and follow-up scans along with a set of healthy brain scans, approximate the patient’s brain anatomy before tumor development. Such estimation provides additional cues for missing correspondences as well as inconsistent intensity profiles, and therefore guides better registration on pathological regions. Together with our symmetric registration framework initialized by joint segmentation-registration using a tumor growth model, we are also able to estimate large deformations between the scans effectively. We apply our method to the scans of 24 glioma patients, achieving the best performance among compared registration methods.

Cortical surface registration or matching facilitates atlasing, cortical morphology-function comparison and statistical analysis. Methods that geodesically shoot surfaces into one another, as currents or varifolds, provide an elegant mathematical framework for generic surface matching and dynamic local features estimation, such as deformation momenta. However, conventional current and varifold matching methods only use the normals of the surface to measure its geometry and guide the warping process, which overlooks the importance of the direction in the convoluted cortical sulcal and gyral folds. To cope with the stated limitation, we decompose each cortical surface into its normal and tangent varifold representations, by integrating principal curvature direction field into the varifold matching framework, thus providing rich information for the direction of cortical folding and better characterization of the cortical geometry. To include more informative cortical geometric features in the matching process, we

adaptively

place control points based on the surface topography, hence the deformation is controlled by points lying on gyral crests (or “hills”) and sulcal fundi (or “valleys”) of the cortical surface, which are the most reliable and important topographic and anatomical landmarks on the cortex. We applied our method for registering the developing cortical surfaces in 12 infants from 0 to 6 months of age. Both of these variants

significantly

improved the matching accuracy in terms of closeness to the target surface and the precision of alignment with regional anatomical boundaries, when compared with several state-of-the-art methods: (1) diffeomorphic spectral matching, (2) current-based surface matching and (3) original varifold-based surface matching.

An atlas is a shape model derived using statistics of a population. Standard models treat local deformations as pure translations and apply linear statistics. They are often inadequate for highly variable anatomical shapes. Non-linear methods has been developed but are generally difficult to implement.

This paper proposes encoding shapes using the special Euclidean group

$\mathbb{SE}(3)$

for model construction.

$\mathbb{SE}(3)$

is a Lie group, so basic linear algebra can be used to analyze data in non-linear higher-dimensional spaces. This group represents non-linear shape variations by decomposing piecewise-local deformations into rotational and translational components.

The method was applied to 49 human liver models that were derived from CT scans. The atlas covered 99% of the population using only three components. Also, the method outperformed the standard method in reconstruction. Encoding shapes as ensembles of elements in the

$\mathbb{SE}(3)$

group is a simple way of constructing compact shape models.

Assessment of structural connectivity patterns of brains can be an important avenue for better understanding mechanisms of structural and functional brain architectures. Therefore, many efforts have been made to estimate and validate axonal pathways via a number of techniques, such as myelin stain, tract-tracing and diffusion MRI (dMRI). The three modalities have their own advantages and are complimentary to each other. From myelin stain data, we can infer rich in-plane information of axonal orientation at micro-scale. Tracttracing data is considered as ‘gold standard’ to estimate trustworthy meso-scale pathways. dMRI currently is the only way to estimate global macro-scale pathways given further validation. We propose a framework to take advantage of these three modalities. Information of the three modalities is integrated to determine the optimal tractography parameters for dMRI fibers and identify crossvalidated fiber bundles that are finally used to construct atlas. We demonstrate the effectiveness of the framework by a collection of experimental results.

Brain atlases are an integral component of neuroimaging studies. However, most brain atlases are fuzzy and lack structural details, especially in the cortical regions. In particular, neonatal brain atlases are especially challenging to construct due to the low spatial resolution and low tissue contrast. This is mainly caused by the image averaging process involved in atlas construction, often smoothing out high-frequency contents that indicate fine anatomical details. In this paper, we propose a novel framework for detail-preserving construction of atlases. Our approach combines space and frequency information to better preserve image details. This is achieved by performing reconstruction in the space-frequency domain given by wavelet transform. Sparse patch-based atlas reconstruction is performed in each frequency subband. Combining the results for all these subbands will then result in a refined atlas. Compared with existing atlases, experimental results indicate that our approach has the ability to build an atlas with more structural details, thus leading to better performance when used to normalize a group of testing neonatal images.

Aligning a pair of images in a mid-space is a common approach to ensuring that deformable image registration is symmetric – that it does not depend on the arbitrary ordering of the input images. The results are, however, generally dependent on the choice of the mid-space. In particular, the set of possible solutions is typically affected by the constraints that are enforced on the two transformations (that deform the two images), which are to prevent the mid-space from drifting too far from the native image spaces. The use of an implicit atlas has been proposed to define the mid-space for registration. In this work, by aligning the atlas to each image in the native image space, we make implicit-atlas-based pairwise registration independent of the mid-space, thereby eliminating the need for anti-drift constraints. We derive a new symmetric data term that only depends on a single transformation morphing one image to the other, and validate it through diffeomorphic registration experiments on brain MR images.

We present a 2D to 3D registration framework that compensates for prostate motion and deformations during freehand prostate biopsies. It has two major components: 1) a trajectory-based rigid registration to account for gross motions of the prostate; and 2) a non-rigid registration constrained by a finite element model (FEM) to adjust for residual motion and deformations. For the rigid alignment, we constrain the ultrasound probe tip in the live 2D imaging plane to the tracked trajectory from the pre-procedure 3D ultrasound volume. This ensures the rectal wall approximately coincides between the images. We then apply a FEM-based technique to deform the volume based on image intensities. We validate the proposed framework on 10 prostate biopsy patients, demonstrating a mean target registration error (TRE) of 4.63 mm and 3.15 mm for rigid and FEM-based components, respectively.

Image-based diagnosis and population study on cardiac problems require automatic segmentation on increasingly large amount of data from different protocols, different views, and different patients. However, current algorithms are often limited to regulated settings such as fixed view and single image from one specific modality, where the supervised learning methods can be easily employed but with restricted usability. In this paper, we propose the unsupervised free-view groupwise M

3

segmentation: a simultaneous segmentation for a group of

M

ulti-modality,

M

ulti-chamber, from

M

ulti-subject images from an arbitrary imaging view. To achieve the segmentation, we particularly develop the Synchronized Spectral Network (SSN) model for the joint decomposing, synchronizing, and clustering the spectral representations of free-view M

3

cardiac images. The SSN model generates a set of synchronized superpixels where the corresponding chamber regions share the same superpixel label, which naturally provides simultaneous cardiac segmentation. The segmentation is quantitatively evaluated by more than 10000 images (MR and CT) from 93 subjects and high dice metric (> 85%) is consistently achieved in validation. Our method provides a powerful segmentation tool for cardiac images under non-regulated imaging environment.

This paper presents an approach to estimate the uncertainty of registration parameters for the large displacement diffeomorphic metric mapping (LDDMM) registration framework. Assuming a local multivariate Gaussian distribution as an approximation for the registration energy at the optimal registration parameters, we propose a method to approximate the covariance matrix as the inverse of the Hessian of the registration energy to quantify registration uncertainty. In particular, we make use of a low-rank approximation to the Hessian to accurately and efficiently estimate the covariance matrix using few eigenvalues and eigenvectors. We evaluate the uncertainty of the LDDMM registration results for both synthetic and real imaging data.

Image registration is often very slow because of the high dimensionality of the images and complexity of the algorithms. Adaptive stochastic gradient descent (ASGD) outperforms deterministic gradient descent and even quasi-Newton in terms of speed. This method, however, only exploits first-order information of the cost function. In this paper, we explore a stochastic quasi-Newton method (s-LBFGS) for non-rigid image registration. It uses the classical limited memory BFGS method in combination with noisy estimates of the gradient. Curvature information of the cost function is estimated once every

L

iterations and then used for the next

L

iterations in combination with a stochastic gradient. The method is validated on follow-up data of 3D chest CT scans (19 patients), using a B-spline transformation model and a mutual information metric. The experiments show that the proposed method is robust, efficient and fast. s-LBFGS obtains a similar accuracy as ASGD and deterministic LBFGS. Compared to ASGD the proposed method uses about 5 times fewer iterations to reach the same metric value, resulting in an overall reduction in run time of a factor of two. Compared to deterministic LBFGS, s-LBFGS is almost 500 times faster.

Registration of Diffusion Weighted Images (DWI) is challenging as the data, in contrast to scalar-valued images, is a composition of both directional and intensity information. The DWI signal is known to be influenced by noise and a wide range of artifacts. Therefore, it is attractive to use similarity measures with invariance properties, such as Mutual Information. However, density estimation from DWI is complicated by directional information. We address this problem by extending Locally Orderless Registration (LOR), a density estimation framework for image similarity, to include directional information. We construct a spatio-directional scale-space formulation of marginal and joint density distributions between two DWI, that takes the projective nature of the directional information into account. This accounts for orientation and magnitude and enables us to use a wide range of similarity measures from the LOR framework. Using Mutual Information, we examine the properties of the scale-space induced by the choice of kernels and illustrate the approach by affine registration.

The alignment of brain imaging data for functional neuroimaging studies is challenging due to the discrepancy between correspondence of morphology, and equivalence of functional role. In this paper we map functional activation areas across individuals by a multi-atlas label fusion algorithm in a functional space. We learn the manifold of resting-state fMRI signals in each individual, and perform manifold alignment in an embedding space. We then transfer activation predictions from a source population to a target subject via multi-atlas label fusion. The cost function is derived from the aligned manifolds, so that the resulting correspondences are derived based on the similarity of intrinsic connectivity architecture. Experiments show that the resulting label fusion predicts activation evoked by various experiment conditions with higher accuracy than relying on morphological alignment. Interestingly, the distribution of this gain is distributed heterogeneously across the cortex, and across tasks. This offers insights into the relationship between intrinsic connectivity, morphology and task activation. Practically, the mechanism can serve as prior, and provides an avenue to infer task-related activation in individuals for whom only resting data is available.

Direct surgical navigation requires registration of an intraoperative imaging modality to a tracking technology, from which a patient image registration can be found. Although electromagnetic tracking is ergonomically attractive, it is used less often than optical tracking because its lower position accuracy offsets its higher orientation accuracy.

We propose a crossing-lines registration method for intraoperative electromagnetic tracking that uses a small disposable device temporarily attached to the patient. The method exploits the orientation accuracy of electromagnetic tracking by calculating directly on probed lines, avoiding the problem of acquiring accurately known probed points for registration. The calibration data can be acquired and computed in less than a minute (50 s ± 12 s). Laboratory tests demonstrated fiducial localization error with sub-degree and sub-millimeter means for 5 observers. A pre-clinical trial, on 10 shoulder models, achieved target registration error of 1.9 degree ± 1.8 degree for line directions and 0.8 mm ± 0.6 mm for inter-line distance. A Board-certified orthopedic surgeon verified that this accuracy easily exceeded the technical needs in shoulder replacement surgery.

This preclinical study demonstrated high application accuracy. The fast registration process and effective intraoperative method is promising for clinical orthopedic interventions where the target anatomy is small bone or has poor surgical exposure, as an adjunct to intraoperative imaging.

This paper presents a robust outlier elimination approach for multimodal retina image registration application. Our proposed scheme is based on the Scale-Invariant Feature Transform (SIFT) feature extraction and Partial Intensity Invariant Feature Descriptors (PIIFD), and we combined with a novel outlier elimination approach to robustly eliminate incorrect putative matches to achieve better registration results. Our proposed approach, which we will henceforth refer to as the residual-scaled-weighted Least Trimmed Squares (RSW-LTS) method, has been designed to enforce an affine transformation geometric constraint to solve the problem of image registration when there is very high percentage of incorrect matches in putatively matched feature points. Our experiments on registration of fundus-fluorescein angiographic image pairs show that our proposed scheme significantly outperforms the Harris-PIIFD scheme. We also show that our proposed RSW-LTS approach outperforms other outlier elimination approaches such as RANSAC (RANdom SAmple Consensus) and MSAC (M-estimator SAmple and Consensus).

This paper presents a new and highly efficient approach for finding correspondences across volumes with large motion. Most existing registration approaches are set in the continuous optimisation domain, which has severe limitations for estimating larger deformations. Feature-based approaches that rely on finding corresponding keypoints have been proposed, but they are prone to erroneous matching due to repetitive features and low contrast areas. This can be overcome by using a discrete optimisation approach. However, finding a constrained search space and regularisation strategy is still an open problem. Our method calculates a dissimilarity distribution over a densely sampled space of displacements for a small number of distinctive keypoints (found in only one volume). A parts-based model is used to infer smooth motion of connected keypoints and regularise the correspondence field. This effective and highly accurate approach is further improved by enforcing the symmetry of uncertainty estimates of displacements. Our method ranks first on one of the most challenging medical registration benchmarks for breath-hold CT scan-pairs of COPD patients, where accurate motion estimation is important for diagnosis.

Intravascular ultrasound (IVUS) is a widely used imaging technique for atherosclerotic plaque assessment, interventionist guidance, stent deploy visualization and, lately, as tissue characterization tool. Some IVUS applications solve the problem of transducer motion by gating a particular phase of the study while others, such as elastography or spatio-temporal vessel reconstruction, combine image data from different cardiac phases, for which the gating solution is not enough. In the latter, it is mandatory for the structures in different cardiac phases to be aligned (cross-sectional registration) and in the correct position along the vessel axis (longitudinal registration). In this paper, a novel method for transversal and longitudinal registration is presented, which minimizes the correlation of the structures between images in a local set of frames. To assess the performance of this method, frames immediately after carina bifurcation were marked at different cardiac phases and the error between registrations was measured. The results shown a longitudinal registration error of 0.3827 ±0.8250 frames.

Multi-compartment diffusion models (MCM) are increasingly used to characterize the brain white matter microstructure from diffusion MRI. We address the problem of interpolation and averaging of MCM images as a simplification problem based on spectral clustering. As a core part of the framework, we propose novel solutions for the averaging of MCM compartments. Evaluation is performed both on synthetic and clinical data, demonstrating better performance for the “covariance analytic” averaging method. We then present an MCM template of normal controls constructed using the proposed interpolation.

Interventional procedures in cardiovascular diseases often require ultrasound (US) image guidance. These US images must be combined with pre-operatively acquired tomographic images to provide a roadmap for the intervention. Spatial alignment of pre-operative images with intra-operative US images can provide valuable clinical information. Existing multi-modal US registration techniques often do not achieve reliable registration due to low US image quality. To address this problem, a novel medical image representation based on a trained decision forest named probabilistic edge map (PEM) is proposed in this paper. PEMs are generic and modality-independent. They generate similar anatomical representations from different imaging modalities and can thus guide a multi-modal image registration algorithm more robustly and accurately. The presented image registration framework is evaluated on a clinical dataset consisting of 10 pairs of 3D US-CT and 7 pairs of 3D US-MR cardiac images. The experiments show that a registration based on PEMs is able to estimate more reliable and accurate inter-modality correspondences compared to other state-of-the-art US registration methods.

Image templates, or atlases, play a critical role in imaging studies by providing a common anatomical coordinate system for analysis of shape and function. It is now common to estimate an atlas as a deformable average of the very images being studied, in order to provide a representative example of the particular population, imaging hardware, protocol, etc. However, when imaging data is aggregated across multiple sites, estimating an atlas from the pooled data fails to account for the variability of these factors across sites. In this paper, we present a hierarchical Bayesian model for diffeomorphic atlas construction of multi-site imaging data that explicitly accounts for the inter-site variability, while providing a global atlas as a common coordinate system for images across all sites. Our probabilistic model has two layers: the first consists of the average diffeomorphic transformations from the global atlas to each site, and the second consists of the diffeomorphic transformations from the site level to the individual input images. Our results on multi-site datasets, both synthetic and real brain MRI, demonstrate the capability of our model to capture inter-site geometric variability and give more reliable alignment of images across sites.

In recent works, 358 cortical landmarks named Dense Individualized Common Connectivity based Cortical Landmarks (DICCCOLs) were identified. Instead of whole-brain parcellation into sub-units, it identified the common brain regions that preserve consistent structural connectivity profile based diffusion tensor imaging (DTI). However, since the DICCCOL system was developed based on connectivity patterns only, morphological and geometric features were not used. Thus, in this paper, we constructed distance networks based on both geodesic distance and Euclidean distance to morphologically profile and characterize DICCCOL landmarks. Based on the distance network derived from 10 templates subjects with DICCCOL, we evaluated the anatomic consistency of each DICCCOL, identified reliable/unreliable DICCCOLs, and modeled the distance network of DICCCOLs. Our results suggested that the most relative consistent connections are long distance connections. Also, both of the distance measurements gave consistent observations and worked well in identifying anatomical consistent and inconsistent DICCCOLs. In the future, distance networks can be potentially applied as a complementary metric to improve the prediction accuracy of DICCCOLs or other ROIs defined on cortical surface.

Non-linear registration is an essential step in neuroimaging, influencing both structural and functional analyses. Although important, how different registration methods influence the results of these analyses is poorly known, with the metrics used to compare methods weakly justified. In this work we propose a framework to simulate true deformation fields derived from manually segmented volumes of interest. We test both state-of-the-art binary and non-binary, volumetric and surface -based metrics against these true deformation fields. Our results show that surface-based metrics are twice as sensitive as volume-based metrics, but are typically less used in non-linear registration evaluations. All analysed metrics poorly explained the true deformation field, with none explaining more than half the variance.

Depth assessment of 3D vascular models visualized on 2D displays is often difficult, especially in complex workspace conditions such as in the operating room. To address these limitations, we propose a new visualization technique for 3D vascular models. Our technique is tailored to static monoscopic 2D representations, as they are often used during surgery. To improve depth assessment, we propose a combination of supporting lines, view-aligned quads, and illustrative shadows. In addition, a hatching scheme that uses different line styles depending on a distance measure is applied to encode vascular shape as well as the distance to tumors. The resulting visualization can be displayed on monoscopic 2D monitors and on 2D printouts without the requirement to use color or intensity gradients. A qualitative study with 15 participants and a quantitative study with 50 participants confirm that the proposed visualization technique significantly improves depth assessment of complex 3D vascular models.

In CT perfusion imaging (CTP) multiple, consecutive 3D CT scans of an organ are made during the administration of contrast agent. This results in a 3D movie of the contrast agent entering and subsequently leaving the organ. Currently, this modality is mainly used for voxelwise analysis of perfusion parameters such as blood flow, blood volume, transit time etc. In this work, we propose to analyze these images in a more global fashion and introduce a method to infer the connectivity of the vascular structure underlying the perfusion – even if the vasculature itself is on a subvoxel scale. This novel approach enables several new applications for CTP. The feasibility of the method is illustrated on clinical data.

Prostate brachytherapy preplanning is the process of determining treatment target volume and arrangement of radioactive seeds w.r.t. the target volume prior to the implantation. Although preplanning is typically performed by a trained expert using a strict set of guidelines, the process remains highly subjective, resulting in significant user-dependent variability in the plans. In this work, we aim to reduce the preplanning variability by automating the seed arrangement process. We propose a novel framework which uses a retrospective treatment dataset to extract common radioactive seed patterns. The framework captures the inter-relation between the treatment volume delineation and seed arrangements through a joint sparse representation of retrospective data. This representation is used to estimate an initial seed arrangement for a new treatment volume, followed by a novel optimization process which captures the clinical guidelines, to fine-tune the seed arrangement. The proposed framework is evaluated on a dataset of 590 brachytherapy treatment cases by 5-fold cross validation. It achieves 86% success rate, when compared to the clinical guidelines and the actual plans.

Recent work on brain tumor growth modeling for glioblastoma using reaction-diffusion equations suggests that the diffusion coefficient and the proliferation rate can be related to clinically relevant information. However, estimating these parameters is difficult due to the lack of identifiability of the parameters, the uncertainty in the tumor segmentations, and the model approximation, which cannot perfectly capture the dynamics of the tumor. Therefore, we propose a method for conducting the Bayesian personalization of the tumor growth model parameters. Our approach estimates the posterior probability of the parameters, and allows the analysis of the parameters correlations and uncertainty. Moreover, this method provides a way to compute the evidence of a model, which is a mathematically sound way of assessing the validity of different model hypotheses. Our approach is based on a highly parallelized implementation of the reaction-diffusion equation, and the Gaussian Process Hamiltonian Monte Carlo (GPHMC), a high acceptance rate Monte Carlo technique. We demonstrate our method on synthetic data, and four glioblastoma patients. This promising approach shows that the infiltration is better captured by the model compared to the speed of growth.

We propose a parametric lumped model (LM) for fast patient-specific computational fluid dynamic simulations of blood flow in elongated vessel networks to alleviate the computational burden of 3D finite element (FE) simulations. We learn the coefficients balancing the local nonlinear hydraulic effects from a training set of precomputed FE simulations. Our LM yields pressure predictions accurate up to 2.76mmHg on 35 coronary trees obtained from 32 coronary computed tomography angiograms. We also observe a very good predictive performance on a validation set of 59 physiological measurements suggesting that FE simulations can be replaced by our LM. As LM predictions can be computed extremely fast, our approach paves the way to use a personalised interactive biophysical model with realtime feedback in clinical practice.

Precise estimation of computational physiological model parameters from patient data is one of the main hurdles towards their clinical applicability. Designing robust estimation algorithms is often a tedious and model-specific process. We propose to use, for the first time to our knowledge, artificial intelligence (AI) concepts to learn how to personalize a computational model, inspired by how an expert manually personalizes. We reformulate the parameter estimation problem in terms of Markov decision process and reinforcement learning. In an off-line phase, the artificial agent, called Vito, automatically learns a representative state-action-state model through data-driven exploration of the computational model under consideration. In other words, Vito learns how the model behaves under change of parameters and how to personalize it. Vito then controls the on-line personalization by exploiting its automatically derived action policy. Because the algorithm is model-independent, personalizing a completely new model would require only adjusting some simple parameters of the agent and defining the observations to match, without the full knowledge of the model itself. Vito was evaluated on two challenging problems: the inverse problem of cardiac electrophysiology and the personalization of a lumped-parameter whole-body circulation model. Obtained results suggested that Vito could achieve equivalent goodness of fit than standard methods, while being more robust (up to 25% higher success rates) and with faster (up to three times) convergence rate. Our AI approach could thus make model personalization algorithms generalizable and self-adaptable to any patient, like a human operator.

The insufflation of the abdomen in laparoscopic liver surgery leads to significant deformation of the liver. The estimation of the shape and position of the liver after insufflation has many important applications, such as providing surface-based registration algorithms used in image guidance with an initial guess and realistic patient-specific surgical simulation.

Our proposed algorithm computes a deformation estimate for a patient subject from a database of known insufflation deformations, as a weighted average. The database is built from pre-operative and intra-operative 3D image segmentations. The estimation pipeline also comprises a biomechanical simulation to incorporate patient-specific boundary conditions (BCs) and eliminate any non-physical deformation arising from the computation of the deformation as a weighted average.

We have evaluated the accuracy of our intra-subject registration, used for the computation of the displacements stored in the database, and our liver deformation predictions based on segmented, in-vivo porcine CT image data from 5 animals and manually selected vascular landmarks. We found root mean squared (RMS) target registration errors (TREs) of 2.96-11.31mm after intra-subject registration. For our estimated deformation, we found an RMS TRE of 5.82-11.47mm for four of the subjects, on one outlier subject the method failed.

3D ultrasound imaging has high potential for various clinical applications, but often suffers from high operator-dependency and the directionality of the acquired data. State-of-the-art systems mostly perform compounding of the image data prior to further processing and visualization, resulting in 3D volumes of scalar intensities. This work presents computational sonography as a novel concept to represent 3D ultrasound as tensor instead of scalar fields, mapping a full and arbitrary 3D acquisition to the reconstructed data. The proposed representation compactly preserves significantly more information about the anatomy-specific and direction-depend acquisition, facilitating both targeted data processing and improved visualization. We show the potential of this paradigm on ultrasound phantom data as well as on clinically acquired data for acquisitions of the femoral, brachial and antebrachial bone.

Vortex ring formation within the cardiac left ventricular (LV) blood flow has recently gained much interest as an efficient blood transportation mechanism and a potential early predictor of the chamber remodeling. In this work we propose a new method for automatic identification of vortex rings in the LV by means of 4D Flow MRI. The proposed method consists of three elements: 1) the 4D Flow MRI flow field is transformed into a 3D vortical scalar field using a well-established fluid dynamics-based vortex detection technique. 2) a shape signature of the cardiac vortex ring isosurface is derived from the probability distribution function of pairwise distances of randomly sampled points over the isosurface 3) a hierarchical clustering is then proposed to simultaneously identify the best isovalue that defines a vortex ring as well as the isosurface that corresponds to a vortex ring in the given vortical scalar field. The proposed method was evaluated in a datasets of 24 healthy controls as well as a dataset of 23 congenital heart disease patients. Results show great promise not only for vortex ring identification but also for allowing an objective quantification of vortex ring formation in the LV.

In this work, we propose to tackle the problem of magnetic resonance (MR)-based radiotherapy treatment planning in the head & neck area by synthesising computed tomography (CT) from MR images using an iterative multi-atlas approach. The proposed method relies on pre-acquired pairs of non-rigidly aligned T2-weighted MRI and CT images of the neck. To synthesise a pseudo CT, all the MRIs in the database are first registered to the target MRI using a robust affine followed by a deformable registration. An initial pseudo CT is obtained by fusing the mapped atlases according to their morphological similarity to the target. This initial pseudo CT is then combined with the target MR image in order to improve both the registration and fusion stages and refine the synthesis in the bone region.

Results showed that the proposed iterative CT synthesis algorithm is able to generate pseudo CT images in a challenging region for registration algorithms. We demonstrate that the robust affine decreases the overall absolute error compared to a single affine transformation, mainly in images with small axial field-of-view, whilst the bone refinement process further reduces the error in the bone region, increasing image sharpness.

The Mean Aneurysm Flow Amplitude ratio (MAFA-ratio) has been proposed to evaluate the efficacy of flow diverting stents during minimally invasive intracranial aneurysm treatment. A method has been described for calculating the MAFA-ratio on high frame-rate digital subtraction angiography (DSA) acquisitions using an optical flow algorithm. In this article we have generated computational fluid dynamics (CFD) simulations using six distinct aneurysms and computed the MAFA-ratios based on these data. Furthermore, the simulations have been used to create virtual angiograms, in order to calculate the MAFA-ratios using the DSA approach. An analysis of the MAFAratios generated by both methods shows that there is a monotone increasing relation between the DSA and CFD based ratios, albeit without a slope being identity. Overall, it can be concluded that the DSA-based ratio is a predictor for the magnitude of aneurysm flow reduction, i.e., for the efficacy of flow diverting stents.

Inverse problem of electrocardiography (ECG) has been extensively investigated as the estimated epicardial potentials (EPs) reflecting underlying myocardial activities. Traditionally, L2-norm regularization methods have been proposed to solve this ill-posed problem. But L2-norm penalty function inherently leads to considerable smoothing of the solution, which reduces the accuracy of distinguishing abnormalities and locating diseased regions. Directly using L1-norm penalty function, however, may greatly increase the computational complexity due to its non-differentiability. In this study, we present a smoothed L0 norm technique in order to directly solve the L0 norm constrained problem. Our method employs a smoothing function to make the L0 norm continuous. Extensive experiments on various datasets, including normal human data, isolated canine data, and WPW syndrome data, were conducted to validate our method. Epicardial potentials mapped during pacing were also reconstructed and visualized on the heart surface. Experimental results show that the proposed method reconstructs more accurate epicardial potentials compared with L1 norm and L2 norm based methods, demonstrating that smoothed L0 norm is a promising method for the noninvasive estimation of epicardial potentials.

Focal cortical dysplasia (FCD), a malformation of cortical development, is a frequent cause of drug-resistant epilepsy. This lesion is histologically classified into Type-IIA (dyslamination, dysmorphic neurons) and Type-IIB (dyslamination, dysmorphic neurons, and balloon cells). Reliable

in-vivo

identification of lesional subtypes is important for preoperative decision-making and surgical prognostics. We propose a novel multi-modal MRI lesion profiling based on multiple surfaces that systematically sample intra- and subcortical tissue. We applied this framework to histologically-verified FCD. We aggregated features describing morphology, intensity, microstructure, and function from T1-weighted, FLAIR, diffusion, and resting-state functional MRI. We observed alterations across multiple features in FCD Type-IIB, while anomalies in IIA were subtle and mainly restricted to FLAIR intensity and regional functional homogeneity. Anomalies in Type-IIB were seen across all intra- and sub-cortical levels, whereas those in Type-IIA clustered at the cortico-subcortical interface. A supervised classifier predicted the FCD subtype with 91% accuracy, validating our profiling framework at the individual level.

The simulation of medical ultrasound from patient-specific data may improve the planning and execution of interventions e.g. in the field of neurosurgery. However, both the long computation times and the limited realism due to lack of acoustic information from tomographic scans prevent a wide adoption of such a simulation. In this work, we address these problems by proposing a novel efficient ultrasound simulation method based on convolutional ray-tracing which directly takes volumetric image data as input. We show how the required acoustic simulation parameters can be derived from a segmented MRI scan of the patient. We also propose an automatic optimization of ultrasonic simulation parameters and tissue-specific acoustic properties from matching ultrasound and MRI scan data. Both qualitative and quantitative evaluation on a database of 14 neurosurgical patients demonstrate the potential of our approach for clinical use.

Noninvasive inference of patient-specific intramural electrical activity from surface electrocardiograms (ECG) lacks a unique solution in the absence of prior assumptions. While 3D cardiac electrophysiological models emerged to be a viable vehicle for constraining this inference with knowledge about the spatiotemporal dynamics of cardiac excitation, it is important for the inference to be robust to errors in these high-dimensional model predictions given the limited ECG data. We present an innovative solution to this problem by exploiting the low-dimensional structure of the solution space – a powerful regularizer in overcoming the lack of measurements –

within

the dynamic inference guided by physiological models. We present the first Bayesian inference framework that allows the exploration of both the spatial sparsity of cardiac excitation and its complex nonlinear spatiotemporal dynamics for an improved inference of patient-specific intramural electrical activity. The benefit of this integration is verified in both synthetic and real-data experiments, where we present one of the first detailed, point-by-point comparison of the reconstructed electrical activity to

in-vivo

catheter mapping data.

Physiological and biophysical models have been proposed to link neural activity to the Blood Oxygen Level-Dependent (BOLD) signal in functional MRI (fMRI). They rely on a set of parameter values that cannot always be extracted from the literature. Their estimation is challenging because there are more than 10 potentially interesting parameters involved in non-linear equations and whose interactions may result in identifiability issues. However, the availability of statistical prior knowledge on these parameters can greatly simplify the estimation task. In this work we focus on the extended Balloon model and propose the estimation of 15 parameters using an Evolutionary Computation (EC) global search method. To combine both the ability to escape local optima and to incorporate prior knowledge, we derive the EC objective function from Bayesian modeling. This novel method provides promising results on a challenging real fMRI data set involving rats with epileptic activity and compares favorably with the conventional Expectation Maximization Gauss-Newton approach.

We present a new technique to overcome a major disadvantage of SPECT-guided surgery, where a 3D image of the distribution of a radiotracer augments the live view of the surgical situs in order to identify radiopositive tissue for resection and subsequent histological analysis. In current systems, the reconstructed SPECT volume is outdated as soon as the situs is modified by further surgical actions, due to tissue displacement. Our technique intraoperatively estimates the displacement of radiopositive tissue, which enables the update of the SPECT image augmentation. After the initial SPECT reconstruction is complete, we deploy a 2D

γ

-camera along with a technique to optimize its placement. We automatically establish a correspondence between regions of interest in the reconstructed volume and the near real-time 2D

γ

images. The 3D displacement of the radiopositive nodules is then continuously estimated based on the processing of the aforementioned

γ

-camera’s output. Initial results show that we can estimate displacements with ±1 mm accuracy.

Analysis of hemodynamics shows great potential to provide indications for the risk of cardiac malformations and is essential for diagnostic purposes in clinical applications. Computational fluid dynamics (CFD) has been established as a valuable tool for the detailed characterization of volumetric blood flow and its effects on the arterial wall. However, studies concentrating on the aortic root have to take the turbulent nature of the flow into account while no satisfactory solution for such simulations exists today. In this paper we propose to combine magnetic resonance imaging (MRI) flow acquisitions, providing excellent data in the turbulent regions while showing only limited reliability in the boundary layer, with CFD simulations which can be used to extrapolate the measured data towards the vessel wall. The solution relies on a partial domain approach, restricting the simulations to the laminar flow domain while using MRI measurements as additional boundary conditions to drive the numerical simulation. In this preliminary work we demonstrate the feasibility of the method on flow phantom measurements while comparing actually measured and simulated flow fields under straight and spiral flow regimes.

We present a method to correct motion in fetal in-utero scan sequences. The proposed approach avoids previously necessary manual segmentation of a region of interest. We solve the problem of non-rigid motion by splitting motion corrupted slices into overlapping patches of finite size. In these patches the assumption of rigid motion approximately holds and they can thus be used to perform a slice-to-volume-based (SVR) reconstruction during which their consistency with the other patches is learned. The learned information is used to reject patches that are not conform with the motion corrected reconstruction in their local areas. We evaluate rectangular and evenly distributed patches for the reconstruction as well as patches that have been derived from super-pixels. Both approaches achieve on 29 subjects aged between 22–37 weeks a sufficient reconstruction quality and facilitate following 3D segmentation of fetal organs and the placenta.

Parallel magnetic resonance imaging (pMRI) is a useful technique to aid clinical diagnosis. In this paper, we develop an accelerated algorithm for joint total variation (JTV) regularized calibrationless Parallel MR image reconstruction. The algorithm minimizes a linear combination of least squares data fitting term and the joint total variation regularization. This model has been demonstrated as a very powerful tool for parallel MRI reconstruction. The proposed algorithm is based on the iteratively reweighted least squares (IRLS) framework, which converges exponentially fast. It is further accelerated by preconditioned conjugate gradient method with a well-designed preconditioner. Numerous experiments demonstrate the superior performance of the proposed algorithm for parallel MRI reconstruction in terms of both accuracy and efficiency.

Ophthalmoscopes have yet to capitalise on novel low-cost miniature optomechatronics, which could disrupt ophthalmic monitoring in rural areas. This paper demonstrates a new design integrating modern components for ophthalmoscopy. Simulations show that the optical elements can be reduced to just two lenses: an aspheric ophthalmoscopic lens and a commodity liquid-lens, leading to a compact prototype. Circularly polarised transpupilary illumination, with limited use so far for ophthalmoscopy, suppresses reflections, while autofocusing preserves image sharpness. Experiments with a human-eye model and cadaver porcine eyes demonstrate our prototype’s clinical value and its potential for accessible imaging when cost is a limiting factor.

C-arm computed tomography reconstruction of multiple cardiac phases could provide a highly useful tool to interventional cardiologists in the catheter laboratory. Today, however, for clinically reasonable acquisition protocols the achievable image quality is still severely limited due to undersampling artifacts. We propose an iterative optimization scheme combining image registration, motion compensation and spatio-temporal regularization to improve upon the state-of-the-art w.r.t. image quality and accuracy of motion estimation. Evaluation of clinical cases indicates an improved visual appearance and temporal consistency, evidenced by a strong decrease in temporal variance in uncontrasted regions accompanied by an increased sharpness of the contrasted left ventricular blood pool boundary. In a phantom study, the universal image quality index proposed by Wang et al. is raised from 0.80 to 0.95, with 1.0 corresponding to a perfect match with the ground truth. The results lay a promising foundation for interventional cardiac functional analysis.

This work introduces a robust framework for predicting Deep Brain Stimulation (DBS) target structures which are not identifiable on standard clinical MRI. While recent high-field MR imaging allows clear visualization of DBS target structures, such high-fields are not clinically available, and therefore DBS targeting needs to be performed on the standard clinical low contrast data. We first learn via regression models the shape relationships between DBS targets and their potential predictors from high-field (7 Tesla) MR training sets. A bagging procedure is utilized in the regression model, reducing the variability of learned dependencies. Then, given manually or automatically detected predictors on the clinical patient data, the target structure is predicted using the learned high quality information. Moreover, we derive a robust way to properly weight different training subsets, yielding higher accuracy when using an ensemble of predictions. The subthalamic nucleus (STN), the most common DBS target for Parkinson’s disease, is used to exemplify within our framework. Experimental validation from Parkinson’s patients shows that the proposed approach enables reliable prediction of the STN from the clinical 1.5T MR data.

Ultrasound is becoming an ever increasingly important modality in medical care. However, underlying physical acquisition principles are prone to image artifacts and result in overall quality variation. Therefore processing medical ultrasound data remains a challenging task. We propose a novel distribution-based measure of assessing the confidence in the signal, which emphasizes uncertainty in attenuated as well as shadow regions. In contrast to the similar recently proposed method that relies on image intensities, the new approach makes use of the enveloped-detected radio-frequency data, facilitating the use of Nakagami speckle statistics. Employing J-divergence as distance measure for the random-walk based algorithm, provides a natural measure of similarity, yielding a more reliable estimate of confidence. For evaluation of the model’s performance, tests are conducted on the application of shadow detection. Additionally, computed maps are presented for different organs such as neck, liver and prostate, showcasing the properties of the model. The probabilistic approach is shown to have beneficial features for image processing tasks.

Thin-cap fibroatheroma (TCFA) is particularly prone to rupture, which may result in myocardial infarction and death. Virtual histology intravascular ultrasound (VH-IVUS) provides quantitative information about plaque composition and enables TCFA identification. However, prospective prediction of future development of TCFA has not been previously possible. The aim of our study was to determine whether subsequent development of TCFA can be predicted from baseline VH-IVUS data. Corresponding VH-IVUS images of baseline and follow-up examinations were identified by a highly automated approach to register IVUS pullback pairs from 24 patients (2,331 image pairs). Next, 20 location-specific VH-based and IVUS-based features including plaque phenotype and morphology, and 15 systemic patient-specific features were extracted and ranked using a support vector machine recursive feature elimination (SVM RFE) technique. SVM was applied to assess the prediction power of different feature sets, by adding the first n-ranked features to the classification procedure (leave-one-patient-out cross validation) iteratively until all features were considered. The experimental results showed that the prospective prediction of TCFA achieves a sensitivity of 72.6% and a specificity of 73.3%, when an optimal set of the five best selected features is used. The results indicate the feasibility of prospective prediction of TCFA formation based on baseline VH-IVUS data.

Sentinel lymph node (sLN) biopsy mostly requires an invasive surgical intervention to remove sLNs under radioguidance. We present an alternative method where live ultrasound is combined with live robotic gamma imaging to provide real-time anatomical and nuclear guidance of punch biopsies. The robotic arm holding a gamma camera is equipped with a system for inside-out tracking to directly retrieve the relative position of the US transducer with respect to itself. Based on this, the system cooperatively positions the gamma camera parallel to the US imaging plane selected by the physician for real-time multi-modal visualization. We validate the feasibility of this approach with a dedicated gelatine/agar biopsy phantom and show that lymph nodes separated by at least 10 mm can be distinguished.

This paper describes the use of Monte Carlo sampling for tomographic image reconstruction. We describe an efficient sampling strategy, based on the Riemannian Manifold Markov Chain Monte Carlo algorithm, that exploits the peculiar structure of tomographic data, enabling efficient sampling of the high-dimensional probability densities that arise in tomographic imaging. Experiments with positron emission tomography (PET) show that the method enables the quantification of the uncertainty associated with tomographic acquisitions and allows the use of arbitrary risk functions in the reconstruction process.

The exposure index (EI) gives a feedback to radiographers on the image quality in digital radiography, but its estimation on clinical images raises many challenges. In this paper we provide a critical overview of state of the art methods that address this problem and we show that more robust results can be obtained by detecting anatomical structures. This new approach implicitly manages the presence of multiple structures in the field-of-view. Moreover, we propose a landmark-based method that, by exploiting redundancy of local estimates, is more robust to potential detection errors.

In this paper, we propose a novel compressive sensing model for dynamic MR reconstruction. With total variation (TV) and nuclear norm (NN) regularization, our method can utilize both spatial and temporal redundancy in dynamic MR images. Due to the non-smoothness and non-separability of TV and NN terms, it is difficult to optimize the primal problem. To address this issue, we propose a fast algorithm by solving a primal-dual form of the original problem. The ergodic convergence rate of the proposed method is

$\mathcal{O}(1/N)$

for N iterations. In comparison with six state-of-the-art methods, extensive experiments on single-coil and multi-coil dynamic MR data demonstrate the superior performance of the proposed method in terms of both reconstruction accuracy and time complexity.

Respiratory motion presents significant challenges for PET/ CT acquisitions, potentially leading to inaccurate SUV quantitation. Non Rigid Registration [NRR] of gated PET images is quite challenging due to large motion, intrinsic noise, and the need to preserve definitive features like tumors. In this work, we use non-local spatio-temporal constraints within group-wise NRR to get a stable framework which can work with few number of PET gates, and handle the above challenges of PET data. Additionally, we propose metrics for measuring alignment and artifacts introduced by NRR which is rarely addressed. Our results are quantitatively compared to related works, on 20 clinical PET cases.

Abnormalities in cerebral glucose metabolism detectable on fluorodeoxyglucose positron emission tomography (FDG PET) can be assessed on a regional or voxel-wise basis. In regional analysis, the average relative uptake over a region of interest is compared with the average relative uptake obtained for normal controls. Prior knowledge is required to determine the regions where abnormal uptake is expected, which can limit its usability. On the other hand, voxel-wise analysis consists of comparing the metabolic activity of the patient to the normal controls voxel-by-voxel, usually in a groupwise space. Voxel-based techniques are limited by the inter-subject morphological and metabolic variability in the normal population, which can limit their sensitivity.

In this paper, we combine the advantages of both regional and voxel-wise approaches through the use of subject-specific PET models for glucose metabolism. By accounting for inter-subject morphological differences, the proposed method aims to remove confounding variation and increase the sensitivity of group-wise approaches. The method was applied to a dataset of 22 individuals: 17 presenting four distinct neurodegenerative syndromes, and 5 controls. The proposed method more accurately distinguishes subgroups in this set, and improves the delineation of disease-specific metabolic patterns.

Advancements in 7T MR imaging bring higher spatial resolution and clearer tissue contrast, in comparison to the conventional 3T and 1.5T MR scanners. However, 7T MRI scanners are less accessible at the current stage due to higher costs. Through analyzing the appearances of 7T images, we could improve both the resolution and quality of 3T images by properly mapping them to 7T-like images; thus, promoting more accurate post-processing tasks, such as segmentation. To achieve this method based on an unique dataset acquired both 3T and 7T images from same subjects, we propose novel multi-level Canonical Correlation Analysis (CCA) method and group sparsity as a hierarchical framework to reconstruct 7T-like MRI from 3T MRI. First, the input 3T MR image is partitioned into a set of overlapping patches. For each patch, the local coupled 3T and 7T dictionaries are constructed by extracting the patches from a neighboring region from all aligned 3T and 7T images in the training set. In the training phase, we have both 3T and 7T MR images scanned from each training subject. Then, these two patch sets are mapped to the same space using multi-level CCA. Next, each input 3T MRI patch is sparsely represented by the 3T dictionary and then the obtained sparse coefficients are utilized to reconstruct the 7T patch with the corresponding 7T dictionary. Group sparsity is further utilized to maintain the consistency between neighboring patches. Such reconstruction is performed hierarchically with adaptive patch size. The experiments were performed on 10 subjects who had both 3T and 7T MR images. Experimental results demonstrate that our proposed method is capable of recovering rich structural details and outperforms other methods, including the sparse representation method and CCA method.

Fat quantification based on the multi-echo Dixon method is gaining importance in clinical practice as it can match the accuracy of spectroscopy but provides high spatial resolution. Accurate quantification protocols, though, are limited to low SNR and suffer from a high noise bias. As the clinically relevant water and fat components are estimated by fitting a non-linear signal model to the data, the uncertainty is further amplified. In this work, we first establish the low-rank property and its locality assumptions for water-fat MRI and, consequently, propose a model-consistent but adaptive spectral denoising. A robust noise estimation in combination with a risk-minimizing threshold adds to a fully-automatic method. We demonstrate its capabilities on abdominal fat quantification data from in-vivo experiments. The denoising reduces the fit error on average by 37% and the uncertainty of the fat fraction by 58% in comparison to the original data while being edge-preserving.

Imaging devices exploit the Nyquist-Shannon sampling theorem to avoid both aliasing and redundant oversampling by design. Conversely, in medical image resampling, images are considered as continuous functions, are warped by a spatial transformation, and are then sampled on a regular grid. In most cases, the spatial warping changes the frequency characteristics of the continuous function and no special care is taken to ensure that the resampling grid respects the conditions of the sampling theorem. This paper shows that this oversight introduces artefacts, including aliasing, that can lead to important bias in clinical applications. One notable exception to this common practice is when multi-resolution pyramids are constructed, with low-pass ”anti-aliasing” filters being applied prior to downsampling. In this work, we illustrate why similar caution is needed when resampling images under general spatial transformations and propose a novel method that is more respectful of the sampling theorem, minimising aliasing and loss of information. We introduce the notion of scale factor point spread function (sfPSF) and employ Gaussian kernels to achieve a computationally tractable resampling scheme that can cope with arbitrary non-linear spatial transformations and grid sizes. Experiments demonstrate significant (

p

 < 10

− 4

) technical and clinical implications of the proposed method.

Magnetic resonance (MR) imaging provides a unique in-vivo capability of visualizing tissue in the human brain non-invasively, which has tremendously improved patient care over the past decades. However, there are still prominent artifacts, such as intensity inhomogeneities due to the use of an array of receiving coils (RC) to measure the MR signal or noise amplification due to accelerated imaging strategies. It is critical to mitigate these artifacts for both visual inspection and quantitative analysis. The cornerstone to address this issue pertains to the knowledge of coil sensitivity profiles (CSP) of the RCs, which describe how the measured complex signal decays with the distance to the RC.

Existing methods for CSP estimation share a number of limitations: (i) they primarily focus on CSP magnitude, while it is known that the solution to the MR image reconstruction problem involves complex CSPs and (ii) they only provide point estimates of the CSPs, which makes the task of optimizing the parameters and acquisition protocol for their estimation difficult. In this paper, we propose a novel statistical framework for estimating complex-valued CSPs. We define a CSP estimator that uses spatial smoothing and additional body coil data for phase normalization. The main contribution is to provide detailed information on the statistical distribution of the CSP estimator, which yields automatic determination of the optimal degree of smoothing for ensuring minimal bias and provides guidelines to the optimal acquisition strategy.

Computed tomography (CT) is a widely used medical technology. Adding 3D imaging to a mobile fluoroscopic C-arm reduces the cost of CT, as a mobile C-arm is much less expensive than a dedicated CT scanner. In this paper we explore the technical challenges to implementing 3D reconstruction on these devices. One of the biggest challenges is the problem of uncertain geometry; mobile C-arms do not have the same geometric consistency that exists in larger dedicated CT scanners. The geometric parameters of an acquisition scan are therefore uncertain, and a naïve reconstruction with these incorrect parameters leads to poor image quality. Our proposed method reconstructs the 3D image using the expectation maximization (EM) framework while jointly estimating the true geometry, thereby improving the feasibility of 3D imaging on mobile C-arms.

Real-time reconstruction in multi-contrast magnetic resonance imaging (MC-MRI) is very challenging due to the slow scanning and reconstruction process. In this study, we propose a novel algorithm to accelerate the MC-MRI reconstruction in the framework of compressed sensing. The problem is formulated as the minimization of the least square data fitting with joint total variation (JTV) regularization term. We first utilized the iterative reweighted least square (IRLS) framework to reformulate the problem. A joint preconditioner is dexterously designed to efficiently compute the inverse of large transform matrix at each iteration. We compared our algorithm with eight cutting-edge compressive sensing MRI algorithms on real MC-MRI dataset. Extensive experiments demonstrate that the proposed algorithm can achieve far better reconstruction performance than all other eight cutting-edge methods.