Medical Image Computing and Computer-Assisted Intervention – MICCAI 2009

The preferred method of treatment for Atrial Fibrillation (AF) is by catheter ablation wherein a catheter is guided into the left atrium through a transseptal puncture. However, the transseptal puncture constrains the catheter, thereby limiting its maneuverability and increasing the difficulty in reaching various locations in the left atrium. In this paper, we address the problem of choosing the optimal transseptal puncture location for performing cardiac ablation to obtain maximum maneuverability of the catheter. We have employed an optimization algorithm to maximize the Global Isotropy Index (GII) to evaluate the optimal transseptal puncture location. As part of this algorithm, a novel kinematic model for the catheter has been developed based on a continuum robot model. Preoperative MR/CT images of the heart are segmented using the open source image-guided therapy software, Slicer 3, to obtain models of the left atrium and septal wall. These models are input to the optimization algorithm to evaluate the optimal transseptal puncture location. Simulation results for the optimization algorithm are presented in this paper.

This paper describes a novel method for improving the navigation and guidance of devices and catheters in electrophysiology and interventional cardiology procedures using volumetric data fusion. The clinical workflow includes the acquisition and reconstruction of CT data from a C-arm X-ray angiographic system and the real-time acquisition of volumetric ultrasound datasets with a new intracardiac real-time 3D ultrasound catheter. Mono- and multi-modal volumetric registration methods, as well as visualization modes, that are suitable for real-time fusion are described, which are the key components of this work. Evaluation on phantom and in-vivo animal data shows that it is feasible to register and track the motion of real-time 3D intracardiac ultrasound in C-arm CT.

Pulmonary valve disease affects a significant portion of the global population and often occurs in conjunction with other heart dysfunctions. Emerging interventional methods enable percutaneous pulmonary valve implantation, which constitute an alternative to open heart surgery. As minimal invasive procedures become common practice, imaging and non-invasive assessment techniques turn into key clinical tools. In this paper, we propose a novel approach for intervention planning as well as morphological and functional quantification of the pulmonary trunk and valve. An abstraction of the anatomic structures is represented through a four-dimensional, physiological model able to capture large pathological variation. A hierarchical estimation, based on robust learning methods, is applied to identify the patient-specific model parameters from volumetric CT scans. The algorithm involves detection of piecewise affine parameters, fast centre-line computation and local surface delineation. The estimated personalized model enables for efficient and precise quantification of function and morphology. This ability may have impact on the assessment and surgical interventions of the pulmonary valve and trunk. Experiments performed on 50 cardiac computer tomography sequences demonstrated the average speed of 202 seconds and accuracy of 2.2mm for the proposed approach. An initial clinical validation yielded a significant correlation between model-based and expert measurements. To the best of our knowledge this is the first dynamic model of the pulmonary trunk and right ventricle outflow track estimated from CT data.

The manipulation of fast moving, delicate tissues in beating heart procedures presents a considerable challenge to surgeons. We present a new robotic force stabilization system that assists surgeons by maintaining a constant contact force with the beating heart. The system incorporates a novel, miniature uniaxial force sensor that is mounted to surgical instrumentation to measure contact forces during surgical manipulation. Using this sensor in conjunction with real-time tissue motion information derived from 3D ultrasound, we show that a force controller with feed-forward motion terms can provide safe and accurate force stabilization in an in vivo contact task against the beating mitral valve annulus. This confers a 50% reduction in force fluctuations when compared to a standard force controller and a 75% reduction in fluctuations when compared to manual attempts to maintain the same force.

This paper presents a new method to reconstruct the beating heart surface based on the non-rigid structure from motion technique using preprocessed endoscopic images. First the images captured at the same phase within each heart cycle are automatically extracted from the original image sequence to reduce the dimension of the deformation subspace. Then the remaining residual non-rigid motion is restricted to lie within a low-dimensional subspace and a probabilistic model is used to recover the 3D structure and camera motion simultaneously. Outliers are removed iteratively based on the reprojection error. Missing data are also recovered with an Expectation Maximization algorithm. As a result the camera can move around the operation scene to build a 3D surface with a wide field-of-view for intra-operative procedures. The method has been evaluated with synthetic data, heart phantom data, and in vivo data from a

da Vinci

surgical system.

Image Guided Radiation Therapy (IGRT) improves radiation therapy for prostate cancer by facilitating precise radiation dose coverage of the object of interest, and minimizing dose to adjacent normal organs. In an effort to optimize IGRT, we developed a fast segmentation-registration-segmentation framework to accurately and efficiently delineate the clinically critical objects in Cone Beam CT images obtained during radiation treatment. The proposed framework started with deformable models automatically segmenting the prostate, bladder, and rectum in planning CT images. All models were built around seed points and involved in the CT image under the influence of image features using the level set formulation. The deformable models were then converted into meshless point sets and underwent a 3D non rigid registration from the planning CT to the treatment CBCT. The motion of deformable models during the registration was constrained by the global shape prior on the target surface during the deformation. The meshless formulation provided a convenient interface between deformable models and the image feature based registration method. The final registered deformable models in the CBCT domain were further refined using the interaction between objects and other available image features. The segmentation results for 15 data sets has been included in the validation study, compared with manual segmentations by a radiation oncologist. The automatic segmentation results achieved a satisfactory convergence with manual segmentations and met the speed requirement for on line IGRT.

Surgical aspirators are one of the most frequently used neurosurgical tools. Effective training on a neurosurgery simulator requires a visually and haptically realistic rendering of surgical aspiration. However, there is little published data on mechanical interaction between soft biological tissues and surgical aspirators. In this study an experimental setup for measuring tissue response is described and results on calf brain and a range of phantom materials are presented. Local graphical and haptic models are proposed. They are simple enough for real-time application, and closely match the observed tissue response. Tissue resection (cutting) with suction is simulated using a volume sculpting approach. A simulation of suction is presented as a demonstration of the effectiveness of the approach.

In prostate brachytherapy, x-ray fluoroscopy has been used for intra-operative dosimetry to provide qualitative assessment of implant quality. More recent developments have made possible 3D localization of the implanted radioactive seeds. This is usually modeled as an assignment problem and solved by resolving the correspondence of seeds. It is, however, NP-hard, and the problem is even harder in practice due to the significant number of hidden seeds. In this paper, we propose an algorithm that can find an optimal solution from multiple projection images with hidden seeds. It solves an equivalent problem with reduced dimensional complexity, thus allowing us to find an optimal solution in polynomial time. Simulation results show the robustness of the algorithm. It was validated on 5 phantom and 18 patient datasets, successfully localizing the seeds with detection rate of ≥ 97.6 % and reconstruction error of ≤ 1.2 mm. This is considered to be clinically excellent performance.

Computer-assisted prostate biopsies became a very active research area during the last years. Prostate tracking makes it possible to overcome several drawbacks of the current standard transrectal ultrasound (TRUS) biopsy procedure, namely the insufficient targeting accuracy which may lead to a biopsy distribution of poor quality, the very approximate knowledge about the actual location of the sampled tissues which makes it difficult to implement focal therapy strategies based on biopsy results, and finally the difficulty to precisely reach non-ultrasound (US) targets stemming from different modalities, statistical atlases or previous biopsy series. The prostate tracking systems presented so far are limited to rigid transformation tracking. However, the gland can get considerably deformed during the intervention because of US probe pressure and patient movements. We propose to use 3D US combined with image-based elastic registration to estimate these deformations. A fast elastic registration algorithm that copes with the frequently occurring US shadows is presented. A patient cohort study was performed, which yielded a statistically significant in-vivo accuracy of 0.83±0.54mm.

Defects in articular cartilage can be repaired through osteochondral transplantation (mosaic arthroplasty), where osteochondral plugs from non-weight-bearing areas of the joint are transferred to the defect site. Incongruity between the plug surface and the adjacent cartilage results in increased contact pressures and poorer outcomes. We compare three methods to predict the desired repair surface for use in computer-assisted mosaic arthroplasty: manual estimation, a cubic spline surface, and a statistical shape atlas of the knee. The cubic spline was found to most accurately match the pre-impact cartilage surface; the atlas was found to match least accurately.

In this paper, we describe a rigid endoscope that transmits a laser beam coaxially to arbitrary points in the endoscopic view, mainly for treatment of twin-to-twin transfusion syndrome. The endoscope consists of a hotmirror for coaxial transmission of visible light and a Nd:YAG laser beam, and galvanometers for controlling the beam irradiation angle. We evaluated the transmission efficiency of the laser power, the spot size through the endoscope and accuracy in positioning the beam. The maximum laser transmission efficiency was 39% and the spot diameter was 2.2–3.2 mm at a distance of 10–20 mm. The positioning accuracy was mostly within 1.0 mm in the endoscopic view at the distance. The average laser power density on the spot was estimated to be 170–370 W/cm

2

, and a chicken liver was successfully coagulated by changing the laser beam irradiation angle.

Endoscopic endonasal skull base surgery (ESBS) requires high accuracy to ensure safe navigation of the critical anatomy at the anterior skull base. Current navigation systems provide approximately 2

mm

accuracy. This level of registration error is due in part from the indirect nature of tracking used. We propose a method to directly track the position of the endoscope using video data. Our method first reconstructs image feature points from video in 3D, and then registers the reconstructed point cloud to pre-operative data (e.g. CT/MRI). After the initial registration, the system tracks image features and maintains the 2D-3D correspondence of image features and 3D locations. These data are then used to update the current camera pose. We present registration results within 1mm, which matches the accuracy of our validation framework.

Spectral domain optical coherence tomography (SD-OCT) is an important tool for the diagnosis of various retinal diseases. The measurements available from SD-OCT volumes can be used to detect structural changes in glaucoma patients before the resulting vision loss becomes noticeable. Eye movement during the imaging process corrupts the data, making measurements unreliable. We propose a method to correct for transverse motion artifacts in SD-OCT volumes after scan acquisition by registering the volume to an instantaneous, and therefore artifact-free, reference image. Our procedure corrects for smooth deformations resulting from ocular tremor and drift as well as the abrupt discontinuities in vessels resulting from microsaccades. We test our performance on 48 scans of healthy eyes and 116 scans of glaucomatous eyes, improving scan quality in 96% of healthy and 73% of glaucomatous eyes.

We present initial prototype and preliminary experimental demonstration of a new class of microsurgical instruments that incorporate common path optical coherence tomography (CP-OCT) capabilities. These instruments may be used freehand or with robotic assistance. We describe a prototype 25 gauge microsurgical pick incorporating a single 125 (m diameter optical fiber interfaced to a Fourier Domain CP-OCT system developed in our laboratory. For initial experimentation, we have interfaced this instrument with an extremely precise, cooperatively controlled robot. We describe the tool, system design, and demonstration of three control methods on simple phantom models: 1) enforce ment of safety constraints preventing unintentional collisions of the instrument with the retinal surface; 2) the ability to scan the probe across a surface while maintaining a constant distance offset; and 3) the ability to place the pick over a subsurface target identified in a scan and then penetrate the surface to hit the target.

Fluoroscopic images contain useful information that is difficult to comprehend due to the collapse of the 3D information into 2D space. Extracting the informative layers and analyzing them separately could significantly improve the task of understanding the image content. Traditional Digital Subtraction Angiography (DSA) is not applicable for coronary angiography because of heart beat and breathing motion. In this work, we propose a layer extraction method for separating transparent motion layers in fluoroscopic image sequences, so that coronary tree can be better visualized.. The method is based on the fact that different anatomical structures possess different motion patterns,

e.g.

, heart is beating fast, while lung is breathing slower. A multiscale implementation is used to further improve the efficiency and accuracy. The proposed approach helps to enhance the visibility of the vessel tree, both visually and quantitatively.

Obtaining detailed, patient-specific blood flow information would be very useful in detecting and monitoring cardio-vascular diseases. Current approaches rely on computational fluid dynamics to achieve this; however, these are hardly usable in the daily clinical routine due to the required technical supervision and long computing times. We propose a fast measurement enhancement method that requires neither supervision nor long computation and it is the objective of this paper to evaluate its performance as compared to the state-of-the-art. To this end a large set of abdominal aortic bifurcation geometries was used to test this technique and the results were compared to measurements and numerical simulations. We find that this method is able to dramatically improve the quality of the measurement information, in particular the flow-derived quantities such as wall shear stress. Additionally, good estimation of unmeasurable quantities such as pressure can be provided. We demonstrate that this approach is a practical and clinically feasible alternative to fully-blown, time-consuming, patient-specific flow simulations.

Anatomical and functional information of cardiac vasculature is a key component of future developments in the field of interventional cardiology. With the technology of C-arm CT it is possible to reconstruct intraprocedural 3-D images from angiographic projection data. Current approaches attempt to add the temporal dimension (4-D) by ECG-gating in order to distinct physical states of the heart. This model assumes that the heart motion is periodic. However, frequently arrhytmic heart signals are observed in a clinical environment. In addition breathing motion can still occur. We present a reconstruction method based on a 4-D time-continuous motion field which is parameterized by the acquisition time and not the quasi-periodic heart phase. The output of our method is twofold. It provides a motion compensated 3-D reconstruction (anatomic information) and a motion field (functional information). In a physical phantom experiment a vessel of size 3.08 mm undergoing a non-periodic motion was reconstructed. The resulting diameters were 3.42 mm and 1.85 mm assuming non-periodic and periodic motion, respectively. Further, for two clinical cases (coronary arteries and coronary sinus) it is demonstrated that the presented algorithm outperforms periodic approaches and is able to handle realistic irregular heart motion.

This paper presents a technique to recover dynamic 3D vascular morphology from a single 3D rotational X-ray angiography acquisition. The dynamic morphology corresponding to a canonical cardiac cycle is represented via a 4D

B

-spline based spatiotemporal deformation. Such deformation is estimated by simultaneously matching the forward projections of a sequence of the temporally deformed 3D reference volume to the entire 2D measured projection sequence. A joint use of two acceleration strategies is also proposed: semi-precomputation of forward projections and registration metric computation based on a narrow-band region-of-interest. Digital and physical phantoms of pulsating cerebral aneurysms have been used for evaluation. Accurate estimation has been obtained in recovering sub-voxel pulsation, even from images with substantial intensity inhomogeneity. Results also demonstrate that the acceleration strategies can reduce memory consumption and computational time without degrading the performance.

This article presents a new algebraic method for reconstructing emission tomography images. This approach is mostly an interval extension of the conventional SIRT algorithm. One of the main characteristic of our approach is that the reconstructed activity associated with each pixel of the reconstructed image is an interval whose length can be considered as an estimate of the impact of the random variation of the measured activity on the reconstructed image. This work aims at investigating a new methodological concept for a reliable and robust quantification of reconstructed activities in scintigraphic images.

In this paper we generalize the Log-Euclidean polyaffine registration framework of Arsigny et al. [1] to deal with articulated structures. This framework has very useful properties as it guarantees the invertibility of smooth geometric transformations. In articulated registration a skeleton model is defined for rigid structures such as bones. The final transformation is affine for the bones and elastic for other tissues in the image. We extend the Arsigny el al.’s method to deal with locally-affine registration of pairs of wires. This enables the possibility of using this registration framework to deal with articulated structures. In this context, the design of the weighting functions, which merge the affine transformations defined for each pair of wires, has a great impact not only on the final result of the registration algorithm, but also on the invertibility of the global elastic transformation. Several experiments, using both synthetic images and hand radiographs, are also presented.

In this paper we present a method for nonrigid registration of myocardial perfusion MR images. Instead of registering pairs of images within the observed sequence, we register the observed sequence to a pseudo ground truth, which is a motion/noise-free sequence estimated from the observed one. As the corresponding images of the two sequences have almost identical intensity distributions, our method overcomes the challenges arising from rapidly varying image intensity and contrast. The pseudo ground truth and the deformation fields for the observed sequence are obtained simultaneously by minimizing an energy functional integrating both the registration error and the spatiotemporal constraints on the pseudo ground truth in an expectation-maximization fashion. We have tested the proposed nonrigid registration method on real cardiac MR perfusion scans, both qualitatively and quantitatively. Experimental results show that the proposed method is able to successfully compensate for the heart motion during contrast enhancement.

In this paper, we present a novel method to create parallax-free panoramic X-ray images of long bones during surgery by making the C-arm rotate around its X-ray source, relative to the patient’s table. In order to ensure that the C-arm motion is a relative pure rotation around its X-ray source, we move the table to compensate for the translational part of the motion based on C-arm pose estimation, for which we employed a Camera Augmented Mobile C-arm system [1] and a visual planar marker pattern. Thus, we are able to produce a parallax-free panoramic X-ray image that preserves the property of linear perspective projection. We additionally implement a method to reduce the error caused by varying intrinsic parameters of C-arm X-ray imaging. The results show that our proposed method can generate a parallax-free panoramic X-ray image, independent of the configuration of bone structures and without the requirement of a fronto-parallel setup or any overlap in the X-ray images. The resulting panoramic images have a negligible difference (below 2 pixels) in the overlap between two consecutive individual X-ray images and have a high visual quality, which promises suitability for intra-operative clinical applications in orthopedic and trauma surgery.

We propose a novel method for deformable tensor–to–tensor registration of Diffusion Tensor Imaging (DTI) data. Our registration method considers estimated diffusion tensors as normally distributed random variables whose covariance matrices describe uncertainties in the mean estimated tensor due to factors such as noise in diffusion weighted images (DWIs), tissue diffusion properties, and experimental design. The dissimilarity between distributions of tensors in two different voxels is computed using the Kullback-Leibler divergence to drive a deformable registration process, which is not only affected by principal diffusivities and principal directions, but also the underlying DWI properties. We in general do not assume the positive definite nature of the tensor space given the pervasive influence of noise and other factors. Results indicate that the proposed metric weights voxels more heavily whose diffusion tensors are estimated with greater certainty and exhibit anisotropic diffusion behavior thus, intrinsically favoring coherent white matter regions whose tensors are estimated with high confidence.

In this paper, we present a novel algorithm for non-rigidly registering two high angular resolution diffusion weighted MRIs (HARDI), each represented by a Gaussian mixture field (GMF). We model the non-rigid warp by a thin-plate spline and formulate the registration problem as the minimization of the L2 distance between the two given GMFs. The key mathematical contributions of this work are, (i) a closed form expression for the derivatives of this objective function with respect to the parameters of the registration and (ii) a novel and simpler re-orientation scheme based on an extension to the ”Preservation of Principle Directions” technique. We present results of our algorithm’s performance on several synthetic and real HARDI data sets.

Over the last decade, remarkable progress has been made in the field of endovascular treatment of aneurysms. Technological advances continue to enable a growing number of patients with cerebral aneurysms to be treated with a variety of endovascular strategies, essentially using detachable platinum coils. Yet, coil embolization remains a very complex medical procedure for which careful planning must be combined with advanced technical skills in order to be successful.

In this paper we propose a method for computing the complex blood flow patterns that take place within the aneurysm, and for simulating the interaction of coils with this flow. This interaction is twofold, first involving the impact of the flow on the coil during the initial stages of its deployment, and second concerning the decrease of blood velocity within the aneurysm, as a consequence of coil packing. We also propose an approach to achieve real-time computation of coil-flow bilateral influence, necessary for interactive simulation. This in turns allows to dynamically plan coil embolization for two key steps of the procedure: choice and placement of the first coils, and assessment of the number of coils necessary to reduce aneurysmal blood velocity and wall pressure.

Real-time three-dimensional (RT3D) echocardiography is the newest generation of three-dimensional (3-D) echocardiography. Segmentation of RT3D echocardiographic images is essential for determining many important diagnostic parameters. In cardiac imaging, since the heart is a moving organ, prior knowledge regarding its shape and motion patterns becomes an important component for the segmentation task. However, most previous cardiac models are either static models (SM), which neglect the temporal coherence of a cardiac sequence or generic dynamical models (GDM), which neglect the inter-subject variability of cardiac motion. In this paper, we present a subject-specific dynamical model (SSDM) which simultaneously handles inter-subject variability and cardiac dynamics (intra-subject variability). It can progressively predict the shape and motion patterns of a new sequence at the current frame based on the shapes observed in the past frames. The incorporation of this SSDM into the segmentation process is formulated in a recursive Bayesian framework. This results in a segmentation of each frame based on the intensity information of the current frame, as well as on the prediction from the previous frames. Quantitative results on 15 RT3D echocardiographic sequences show that automatic segmentation with SSDM is superior to that of either SM or GDM, and is comparable to manual segmentation.

Patients with repaired Tetralogy of Fallot commonly suffer from chronic pulmonary valve regurgitations and extremely dilated right ventricle (RV). To reduce risk factors, new pulmonary valves must be re-implanted. However, establishing the best timing for re-intervention is a clinical challenge because of the large variability in RV shape and in pathology evolution. This study aims at quantifying the regional impacts of growth and regurgitations upon the end-diastolic RV anatomy. The ultimate goal is to determine, among clinical variables, predictors for the shape in order to build a statistical model that predicts RV remodelling. The proposed approach relies on a

forward

model based on currents and LDDMM algorithm to estimate an unbiased template of 18 patients and the deformations towards each individual shape. Cross-sectional multivariate analyses are carried out to assess the effects of body surface area, tricuspid and transpulmonary valve regurgitations upon the RV shape. The statistically significant deformation modes were found clinically relevant. Canonical correlation analysis yielded a generative model that was successfully tested on two new patients.

We propose a recursive Bayesian model for the delineation of coronary arteries from 3D CT angiograms (cardiac CTA) and discuss the use of discrete minimal path techniques as an efficient optimization scheme for the propagation of model realizations on a discrete graph. Design issues such as the definition of a suitable accumulative metric are analyzed in the context of our probabilistic formulation.

Our approach jointly optimizes the vascular centerline and associated radius on a 4D space+scale graph. It employs a simple heuristic scheme to dynamically limit scale-space exploration for increased computational performance. It incorporates prior knowledge on radius variations and derives the local data likelihood from a multiscale, oriented gradient flux-based feature. From minimal cost path techniques, it inherits practical properties such as computational efficiency and workflow versatility. We quantitatively evaluated a two-point interactive implementation on a large and varied cardiac CTA database. Additionally, results from the Rotterdam Coronary Artery Algorithm Evaluation Framework are provided for comparison with existing techniques. The scores obtained are excellent (97.5% average overlap with ground truth delineated by experts) and demonstrate the high potential of the method in terms of robustness to anomalies and poor image quality.

Research has shown that tumor vascular markers (TVMs) may serve as potential OCa biomarkers for prognosis prediction. One such TVM is ESM-1, which can be visualized by staining ovarian Tissue Microarrays (TMA) with an antibody to ESM-1. The ability to quickly and quantitatively estimate vascular stained regions may yield an image based metric linked to disease survival and outcome. Automated segmentation of the vascular stained regions on the TMAs, however, is hindered by the presence of spuriously stained false positive regions. In this paper, we present a general, robust and efficient unsupervised segmentation algorithm, termed Hierarchical Normalized Cuts (HNCut), and show its application in precisely quantifying the presence and extent of a TVM on OCa TMAs. The strength of HNCut is in the use of a hierarchically represented data structure that bridges the mean shift (MS) and the normalized cuts (NCut) algorithms. This allows HNCut to efficiently traverse a pyramid of the input image at various color resolutions, efficiently and accurately segmenting the object class of interest (in this case ESM-1 vascular stained regions) by simply annotating half a dozen pixels belonging to the target class. Quantitative and qualitative analysis of our results, using 100 pathologist annotated samples across multiple studies, prove the superiority of our method (sensitivity 81%, Positive predictive value (PPV), 80%) versus a popular supervised learning technique, Probabilistic Boosting Trees (sensitivity, PPV of 76% and 66%).

Segmentation of low contrast objects is an important task in clinical applications like lesion analysis and vascular wall remodeling analysis. Several solutions to low contrast segmentation that exploit high-level information have been previously proposed, such as shape priors and generative models. In this work, we incorporate

a priori

distributions of intensity and low-level image information into a nonparametric dissimilarity measure that defines a local indicator function for the likelihood of belonging to a foreground object. We then integrate the indicator function into a level set formulation for segmenting low contrast structures. We apply the technique to the clinical problem of positive remodeling of the vessel wall in cardiac CT angiography images. We present results on a dataset of twenty five patient scans, showing improvement over conventional gradient-based level sets.

The diagnosis of colorectal cancer is usually supported by a staging system, such as the Duke or TNM system. In this work we discuss computer–aided pit–pattern classification of surface structures observed during high–magnification colonoscopy in order to support dignity assessment of colonic polyps. This is considered a quite promising approach because it allows in vivo staging of colorectal lesions. Since recent research work has shown that the characteristic surface structures of the colon mucosa exhibit texture characteristics, we employ a set of texture image features in the wavelet-domain and propose a novel classifier combination approach which is similar to a combination of experts. The experimental results of our work show superior classification performance compared to previous approaches on both a two-class (non-neoplastic vs. neoplastic) and a more complicated six-class (pit–pattern) classification problem.

Automated segmentation of the esophagus in CT images is of high value to radiologists for oncological examinations of the mediastinum. It can serve as a guideline and prevent confusion with pathological tissue. However, segmentation is a challenging problem due to low contrast and versatile appearance of the esophagus. In this paper, a two step method is proposed which first finds the approximate shape using a “detect and connect” approach. A classifier is trained to find short segments of the esophagus which are approximated by an elliptical model. Recently developed techniques in discriminative learning and pruning of the search space enable a rapid detection of possible esophagus segments. Prior shape knowledge of the complete esophagus is modeled using a Markov chain framework, which allows efficient inferrence of the approximate shape from the detected candidate segments. In a refinement step, the surface of the detected shape is non-rigidly deformed to better fit the organ boundaries. In contrast to previously proposed methods, no user interaction is required. It was evaluated on 117 datasets and achieves a mean segmentation error of 2.28mm with less than 9s computation time.

Automatic segmentation of structures with missing or invisible borders is a challenging task. Since structures in the lungs are related, humans use contextual and shape information to infer the position of invisible borders. An example of a task in which the borders are often incomplete or invisible is the segmentation of the pulmonary lobes. In this paper, a fully automatic segmentation of the pulmonary lobes in chest CT scans is presented. The method is especially designed to be robust to incomplete fissures by incorporating contextual information from automatic lung, fissure, and bronchial tree segmentations, as well as shape information. Since the method relies on the result of automatic segmentations, it is important that the method is robust against failure of one or more of these segmentation methods. In an extensive experiment on 10 chest CT scans with manual segmentations, the robustness of the method to incomplete fissures and missing input segmentations is shown. In a second experiment on 100 chest CT scans with incomplete fissures, the method is shown to perform well.

Spatial priors, such as probabilistic atlases, play an important role in MRI segmentation. However, the availability of comprehensive, reliable and suitable manual segmentations for atlas construction is limited. We therefore propose a joint segmentation of corresponding, aligned structures in the entire population that does not require a probability atlas. Instead, a latent atlas, initialized by a single manual segmentation, is inferred from the evolving segmentations of the ensemble. The proposed method is based on probabilistic principles but is solved using partial differential equations (PDEs) and energy minimization criteria. We evaluate the method by segmenting 50 brain MR volumes. Segmentation accuracy for cortical and subcortical structures approaches the quality of state-of-the-art atlas-based segmentation results, suggesting that the

latent atlas

method is a reasonable alternative when existing atlases are not compatible with the data to be processed.

We propose a novel parametric deformable model controlled by shape and visual appearance priors learned from a training subset of co-aligned medical images of goal objects. The shape prior is derived from a linear combination of vectors of distances between the training boundaries and their common centroid. The appearance prior considers gray levels within each training boundary as a sample of a Markov-Gibbs random field with pairwise interaction. Spatially homogeneous interaction geometry and Gibbs potentials are analytically estimated from the training data. To accurately separate a goal object from an arbitrary background, empirical marginal gray level distributions inside and outside of the boundary are modeled with adaptive linear combinations of discrete Gaussians (LCDG). Due to the analytical shape and appearance priors and a simple Expectation-Maximization procedure for getting the object and background LCDG, our segmentation is considerably faster than with most of the known geometric and parametric models. Experiments with various goal images confirm the robustness, accuracy, and speed of our approach.

Modeling and analysis of MR images of the early developing human brain is a challenge because of the transient nature of different tissue classes during brain growth. To address this issue, a statistical model that can capture the spatial variation of structures over time is needed. Here, we present an approach to building a spatio-temporal model of tissue distribution in the developing brain which can incorporate both developed tissues as well as transient tissue classes such as the germinal matrix by using constrained higher order polynomial models. This spatio-temporal model is created from a set of manual segmentations through groupwise registration and voxelwise non-linear modeling of tissue class membership, that allows us to represent the appearance as well as disappearance of the transient brain structures over time. Applying this model to atlas-based segmentation, we generate age-specific tissue probability maps and use them to initialize an EM segmentation of the fetal brain tissues. The approach is evaluated using clinical MR images of young fetuses with gestational ages ranging from 20.57 to 24.71 weeks. Results indicate improvement in performance of atlas-based EM segmentation provided by higher order temporal models that capture the variation of tissue occurrence over time.

We propose a new methodology to analyze the anatomical variability of a set of longitudinal data (population scanned at several ages). This method accounts not only for the usual 3D anatomical variability (geometry of structures), but also for possible changes in the dynamics of evolution of the structures. It does not require that subjects are scanned the same number of times or at the same ages. First a regression model infers a continuous evolution of shapes from a set of observations of the same subject. Second, spatiotemporal registrations deform jointly (1) the geometry of the evolving structure via 3D deformations and (2) the dynamics of evolution via time change functions. Third, we infer from a population a prototype scenario of evolution and its 4D variability. Our method is used to analyze the morphological evolution of 2D profiles of hominids skulls and to analyze brain growth from amygdala of autistics, developmental delay and control children.

This paper investigates an approach to model the space of brain images through a low-dimensional manifold. A data driven method to learn a manifold from a collections of brain images is proposed. We hypothesize that the space spanned by a set of brain images can be captured, to some approximation, by a low-dimensional manifold, i.e. a parametrization of the set of images. The approach builds on recent advances in manifold learning that allow to uncover nonlinear trends in data. We combine this manifold learning with distance measures between images that capture shape, in order to learn the underlying structure of a database of brain images. The proposed method is generative. New images can be created from the manifold parametrization and existing images can be projected onto the manifold. By measuring projection distance of a held out set of brain images we evaluate the fit of the proposed manifold model to the data and we can compute statistical properties of the data using this manifold structure. We demonstrate this technology on a database of 436 MR brain images.

Human cortical folding pattern has been studied for decades. This paper proposes a gyrus scale folding pattern analysis technique via cortical surface profiling. Firstly, we sample the cortical surface into 2D profiles and model them using power function. This step provides both the flexibility of representing arbitrary shape by profiling and the compactness of representing shape by parametric modeling. Secondly, based on the estimated model parameters, we extract affine-invariant features on the cortical surface and apply the affinity propagation clustering algorithm to parcellate the cortex into regions with different shape patterns. Finally, a second-round surface profiling is performed on the parcellated cortical regions, and the number of hinges is detected to describe the gyral folding pattern. Experiments demonstrate that our method could successfully classify human gyri into 2-hinge, 3-hinge and 4-hinge gyri. The proposed method has the potential to significantly contribute to automatic segmentation and recognition of cortical gyri.

It has been shown that brain structures in normal aging undergo significant changes attributed to neurodevelopmental and neurodegeneration processes as a lifelong, dynamic process. Modeling changes in healthy aging will be necessary to explain differences to neurodegenerative patterns observed in mental illness and neurological disease. Driving application is the analysis of brain white matter properties as a function of age, given a database of diffusion tensor images (DTI) of 86 subjects well-balanced across adulthood. We present a methodology based on constrained PCA (CPCA) for fitting age-related changes of white matter diffusion of fiber tracts. It is shown that CPCA applied to tract functions of diffusion isolates population noise and retains age as a smooth change over time, well represented by the first principal mode. CPCA is therefore applied to a functional data analysis (FDA) problem. Age regression on tract functions reveals a nonlinear trajectory but also age-related changes varying locally along tracts. Four tracts with four different tensor-derived scalar diffusion measures were analyzed, and leave-one-out validation of data compression is shown.

This study aims to identify the minimum requirements for an accurate model of the diffusion MR signal in white matter of the brain. We construct a hierarchy of two-compartment models of white matter from combinations of simple models for the intra and extra-cellular spaces. We devise a new diffusion MRI protocol that provides measurements with a wide range of parameters for diffusion sensitization both parallel and perpendicular to white matter fibres. We use the protocol to acquire data from a fixed rat brain, which allows us to fit, study and compare the different models. The results show that models which incorporate pore size describe the measurements most accurately. The best fit comes from combining a full diffusion tensor (DT) model of the extra-cellular space with a cylindrical intra-cellular component.

Here we introduce multivariate tensor-based surface morphometry using holomorphic one-forms to study brain anatomy. We computed new statistics from the Riemannian metric tensors that retain the full information in the deformation tensor fields. We introduce two different holomorphic one-forms that induce different surface conformal parameterizations. We applied this framework to 3D MRI data to analyze hippocampal surface morphometry in Alzheimer’s Disease (AD; 26 subjects), lateral ventricular surface morphometry in HIV/AIDS (19 subjects) and cortical surface morphometry in Williams Syndrome (WS; 80 subjects). Experimental results demonstrated that our method powerfully detected brain surface abnormalities. Multivariate statistics on the local tensors outperformed other TBM methods including analysis of the Jacobian determinant, the largest eigenvalue, or the pair of eigenvalues, of the surface Jacobian matrix.

We introduce a framework for computing geometrical properties of white matter fibres directly from diffusion tensor fields. The key idea is to isolate the portion of the

gradient

of the tensor field corresponding to local variation in tensor orientation, and to project it onto a coordinate frame of tensor eigenvectors. The resulting eigenframe-centered representation makes it possible to define scalar geometrical measures that describe the underlying white matter fibres, directly from the diffusion tensor field and its gradient, without requiring prior tractography. We define two new scalar measures of (1) fibre dispersion and (2) fibre curving, and we demonstrate them on synthetic and

in-vivo

datasets. Finally, we illustrate their applicability in a group study on schizophrenia.

With increasing demand on intra-operative navigation and motion compensation during robotic assisted minimally invasive surgery, real-time 3D deformation recovery remains a central problem. Currently the majority of existing methods rely on salient features, where the inherent paucity of distinctive landmarks implies either a semi-dense reconstruction or the use of strong geometrical constraints. In this study, we propose a gaze-contingent depth reconstruction scheme by integrating human perception with semi-dense stereo and

p-q

based shading information. Depth inference is carried out in real-time through a novel application of Bayesian chains without smoothness priors. The practical value of the scheme is highlighted by detailed validation using a beating heart phantom model with known geometry to verify the performance of gaze-contingent 3D surface reconstruction and deformation recovery.

In image-guided interventions, anatomical models of organs are often generated from pre-operative images and further employed in planning and guiding therapeutic procedures. However, the accuracy of these models, along with their registration to the subject are crucial for successful therapy delivery. These factors are amplified when manipulating soft tissue undergoing large deformations, such as the heart. When used in guiding beating-heart procedures, pre-operative models may not be sufficient for guidance and they are often complemented with real-time, intra-operative cardiac imaging. Here we demonstrate via

in vitro

endocardial “therapy” that ultrasound-enhanced model-guided navigation provides sufficient guidance to preserve a clinically-desired targeting accuracy of under 3 mm independently of the model-to-subject misregistrations. These results emphasize the direct benefit of integrating real-time imaging within intra-operative visualization environments considering that model-to-subject misalignments are often encountered clinically.

We present an approach to derive patient specific coronary models from ECG-gated CTA data and their application for the alignment of CTA with mono-plane X-ray imaging during interventional cardiology. A 4D (3D+t) deformation model of the coronary arteries is derived by (i) extraction of a 3D coronary model at an appropriate cardiac phase and (ii) non-rigid registration of the CTA images at different ECG phases to obtain a deformation model. The resulting 4D coronary model is aligned with the X-ray data using a novel 2D+t/3D+t registration approach. Model consistency and accuracy is evaluated using manually annotated coronary centerlines at systole and diastole as reference. Improvement of registration robustness by using the 2D+t/3D+t registration is successfully demonstrated by comparison of the actual X-ray cardiac phase with the automatically determined best matching phase in the 4D coronary model.

Many vascular pathologies can now be treated in a minimally invasive way thanks to interventional radiology. Instead of open surgery, it allows to reach the lesion of the arteries with therapeutic devices through a catheter. As a particular case, intracranial aneurysms are treated by filling the localized widening of the artery with a set of coils to prevent a rupture due to the weakened arterial wall. Considering the location of the lesion, close to the brain, and its very small size, the procedure requires a combination of careful planning and excellent technical skills. An interactive and reliable simulation, adapted to the patient anatomy, would be an interesting tool for helping the interventional neuroradiologist plan and rehearse a coil embolization procedure. This paper describes an original method to perform interactive simulations of coil embolization and proposes a clinical metric to quantitatively measure how the first coil fills the aneurysm. The simulation relies on an accurate reconstruction of the aneurysm anatomy and a real-time model of the coil for which sliding and friction contacts are taken into account. Simulation results are compared to real embolization procedure and exhibit good adequacy.

We present a method for realtime online 3d reconstruction of a guide-wire or catheter using 2d X-ray images, which do not have to be recorded from different viewpoints. No special catheters or sensors are needed. Given a 3d patient data set and the projection parameters, we use recursive probability density propagation to estimate a probability distribution of the current positions of guide-wire parts. Based on this distribution, we extract the optimal guide-wire position using regularization techniques. We describe the guide-wire position by a uniform cubic B-spline. Experiments on simulated and phantom data demonstrate the high accuracy and robustness of our approach.

Radio-frequency catheter ablation of the pulmonary veins attached to the left atrium is usually carried out under fluoroscopy guidance. Two-dimensional X-ray navigation may involve overlay images derived from a static pre-operative 3-D volumetric data set to add anatomical details. However, respiratory motion may impair the utility of static overlay images for catheter navigation. We developed a system for image-based 3-D motion estimation and compensation as a solution to this problem for which no previous solution is yet known. It is based on 3-D catheter tracking involving 2-D/3-D registration. A biplane X-ray C-arm system is used to image a special circumferential (lasso) catheter from two directions. In the first step of the method, a 3-D model of the device is reconstructed. 3-D respiratory motion at the site of ablation is then estimated by tracking the reconstructed model in 3-D from biplane fluoroscopy. In our experiments, the circumferential catheter was tracked in 231 biplane fluoro frames (462 monoplane fluoro frames) with an average 2-D tracking error of 1.0 mm ± 0.5 mm.

This contribution reports the development and initial testing of a Mobile Robot System for Surgical Craniotomy, the Craniostar. A kinematic system based on a unicycle robot is analysed to provide local positioning through two spiked wheels gripping directly onto a patients skull. A control system based on a shared control system between both the Surgeon and Robot is employed in a hand-held design that is tested initially on plastic phantom and swine skulls. Results indicate that the system has substantially lower risk than present robotically assisted craniotomies, and despite being a hand-held mobile robot, the Craniostar is still capable of sub-millimetre accuracy in tracking along a trajectory and thus achieving an accurate transfer of pre-surgical plan to the operating room procedure, without the large impact of current medical robots based on modified industrial robots.

In robot-assisted procedures, the surgeon’s ability can be enhanced by navigation guidance through the use of virtual fixtures or active constraints. This paper presents a real-time modeling scheme for dynamic active constraints with fast and simple mesh adaptation under cardiac deformation and changes in anatomic structure. A smooth tubular pathway is constructed which provides assistance for a flexible hyper-redundant robot to circumnavigate the heart with the aim of undertaking bilateral pulmonary vein isolation as part of a modified maze procedure for the treatment of debilitating arrhythmia and atrial fibrillation. In contrast to existing approaches, the method incorporates detailed geometrical constraints with explicit manipulation margins of the forbidden region for an

entire

articulated surgical instrument, rather than just the end-effector itself. Detailed experimental validation is conducted to demonstrate the speed and accuracy of the instrument navigation with and without the use of the proposed dynamic constraints.

We developed a nonmagnetic rigid and flexible outer sheath with pneumatic interlocking mechanism using flexible toothed links and a wire-driven bending distal end. The outer sheath can be switched between rigid and flexible modes easily depending on surgical scenes, and the angle of its distal end can be controlled by three nylon wires. All components of flexible parts are made of MRI-compatible nonmagnetic plastics. We manufactured the device with 300-mm long, 16-mm outer diameter, 7-mm inner diameter and 90-mm bending distal end. Holding power of the device in rigid mode was maximum 3.6 N, which was sufficient for surgical tasks in body cavity. In vivo experiment using a swine, our device performed smooth insertion of a flexible endoscope and a biopsy forceps into reverse side of the liver, intestines and spleen with a curved path. In conclusion, our device shows availability of secure approach of surgical instruments into deep cavity.

This paper addresses automatic skill assessment in robotic minimally invasive surgery. Hidden Markov models (HMMs) are developed for individual surgical gestures (or

surgemes

) that comprise a typical bench-top surgical training task. It is known that such HMMs can be used to recognize and segment surgemes in previously unseen trials [1]. Here, the topology of each surgeme HMM is designed in a data-driven manner, mixing trials from multiple surgeons with varying skill levels, resulting in HMM states that model skill-specific

sub-gestures

. The sequence of HMM states visited while performing a surgeme are therefore indicative of the surgeon’s skill level. This expectation is confirmed by the average edit distance between the state-level “transcripts” of the same surgeme performed by two surgeons with different expertise levels. Some surgemes are further shown to be more indicative of skill than others.

Evaluating surgical skill is a time consuming, subjective, and difficult process. This paper compares two methods of identifying the skill level of a subject given motion data from a benchtop surgical task. In the first method, we build discrete Hidden Markov Models at the

task level

, and test against these models. In the second method, we build discrete Hidden Markov Models of surgical gestures, called

surgemes

, and evaluate skill at this level. We apply these techniques to 57 data sets collected from the da Vinci surgical system. Our current techniques have achieved accuracy levels of 100% using task level models and known gesture segmentation, 95% with task level models and unknown gesture segmentation, and 100% with the surgeme level models in correctly identifying the skill level. We observe that, although less accurate, the second method requires less prior label information. Also, the surgeme level classification provided more insights into what subjects did well, and what they did poorly.

In recent years there has been an ever increasing amount of research and development of technologies and methodologies aimed at improving the safety of advanced surgery. In this context, several training methods and metrics have been proposed, in particular for laparoscopy, both to improve the surgeon’s abilities and also to assess her/his skills. For neurosurgery, however, the extremely small movements and sizes involved have prevented until now the development of similar methodologies and systems.

In this paper we present the development of the ultra-miniaturized Inertial Measurement Unit WB3 (at present the smallest, lightest, and best performing in the world) for practical application in neurosurgery as skill assessment tool. This paper presents the feasibility study for quantitative discrimination of movements of experienced surgeons and beginners in a simple pick and place scenario.

The purpose of this study is to develop a new endoscope for performing simple surgical tasks inside a cardiac atrium/chamber filled with blood, i.e., for performing “off-pump” cardiac surgeries. In general, it is very difficult to observe the inner wall of the vessels containing circulating blood because the light from the endoscope is scattered by the red blood cells. “Plasma flushing” performed using the separator system is developed to observe the inner side of the heart filled with blood and to remove blood cells from the front of the endoscope tip. The system was used in in vitro quantitative measurement of the device performance and in vivo experiments on a swine. In these experiments, we successfully obtained high-resolution images of the interior of the heart during off-pump surgery.

An open problem in endoscopic surgery (especially with flexible endoscopes) is the absence of a stable horizon in endoscopic images. With our ”Endorientation” approach image rotation correction, even in non-rigid endoscopic surgery (particularly NOTES), can be realized with a tiny MEMS tri-axial inertial sensor placed on the tip of an endoscope. It measures the impact of gravity on each of the three orthogonal accelerometer axes. After an initial calibration and filtering of these three values the rotation angle is estimated directly. Achievable repetition rate is above the usual endoscopic video frame rate of 30Hz; accuracy is about one degree. The image rotation is performed in real-time by digitally rotating the analog endoscopic video signal. Improvements and benefits have been evaluated in animal studies: Coordination of different instruments and estimation of tissue behavior regarding gravity related deformation and movement was rated to be much more intuitive with a stable horizon on endoscopic images.

This paper describes the first accomplishment of the Time-of-Flight (ToF) measurement principle via endoscope optics. The applicability of the approach is verified by in-vitro experiments. Off-the-shelf ToF camera sensors enable the per-pixel, on-chip, real-time, marker-less acquisition of distance information. The transfer of the emerging ToF measurement technique to endoscope optics is the basis for a new generation of ToF rigid or flexible 3-D endoscopes. No modification of the endoscope optic itself is necessary as only an enhancement of illumination unit and image sensors is necessary. The major contribution of this paper is threefold: First, the accomplishment of the ToF measurement principle via endoscope optics; second, the development and validation of a complete calibration and post-processing routine; third, accomplishment of extensive in-vitro experiments. Currently, a depth measurement precision of 0.89 mm at 20 fps with 3072 3-D points is achieved.

The aim of this study is to automatically assess reendothelialization of stents at an accuracy of down to a few microns by analyzing endovascular optical coherence tomography (OCT) sequences. Vessel wall and struts are automatically detected and complete distance map is then computed from sparse distances measured between wall and struts by thin-plate spline (TPS) interpolation. A reendothelialization score is mapped onto the geometry of the coronary artery segment. Accuracy and robustness are increased by taking into account the inhomogeneity of datapoints and integrating in the same framework orthogonalized forward selection of support points, optimal selection of regularization parameters by generalized cross-validation (GCV) and rejection of detection outliers. The comparison against manual expert measurements for a phantom study and 12 in vivo stents demonstrates no significant discordance with variability of the order of the strut thickness.

The quest for providing tissue characterization and functional mapping during minimally invasive surgery (MIS) has motivated the development of new surgical tools that extend the current functional capabilities of MIS. Miniaturized optical probes can be inserted into the instrument channel of standard endoscopes to reveal tissue cellular and subcellular microstructures, allowing excision-free optical biopsy. One of the limitations of such a point based imaging and tissue characterization technique is the difficulty of tracking probed sites in vivo. This prohibits large area surveillance and integrated functional mapping. The purpose of this paper is to present an image-based tracking framework by combining a semi model-based instrument tracking method with vision-based simultaneous localization and mapping. This allows the mapping of all spatio-temporally tracked biopsy sites, which can then be re-projected back onto the endoscopic video to provide a live augmented view in vivo, thus facilitating re-targeting and serial examination of potential lesions. The proposed method has been validated on phantom data with known ground truth and the accuracy derived demonstrates the strength and clinical value of the technique. The method facilitates a move from the current point based optical biopsy towards large area multi-scale image integration in a routine clinical environment.

Tracking biopsy sites in endoscopic images can be useful to provide a visual aid for the guidance of surgical tools, for example when endoscopic guided biopsy is required. A new method for re-localisation of these sites is presented in this paper. It makes use of epipolar geometry properties between three images of the same site observed from different viewpoints with an endoscope. Two epipolar lines are derived from the two first images in the third image where the site needs to be re-localised. Their intersection corresponds to the location of the biopsy site. This method was tested with gastroscopic data from 2 patients with 9 series of three images of the oesophagus. The re-localisation error was estimated at less than 1.5 millimetres by a clinical endoscopist, which is sufficient for most clinical endoscopic applications.

Recent advances in biophotonics have enabled

in-vivo

, in-situ histopathology for routine clinical applications. The non-invasive nature of these optical ‘biopsy’ techniques, however, entails the difficulty of identifying previously visited biopsy locations, particularly for surveillance examinations. This paper presents a novel region-matching approach for narrow-band endoscopy to facilitate retargeting the optical biopsy sites. The task of matching sparse affine covariant image regions is modelled in a Markov Random Field (MRF) framework. The proposed model incorporates appearance based region similarities as well as spatial correlations of neighbouring regions. In particular, a geometric constraint that is robust to deviations in relative positioning of the detected regions is introduced. In the proposed model, the appearance and geometric constraints are evaluated in the same space (photometry), allowing for their seamless integration into the MRF objective function. The performance of the method as compared to the existing state-of-the-art is evaluated with both

in-vivo

and simulation datasets with varying levels of visual complexities.

Tracked ultrasound elastography can be used for guidance in partial breast radiotherapy by visualizing the hard scar tissue around the lumpectomy cavity. For clinical success, the elastography method needs to be robust to the sources of decorrelation between ultrasound images, specifically fluid motions inside the cavity, change of the appearance of speckles caused by compression or physiologic motions, and out-of-plane motion of the probe. In this paper, we present a novel elastography technique that is based on analytic minimization of a regularized cost function. The cost function incorporates similarity of RF data intensity and displacement continuity, making the method robust to small decorrelations present throughout the image. We also exploit techniques from robust statistics to make the method resistant to large decorrelations caused by sources such as fluid motion. The analytic displacement estimation works in real-time. Moreover, the tracked data, used for targeting the radiotherapy, is exploited for discarding frames with excessive out-of-plane motion. Simulation, phantom and patient results are presented.

Anesthetic nerve blocks are a common therapy performed in hospitals around the world to alleviate acute and chronic pain. Tracking systems have shown considerable promise in other forms of therapy, but little has been done to apply this technology in the field of anesthesia. We are developing a guidance system for combining tracked needles with non-invasive ultrasound (US) and patient-specific geometric models. In experiments with phantoms two augmented reality (AR) guidance systems were compared to the exclusive use of US for lumbar facet injection therapy. Anesthetists and anesthesia residents were able to place needles within 0.57

mm

of the intended targets using our AR systems compared to 5.77

mm

using US alone. A preliminary cadaver study demonstrated the system was able to accurately place radio opaque dye on targets. The combination of real time US with tracked tools and AR guidance has the potential to replace CT and fluoroscopic guidance, thus reducing radiation dose to patients and clinicians, as well as reducing health care costs.

For patients with neck problems valuable functional and diagnostic information can be obtained from a fluoroscopy video of a flexion-extension movement of the cervical spine. In most cases physicians have to manually extract the vertebrae, making the analysis of these video sequences tedious and time consuming. In this paper we propose an automatic fast and precise method for tracking cervical vertebrae. Our method relies only on a rough selection of template areas of each vertebra in a single frame of the video sequence. Compared to existing automated methods, no contours need to be extracted and no vertebra segmentation is required. Tracking is done with a normalized gradient field, using only the gradient orientations as features. Experimental results show that the algorithm is robust and able to track the vertebrae accurately even if they are partially occluded or if a disc prosthesis is present.

Intraoperative robotic and computer-guided assistances are now commonly used in total hip arthroplasty (THA) for accurate execution of the preoperative plan. Although the preoperative plan to be accurately executed is critical, it is still interactively prepared in a time-consuming and subjective manner. In this paper, atlas-based approach to automated surgical planning of the acetabular cup in THA is described to stabilize its quality as well as reduce its time-consuming nature. Surgeon’s expertise is embedded in two types of statistical atlases, which are constructed from training datasets of CT-based 3D plans prepared by experienced surgeons. One is a statistical shape model which encodes global spatial relationships between the patient anatomy and implant. The other is the statistical map of residual bone thickness on the implant surface, which encodes local spatial constraints of the anatomy and implant. Given the 3D pelvis shape of the patient, we formulate a procedure to determine the best size and position of the acetabular cup which satisfy the constraints derived from the two statistical atlases. We validated the proposed planning method by retrospective study using the datasets which were actually used in the THA surgery.

Vitreoretinal surgeries require accuracy and dexterity that is often beyond the capabilities of human surgeons. Untethered robotic devices that can achieve the desired precision have been proposed, and localization information is required for their control. Since the interior of the human eye is externally observable, vision can be used for their localization. In this paper we examine the effects of the human eye optics on imaging and localizing intraocular devices. We propose a method for wide-angle intraocular imaging and localization. We demonstrate accurate localization with experiments in a model eye.

The automation and speedup of interventional therapy and diagnostic workflows is a crucial issue. One way to improve these workflows is to accelerate the image acquisition procedures by fully automating the patient setup. This paper describes a system that performs this task without the use of markers or other prior assumptions. It returns metric coordinates of the 3-D body shape in real-time for inverse positioning. This is achieved by the application of an emerging technology, called Time-of-Flight (ToF) sensor. A ToF sensor is a cost-efficient, off-the-shelf camera which provides more than 40,000 3-D points in real-time. The first contribution of this paper is the incorporation of this novel imaging technology (ToF) in interventional imaging. The second contribution is the ability of a C-arm system to position itself with respect to the patient prior to the acquisition. We are using the 3-D surface information of the patient to partition the body into anatomical sections. This is achieved by a fast two-stage classification process. The system computes the ISO-center for each detected region. To verify our system we performed several tests on the ISO-center of the head. Firstly, the reproducibility of the head ISO-center computation was evaluated. We achieved an accuracy of (x: 1.73±1.11 mm/y: 1.87±1.31 mm/z: 2.91±2.62 mm). Secondly, a C-arm head scan of a body phantom was setup. Our system automatically aligned the ISO-center of the head with the C-arm ISO-center. Here we achieved an accuracy of ± 1 cm, which is within the accuracy of the patient table control.

To help surgeons to pre-operatively select the target location for DBS electrodes, functional atlases based on intra-operatively acquired data have been created in the past. Recently, many groups have reported on the occurrence of brain shift in stereotactic surgery and its impact on the procedure but not on the creation of such atlases. Due to brain shift, the pre- and intra-operative coordinates of anatomic structures are different. When building large population atlases, which rely on pre-operative images for normalization purposes, it is thus necessary to correct for this difference. In this paper, we propose a method to achieve this. We show evidence that electrophysiological maps built using corrected and uncorrected data are different and that the maps created using shift-corrected data correlate better than those created using uncorrected data with the final position of the implant. These findings suggest that brain-shift correction of intra-operatively recorded data is feasible for the construction of accurate shift-corrected electrophysiological atlases.

A natural requirement in pairwise image registration is that the resulting deformation is independent of the order of the images. This constraint is typically achieved via a symmetric cost function and has been shown to reduce the effects of local optima. Consequently, symmetric registration has been successfully applied to pairwise image registration as well as the spatial alignment of individual images with a template. However, recent work has shown that the relationship between an image and a template is fundamentally asymmetric. In this paper, we develop a method that reconciles the practical advantages of symmetric registration with the asymmetric nature of image-template registration by adding a simple correction factor to the symmetric cost function. We instantiate our model within a log-domain diffeomorphic registration framework. Our experiments show exploiting the asymmetry in image-template registration improves alignment in the image coordinates.

Thirion’s Demons [1] is a popular algorithm for nonrigid image registration because of its linear computational complexity and ease of implementation. It approximately solves the diffusion registration problem [2] by successively estimating force vectors that drive the deformation toward alignment and smoothing the force vectors by Gaussian convolution. In this article, we show how the Demons algorithm can be generalized to allow image-driven locally adaptive regularization [3,4] in a manner that preserves both the linear complexity and ease of implementation of the original Demons algorithm. We show that the proposed algorithm exhibits lower target registration error and requires less computational effort than the original Demons algorithm on the registration of serial chest CT scans of patients with lung nodules.

A variety of pixel and feature based methods have been proposed for registering multiple views of anatomy visible in studies obtained using diagnostic, minimally invasive imaging. A given registration method may outperform another depending on anatomical variations, imaging conditions, and imaging sensor performance, and it is often difficult a priori to determine the best registration method for a particular application. To address this problem, we propose a registration framework that pools the results of multiple registration methods using a decision function for validating registrations. We refer to this as

meta registration.

We demonstrate that our framework outperforms several individual registration methods on the task of registering multiple views of Crohn’s disease lesions sampled from a Capsule Endoscopy (CE) study database. We also report on preliminary work on assessing the quality of registrations obtained, and the possibility of using such assessment in the registration framework.

The registration of various data is a challenging task in medical image processing and a highly frequented area of research. Most of the published approaches tend to fail sporadically on different data sets. This happens due to two major problems. First, local optimization strategies induce a high risk when optimizing nonconvex functions. Second, similarity measures might fail if they are not suitable for the data. Thus, researchers began to combine multiple measures by weighted sums. In this paper, we show severe limitations of such summation approaches. We address both issues by a gradient-based vector optimization algorithm that uses multiple similarity measures. It gathers context information from the iteration process to detect and suppress failing measures. The new approach is evaluated by experiments from the field of 2D-3D registration. Besides its generic character with respect to arbitrary data, the main benefit is a highly robust iteration behavior, where even very poor initial guesses of the transform result in good solutions.

In this paper, we propose a framework for learning the parameters of registration cost functions – such as the tradeoff between the regularization and image similiarity term – with respect to a specific task. Assuming the existence of labeled training data, we specialize the framework for the task of localizing hidden labels via image registration. We learn the parameters of the weighted sum of squared differences (wSSD) image similarity term that are optimal for the localization of Brodmann areas (BAs) in a new subject based on cortical geometry. We demonstrate state-of-the-art localization of V1, V2, BA44 and BA45.

We introduce a new hybrid approach for spline-based elastic registration of multimodal medical images. The approach uses point landmarks as well as intensity information based on local analytic measures for joint entropy and mutual information. The information-theoretic similarity measures are computationally efficient and can be optimized independently for each voxel. We have applied our approach to synthetic images, brain phantom images, as well as clinically relevant multimodal medical images. We also compared our measures with previous measures.

In this paper we present a novel method for performing image registration of different modalities. Mutual Information (MI) is an established method for performing such registration. However, it is recognised that standard MI is not without some problems, in particular it does not utilise spatial information within the images. Various modifications have been proposed to resolve this, however these only offer slight improvement to the accuracy of registration. We present Feature Neighbourhood Mutual Information (FNMI) that combines both image structure and spatial neighbourhood information which is efficiently incorporated into Mutual Information by approximating the joint distribution with a covariance matrix (c.f. Russakoff’s Regional Mutual Information). Results show that our approach offers a very high level of accuracy that improves greatly on previous methods. In comparison to Regional MI, our method also improves runtime for more demanding registration problems where a higher neighbourhood radius is required. We demonstrate our method using retinal fundus photographs and scanning laser ophthalmoscopy images, two modalities that have received little attention in registration literature. Registration of these images would improve accuracy when performing demarcation of the optic nerve head for detecting such diseases as glaucoma.

This paper proposes a novel multimodal image registration method which can fully utilize the multimodal information and result in a more accurate unified deformation field. Different from the existing methods which fuse the information at the image/intensity level, the proposed method fuses the multimodal information at the feature level through Gabor wavelets transformation. At this level, complementary and redundant information is distinguished reliably and efficiently, and then combined and removed respectively. Experiments on both simulated and real T1+DTI image sets illustrate that the proposed method can effectively incorporate better characterization for white matter (WM) from the DTI and for gray matter (GM) from the T1 image and lead to a more accurate and efficient multimodal image registration which paves the way for the subsequent multimodal population-based studies.

We introduce an efficient search strategy to substantially accelerate feature based registration. Previous feature based registration algorithms often use truncated search strategies in order to achieve small computation times. Our new accelerated search strategy is based on the realization that the search for corresponding features can be dramatically accelerated by utilizing Johnson-Lindenstrauss dimension reduction. Order of magnitude calculations for the search strategy we propose here indicate that the algorithm proposed is more than a million times faster than previously utilized naive search strategies, and this advantage in speed is directly translated into an advantage in accuracy as the fast speed enables more comparisons to be made in the same amount of time. We describe the accelerated scheme together with a full complexity analysis. The registration algorithm was applied to large transmission electron microscopy (TEM) images of neural ultrastructure. Our experiments demonstrate that our algorithm enables alignment of TEM images with increased accuracy and efficiency compared to previous algorithms.

Registration of Diffusion-Weighted MR Images (DW-MRI) can be achieved by registering the corresponding 2nd-order Diffusion Tensor Images (DTI). However, it has been shown that higher-order diffusion tensors (e.g. order-4) outperform the traditional DTI in approximating complex fiber structures such as fiber crossings. In this paper we present a novel method for unbiased group-wise non-rigid registration and atlas construction of 4th-order diffusion tensor fields. To the best of our knowledge there is no other existing method to achieve this task. First we define a metric on the space of positive-valued functions based on the Riemannian metric of real positive numbers (denoted by ℝ

 + 

). Then, we use this metric in a novel functional minimization method for non-rigid 4th-order tensor field registration. We define a cost function that accounts for the 4th-order tensor re-orientation during the registration process and has analytic derivatives with respect to the transformation parameters. Finally, the tensor field atlas is computed as the minimizer of the variance defined using the Riemannian metric. We quantitatively compare the proposed method with other techniques that register scalar-valued or diffusion tensor (rank-2) representations of the DWMRI.

In this paper, we propose a generalized group-wise non-rigid registration strategy for multiple unlabeled point-sets of unequal cardinality, with no bias toward any of the given point-sets. To quantify the divergence between the probability distributions – specifically Mixture of Gaussians – estimated from the given point sets, we use a recently developed information-theoretic measure called Jensen-Renyi (JR) divergence. We evaluate a closed-form JR divergence between multiple probabilistic representations for the general case where the mixture models differ in variance and the number of components. We derive the analytic gradient of the divergence measure with respect to the non-rigid registration parameters, and apply it to numerical optimization of the group-wise registration, leading to a computationally efficient and accurate algorithm. We validate our approach on synthetic data, and evaluate it on 3D cardiac shapes.

Groupwise registration has been recently introduced for simultaneous registration of a group of images with the goal of constructing an unbiased atlas. To this end, direct application of information-theoretic entropy measures on image intensity has achieved various successes. However, simplistic voxelwise utilization of image intensity often neglects important contextual information, which can be provided by more comprehensive geometric and statistical features. In this paper, we employ attribute vectors, instead of image intensities, to guide groupwise registration. In particular, for each voxel, the attribute vector is computed from its multiple-scale neighborhoods to capture geometric information at different scales. Moreover, the probability density function (PDF) of each attribute in the vector is then estimated from the local neighborhood, providing a statistical summary of the underlying anatomical structure. For the purpose of registration, Jensen-Shannon (JS) divergence is used to measure the PDF dissimilarity of each attribute at corresponding locations of different individual images. By minimizing the overall JS divergence in the whole image space and estimating the deformation field of each image simultaneously, we can eventually register all images and build an unbiased atlas. Experimental results indicate that our method yields better registration quality, compared with a popular groupwise registration method.

The construction and application of statistical models of deformations based on non-rigid image registration methods have gained recent popularity. This paper presents the application of such a model to restricting a general-purpose registration algorithm to anatomically plausible solutions. Specifically, the Morphon registration method is used for atlas-based segmentation of bone scintigraphy images. From a training set of 734 images, a model of characteristic deformation fields is built and used for regularizing the registration of 113 test images. Results show that around 300 training images and 30 principal modes are sufficient for building a useful model. The segmentation succeeded in 106 of 113 test images.

In this paper we propose a novel framework to unite a population to an optimal (unknown) pose through their mutual deformation. The registration criterion comprises three terms, the first imposes compactness on appearance of the registered population at the pixel level, the second tries to minimize the individual distances between all possible pairs of images, while the last is a regularization one imposing smoothness on the deformation fields. The problem is reformulated as a graphical model that consists of hidden (deformation fields) and observed variables (intensities). A novel deformation grid-based scheme is proposed that guarantees the diffeomorphism of the deformation and is computationally favorably compared to standard deformation methods. Towards addressing important deformations we propose a compositional approach where the deformations are recovered through the sub-optimal solutions of successive discrete MRFs by using efficient linear programming. Promising experimental results using real 2D data demonstrate the potentials of our approach.

Geodesic registration methods have been used to solve the large deformation registration problems, which are hard to solve with conventional registration methods. However, analytically defined geodesics may not coincide with anatomically optimal paths of registration. In this paper we propose a novel and efficient method for large deformation registration by learning the underlying structure of the data using a manifold learning technique. In this method a large deformation between two images is decomposed into a series of small deformations along the shortest path on the graph that approximates the metric structure of data. Furthermore, the graph representation allows us to estimate the optimal group template by minimizing geodesic distances. We demonstrate the advantages of the proposed method with synthetic 2D images and real 3D mice brain volumes.

We present a new method for the non-rigid registration of serial mouse microCT images which undergo potentially large changes in the positions of the legs due to articulation. While non-rigid registration methods have been extensively used in the evaluation of individual organs, application in whole body imaging has been limited, primarily because the scale of possible displacements and deformations is large resulting in poor convergence of most methods. Our method is based on the extended demons algorithm that uses a level-set representation of the mouse skin and skeleton as an input, and composed of three steps reflecting the natural physical movements of bony structures. We applied our method to the registration of serial microCT mouse images demonstrating encouraging performances as compared to competitive techniques.

In this paper, we propose a new feature based non-rigid image registration method for dealing with two important issues. First, in order to establish reliable anatomical correspondence between template and subject images, efficient and distinctive region descriptor is needed as intensity information alone maybe insufficient. Second, since interference factors such as monotonic gray-level bias fields are commonly existed during the imaging process, the registration algorithm should be robust against such factors. There are two main contributions presented in this paper. (1) A new region descriptor, named uniform gradient spherical pattern (UGSP), is proposed to extract the geometric features from input images. UGSP encodes second order voxel interaction information. (2) The UGSP feature is rotation and monotonic gray-level bias field invariant. The proposed method is integrated with the Markov random field (MRF) labeling framework to formulate the registration process. The

α

-expansion algorithm is used to optimize the corresponding MRF energy function. The proposed method is evaluated on both the simulated and real 3D databases obtained from BrainWeb and IBSR respectively and compared with other state-of-the-art registration methods. Experimental results show that the proposed method gives the highest registration accuracy among all the compared methods on both databases.

Registration of brain structures should bring anatomically equivalent areas into correspondence which is usually done using information from structural MRI modalities. Correspondence can be improved by using other image modalities that provide complementary data. In this paper we propose and evaluate two novel surface registration algorithms which improve within-surface correspondence in brain structures. Both approaches use a white-matter tract similarity function (derived from probabilistic tractography) to match areas of similar connectivity patterns. The two methods differ in the way the deformation field is calculated and in how the multi-scale registration framework is implemented. We validated both algorithms using artificial and real image examples, in both cases showing high registration consistency and the ability to find differences in thalamic sub-structures between Alzheimer’s disease and control subjects. The results suggest differences in thalamic connectivity predominantly in the medial dorsal parts of the left thalamus.

Constructing a one to one correspondence between whole brain MR image scans is a problem of critical importance in neuroimaging analyses. We present a framework to combine the strength of both surface-based and volumetric-based analyses for consistent, bijective data transfer between brain coordinate systems.

We propose a novel algorithm, called

Fast Tensor Image Morphing for Elastic Registration

or F-TIMER. F-TIMER leverages multiscale tensor regional distributions and local boundaries for hierarchically driving deformable matching of tensor image volumes. Registration is achieved by aligning a set of automatically determined structural landmarks, via solving a soft correspondence problem. Based on the estimated correspondences, thin-plate splines are employed to generate a smooth, topology preserving, and dense transformation, and to avoid arbitrary mapping of non-landmark voxels. To mitigate the problem of local minima, which is common in the estimation of high dimensional transformations, we employ a hierarchical strategy where a small subset of voxels with more distinctive attribute vectors are first deployed as landmarks to estimate a relatively robust low-degrees-of-freedom transformation. As the registration progresses, an increasing number of voxels are permitted to participate in refining the correspondence matching. A scheme as such allows less conservative progression of the correspondence matching towards the optimal solution, and hence results in a faster matching speed. Results indicate that better accuracy can be achieved by F-TIMER, compared with other deformable registration algorithms [1, 2], with significantly reduced computation time cost of 4–14 folds.

Neuroimaging at the group level requires spatial normalization of individual structural data. We propose a geometric approach that consists in matching a series of cortical surfaces through diffeomorphic registration of their sulcal imprints. The resulting 3D transforms naturally extends to the entire MRI volumes. The DIffeomorphic Sulcal-based COrtical (DISCO) registration integrates two recent technical outcomes: 1) the automatic extraction, identification and simplification of numerous sulci from T1-weighted MRI data series hereby revealing the sulcal imprint and 2) the measure-based diffeomorphic registration of those crucial anatomical landmarks. We show how the DISCO registration may be used to elaborate a sulcal template which optimizes the distribution of constraints over the entire cortical ribbon. DISCO was evaluated through a group of 20 individual brains. Quantitative and qualitative indices attest how this approach may improve both alignment of sulcal folds and overlay of gray and white matter volumes at the group level.

The human lungs are divided into five independent compartments called lobes. The lobar fissures separate the lung lobes. It is hypothesized that the lobar surfaces slide against each other during respiration. We propose a method to evaluate the sliding motion of the lobar surfaces during respiration using lobe-by-lobe mass-preserving non-rigid image registration. We measure lobar sliding by evaluating the relative displacement on both sides of the fissure. The results show a superior-inferior gradient in the magnitude of lobar sliding. We compare whole-lung-based registration accuracy to lobe-by-lobe registration accuracy using vessel bifurcation landmarks.

Non-rigid registration accuracy assessment is typically performed by evaluating the target registration error at manually placed landmarks. For 4D-CT lung data, we compare two sets of landmark distributions: a smaller set primarily defined on vessel bifurcations as commonly described in the literature and a larger set being well-distributed throughout the lung volume. For six different registration schemes (three in-house schemes and three schemes frequently used by the community) the landmark error is evaluated and found to depend significantly on the distribution of the landmarks. In particular, lung regions near to the pleura show a target registration error three times larger than near-mediastinal regions. While the inter-method variability on the landmark positions is rather small, the methods show discriminating differences with respect to consistency and local volume change. In conclusion, both a well-distributed set of landmarks and a deformation vector field analysis are necessary for reliable non-rigid registration accuracy assessment.

The computation of accurate motion fields is a crucial aspect in 4D medical imaging. It is usually done using a non-linear registration without further modeling of physiological motion properties. However, a globally homogeneous smoothing (regularization) of the motion field during the registration process can contradict the characteristics of motion dynamics. This is particularly the case when two organs slip along each other which leads to discontinuities in the motion field. In this paper, we present a diffusion-based model for incorporating physiological knowledge in image registration. By decoupling normal- and tangential-directed smoothing, we are able to estimate slipping motion at the organ borders while ensuring smooth motion fields in the inside and preventing gaps to arise in the field. We evaluate our model focusing on the estimation of respiratory lung motion. By accounting for the discontinuous motion of visceral and parietal pleurae, we are able to show a significant increase of registration accuracy with respect to the target registration error (TRE).

Recent US systems allow the real-time acquisition of volume data, either by freehand 3D techniques or novel transducer hardware. However, the acquisition of large volumes is limited by the field of view of the US transducer and anatomical scanning windows into the patient. Mosaicing of several 3D US scans has been proposed to generate large US volumes. Still, US imaging specific characteristics and artifacts make it challenging to create high quality mosaics. For many clinical cases, especially interventions, additional high quality CT data is available. In this paper we present a novel multi-variate, multi-modal 3D US registration and mosaicing approach, which reduces the effects of ultrasound imaging artifacts on mosaic quality, by incorporating information from co-registered CT.

Radio frequency ablation of liver cancer is a minimally invasive alternative to open surgery. Typically, the preoperative planning is done on an MR (or CT) scan, while the intervention relies on ultrasound (US) guidance. Registration of intra-operative US and preoperative MR (or CT) would assist navigation and increase the confidence of RFA needle positioning. In this paper we present a novel method for registration of US and MR images of the liver. Hepatic vessels are extracted from 2D US by an algorithm that models US dynamics. It generates 2D probability maps representing hepatic vessels which are then combined into probability volumes. A multi-resolution registration framework performs registration of the pre-processed MR with two 3D vessel probability images. The accuracy, robustness and speed of the method were assessed by registering eight US/MR datasets. High robustness (86%) and reasonable accuracy (1.98°, 4.10mm), acceptable for the RFA clinical application, suggest that the method has a good potential for intra-operative use.

We address the problem of the viewing-angle dependency of ultrasound images for registration. The reflected signal from large scale tissue boundaries is dependent on the incident angle of the beam. This applies an implicit weighting on the ultrasound image, dependent on the viewing-angle, which negatively affects the registration process, especially when utilizing curved linear transducers. We show that a simple reweighting of the images, considering a common physical model for ultrasound imaging, is not feasible. We therefore introduce a new matching function, separating reflectivity and scattering regions, which are the results of two different types of physical interactions of the ultrasound beam with the tissue. We use the local phase for identifying regions of reflectivity, and consider it as one part of our matching function, combining feature- and intensity-based aspects. First experiments provide good results for this novel registration approach.

A method is described for registering preoperative magnetic resonance (MR) to intraoperative transrectal ultrasound (TRUS) images of the prostate gland. A statistical motion model (SMM) of the prostate is first built using training data provided by biomechanical simulations of the motion of a patient-specific finite element model, derived from a preoperative MR image. The SMM is then registered to a 3D TRUS image by maximising the likelihood of the shape of an SMM instance given a voxel-intensity-based feature, which represents an estimate of normal vector at the surface of the prostate gland. Using data acquired from 7 patients, the accuracy of registering T2 MR to 3D TRUS images was evaluated using anatomical landmarks inside the gland. The results show that the proposed registration method has a root-mean-square target registration error of 2.66 mm.

We present an accurate and efficient technique to combine and rasterize multiple 3D ultrasound (3DUS) image volumes originally presented in spherical coordinates into a single, 3D Cartesian image that uniformly samples the total field of view. To ensure the consistency of merged image content in overlapping regions, image re-registration was performed by maximizing mutual information (MI). The technique was applied to 22 3DUS image volumes obtained during five neurosurgical patient cases. The computational cost of the approach increases linearly with the number of images involved (average time to combine and rasterize one pair of 3DUS images was 1.5 sec). Interpolation was approximately 20% more accurate in overlapping regions when re-registration was performed before rasterization and minimized feature loss and/or blurring that was evident without re-registration. In addition, we report the average translational (35.2 mm) and rotational (38.5

o

) capture ranges for the MI re-registration of two volumetric 3DUS images. The technique is applicable in any clinical application in which volumetric true 3DUS is acquired.

Registration of intraoperative ultrasound (US) with preoperative computed tomography (CT) data for interventional guidance is a subject of immense interest, particularly for percutaneous spinal injections. We propose a biomechanically constrained group-wise registration of US to CT images of the lumbar spine. Each vertebra in CT is treated as a sub-volume and transformed individually. The sub-volumes are then reconstructed into a single volume. The algorithm simulates an US image from the CT data at each iteration of the registration. This simulated US image is used to calculate an intensity based similarity metric with the real US image. A biomechanical model is used to constrain the displacement of the vertebrae relative to one another. Covariance Matrix Adaption - Evolution Strategy (CMA-ES) is utilized as the optimization strategy. Validation is performed on CT and US images from a phantom designed to preserve realistic curvatures of the spine. The technique is able to register initial misalignments of up to 20mm with a success rate of 82%, and those of up to 10mm with a success rate of 98.6%.

This paper presents a general PDE-framework for registration of contrast enhanced images. The approach directly applies the idea of separating the contrast enhancement term from the images in the regularization terms. In our formulation, we stay consistent with existing non-parametric image registration techniques, however, we carry an additional contrast enhancement term throughout. A mathematically rigorous approach is pursued which can exploit various forms of regularization. In this paper, our experiments are built based on diffusion regularization for both contrast enhancement and the deformation field.

The widely used procedure of evaluation of cup orientation following total hip arthroplasty using single standard anteroposterior (AP) radiograph is known inaccurate, largely due to the wide variability in individual pelvic orientation relative to X-ray plate. 2D/3D rigid image registration methods have been introduced for an accurate determination of the post-operative cup alignment with respect to an anatomical reference extracted from the CT data. Although encouraging results have been reported, their extensive usage in clinical routine is still limited. This may be explained by their requirement of a CAD model of the prosthesis, which is often difficult to be organized from the manufacturer due to the proprietary issue, and by their requirement of a pre-operative CT scan, which is not available for most retrospective studies. To address these issues, we developed and validated a statistically deformable 2D/3D registration approach for accurate determination of post-operative cup orientation. No CAD model and pre-operative CT data is required any more. Quantitative and qualitative results evaluated on cadaveric and clinical datasets are given, which indicate the validity of the approach.

In this paper we propose a novel similarity metric and a method for deformable registration of two images for a specific clinical application. The basic assumption in almost all deformable registration approaches is that there exist explicit correspondences between pixels across the two images. This principle is used to design image (dis)similarity metrics, such as sum of squared differences (SSD) or mutual information (MI). This assumption is strongly violated, for instance, within specific regions of images from abdominal or pelvic section of a patient taken at two different time points. Nevertheless, in some clinical applications, it is required to compute a smooth deformation field for all the regions within the image including the boundaries of such regions. In this paper, we propose a deformable registration method, which utilizes a priori intensity distributions of the regions delineated on one of the images to devise a new similarity measure that varies across regions of the image to establish a smooth and robust deformation field. We present validation results of the proposed method in mapping bladder, prostate, and rectum contours of computer tomography (CT) volumes of 10 patients taken for prostate cancer radiotherapy treatment planning and verification.

CT guided tumor ablation of the liver often suffers from a lack of visualization of the target tumor and surrounding critical structures. This information is available on pre-operative contrast enhanced MR images and a non-rigid registration technique is desirable. However while registration methods have been successfully tested retrospectively on patient data, very few have been incorporated into clinical procedures. A non-rigid registration technique has been evaluated, optimized and validated to be able to perform registration of the liver between MR to CT images, and between intra-operative CT images. The method requires pre-processing and segmentation of the liver, and presents an accuracy of approximately 2mm. A clinical feasibility study has been conducted in 5 liver ablation cases. The method helps clinicians enhance interventional planning, confirm ablation probe location with respect to the tumor, and in the case of cryotherapy, evaluate tumor coverage by the ice ball.

Fractional anisotropy (FA), a very widely used measure of fiber integrity based on diffusion tensor imaging (DTI), is a problematic concept as it is influenced by several quantities including the number of dominant fiber directions within each voxel, each fiber’s anisotropy, and partial volume effects from neighboring gray matter. With High-angular resolution diffusion imaging (HARDI) and the tensor distribution function (TDF), one can reconstruct multiple underlying fibers per voxel and their individual anisotropy measures by representing the diffusion profile as a probabilistic mixture of tensors. We found that FA, when compared with TDF-derived anisotropy measures, correlates poorly with individual fiber anisotropy, and may sub-optimally detect disease processes that affect myelination. By contrast, mean diffusivity (MD) as defined in standard DTI appears to be more accurate. Overall, we argue that novel measures derived from the TDF approach may yield more sensitive and accurate information than DTI-derived measures.

We present a method for the fully automated extraction of the cingulum using diffusion tensor imaging (DTI) data. We perform whole-brain tractography and initialize tract selection in the cingulum with a registered DTI atlas. Tracts are parameterized from which tract co-linearity is derived. The tract set, filtered on the basis of co-linearity with the cingulum shape, yields an improved segmentation of the cingulum and is subsequently optimized in an iterative fashion to further improve the tract selection. We evaluate the method using a large DTI database of 500 subjects from the general population and show robust extraction of tracts in the entire cingulate bundle in both hemispheres. We demonstrate the use of the extracted fiber-tracts to compare left and right cingulate bundles. Our asymmetry analysis shows a higher fractional anisotropy in the left anterior part of the cingulum compared to the right side, and the opposite effect in the posterior part.

Estimating the complete set of white matter fascicles (the projectome) from diffusion data requires evaluating an enormous number of potential pathways; consequently, most algorithms use computationally efficient greedy methods to search for pathways. The limitation of this approach is that critical global parameters - such as data prediction error and white matter volume conservation - are not taken into account. We describe BlueMatter, a parallel algorithm for global projectome evaluation, which uniquely accounts for global prediction error and volume conservation. Leveraging the BlueGene/L supercomputing architecture, BlueMatter explores a massive database of 180 billion candidate fascicles. The candidates are derived from several sources, including atlases and mutliple tractography algorithms. Using BlueMatter we created the highest resolution, volume-conserved projectome of the human brain.

We propose a method to create a dual tensor atlas from multiple coregistered non-HARDI datasets. Increased angular resolution is ensured by random variations of subject positioning in the scanner and different local rotations applied during coregistration resulting in dispersed gradient directions. Simulations incorporating residual coregistration misalignments show that using 10 subjects should already double the angular resolution, even at a relatively low

b

-value of

b

 = 1000 smm

− 2

. Commisural corpus callosum fibers reconstructed by our method closely approximated those found in a HARDI dataset.

High angular resolution diffusion imaging (HARDI) has become an important magnetic resonance technique for in vivo imaging. Current techniques for estimating the diffusion orientation distribution function (ODF), i.e., the probability density function of water diffusion along any direction, do not enforce the estimated ODF to be nonnegative or to sum up to one. Very often this leads to an estimated ODF which is not a proper probability density function. In addition, current methods do not enforce any spatial regularity of the data. In this paper, we propose an estimation method that naturally constrains the estimated ODF to be a proper probability density function and regularizes this estimate using spatial information. By making use of the spherical harmonic representation, we pose the ODF estimation problem as a convex optimization problem and propose a coordinate descent method that converges to the minimizer of the proposed cost function. We illustrate our approach with experiments on synthetic and real data.

In this paper, we compare a representative selection of current state-of-the-art algorithms in diffusion-weighted magnetic resonance imaging (dwMRI) tractography, and propose a novel way to quantitatively define the connectivity between brain regions. As criterion for the comparison, we quantify the connectivity computed with the different methods. We provide initial results using diffusion tensor, spherical deconvolution, ball-and-stick model, and persistent angular structure (PAS) along with deterministic and probabilistic tractography algorithms on a human DWI dataset. The connectivity is presented for a representative selection of regions in the brain in matrices and connectograms.Our results show that fiber crossing models are able to reveal connections between more brain areas than the simple tensor model. Probabilistic approaches show in average more connected regions but lower connectivity values than deterministic methods.

We describe a technique to simultaneously estimate a weighted, positive-definite multi-tensor fiber model and perform tractography. Existing techniques estimate the local fiber orientation at each voxel independently so there is no running knowledge of confidence in the estimated fiber model. We formulate fiber tracking as recursive estimation: at each step of tracing the fiber, the current estimate is guided by the previous. To do this we model the signal as a weighted mixture of Gaussian tensors and perform tractography within a filter framework. Starting from a seed point, each fiber is traced to its termination using an unscented Kalman filter to simultaneously fit the local model and propagate in the most consistent direction. Further, we modify the Kalman filter to enforce model constraints, i.e. positive eigenvalues and convex weights. Despite the presence of noise and uncertainty, this provides a causal estimate of the local structure at each point along the fiber. Synthetic experiments demonstrate that this approach significantly improves the angular resolution at crossings and branchings while consistently estimating the mixture weights.

In vivo

experiments confirm the ability to trace out fibers in areas known to contain such crossing and branching while providing inherent path regularization.

In this paper we deal with the problem of quantification of diffusion tensor (DT) data sets. A set of measures and a 2D tract mapping technique are proposed to analyze the fiber structures in brain white matter and to allow for comparisons between different subjects, either patients or controls. Features such as integrity, discontinuity and connectivity of the fiber bundles are proposed and analyzed, taking into account longitudinal and transverse information of the fiber bundle under study. The performance of the proposed characterization framework is shown analyzing the corticospinal tracts of control data sets and pathological cases, comparing the measures between controls and patients and also between the right and left hemispheres. A reproducibility study is also performed to show the robustness of the proposed measures.

Compared with Diffusion Tensor Imaging (DTI), High Angular Resolution Imaging (HARDI) can better explore the complex microstructure of white matter. Orientation Distribution Function (ODF) is used to describe the probability of the fiber direction. Fisher information metric has been constructed for probability density family in Information Geometry theory and it has been successfully applied for tensor computing in DTI. In this paper, we present a state of the art Riemannian framework for ODF computing based on Information Geometry and sparse representation of orthonormal bases. In this Riemannian framework, the exponential map, logarithmic map and geodesic have closed forms. And the weighted Frechet mean exists uniquely on this manifold. We also propose a novel scalar measurement, named Geometric Anisotropy (GA), which is the Riemannian geodesic distance between the ODF and the isotropic ODF. The Renyi entropy

$H_{\frac{1}{2}}$

of the ODF can be computed from the GA. Moreover, we present an Affine-Euclidean framework and a Log-Euclidean framework so that we can work in an Euclidean space. As an application, Lagrange interpolation on ODF field is proposed based on weighted Frechet mean. We validate our methods on synthetic and real data experiments. Compared with existing Riemannian frameworks on ODF, our framework is model-free. The estimation of the parameters, i.e. Riemannian coordinates, is robust and linear. Moreover it should be noted that our theoretical results can be used for any probability density function (PDF) under an orthonormal basis representation.

Least Squares (LS) and its weighted version are standard techniques to estimate the Diffusion Tensor (DT) from Diffusion Weighted Images (DWI). They require to linearize the problem by computing the logarithm of the DWI. For the single-coil Rician noise model it has been shown that this model does not introduce a significant bias, but for multiple array coils and parallel imaging, the noise cannot longer be modeled as Rician. As a result the validity of LS approaches is not assured. An analytical study of noise statistics for a multiple coil system is carried out, together with the Weighted LS formulation and noise analysis for this model. Results show that the bias in the computation of the components of the DT may be comparable to their variance in many cases, stressing the importance of unbiased filtering previous to DT estimation.

This paper introduces a novel framework for global diffusion MRI tractography inspired from a spin glass model. The entire white matter fascicle map is parameterized by pieces of fibers called spins. Spins are encouraged to move and rotate to align with the main fiber directions, and to assemble into longer chains of low curvature. Moreover, they have the ability to adapt their quantity in regions where the spin concentration is not sufficient to correctly model the data. The optimal spin glass configuration is retrieved by an iterative minimization procedure, where chains are finally assimilated to fibers. As a result, all brain fibers appear as growing simultaneously until they merge with other fibers or reach the domain boundaries. In case of an ambiguity within a region like a crossing, the contribution of all neighboring fibers is used determine the correct neural pathway. This framework is tested on a MR phantom representing a 45° crossing and a real brain dataset. Notably, the framework was able to retrieve the triple crossing between the callosal fibers, the corticospinal tract and the arcuate fasciculus.

Determining cortical functional areas is an important goal for neurosciences and clinical neurosurgery. This paper presents a method for connectivity-based parcellation of the entire human cortical surface, exploiting the idea that each cortex region has a specific connection profile. The connectivity matrix of the cortex is computed using analytical Q-ball-based tractography. The parcellation is achieved independently for each subject and applied to the subset of the cortical surface endowed with enough connections to estimate safely a connectivity profile, namely the top of the cortical gyri. The key point of the method lies in a twofold reduction of the connectivity matrix dimension. First, parcellation amounts to iterating the clustering of Voronoï patches of the cortical surface into parcels endowed with homogeneous profiles. The parcels without intersection with the patch boundaries are selected for the final parcellation. Before clustering a patch, the complete profiles are collapsed into short profiles indicating connectivity with a set of putative cortical areas. These areas are supposed to correspond to the catchment basins of the watershed of the density of connection to the patch computed on the cortical surface. The results obtained for several brains are compared visually using a coordinate system.

This paper presents a belief propagation approach to the segmentation of the major white matter tracts in diffusion tensor images of the human brain. Unlike tractography methods that sample multiple fibers to be bundled together, we define a Markov field directly on the diffusion tensors to separate the main fiber tracts at the voxel level. A prior model of shape and direction guides a full segmentation of the brain into known fiber tracts; additional, unspecified fibers; and isotropic regions. The method is evaluated on various data sets from an atlasing project, healthy subjects, and multiple sclerosis patients.

A methodology to build a realistic phantom for the assessment of filtering performance in Diffusion Weighted Images (DWI) is presented. From a real DWI data–set, a regularization process is carried out taking into account the diffusion model. This process drives to a model which accurately preserves the structural characteristics of actual DWI volumes, being in addition regular enough to be considered as a

noise–free

data–set and therefore to be used as a ground–truth. We compare our phantom with a kind of simplified phantoms commonly used in the literature (those based on homogeneous cross sections), concluding that the latter may introduce important biases in common quality measures used in the filtering performance assessment, and even drive to erroneous conclusions in the comparison of different filtering techniques.

The automatic analysis of longitudinal changes between Diffusion Tensor Imaging (DTI) acquisitions is a promising tool for monitoring disease evolution. However, few works address this issue and existing methods are generally limited to the detection of changes between scalar images characterizing diffusion properties, such as Fractional Anisotropy or Mean Diffusivity, while richer information can be exploited from the whole set of Apparent Diffusion Coefficient (ADC) images that can be derived from a DTI acquisition. In this paper, we present a general framework for detecting changes between two sets of ADC images and we investigate the performance of four statistical tests. Results are presented on both simulated and real data in the context of the follow-up of multiple sclerosis lesion evolution.

Information from the full diffusion tensor (DT) was used to compute voxel-wise genetic contributions to brain fiber microstructure. First, we designed a new multivariate intraclass correlation formula in the log-Euclidean framework [1]. We then analyzed used the full multivariate structure of the tensor in a multivariate version of a voxel-wise maximum-likelihood structural equation model (SEM) that computes the variance contributions in the DTs from genetic (A), common environmental (C) and unique environmental (E) factors. Our algorithm was tested on DT images from 25 identical and 25 fraternal twin pairs. After linear and fluid registration to a mean template, we computed the intraclass correlation and Falconer’s heritability statistic for several scalar DT-derived measures and for the full multivariate tensors. Covariance matrices were found from the DTs, and inputted into SEM. Analyzing the full DT enhanced the detection of A and C effects. This approach should empower imaging genetics studies that use DTI.

In this paper, we present a fast numerical scheme to estimate Partition Functions (PF) of 3D Ising fields. Our strategy is applied to the context of the joint detection-estimation of brain activity from functional Magnetic Resonance Imaging (fMRI) data, where the goal is to automatically recover activated regions and estimate region-dependent hemodynamic filters. For any region, a specific binary Markov random field may embody spatial correlation over the hidden states of the voxels by modeling whether they are activated or not. To make this spatial regularization fully adaptive, our approach is first based upon a classical path-sampling method to approximate a small subset of

reference

PFs corresponding to prespecified regions. Then, the proposed extrapolation method allows us to approximate the PFs associated with the Ising fields defined over the remaining brain regions. In comparison with preexisting approaches, our method is robust to topological inhomogeneities in the definition of the

reference

regions. As a result, it strongly alleviates the computational burden and makes spatially adaptive regularization of whole brain fMRI datasets feasible.

Inter-subject parcellation of functional Magnetic Resonance Imaging (fMRI) data based on a standard General Linear Model (GLM) and spectral clustering was recently proposed as a means to alleviate the issues associated with spatial normalization in fMRI. However, for all its appeal, a GLM-based parcellation approach introduces its own biases, in the form of a priori knowledge about the shape of Hemodynamic Response Function (HRF) and task-related signal changes, or about the subject behaviour during the task.

In this paper, we introduce a data-driven version of the spectral clustering parcellation, based on Independent Component Analysis (ICA) and Partial Least Squares (PLS) instead of the GLM. First, a number of independent components are automatically selected. Seed voxels are then obtained from the associated ICA maps and we compute the PLS latent variables between the fMRI signal of the seed voxels (which covers regional variations of the HRF) and the principal components of the signal across all voxels. Finally, we parcellate all subjects data with a spectral clustering of the PLS latent variables.

We present results of the application of the proposed method on both single-subject and multi-subject fMRI datasets. Preliminary experimental results, evaluated with intra-parcel variance of GLM t-values and PLS derived t-values, indicate that this data-driven approach offers improvement in terms of parcellation accuracy over GLM based techniques.

In neuroimaging cluster-based inference has generally been found to be more powerful than voxel-wise inference [1]. However standard cluster-based methods assume stationarity (constant smoothness), while under nonstationarity clusters are larger in smooth regions just by chance, making false positive risk spatially variant. Hayasaka et al. [2] proposed a Random Field Theory (RFT) based nonstationarity adjustment for cluster inference and validated the method in terms of controlling the overall family-wise false positive rate. The RFT-based methods, however, have never been directly assessed in terms of homogeneity of local false positive risk. In this work we propose a new cluster size adjustment that accounts for local smoothness, based on local empirical cluster size distributions and a two-pass permutation method. We also propose a new approach to measure homogeneity of local false positive risk, and use this method to compare the RFT-based and our new empirical adjustment methods. We apply these techniques to both cluster-based and a related inference, threshold-free cluster enhancement (TFCE). Using simulated and real data we confirm the expected heterogeneity in false positive risk with unadjusted cluster inference but find that RFT-based adjustment does not fully eliminate heterogeneity; we also observe that our proposed empirical adjustment dramatically increases the homogeneity and TFCE inference is generally quite robust to nonstationarity.

We present a method for controlling a dynamical system using real-time fMRI. The objective for the subject in the MR scanner is to balance an inverted pendulum by activating the left or right hand or resting. The brain activity is classified each second by a neural network and the classification is sent to a pendulum simulator to change the force applied to the pendulum. The state of the inverted pendulum is shown to the subject in a pair of VR goggles. The subject was able to balance the inverted pendulum during several minutes, both with real activity and imagined activity. In each classification 9000 brain voxels were used and the response time for the system to detect a change of activity was on average 2-4 seconds. The developments here have a potential to aid people with communication disabilities, such as locked in people. Another future potential application can be to serve as a tool for stroke and Parkinson patients to be able to train the damaged brain area and get real-time feedback for more efficient training.

The standard general linear model (GLM) for rapid event-related fMRI design protocols typically ignores reduction in hemodynamic responses in successive stimuli in a train due to incomplete recovery from the preceding stimuli. To capture this adaptation effect, we incorporate a region-specific adaptation model into GLM. The model quantifies the rate of adaptation across brain regions, which is of interest in neuroscience. Empirical evaluation of the proposed model demonstrates its potential to improve detection sensitivity. In the fMRI experiments using visual and auditory stimuli, we observed that the adaptation effect is significantly stronger in the visual area than in the auditory area, suggesting that we must account for this effect to avoid bias in fMRI detection.

Most fMRI studies use voxel-wise statistics to carry out intra-subject as well as inter-subject analysis. We show that statistics derived from voxel-wise comparisons are likely to be noisy and error prone, especially for inter-subject comparisons. In this paper we propose a novel metric called

weighted cluster coverage

to compare two activation maps. This metric is based on the intersection of spatially contiguous clusters of activations. It is found to be more robust than voxel-wise comparisons and could potentially lead to more statistical power in fMRI-based group studies.