Statistical Atlases and Computational Models of the Heart. ACDC and MMWHS Challenges

A cardiac motion atlas provides a space of reference in which the cardiac motion fields of a cohort of subjects can be directly compared. From such atlases, descriptors can be learned for subsequent diagnosis and characterization of disease. Traditionally, such atlases have been formed from imaging data acquired using a single modality. In this work we propose a framework for building a multimodal cardiac motion atlas from MR and ultrasound data and incorporate a multiview classifier to exploit the complementary information provided by the two modalities. We demonstrate that our novel framework is able to detect non ischemic dilated cardiomyopathy patients from ultrasound data alone, whilst still exploiting the MR based information from the multimodal atlas. We evaluate two different approaches based on multiview learning to implement the classifier and achieve an improvement in classification performance from 77.5% to 83.50% compared to the use of US data without the multimodal atlas.

Registration and segmentation of anatomical structures are two well studied problems in medical imaging. Optimizing segmentation and registration jointly has been proven to improve results for both challenges. In this work, we propose a joint optimization scheme for registration and segmentation using dictionary learning based descriptors. Our joint registration and segmentation aims to solve an optimization function, which enables better performance for both of these ill-posed processes. We build two dictionaries for background and myocardium for square patches extracted from training images. Based on dictionary learning residuals and sparse representations defined on these pre-trained dictionaries, a Markov Random Field (MRF) based joint optimization scheme is built. The algorithm proceeds iteratively updating the dictionaries in an online fashion. The accuracy of the proposed method is illustrated on Cardiac Phase-resolved Blood Oxygen-Level-Dependent (CP-BOLD) MRI and standard cine Cardiac MRI data from MICCAI 2013 SATA Segmentation Challenge. The proposed joint segmentation and registration method achieves higher dice accuracy for myocardium segmentation compared to its variants.

A fully automatic approach for the segmentation of the left ventricle (LV) myocardium in porcine cardiac cine MRI images is proposed based on deep convolutional neural networks (CNN). We trained a 56-layer residual learning CNN (ResNet-56) from scratch on a set of porcine cine MRI images acquired internally, and another CNN via transfer learning by fine tuning a network previously trained on a public human cine MRI dataset. A leave-one-out validation was performed on an 8-specimen porcine cardiac cine MRI dataset (3,600 slices). Comparisons with manual segmentations show that both CNN models are able to produce precise results (99.94% “good” segmentations), while the CNN trained through transfer learning performs better by achieving Dice similarity coefficient (DSC) of 0.86, Hausdorff distance (HD) of 4.01 mm, and overall average perpendicular distance (APD) of 1.04 mm on average.

The left atrial appendage closure surgery is the main treatment of thrombosis in patients with atrial fibrillation. Left atrial appendage neck is the region of implanting occluder in the surgery. The occluder is strictly matched with the neck to prevent the occluder piercing the endocardium or falling off and avoid threatening patients’ lives. So we build a model of left atrial appendage neck based on the segmentation result of maximal volume phase from CT data. The model automatically generated by our approach is a circumscribed cylinder to represent the irregular columnar neck. This circumscribed cylinder can help to determine the diameter of the closure device before the surgery, the implanted position and pose of the device during the surgery. Specifically, we successfully solved these problems including the auto-detection of the boundary points of left atrial appendage ostium, the establishment of the standard coordinate system, the auto-calculation of the cylinder height and the polychromatic display of occlusive tension on the neck. We built our model on 100 patients’ data and dissected the three pig hearts to do comparative experiments. Tests were performed in 67 occlusion surgeries with the success rate of 97.01\(\%\). These indicate that our approach can precisely and non-invasively model the left atrial appendage neck for assisting closure surgeries.

The detection of substances in the left atrial appendage (LAA) is essential in evaluating disease development and treatment planning in patients with atrial fibrillation. The advent of 4D-CT bringing high spatiotemporal resolution, we present a new approach for the detection of substances in the LAA by spatiotemporal motion analysis and make a detailed judgment and analysis of spatial distribution and classification of most objects in the LAA. The noise interference is also eliminated properly. This approach requires the extraction of the optical flow field for all adjacent phases in a cardiac cycle of 20 phases. According to the optical flow information of 19 optical flow fields, we adopt the nearest neighbor interpolation method to establish the motion trajectory of the key voxels in a whole cardiac cycle. Then we create a hierarchical clustering tree by calculating the similarity between the tracks based on hierarchical clustering and find the corresponding classification for every track. Different classifications of tracks represent the divisions of substances in the LAA. Finally, we perform the stress and strain detection of the critical lump using time-frequency analysis of their trajectories. Tests and validations of our approach were performed on 32 data sets (artificial diagnosis of echocardiography and 4-D CT). The frequency responded range to stress and strain of different substances was obtained, which included normal circulation blood, mild, moderate and severe SEC blood, initial jelling thrombi, old or calcified thrombi, organic thrombi and pectinate muscles. Our results are consistent with the two artificial diagnoses. Furthermore, they can refine the identification of substances such as their texture and tiny sizes.

Delayed Enhancement (DE) Cardiac Magnetic Resonance (CMR) allows practitioners to identify fibrosis in the myocardium. It is of importance in the differential diagnosis and therapy selection in Hypertrophic Cardiomyopathy (HCM). However, most clinical semiautomatic scar quantification methods present high intra- and interobserver variability in the case of HCM. Automatic methods relying on mixture model estimation of the myocardial intensity distribution are also subject to variability due to inaccuracies of the myocardial mask. In this paper, the CINE-CMR image information is incorporated to the estimation of the DE-CMR tissue distributions, without assuming perfect alignment between the two modalities nor the same label partitions in them. For this purpose, we propose an expectation maximization algorithm that estimates the DE-CMR distribution parameters, as well as the conditional probabilities of the DE-CMR labels with respect to the labels of CINE-CMR, with the latter being an input of the algorithm. Our results show that, compared to applying the EM using only the DE-CMR data, the proposed algorithm is more accurate in estimating the myocardial tissue parameters and obtains higher likelihood of the fibrosis voxels, as well as a higher Dice coefficient of the subsequent segmentations.

Statistical atlases of myocardial fiber directions have great utility in modelling applications. The first step in building atlases requires a registration of the hearts to a template. In this paper, we performed groupwise registration on a small database of explanted pig hearts (\(N=4\)) and coupled it with a multilevel pairwise registration framework in order to generate an average cardiac geometry. The scheme implemented in our experiments effectively registers and normalizes the hearts despite a high variability in cardiac measurements. In addition, we adopted an intuitive averaging technique on the transformed versions of each heart to obtain a new reference geometry at every iteration. This reduces biases that may be introduced by the selection of an initial reference geometry in the construction of an average cardiac geometry. The next step will focus on improving current results by using a larger database of heart samples.

In this paper, we propose a fully automatic MRI cardiac segmentation method based on a novel deep convolutional neural network (CNN) designed for the 2017 ACDC MICCAI challenge. The novelty of our network comes with its embedded shape prior and its loss function tailored to the cardiac anatomy. Our model includes a cardiac center-of-mass regression module which allows for an automatic shape prior registration. Also, since our method processes raw MR images without any manual preprocessing and/or image cropping, our CNN learns both high-level features (useful to distinguish the heart from other organs with a similar shape) and low-level features (useful to get accurate segmentation results). Those features are learned with a multi-resolution conv-deconv “grid” architecture which can be seen as an extension of the U-Net.

Experimental results reveal that our method can segment the left and right ventricles as well as the myocardium from a 3D MRI cardiac volume in 0.4 s with an average Dice coefficient of 0.90 and an average Hausdorff distance of \(10.4\) mm.

Computer-aided diagnosis of cardiovascular diseases (CVDs) with cine-MRI is an important research topic to enable improved stratification of CVD patients. However, current approaches that use expert visualization or conventional clinical indices can lack accuracy for borderline classications. Advanced statistical approaches based on eigen-decomposition have been mostly concerned with shape and motion indices. In this paper, we present a new approach to identify CVDs from cine-MRI by estimating large pools of radiomic features (statistical, shape and textural features) encoding relevant changes in anatomical and image characteristics due to CVDs. The calculated cine-MRI radiomic features are assessed using sequential forward feature selection to identify the most relevant ones for given CVD classes (e.g. myocardial infarction, cardiomyopathy, abnormal right ventricle). Finally, advanced machine learning is applied to suitably integrate the selected radiomics for final multi-feature classification based on Support Vector Machines (SVMs). The proposed technique was trained and cross-validated using 100 cine-MRI cases corresponding to five different cardiac classes from the ACDC MICCAI 2017 challenge (https://​www.​creatis.​insa-lyon.​fr/​Challenge/​acdc/​index.​html). All cases were correctly classified in this preliminary study, indicating potential of using large-scale radiomics for MRI-based diagnosis of CVDs.

We present a fast fully automatic method for cardiac segmentation in ED and ES short axis MRI. At first we extract a region where the whole heart is situated, using a new, time-based approach. Then, the segmentation in LV, myocardium and right ventricle (RV) is obtained for a slice in a basal ED slice where both cavities are well distinguished. The extracted regions are tracked for the whole slice sequence backwards and forwards in ED. In all cases the segmentation is based on MRF optimization in four classes, two for the blood areas, and one for the myocardium and the background. Subsequently the segmentation in the ES images is based on the result of ED segmentation. As the epicardium is not well delineated, a smoothing process based on spline curves is used for obtaining the final result. We consider that, with an unsupervised method, we have obtained good results for LV and satisfactory for the RV and the myocardium on the ACDC 2017 datasets.

Segmentation of the heart in cardiac cine MR is clinically used to quantify cardiac function. We propose a fully automatic method for segmentation and disease classification using cardiac cine MR images. A convolutional neural network (CNN) was designed to simultaneously segment the left ventricle (LV), right ventricle (RV) and myocardium in end-diastole (ED) and end-systole (ES) images. Features derived from the obtained segmentations were used in a Random Forest classifier to label patients as suffering from dilated cardiomyopathy, hypertrophic cardiomyopathy, heart failure following myocardial infarction, right ventricular abnormality, or no cardiac disease.

The method was developed and evaluated using a balanced dataset containing images of 100 patients, which was provided in the MICCAI 2017 automated cardiac diagnosis challenge (ACDC). Segmentation and classification pipeline were evaluated in a four-fold stratified cross-validation. Average Dice scores between reference and automatically obtained segmentations were 0.94, 0.88 and 0.87 for the LV, RV and myocardium. The classifier assigned 91% of patients to the correct disease category. Segmentation and disease classification took 5 s per patient.

The results of our study suggest that image-based diagnosis using cine MR cardiac scans can be performed automatically with high accuracy.

Accurate segmentation of the heart is an important step towards evaluating cardiac function. In this paper, we present a fully automated framework for segmentation of the left (LV) and right (RV) ventricular cavities and the myocardium (Myo) on short-axis cardiac MR images. We investigate various 2D and 3D convolutional neural network architectures for this task. Experiments were performed on the ACDC 2017 challenge training dataset comprising cardiac MR images of 100 patients, where manual reference segmentations were made available for end-diastolic (ED) and end-systolic (ES) frames. We find that processing the images in a slice-by-slice fashion using 2D networks is beneficial due to a relatively large slice thickness. However, the exact network architecture only plays a minor role. We report mean Dice coefficients of 0.950 (LV), 0.893 (RV), and 0.899 (Myo), respectively with an average evaluation time of 1.1 s per volume on a modern GPU.

Cardiac magnetic resonance imaging improves on diagnosis of cardiovascular diseases by providing images at high spatiotemporal resolution. Manual evaluation of these time-series, however, is expensive and prone to biased and non-reproducible outcomes. In this paper, we present a method that addresses named limitations by integrating segmentation and disease classification into a fully automatic processing pipeline. We use an ensemble of UNet inspired architectures for segmentation of cardiac structures such as the left and right ventricular cavity (LVC, RVC) and the left ventricular myocardium (LVM) on each time instance of the cardiac cycle. For the classification task, information is extracted from the segmented time-series in form of comprehensive features handcrafted to reflect diagnostic clinical procedures. Based on these features we train an ensemble of heavily regularized multilayer perceptrons (MLP) and a random forest classifier to predict the pathologic target class. We evaluated our method on the ACDC dataset (4 pathology groups, 1 healthy group) and achieve dice scores of 0.945 (LVC), 0.908 (RVC) and 0.905 (LVM) in a cross-validation over the training set (100 cases) and 0.950 (LVC), 0.923 (RVC) and 0.911 (LVM) on the test set (50 cases). We report a classification accuracy of \(94 \%\) on a training set cross-validation and \(92\%\) on the test set. Our results underpin the potential of machine learning methods for accurate, fast and reproducible segmentation and computer-assisted diagnosis (CAD).

In this paper, we develop a 2D and 3D segmentation pipelines for fully automated cardiac MR image segmentation using Deep Convolutional Neural Networks (CNN). Our models are trained end-to-end from scratch using the ACD Challenge 2017 dataset comprising of 100 studies, each containing Cardiac MR images in End Diastole and End Systole phase. We show that both our segmentation models achieve near state-of-the-art performance scores in terms of distance metrics and have convincing accuracy in terms of clinical parameters. A comparative analysis is provided by introducing a novel dice loss function and its combination with cross entropy loss. By exploring different network structures and comprehensive experiments, we discuss several key insights to obtain optimal model performance, which also is central to the theme of this challenge.

In this paper, we propose a fully automatic method for segmentation of left ventricle, right ventricle and myocardium from cardiac Magnetic Resonance (MR) images using densely connected fully convolutional neural network. Dense Convolutional neural network (DenseNet) facilitates multi-path flow for gradients between layers during training by back-propagation and feature propagation. DenseNet also encourages feature reuse & thus substantially reduces the number of parameters while maintaining good performance, which is ideal in scenarios with limited data. The training data was subjected to Fourier analysis and classical computer vision (CV) techniques for Region of Interest (ROI) extraction. The parameters of the network were optimized by training with a dual cost function i.e. weighted cross-entropy and Dice co-efficient. For the task of automated heart diagnosis, cardiac parameters such as ejection fraction, volumes of ventricles etc. where calculated from segmentation masks predicted by the network at the end systole and diastole phases. Further these parameters were used as features to train a Random forest classifier. On the exclusively held-out test set (10% of training set) the proposed method for segmentation task achieved a mean dice score of 0.92, 0.87 and 0.86 for left ventricle, right ventricle and myocardium respectively. For automated cardiac disease diagnosis, the Random Forest classifier achieved an accuracy of 90%.

Segmenting ventricular structures from cardiovascular MR scan is important for quantitative evaluation of heart. Manual delineation is time-consuming and tedious and lack of reproductivity. Considering MR image quality, heart variance, spatial inconsistency and motion artifacts during scanning, it is still a non-trivial task for automatic segmentation methods. In this paper, we propose a general and fully automatic solution to concurrently segment three important ventricular structures. Rooting in the deep learning trend, our method starts from 3D Fully Convolutional Network (3D FCN). We then enhance the 3D FCN with two well-verified blocks: (1) we conduct transfer learning between a pre-trained C3D model and our 3D FCN to get good initialization and thus suppress overfitting. (2) since boosting the gradient flow in network is beneficial to promote segmentation performance, we attach several auxiliary loss functions so as to expose early layers to better supervision. Because the volume size imbalance among different ventricular structures often biases the training of our 3D FCN, to this end, we investigate the capacity of different loss functions and propose a Multi-class Dice Similarity Coefficient (mDSC) based loss function to re-weight the training for all classes. We verified our method, especially the significance of mDSC, on the Automated Cardiac Diagnosis Challenge 2017 datasets for MR image segmentation. Extensive experimental results demonstrate the promising performance of our method.

Automatic and accurate segmentation of Left Ventricle (LV) and Right Ventricle (RV) in cine-MRI is required to analyze cardiac function and viability. We present a fully convolutional neural network to efficiently segment LV and RV as well as myocardium. The network is trained end-to-end from scratch. Average dice scores from five-fold cross-validation on the ACDC training dataset were 0.94, 0.89, and 0.88 for LV, RV, and myocardium. Experimental results show the robustness of the proposed architecture.

Segmentation of the myocardium is a key step for image guided diagnosis in many cardiac diseases. In this article, we propose an automatic multi-atlas segmentation framework which relies on a very fast registration algorithm trained with convolutional neural networks. The speed of this registration method allows us to use a high number of templates in the multi-atlas segmentation while remaining computationally tractable. The performance of the propose approach is evaluated on a dataset of 100 end-diastolic and end-systolic MRI images of the STACOM 2017 Automated Cardiac Diagnosis Challenge (ACDC).

Segmenting cardiovascular volumes plays a crucial role for clinical applications, especially parsing the whole heart into fine-grained structures. However, conquering fuzzy boundaries and differentiating branchy structures in cardiovascular volume images still remain a challenging task. In this paper, we propose a general and fully automatic solution for fine-grained whole heart partition. The proposed framework originates from the 3D Fully Convolutional Network, and is reinforced in the following aspects: (1) By inheriting the knowledge from a pre-trained C3D Network, our network launches with a good initialization and gains capabilities in coping with overfitting. (2) We triggered several auxiliary loss functions on shallow layers to promote gradient flow and thus alleviate the training difficulties associated with deep neural networks. (3) Considering the obvious volume imbalance among different substructures, we introduced a Multi-class Dice Similarity Coefficient based metric to efficiently balance the training for all classes. We evaluated our method on the MM-WHS Challenge 2017 datasets. Extensive experimental results demonstrated the promising performance of our method. Our framework achieves promising results across different modalities and is general to be referred in other volumetric segmentation tasks.

We propose a pipeline of two fully convolutional networks for automatic multi-label whole heart segmentation from CT and MRI volumes. At first, a convolutional neural network (CNN) localizes the center of the bounding box around all heart structures, such that the subsequent segmentation CNN can focus on this region. Trained in an end-to-end manner, the segmentation CNN transforms intermediate label predictions to positions of other labels. Thus, the network learns from the relative positions among labels and focuses on anatomically feasible configurations. Results on the MICCAI 2017 Multi-Modality Whole Heart Segmentation (MM-WHS) challenge show that the proposed architecture performs well on the provided CT and MRI training volumes, delivering in a three-fold cross validation an average Dice Similarity Coefficient over all heart substructures of 88.9% and 79.0%, respectively. Moreover, on the MM-WHS challenge test data we rank first for CT and second for MRI with a whole heart segmentation Dice score of 90.8% and 87%, respectively, leading to an overall first ranking among all participants.

Non-invasive detection of cardiovascular disorders from radiology scans requires quantitative image analysis of the heart and its substructures. There are well-established measurements that radiologists use for diseases assessment such as ejection fraction, volume of four chambers, and myocardium mass. These measurements are derived as outcomes of precise segmentation of the heart and its substructures. The aim of this paper is to provide such measurements through an accurate image segmentation algorithm that automatically delineates seven substructures of the heart from MRI and/or CT scans. Our proposed method is based on multi-planar deep convolutional neural networks (CNN) with an adaptive fusion strategy where we automatically utilize complementary information from different planes of the 3D scans for improved delineations. For CT and MRI, we have separately designed three CNNs (the same architectural configuration) for three planes, and have trained the networks from scratch for voxel-wise labeling for the following cardiac structures: myocardium of left ventricle (Myo), left atrium (LA), left ventricle (LV), right atrium (RA), right ventricle (RV), ascending aorta (Ao), and main pulmonary artery (PA). We have evaluated the proposed method with 4-fold-cross-validation on the multi-modality whole heart segmentation challenge (MM-WHS 2017) dataset. A precision and dice index of 0.93 and 0.90, and 0.87 and 0.85 were achieved for CT and MR images, respectively. Cardiac CT volume was segmented in about 50 s, with cardiac MRI segmentation requiring around 17 s with multi-GPU/CUDA implementation.

Atlas-based segmentation is a well-known method for segmentation of medical images. In particular, this method could be used in an efficient way to automatically segment heart structures in MRI or CT scans. We propose, in this paper a more adaptive and interactive atlas-based segmentation method. The model presented combines several local probabilistic atlases with a topological graph. The local atlases provide more refined information about the structures’ shape while the spatial relationships between the atlases are learned and stored in a graph. Hence, local registrations need less computational time and the image segmentation can be guided by the user in an incremental way. Following this step, a pixel classification is performed with a hidden Markov random field that integrates the learned a priori information with the pixel intensities that originate from different modalities. Finally, an a posteriori correction is performed using Adaboost classifiers in order to correct voxels in the border of the seek region and improve the precision of the results. The proposed method is tested on CT scan and MRI images of the heart coming from the MM-WHS challenge.

Accurate whole-heart segmentation from multi-modality medical images (MRI, CT) plays an important role in many clinical applications, such as precision surgical planning and improvement of diagnosis and treatment. This paper presents a deeply-supervised 3D U-Net for fully automatic whole-heart segmentation by jointly using the multi-modal MRI and CT images. First, a 3D U-Net is employed to coarsely detect the whole heart and segment its region of interest, which can alleviate the impact of surrounding tissues. Then, we artificially enlarge the training set by extracting different regions of interest so as to train a deep network. We perform voxel-wise whole-heart segmentation with the end-to-end trained deeply-supervised 3D U-Net. Considering that different modality information of the whole heart has a certain complementary effect, we extract multi-modality features by fusing MRI and CT images to define the overall heart structure, and achieve final results. We evaluate our method on cardiac images from the multi-modality whole heart segmentation (MM-WHS) 2017 challenge.

To assist 3D cardiac image analysis, we propose an automatic whole heart segmentation using a deep learning framework combined with shape context information that is encoded in volumetric shape models. The proposed processing pipeline consists of three major steps: scout segmentation with orthogonal 2D U-nets, shape context estimation and refining segmentation with U-net and shape context. The proposed method was evaluated using the MMWHS challenge data. Two sets of networks were trained separately for contrast-enhanced CT and MRI. On the 20 training datasets, using 5-fold cross-validation, the average Dice coefficients for the left ventricle, the right ventricle, the left atrium, the right atrium and the myocardium of the left ventricle were 0.895, 0.795, 0.847, 0.821, 0.807 for MRI and 0.935, 0.825, 0.908, 0.881, 0.879 for CT, respectively. Further improvement may be possible given more training data or advanced data augmentation strategy.

Whole heart segmentation in CT images is a significant prerequisite for clinical diagnosis or treatment. In this work, we present a three-step multi-atlas-based method for obtaining a segmentation of the whole heart. In the first step, the region of the heart was detected by aligning the down-sampled patient CT with the low-resolution atlas images. The detected region of heart was used to crop the original patient image. In the second step, the registration between high-resolution atlas images and cropped original patient images was performed to obtain the precise segmentation of the heart. In the third step, the registration was performed again by minimizing the dissimilarity within the heart region. Finally, the labels of four cardiac chambers, aorta and pulmonary artery were generated according to the similarity between the deformed atlas images and the patient image. A leave-one-out experiment has been performed on the 20 training datasets of MM-WHS 2017 challenge. The average Dice coefficient between our segmentation results and the manual segmentation results is 0.9051. The mean and standard deviation of Dice coefficients of each structure (i.e. LV, RV, LA, RA, Myo, Ao, PA) are 0.9601 ± 0.0324, 0.9344 ± 0.0418, 0.9594 ± 0.0316, 0.8836 ± 0.0826, 0.8724 ± 0.0707, 0.9295 ± 0.0883, 0.7966 ± 0.1149 respectively.