Statistical 4D graphs for multi-organ abdominal segmentation from multiphase CT

Abstract

The interpretation of medical images benefits from anatomical and physiological priors to optimize computer-aided diagnosis applications. Diagnosis also relies on the comprehensive analysis of multiple organs and quantitative measures of soft tissue. An automated method optimized for medical image data is presented for the simultaneous segmentation of four abdominal organs from 4D CT data using graph cuts. Contrast-enhanced CT scans were obtained at two phases: non-contrast and portal venous. Intra-patient data were spatially normalized by non-linear registration. Then 4D convolution using population training information of contrast-enhanced liver, spleen and kidneys was applied to multiphase data to initialize the 4D graph and adapt to patient-specific data. CT enhancement information and constraints on shape, from Parzen windows, and location, from a probabilistic atlas, were input into a new formulation of a 4D graph. Comparative results demonstrate the effects of appearance, enhancement, shape and location on organ segmentation. All four abdominal organs were segmented robustly and accurately with volume overlaps over 93.6% and average surface distances below 1.1 mm.

Introduction

In CT-based clinical abdominal diagnosis, radiologists rely on analyzing multiphase computed tomography (CT) data, as soft tissue enhancement can be an indicator of abnormality. Contrast-enhanced CT has proven exceptionally useful to improving diagnosis due to the ability to differentiate tumors from healthy tissue. For instance, the level of enhancement in the tumor is an important indication of malignancy and can be used to better classify abdominal abnormalities (Fritz et al., 2006, Voci et al., 2000). This routine clinical acquisition protocol makes multiphase data (with/without contrast) readily available.

Diagnosis also relies on the comprehensive analysis of groups of organs and quantitative measures of soft tissue, as the volumes and shapes of organs can be indicators of disorders. When presented three-dimensional (3D) patient data, such as CT, radiologists typically analyze them organ-by-organ and slice-by-slice until the entire image data are covered. This allows detecting multi-diseases from multi-organs.

Computer-aided diagnosis (CAD) and medical image analysis traditionally focus on organ or disease-based applications. However there is a strong incentive to migrate toward the automated simultaneous segmentation and analysis of multiple organs for comprehensive diagnosis or pre-operative planning and guidance. Additionally, the interpretation of medical images should benefit from anatomical and physiological priors, such as shape and appearance; synergistic combinations of priors were seldom incorporated in the optimization of CAD.

A variety of methods have been proposed for the segmentation of individual abdominal organs from CT images, in particular CT after contrast enhancement. The liver enjoyed special attention in recent literature (Delingette and Ayache, 2005, Heimann et al., 2009, Linguraru et al., 2010, Okada et al., 2008a, Soler et al., 2001, Song et al., 2009, Wimmer et al., 2009), kidneys were analyzed sporadically (Ali et al., 2007, Shim et al., 2009, Spiegel et al., 2009), while the spleen (Danelson and Stitzel, 2008, Linguraru et al., 2010) and pancreas (Shimizu et al., 2010a) were segmented less frequently. Model driven approaches have been both popular and successful (Soler et al., 2001, Song et al., 2009), including active and statistical shape models (Okada et al., 2008a, So and Chung, 2009, Wimmer et al., 2009) and atlas-based segmentation (Linguraru et al., 2010, Okada et al., 2008a, Shimizu et al., 2010a). Level sets and geodesic active contours were frequently involved in these techniques (Heimann et al., 2009, Linguraru et al., 2010, Wimmer et al., 2009). Occasionally, graph cuts were employed (Ali et al., 2007, Shim et al., 2009).

Recently, the simultaneous segmentation of multiple abdominal organs has been addressed in publications (Linguraru and Summers, 2008, Okada et al., 2008b, Park et al., 2003, Seifert et al., 2009, Shimizu et al., 2007, Shimizu et al., 2010a). Most of these methods rely on some form of prior knowledge of the organs, for example probabilistic atlases (Park et al., 2003, Reyes et al., 2009, Shimizu et al., 2007) and statistical models (Okada et al., 2008b), which are sensitive to initialization/registration. An initial segmentation is typically achieved and subsequently refined. The relation between organs and manual landmarks was used in Park et al. (2003). An efficient optimization of level sets techniques for general multi-class segmentation was proposed in Bae and Tai (2009), paving the way for the discrete optimization of graph cuts with nonsubmodular functions in El-Zehiry and Grady (2011).

Notably, a hierarchical multi-organ statistical atlas was developed by Okada et al. (2008b). The analysis was restricted to the liver area due to the large variations to be statistically modeled for inter-organ relationships. Also recently, Seifert et al. (2009) proposed a semantic navigation for fast multi-organ segmentation from CT data. The method estimated the organ location, orientation and size using automatically detected anatomical landmarks and machine learning techniques. Decision forests were additionally proposed in Montillo et al. (2011) to classify multiple organs from CT volumes. The method achieved high prediction accuracy and was fast, but its segmentation overlap was low. Another interesting concept was presented in Zhan et al. (2008) for the scheduling problem of multi-organ segmentation to maximize the performance of CAD systems designed to analyze the whole human body.

In addition, multiphase contrast-enhanced CT data were employed in abdominal multi-organ analysis (Hu et al., 2004, Linguraru and Summers, 2008, Sadananthan et al., 2010). In Hu et al. (2004), the segmentation was based on independent component analysis in a variational Bayesian mixture, while in Sakashita et al. (2007), expectation–maximization and principal component analysis were combined. A 4D convolution was proposed in Linguraru and Summers (2008) constrained by a training model of abdominal soft tissue enhancement. These intensity-based methods are hampered by the high variability of abdominal intensity and texture.

Graph cuts (Boykov and Jolly, 2001) have become popular for image segmentation due to their ability to handle highly textured data via a numerically robust global optimization. The segmentation uses hard constraints from user defined areas of “object” and “background” and additional soft constraints from boundaries and region information. The value of the method was immediate for medical data (the segmentation of bone from CT and kidney from magnetic resonance imaging – MRI) and video sequences (2D + time; Boykov and Jolly, 2001). Graph cuts were also used to track objects from occluding scenes in Khan and Shah (2009).

To reduce the sensitivity to initialization, global geodesics were computed via graph cuts (Boykov and Kolmogorov, 2003) and used to segment the liver, lung and heart. This method imposed length/area constraints for object boundaries and relied on consistent edge weights to obtain geometric properties. While introducing the theoretical advantages of graph cuts, there was no validation of medical data segmentation provided in Boykov and Jolly, 2001, Boykov and Kolmogorov, 2003.

Combining the length/area concept with the computation of flux, the geometric interpretation can be seen as a shape prior in the construction of the graph (Kolmogorov and Boykov, 2005, Vu and Manjunath, 2008). Multiple objects can be consecutively segmented (Vu and Manjunath, 2008). The shape model was implemented as a density estimation for shape priors initially proposed for level sets in Cremers et al. (2006), but a symmetric shape distance can be biased if the initialization is poor. A multi-region segmentation via graph-cuts was recently proposed by Delong and Boykov (2009) with separate appearance models for each region. Their approach uses distance priors between regions, but no explicit shape priors, and was not quantitatively validated.

The introduction of shape into graph cuts has been an area of active research. Compact shape priors were used in Das et al. (2008), but medical data often involve complex shapes. A star shape descriptor was introduced in Veksler (2008), but only shapes complying with a generic star shape were extracted. Shape priors were embedded into the weights on the edges in the graph by using a level-set formulation in Freedman and Zhang (2005), but this interactive method was robust only to small shape variations. Finally, a kernel principle component analysis was used to learn a statistical model of relevant shapes in Malcolm et al. (2007) in a Bayesian formulation to perform segmentation via graph cuts in four natural images.

All the above graph cut techniques suffer from the manual initialization of the segmentation.

Following the theoretical advances of graph cuts techniques, several medical image analysis applications have been proposed. The validation of these applications is generally more thorough.

An automated graph cut technique was used in García-Lorenzo et al. (2009) to segment multiple sclerosis lesions from MRI of the brain using an expectation maximization initialization. Statistical models of intensity and spatial distribution from MRI data were registered and used to construct a graph for the segmentation of the hippocampus in van der Lijn et al., 2008, Lötjönen et al., 2010. An interesting example of using graph-cuts to solve non-rigid registration for brain MRI was presented in So and Chung (2009). In general, registration methods are more robust on brain MRI data than the abdomen, which has higher shape and appearance variability.

Brain tumors were automatically segmented from MRI using integrated probabilistic boosting trees into graph cuts to handle intra-patient intensity heterogeneity (Wels et al., 2008). Graph cuts were also used to refine the manual segmentation of breast tumors from MRI data (Zheng et al., 2007). The skull was accurately removed from MRI images in Sadananthan et al. (2010) using intensity thresholding for initialization. Then foot bones were segmented interactively from CT in Liu et al. (2008). Using an acquisition protocol for plaque reconstruction, carotid plaques were segmented semi-automatically from ultrasound images in Seabra et al. (2009).

In Ali et al., 2007, Ben Ayed et al., 2009, Chen and Shapiro, 2008, Lin et al., 2005 model-based information was included for the segmentation of heart, spleen and kidneys. The models were aligned using markers in Ali et al., 2007, Lin et al., 2005, manual placements in axial slices in Chen and Shapiro (2008) and intra-model constraints given in the first frame of the cardiac cycle in Ben Ayed et al. (2009). Shape priors were employed in Bauer et al., 2010, Esneault et al., 2010 to reconstruct the liver vasculature and lung airways; the cuts in the graph were constrained by a tubular filter. Probabilistic shape-based energies for graph-cuts were combined with image intensity in a non-parametric iterative model in Freiman et al. (2010) for the segmentation of the kidneys. Also, in Shimizu et al. (2010b), shape priors and neighboring constraints were incorporated using signed distances from boundaries to segment the liver.

In other types of biomedical applications, a multi-level automated graph-cut algorithm was used in Al-Kofahi et al. (2009) to segment cell nuclei; the seed points were detected by a Laplacian-of-Gaussian filter in a method designed for histopathology data. A graph cuts optimization was presented in Deleus and van Hulle (2009) for the parcellation of the brain from functional MRI. In Gramfort et al. (2010), a data-driven graph approach was implemented to estimate the variability of neural responses on magnetoencephalography or electroencephalography data. Finally, a study of the effect of weights and topology on the construction of graphs can be found in Grady and Jolly (2008).

Abdominal multi-organ segmentation remains a challenging task because the sizes, shapes and locations of the organs vary significantly in different subjects. Moreover, these organs have similar appearance in CT images, even in contrast-enhanced data, and are in close proximity to each other.

An advantage when handling medical data is the available prior information regarding organ location, shape and appearance. Although highly variable between patients and in the presence of disease, abdominal organs satisfy basic rules of anatomy and physiology. Hence, the incorporation of statistical models into algorithms for medical data analysis greatly benefits the segmentation of abdominal images. For example, the enhancement of soft tissue in CT images is not only a marker of disease, but also an indicator of tissue or organ type, as contrast agent intake is tissue specific. As presented in the previous sections, certain levels of model-based information have been included in abdominal segmentation and to a reduced extent into graph cuts. They generally suffer from manual initialization and do not address multi-organ segmentation.

An integrated statistical model for medical data is introduced in this paper and incorporated into a graph-based approach. We propose a new formulation of a 4D directional graph to automatically segment abdominal organs, at this stage the liver, spleen and left and right kidneys using graph cuts. The statistical priors comprise location probabilities that are intrinsic to medical data, an enhancement constraint characteristic to the clinical protocols using abdominal CT and an unbiased asymmetric shape measure. The method is optimized globally and starts with training (entire patient population) 4D intensity data to automatically initialize the graph, then migrating to patient specific information for better specificity. Comparative results at different stages of the algorithm show the effects of appearance, shape and location on the accuracy of organ segmentation.