Statistical shape models for 3D medical image segmentation: A review
Introduction
In the last two decades, model-based segmentation approaches have been established as one of the most successful methods for image analysis. By matching a model which contains information about the expected shape and appearance of the structure of interest to new images, the segmentation is conducted in a top-down fashion. Due to the inherent a-priori information, this approach is more stable against local image artifacts and perturbations than conventional low-level algorithms. While a single template shape is an adequate model for industrial applications where mass-produced, rigid objects need to be detected, this method is prone to fail in case of biological objects due to their considerable natural variability. Information about common variations thus has to be included in the model. A straight-forward approach to gather this information is to examine a number of training shapes by statistical means, leading to statistical shape models (SSMs).
In this paper, we will review methods and procedures for generating, training and employing statistical models of shape and appearance for 3D medical image segmentation. Specifically, we will discuss work on discrete, parametric models which can be trained from a set of example data. Probably the best-known methods in that area are the Active Shape models (Cootes et al., 1995) and Active Appearance models (Cootes et al., 2001) by Cootes et al. In addition, we will discuss related concepts and alternative approaches, all within the context of statistical shape models. Due to the constantly increasing importance of 3D imaging and the urgent need for segmentation in that particular area, we will concentrate on methods for volumetric images. However, many modeling methods have only been applied to 2D so far. To the extent that such methods can be generalized or extended to the 3D case, we have included them in the present review. We have furthermore included some methods which have shown to be very successful in 2D, but which are technically not feasible in 3D – simply to emphasize the difference.
The objective of this article is to provide the reader with a summary of the current state of the art with regard to 3D statistical shape models, to demonstrate what has been done until now, but also to present some ideas of what might yet be done. To ensure comprehensive coverage, we have screened all publications included in IEEE Transactions on Medical Imaging and Medical Image Analysis during the last 10 years for articles related to shape models. In addition, we have included a large number of articles from other international journals, but also numerous conference and workshop papers which present good ideas, but which have not been published in any journal yet. Our main source of references was the Internet; we have searched for the terms shape model and statistical model on PubMed, IEEE-Xplore, Citeseer and Google. We have also followed the references encountered in papers from these sites, until we had collected a comprehensive library of more than 400 articles on the topic. In case we encountered several papers from one author about the same subject, we generally picked the most detailed one for this review.
Before reviewing statistical shape models, let us first define what we regard as related work that will not be discussed further in this article.
Kass et al. started the use of deformable models for image segmentation in their seminal snakes paper (Kass et al., 1988). Their main idea is that model evolution is driven by two energies: an external energy that adapts the model to the image data and an internal energy that stabilizes its shape based on general smoothness constraints. Shortly afterwards, Terzopoulos et al. (1988) generalized the concept (which initially had only been applied to 2D examples) to 3D shapes. In (Delingette et al., 1994), Delingette introduced the deformable simplex mesh, which features a stable internal energy that can easily be customized to deform toward a specific template shape. Using a different approach, McInerney and Terzopoulos (1999) presented a method of how to implement topology changes for deformable surfaces. After almost two decades of deformable models, several review articles on the topic have been published, notably by McInerney and Terzopoulos, 1996, Jain et al., 1998, Montagnat et al., 2001. We disregarded these methods in this review because the underlying deformation algorithms do not incorporate learned constraints of shape variability. Although freely-deformable models can be customized to represent specific shapes (and often are), the stabilizing forces or energies are based on general smoothness properties and are not driven by statistical information.
Level-sets were introduced by Osher and Sethian (1988) and made popular for computer vision and image analysis by Malladi et al. (1995). They feature an implicit shape representation and can be employed with regional or edge-based features. Leventon et al. (2000) extended the original energy formulation by an additional term which deforms the contour towards a previously learned shape model. A frequent criticism is that the signed distance maps which the shape model is based on, do not form a linear space, which can lead to invalid shapes if training samples vary too much. Nevertheless, the approach quickly gained popularity and was extended in several directions, among others by Tsai et al. (2003) who employ Leventon’s modeling method with a region-based energy functional. Recently, Pohl et al. (2006) presented a method of embedding the signed distance maps into the linear LogOdds space, which could solve the modeling problems. To keep this review at a reasonable length, we had to ignore level-set theory and techniques: The conceptual differences between the implicit representation and the discrete models we intend to focus on would have required a special treatment for all following sections. For an overview of statistical approaches to level-set segmentation – including prior shape knowledge – we refer the reader to the recent survey by Cremers et al. (2007).
In order to present a systematic overview of the topic, we have divided this article into several parts, each highlighting a specific aspect of statistical shape models: In Section 2, we will start with presenting different possibilities of how to represent shapes for statistical analysis. Subsequently, we will explain how to extract the principal modes of variation from a set of training shapes in Section 3. A general requirement for this step is that the correspondences between all shapes of the training set are known, a topic which will be discussed in Section 4. After that, in Section 5, we will present techniques to model the appearance of the examined object. The different algorithms that employ shape and appearance models for image analysis and segmentation will be discussed in Section 6. Subsequently, we will present an overview of the areas of application in medical imaging, which have been tackled with three-dimensional SSMs in Section 7. Before concluding the review, we will recapitulate the main points and predict future developments in Section 8.
Section snippets
Shape representation
Training data for SSMs in the medical field will most likely consist of segmented volumetric images. Depending on the segmentation method used, the initial representation might be binary voxel data, fuzzy voxel data (e.g. from probabilistic methods), or surface meshes. Data originating from other sources of acquisition, e.g. surface scanning, might be represented differently. In any case, all shape representations can be converted into each other, and the choice of shape representation is the
Shape model construction
Constructing a statistical shape model basically consists of extracting the mean shape and a number of modes of variation from a collection of training samples. Obviously, the methods employed strongly depend on the chosen shape representation. Due to the dominant role of landmark-based point distribution models, in this section we will concentrate on PDMs and only briefly deal with the corresponding procedures for other representations. An essential requirement for building shape models with
Shape correspondence
Modeling the statistics of a class of shapes requires a set of appropriate training shapes with well-defined correspondences. Depending on the chosen representation, the methods of how to best define these correspondences vary. In any case, establishing dense point correspondences between all shapes of the training set is generally the most challenging part of 3D model construction, and at the same time one of the major factors influencing model quality (the other one being the local gray-value
Appearance models
The majority of published works uses shape models for image segmentation, i.e. after construction the model is fitted to new, previously unseen data. For this purpose, a model of the appearance of the structure of interest is required. Although the first version of shape models simply adapted to the strongest edges in the image (Cootes et al., 1995), the state of the art quickly developed towards specialized, statistical models of appearance. As the shape model, these appearance models have to
Search algorithms
Due to the large size of the search space in 3D, most methods applied to locate an SSM in new image data use local search algorithms that require an initial estimate of the model pose. In the first subsection, we will review several approaches for this initialization, including some global search algorithms that deliver a complete solution for shape and pose. Subsequently, we will deal with the popular Active Shape models and Active Appearance models with some of their variants. In a next step,
Applications
In this section, we will present applications of 3D SSMs for medical image analysis. The focus is on the medical application, i.e. we will only list those papers which state a real-world problem solvable with the approach. Studies where a shape model is built only to demonstrate the feasibility of the modeling approach will not be considered.
Discussion
In the preceding sections, we have reviewed the current state of the art in statistical shape modeling. To conclude our survey, we will recapitulate the main points and make some predictions on future developments in the field.
Acknowledgements
Tobias Heimann was supported by the German Research Foundation DFG under grant WO 1218/2-1. We would like to thank Marleen de Bruijne from the DIKU group at the University of Copenhagen for many helpful comments and suggestions regarding this manuscript. In addition, Klaus Fritzsche, Tobias Schwarz, Ivo Wolf and Sascha Zelzer from our own group have contributed valuable comments.
References (250)
- et al.
Efficient and generalizable statistical models of shape and appearance for analysis of cardiac MRI
Med. Image Anal.
(2008) - et al.
Instantiation and registration of statistical shape models of the femur and pelvis using 3D ultrasound imaging
Med. Image Anal.
(2008) - et al.
Evaluation of image features and search strategies for segmentation of bone structures in radiographs using active shape models
Med. Image Anal.
(2002) Biological shape and visual science
J. Theor. Biol.
(1973)- et al.
Building 3D sulcal models using local geometry
Med. Image Anal.
(2001) - et al.
Combining point distribution models with shape models based on finite-element analysis
Image Vis. Comput.
(1995) - et al.
Active shape models – their training and application
Comput. Vis. Image Und.
(1995) - et al.
Shape statistics in kernel space for variational image segmentation
Pattern Recogn.
(2003) - et al.
Automatic feature localisation with constrained local models
Pattern Recogn.
(2008) - et al.
Automatic shape model building based on principal geodesic analysis bootstrapping
Med. Image Anal.
(2008)