Elsevier

Medical Image Analysis

Volume 13, Issue 4, August 2009, Pages 543-563
Medical Image Analysis

Statistical shape models for 3D medical image segmentation: A review

https://doi.org/10.1016/j.media.2009.05.004Get rights and content

Abstract

Statistical shape models (SSMs) have by now been firmly established as a robust tool for segmentation of medical images. While 2D models have been in use since the early 1990s, wide-spread utilization of three-dimensional models appeared only in recent years, primarily made possible by breakthroughs in automatic detection of shape correspondences. In this article, we review the techniques required to create and employ these 3D SSMs. While we concentrate on landmark-based shape representations and thoroughly examine the most popular variants of Active Shape and Active Appearance models, we also describe several alternative approaches to statistical shape modeling. Structured into the topics of shape representation, model construction, shape correspondence, local appearance models and search algorithms, we present an overview of the current state of the art in the field. We conclude with a survey of applications in the medical field and a discussion of future developments.

Introduction

In the last two decades, model-based segmentation approaches have been established as one of the most successful methods for image analysis. By matching a model which contains information about the expected shape and appearance of the structure of interest to new images, the segmentation is conducted in a top-down fashion. Due to the inherent a-priori information, this approach is more stable against local image artifacts and perturbations than conventional low-level algorithms. While a single template shape is an adequate model for industrial applications where mass-produced, rigid objects need to be detected, this method is prone to fail in case of biological objects due to their considerable natural variability. Information about common variations thus has to be included in the model. A straight-forward approach to gather this information is to examine a number of training shapes by statistical means, leading to statistical shape models (SSMs).

In this paper, we will review methods and procedures for generating, training and employing statistical models of shape and appearance for 3D medical image segmentation. Specifically, we will discuss work on discrete, parametric models which can be trained from a set of example data. Probably the best-known methods in that area are the Active Shape models (Cootes et al., 1995) and Active Appearance models (Cootes et al., 2001) by Cootes et al. In addition, we will discuss related concepts and alternative approaches, all within the context of statistical shape models. Due to the constantly increasing importance of 3D imaging and the urgent need for segmentation in that particular area, we will concentrate on methods for volumetric images. However, many modeling methods have only been applied to 2D so far. To the extent that such methods can be generalized or extended to the 3D case, we have included them in the present review. We have furthermore included some methods which have shown to be very successful in 2D, but which are technically not feasible in 3D – simply to emphasize the difference.

The objective of this article is to provide the reader with a summary of the current state of the art with regard to 3D statistical shape models, to demonstrate what has been done until now, but also to present some ideas of what might yet be done. To ensure comprehensive coverage, we have screened all publications included in IEEE Transactions on Medical Imaging and Medical Image Analysis during the last 10 years for articles related to shape models. In addition, we have included a large number of articles from other international journals, but also numerous conference and workshop papers which present good ideas, but which have not been published in any journal yet. Our main source of references was the Internet; we have searched for the terms shape model and statistical model on PubMed, IEEE-Xplore, Citeseer and Google. We have also followed the references encountered in papers from these sites, until we had collected a comprehensive library of more than 400 articles on the topic. In case we encountered several papers from one author about the same subject, we generally picked the most detailed one for this review.

Before reviewing statistical shape models, let us first define what we regard as related work that will not be discussed further in this article.

Kass et al. started the use of deformable models for image segmentation in their seminal snakes paper (Kass et al., 1988). Their main idea is that model evolution is driven by two energies: an external energy that adapts the model to the image data and an internal energy that stabilizes its shape based on general smoothness constraints. Shortly afterwards, Terzopoulos et al. (1988) generalized the concept (which initially had only been applied to 2D examples) to 3D shapes. In (Delingette et al., 1994), Delingette introduced the deformable simplex mesh, which features a stable internal energy that can easily be customized to deform toward a specific template shape. Using a different approach, McInerney and Terzopoulos (1999) presented a method of how to implement topology changes for deformable surfaces. After almost two decades of deformable models, several review articles on the topic have been published, notably by McInerney and Terzopoulos, 1996, Jain et al., 1998, Montagnat et al., 2001. We disregarded these methods in this review because the underlying deformation algorithms do not incorporate learned constraints of shape variability. Although freely-deformable models can be customized to represent specific shapes (and often are), the stabilizing forces or energies are based on general smoothness properties and are not driven by statistical information.

Level-sets were introduced by Osher and Sethian (1988) and made popular for computer vision and image analysis by Malladi et al. (1995). They feature an implicit shape representation and can be employed with regional or edge-based features. Leventon et al. (2000) extended the original energy formulation by an additional term which deforms the contour towards a previously learned shape model. A frequent criticism is that the signed distance maps which the shape model is based on, do not form a linear space, which can lead to invalid shapes if training samples vary too much. Nevertheless, the approach quickly gained popularity and was extended in several directions, among others by Tsai et al. (2003) who employ Leventon’s modeling method with a region-based energy functional. Recently, Pohl et al. (2006) presented a method of embedding the signed distance maps into the linear LogOdds space, which could solve the modeling problems. To keep this review at a reasonable length, we had to ignore level-set theory and techniques: The conceptual differences between the implicit representation and the discrete models we intend to focus on would have required a special treatment for all following sections. For an overview of statistical approaches to level-set segmentation – including prior shape knowledge – we refer the reader to the recent survey by Cremers et al. (2007).

In order to present a systematic overview of the topic, we have divided this article into several parts, each highlighting a specific aspect of statistical shape models: In Section 2, we will start with presenting different possibilities of how to represent shapes for statistical analysis. Subsequently, we will explain how to extract the principal modes of variation from a set of training shapes in Section 3. A general requirement for this step is that the correspondences between all shapes of the training set are known, a topic which will be discussed in Section 4. After that, in Section 5, we will present techniques to model the appearance of the examined object. The different algorithms that employ shape and appearance models for image analysis and segmentation will be discussed in Section 6. Subsequently, we will present an overview of the areas of application in medical imaging, which have been tackled with three-dimensional SSMs in Section 7. Before concluding the review, we will recapitulate the main points and predict future developments in Section 8.

Section snippets

Shape representation

Training data for SSMs in the medical field will most likely consist of segmented volumetric images. Depending on the segmentation method used, the initial representation might be binary voxel data, fuzzy voxel data (e.g. from probabilistic methods), or surface meshes. Data originating from other sources of acquisition, e.g. surface scanning, might be represented differently. In any case, all shape representations can be converted into each other, and the choice of shape representation is the

Shape model construction

Constructing a statistical shape model basically consists of extracting the mean shape and a number of modes of variation from a collection of training samples. Obviously, the methods employed strongly depend on the chosen shape representation. Due to the dominant role of landmark-based point distribution models, in this section we will concentrate on PDMs and only briefly deal with the corresponding procedures for other representations. An essential requirement for building shape models with

Shape correspondence

Modeling the statistics of a class of shapes requires a set of appropriate training shapes with well-defined correspondences. Depending on the chosen representation, the methods of how to best define these correspondences vary. In any case, establishing dense point correspondences between all shapes of the training set is generally the most challenging part of 3D model construction, and at the same time one of the major factors influencing model quality (the other one being the local gray-value

Appearance models

The majority of published works uses shape models for image segmentation, i.e. after construction the model is fitted to new, previously unseen data. For this purpose, a model of the appearance of the structure of interest is required. Although the first version of shape models simply adapted to the strongest edges in the image (Cootes et al., 1995), the state of the art quickly developed towards specialized, statistical models of appearance. As the shape model, these appearance models have to

Search algorithms

Due to the large size of the search space in 3D, most methods applied to locate an SSM in new image data use local search algorithms that require an initial estimate of the model pose. In the first subsection, we will review several approaches for this initialization, including some global search algorithms that deliver a complete solution for shape and pose. Subsequently, we will deal with the popular Active Shape models and Active Appearance models with some of their variants. In a next step,

Applications

In this section, we will present applications of 3D SSMs for medical image analysis. The focus is on the medical application, i.e. we will only list those papers which state a real-world problem solvable with the approach. Studies where a shape model is built only to demonstrate the feasibility of the modeling approach will not be considered.

Discussion

In the preceding sections, we have reviewed the current state of the art in statistical shape modeling. To conclude our survey, we will recapitulate the main points and make some predictions on future developments in the field.

Acknowledgements

Tobias Heimann was supported by the German Research Foundation DFG under grant WO 1218/2-1. We would like to thank Marleen de Bruijne from the DIKU group at the University of Copenhagen for many helpful comments and suggestions regarding this manuscript. In addition, Klaus Fritzsche, Tobias Schwarz, Ivo Wolf and Sascha Zelzer from our own group have contributed valuable comments.

References (250)

  • R.H. Davies et al.

    Building optimal 2D statistical shape models

    Image Vis. Comput.

    (2003)
  • M. de Bruijne et al.

    Interactive segmentation of abdominal aortic aneurysms in CTA images

    Med. Image Anal.

    (2004)
  • S. Duchesne et al.

    Appearance-based segmentation of medial temporal lobe structures

    NeuroImage

    (2002)
  • L. Ferrarini et al.

    GAMEs: Growing and adaptive meshes for fully automatic shape modeling and analysis

    Med. Image Anal.

    (2007)
  • M. Fleute et al.

    Incorporating a statistically based shape model into a system for computer-assisted anterior cruciate ligament surgery

    Med. Image Anal.

    (1999)
  • Y. Freund et al.

    A decision-theoretic generalization of on-line learning and an application to boosting

    J. Comput. Syst. Sci.

    (1997)
  • S. Gold et al.

    New algorithms for 2D and 3D point matching: Pose estimation and correspondence

    Pattern Recogn.

    (1998)
  • P. Golland et al.

    Detection, analysis of statistical differences in anatomical shape

    Med. Image Anal.

    (2005)
  • D.J. Hawkes et al.

    Tissue deformation and shape models in image-guided interventions: a discussion paper

    Med. Image Anal.

    (2005)
  • P.R. Andresen et al.

    Surface-bounded growth modeling applied to human mandibles

    IEEE Trans. Med. Imaging

    (2000)
  • Bailleul, J., Ruan, S., Bloyet, D., Romaniuk, B., 2004. Segmentation of anatomical structures from 3D brain MRI using...
  • Baker, S., Matthews, I., 2001. Equivalence and efficiency of image alignment algorithms. In: Proc. IEEE CVPR, vol....
  • R. Bansal et al.

    Statistical analyses of brain surfaces using gaussian random fields on 2-D manifolds

    IEEE Trans. Med. Imaging

    (2007)
  • V. Barnett et al.

    Outliers in Statistical Data

    (1994)
  • R. Beichel et al.

    Robust active appearance models and their application to medical image analysis

    IEEE Trans. Med. Imaging

    (2005)
  • Beichel, R., Gotschuli, G., Sorantin, E., Leberl, F., Sonka, M., 2002. Diaphragm dome surface segmentation in CT data...
  • P.J. Besl et al.

    A method for registration of 3-D shapes

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1992)
  • J.M. Blackall et al.

    A statistical model of respiratory motion and deformation of the liver

  • F.L. Bookstein

    Morphometric Tools for Landmark Data

    (2003)
  • J. Bosch et al.

    Automatic segmentation of echocardiographic sequences by active appearance motion models

    IEEE Trans. Med. Imaging

    (2002)
  • M. Brejl et al.

    Object localization and border detection criteria design in edge-based image segmentation: automated learning from examples

    IEEE Trans. Med. Imaging

    (2000)
  • A.D. Brett et al.

    A method of automated landmark generation for automated 3D PDM construction

    Image Vis. Comput.

    (1999)
  • Brett, A.D., Taylor, C.J., 2000. Automated construction of 3D shape models using harmonic maps. In: Proc. Medical Image...
  • Broadhurst, R., Stough, J., Pizer, S., Chaney, E., 2006. A statistical appearance model based on intensity quantile...
  • Broadhurst, R.E., Stough, J., Pizer, S.M., Chaney, E.L., 2005. Histogram statistics of local model-relative image...
  • J. Cates et al.

    Shape modeling and analysis with entropy-based particle systems

  • C.S.K. Chan et al.

    Integration of ultrasound based registration with statistical shape models for computer assisted orthopaedic surgery

  • Cootes, T.F., Edwards, G.J., Taylor, C.J., 1998. Active appearance models. In: H. Burkhardt, B. Neumann (Eds.), Proc....
  • T.F. Cootes et al.

    A comparative evaluation of active appearance model algorithms

  • T.F. Cootes et al.

    Active appearance models

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2001)
  • T.F. Cootes et al.

    Comparing variations on the active appearance model algorithm

  • T.F. Cootes et al.

    Active shape model search using local grey-level models: A quantitative evaluation

  • Cootes, T.F., Taylor, C.J., 1994. Using grey-level models to improve active shape model search. In: Proc. ICPR, vol....
  • T.F. Cootes et al.

    Data driven refinement of active shape model search

  • T.F. Cootes et al.

    A mixture model for representing shape variation

    Image Vis. Comput.

    (1999)
  • Cootes, T.F., Taylor, C.J., 2001. Constrained active appearance models. In: Proc. IEEE ICCV, vol....
  • Cootes, T.F., Taylor, C.J., 2001. On representing edge structure for model matching. In: Proc. IEEE CVPR, vol....
  • Cootes, T.F., Taylor, C.J., 2004. Statistical models of appearance for computer vision. Tech. rep., University of...
  • T.F. Cootes et al.

    Training models of shape from sets of examples

  • T.F. Cootes et al.

    Active shape models: Evaluation of a multi-resolution method for improving search

  • Cited by (0)

    View full text