Skip to main content
SpringerLink
Log in

Measures for Benchmarking of Automatic Correspondence Algorithms

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Automatic localisation of correspondences for the construction of Statistical Shape Models from examples has been the focus of intense research during the last decade. Several algorithms are available and benchmarking is needed to rank the different algorithms. Prior work has argued that the quality of the models produced by the algorithms can be evaluated by measuring compactness, generality and specificity. In this paper severe problems with these standard measures are analysed both theoretically and experimentally both on natural and synthetic datasets. We also propose that a Ground Truth Correspondence Measure (GCM) is used for benchmarking and in this paper benchmarking is performed on several state of the art algorithms using seven real and one synthetic dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Baumberg, A., Hogg, D.: Learning flexible models from image sequences. In: Proc. European Conf. on Computer Vision, ECCV’94, pp. 299–308 (1994)

  2. Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24(24), 509–522 (2002)

    Article  Google Scholar 

  3. Benayoun, A., Ayache, N., Cohen, I.: Adaptive meshes and nonrigid motion computation. In: Proc. International Conference on Pattern Recognition, Jerusalem, Israel, pp. 730–732 (1994)

  4. Bookstein, F.: Landmark methods for forms without landmarks: morphometrics of group differences in outline shape. Med. Image Anal. 3, 225–243 (1999)

    Google Scholar 

  5. Chui, H., Rangarajan, A.: A feature registration framework using mixture models. In: IEEE Workshop on Mathematical Methods in Biomedical Image Analysis (MMBIA), pp. 190–197 (2000)

  6. Chui, H., Rangarajan, A.: A new algorithm for non-rigid point matching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. II, pp. 44–51 (2000)

  7. Davies, R.: Learning shape: optimal models for analysing natural variability. PhD thesis, University of Manchester (2002)

  8. Davies, R., Twining, C., Cootes, T., Waterton, J., Taylor, C.: A minimum description length approach to statistical shape modeling. IEEE Trans. Med. Imaging 21(5), 525–537 (2002)

    Article  Google Scholar 

  9. Davies, R.H., Cootes, T.F., Waterton, J.C., Taylor, C.J.: An efficient method for constructing optimal statistical shape models. In: Medical Image Computing and Computer-Assisted Intervention MICCAI’2001, pp. 57–65 (2001)

  10. Davies, R.H., Twining, C.J., Allen, P.D., Cootes, T.F., Taylor, C.J.: Shape discrimination in the hippocampus using an MDL model. In: Information Processing in Medical Imaging (2003)

  11. Dryden, I.L., Mardia, K.V.: Statistical Shape Analysis. Wiley, New York (1998)

    MATH  Google Scholar 

  12. Ericsson, A.: Automatic shape modelling and applications in medical imaging. Technical report, Mathematics LTH, Centre for Mathematical Sciences, Box 118, 22100 Lund, Sweden, November 2003

  13. Ericsson, A.: Automatic shape modelling with applications in medical imaging. PhD thesis, Lund University, Centre for Mathematical Sciences, Box 118, 22100 Lund, Sweden, September 2006

  14. Ericsson, A., Åström, K.: An affine invariant deformable shape representation for general curves. In: Proc. 9th Int. Conf. on Computer Vision, Nice, France, pp. 1142–1149 (2003)

  15. Ericsson, A., Åström, K.: Minimizing the description length using steepest descent. In: Proc. British Machine Vision Conference, Norwich, United Kingdom, vol. 2, pp. 93–102 (2003)

  16. Ericsson, A., Karlsson, J.: Aligning shapes by minimising the description length. In: Proc. Scandinavian Conf. on Image Analysis, SCIA’05, Joensuu, Finland, vol. 3540/2005, pp. 709–718 (2005)

  17. Fisher, R.: Caviar project. Ground truth labelled video sequences (2005), available at http://homepages.inf.ed.ac.uk/rbf/CAVIAR/

  18. Gower, J.: Generalized procrustes analysis. Psychometrica 40, 33–50 (1975)

    Article  MATH  MathSciNet  Google Scholar 

  19. Hill, A., Taylor, C.: Automatic landmark generation for point distribution models. In: Proc. British Machine Vision Conference, pp. 429–438 (1994)

  20. Hill, A., Taylor, C.: A framework for automatic landmark indentification using a new method of nonrigid correspondence. IEEE Trans. Pattern Anal. Mach. Intell. 22, 241–251 (2000)

    Article  Google Scholar 

  21. Kambhamettu, C., Goldgof, D.: Points correspondences recovery in non-rigid motion. In: Proc. Conf. Computer Vision and Pattern Recognition, CVPR’92, pp. 222–237 (1992)

  22. Karlsson, J., Ericsson, A., Åström, K.: Parameterisation invariant statistical shape models. In: Proc. International Conference on Pattern Recognition, Cambridge, UK (2004)

  23. Kelemen, A., Szekely, G., Gerig, G.: Elastic model-based segmentation of 3D neuroradiological data sets. IEEE Trans. Med. Imaging 18(10), 828–839 (1999)

    Article  Google Scholar 

  24. Kotcheff, A., Taylor, C.: Automatic construction of eigenshape models by direct optimization. Med. Image Anal. 2, 303–314 (1998)

    Article  Google Scholar 

  25. Mao, Z., Ju, X., Siebert, J., Cockshott, W., Ayoub, A.: Constructing dense correspondences for the analysis of 3D facial morphology. Pattern Recognit. Lett. 27(6), 597–608 (2006)

    Article  Google Scholar 

  26. Martin, D.R., Fowlkes, C.C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans. Pattern Anal. Mach. Intell. 26(5), 530–549 (2004)

    Article  Google Scholar 

  27. Richter, J., Ericsson, A., Åström, K., Kahl, F., Edenbrant, L.: Automated interpretation of cardiac scintigrams. In: Proc. 13th Scandinavian Conf. on Image Analysis, Gothenburg, Sweden (2003)

  28. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1/2/3), 7–42 (2002)

    Article  MATH  Google Scholar 

  29. Schestowitz, R., Twining, C., Cootes, T., Petrović, V., Taylor, C., Crum, B.: Assessing the accuracy of non-rigid registration with and without ground truth. In: Proc. IEEE International Symposium on Biomedical Imaging (2006)

  30. Sebastian, T., Klein, P., Kimia, B.: Constructing 2D curve atlases. In: IEEE Workshop on Mathematical Methods in Biomedical Image Analysis, pp. 70–77 (2000)

  31. Sebastian, T., Klein, P., Kimia, B.: On aligning curves. IEEE Trans. Pattern Anal. Mach. Intell. 25(1), 116–125 (2003)

    Article  Google Scholar 

  32. Sharvit, D., Chan, J., Tek, H., Kimia, B.: Symmetry-based indexing of image databases. J. Vis. Commun. Image Represent. 4(4), 366–380 (1998)

    Article  Google Scholar 

  33. Sternby, J., Ericsson, A.: Core points—a framework for structural parameterization. In: Proc. Internation Conference on Document Analysis and Recognition, ICDAR’05, Seoul, Korea (2005)

  34. Styner, M., Rajamani, K., Nolte, L., Zsemlye, G., Szekely, G., Taylor, C., Davies, R.H.: Evaluation of 3D correspondence methods for model building. In: Information Processing in Medical Imaging (IPMI), pp. 63–75 (2003)

  35. Tagare, H.: Shape-based nonrigid correspondence with application to heart motion analysis. IEEE Trans. Med. Imaging 18, 570–579 (1999)

    Article  Google Scholar 

  36. Thodberg, H.H.: Minimum description length shape and appearance models. In: Image Processing Medical Imaging, IPMI (2003)

  37. Thodberg, H.H., Olafsdottir, H.: Adding curvature to minimum description length shape models. In: Proc. British Machine Vision Conference (2003)

  38. Twining, C., Taylor, C.: Specificity as a graph-based estimator of cross-entropy. In: Proc. British Machine Vision Conference, Edinburgh, United Kingdom, vol. 2, pp. 459–468 (2006)

  39. Wang, Y., Peterson, B., Staib, L.: Shape-based 3D surface correspondence using geodesics and local geometry. In: Proc. Conf. Computer Vision and Pattern Recognition, CVPR’00, pp. 644–651 (2000)

  40. Zheng, Y., Doermann, D.: Robust point matching for non-rigid shapes: a relaxation labeling based approach. Technical Report: Lamp-tr-117, University of Maryland, College Park (2004)

Download references

Author information

Authors and Affiliations

  1. Centre for Mathematical Sciences, Lund University, Lund, Sweden

    Anders Ericsson & Johan Karlsson

Authors
  1. Anders Ericsson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Johan Karlsson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Johan Karlsson.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ericsson, A., Karlsson, J. Measures for Benchmarking of Automatic Correspondence Algorithms. J Math Imaging Vis 28, 225–241 (2007). https://doi.org/10.1007/s10851-007-0018-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-007-0018-5

Keywords

Search

Navigation