Skip to main content
Log in

A Variational Framework for Region-Based Segmentation Incorporating Physical Noise Models

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Image segmentation is one of the fundamental problems in computer vision and image processing. In the recent years mathematical models based on partial differential equations and variational methods have led to superior results in many applications, e.g., medical imaging. A majority of works on image segmentation implicitly assume the given image to be biased by additive Gaussian noise, for instance the popular Mumford-Shah model. Since this assumption is not suitable for a variety of problems, we propose a region-based variational segmentation framework to segment also images with non-Gaussian noise models. Motivated by applications in biomedical imaging, we discuss the cases of Poisson and multiplicative speckle noise intensively. Analytical results such as the existence of a solution are verified and we investigate the use of different regularization functionals to provide a-priori information regarding the expected solution. The performance of the proposed framework is illustrated by experimental results on synthetic and real data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Acar, R., Vogel, C.R.: Analysis of bounded variation penalty methods for ill-posed problems. Inverse Probl. 10, 1217–1229 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  2. Adams, R.A.: Sobolev Spaces. Pure and Applied Mathematics, vol. 65. Academic Press, New York (1975)

    MATH  Google Scholar 

  3. Adams, R.A., Fournier, J.J.F.: Sobolev Spaces. Pure and Applied Mathematics, vol. 140. Elsevier, Amsterdam (2003)

    MATH  Google Scholar 

  4. Ambrosio, L., Tortorelli, V.M.: Approximation of functionals depending on jumps by elliptic functionals via Γ-convergence. Commun. Pure Appl. Math. 43, 999–1036 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  5. Ambrosio, L., Fusco, N., Pallara, D.: Functions of Bounded Variation and Free Discontinuity Problems. Oxford Mathematical Monographs. Oxford University Press, Oxford (2000)

    MATH  Google Scholar 

  6. Aubert, G., Aujol, J.-F.: A variational approach to removing multiplicative noise. SIAM J. Appl. Math. 68, 925–946 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  7. Aubert, G., Kornprobst, P.: Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, 2nd edn. Applied Mathematical Sciences, vol. 147. Springer, Berlin (2006)

    Google Scholar 

  8. Aujol, J.-F.: Some first-order algorithms for total variation based image restoration. J. Math. Imaging Vis. 34, 307–327 (2009)

    Article  MathSciNet  Google Scholar 

  9. Barbu, V., Precupanu, T.: Convexity and Optimization in Banach Spaces. Sijthoff & Noordhoff, Rockville (1978)

    Google Scholar 

  10. Benning, M., Kösters, T., Wübbeling, F., Schäfers, K., Burger, M.: A nonlinear variational method for improved quantification of myocardial blood flow using dynamic \(\mathrm{H}_{2} ^{15}\)O PET. In: Nuclear Science Symposium Conference Record, pp. 4472–4477 (2008)

    Google Scholar 

  11. Bertero, M., Lanteri, H., Zanni, L.: Iterative image reconstruction: a point of view. In: Censor, Y., Jiang, M., Louis, A. (eds.) Mathematical Methods in Biomedical Imaging and Intensity-Modulated Radiation Therapy (IMRT). Publications of the Scuola Normale, CRM Series, vol. 7, pp. 37–63 (2008)

    Google Scholar 

  12. Bresson, X., Esedoglu, S., Vandergheynst, P., Thiran, J.-P., Osher, S.: Fast global minimization of the active contour/snake model. J. Math. Imaging Vis. 28, 151–167 (2007)

    Article  MathSciNet  Google Scholar 

  13. Brox, T., Weickert, J.: Level set segmentation with multiple regions. IEEE Trans. Image Process. 15, 3213–3218 (2006)

    Article  Google Scholar 

  14. Burger, M., Franek, M., Schönlieb, C.-B.: Regularized regression and density estimation based on optimal transport. Appl. Math. Res. Express 2012 (2012), 45 pp.

  15. Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. Int. J. Comput. Vis. 22, 61–79 (1997)

    Article  MATH  Google Scholar 

  16. Caselles, V., Chambolle, A., Novaga, M.: The discontinuity set of solutions of the TV denoising problem and some extensions. Multiscale Model. Simul. 6, 879–894 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  17. Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20, 89–97 (2004)

    Article  MathSciNet  Google Scholar 

  18. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40, 120–145 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  19. Chan, T.F., Shen, J.: Image Processing and Analysis: Variational, PDE, Wavelet and Stochastic Methods. SIAM, Philadelphia (2005)

    Book  Google Scholar 

  20. Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10, 266–277 (2001)

    Article  MATH  Google Scholar 

  21. Chan, T.F., Golub, G.H., Mulet, P.: A nonlinear primal-dual method for total variation-based image restoration. SIAM J. Sci. Comput. 20, 1964–1977 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  22. Chan, T.F., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM J. Appl. Math. 66, 1632–1648 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  23. Chesnaud, C., Réfrégier, P., Boulet, V.: Statistical region snake-based segmentation adapted to different physical noise models. IEEE Trans. Pattern Anal. Mach. Intell. 21, 1145–1157 (1999)

    Article  Google Scholar 

  24. Chung, G., Vese, L.A.: Energy minimization based segmentation and denoising using a multilayer level set approach. In: Proceedings of the 5th International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition. LNCS, vol. 3757, pp. 439–455. Springer, Berlin (2005)

    Chapter  Google Scholar 

  25. Cremers, D., Rousson, M., Deriche, R.: A review of statistical approaches to level set segmentation: integrating color, texture, motion and shape. Int. J. Comput. Vis. 72, 195–215 (2007)

    Article  Google Scholar 

  26. Cremers, D., Pock, T., Kolev, K., Chambolle, A.: Convex relaxation techniques for segmentation, stereo, and multiview reconstruction. In: Markov Random Fields for Vision and Image Processing. MIT Press, New York (2011)

    Google Scholar 

  27. Dey, N., Blanc-Féraud, L., Zimmer, C., Roux, P., Kam, Z., Olivio-Marin, J.-C., Zerubia, J.: 3D microscopy deconvolution using Richardson-Lucy algorithm with total variation regularization. Tech. Rep. 5272, Institut National de Recherche en Informatique et en Automatique (2004)

  28. Ekeland, I., Temam, R.: Convex Analysis and Variational Problems. Studies in Mathematics and Its Applications, vol. 1. North-Holland, Amsterdam (1976)

    MATH  Google Scholar 

  29. Elman, H.C., Golub, G.H.: Inexact and preconditioned Uzawa algorithms for saddle point problems. SIAM J. Numer. Anal. 31, 1645–1661 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  30. Fortin, M., Glowinski, R.: Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems. Studies in Mathematics and Its Applications, vol. 15. Elsevier, Amsterdam (1983)

    MATH  Google Scholar 

  31. Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. J. Appl. Stat. 20, 25–62 (1993)

    Article  Google Scholar 

  32. Geman, S., McClure, D.E.: Bayesian image analysis: an application to single photon emission tomography. In: Statistical Computation Section, American Statistical Association, pp. 12–18 (1985)

    Google Scholar 

  33. Ghanem, A., et al.: Triggered replenishment imaging reduces variability of quantitative myocardial contrast echocardiography and allows assessment of myocardial blood flow reserve. Echocardiography 24, 149–158 (2007)

    Article  Google Scholar 

  34. Gianazza, U., Savaré, G., Toscani, G.: The Wasserstein gradient flow of the Fisher information and the quantum drift-diffusion equation. Arch. Ration. Mech. Anal. 194, 133–220 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  35. Giusti, E.: Minimal Surfaces and Functions of Bounded Variation. Monographs in Mathematics, vol. 80. Birkhäuser, Basel (1984)

    Book  MATH  Google Scholar 

  36. Glowinski, R., Le Tallec, P.: Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. Studies in Applied Mathematics, vol. 9. SIAM, Philadelphia (1989)

    Book  MATH  Google Scholar 

  37. Goldluecke, B., Cremers, D.: Convex relaxation for multilabel problems with product label spaces. In: Proceedings of the 11th European Conference on Computer Vision. LNCS, vol. 6315, pp. 225–238. Springer, Berlin (2010)

    Google Scholar 

  38. Goldstein, T., Osher, S.: The split Bregman method for L 1-regularized problems. SIAM J. Imaging Sci. 2, 323–343 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  39. Goldstein, T., Bresson, X., Osher, S.: Geometric applications of the split Bregman method: segmentation and surface reconstruction. J. Sci. Comput. 45, 272–293 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  40. Helin, T., Lassas, M.: Hierarchical models in statistical inverse problems and the Mumford-Shah functional. Inverse Probl. 27, 015008 (2011). 32 pp.

    Article  MathSciNet  Google Scholar 

  41. Hell, S.W.: Toward fluorescence nanoscopy. Nat. Biotechnol. 21, 1347–1355 (2003)

    Article  Google Scholar 

  42. Ito, K., Kunisch, K.: Lagrange Multiplier Approach to Variational Problems and Applications. Advances in Design and Control, vol. 15. SIAM, Philadelphia (2008)

    Book  MATH  Google Scholar 

  43. Jin, Z., Yang, X.: A variational model to remove the multiplicative noise in ultrasound images. J. Math. Imaging Vis. 39, 62–74 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  44. Kass, M., Witkin, A., Terzopoulos, D.: Snakes: active contour models. Int. J. Comput. Vis. 1, 321–331 (1988)

    Article  Google Scholar 

  45. Krissian, K., Kikinis, R., Westin, C.-F., Vosburgh, K.: Speckle-constrained filtering of ultrasound images. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 547–552 (2005)

    Google Scholar 

  46. Lantéri, H., Theys, C.: Restoration of astrophysical images—the case of Poisson data with additive Gaussian noise. EURASIP J. Appl. Signal Process. 15, 2500–2513 (2005)

    Google Scholar 

  47. Le, T., Chartrand, R., Asaki, T.J.: A variational approach to reconstructing images corrupted by Poisson noise. J. Math. Imaging Vis. 27, 257–263 (2007)

    Article  MathSciNet  Google Scholar 

  48. Lellmann, J., Becker, F., Schnörr, C.: Convex optimization for multi-class image labeling with a novel family of total variation based regularizers. In: Proceedings of the IEEE 12th International Conference on Computer Vision, pp. 646–653 (2009)

    Google Scholar 

  49. Lellmann, J., Lenzen, F., Schnörr, C.: Optimality bounds for a variational relaxation of the image partitioning problem. In: Proceedings of the 8th International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition. LNCS, vol. 6819, pp. 132–146. Springer, Berlin (2011)

    Google Scholar 

  50. Li, C., Xu, C., Gui, C., Fox, M.D.: Level set evolution without re-initialization: a new variational formulation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 430–436 (2005)

    Google Scholar 

  51. Llacer, J., Núñez, J.: Iterative maximum likelihood and Bayesian algorithms for image reconstruction in astronomy. In: White, R.L., Allen, R.J. (eds.) The Restoration of Hubble Space Telescope Images, pp. 62–69. The Space Telescope Science Institute, Baltimore (1990)

    Google Scholar 

  52. Loupas, T., McDicken, W.N., Allan, P.L.: An adaptive weighted median filter for speckle suppression in medical ultrasonic images. IEEE Trans. Circuits Syst. 36, 129–135 (1989)

    Article  Google Scholar 

  53. Martin, P., Réfrégier, P., Goudail, F., Guérault, F.: Influence of the noise model on level set active contour segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 26, 799–803 (2004)

    Article  Google Scholar 

  54. Megginson, R.E.: An Introduction to Banach Space Theory. Graduate Texts in Mathematics, vol. 183. Springer, Berlin (1998)

    Book  MATH  Google Scholar 

  55. Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42, 577–685 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  56. Nečas, J.: Les Méthodes Directes en Théorie des Équations Elliptiques. Academia/Masson et Cie, Prague/Paris (1967)

    MATH  Google Scholar 

  57. Noble, J.A., Boukerroui, D.: Ultrasound image segmentation: a survey. IEEE Trans. Med. Imaging 25, 987–1010 (2006)

    Article  Google Scholar 

  58. Obereder, A., Scherzer, O., Kovac, A.: Bivariate density estimation using BV regularisation. Comput. Stat. Data Anal. 51, 5622–5634 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  59. Osher, S., Fedkiw, R.: Level Set Methods and Dynamic Implicit Surfaces. Applied Mathematical Sciences, vol. 153. Springer, Berlin (2003)

    MATH  Google Scholar 

  60. Osher, S., Sethian, J.A.: Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations. J. Comput. Phys. 79, 12–49 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  61. Paragois, N., Deriche, R.: Geodesic active regions: a new paradigm to deal with frame partition problems in computer vision. J. Vis. Commun. Image Represent. 13, 249–268 (2002)

    Article  Google Scholar 

  62. Pirich, C., Schwaiger, M.: The clinical role of positron emission tomography in management of the cardiac patient. Port. J. Cardiol. 19(Suppl 1), 89–100 (2000)

    Google Scholar 

  63. Pock, T., Chambolle, A., Cremers, D., Bischof, H.: A convex relaxation approach for computing minimal partitions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 810–817 (2009)

    Google Scholar 

  64. Pock, T., Cremers, D., Bischof, H., Chambolle, A.: Global solutions of variational models with convex regularization. SIAM J. Imaging Sci. 3, 1122–1145 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  65. Resmerita, E., Anderssen, R.S.: Joint additive Kullback-Leibler residual minimization and regularization for linear inverse problems. Math. Methods Appl. Sci. 30, 1527–1544 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  66. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)

    Article  MATH  Google Scholar 

  67. Rudin, L., Lions, P.-L., Osher, S.: Multiplicative denoising and deblurring: theory and algorithms. In: Geometric Level Set Methods in Imaging, Vision, and Graphics, pp. 103–119. Springer, Berlin (2003)

    Chapter  Google Scholar 

  68. Sarti, A., Corsi, C., Mazzini, E., Lamberti, C.: Maximum likelihood segmentation of ultrasound images with Rayleigh distribution. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 52, 947–960 (2005)

    Article  Google Scholar 

  69. Sawatzky, A.: (Nonlocal) Total Variation in Medical Imaging. PhD thesis, University of Münster (2011). CAM Report 11-47, UCLA

  70. Sawatzky, A., Brune, C., Müller, J., Burger, M.: Total variation processing of images with Poisson statistics. In: Jiang, X., Petkov, N. (eds.) Computer Analysis of Images and Patterns. Lecture Notes in Computer Science, vol. 5702, pp. 533–540 (2009)

    Chapter  Google Scholar 

  71. Schäfers, K.P., et al.: Absolute quantification of myocardial blood flow with \(\mathrm{H}_{2} ^{15}\)O and 3-dimensional PET: an experimental validation. J. Nucl. Med. 43, 1031–1040 (2002)

    Google Scholar 

  72. Shepp, L.A., Vardi, Y.: Maximum likelihood reconstruction for emission tomography. IEEE Trans. Med. Imaging 1, 113–122 (1982)

    Article  Google Scholar 

  73. Smereka, P.: Semi-implicit level set methods for curvature and surface diffusion motion. J. Sci. Comput. 19, 439–456 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  74. Snyder, D.L., Hammoud, A.M., White, R.L.: Image recovery from data acquired with a charge-coupled-device camera. J. Opt. Soc. Am. A, Opt. Image Sci. Vis. 10, 1014–1023 (1993)

    Article  Google Scholar 

  75. Soret, M., Bacharach, S.L., Buvat, I.: Partial-volume effect in PET tumor imaging. J. Nucl. Med. 48, 932–945 (2007)

    Article  Google Scholar 

  76. Stypmann, J., et al.: Dilated cardiomyopathy in mice deficient for the lysosomal cysteine peptidase cathepsin L. Proc. Natl. Acad. Sci. USA 99, 6234–6239 (2002)

    Article  Google Scholar 

  77. Tur, M., Chin, K.C., Goodman, J.W.: When is speckle noise multiplicative? Appl. Opt. 21, 1157–1159 (1982)

    Article  Google Scholar 

  78. Vardi, Y., Shepp, L.A., Kaufman, L.: A statistical model for positron emission tomography. J. Am. Stat. Assoc. 80, 8–20 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  79. Vese, L.A., Chan, T.F.: A multiphase level set framework for image segmentation using the Mumford and Shah model. Int. J. Comput. Vis. 50, 271–293 (2002)

    Article  MATH  Google Scholar 

  80. Villani, C.: Topics in Optimal Transportation. Graduate Studies in Mathematics, vol. 58. American Mathematical Society, New York (2003)

    MATH  Google Scholar 

  81. Vovk, U., Pernuš, F., Likar, B.: A review of methods for correction of intensity inhomogeneity in MRI. IEEE Trans. Med. Imaging 26, 405–421 (2007)

    Article  Google Scholar 

  82. Wellnhofer, E., et al.: Angiographic assessment of cardiac allograft vasculopathy: results of a consensus conference of the task force for thoracic organ transplantation of the German cardiac society. Transpl. Int. 23, 1094–1104 (2010)

    Article  Google Scholar 

  83. Wernick, M.N., Aarsvold, J.N. (eds.): Emission Tomography: The Fundamentals of PET and SPECT. Elsevier, Amsterdam (2004)

    Google Scholar 

  84. Wirtz, D.: SEGMEDIX: Development and application of a medical image segmentation framework. Master’s thesis, University of Münster (2009) www.agh.ians.uni-stuttgart.de/uploads/media/DA_Wirtz.pdf

  85. Xiao, G., Brady, M., Noble, J.A., Zhang, Y.: Segmentation of ultrasound B-mode images with intensity inhomogeneity correction. IEEE Trans. Med. Imaging 21, 48–57 (2002)

    Article  Google Scholar 

Download references

Acknowledgements

This study has been supported by the German Research Foundation DFG, SFB 656 MoBil (project B2, B3, C3), as well as DFG project BU 2327/1. The authors thank Jörg Stypmann (University Hospital of Münster) for providing medical ultrasound data. Furthermore, we thank Florian Büther and Klaus Schäfers (EIMI, WWU Münster) for providing PET data. The authors thank Tanja Teuber (TU Kaiserslautern) for useful hints on the speckle noise model.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Alex Sawatzky.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sawatzky, A., Tenbrinck, D., Jiang, X. et al. A Variational Framework for Region-Based Segmentation Incorporating Physical Noise Models. J Math Imaging Vis 47, 179–209 (2013). https://doi.org/10.1007/s10851-013-0419-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-013-0419-6

Keywords

Navigation