Skip to main content
SpringerLink
Log in

A Cortical Based Model of Perceptual Completion in the Roto-Translation Space

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

We present a mathematical model of perceptual completion and formation of subjective surfaces, which is at the same time inspired by the architecture of the visual cortex, and is the lifting in the 3-dimensional rototranslation group of the phenomenological variational models based on elastica functional. The initial image is lifted by the simple cells to a surface in the rototranslation group and the completion process is modeled via a diffusion driven motion by curvature. The convergence of the motion to a minimal surface is proved. Results are presented both for modal and amodal completion in classic Kanizsa images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. L. Ambrosio and S. Masnou, “A direct variational approach to a problem arising in image recostruction,” Interfaces and Free Boundaries, Vol. 5, No. 1, pp. 63–81, 2003.

    MathSciNet  MATH  Google Scholar 

  2. G. Barles and C. Georgelin, “A simple proof for the convergence for an approximation scheme for computing motions by mean curvature,” SIAM J. Numerical Analysis, Vol. 32, pp. 484–500, 1995.

    Article  MathSciNet  MATH  Google Scholar 

  3. O. Bar, H. Sompolinsky, and R. Ben-Yishai, “ Theory of orientation tuning in visual cortex” Proc. Natl. Acad. Sci. U.S.A., Vol. 92, pp. 3844–3848, 1995.

    Google Scholar 

  4. A. Bellaiche, “The tangent space in sub-Riemannian geometry” in Proceedings of the satellite Meeting of the 1st European, Congress of Mathematics ‘Journees nonholonomes: Geometrie sous-riemannienne, theorie du controle, robotique,’ Paris, France, June 30–July 1, 1992. Basel: Birkhäuser. Prog. Math., Vol. 144, pp. 1–78, 1996.

  5. G. Bellettini and R. March, “An image segmentation variational model with free discontinuities and contour curvature,” Math. Mod. Meth. Appl. Sci., Vol. 14, pp. 1–45, 2004.

    Article  MathSciNet  MATH  Google Scholar 

  6. C. Ballester, M. Bertalmio, V. Caselles, G. Sapiro, and J. Verdera, “Filling-in by interpolation of vector fields and gray levels,” IEEE Transactions on Image Processing, Vol. 10, No. 8, pp. 1200–1211, 2001.

    Article  MathSciNet  MATH  Google Scholar 

  7. J. Bence, B. Merriman, and S. Osher, “Diffusion generated motion by mena curvature,” in Computational Crystal Growers Workshop, J. Taylor Sel. Taylor (Ed).

  8. A. Bonfiglioli, E. Lanconelli, and F. Uguzzoni, “Fundamental solutions for non-divergence form operators on stratified groups,” Trans. Amer. Math. Soc., Vol. 356, No. 7, pp. 2709–2737, 2004.

    Article  MathSciNet  MATH  Google Scholar 

  9. L. Capogna, D. Danielli, and N. Garofalo, “The geometric Sobolev embedding for vector fields and the isoperimetric inequality” Comm Anal. Geo., Vol. 12, pp. 203–215, 1994.

    MathSciNet  Google Scholar 

  10. M. Carandini and D.L. Ringach, “Predictions of a recurrent model of orientation selectivity,” Vision Res., Vol. 37, pp. 3061–3071, 1997.

    Article  Google Scholar 

  11. G. Citti, M. Manfredini, and A. Sarti, “Neuronal oscillations in the visual cortex: Γ-convergence to the Riemannian Mumford-Shah functional” SIAM Journal of Mathematical Analysis, Vol. 35, No. 6, pp. 1394–1419, 2004.

    Article  MathSciNet  MATH  Google Scholar 

  12. J.G. Daugman, “Uncertainty—relation for resolution in space spatial frequency and orientation optimized by two dimensional visual cortical filters,” J. Opt. Soc. Amer., Vol. 2, No. 7, pp. 1160–1169, 1985.

    Google Scholar 

  13. E. De Giorgi, “Some remarks on Γ convergence and least square methods,” in Composite Media and Homogeniziation Theory G. Dal Masoand and G. F. Dell’Antonio (Eds.), Birkhauser Boston, 1991, pp. 153–142.

  14. S. Esedoglu and R. March, “Segmentation with Deph but without detecting junctions,” Journal of Mathematical Imaging and Vision, Vol. 18, pp. 7–15, 2003.

    Article  MathSciNet  MATH  Google Scholar 

  15. A.K. Engel, P. Konig, C.M. Gray, and W. Singer, “Stimulus dependent neuronal oscillations, in cat visual cortex: Intercolumnar interaction as determined by cross-correlation analysis” European Journal of Neuroscience, Vol. 2, pp. 558–606, 1990.

    Article  Google Scholar 

  16. A.K. Engel, A.K. Kreiter, P. Konig, and W. Singer, “Syncronization of oscillatory neuronal responses between striate and extrastriate visual cortical areas of the cat,” PNAS, Vol. 88, pp. 6048–6052, 1991.

  17. L. Evans, “Convergence of an Algorithm for mean curvature motion,” Indiana Univ. Math. J., Vol. 42, No. 2, pp. 553–557, 1993.

    Article  Google Scholar 

  18. B. Franchi, R. Serapioni, and F. Serra Cassano, “On the structure of finite perimeter sets in step 2 Carnot groups,” J. Geom. Anal., Vol. 13, No. 3, pp. 421–466, 2003.

    MathSciNet  MATH  Google Scholar 

  19. G.B. Folland, “Subelliptic estimates and function spaces on nilpotent Lie groups,” Ark. Mat., Vol. 13, pp. 161–207, 1975.

    Article  MATH  MathSciNet  Google Scholar 

  20. G.B. Folland, “On the Rothschild-Stein lifting theorem,” Commun. Partial Differ. Equations Vol. 2, pp. 165–191, 1977.

    MATH  MathSciNet  Google Scholar 

  21. G.B. Folland and E.M. Stein, “Estimates for the \(\bar\partial_b\) complex and analysis on the Heisenberg group,” Comm. Pure Appl. Math., Vol. 20, pp. 429–522, 1974.

    MathSciNet  Google Scholar 

  22. R. Goodman, “Lifting vector fields to nilpotent Lie groups,” J. Math. Pures Appl., Vol. 57, pp. 77–85, 1978.

    MATH  MathSciNet  Google Scholar 

  23. C.D. Gilbert, A. Das, M. Ito, M. Kapadia, and G. Westheimer, “Spatial integration and cortical dynamics,” Proceedings of the National Academy of Sciences USA, Vol. 93, pp. 615–622.

  24. C.M. Gray, P. Konig, A.K. Engel, and W. Singer, “Oscillatory responses in cat visual cortex exhibit inter-columnar syncronization which reflects global stimulus properties,” Nature, Vol. 338, pp. 334–337, 1989.

    Article  Google Scholar 

  25. S. Grossberg and E. Mingolla, “Neural dynamics of perceptual grouping: Textures, boundaries and emergent segmentations,” in Perception and Psychophysics, 1985.

  26. Field, A. Heyes, and R.F. Hess, “Contour integration by the human visual system: Evidence for a local Association Field,” Vision Research, Vol. 33, pp. 173–193, 1993.

    Article  Google Scholar 

  27. W.C. Hoffman, “The visual cortex is a contact bundle,” Applied Mathematics and Computation, Vol 32, pp. 137–167, 1989.

    Article  MATH  MathSciNet  Google Scholar 

  28. W.C. Hoffman and M. Ferraro, “Lie transformation groups, integral transforms, and invariant pattern recognition,” Spatial Vision, Vol. 8, pp. 33–44, 1994.

    Google Scholar 

  29. H. Hörmander, “Hypoelliptic second-order differential equations,” Acta Math., Vol. 119, pp. 147–171, 1967.

    MATH  MathSciNet  Google Scholar 

  30. H. Hörmander and A. Melin, “Free systems of vector fields,” Ark. Mat, Vol. 16, pp. 83–88, 1978.

    Article  MathSciNet  MATH  Google Scholar 

  31. D. Hubel and T. Wiesel, “Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex,” Journal of Physiology, Vol. 160, pp. 106–154, 1962.

    Google Scholar 

  32. D. Jerison and A. Sánchez-Calle, “Subelliptic, second order differential operators,” Complex analysis III, Proc. Spec. Year, College Park/Md. 1985–86, Lect. Notes Math. 1277, pp. 46–77, 1987.

  33. J.P. Jones and L.A. Palmer “An evaluation of the two-dimensional gabor filter model of simple receptive fields in cat striate cortex,” J. Neurophysiology, Vol. 58, pp. 1233–1258, 1987.

    Google Scholar 

  34. G. Kanizsa, Grammatica del vedere, Il Mulino, Bologna, 1980.

    Google Scholar 

  35. G. Kanizsa, Organization in Vision, Hardcover, 1979.

  36. M.K. Kapadia, M. Ito, C.D. Gilbert, and G. Westheimer, “Improvement in visual sensitivity by changes in local context: Parallel studies in human observers and in V1 of alert monkeys,” Neuron, Vol. 15, pp. 843–856, 1995.

    Article  Google Scholar 

  37. S. Kusuoka and D. Stroock, “Applications of the Malliavin calculus III,” J. Fac. Sci. Univ. Tokio, Sect. IA, Math, Vol. 34, pp. 391–442, 1987.

    Google Scholar 

  38. S. Kusuoka and D. Stroock, “Long time estimates for the heat kernel associated with a uniformly subelliptic symmetric second order operator,” Ann. of Math., Vol. 127, pp. 165–189, 1988.

    Article  MathSciNet  Google Scholar 

  39. I. Kovacs and B. Julesz, “A closed curve is much more than an incomplete one: effect of closure in figure-ground segmentation,” PNAS, 90, pp. 7495–7497, 1993.

    Google Scholar 

  40. I. Kovacs and B. Julesz, “Perceptual sensitivity maps within globally defined visual shapes,” Nature, Vol. 370, pp. 644–646, 1994.

    Article  Google Scholar 

  41. LeVeque and J. Randall, Nonlinear conservation laws and finite volume methods. (English) Steiner, Oskar et al., Computational methods for astrophysical fluid flow. Saas-Fee advanced course 27. Lecture notes 1997. Swiss Society for Astrophysics and Astronomy. Berlin: Springer, pp. 1–159, 1998.

    Google Scholar 

  42. S. Marcelja, “Mathematical description of the response of simple cortical cells,” J. Opt. Soc. Amer., Vol. 70, pp. 1297–1300, 1980.

    Article  MathSciNet  Google Scholar 

  43. V. Magnani, “Differentiability and area formula on stratified Lie groups,” Houston J. Math., Vol. 27, No. 2, pp. 297–323, 2001.

    MATH  MathSciNet  Google Scholar 

  44. D. Mumford, M. Nitzberg, and T. Shiota, Filtering, Segmentation and Deph, Springer-Verlag: Berlin, 1993.

    Google Scholar 

  45. S. Masnou and J.M. Norel, “Level lines based disocclusion,” Proc. 5th. IEEE International Conference on Image Processing, Chicago, Illinois, October 4–7, 1998.

  46. K.D. Miller, A. Kayser, and N.J. Priebe, “Contrast-dependent nonlinearities arise locally in a model of contrast-invariant orientation tuning,” J. Neurophysiol., Vol. 85, pp. 2130–2149, 2001.

    Google Scholar 

  47. E. Mingolla, “Le unità della visione,” IX Kanitza lecture, Trieste symposium on perception and cognition, 26 October 2001.

  48. A. Nagel, E.M. Stein, and S. Wainger, “Balls and metrics defined by vector fields I: Basic properties,” Acta Math., Vol. 155, pp. 103–147, 1985.

    MathSciNet  MATH  Google Scholar 

  49. S.B. Nelson, M. Sur, and D.C. Somers, “An emergent model of orientation selectivity in cat visual cortical simples cells,” J. Neurosci., Vol. 15, pp. 5448–5465, 1995.

    Google Scholar 

  50. S.D. Pauls, “A notion of rectifiability modeled on Carnot groups,” Indiana Univ. Math. J., Vol. 53, No. 1, pp. 49–81, 2004.

    Article  MATH  MathSciNet  Google Scholar 

  51. S.D. Pauls, “Minimal surfaces in the Heisenberg group,” Geom. Dedicata, Vol. 104, pp. 201–231, 2004.

    Article  MATH  MathSciNet  Google Scholar 

  52. P. Perona, “Deformable kernels for early vision,” IEEE-PAMI, Vol. 17, No. 5, pp. 488–499, 1995.

    Google Scholar 

  53. J. Petitot, “Phenomenology of Perception, Qualitative Physics and Sheaf Mereology,” Proceedings of the 16th International Wittgenstein Symposium, Vienna, Verlag Hölder-Pichler-Tempsky, 1994, pp. 387–408.

    Google Scholar 

  54. J. Petitot and Y. Tondut, “Vers une Neuro-geometrie. Fibrations corticales, structures de contact et contours subjectifs modaux, Mathématiques, Informatique et Sciences Humaines,” EHESS, Paris, Vol. 145, pp. 5–101, 1998.

    Google Scholar 

  55. J. Petitot, Morphological Eidetics for Phenomenology of Perception, in Naturalizing Phenomenology: Issues in Contemporary Phenomenology and Cognitive Science, J. Petitot, F.J. Varela, J.-M. Roy, B. Pachoud (Eds.), Stanford, Stanford University Press, 1998, pp. 330–371.

    Google Scholar 

  56. N.J. Priebe, K.D. Miller, T.W. Troyer, and A.E. Krukowsky, “Contrast-invariant orientation tuning in cat visual cortex: Thalamocortical input tuning and correlation-based intracortical connectivity.” J. Neurosci., Vol. 18, pp. 5908–5927, 1998.

    Google Scholar 

  57. L. Rothschild and E.M. Stein, “Hypoelliptic differential operators and nihilpotent Lie groups,” Acta Math., Vol. 137, pp. 247–320, 1977.

    MathSciNet  MATH  Google Scholar 

  58. A. Sarti, R. Malladi and J.A. Sethian, Subjective surfaces: A method for completion of missing boundaries, in Proceedings of the National Academy of Sciences of the United States of America, Vol. 12, No.97, pp. 6258–6263, 2000.

    Article  MathSciNet  Google Scholar 

  59. A. Sarti, G. Citti, and M. Manfredini, “From neural oscillations to variational problems in the visual cortex,” Journal of Physiology, Vol. 97, No. 2–3, pp. 87–385, 2003.

    Google Scholar 

  60. M. Shelley, D.J. Wielaard, D. McLaughlin and R. Shapley, “A neuronal network model of macaque primary visual cortex (v1): Orientation selectivity and dynamics in the input layer 4calpha”. Proc. Natl. Acad. Sci. U.S.A., Vol. 97, pp. 8087–8092, 2000.

    Article  Google Scholar 

  61. S.C. Yen and L.H. Finkel, “Extraction of perceptually salient contours by striate cortical networks,” Vision Res., Vol. 38, No. 5, pp. 719–741, 1998.

    Article  Google Scholar 

  62. Y.Q. Song and X.P. Yang, “BV function in the Heisenberg group,” Chinese Ann. Math. Ser. A, Vol. 24, No. 5, pp. 541–554, 2003; translation in Chinese J. Contemp. Math., Vol. 24, No. 4, pp. 301–316, 2004.

    MathSciNet  MATH  Google Scholar 

  63. E.M. Stein, Harmonic Analysis, Princeton University Press, 1993.

  64. S.K. Vodop’yanov and A.D. Ukhlov, “Approximately differentiable transformations and change of variables on nilpotent groups,” Sib. Math. J., Vol. 37, No.1, pp. 62–78, 1996, translation from Sib. Mat. Zh., Vol. 37, No. 1, pp. 70–89, 1996.

    Article  MathSciNet  MATH  Google Scholar 

  65. V.S. Varadarajan, “Lie groups, Lie algebras, and their representations,” Graduate Texts in Mathematics. 102, New York, Springer, 1984.

    MATH  Google Scholar 

  66. N.T. Varopoulos, L. Saloff-Coste, and T. Coulhon, Analysis and geometry on groups Cambridge texts in Mathematics, 100, Cambridge University Press, Cambredge, 1992.

    Google Scholar 

  67. F.W. Warner, Foundations of differentiable manifolds and Lie groups. Glenview, Illinois-London: Scott, Foresman & Comp. 270, 1971.

    MATH  Google Scholar 

  68. C. Wang, “The comparsion principle for viscosity solutions of fully nonlinear subelliptic equations in Carnot groups,” Preprint.

  69. F. Worgotter and C. Koch, “A detailed model of the primary visual pathway in the cat: Comparison of afferent excitatory and intracortical inhibitory connection schemes for orientation selectivity,” J. Neurosci., Vol. 11, pp. 1959–1979, 1991.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Bologna, Bologna

    G. Citti

  2. Department of Electronics, Information and Systems, University of Bologna, Bologna

    A. Sarti

Authors
  1. G. Citti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Sarti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to G. Citti.

Additional information

The work was supported by University of Bologna: founds for selected research topics.

Giovanna Citti is full professor of Mathematical Analysis at University of Bologna, and she is coordinator, together with A.Sarti, of the local interdipartimental group of “Neuromathematics and Visual Cognition”. Her principal research interests are existence and regularity of solution of nonlinear subelliptic equations represented as sum of squares of vector fields, whose associated geometry is subriemannian. Besides she is interested in applications of instruments of real analysis in Lie Groups and subriemannian geometry to visual perception, and to the study of the functionality of the visual cortex.

Alessandro Sarti received the Ph.D. degree in bioengineering from the University of Bologna in 1996. From 1997 to 2000 he was appointed with a Postdoc position at the Mathematics Department of the University of California, Berkeley, and the Mathematics Department of the Lawrence Berkeley National Laboratory in Berkeley. Since 2001 he got a permanent position at the University of Bologna. He is associate to CREA, Ecole Polytechnique, Paris, France. With Giovanna Citti, he is the scientific responsible of the interdipartimental group of “Neuromathematics and Visual Cognition.” In the last years he gave lectures at the University of Yale, University of California at Los Angeles, University of California at Berkeley, Freie Universitat Berlin, Ecole Normale Superieure Cachan, Ecole Polytechnique, Scuola Normale Superiore di Pisa.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Citti, G., Sarti, A. A Cortical Based Model of Perceptual Completion in the Roto-Translation Space. J Math Imaging Vis 24, 307–326 (2006). https://doi.org/10.1007/s10851-005-3630-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-005-3630-2

Keywords

Search

Navigation