Skip to main content
SpringerLink
Log in

Neural theory for the perception of causal actions

  • Original Article
  • Published:
Psychological Research Aims and scope Submit manuscript

Abstract

The efficient prediction of the behavior of others requires the recognition of their actions and an understanding of their action goals. In humans, this process is fast and extremely robust, as demonstrated by classical experiments showing that human observers reliably judge causal relationships and attribute interactive social behavior to strongly simplified stimuli consisting of simple moving geometrical shapes. While psychophysical experiments have identified critical visual features that determine the perception of causality and agency from such stimuli, the underlying detailed neural mechanisms remain largely unclear, and it is an open question why humans developed this advanced visual capability at all. We created pairs of naturalistic and abstract stimuli of hand actions that were exactly matched in terms of their motion parameters. We show that varying critical stimulus parameters for both stimulus types leads to very similar modulations of the perception of causality. However, the additional form information about the hand shape and its relationship with the object supports more fine-grained distinctions for the naturalistic stimuli. Moreover, we show that a physiologically plausible model for the recognition of goal-directed hand actions reproduces the observed dependencies of causality perception on critical stimulus parameters. These results support the hypothesis that selectivity for abstract action stimuli might emerge from the same neural mechanisms that underlie the visual processing of natural goal-directed action stimuli. Furthermore, the model proposes specific detailed neural circuits underlying this visual function, which can be evaluated in future experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Allison, T., Puce, A., & McCarthy, G. (2000). Social perception from visual cues: Role of the STS region. Trends in Cognitive Sciences, 4(7), 267–278.

    Article  PubMed  Google Scholar 

  • Baker, C. L., Saxe, R., & Tenenbaum, J. B. (2009). Action understanding as inverse planning. Cognition, 113(3), 329–349.

    Article  PubMed  Google Scholar 

  • Barraclough, N. E., Keith, R. H., Xiao, D., Oram, M. W., & Perrett, D. I. (2009). Visual adaptation to goal-directed hand actions. Journal of Cognitive Neuroscience, 21(9), 1806–1820.

    Article  PubMed  Google Scholar 

  • Barrett, H. C., Todd, P. M., Miller, G. F., & Blythe, P. (2005). Accurate judgments of intention from motion alone: A cross-cultural study. Evolution and Human Behavior, 26, 313–331.

    Article  Google Scholar 

  • Bassili, F. (1976). Temporal and spatial contingencies in the perception of social events. Journal of Personality and Social Psychology, 33(6), 680–685.

    Article  Google Scholar 

  • Beardsley, S. A., & Vaina, L. M. (2001). A laterally interconnected neural architecture in MST accounts for psychophysical discrimination of complex motion patterns. Journal of Computational Neuroscience, 10(3), 255–280.

    Article  PubMed  Google Scholar 

  • Beasley, N. A. (1968). The extent of individual differences in the perception of causality. Canadian Journal of Psychology, 22(5), 399–407.

    Article  PubMed  Google Scholar 

  • Blakemore, S. J., & Decety, J. (2001). From the perception of action to the understanding of intention. Nature Reviews Neuroscience, 2(8), 561–567.

    Article  PubMed  Google Scholar 

  • Blythe, P. W., Todd, P. M., & Miller, G. F. (1999). How motion reveals intention: Categorizing social interactions. In G. Gigerenzer & P. M. Todd (Eds.), Simple heuristics that make us smart (pp. 257–285). Oxford: Oxford University Press.

    Google Scholar 

  • Bonaiuto, J., & Arbib, M. A. (2010). Extending the mirror neuron system model, II: What did I just do? A new role for mirror neurons. Biological Cybernetics, 102(4), 341–359.

    Article  PubMed  Google Scholar 

  • Brass, M., Schmitt, R. M., Spengler, S., & Gergely, G. (2007). Investigating action understanding: Inferential processes versus action simulation. Current Biology, 17(24), 2117–2121.

    Article  PubMed  Google Scholar 

  • Castelli, F., Happe, F., Frith, U., & Frith, C. (2000). Movement and mind: A functional imaging study of perception and interpretation of complex intentional movement patterns. Neuroimage, 12(3), 314–325.

    Article  PubMed  Google Scholar 

  • Chersi, F. (2011). Neural mechanisms and models underlying joint action. Experimental Brain Research, 211(3–4), 643–653.

    Article  Google Scholar 

  • Choi, H., & Scholl, B. J. (2006). Measuring causal perception: Connections to representational momentum? Acta psychologica (Amst), 123(1–2), 91–111.

    Article  Google Scholar 

  • Dasser, V., Ulbaek, I., & Premack, D. (1989). The perception of intention. Science, 243(4889), 365–367.

    Article  PubMed  Google Scholar 

  • de Lange, F. P., Spronk, M., Willems, R. M., Toni, I., & Bekkering, H. (2008). Complementary systems for understanding action intentions. Current Biology, 18(6), 454–457.

    Article  PubMed  Google Scholar 

  • Deco, G., & Rolls, E. T. (2005). Attention, short-term memory, and action selection: A unifying theory. Progress in Neurobiology, 76(4), 236–256.

    Article  PubMed  Google Scholar 

  • Di Carlo, J. J., & Maunsell, J. H. R. (2003). Anterior inferotemporal neurons of monkeys engaged in object recognition can be highly sensitive to object. Neurophysiology, 89, 3264–3278.

    Article  Google Scholar 

  • Dittrich, W. H., & Lea, S. E. (1994). Visual perception of intentional motion. Perception, 23(3), 253–268.

    Article  PubMed  Google Scholar 

  • Fleischer, F., Casile, A., & Giese, M. A. (2009). Bio-inspired approach for the recognition of goal-directed hand actions. In X. Jiang & N. Petkov (Eds.), Conference on Computer Analysis of Images and Patterns (CAIP), LCNS (Vol. 5702, pp. 714–722).

  • Fleischer, F., & Giese, M. A. (2010). Computational Mechanisms of the Visual Processing of Action Stimuli. In K. Johnson & M. Shiffrar (Eds.), Perception of the human body in motion: Findings, theory and practice (Vol. in press). New York: Oxford University Press.

  • Fogassi, L., Ferrari, P. F., Gesierich, B., Rozzi, S., Chersi, F., & Rizzolatti, G. (2005). Parietal lobe: From action organization to intention understanding. Science, 29, 662–667.

    Google Scholar 

  • Fonlupt, P. (2003). Perception and judgement of physical causality involve different brain structures. Brain Research. Cognitive Brain Research, 17(2), 248–254.

    Article  PubMed  Google Scholar 

  • Frith, C. D., & Frith, U. (1999). Interacting minds—a biological basis. Science, 286(5445), 1692–1695.

    Article  PubMed  Google Scholar 

  • Frith, U., & Frith, C. D. (2003). Development and neurophysiology of mentalizing. Philosophical Transactions of the Royal Society of London. Series B, Biological sciences, 358(1431), 459–473.

    Article  PubMed  Google Scholar 

  • Fugelsang, J. A., Roser, M. E., Corballis, P. M., Gazzaniga, M. S., & Dunbar, K. N. (2005). Brain mechanisms underlying perceptual causality. Brain Research. Cognitive Brain Research, 24(1), 41–47.

    Article  PubMed  Google Scholar 

  • Giese, M. A., & Poggio, T. (2003). Neural mechanisms for the recognition of biological movements. Nature Reviews Neuroscience, 4(3), 179–192.

    Article  PubMed  Google Scholar 

  • Hamilton, A. F., & Grafton, S. T. (2008). Action outcomes are represented in human inferior frontoparietal cortex. Cerebral Cortex, 18(5), 1160–1168.

    Article  PubMed  Google Scholar 

  • Heider, F., & Simmel, M. (1944). An experimental study of apparent behavior. American Journal of Psychology, 57, 243–249.

    Article  Google Scholar 

  • Jastorff, J., Clavagnier, S., Gergely, G., & Orban, G. A. (2011). Neural mechanisms of understanding rational actions: Middle temporal gyrus activation by contextual violation. Cerebral Cortex, 21(2), 318–329.

    Article  PubMed  Google Scholar 

  • Jellema, T., & Perrett, D. I. (2006). Neural representations of perceived bodily actions using a categorical frame of reference. Neuropsychologia, 44(9), 1535–1546.

    Article  PubMed  Google Scholar 

  • Kanizsa, G., & Vicario, G. (1968). The perception of intentional reaction. In G. Kanizsa & G. Vicario (Eds.), Experimental research on perception (pp. 71–126). Trieste: University of Trieste.

    Google Scholar 

  • Koenderink, J. J., van Doorn, A. J., & van de Grind, W. A. (1985). Spatial and temporal parameters of motion detection in the peripheral visual field. Journal of the Optical Society of America A: Optics, Image Science, and Vision, 2(2), 252–259.

    Article  Google Scholar 

  • Kourtzi, Z., & Connor, C. E. (2011). Neural representations for object perception: Structure, category, and adaptive coding. Annual Review of Neuroscience, 34, 45–67.

    Article  PubMed  Google Scholar 

  • Leslie, A. M., & Keeble, S. (1987). Do six-month-old infants perceive causality? Cognition, 25(3), 265–288.

    Article  PubMed  Google Scholar 

  • Martin, A., & Weisberg, J. (2003). Neural foundations for understanding social and mechanical concepts. Cognitive Neuropsychology, 20(3–6), 575–587.

    Article  PubMed  Google Scholar 

  • McAleer, P., & Pollick, F. E. (2008). Understanding intention from minimal displays of human activity. Behavior Research Methods, 40(3), 830–839.

    Article  PubMed  Google Scholar 

  • Michotte, A. (1946/1963). The Perception of Causality (Translated by T.R. Miles and E. Miles). London: Methuen: Basic Books.

  • Nelissen, K., Borra, E., Gerbella, M., Rozzi, S., Luppino, G., Vanduffel, W., et al. (2011). Action observation circuits in the macaque monkey cortex. Journal of Neuroscience, 31(10), 3743–3756.

    Article  PubMed  Google Scholar 

  • Oakes, L. M., & Kannass, K. N. (1999). That’s the way the ball bounces: Infants’ and adults’ perception of spatial and temporal contiguity in collisions involving bouncing balls. Developmental Science, 2(1), 86–101.

    Article  Google Scholar 

  • Ohnishi, T., Moriguchi, Y., Matsuda, H., Mori, T., Hirakata, M., Imabayashi, E., et al. (2004). The neural network for the mirror system and mentalizing in normally developed children: An fMRI study. NeuroReport, 15(9), 1483–1487.

    Article  PubMed  Google Scholar 

  • Op De Beeck, H., & Vogels, R. (2000). Spatial sensitivity of macaque inferior temporal neurons. Journal of Comparative Neurology, 426(4), 505–518.

    Article  PubMed  Google Scholar 

  • Oztop, E., Kawato, M., & Arbib, M. (2006). Mirror neurons and imitation: A computationally guided review. Neural Networks, 19(3), 254–271.

    Article  PubMed  Google Scholar 

  • Perrett, D. I., Harries, M. H., Bevan, R., Thomas, S., Benson, P. J., Mistlin, A. J., Chitty, A. J., Hietanen, J. K., & Ortega, J. E. (1989). Frameworks of analysis for the neural representation of animate objects and actions. Journal of Experimental Biology, 146, 87–113.

    Google Scholar 

  • Pessoa, L., & Adolphs, R. (2010). Emotion processing and the amygdala: From a ‘low road’ to ‘many roads’ of evaluating biological significance. Nature Reviews Neuroscience, 11(11), 773–783.

    Article  PubMed  Google Scholar 

  • Petroni, A., Baguear, F., & Della-Maggiore, V. (2010). Motor resonance may originate from sensorimotor experience. Journal of Neurophysiology, 104(4), 1867–1871.

    Article  PubMed  Google Scholar 

  • Pouget, A., & Sejnowski, T. J. (1997). Spatial transformations in the parietal cortex using basis functions. Journal of Cognitive Neuroscience, 9(2), 222–237.

    Article  Google Scholar 

  • Prinz, W. (1997). Perception and Action Planning. European Journal of Cognitive Psychology, 9(2), 129–154.

    Article  Google Scholar 

  • Reithler, J., van Mier, H. I., Peters, J. C., & Goebel, R. (2007). Nonvisual motor learning influences abstract action observation. Current Biology, 17(14), 1201–1207.

    Article  PubMed  Google Scholar 

  • Riesenhuber, M., & Poggio, T. (1999). Hierarchical models of object recognition in cortex. Nature Neuroscience, 2(11), 1019–1025.

    Article  PubMed  Google Scholar 

  • Rips, L. J. (2011). Split identity: Intransitive judgments of the identity of objects. Cognition, 119(3), 356–373.

    Article  PubMed  Google Scholar 

  • Rizzolatti, G., Fogassi, L., & Gallese, V. (2001). Neurophysiological mechanisms underlying the understanding and imitation of action. Nature Reviews Neuroscience, 2(9), 661–670.

    Article  PubMed  Google Scholar 

  • Rizzolatti, G., & Sinigaglia, C. (2010). The functional role of the parietofrontal mirror circuit: Interpretations and misinterpretations. Nature Reviews Neuroscience, 11(4), 264–274.

    Article  PubMed  Google Scholar 

  • Rochat, P., Morgan, R., & Carpenter, M. (1997). Young infants’ sensitivity to movement information specifying social causality. Cognitive Development, 12, 537–561.

    Article  Google Scholar 

  • Roser, M. E., Fugelsang, J. A., Dunbar, K. N., Corballis, P. M., & Gazzaniga, M. S. (2005). Dissociating processes supporting causal perception and causal inference in the brain. Neuropsychology, 19(5), 591–602.

    Article  PubMed  Google Scholar 

  • Salinas, E., & Abbott, L. F. (1995). Transfer of coded information from sensory to motor networks. Journal of Neuroscience, 15(10), 6461–6474.

    PubMed  Google Scholar 

  • Saxe, R., & Carey, S. (2006). The perception of causality in infancy. Acta Psychologica (Amst), 123(1–2), 144–165.

    Article  Google Scholar 

  • Saxe, R., Xiao, D. K., Kovacs, G., Perrett, D. I., & Kanwisher, N. (2004). A region of right posterior superior temporal sulcus responds to observed intentional actions. Neuropsychologia, 42(11), 1435–1446.

    Article  PubMed  Google Scholar 

  • Schlottmann, A., & Anderson, N. H. (1993). An information integration approach to phenomenal causality. Memory & Cognition, 21(6), 785–801.

    Article  Google Scholar 

  • Schlottmann, A., Ray, E. D., Mitchell, A., & Demetriou, N. (2006). Perceived physical and social causality in animated motions: Spontaneous reports and ratings. Acta Psychologica (Amst), 123(1–2), 112–143.

    Article  Google Scholar 

  • Schlottmann, A., & Shanks, D. R. (1992). Evidence for a distinction between judged and perceived causality. Quarterly Journal of Experimental Psychology. A, Human Experimental Psychology, 44(2), 321–342.

    Google Scholar 

  • Scholl, B. J., & Tremoulet, P. D. (2000). Perceptual causality and animacy. Trends in Cognitive Sciences, 4(8), 299–309.

    Article  PubMed  Google Scholar 

  • Schubotz, R. I., & von Cramon, D. Y. (2004). Sequences of abstract nonbiological stimuli share ventral premotor cortex with action observation and imagery. Journal of Neuroscience, 24(24), 5467–5474.

    Article  PubMed  Google Scholar 

  • Schultz, J., Imamizu, H., Kawato, M., & Frith, C. D. (2004). Activation of the human superior temporal gyrus during observation of goal attribution by intentional objects. Journal of Cognitive Neuroscience, 16(10), 1695–1705.

    Article  PubMed  Google Scholar 

  • Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., & Poggio, T. (2007). Robust object recognition with cortex-like mechanisms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(3), 411–426.

    Article  PubMed  Google Scholar 

  • Smith, A. T., & Snowden, R. J. (Eds.). (1994). Visual detection of motion. London: Academic Press Limited.

    Google Scholar 

  • Straube, B., & Chatterjee, A. (2010). Space and time in perceptual causality. Frontiers in Human Neuroscience, 4, 28.

    PubMed  Google Scholar 

  • Tessitore, G., Prevete, R., Catanzariti, E., & Tamburrini, G. (2010). From motor to sensory processing in mirror neuron computational modelling. Biological Cybernetics, 103(6), 471–485.

    Article  PubMed  Google Scholar 

  • Treue, S. (2001). Neural correlates of attention in primate visual cortex. Trends in Neurosciences, 24(5), 295–300.

    Article  PubMed  Google Scholar 

  • Treue, S., & Maunsell, J. H. R. (2006). Feature-based attention in visual cortex. Trends in Neurosciences, 29(6), 317–322.

    Article  PubMed  Google Scholar 

  • Ullman, S. (2007). Object recognition and segmentation by a fragment-based hierarchy. Trends in Cognitive Sciences, 11(2), 58–64.

    Article  PubMed  Google Scholar 

  • Van Overwalle, F., & Baetens, K. (2009). Understanding others’ actions and goals by mirror and mentalizing systems: A meta-analysis. Neuroimage, 48(3), 564–584.

    Article  PubMed  Google Scholar 

  • White, P. A., & Milne, A. (1997). Phenomenal causality: Impressions of pulling in the visual perception of objects in motion. American Journal of Psychology, 110(4), 573–602.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

We thank H. Alhumsi for help with the data collection, D. Endres for stimulating discussions, M. Angelovska for help with the graphical illustrations, and K. Festl for help with the data analysis. This work was supported by the EC FP7 Projects TANGO (Grant FP7-249858-TP3) and AMARSi (Grant FP7-ICT-248311), the Deutsche Forschungsgemeinschaft Grant GI 305/4-1 and the Hermann and Lilly Schilling Foundation.

Author information

Authors and Affiliations

  1. Section for Computational Sensomotorics, Department of Cognitive Neurology, Hertie Institute for Clinical Brain Research and Centre for Integrative Neuroscience, University Clinic Tübingen, Frondsbergstrasse 23, 72070, Tübingen, Germany

    Falk Fleischer, Andrea Christensen & Martin A. Giese

  2. Department of Cognitive Neurology, Hertie Institute for Clinical Brain Research and Centre for Integrative Neuroscience, University Clinic Tübingen, Hoppe-Seyler-Straße 3, 72076, Tübingen, Germany

    Vittorio Caggiano & Peter Thier

  3. McGovern Institute for Brain Research Massachusetts Institute of Technology (MIT), 77 Massachussetts Avenue, Building 46, Room 6177, Cambridge, MA, 02139, USA

    Vittorio Caggiano

Authors
  1. Falk Fleischer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea Christensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vittorio Caggiano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peter Thier
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Martin A. Giese
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martin A. Giese.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (AVI 60 kb)

Supplementary material 2 (AVI 74 kb)

Supplementary material 3 (AVI 66 kb)

Supplementary material 4 (AVI 60 kb)

Supplementary material 5 (AVI 66 kb)

Supplementary material 6 (AVI 70 kb)

Supplementary material 7 (AVI 81 kb)

Supplementary material 8 (AVI 61 kb)

Supplementary material 9 (AVI 60 kb)

Supplementary material 10 (AVI 42 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fleischer, F., Christensen, A., Caggiano, V. et al. Neural theory for the perception of causal actions. Psychological Research 76, 476–493 (2012). https://doi.org/10.1007/s00426-012-0437-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00426-012-0437-9

Keywords

Search

Navigation