Abstract
Action-compatibility effects (ACEs) arise due to incongruity between perceptuo-motor traces stored in memory and the perceptuo-motor demands of a retrieval task. Recent research has suggested that ACEs arising during spatial memory retrieval are additionally modulated by individual differences in how experienced participants are with a college campus environment. However, the extent and nature of experience with a real-world environment is difficult to assess and control, and characteristics of the retrieval task itself might modulate ACEs during spatial memory retrieval. The present study provides a more controlled and in-depth examination of how individual differences and task-based factors interact to shape ACEs when participants retrieve spatial memories. In two experiments, participants with varied video game experience learned a virtual environment and then used the computer mouse to verify spatial relationships from different perspectives. Mouse trajectories demonstrated ACEs, differing by retrieval perspective and video game experience. Videogame experts demonstrated the ACE based on learned spatial relationships during egocentric retrieval only, whereas videogame novices showed the ACE based on semantic processing of directional terms only. Specifically, gaming experts invoke perspective-specific perceptuo-motor associations to retrieve spatial knowledge, whereas non-experts are influenced by semantically based associations specific to the retrieval task. Results are discussed in the context of action-compatibility effects, the intentional weighting hypothesis, and the flexible encoding and retrieval of spatial information.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anguera, J. A., Boccanfuso, J., Rintoul, J. L., Faraji, F., Janowich, J., Kong, E., … Al-Hashimi, O. (2013). Video game training enhances cognitive control in older adults. Nature, 501(7465), 97–101. https://doi.org/10.1038/nature12486.
Basak, C., Boot, W. R., Voss, M. W., & Kramer, A. F. (2008). Can training in a real-time strategy video game attenuate cognitive decline in older adults? Psychology and Aging, 23(4), 765–777. https://doi.org/10.1037/a0013494.
Boot, W. R., Blakely, D. P., & Simons, D. J. (2011). Do action video games improve perception and cognition? Frontiers in Psychology, 2(SEP), 1–6. https://doi.org/10.3389/fpsyg.2011.00226.
Boot, W. R., Kramer, A. F., Simons, D. J., Fabiani, M., & Gratton, G. (2008). The effects of video game playing on attention, memory, and executive control. Acta Psychologica, 129(3), 387–398. https://doi.org/10.1016/j.actpsy.2008.09.005.
Boumenir, Y., Rebillard, G., & Dresp-Langley, B. Short-Term Memory Representation Of A Complex And Non-Familiar Environment After Brief Exposure. WebmedCentral BRAIN 2011;2(1):WMC001523. https://doi.org/10.9754/journal.wmc.2011.001523.
Brunyé, T. T., Burte, H., Houck, L. A., & Taylor, H. A. (2015). The map in our head is not oriented north: evidence from a real-world environment. PloS one, 10(9), e0135803. https://doi.org/10.1371/journal.pone.0135803.
Brunyé, T. T., Gardony, A., Mahoney, C. R., & Taylor, H. A. (2012). Going to town: Visualized perspectives and navigation through virtual environments. Computers in Human Behavior, 28(1), 257–266. https://doi.org/10.1016/j.chb.2011.09.008.
Brunye, T. T., Mahoney, C. R., & Taylor, H. A. (2010). Moving through imagined space: Mentally simulating locomotion during spatial description reading. Acta Psychologica, 134(1), 110–124. https://doi.org/10.1016/j.actpsy.2010.01.003.
Brunye, T.,T., Ditman, T., Mahoney, C. R., Augustyn, J. S., & Taylor, H. A. (2009). When you and I share perspectives: Pronouns modulate perspective taking during narrative comprehension. Psychological Science, 20(1), 27–32. https://doi.org/10.1111/j.1467-9280.2008.02249.x.
Carlson, L. A., & Kenny, R. (2006). Interpreting spatial terms involves simulating interactions. Psychonomic Bulletin & Review, 13(4), 682–688. https://doi.org/10.3758/BF03193981.
Cobb, S. V. G., Nichols, S., Ramsey, A., & Wilson, J. R. (1999). Virtual Reality-Induced Symptoms and Effects (VRISE). Presence, 8(2), 169–186. https://doi.org/10.1162/105474699566152.
Franklin, N., & Tversky, B. (1990). Searching imagined environments. Journal of Experimental Psychology. General, 119(1), 63–76. https://doi.org/10.1037/0096-3445.119.1.63.
Freeman, J. B., & Ambady, N. (2010). MouseTracker: Software for studying real-time mental processing using a computer mouse-tracking method. Behavior Research Methods, 42(1), 226–241. https://doi.org/10.3758/BRM.42.1.226.
Friedman, A., & Montello, D. R. (2006). Global-scale location and distance estimates: Common representations and strategies in absolute and relative judgments. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32(2), 333–346. https://doi.org/10.1037/0278-7393.32.3.333.
Gaunet, F., Vidal, M., Kemeny, A., & Berthoz, A. (2001). Active, passive and snapshot exploration in a virtual environment: influence on scene memory, reorientation and path memory. Cognitive Brain Research, 11(3), 409–420. https://doi.org/10.1016/S0926-6410(01)00013-1.
Glenberg, A. M., & Kaschak, M. P. (2002). Grounding language in action.. Psychonomic Bulletin & Review, 9(3), 558–565. https://doi.org/10.3758/BF03196313.
Glenberg, A. M., Lopez-Mobilia, G., McBeath, M., Toma, M., Sato, M., & Cattaneo, L. (2010). Knowing beans: Human mirror mechanisms revealed through motor adaptation. Frontiers in Human Neuroscience, 4 https://doi.org/10.3389/fnhum.2010.00204.
Hommel, B. (2007). Feature integration across perception and action: Event files affect response choice. Psychological research, 71(1), 42–63. https://doi.org/10.1007/s00426-005-0035-1.
Hommel, B. (2011). The Simon effect as tool and heuristic. Acta psychologica, 136(2), 189–202. https://doi.org/10.1016/j.actpsy.2010.04.011.
Hommel, B. (2017). Goal-directed actions. The Oxford Handbook of Causal Reasoning, p. 265.
Hommel, B., Gehrke, J., & Knuf, L. (2000). Hierarchical coding in the perception and memory of spatial layouts. Psychological Research/Psychologische Forschung, 64(1), 1–10. https://doi.org/10.1007/s004260000032.
Hommel, B., Müsseler, J., Aschersleben, G., & Prinz, W. (2001). The theory of event coding (TEC): A framework for perception and action planning. Behavioral and brain sciences, 24(5), 910–926. https://doi.org/10.1017/S0140525X01520105.
Jansen-Osmann, P., & Berendt, B. (2002). Investigating distance knowledge using virtual environments. Environment and Behavior, 34(2), 178–193. https://doi.org/10.1177/0013916502034002002.
Klatzky, R. L., Loomis, J. M., Beall, A. C., Chance, S. S., & Golledge, R. G. (1998). Spatial updating of self-position and orientation during real, imagined, and virtual locomotion. Psychological Science, 9(4), 293–298. https://doi.org/10.1111/1467-9280.00058.
Knoblich, G., & Flach, R. (2001). Predicting the effects of actions: Interactions of perception and action.. Psychological Science, 12(6), 467–467-472. https://doi.org/10.1111/1467-9280.00387.
Levinson, S. C. (1996). Frames of reference and molyneux’s question: Crosslinguistic evidence. Cambridge, MA: The MIT Press.
Li, H., & Giudice, N. A. (2013). The Effects of Immersion and Body-based Rotation on Learning Multi-level Indoor Virtual Environments. Proceedings of the Fifth ACM SIGSPATIAL International Workshop on Indoor Spatial Awareness - ISA’13, 8–15. https://doi.org/10.1145/2533810.2533811.
Li, L., & Warren, W. H. (2002). Retinal flow is sufficient for steering during observer rotation. Psychological Science, 13(5), 485–490. https://doi.org/10.1111/1467-9280.00486.
Longo, M. R., & Lourenco, S. F. (2007). Spatial attention and the mental number line: Evidence for characteristic biases and compression. Neuropsychologia,. 45(7), 1400–1407. https://doi.org/10.1016/j.neuropsychologia.2006.11.002.
Loomis, J. M., Blascovich, J. J., & Beall, A. C. (1999). Immersive virtual environment technology as a basic research tool in psychology. Behavior Research Methods, Instruments, & Computers, 31(4), 557–564. https://doi.org/10.3758/BF03200735.
Majid, A., Bowerman, M., Kita, S., Haun, D. B. M., & Levinson, S. C. (2004). Can language restructure cognition? the case for space. Trends in Cognitive Sciences, 8(3), 108–114. https://doi.org/10.1016/j.tics.2004.01.003.
Memelink, J., & Hommel, B. (2013). Intentional weighting: a basic principle in cognitive control. Psychological Research Psychologische Forschung, 77(3), 249–259. https://doi.org/10.1007/s00426-012-0435-y.
Montello, D. R., Waller, D., Hegarty, M., & Richardson, A. E. (2004). Spatial memory of real environments, virtual environments, and maps. Human spatial memory: Remembering where, 251–285.
Munafo, J., Diedrick, M., & Stoffregen, T. A. (2016). The virtual reality head-mounted display Oculus Rift induces motion sickness and is sexist in its effects. Experimental Brain Research, 1–13. https://doi.org/10.1007/s00221-016-4846-7.
Murias, K., Kwok, K., Castillejo, A. G., Liu, I., & Iaria, G. (2016). The effects of video game use on performance in a virtual navigation task. Computers in Human Behavior, 58, 398–406. https://doi.org/10.1016/j.chb.2016.01.020.
Parsons, L. M. (1994). Temporal and kinematic properties of motor behavior reflected in mentally simulated action. Journal of Experimental Psychology: Human Perception and Performance, 20(4), 709–730. https://doi.org/10.1037/0096-1523.20.4.709.
Prinz, W. (1997). Perception and action planning. European Journal of Cognitive Psychology, 9(2), 129–154. https://doi.org/10.1080/713752551.
Regier, T., & Carlson, L. A. (2001). Grounding spatial language in perception: An empirical and computational investigation. Journal of Experimental Psychology: General, 130(2), 273–298. https://doi.org/10.1037/0096-3445.130.2.273.
Repp, B. H., & Knoblich, G. (2004). Perceiving action identity: How pianists recognize their own performances.. Psychological Science, 15(9), 604–604-609. https://doi.org/10.1111/j.0956-7976.2004.00727.x.
Richardson, A. E., & Collaer, M. L. (2011). Virtual navigation performance: the relationship to field of view and prior video gaming experience. Perceptual and motor skills, 112(2), 477–498. https://doi.org/10.2466/22.24.PMS.112.2.477-498.
Richardson, A. E., Montello, D. R., & Hegarty, M. (1999). Spatial knowledge acquisition from maps and from navigation in real and virtual environments.. Memory & Cognition, 27(4), 741 – 50. https://doi.org/10.3758/BF03211566.
Richardson, A. E., Powers, M. E., & Bousquet, L. G. (2011). Video game experience predicts virtual, but not real navigation performance. Computers in Human Behavior, 27(1), 552–560. https://doi.org/10.1016/j.chb.2010.10.003.
Richardson, D. C., Dale, R., & Spivey, M. J. (2007). Eye movements in language and cognition: A brief introduction. In M. Gonzalez-Marquez, I. Mittelberg, S. Coulson & M. J. Spivey (Eds.), Methods in cognitive linguistics (pp. 323–344). Amsterdam: John Benjamins Publishing Company.
Riecke, B. E., Cunningham, D. W., & Bülthoff, H. H. (2007). Spatial updating in virtual reality: the sufficiency of visual information. Psychological research, 71(3), 298–313. https://doi.org/10.1007/s00426-006-0085-z.
Rizzolatti, G., & Craighero, L. (2004). The mirror-neuron system. Annual Review of Neuroscience, 27, 169–192. https://doi.org/10.1146/annurev.neuro.27.070203.144230.
Ruddle, R. A., & Lessels, S. (2006). For efficient navigational search, humans require full physical movement, but not a rich visual scene. Psychological Science, 17(6), 460–465. https://doi.org/10.1111/j.1467-9280.2006.01728.x.
Ruddle, R. A., Payne, S. J., & Jones, D. M. (1999). Navigating Large-Scale Virtual Environments: What Differences Occur Between Helmet-Mounted and Desk-Top Displays? Abstract. Presence, 8(2), 157–168.
Ruddle, R. A., Volkova, E., Mohler, B., & Bülthoff, H. H. (2011). The effect of landmark and body-based sensory information on route knowledge. Memory & Cognition, 39(4), 686–699. https://doi.org/10.3758/s13421-010-0054-z.
Santos, S. B., Dias, P., Pimentel, A., Baggerman, J. W., Ferreira, C., Silva, S., & Madeira, J. (2009). Head-mounted display versus desktop for 3D navigation in virtual reality: A user study. Multimedia Tools and Applications, 41(1), 161–181. https://doi.org/10.1007/s11042-008-0223-2.
Sharples, S., Cobb, S., Moody, A., & Wilson, J. R. (2008). Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems. Displays, 29(2), 58–69. https://doi.org/10.1016/j.displa.2007.09.005.
Shelton, A. L., & McNamara, T. P. (2004). Orientation and perspective dependence in route and survey learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30(1), 158–170. https://doi.org/10.1037/0278-7393.30.1.158.
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632.
Sparrow, B., Liu, J., & Wegner, D. M. (2011). Google effects on memory: Cognitive consequences of having information at our fingertips. Science, 333(6043), 776–778. https://doi.org/10.1126/science.1207745.
Spence, I., & Feng, J. (2010). Video Games and Spatial Cognition. Review of General Psychology, 14(2), 92–104. https://doi.org/10.1037/a0019491.
Spivey, M., Richardson, D., & Dale, R. (2009). The movement of eye and hand as a window into language and cognition. In E. Morsella, J. A. Bargh & P. M. Gollwitzer (Eds.), (pp. 225–249). New York, NY: Oxford University Press.
Stanney, K. M., Hale, K. S., Nahmens, I., & Kennedy, R. S. (2003). What to Expect from Immersive Virtual Environment Exposure: Influences of Gender, Body Mass Index, and Past Experience. Human Factors, 45(3), 504–520. https://doi.org/10.1518/hfes.45.3.504.27254.
Taylor, H. A., & Tversky, B. (1992). Spatial mental models derived from survey and route descriptions. Journal of Memory and Language, 31(2), 261–292. https://doi.org/10.1016/0749-596X(92)90014-O.
Thorndyke, P. W., & Hayes-Roth, B. (1982). Differences in spatial knowledge acquired from maps and navigation.. Cognitive Psychology, 14(4), 560–589. https://doi.org/10.1016/0010-0285(82)90019-6.
Tversky, B. (2005). How to get around by mind and body: Spatial thought, spatial action. In A. Zilhao (Ed.), Cognition, evolution, and rationality: A cognitive science for the XXIst century. London: Routledge.
Waller, D. (2000). Individual Differences in Spatial Learning From Computer-Simulated Environments. Journal of Experimental Psychology. Applied, 6(4), 307–321. https://doi.org/10.1037//1076-898x.6.4.307.
Wang, Q., Taylor, H. A., & Brunyé, T. T. (2012). When going the right way is hard to do: Distinct phases of action compatibility in spatial knowledge development. Acta Psychologica, 139(3), 449–457. https://doi.org/10.1016/j.actpsy.2012.01.006.
Willemsen, P., Colton, M. B., Creem-Regehr, S., & Thompson, W. B. (2009). The effects of head-mounted display mechanical properties and field of view on distance judgments in virtual environments. ACM Transactions on Applied Perception, 6(2) https://doi.org/10.1145/1498700.1498702.
Wilson, M., & Knoblich, G. (2005). The case for motor involvement in perceiving conspecifics. Psychological Bulletin, 131(3), 460–473. https://doi.org/10.1037/0033-2909.131.3.460.
Witt, J. K., Proffitt, D. R., & Epstein, W. (2004). Perceiving distance: A role of effort and intent. Perception, 33(5), 577–590. https://doi.org/10.1068/p5090.
Zetzsche, C., Wolter, J., Galbraith, C., & Schill, K. (2009). Representation of space: Image-like or sensorimotor? Spatial Vision, 22(5), 409–424. https://doi.org/10.1163/156856809789476074.
Zwaan, R. A., & Taylor, L. J. (2006). Seeing, acting, understanding: Motor resonance in language comprehension. Journal of Experimental Psychology: General, 135(1), 1–11. https://doi.org/10.1037/0096-3445.135.1.1.
Mizell, D. W., Jones, S. P., Slater, M., & Spanlang, B. (2002). Comparing immersive virtual reality with other display modes for visualizing complex 3D geometry. University College London, technical report. doi: 10.1.1.99.5391&rep=rep1&type=pdf.
Funding
This research was funded by the National Natural Science Foundation of China (31,400,865).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical approval
All procedures performed in present studies involving human participants were in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed Consent
All participants gave their informed consent prior to their inclusion in the study.
Conflict of interest
The authors declare that they have no conflict of interest.
Rights and permissions
About this article
Cite this article
Wang, Q., Taylor, H.A. & Brunyé, T.T. Action compatibility in spatial knowledge developed through virtual navigation. Psychological Research 84, 177–191 (2020). https://doi.org/10.1007/s00426-018-0972-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00426-018-0972-0