Skip to main content

Representing 3D Space in Working Memory: Spatial Images from Vision, Hearing, Touch, and Language

  • Chapter
  • First Online:
Multisensory Imagery

Abstract

The chapter deals with a form of transient spatial representation referred to as a spatial image. Like a percept, it is externalized, scaled to the environment, and can appear in any direction about the observer. It transcends the concept of modality, as it can be based on inputs from the three spatial senses, from language, and from long-term memory. Evidence is presented that supports each of the claimed properties of the spatial image, showing that it is quite different from a visual image. Much of the evidence presented is based on spatial updating. A major concern is whether spatial images from different input modalities are functionally equivalent—that once instantiated in working memory, the spatial images from different modalities have the same functional characteristics with respect to subsequent processing, such as that involved in spatial updating. Going further, the research provides some evidence that spatial images are amodal (i.e., do not retain modality-specific features).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 229.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 299.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 299.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Röder and Rösler (1998) used the term “spatial image” in connection with the study of internal scanning of images from both vision and touch. Because their study focused on 2D maps, their use of the term seems to refer to a concept narrower than ours. Conversely, in their review of behavioral and neural research on mental imagery, Struiksma et al. (2009) use “spatial image” as a more encompassing term than we do.

References

  • Amorim MA, Glasauer S, Corpinot K, Berthoz A (1997) Updating an object’s orientation and location during nonvisual navigation: a comparison between two processing modes. Atten Percept Psychophys 59:404–418

    CAS  Google Scholar 

  • Andersen RA (1997) Multimodal integration for the representation of space in the posterior parietal cortex. Philos Trans R Soc Lond B Biol Sci 352:1421–1428

    PubMed  CAS  Google Scholar 

  • Ashmead DH, DeFord LD, Northington A (1995) Contribution of listeners’ approaching motion to auditory distance perception. J Exp Psychol Hum Percept Perform 21:239–256

    PubMed  CAS  Google Scholar 

  • Auvray M, Myin E (2009) Perception with compensatory devices: from sensory substitution to sensorimotor extension. Cogn Sci 33:1036–1058

    PubMed  Google Scholar 

  • Avraamides MN (2003) Spatial updating of environments described in texts. Cogn Psychol 47:402–431

    PubMed  Google Scholar 

  • Avraamides MN, Kelly JW (2008) Multiple systems of spatial memory and action. Cogn Process 9:93–106

    PubMed  Google Scholar 

  • Avraamides MN, Kelly JW (2010) Multiple systems of spatial memory: evidence from described scenes. J Exp Psychol Learn Mem Cogn 36:635–645

    PubMed  Google Scholar 

  • Avraamides MN, Loomis JM, Klatzky RL, Golledge RG (2004) Functional equivalence of spatial representations derived from vision and language: evidence from allocentric judgments. J Exp Psychol Learn Mem Cogn 30:801–814

    Google Scholar 

  • Bennett CR, Giudice NA, Klatzky RL, Loomis JM (2011) Spatial images developed through extended touch: comparing updating performance between haptic and visual learning. Poster presented at the 52nd Annual Psychonomics Society Meeting, Seattle, WA, November 2011

    Google Scholar 

  • Blajenkova O, Kozhevnikov M, Motes M (2006) Object-spatial imagery: a new self-report imagery questionnaire. Appl Cogn Psychol 20:239–263

    Google Scholar 

  • Blazhenkova O, Kozhevnikov M (2010) Visual-object ability: a new dimension of non-verbal intelligence. Cognition 117:276–301

    PubMed  Google Scholar 

  • Bryant DJ (1997) Representing space in language and perception. Mind Lang 12:239–264

    Google Scholar 

  • Bryant DJ, Tversky B, Franklin N (1992) Internal and external spatial frameworks for representing described scenes. J Mem Lang 31:74–98

    Google Scholar 

  • Burgess N (2006) Spatial memory: how egocentric and allocentric combine. Trends Cogn Sci 10:551–557

    PubMed  Google Scholar 

  • Burgess N (2008) Spatial cognition and the brain. Ann N Y Acad Sci 1124:77–97

    PubMed  Google Scholar 

  • Byrne P, Becker S, Burgess N (2007) Remembering the past and imagining the future: a neural model of spatial memory and imagery. Psychol Rev 114:340–375

    PubMed  Google Scholar 

  • Cattaneo Z, Vecchi T (2011) Blind Vision: The Neuroscience of Visual Impairment. MIT Press, Cambridge, MA

    Google Scholar 

  • Cavanna AE, Trimble MR (2006) The precuneus: a review of its functional anatomy and behavioral correlates. Brain 129:564–583

    PubMed  Google Scholar 

  • Chang SW, Papadimitriou C, Snyder LH (2009) Using a compound gain field to compute a reach plan. Neuron 64:744–755

    PubMed  CAS  Google Scholar 

  • Chebat DR, Schneider F, Kupers R, Ptito M (2011) Navigation with a sensory substitution device in congenitally blind individuals. Neuroreport 22:342–347

    PubMed  Google Scholar 

  • Denis M, Cocude M (1989) Scanning visual images generated from verbal descriptions. Eur J Cogn Psychol 1:293–307

    Google Scholar 

  • De Vega M, Rodrigo MJ (2001) Updating spatial layouts mediated by pointing and labelling under physical and imaginary rotation. Eur J Cogn Psychol 13:369–393

    Google Scholar 

  • Easton RD, Sholl MJ (1995) Object–array structure, frames of reference, and retrieval of spatial knowledge. J Exp Psychol Learn Mem Cogn 21:483–500

    PubMed  CAS  Google Scholar 

  • Farah MJ, Hammond KM, Levine DN, Calvanio R (1988) Visual and spatial mental imagery: dissociable systems of representation. Cogn Psychol 20:439–462

    PubMed  CAS  Google Scholar 

  • Farrell MJ, Robertson IH (1998) Mental rotation and the automatic updating of body-centered spatial relationships. J Exp Psychol Learn Mem Cogn 24:227–233

    Google Scholar 

  • Farrell MJ, Thomson JA (1999) On-line updating of spatial information during locomotion without vision. J Mot Behav 3:39–53

    Google Scholar 

  • Fernandez-Ruiz J, Goltz HH, DeSouza JFX, Vilis T, Crawford JD (2007) Human parietal “reach region” primarily encodes intrinsic visual direction, not extrinsic movement direction, in a visual–motor dissociation task. Cereb Cortex 17:2283–2292

    PubMed  Google Scholar 

  • Franklin N, Tversky B (1990) Searching imagined environments. J Exp Psychol Gen 119:63–76

    Google Scholar 

  • Fukusima SS, Loomis JM, Da Silva JA (1997) Visual perception of egocentric distance as assessed by triangulation. J Exp Psychol Hum Percept Perform 23:86–100

    PubMed  CAS  Google Scholar 

  • Giudice NA, Betty MR, Loomis JM (2011) Functional equivalence of spatial images from touch and vision: evidence from spatial updating in blind and sighted individuals. J Exp Psychol Learn Mem Cogn 37:621–634

    PubMed  Google Scholar 

  • Giudice NA, Klatzky RL, Loomis JM (2009) Evidence for amodal representations after bimodal learning: integration of haptic-visual layouts into a common spatial image. Spat Cogn Comput 9:287–304

    Google Scholar 

  • Giudice NA, Klatzky RL, Bennett CR, Loomis JM (in press) Combining locations from working memory and long-term memory into a common spatial image. Spat Cogn Comput

    Google Scholar 

  • Giudice NA, Legge GE (2008) Blind navigation and the role of technology. In: Helal A, Mokhtari M, Abdulrazak B (eds) Engineering Handbook of Smart Technology for Aging, Disability, and Independence. Wiley, New York

    Google Scholar 

  • Gordon MS, Rosenblum LD (2004) Perception of sound-obstructing surfaces using body-scaled judgments. Ecol Psychol 16:87–113

    Google Scholar 

  • Harrison AM (2007) Reversal of the Alignment Effect: Influence of Visualization and Spatial Set Size. In: Proceedings of the 29th Annual Conference of the Cognitive Science Society. Curran Associates, Red Hook, NY

    Google Scholar 

  • Hegarty M, Kozhevnikov M (1999) Types of visual-spatial representations and mathematical problem solving. J Educ Psychol 91:684–689

    Google Scholar 

  • Hodgson E, Waller D (2006) Lack of set size effects in spatial updating: evidence for offline updating. J Exp Psychol Learn Mem Cogn 32:854–866

    PubMed  Google Scholar 

  • Hollins M, Kelley EK (1988) Spatial updating in blind and sighted people. Percept Psychophys 43:380–388

    PubMed  CAS  Google Scholar 

  • Horn DL, Loomis JM (2004) Spatial updating of targets in front and behind. Paidéia 14:75–81

    Google Scholar 

  • Hughes B (2001) Active artificial echolocation and the nonvisual perception of aperture passability. Hum Mov Sci 20:371–400

    PubMed  CAS  Google Scholar 

  • Huttenlocher J, Hedges LV, Duncan S (1991) Categories and particulars: prototype effects in estimating spatial location. Psychol Rev 98:352–376

    PubMed  CAS  Google Scholar 

  • Kelly JW, Avraamides M, Loomis JM (2007) Sensorimotor alignment effects in the learning environment and in novel environments. J Exp Psychol Learn Mem Cogn 33:1092–1107

    PubMed  Google Scholar 

  • Klatzky RL, Lippa Y, Loomis JM, Golledge RG (2003) Encoding, learning, and spatial updating of multiple object locations specified by 3-D sound, spatial language, and vision. Exp Brain Res 149:48–61

    PubMed  Google Scholar 

  • Klatzky RL, Loomis JM, Beall AC, Chance SS, Golledge RG (1998) Spatial updating of ­self-position and orientation during real, imagined, and virtual locomotion. Psychol Sci 9:293–298

    Google Scholar 

  • Knauff M (2009) A neuro-cognitive theory of deductive relational reasoning with mental models and visual images. Spat Cogn Comput 9:109–137

    Google Scholar 

  • Koch C (2003) The Quest for Consciousness. Roberts and Co., Englewood, CO

    Google Scholar 

  • Kosslyn SM, Thompson WL (2003) When is early visual cortex activated during visual mental imagery? Psychol Bull 129:723–746

    PubMed  Google Scholar 

  • Kozhevnikov M, Hegarty M, Mayer R (2002) Revising the cognitive style dimension: evidence for two types of visualizers. Cogn Instr 20:47–77

    Google Scholar 

  • Kozhevnikov M, Kosslyn S, Shephard J (2005) Spatial versus object visualizers: a new characterization of visual cognitive style. Mem Cognit 33:710–726

    PubMed  Google Scholar 

  • Lacey S, Lin JB, Sathian K (2011) Object and spatial imagery dimensions in visuo-haptic representations. Exp Brain Res 213:267–273

    PubMed  Google Scholar 

  • Lehar S (2003) The World in your Head: A Gestalt View of the Mechanism of Conscious experience. Erlbaum, Mahwah, NJ

    Google Scholar 

  • Levesque V (2009) Virtual display of tactile graphics and Braille by lateral skin deformation. Unpublished manuscript, Department of Electrical and Computer Engineering, McGill University, Montreal, Canada. Available online at http://vlevesque.com/papers/Levesque-PhD.pdf

  • Levine M, Jankovic IN, Palij M (1982) Principles of spatial problem solving. J Exp Psychol Gen 111:157–175

    Google Scholar 

  • Lyon DR, Gunzelmann GM (2011) Functional equivalence and spatial path memory. Q J Exp Psychol 64:2081–2087

    Google Scholar 

  • Loarer E, Savoyant A (1991) Visual Imagery in Locomotor Movement Without Vision. In: Logie RH, Denis M (eds) Mental Images in Human Cognition. Elsevier, The Hague

    Google Scholar 

  • Loomis JM (1992) Distal attribution and presence. Presence (Camb) 1:113–119

    Google Scholar 

  • Loomis JM, Beall AC (2004) Model-based control of perception/action. In: Vaina L, Beardsley S, Rushton S (eds) Optic Flow and Beyond. Kluwer Academic Publishers, Boston, MA

    Google Scholar 

  • Loomis JM, Da Silva JA, Fujita N, Fukusima SS (1992) Visual space perception and visually directed action. J Exp Psychol Hum Percept Perform 18:906–921

    PubMed  CAS  Google Scholar 

  • Loomis JM, Klatzky RL (2007) Functional equivalence of spatial representations from vision, touch, and hearing: relevance for sensory substitution. In: Rieser JJ, Ashmead DH, Ebner FF, Corn AL (eds) Blindness and Brain Plasticity in Navigation and Object Perception. Lawrence Erlbaum Associates, New York

    Google Scholar 

  • Loomis JM, Klatzky RL, Giudice NA (2012) Sensory substitution of vision: importance of perceptual and cognitive processing. In: Manduchi R, Kurniawan S (eds) Assistive Technology for Blindness and Low Vision. CRC, Boca Raton, FL

    Google Scholar 

  • Loomis JM, Klatzky RL, Philbeck JW, Golledge RG (1998) Assessing auditory distance perception using perceptually directed action. Percept Psychophys 60:966–980

    PubMed  CAS  Google Scholar 

  • Loomis JM, Lippa Y, Klatzky RL, Golledge RG (2002) Spatial updating of locations specified by 3-D sound and spatial language. J Exp Psychol Learn Mem Cogn 28:335–345

    PubMed  Google Scholar 

  • Loomis JM, Philbeck JW (2008) Measuring spatial perception with spatial updating and action. In: Klatzky RL, Behrmann M, MacWhinney B (eds) Embodiment, Ego-Space, and Action. Taylor & Francis, New York

    Google Scholar 

  • May M (2004) Imaginal perspective switches in remembered environments: transformation versus interference accounts. Cogn Psychol 48:163–206

    PubMed  Google Scholar 

  • Mazard A, Tzourio-Mazoyer N, Crivello F, Mazoyer B, Mellet E (2004) A PET meta-analysis of object and spatial imagery. Eur J Cogn Psychol 16:673–695

    Google Scholar 

  • McNamara TP (2003) How are the locations of objects in the environment represented in memory? In: Freksa C, Brauer W, Habel C, Wender K (eds) Spatial Cognition III: Routes and Navigation, Human Memory and Learning, Spatial Representation and Spatial Reasoning. Springer, Berlin

    Google Scholar 

  • Medendorp WP, Tweed DB, Crawford JD (2003) Motion parallax is computed in the updating of human spatial memory. J Neurosci 23:8135–8142

    PubMed  CAS  Google Scholar 

  • Milner AD, Goodale MA (2008) Two visual systems re-viewed. Neuropsychologia 46:774–785

    PubMed  CAS  Google Scholar 

  • Motes MA, Malach R, Kozhevnikov M (2008) Object-processing neural efficiency differentiates object from spatial visualizers. Neuroreport 19:1727–1731

    PubMed  Google Scholar 

  • Mou W, McNamara TP, Valiquette CM, Rump B (2004) Allocentric and egocentric updating of spatial memories. J Exp Psychol Learn Mem Cogn 30:142–157

    PubMed  Google Scholar 

  • Newell FN, Woods AT, Mernagh M, Bülthoff HH (2005) Visual, haptic and crossmodal recognition of scenes. Exp Brain Res 161:233–242

    PubMed  Google Scholar 

  • O’Keefe J, Nadel L (1978) The Hippocampus as a Cognitive Map. Oxford University Press, Oxford

    Google Scholar 

  • Ooi TL, Wu B, He ZJJ (2001) Distance determined by the angular declination below the horizon. Nature 414:197–200

    PubMed  CAS  Google Scholar 

  • Ooi TL, Wu B, He ZJ (2006) Perceptual space in the dark affected by the intrinsic bias of the visual system. Perception 35:605–624

    PubMed  Google Scholar 

  • Pascual-Leone A, Hamilton R (2001) The metamodal organization of the brain. In: Casanova C, Ptito M (eds) Vision: from Neurons to Cognition (vol 134, Ch 27: Progress in Brain Research). Elsevier, Amsterdam

    Google Scholar 

  • Pham QC, Hicheur H (2009) On the open-loop and feedback processes that underlie the formation of trajectories during visual and nonvisual locomotion in humans. J Neurophysiol 102:2800–2815

    PubMed  Google Scholar 

  • Philbeck JW, Loomis JM (1997) Comparison of two indicators of visually perceived egocentric distance under full-cue and reduced-cue conditions. J Exp Psychol Hum Percept Perform 23:72–85

    PubMed  CAS  Google Scholar 

  • Philbeck JW, Loomis JM, Beall AC (1997) Visually perceived location is an invariant in the control of action. Percept Psychophys 59:601–612

    PubMed  CAS  Google Scholar 

  • Presson CC, Montello DR (1994) Updating after rotational and translational body movements: coordinate structure of perspective space. Perception 23:1447–1455

    PubMed  CAS  Google Scholar 

  • Riecke BE, von der Heyde M, Bülthoff HH (2005) Visual cues can be sufficient for triggering automatic, reflex-like spatial updating. ACM Trans Appl Percept 2:183–215

    Google Scholar 

  • Rieser JJ (1989) Access to knowledge of spatial structure at novel points of observation. J Exp Psychol Learn Mem Cogn 15:1157–1165

    PubMed  CAS  Google Scholar 

  • Rieser JJ, Ashmead DH, Talor CR, Youngquist GA (1990) Visual perception and the guidance of locomotion without vision to previously seen targets. Perception 19:675–689

    PubMed  CAS  Google Scholar 

  • Rieser JJ, Garing AE, Young MF (1994) Imagery, action, and young children’s spatial orientation: it’s not being there that counts, it’s what one has in mind. Child Dev 65:1262–1278

    PubMed  CAS  Google Scholar 

  • Rieser JJ, Guth DA, Hill EW (1986) Sensitivity to perspective structure while walking without vision. Perception 15:173–188

    PubMed  CAS  Google Scholar 

  • Rieser JJ, Rider E (1991) Young children’s spatial orientation with respect to multiple targets when walking without vision. Dev Psychol 27:97–107

    Google Scholar 

  • Röder B, Rösler F (1998) Visual input does not facilitate the scanning of spatial images. J Mental Imagery 22:165–182

    Google Scholar 

  • Russell B (1948) Human Knowledge: Its Scope and Limits. Simon and Schuster, New York

    Google Scholar 

  • Sahm CS, Creem-Regehr SH, Thompson WB, Willemsen P (2005) Throwing versus walking as indicators of distance perception in similar real and virtual environments. ACM Trans Appl Percept 2:35–45

    Google Scholar 

  • Shelton AL, McNamara TP (2004) Spatial memory and perspective taking. Mem Cognit 32:416–426

    PubMed  Google Scholar 

  • Shepard RN, Metzler J (1971) Mental rotation of three-dimensional objects. Science 171:701–703

    PubMed  CAS  Google Scholar 

  • Siegle JH, Campos JL, Mohler BJ, Loomis JM, Bülthoff HH (2009) Measurement of instantaneous perceived self-motion using continuous pointing. Exp Brain Res 195:429–444

    PubMed  Google Scholar 

  • Smythies JR (1994) The walls of Plato’s Cave. The Science and Philosophy of Brain, Consciousness, and Perception. Avebury, Aldershot

    Google Scholar 

  • Stevenson RJ, Case TI (2005) Olfactory imagery: a review. Psychon Bull Rev 12:244–264

    PubMed  Google Scholar 

  • Struiksma ME, Noordzij ML, Postma A (2009) What is the link between language and spatial images? Behavioral and neural findings in blind and sighted individuals. Acta Psychol 132:145–156

    Google Scholar 

  • Tatler BW, Land MF (2011) Vision and the representation of the surroundings in spatial memory. Philos Trans R Soc B 366:596–610

    Google Scholar 

  • Taylor HA, Tversky B (1992) Spatial mental models derived from survey and route descriptions. J Mem Lang 31:261–282

    Google Scholar 

  • Teng S, Puri A, Whitney D (2012) Ultrafine spatial acuity of blind expert human echolocators. Exp Brain Res 216(4):483–488

    PubMed  Google Scholar 

  • Thomson JA (1983) Is continuous visual monitoring necessary in visually guided locomotion? J Exp Psychol Hum Percept Perform 9:427–443

    PubMed  CAS  Google Scholar 

  • Thompson WB, Willemsen P, Gooch AA, Creem-Regehr SH, Loomis JM, Beall AC (2004) Does the quality of the computer graphics matter when judging distances in visually immersive environments. Presence (Camb) 13:560–571

    Google Scholar 

  • Tolman EC (1948) Cognitive maps in rats and men. Psychol Rev 55:189–208

    PubMed  CAS  Google Scholar 

  • Waller D, Hodgson E (2006) Transient and enduring spatial representations under disorientation and self-rotation. Mem Cognit 32:867–882

    Google Scholar 

  • Waller D, Montello DR, Richardson AE, Hegarty M (2002) Orientation specificity and spatial updating of memories for layouts. J Exp Psychol Learn Mem Cogn 28:1051–1063

    PubMed  Google Scholar 

  • Wang RF, Spelke ES (2000) Updating egocentric representations in human navigation. Cognition 77:215–250

    PubMed  CAS  Google Scholar 

  • Wang RF, Crowell JA, Simons DJ, Irwin DE, Kramer AF, Ambinder MS et al (2006) Spatial updating relies on an egocentric representation of space: effects of the number of objects. Psychon Bull Rev 13:281–286

    PubMed  CAS  Google Scholar 

  • Wiener JM, Berthoz A, Wolbers T (2010) Dissociable cognitive mechanisms underlying human path integration. Exp Brain Res 208:61–71

    PubMed  Google Scholar 

  • Wolbers T, Hegarty M, Büchel C, Loomis JM (2008) Spatial updating: How the brain keeps track of changing object locations during observer motion. Nat Neurosci 11:1223–1230

    PubMed  CAS  Google Scholar 

  • Wu B, Ooi TL, He ZJ (2004) Perceiving distances accurately by a directional process of integrating ground information. Nature 428:73–77

    PubMed  CAS  Google Scholar 

  • Zwaan RA, Radvansky GA (1998) Situation models in language comprehension and memory. Psychol Bull 123:162–185

    PubMed  CAS  Google Scholar 

Download references

Acknowledgments

Preparation of this chapter was supported by NIH grant R01EY016817 awarded to the three authors. Roberta Klatzky also acknowledges the support of the Alexander von Humboldt Foundation. The authors thank Rebecca Lawson, Simon Lacey, Maria Kozhevnikov, Marios Avraamides, and Jonathan Kelly for helpful comments on earlier versions of the manuscript.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jack M. Loomis .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Loomis, J.M., Klatzky, R.L., Giudice, N.A. (2013). Representing 3D Space in Working Memory: Spatial Images from Vision, Hearing, Touch, and Language. In: Lacey, S., Lawson, R. (eds) Multisensory Imagery. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-5879-1_8

Download citation

Publish with us

Policies and ethics