Abstract
In Experiment 1, participants were presented with pairs of stimuli (one visual and the other tactile) from the left and/or right of fixation at varying stimulus onset asynchronies and were required to make unspeeded temporal order judgments (TOJs) regarding which modality was presented first. When the participants adopted an uncrossed-hands posture, just noticeable differences (JNDs) were lower (i.e., multisensory TOJs were more precise) when stimuli were presented from different positions, rather than from the same position. This spatial redundancy benefit was reduced when the participants adopted a crossed-hands posture, suggesting a failure to remap visuotactile space appropriately. In Experiment 2, JNDs were also lower when pairs of auditory and visual stimuli were presented from different positions, rather than from the same position. Taken together, these results demonstrate that people can use redundant spatial cues to facilitate their performance on multisensory TOJ tasks and suggest that previous studies may have systematically overestimated the precision with which people can make such judgments. These results highlight the intimate link between spatial and temporal factors in determining our perception of the multimodal objects and events in the world around us.
Article PDF
Similar content being viewed by others
References
Allen, P. G., &Kolers, P. A. (1981). Sensory specificity of apparent motion.Journal of Experimental Psychology: Human Perception & Performance,7, 1318–1326.
Anllo-Vento, L., &Hillyard, S. A. (1996). Selective attention to the color and direction of moving stimuli: Electrophysiological correlates of hierarchical feature selection.Perception & Psychophysics,58, 191–206.
Anzola, G. P., Bertoloni, G., Buchtel, H. A., &Rizzolatti, G. (1977). Spatial compatibility and anatomical factors in simple and choice reaction time.Neuropsychologia,15, 295–302.
Bald, L., Berrien, F. K., Price, J. B., &Sprague, R. O. (1942). Errors in perceiving the temporal order of auditory and visual stimuli.Journal of Applied Psychology,26, 382–388.
Bertelson, P. (1998). Starting from the ventriloquist: The perception of multimodal events. In M. Sabourin, F. I. M. Craik, & M. Robert (Eds.),Advances in psychological science, Vol. 2. Biological and cognitive aspects (pp. 419–439). Hove, U.K.: Psychological Press. or]Bertelson, P., & Aschersleben, G. (2001, January).Crossmodal interaction on the time dimension: Evidence from auditory-visual time order judgment. Paper presented at the London meeting of the Experimental Psychology Society.
Bishop, C. M. (1995).Neural networks for pattern recognition. Oxford: Oxford University Press, Clarendon Press.
Bloch, A. M. (1887). Exnerschen ähnliche Versuche mit analogen Ergebnissen [Replication of Exner’s studies with analogous results].Revue Scientifique,XXXIX, S.585.
Bradshaw, J. L., Howard, M. J., Pierson, J. M., Phillips, J., &Bradshaw, J. A. (1992). Effects of expectancy and attention in vibrotactile choice reaction time tasks.Quarterly Journal of Experimental Psychology,44A, 509–528.
Burtt, H. E. (1917). Tactile illusions of movement.Journal of Experimental Psychology,2, 371–385.
Bushara, K. O., Grafman, J., &Hallett, M. (2001). Neural corre lates of auditory-visual stimulus onset asynchrony detection.Journal of Neuroscience,21, 300–304.
Carmon, A., &Nachshon, I. (1971). Effect of unilateral brain damage on perception of temporal order.Cortex,7, 410–418.
Coren, S., Ward, L. M., &Enns, J. T. (1999).Sensation and perception (5th Ed.). Fort Worth: Harcourt Brace.
Dennett, D. C. (1992). Temporal anomalies of consciousness. In Y. Christen & P. S. Churchland (Eds.),Neurophilosophy and Alzheimer’s disease. Berlin: Springer-Verlag.
Dinnerstein, A. J., &Blitz, B. (1964). Perceptual delays and speed of reading and tapping.Perceptual & Motor Skills,19, 867–873.
Dinnerstein, A. J., &Zlotogura, P. (1968). Intermodal perception of temporal order and motor skills: Effects of age.Perceptual & Motor Skills,26, 987–1000.
Driver, J., &Spence, C. (2000). Multisensory perception: Beyond modularity and convergence.Current Biology,10, R731-R735.
Duncan, J. (1984). Selective attention and the organization of visual information.Journal of Experimental Psychology: General,113, 501–517.
Efron, R. (1963). The effect of handedness on the perception of simultaneity and temporal order.Brain,86, 261–284.
Egly, R., Driver, J., &Rafal, R. D. (1994). Shifting visual attention between objects and locations: Evidence from normal and parietal lesion subjects.Journal of Experimental Psychology: General,123, 161–177.
Estes, W. K. (1956). The problem of interference from curves based on group data.Psychological Bulletin,53, 134–140.
Exner, S. (1875). Experimentelle Untersuchung der einfachsten psychischen Processe [Experimental study of the most simple psychological processes].Archiv für die geschichte Physiologie (Pflüger’s Archive),11, 403–432.
Fendrich, R., &Corballis, P. M. (2001). The temporal cross-capture of audition and vision.Perception & Psychophysics,63, 719–725.
Finney, D. J. (1964).Probit analysis: Statistical treatment of the sigmoid response curve (2nd ed.). Cambridge: Cambridge University Press.
Fraisse, P. (1963).The psychology of time. Evanston, NY: Harper & Row.
Gengel, R. W., &Hirsh, I. J. (1970). Temporal order: The effect of single versus repeated presentations, practice, and verbal feedback.Perception & Psychophysics,7, 209–211.
Grossberg, S., &Grunewald, A. (1997). Cortical synchronization and perceptual framing.Journal of Cognitive Neuroscience,9, 117–132.
Guinzberg, R. L. (1928). È possibile l’apprendimento di sensazioni eterogenee come perfettamente simultanee? (Is it possible to learn that heterogenous sensations are perfectly simultaneous?)Archivi Italiani di Psicologia,6, 103–114.
Hamlin, A. J. (1895). On the least observable interval between stimuli addressed to disparate senses and to different organs of the same sense.American Journal of Psychology,6, 564–575.
Hirsh, I. J. (1988). Auditory perception and speech. In R. C. Atkinson, R. J. Herrnstein, G. Lindsey, & R. D. Luce (Eds.),Stevens’ Handbook of experimental psychology: Perception and motivation (2nd ed., pp. 377–408). New York: Wiley.
Hirsh, I. J., &Fraisse, P. (1964). Simultanéité et succession de stimuli hétérogènes [Simultaneity and succession of heterogeneous stimuli].L’Année Psychologique,64, 1–19.
Hirsh, I. J., &Sherrick, C. E., Jr.. (1961). Perceived order in different sense modalities.Journal of Experimental Psychology,62, 423–432.
Jaśkowski, P., Jaroszyk, F., &Hojan-Jerierska, D. (1990). Temporalorder judgments and reaction time for stimuli of different modalities.Psychological Research,52, 35–38.
Kennett, S., Eimer, M., Spence, C., &Driver, J. (2001). Tactile-visual links in exogenous spatial attention under different postures: Convergent evidence from psychophysics and ERPs.Journal of Cognitive Neuroscience,13, 462–478.
Kennett, S., Spence, C., &Driver, J. (2002). Visuo-tactile links in covert exogenous spatial attention remap across changes in unseen hand posture.Perception & Psychophysics,64, 1083–1094.
King, A. J., &Palmer, A. R. (1985). Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus.Experimental Brain Research,60, 492–500.
McFarland.D. J., Cacace, A. T., &Setzen, G. (1998). Temporalorder discrimination for selected auditory and visual stimulus dimensions.Journal of Speech, Language, & Hearing Research,41, 300–314.
Meredith, M. A., &Stein, B. E. (1983). Interactions among converging sensory inputs in the superior colliculus.Science,221, 389–391.
Millar, S. (1994).Understanding and representing space: Theory and evidence from studies with blind and sighted children. Oxford: Oxford University Press.
Morein-Zamir, S., Soto-Faraco, S., &Kingstone, A. F. (2001). Captured vision: Sounds influence the perception of visual temporal order.Abstracts of the Psychonomic Society,6, 91.
Moutoussis, K., &Zeki, S. (1997). A direct demonstration of perceptual asynchrony in vision.Proceedings of the Royal Society of London: Series B,264, 393–399.
Neisser, U. (1976).Cognition and reality. San Francisco: Freeman.
Nelder, J. A., & Mead, R. (1965). A simplex method for function minimisation.Computer Journal,7, 308–313.
Nishida, S., &Johnston, A. (2002). Marker correspondence, not processing latency, determines temporal binding of visual attributes.Current Biology,12, 359–368.
O’Connor, N., &Hermelin, B. (1978).Seeing and hearing and space and time. New York: Academic Press.
O’Craven, K. M., Downing, P. E., &Kanwisher, N. (1999). fMRI evidence for objects as the units of attentional selection.Nature,401, 584–587.
Piéron, H. (1952).The sensations: Their functions, processes and mechanisms. New Haven: Yale University Press.
Pöppel, E. (1985).Grenzen des Bewußtseins. Stuttgart: Deutsche Verlags-Anstal. [Translated asMindworks: Time and conscious experience (1988). New York: Harcourt Brace Jovanovich.]
Pöppel, E., Schill, K., &Steinbüchel, N. von (1990). Sensory integration within temporally neutral system states: A hypothesis.Naturwissenschaften,77, 89–91. or]Raizada, R. D. S., & Poldrack, R. A. (2001). Event-related fMRI of audio-visual simultaneity perception.Society for Neuroscience Abstracts, Poster No. 511.14.
Rao, S. M., Mayer, A. R., &Harrington, D. L. (2001). The evolution of brain activation during temporal processing.Nature Neuroscience,4, 317–323.
Robertson, G. E., Hairston, W. D., Wallace, M. T., Stein, B. E., Laurienti, P. J., & Schirillo, J. A. (2001). Unifying multisensory signals across time and space.Society for Neuroscience Abstracts, Poster No. 511.19.
Röder, B., Spence, C., &Rösler, F. (2002). Assessing the effect of posture changes on tactile inhibition of return.Experimental Brain Research,143, 453–462.
Rose, S. A., Feldman, J. F., Jankowski, J. J., &Futterweit, L. R. (1999). Visual and auditory temporal processing, cross-modal transfer, and reading.Journal of Learning Disabilities,32, 256–266.
Rutschmann, J., &Link, R. (1964). Perception of temporal order of stimuli differing in sense mode and simple reaction time.Perceptual & Motor Skills,18, 345–352.
Sade, A., &Spitzer, H. (1998). The effects of attentional spread and attentional effort on orientation discrimination.Spatial Vision,11, 367–383.
Scheier, C. R., Nijhawan, R., &Shimojo, S. (1999). Sound alters visual temporal resolution [Abstract].Investigative Ophthalmology & Visual Science,40, S792.
Schröger, E., &Wolff, C. (1997). Fast preattentive processing of location: A functional basis for selective listening in humans.Neuroscience Letters,232, 5–8.
Sherrick, C. E. (1976). The antagonisms of hearing and touch. In S. K. Hirsh, D. H. Eldredge, I. J. Hirsh, & S. R. Silverman (Eds.),Hearing and Davis: Essays honoring Hallowell Davis (pp. 149–158). St. Louis: Washington University Press.
Shore, D. I., Spence, C., &Klein, R. M. (2001). Visual prior entry.Psychological Science,12, 205–212.
Shore, D. I., Spry, E., &Spence, C. (2002a). Confusing the mind by crossing the hands.Cognitive Brain Research,14, 153–163.
Shore, D. I., Spry, E., & Spence, C. (2002b).Spatial modulation of tactile temporal order judgments. Manuscript submitted for publication
Simon, J. R. (1990). The effects of an irrelevant directional cue on human information processing. In R.W. Proctor & T. G. Reeve (Eds.),Stimulus-response compatibility (pp. 31–86). Amsterdam: Elsevier.
Slutsky, D. A., &Recanzone, G. H. (2001). Temporal and spatial dependency of the ventriloquism effect.NeuroReport,12, 7–10.
Smith, W. F. (1933). The relative quickness of visual and auditory perception.Journal of Experimental Psychology,16, 239–257.
Spence, C., &Driver, J. (1997). On measuring selective attention to an expected sensory modality.Perception & Psychophysics,59, 389–403.
Spence, C., Nicholls, M. E. R., &Driver, J. (2001). The cost of expecting events in the wrong sensory modality.Perception & Psychophysics,63, 330–336.
Spence, C., Nicholls, M. E. R., Gillespie, N., &Driver, J. (1998). Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision.Perception & Psychophysics,60, 544–557.
Spence, C., Pavani, F., &Driver, J. (2000). Crossmodal links between vision and touch in covert endogenous spatial attention.Journal of Experimental Psychology: Human Perception & Performance,26, 1298–1319.
Spence, C., Shore, D. I., &Klein, R. M. (2001). Multisensory prior entry.Journal of Experimental Psychology: General,130, 799–832.
Stein, B. E., &Meredith, M. A. (1993). The merging of the senses. Cambridge, MA: MIT Press.
Stone, J. V., Hunkin, N. M., Porrill, J., Wood, R., Keeler, V., Beanland, M., Port, M., &Porter, N. R. (2001). When is now? Perception of simultaneity.Proceedings of the Royal Society of London: Series B,268, 31–38.
Strybel, T. Z., Manligas, C. L., Chan, O., &Perrott, D. R. (1990). A comparison of the effects of spatial separation on apparent motion in the auditory and visual modalities.Perception & Psychophysics,47, 439–448.
Swanson, W. H., &Birch, E. E. (1992). Extracting thresholds from noisy psychophysical data.Perception & Psychophysics,51, 409–422. or]Taylor, B. (1978).Dimensional redundancy in the processing of vibrotactile temporal order. Unpublished doctoral dissertation, Princeton University.
Teatini, G., Farnè, M., Verzella, F., &Berruecos, P., Jr. (1976). Perception of temporal order: Visual and auditory stimuli.Giornale Italiano di Psicologia,3, 157–164.
Urbach, D., &Spitzer, H. (1995). Attentional effort modulated by task difficulty.Vision Research,35, 2169–2177.
van de Par, S., Kohlrausch, A., &Juola, J. F. (1999). Judged synchrony/asynchrony for light-tone pairs.Abstracts of the Psychonomic Society,4, 60.
Watt, R. J. (1991).Understanding vision. London: Academic Press.
Weiland, B. A. (1960). The interaction of space and time in cutaneous perception.American Journal of Psychology,73, 248–255.
Welch, R. B., DuttonHurt, L. D., &Warren, D. H. (1986). Contributions of audition and vision to temporal rate perception.Perception & Psychophysics,39, 294–300.
Welch, R. B., &Warren, D. H. (1980). Immediate perceptual response to intersensory discrepancy.Psychological Bulletin,3, 638–667.
Whipple, G. M., Sanford, E. C., &Colgrove, F. W. (1899). Minor studies from the psychological laboratory of Clark University: On nearly simultaneous clicks and flashes. The time required for recognition: Notes on mental standards of length.American Journal of Psychology,10, 280–295.
Widmann, A., Scharf, B., &Schröger, E. (2000). Frequency and location discrimination: Which is faster? In A. Schick, M. Meis, & C. Reckhardt (Eds.),Contributions to Psychological Acoustics: Results of the Eighth Oldenburg Symposium on Psychological Acoustics (pp. 373–380). Oldenburg: Bibliotheks- und Informationssystem der Universität Oldenburg.
Yamamoto, S., &Kitazawa, S. (2001a). Reversal of subjective temporal order due to arm crossing.Nature Neuroscience,4, 759–765.
Yamamoto, S., &Kitazawa, S. (2001b). Sensation at the tips of invisible tools.Nature Neuroscience,4, 979–980.
Zampini, M., Guest, S., Shore, D. I., & Spence, C. (2003).Audiovisual synchrony judgments. Manuscript submitted for publication.
Author information
Authors and Affiliations
Corresponding authors
Additional information
C.S. and D.I.S. were funded by a Network Grant from the McDonnell-Pew Centre for Cognitive Neuroscience, University of Oxford, and D.I.S. was funded by a grant from the Natural Science and Engineering Research Council of Canada.
Rights and permissions
About this article
Cite this article
Spence, C., Baddeley, R., Zampini, M. et al. Multisensory temporal order judgments: When two locations are better than one. Perception & Psychophysics 65, 318–328 (2003). https://doi.org/10.3758/BF03194803
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03194803