Davide Rocchesso
Stefano Delle Monache
Skip Abstract Section
Abstract
As tables, boards, and walls become surfaces where interaction can be supported by auditory displays, it becomes important to know how accurately and effectively a spatial gesture can be rendered by means of an array of loudspeakers embedded in the surface. Two experiments were designed and performed to assess: (i) how sequences of sound pulses are perceived as gestures when the pulses are distributed in space and time along a line; (ii) how the timing of pulses affects the perceived and reproduced continuity of sequences; and (iii) how effectively a second parallel row of speakers can extend sonic gestures to a two-dimensional space. Results show that azimuthal trajectories can be effectively replicated and that switching between discrete and continuous gestures occurs within the range of inter-pulse interval from 75 to 300ms. The vertical component of sonic gestures cannot be reliably replicated.
- Adcock, M. and Barrass, S. 2004. Cultivating design patterns for auditory displays. In Proceedings of the International Conference on Auditory Display.Google Scholar
- Andersen, T. H. and Zhai, S. 2008. “Writing with music”: Exploring the use of auditory feedback in gesture interfaces. ACM Trans. Appl. Percept. 7, 17:1--17:24. Google ScholarDigital Library
- Bakker, S., van den Hoven, E., and Eggen, B. 2010. Exploring interactive systems using peripheral sounds. In Haptic and Audio Interaction Design, R. Nordahl et al. eds., Lecture Notes in Computer Science, vol. 6306. Springer, Berlin, 55--64. Google ScholarDigital Library
- Begault, D. R., Anderson, M. R., and McClain, M. U. 2006. Spatially modulated auditory alerts for aviation. J. Audio Eng. Soc 54, 4, 276--282.Google Scholar
- Bremer, C. D., Pittenger, J. B., Warren, R., and Jenkins, J. J. 1977. An illusion of auditory saltation similar to the cutaneous “rabbit”. Amer. J. Psychol. 90, 4, 645--654.Google ScholarCross Ref
- Brown, L. and Brewster, S. 2003. Drawing by ear: Interpreting sonified line graphs. In Proceedings of the International Conference on Auditory Display. E. Brazil and B. Shinn-Cunningham, Eds., 152--156.Google Scholar
- Butler, R. A. and Belendiuk, K. 1977. Spectral cues utilized in the localization of sound in the median sagittal plane. J. Acoustical Soc. Amer. 61, 5, 1264--1269.Google ScholarCross Ref
- Caramiaux, B., Susini, P., Bianco, T., Bevilacqua, F., Houix, O., Schnell, N., and Misdariis, N. 2011. Gestural embodiment of environmental sounds: An experimental study. In Proceedings of the International Conference on New Interfaces for Musical Expression. A. R. Jensenius et al., Eds., 144--148Google Scholar
- Carlile, S. and Best, V. 2002. Discrimination of sound source velocity in human listeners. J. Acoustical Soc. Amer. 111, 2, 1026--1035.Google ScholarCross Ref
- De Bruyn, L., Moelants, D., and Leman, M. 2011. An embodied approach to testing musical empathy in participants with an autism spectrum disorder. Music and Medicine 4, 28--36.Google ScholarCross Ref
- Delle Monache, S., Polotti, P., and Rocchesso, D. 2010. A toolkit for explorations in sonic interaction design. In Proceedings of the 5th Audio-Mostly Conference: A Conference on Interaction with Sound (AM'10), ACM, New York, 1:1--1:7. Google ScholarDigital Library
- Delle Monache, S. and Rocchesso, D. 2011. The curse of the where-rabbit: research through design of auditory trajectories. In Proceedings of the 9th ACM SIGCHI Italian Chapter International Conference on Computer-Human Interaction: Facing Complexity (CHItaly). ACM, New York, 79--84. Google ScholarDigital Library
- Desmet, F., Leman, M., Lesaffre, M., and De Bruyn, L. 2010. Statistical analysis of human body movement and group interactions in response to music. In Advances in Data Analysis, Data Handling and Business Intelligence, A. Fink et al., Eds., Springer, Berlin, 399--408.Google Scholar
- Deutsch, D. 2010. Hearing music in ensembles. Physics Today 63, 2, 40--45.Google ScholarCross Ref
- Erkut, C., Iylha, A., and Rocchesso, D. 2013. Heigh ho: Rhythmicity in sonic interaction. In Sonic Interaction Design, K. Franinovic and S. Serafin, Eds., MIT Press, Cambridge, MA. To appear.Google Scholar
- Getzmann, S. 2008. The effect of spectral difference on auditory saltation. Experiment. Psychol. 55, 1, 64--71.Google ScholarCross Ref
- Goldreich, D. 2007. A Bayesian perceptual model replicates the cutaneous rabbit and other tactile spatiotemporal illusions. PLoS ONE 2, 3, e333+.Google Scholar
- Hartmann, W. M. 1999. How we localize sound. Phys. Today 52, 11, 24--29.Google ScholarCross Ref
- Ihlefeld, A. and Shinn-Cunningham, B. G. 2011. Effect of source spectrum on sound localization in an everyday reverberant room. J. Acoustical Soc. Amer. 130, 1, 324+.Google ScholarCross Ref
- Jensenius, A. R., Wanderley, M. M., Godøy, R. I., and Leman, M. 2010. Musical gestures: concepts and methods inresearch. In Musical Gestures: Sound, Movement, and Meaning, R. I. Godøy and M. Leman, Eds., Routledge, New York, 12--35.Google Scholar
- Kilian, K. 2009. From brand identity to audio branding. In Audio Branding. Brands, Sound and Communication, K. Bronner and R. Hirt, Eds., Nomos, Baden-Baden, 35--49.Google Scholar
- Kubovy, M. and Schutz, M. 2010. Audio-visual objects. Rev. Philos. Psychol. 1, 41--61. 10.1007/s13164-009-0004-5.Google ScholarCross Ref
- Lakatos, S. 1993a. Recognition of complex auditory-spatial patterns. Perception 22, 3, 363--374.Google ScholarCross Ref
- Lakatos, S. 1993b. Temporal constraints on apparent motion in auditory space. Percept. Psychophys. 54, 2, 139--144.Google ScholarCross Ref
- Lemmens, P., Crompvoets, F., Brokken, D., van den Eerenbeemd, J., and de Vries, G.-J. 2009. A body-conforming tactile jacket to enrich movie viewing. In Proceedings of the World Haptics - 3rd Joint EuroHaptics Cconference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. IEEE, Los Alamitos, CA, 7--12. Google ScholarDigital Library
- Mills, A. W. 1958. On the minimum audible angle. J. Acoustical Soc. Amer. 30, 4, 237--246.Google ScholarCross Ref
- Muller, M. 2007. Information Retrieval for Music and Motion. Springer, Berlin. Google ScholarDigital Library
- Muller-Tomfelde, C. and Steiner, S. 2001. Audio-enhanced collaboration at an interactive electronic whiteboard. In Proceedings of the International Conference on Auditory Display. 267--271.Google Scholar
- Neuhoff, J. 2004. Auditory motion and localization. In Ecological Psychoacoustics, J. Neuhoff, Ed., Academic Press, New York, 87--111.Google Scholar
- Patel, N. S. and Hughes, D. E. 2012. Revolutionizing human-computer interfaces: The auditory perspective. Interactions 19, 34--37. Google ScholarDigital Library
- Perrott, D. R. and Saberi, K. 1990. Minimum audible angle thresholds for sources varying in both elevation and azimuth. J. Acoustical Soc. Amer. 87, 4, 1728--1731.Google ScholarCross Ref
- Phillips, D. P. and Hall, S. E. 2001. Spatial and temporal factors in auditory saltation. J. Acoustical Soc. Amer. 110, 3, 1539--1547.Google ScholarCross Ref
- Raisamo, J., Raisamo, R., and Surakka, V. 2009. Evaluating the effect of temporal parameters for vibrotactile saltatory patterns. In Proceedings of the International Conference on Multimodal Interfaces (ICMI-MLMI'09), ACM, New York, 319--326. Google ScholarDigital Library
- Rakerd, B. and Hartmann, W. M. 1985. Localization of sound in rooms, ii: The effects of a single reflecting surface. J. Acoustical Soc. Amer. 78, 2, 524--533.Google ScholarCross Ref
- Rocchesso, D. and Delle Monache, S. 2011. Spatio-temporal unfolding of sound sequences. In Proceedings of the Sound and Music Computing Conference, 265--272.Google Scholar
- Ruff, R. M. and Perret, E. 1982. Spatial mapping of two-dimensional sound patterns presented sequentially. Percept. Motor Skills 55, 1, 155--163.Google ScholarCross Ref
- Rusconi, E., Kwan, B., Giordano, B. L., Umilta, C., and Butterworth, B. 2006. Spatial representation of pitch height: The SMARC effect. Cognition 99, 2, 113--129.Google Scholar
- Saberi, K. and Perrott, D. R. 1990. Minimum audible movement angles as a function of sound source trajectory. J. Acoustical Soc. Amer. 88, 6, 2639--2644.Google ScholarCross Ref
- Shams, L., Kamitani, Y., and Shimojo, S. 2002. Visual illusion induced by sound. Cognitive Brain Res. 14, 1, 147--152.Google ScholarCross Ref
- Shore, D. I., Hall, S. E., and Klein, R. M. 1998. Auditory saltation: A new measure for an old illusion. J. Acoustical Soc. Amer. 103, 6, 3730--3733.Google ScholarCross Ref
- Tolkmitt, F. J. and Brindley, R. 1977. Auditory perception of spatiotemporal patterns. Amer. J. Psychol. 90, 1, 73--83.Google ScholarCross Ref
- Van Valkenburg, D. and Kubovy, M. 2003. In defense of the theory of indispensable attributes. Cognition 87, 225--233.Google ScholarCross Ref
- Wang, Z. and Ben-Arie, J. 1996. Conveying visual information with spatial auditory patterns. IEEE Trans. Speech Audio Process. 4, 6, 446--455.Google ScholarCross Ref
- Ware, C. 2004. Information Visualization: Perception for Design. Morgan Kaufmann, San Francisco, CA. Google ScholarDigital Library
- Wright, M., Cassidy, R J., and F., Z. M. 2004. Audio and gesture latency measurements on Linux and OSX. In Proceedings of the International Computer Music Conference. 423--429.Google Scholar
- Zahorik, P., Brungart, D. S., and Bronkhorst, A. W. 2005. Auditory distance perception in humans: A summary of past and present research. Acta Acustica 91, 3, 409--420.Google Scholar
- Zbyszynski, M., Wright, M., Momeni, A., and Cullen, D. 2007. Ten years of tablet musical interfaces at CNMAT. In Proceedings of the 7th International Conference on New interfaces for Musical Expression. ACM, New York, 100--105. Google ScholarDigital Library
Index Terms
- Perception and replication of planar sonic gestures
Recommendations
The Perception of Sound Movements as Expressive Gestures
Sound, Music, and MotionAbstractThis paper is a preliminary attempt to investigate the perception of sound movements as expressive gestures. The idea is that if sound movement is used as a musical parameter, a listener (or a subject) should be able to distinguish among different ...
Effects of Robot Sound on Auditory Localization in Human-Robot Collaboration
HRI '18: Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot InteractionAuditory cues facilitate situational awareness by enabling humans to infer what is happening in the nearby environment. Unlike humans, many robots do not continuously produce perceivable state-expressive sounds. In this work, we propose the use of ...
Comments