Skip to main content

2018 | OriginalPaper | Buchkapitel

13. Media Poetics and Cognition in Colocative Audiovisual Displays

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Colocative display is the technique of sonically articulating the screen area of audiovisual media by dynamically placing and animating the associated sounds in spatial localization to their visual cues. Two such systems have been designed to date, the author’s Pixelphonics system and the Allosphere facility at University of California at Santa Barbara. While both technologies are prototypes, and thus lacking in a rich historical tradition that might inform what film theorist David Bordwell has called historical poetics, Bordwell’s concept of analytical and theoretical poetics can be fruitfully brought to bear to elucidate the general principles for making colocative audiovisual media and applications. The poetics, or ‘principles of making’ colocative media are situated within a discussion of the empirical dimensions of auditory localization, cognition and attentional resources, general audiovisual practices, acoustics and phenomenology. A new concept, that of the soundscene, is introduced to hone in on the particular design affordances of colocative displays. This inquiry blends second and third wave HCI approaches in its hybridization of humanist media theories with cognitivist-attentional usability perspectives.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
Formally described in Apparatus, Method and System for Co-locating Visual Images and Associated Sound (U.S. Provisional Patent No. 62/482725, 2017).
 
Literatur
Zurück zum Zitat Altman R (1992) Sound space. In: Altman R (ed) Sound theory sound practice. Routledge, New York/London Altman R (1992) Sound space. In: Altman R (ed) Sound theory sound practice. Routledge, New York/London
Zurück zum Zitat Belton J (1992) 1950s magnetic sound: the frozen revolution. In: Altman R (ed) Sound theory sound practice. Routledge, New York/London Belton J (1992) 1950s magnetic sound: the frozen revolution. In: Altman R (ed) Sound theory sound practice. Routledge, New York/London
Zurück zum Zitat Bordwell D (2002) Intensified continuity visual style in contemporary American film. Film Q 55(3):16–28CrossRef Bordwell D (2002) Intensified continuity visual style in contemporary American film. Film Q 55(3):16–28CrossRef
Zurück zum Zitat Bordwell D (2007) The poetics of cinema. Routledge, New York Bordwell D (2007) The poetics of cinema. Routledge, New York
Zurück zum Zitat Bregman A (1990) Auditory scene analysis. MIT Press, Cambridge, MA Bregman A (1990) Auditory scene analysis. MIT Press, Cambridge, MA
Zurück zum Zitat Bresson R (1985) Notes on sound. In: Weis E, Belton J (eds) Film sound: theory and practice. Columbia University Press, New York Bresson R (1985) Notes on sound. In: Weis E, Belton J (eds) Film sound: theory and practice. Columbia University Press, New York
Zurück zum Zitat Carlile S, Leung J (2016) The perception of auditory motion. Trends Hearing 20:1–19CrossRef Carlile S, Leung J (2016) The perception of auditory motion. Trends Hearing 20:1–19CrossRef
Zurück zum Zitat Chion M (2013) Sensory aspects of contemporary cinema. In: Richardson J, Gorbman C, Vernallis C (eds) The Oxford handbook of new audiovisual aesthetics. Oxford University Press, Oxford. Amazon Kindle edition Chion M (2013) Sensory aspects of contemporary cinema. In: Richardson J, Gorbman C, Vernallis C (eds) The Oxford handbook of new audiovisual aesthetics. Oxford University Press, Oxford. Amazon Kindle edition
Zurück zum Zitat Coleman S (2005) Badgered. On: a collection of 2005 academy award nominated short films (DVD). Magnolia Home Entertainment Coleman S (2005) Badgered. On: a collection of 2005 academy award nominated short films (DVD). Magnolia Home Entertainment
Zurück zum Zitat Connor S (2013) Sounding out film. In: Richardson J, Gorbman C, Vernallis C (eds) The Oxford handbook of new audiovisual aesthetics. Oxford University Press, Oxford. Amazon Kindle edition Connor S (2013) Sounding out film. In: Richardson J, Gorbman C, Vernallis C (eds) The Oxford handbook of new audiovisual aesthetics. Oxford University Press, Oxford. Amazon Kindle edition
Zurück zum Zitat Donnelly KJ (2013) Extending film aesthetics: audio beyond visuals. In: Richardson J, Gorbman C, Vernallis C (eds) The Oxford handbook of new audiovisual aesthetics. Oxford University Press, Oxford. Amazon Kindle edition Donnelly KJ (2013) Extending film aesthetics: audio beyond visuals. In: Richardson J, Gorbman C, Vernallis C (eds) The Oxford handbook of new audiovisual aesthetics. Oxford University Press, Oxford. Amazon Kindle edition
Zurück zum Zitat Gibson JJ (1966) The senses considered as perceptual systems. Houghton Mifflin Co., Boston Gibson JJ (1966) The senses considered as perceptual systems. Houghton Mifflin Co., Boston
Zurück zum Zitat Godfroy M, Roumes C, Dauchy P (2003) Spatial variations of visual-auditory fusion areas. Perception 32:1233–1245CrossRef Godfroy M, Roumes C, Dauchy P (2003) Spatial variations of visual-auditory fusion areas. Perception 32:1233–1245CrossRef
Zurück zum Zitat He JJ (2017) Literature review on spatial audio. In: Gan WS, Kuo CCJ, Zheng TF, Barni M (eds) Spatial audio reproduction with primary ambient extraction, Springer briefs in signal processing. Springer, BaselCrossRef He JJ (2017) Literature review on spatial audio. In: Gan WS, Kuo CCJ, Zheng TF, Barni M (eds) Spatial audio reproduction with primary ambient extraction, Springer briefs in signal processing. Springer, BaselCrossRef
Zurück zum Zitat Herbert R (2017) Sonic subjectivities. In: Cobussen M, Meelberg V, Truax B (eds) The Routledge companion to sounding art. Routledge, New York. Amazon Kindle edition Herbert R (2017) Sonic subjectivities. In: Cobussen M, Meelberg V, Truax B (eds) The Routledge companion to sounding art. Routledge, New York. Amazon Kindle edition
Zurück zum Zitat Höllerer T, Kuchera-Morin J, Amatriain X (2007) The allosphere: a large-scale immersive surround-view instrument. In: EDT’07: proceedings of the 2007 workshop on emerging displays technologies: images and beyond: the future of displays and interaction, San Diego, CA, USA, Aug, 2007, Article No. 3. doi 101145/12782401278243 Höllerer T, Kuchera-Morin J, Amatriain X (2007) The allosphere: a large-scale immersive surround-view instrument. In: EDT’07: proceedings of the 2007 workshop on emerging displays technologies: images and beyond: the future of displays and interaction, San Diego, CA, USA, Aug, 2007, Article No. 3. doi 101145/12782401278243
Zurück zum Zitat Holman T (2007) Surround sound up and running. Focal Press, New York Holman T (2007) Surround sound up and running. Focal Press, New York
Zurück zum Zitat Horst S (2005) Phenomenology and psychophysics. Phenomenol Cogn Sci 4:1–21CrossRef Horst S (2005) Phenomenology and psychophysics. Phenomenol Cogn Sci 4:1–21CrossRef
Zurück zum Zitat Kerins M (2010) Beyond Dolby (stereo): cinema in the digital sound age. Indiana University Press, Bloomington Kerins M (2010) Beyond Dolby (stereo): cinema in the digital sound age. Indiana University Press, Bloomington
Zurück zum Zitat Kim-Cohn S (2017) Dams, weirs, and damn weird ears– post-ergonal sound. Auditory capital, media publics and the sounding arts. In: Cobussen M, Meelberg V, Truax B (eds) The Routledge companion to sounding art. Routledge, New York. Amazon Kindle edition Kim-Cohn S (2017) Dams, weirs, and damn weird ears– post-ergonal sound. Auditory capital, media publics and the sounding arts. In: Cobussen M, Meelberg V, Truax B (eds) The Routledge companion to sounding art. Routledge, New York. Amazon Kindle edition
Zurück zum Zitat Kuchera-Morin J, Wright M, Wakefield G, Roberts C, Adderton D, Sajadi B, Höllerer T, Majumder A (2014) Immersive full-surround multi-user system design. Comput Graph 40:10–21CrossRef Kuchera-Morin J, Wright M, Wakefield G, Roberts C, Adderton D, Sajadi B, Höllerer T, Majumder A (2014) Immersive full-surround multi-user system design. Comput Graph 40:10–21CrossRef
Zurück zum Zitat LaBelle B (2017) Restless acoustics, emergent publics. In: Cobussen M, Meelberg V, Truax B (eds) The Routledge companion to sounding art. Routledge, New York. Amazon Kindle edition LaBelle B (2017) Restless acoustics, emergent publics. In: Cobussen M, Meelberg V, Truax B (eds) The Routledge companion to sounding art. Routledge, New York. Amazon Kindle edition
Zurück zum Zitat Lacey K (2017) Auditory capital, media publics and the sounding arts. In: Cobussen M, Meelberg V, Truax B (eds) The Routledge companion to sounding art. Routledge, New York. Amazon Kindle edition Lacey K (2017) Auditory capital, media publics and the sounding arts. In: Cobussen M, Meelberg V, Truax B (eds) The Routledge companion to sounding art. Routledge, New York. Amazon Kindle edition
Zurück zum Zitat Makous JC, Middlebrooks JC (1990) Two-dimensional sound localization by human listeners. J Acoust Soc Am 87:2188–2200CrossRef Makous JC, Middlebrooks JC (1990) Two-dimensional sound localization by human listeners. J Acoust Soc Am 87:2188–2200CrossRef
Zurück zum Zitat Middlebrooks JC, Green DM (1991) Sound localization by human listeners. Annu Rev Psychol 42:135–159CrossRef Middlebrooks JC, Green DM (1991) Sound localization by human listeners. Annu Rev Psychol 42:135–159CrossRef
Zurück zum Zitat Richardson DC, Spivey MJ (2000) Representation, space and Hollywood Squares: looking at things that aren’t there anymore. Cognition 76(3):269–295CrossRef Richardson DC, Spivey MJ (2000) Representation, space and Hollywood Squares: looking at things that aren’t there anymore. Cognition 76(3):269–295CrossRef
Zurück zum Zitat Stillwell RJ (2013) Audiovisual space in an era of technological convergence. In: Richardson J, Gorbman C, Vernallis C (eds) The Oxford handbook of new audiovisual aesthetics. Oxford University Press, Oxford. Amazon Kindle edition Stillwell RJ (2013) Audiovisual space in an era of technological convergence. In: Richardson J, Gorbman C, Vernallis C (eds) The Oxford handbook of new audiovisual aesthetics. Oxford University Press, Oxford. Amazon Kindle edition
Zurück zum Zitat Ursino M, Magosso E, Cuppini C (2011) Sensory fusion. In: Cutsuridis V, Hussain A, Taylor JG (eds) Perception-action cycle: models, architectures, and hardware. Springer series in cognitive and neural systems. Springer International Publishing AG, Basel, pp 23–62CrossRef Ursino M, Magosso E, Cuppini C (2011) Sensory fusion. In: Cutsuridis V, Hussain A, Taylor JG (eds) Perception-action cycle: models, architectures, and hardware. Springer series in cognitive and neural systems. Springer International Publishing AG, Basel, pp 23–62CrossRef
Zurück zum Zitat Wickens CD (2002) Multiple resources and performance prediction. Theoretical issues in ergonomic. Science 3(2):166 Wickens CD (2002) Multiple resources and performance prediction. Theoretical issues in ergonomic. Science 3(2):166
Zurück zum Zitat Wickens CD (2008) Multiple resources and mental workload. Hum Factors 50(3):449–455CrossRef Wickens CD (2008) Multiple resources and mental workload. Hum Factors 50(3):449–455CrossRef
Zurück zum Zitat Wright RD, Ward LM (2008) Orienting of attention. Oxford University Press, Oxford Wright RD, Ward LM (2008) Orienting of attention. Oxford University Press, Oxford
Metadaten
Titel
Media Poetics and Cognition in Colocative Audiovisual Displays
verfasst von
Michael Filimowicz
Copyright-Jahr
2018
DOI
https://doi.org/10.1007/978-3-319-73356-2_13

Neuer Inhalt