Skip to main content
Erschienen in:

21.06.2021 | Original Paper

Interactive exploration of a hierarchical spider web structure with sound

verfasst von: Isabelle Su, Ian Hattwick, Christine Southworth, Evan Ziporyn, Ally Bisshop, Roland Mühlethaler, Tomás Saraceno, Markus J. Buehler

Erschienen in: Journal on Multimodal User Interfaces | Ausgabe 1/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

3D spider webs exhibit highly intricate fiber architectures and owe their outstanding performance to a hierarchical organization that spans orders of magnitude in length scale from the molecular silk protein, to micrometer-sized fibers, and up to cm-scale web. Similarly, but in a completely different physical manifestation, music has a hierarchical structure composed of elementary sine wave building blocks that can be combined with other waveforms to create complex timbres, which are then arranged within larger-scale musical compositions. Although apparently different, spider webs and music have many similarities, as we point out in this work. Here, we propose an intuitive and interactive way to explore and visualize a 3D Cyrtophora citricola spider web geometry that has been digitally modeled with micron-scale details from full-scale laboratory experiments. We use model-based sonification to translate the web architecture into sound, allowing for aural perception and interpretation of its essential topological features. We implement this sonification using Unity3D and Max/MSP to create an interactive spider web environment in which a user travels through a virtual spider web. Each silk fiber in their field of view is sonified using different sine waves. Together, the sonified fibers create new and more complex timbres that reflects the architecture of 3D spider webs. These concepts are implemented into a spider web-based instrument for live performances, art installations and data exploration. It provides an unprecedented and creative way to immerse the composer, audience and user in an immersive multimedia experience generated by the complexity of a 3D spider web.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Su I, Qin Z, Bisshop A et al (2021) Sonification of a 3-D spider web and reconstitution for musical composition using granular synthesis. Comput Music J (in Press) Su I, Qin Z, Bisshop A et al (2021) Sonification of a 3-D spider web and reconstitution for musical composition using granular synthesis. Comput Music J (in Press)
2.
Zurück zum Zitat Hermann T, Hunt A, Neuhoff JG (2011) The sonification handbook. Logos Verlag, Berlin Hermann T, Hunt A, Neuhoff JG (2011) The sonification handbook. Logos Verlag, Berlin
3.
Zurück zum Zitat Bovermann T, Hermann T, Ritter H (2006) Tangible data scanning sonification model. Georgia Institute of Technology, Atlanta Bovermann T, Hermann T, Ritter H (2006) Tangible data scanning sonification model. Georgia Institute of Technology, Atlanta
4.
Zurück zum Zitat Kramer G, Walker BN, Bonebright T et al (1999) The sonification report: status of the field and research agenda. Report prepared for the national science foundation by members of the international community for auditory display. Int Community Audit Disp (ICAD), St Fe, NM Kramer G, Walker BN, Bonebright T et al (1999) The sonification report: status of the field and research agenda. Report prepared for the national science foundation by members of the international community for auditory display. Int Community Audit Disp (ICAD), St Fe, NM
6.
Zurück zum Zitat Idreos S, Papaemmanouil O, Chaudhuri S (2015) Overview of data exploration techniques. In: Proceedings of the 2015 ACM SIGMOD international conference on management of data, pp 277–281 Idreos S, Papaemmanouil O, Chaudhuri S (2015) Overview of data exploration techniques. In: Proceedings of the 2015 ACM SIGMOD international conference on management of data, pp 277–281
7.
Zurück zum Zitat Papachristodoulou P, Betella A, Verschure P (2014) Sonification of large datasets in a 3D immersive environment: a neuroscience case study. In: ACHI2014: the seventh international conference on advances in computer-human interactions, pp 35–40 Papachristodoulou P, Betella A, Verschure P (2014) Sonification of large datasets in a 3D immersive environment: a neuroscience case study. In: ACHI2014: the seventh international conference on advances in computer-human interactions, pp 35–40
8.
Zurück zum Zitat Yu CH, Qin Z, Martin-Martinez F, Buehler MJ (2019) A self-consistent sonification method to translate amino acid sequences into musical compositions and application in protein design using AI. ACS Nano 13:7471–7482CrossRef Yu CH, Qin Z, Martin-Martinez F, Buehler MJ (2019) A self-consistent sonification method to translate amino acid sequences into musical compositions and application in protein design using AI. ACS Nano 13:7471–7482CrossRef
9.
Zurück zum Zitat Jeon M, Smith MT, Walker JW, Kuhl SA (2014) Constructing the immersive interactive sonification platform (iISoP). In: Streitz N, Markopoulos P (eds) Distributed, ambient, and pervasive interactions. Springer, Cham, pp 337–348CrossRef Jeon M, Smith MT, Walker JW, Kuhl SA (2014) Constructing the immersive interactive sonification platform (iISoP). In: Streitz N, Markopoulos P (eds) Distributed, ambient, and pervasive interactions. Springer, Cham, pp 337–348CrossRef
11.
Zurück zum Zitat Gu GX, Su I, Sharma S et al (2016) Three-dimensional-printing of bio-inspired composites. J Biomech Eng 138:21006CrossRef Gu GX, Su I, Sharma S et al (2016) Three-dimensional-printing of bio-inspired composites. J Biomech Eng 138:21006CrossRef
15.
Zurück zum Zitat Giesa T, Spivak D, Buehler M (2011) Reoccurring patterns in hierarchical protein materials and music: the power of analogies. Bionanoscience 1:153CrossRef Giesa T, Spivak D, Buehler M (2011) Reoccurring patterns in hierarchical protein materials and music: the power of analogies. Bionanoscience 1:153CrossRef
18.
20.
Zurück zum Zitat Mortimer B, Gordon S, Drodge D et al (2013) Sonic properties of silks. XIV Int Conf Invertebr Sound Vib ISV 2013:43 Mortimer B, Gordon S, Drodge D et al (2013) Sonic properties of silks. XIV Int Conf Invertebr Sound Vib ISV 2013:43
21.
Zurück zum Zitat Saraceno T, Bisshop A, Krell A, Mühlethaler R (2019) Arachnid orchestras: artistic research in vibrational interspecies communication. In: Biotremology: studying vibrational behavior. Springer, pp 485–509 Saraceno T, Bisshop A, Krell A, Mühlethaler R (2019) Arachnid orchestras: artistic research in vibrational interspecies communication. In: Biotremology: studying vibrational behavior. Springer, pp 485–509
23.
Zurück zum Zitat Wulff C (2010) Zu Hause bei der Schwarzen Witwe oder das letzte Projekt mit Rolf-Dieter Düppe. Festschrift anlässlich der Pensionierung von Dr-Ing Rolf-Dieter Düppe nach 36 Jahren am Inst für Photogramm und Kartographie; Schriftenr/Fachrichtung Geodäsie 30:8 Wulff C (2010) Zu Hause bei der Schwarzen Witwe oder das letzte Projekt mit Rolf-Dieter Düppe. Festschrift anlässlich der Pensionierung von Dr-Ing Rolf-Dieter Düppe nach 36 Jahren am Inst für Photogramm und Kartographie; Schriftenr/Fachrichtung Geodäsie 30:8
24.
Zurück zum Zitat Luhmann T (2017) Nahbereichsphotogrammetrie, Errata. Wichmann Verlag, Heidelberg Luhmann T (2017) Nahbereichsphotogrammetrie, Errata. Wichmann Verlag, Heidelberg
25.
Zurück zum Zitat Luhmann T (2010) 3D-Rekonstruktion von Spinnennetzen. Wichmann Verlag, Heidelberg Luhmann T (2010) 3D-Rekonstruktion von Spinnennetzen. Wichmann Verlag, Heidelberg
26.
Zurück zum Zitat Papachristodoulou P, Betella A, Manzolli J, Verschure PFMJ (2015) Augmenting the navigation of complex data sets using sonification: a case study with BrainX3 Panagiota. In: 2015 IEEE 2nd VR Workshop on sonic interactions for virtual environments, SIVE 2015—proceedings. Institute of Electrical and Electronics Engineers Inc., pp 3–8 Papachristodoulou P, Betella A, Manzolli J, Verschure PFMJ (2015) Augmenting the navigation of complex data sets using sonification: a case study with BrainX3 Panagiota. In: 2015 IEEE 2nd VR Workshop on sonic interactions for virtual environments, SIVE 2015—proceedings. Institute of Electrical and Electronics Engineers Inc., pp 3–8
27.
Zurück zum Zitat Rossing TD, SpringerLink (2010) The science of string instruments. [electronic resource]. Springer, New York, 2010 Rossing TD, SpringerLink (2010) The science of string instruments. [electronic resource]. Springer, New York, 2010
28.
Zurück zum Zitat Bennett WR, Holland CK (2018) The science of musical sound. Springer, BerlinCrossRef Bennett WR, Holland CK (2018) The science of musical sound. Springer, BerlinCrossRef
33.
Zurück zum Zitat MATLAB (2019) 9.6.0.1099231 (R2019a). The MathWorks Inc., Natick, Massachusetts MATLAB (2019) 9.6.0.1099231 (R2019a). The MathWorks Inc., Natick, Massachusetts
35.
Zurück zum Zitat Lindborg P (2018) Interactive sonification of weather data for the locust wrath, a multimedia dance performance. Leonardo 51:466–474CrossRef Lindborg P (2018) Interactive sonification of weather data for the locust wrath, a multimedia dance performance. Leonardo 51:466–474CrossRef
37.
Zurück zum Zitat Zotkin DYN, Hwang J, Duraiswaini R, Davis LS (2003) HRTF personalization using anthropometric measurements. In: 2003 IEEE workshop on applications of signal processing to audio and acoustics (IEEE Cat. No.03TH8684). pp 157–160 Zotkin DYN, Hwang J, Duraiswaini R, Davis LS (2003) HRTF personalization using anthropometric measurements. In: 2003 IEEE workshop on applications of signal processing to audio and acoustics (IEEE Cat. No.03TH8684). pp 157–160
Metadaten
Titel
Interactive exploration of a hierarchical spider web structure with sound
verfasst von
Isabelle Su
Ian Hattwick
Christine Southworth
Evan Ziporyn
Ally Bisshop
Roland Mühlethaler
Tomás Saraceno
Markus J. Buehler
Publikationsdatum
21.06.2021
Verlag
Springer International Publishing
Erschienen in
Journal on Multimodal User Interfaces / Ausgabe 1/2022
Print ISSN: 1783-7677
Elektronische ISSN: 1783-8738
DOI
https://doi.org/10.1007/s12193-021-00375-x

Weitere Artikel der Ausgabe 1/2022

Journal on Multimodal User Interfaces 1/2022 Zur Ausgabe