Skip to main content
Top
Published in: Journal on Multimodal User Interfaces 1/2022

21-06-2021 | Original Paper

Interactive exploration of a hierarchical spider web structure with sound

Authors: Isabelle Su, Ian Hattwick, Christine Southworth, Evan Ziporyn, Ally Bisshop, Roland Mühlethaler, Tomás Saraceno, Markus J. Buehler

Published in: Journal on Multimodal User Interfaces | Issue 1/2022

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

3D spider webs exhibit highly intricate fiber architectures and owe their outstanding performance to a hierarchical organization that spans orders of magnitude in length scale from the molecular silk protein, to micrometer-sized fibers, and up to cm-scale web. Similarly, but in a completely different physical manifestation, music has a hierarchical structure composed of elementary sine wave building blocks that can be combined with other waveforms to create complex timbres, which are then arranged within larger-scale musical compositions. Although apparently different, spider webs and music have many similarities, as we point out in this work. Here, we propose an intuitive and interactive way to explore and visualize a 3D Cyrtophora citricola spider web geometry that has been digitally modeled with micron-scale details from full-scale laboratory experiments. We use model-based sonification to translate the web architecture into sound, allowing for aural perception and interpretation of its essential topological features. We implement this sonification using Unity3D and Max/MSP to create an interactive spider web environment in which a user travels through a virtual spider web. Each silk fiber in their field of view is sonified using different sine waves. Together, the sonified fibers create new and more complex timbres that reflects the architecture of 3D spider webs. These concepts are implemented into a spider web-based instrument for live performances, art installations and data exploration. It provides an unprecedented and creative way to immerse the composer, audience and user in an immersive multimedia experience generated by the complexity of a 3D spider web.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Literature
1.
go back to reference Su I, Qin Z, Bisshop A et al (2021) Sonification of a 3-D spider web and reconstitution for musical composition using granular synthesis. Comput Music J (in Press) Su I, Qin Z, Bisshop A et al (2021) Sonification of a 3-D spider web and reconstitution for musical composition using granular synthesis. Comput Music J (in Press)
2.
go back to reference Hermann T, Hunt A, Neuhoff JG (2011) The sonification handbook. Logos Verlag, Berlin Hermann T, Hunt A, Neuhoff JG (2011) The sonification handbook. Logos Verlag, Berlin
3.
go back to reference Bovermann T, Hermann T, Ritter H (2006) Tangible data scanning sonification model. Georgia Institute of Technology, Atlanta Bovermann T, Hermann T, Ritter H (2006) Tangible data scanning sonification model. Georgia Institute of Technology, Atlanta
4.
go back to reference Kramer G, Walker BN, Bonebright T et al (1999) The sonification report: status of the field and research agenda. Report prepared for the national science foundation by members of the international community for auditory display. Int Community Audit Disp (ICAD), St Fe, NM Kramer G, Walker BN, Bonebright T et al (1999) The sonification report: status of the field and research agenda. Report prepared for the national science foundation by members of the international community for auditory display. Int Community Audit Disp (ICAD), St Fe, NM
6.
go back to reference Idreos S, Papaemmanouil O, Chaudhuri S (2015) Overview of data exploration techniques. In: Proceedings of the 2015 ACM SIGMOD international conference on management of data, pp 277–281 Idreos S, Papaemmanouil O, Chaudhuri S (2015) Overview of data exploration techniques. In: Proceedings of the 2015 ACM SIGMOD international conference on management of data, pp 277–281
7.
go back to reference Papachristodoulou P, Betella A, Verschure P (2014) Sonification of large datasets in a 3D immersive environment: a neuroscience case study. In: ACHI2014: the seventh international conference on advances in computer-human interactions, pp 35–40 Papachristodoulou P, Betella A, Verschure P (2014) Sonification of large datasets in a 3D immersive environment: a neuroscience case study. In: ACHI2014: the seventh international conference on advances in computer-human interactions, pp 35–40
8.
go back to reference Yu CH, Qin Z, Martin-Martinez F, Buehler MJ (2019) A self-consistent sonification method to translate amino acid sequences into musical compositions and application in protein design using AI. ACS Nano 13:7471–7482CrossRef Yu CH, Qin Z, Martin-Martinez F, Buehler MJ (2019) A self-consistent sonification method to translate amino acid sequences into musical compositions and application in protein design using AI. ACS Nano 13:7471–7482CrossRef
9.
go back to reference Jeon M, Smith MT, Walker JW, Kuhl SA (2014) Constructing the immersive interactive sonification platform (iISoP). In: Streitz N, Markopoulos P (eds) Distributed, ambient, and pervasive interactions. Springer, Cham, pp 337–348CrossRef Jeon M, Smith MT, Walker JW, Kuhl SA (2014) Constructing the immersive interactive sonification platform (iISoP). In: Streitz N, Markopoulos P (eds) Distributed, ambient, and pervasive interactions. Springer, Cham, pp 337–348CrossRef
11.
go back to reference Gu GX, Su I, Sharma S et al (2016) Three-dimensional-printing of bio-inspired composites. J Biomech Eng 138:21006CrossRef Gu GX, Su I, Sharma S et al (2016) Three-dimensional-printing of bio-inspired composites. J Biomech Eng 138:21006CrossRef
15.
go back to reference Giesa T, Spivak D, Buehler M (2011) Reoccurring patterns in hierarchical protein materials and music: the power of analogies. Bionanoscience 1:153CrossRef Giesa T, Spivak D, Buehler M (2011) Reoccurring patterns in hierarchical protein materials and music: the power of analogies. Bionanoscience 1:153CrossRef
18.
20.
go back to reference Mortimer B, Gordon S, Drodge D et al (2013) Sonic properties of silks. XIV Int Conf Invertebr Sound Vib ISV 2013:43 Mortimer B, Gordon S, Drodge D et al (2013) Sonic properties of silks. XIV Int Conf Invertebr Sound Vib ISV 2013:43
21.
go back to reference Saraceno T, Bisshop A, Krell A, Mühlethaler R (2019) Arachnid orchestras: artistic research in vibrational interspecies communication. In: Biotremology: studying vibrational behavior. Springer, pp 485–509 Saraceno T, Bisshop A, Krell A, Mühlethaler R (2019) Arachnid orchestras: artistic research in vibrational interspecies communication. In: Biotremology: studying vibrational behavior. Springer, pp 485–509
23.
go back to reference Wulff C (2010) Zu Hause bei der Schwarzen Witwe oder das letzte Projekt mit Rolf-Dieter Düppe. Festschrift anlässlich der Pensionierung von Dr-Ing Rolf-Dieter Düppe nach 36 Jahren am Inst für Photogramm und Kartographie; Schriftenr/Fachrichtung Geodäsie 30:8 Wulff C (2010) Zu Hause bei der Schwarzen Witwe oder das letzte Projekt mit Rolf-Dieter Düppe. Festschrift anlässlich der Pensionierung von Dr-Ing Rolf-Dieter Düppe nach 36 Jahren am Inst für Photogramm und Kartographie; Schriftenr/Fachrichtung Geodäsie 30:8
24.
go back to reference Luhmann T (2017) Nahbereichsphotogrammetrie, Errata. Wichmann Verlag, Heidelberg Luhmann T (2017) Nahbereichsphotogrammetrie, Errata. Wichmann Verlag, Heidelberg
25.
go back to reference Luhmann T (2010) 3D-Rekonstruktion von Spinnennetzen. Wichmann Verlag, Heidelberg Luhmann T (2010) 3D-Rekonstruktion von Spinnennetzen. Wichmann Verlag, Heidelberg
26.
go back to reference Papachristodoulou P, Betella A, Manzolli J, Verschure PFMJ (2015) Augmenting the navigation of complex data sets using sonification: a case study with BrainX3 Panagiota. In: 2015 IEEE 2nd VR Workshop on sonic interactions for virtual environments, SIVE 2015—proceedings. Institute of Electrical and Electronics Engineers Inc., pp 3–8 Papachristodoulou P, Betella A, Manzolli J, Verschure PFMJ (2015) Augmenting the navigation of complex data sets using sonification: a case study with BrainX3 Panagiota. In: 2015 IEEE 2nd VR Workshop on sonic interactions for virtual environments, SIVE 2015—proceedings. Institute of Electrical and Electronics Engineers Inc., pp 3–8
27.
go back to reference Rossing TD, SpringerLink (2010) The science of string instruments. [electronic resource]. Springer, New York, 2010 Rossing TD, SpringerLink (2010) The science of string instruments. [electronic resource]. Springer, New York, 2010
28.
go back to reference Bennett WR, Holland CK (2018) The science of musical sound. Springer, BerlinCrossRef Bennett WR, Holland CK (2018) The science of musical sound. Springer, BerlinCrossRef
33.
go back to reference MATLAB (2019) 9.6.0.1099231 (R2019a). The MathWorks Inc., Natick, Massachusetts MATLAB (2019) 9.6.0.1099231 (R2019a). The MathWorks Inc., Natick, Massachusetts
35.
go back to reference Lindborg P (2018) Interactive sonification of weather data for the locust wrath, a multimedia dance performance. Leonardo 51:466–474CrossRef Lindborg P (2018) Interactive sonification of weather data for the locust wrath, a multimedia dance performance. Leonardo 51:466–474CrossRef
37.
go back to reference Zotkin DYN, Hwang J, Duraiswaini R, Davis LS (2003) HRTF personalization using anthropometric measurements. In: 2003 IEEE workshop on applications of signal processing to audio and acoustics (IEEE Cat. No.03TH8684). pp 157–160 Zotkin DYN, Hwang J, Duraiswaini R, Davis LS (2003) HRTF personalization using anthropometric measurements. In: 2003 IEEE workshop on applications of signal processing to audio and acoustics (IEEE Cat. No.03TH8684). pp 157–160
Metadata
Title
Interactive exploration of a hierarchical spider web structure with sound
Authors
Isabelle Su
Ian Hattwick
Christine Southworth
Evan Ziporyn
Ally Bisshop
Roland Mühlethaler
Tomás Saraceno
Markus J. Buehler
Publication date
21-06-2021
Publisher
Springer International Publishing
Published in
Journal on Multimodal User Interfaces / Issue 1/2022
Print ISSN: 1783-7677
Electronic ISSN: 1783-8738
DOI
https://doi.org/10.1007/s12193-021-00375-x

Other articles of this Issue 1/2022

Journal on Multimodal User Interfaces 1/2022 Go to the issue

Premium Partner