Top

Journal on Multimodal User Interfaces

Published in:

18-10-2023 | Original Paper

Multimodal exploration in elementary music classroom

Authors: Martha Papadogianni, Ercan Altinsoy, Areti Andreopoulou

Published in: Journal on Multimodal User Interfaces | Issue 1/2024

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Vibrotactile feedback can support and enhance various musical learning processes of both children and adults, thus improving their musical experience. However, until now, to the best knowledge of the authors, only a few studies have focused on the auditory-tactile music perception in childhood. In this paper, we evaluate a bimodal auditory-tactile teaching approach for elementary school children. The experiment was designed based on the two-group-pretest-posttest design. Children allocated to the experimental group were subjected to auditory-tactile music training, while those allocated to the control-group completed the same training sessions stimulated by audio-only signals. Both groups received common, especially designed musical activities, divided into sixteen sessions. In order to evaluate the experimental procedure, children completed an auditory ability test in three different timepoints: before, half-way through, and at the end of the study. Data from 60 children were analyzed. The main aim of the study was to explore beat sensitivity in simple and complex music metrical structures and examine whether auditory-tactile stimulation could enhance the children’s ability to recognize beat within various musical contexts. Results demonstrate notable performance improvements within the experimental group, suggesting that a multimodal music experience can improve beat induction abilities of both simple and complex meters for elementary school children.

previous article Hearing loss prevention at loud music events via real-time visuo-haptic feedback

next article Comparing alternative modalities in the context of multimodal human–robot interaction

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Einarson K, Trainor L (2016) Hearing the beat: young children’s perceptual sensitivity to beat alignment varies according to metric structure. Music Percept Interdiscip J 34:56–70. https://doi.org/10.1525/.2016.34.1.56CrossRef

Nozaradan S, Peretz I, Missal M, Mouraux A (2011) Tagging the neuronal entrainment to beat and meter. J Neurosci 31(28):10234–40. https://doi.org/10.1523/JNEUROSCI.0411-11.2011CrossRefPubMedPubMedCentral

Repp BH, Su YH (2013) Sensorimotor synchronization: a review of recent research (2006–2012). Psychon Bull Rev 20(3):403–52. https://doi.org/10.3758/s13423-012-0371-2CrossRefPubMed

Patel AD, Iversen JR (2014) The evolutionary neuroscience of musical beat perception: the action simulation for auditory prediction (ASAP) hypothesis. Front Syst Neurosci 13(8):57. https://doi.org/10.3389/fnsys.2014.00057CrossRef

Burger B, Thompson M, Luck G, Saarikallio S, Toiviainen P (2012) Music moves us: beat-related musical features influence regularity of music-induced movement. In: Cambouropoulos E, Tsougras C, Mavromatis P, and Pastiadis K (eds) Proceedings of the 12th international conference on music perception and cognition (ICMPC) and the 8th triennial conference of the european society for the cognitive sciences of music (ESCOM) July 23–28, 2012 Thessaloniki, Greece. Aristotle University of Thessaloniki, pp 183–187. http://icmpc-escom2012.web.auth.gr/sites/default/files/papers/183_Proc.pdf

Honing H (2012) Without it no music: beat induction as a fundamental musical trait. Ann N Y Acad Sci 1252:85–91. https://doi.org/10.1111/j.1749-6632.2011.06402.xADSCrossRefPubMed

Haumann NT, Vuust P, Bertelsen F, Garza-Villarreal EA (2018) Influence of musical enculturation on brain responses to metric deviants. Front Neurosci 18(12):218. https://doi.org/10.3389/fnins.2018.00218CrossRef

Kalender B, Trehub SE, Schellenberg EG (2012) Cross-cultural differences in meter perception. Psychol Res 77(2):196–203. https://doi.org/10.1007/s00426-012-0427-yCrossRefPubMed

Van der Weij B, Pearce MT, Honing H (2017) A probabilistic model of meter perception: simulating enculturation. Front Psychol 8:824. https://doi.org/10.3389/fpsyg.2017.00824CrossRefPubMedPubMedCentral

10.

Large EW, Snyder JS (2009) Pulse and meter as neural resonance. Ann N Y Acad Sci 1169:46–57. https://doi.org/10.1111/j.1749-6632.2009.04550.xADSCrossRefPubMed

11.

Grahn JA, Henry MJ, McAuley JD (2011) FMRI investigation of cross-modal interactions in beat perception: audition primes vision, but not vice versa. Neuroimage 54(2):1231–43. https://doi.org/10.1016/j.neuroimage.2010.09.033CrossRefPubMed

12.

Elliott MT, Wing AM, Welchman AE (2010) Multisensory cues improve sensorimotor synchronisation. Eur J Neurosci 31(10):1828–35. https://doi.org/10.1111/j.1460-9568.2010.07205.xCrossRefPubMed

13.

Grondin S, McAuley D (2009) Duration discrimination in crossmodal sequences. Perception 38(10):1542–59. https://doi.org/10.1068/p6359CrossRefPubMed

14.

Kosonen K, Raisamo R (2006) Rhythm perception through different modalities. In: Proceedings of Eurohaptics 2006, pp 365–370

15.

Lerens E, De Volder A, Araneda R (2013) Perception of rhythm through auditory, vibro-tactile and visual stimulations: an fMRI study. Multisens Res 26:117–117. https://doi.org/10.1163/22134808-000S0085CrossRef

16.

Butler JS, Foxe JJ, Fiebelkorn IC, Mercier MR, Molholm S (2012) Multisensory representation of frequency across audition and touch: high density electrical mapping reveals early sensory-perceptual coupling. J Neurosci 32(44):15338–44. https://doi.org/10.1523/JNEUROSCI.1796-12.2012CrossRefPubMedPubMedCentral

17.

Jokiniemi M, Raisamo R, Lylykangas J, Surakka V (2008) Crossmodal rhythm perception. In: Pirhonen A, Brewster S (eds) Haptic and audio interaction design, vol 5270. HAID 2008. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87883-4_12CrossRef

18.

Godde B, Stauffenberg B, Spengler F, Dinse HR (2000) Tactile coactivation-induced changes in spatial discrimination performance. J Neurosci 20(4):1597–604. https://doi.org/10.1523/JNEUROSCI.20-04-01597.2000CrossRefPubMedPubMedCentral

19.

Brochard R, Touzalin P, Després O, Dufour A (2008) Evidence of beat perception via purely tactile stimulation. Brain Res. 5(1223):59–64. https://doi.org/10.1016/j.brainres.2008.05.050CrossRef

20.

Bach-y-Rita P (2004) Tactile sensory substitution studies. Ann N Y Acad Sci 1013(1):83–91. https://doi.org/10.1196/annals.1305.006ADSCrossRefPubMed

21.

Nanayakkara SC, Wyse L, Ong SH, Taylor EA (2013) Enhancing musical experience for the hearing-impaired using visual and haptic displays. Hum Comput Interact 28(2):115–160. https://doi.org/10.1080/07370024.2012.697006CrossRef

22.

Jack R, McPherson DA, Stockman DT (2015) Designing tactile musical devices with and for deaf users: a case study. In: Proceedings of the international conference on the multimodal experience of music, pp 23–25

23.

Tranchant P, Shiell MM, Giordano M, Nadeau A, Peretz I, Zatorre RJ (2017) Feeling the beat: bouncing synchronization to vibrotactile music in hearing and early deaf people. Front Neurosci 12(11):507. https://doi.org/10.3389/fnins.2017.00507CrossRef

24.

Phillips-Silver J, Toiviainen P, Gosselin N, Turgeon C, Lepore F, Peretz I (2015) Cochlear implant users move in time to the beat of drum music. Hear Res 321:25–34. https://doi.org/10.1016/j.heares.2014.12.007CrossRefPubMed

25.

Petry B, Illandara T, Elvitigala D, Nanayakkara S (2018) Supporting rhythm activities of deaf children using music-sensory-substitution systems. In: Proceedings of the 2018 CHI conference on human factors in computing systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, pp 1–10. https://doi.org/10.1145/3173574.3174060

26.

Miura S, Sugimoto M (2006) Supporting children’s rhythm learning using vibration devices. In: CHI ’06 extended abstracts on human factors in computing systems (CHI EA ’06). Association for Computing Machinery, New York, NY, USA, pp 1127–1132. https://doi.org/10.1145/1125451.1125664

27.

Trainor LJ, Corrigall KA (2010) Music acquisition and effects of musical experience. In: Jones MR, Fay RR, Popper AN (eds) Music perception. Springer, New York, pp 89–127. https://doi.org/10.1007/978-1-4419-6114-3_4CrossRef

28.

Reifinger JL (2006) Skill development in rhythm perception and performance: a review of literature. Update Appl Res Music Educ 25(1):15–27. https://doi.org/10.1177/87551233060250010103CrossRef

29.

Savage PE, Brown S, Sakai E, Currie TE (2015) Statistical universals reveal the structures and functions of human music. Proc Natl Acad Sci U S A 112(29):8987–92. https://doi.org/10.1073/pnas.1414495112ADSCrossRefPubMedPubMedCentral

30.

Hannon EE, Trainor LJ (2007) Music acquisition: effects of enculturation and formal training on development. Trends Cogn Sci 11(11):466–72. https://doi.org/10.1016/j.tics.2007.08.008CrossRefPubMed

31.

Soley G, Hannon EE (2010) Infants prefer the musical meter of their own culture: a cross-cultural comparison. Dev Psychol 46(1):286–92. https://doi.org/10.1037/a0017555CrossRefPubMed

32.

Cirelli LK, Spinelli C, Nozaradan S, Trainor LJ (2016) Measuring neural entrainment to beat and meter in infants: effects of music background. Front Neurosci 24(10):229. https://doi.org/10.3389/fnins.2016.00229CrossRef

33.

Trehub SE, Hannon EE (2006) Infant music perception: Domain-general or domain-specific mechanisms? Cognition 100(1):73–99. https://doi.org/10.1016/j.cognition.2005.11.006CrossRefPubMed

34.

Trehub SE (2003) The developmental origins of musicality. Nat Neurosci 6(7):669–73. https://doi.org/10.1038/nn1084CrossRefPubMed

35.

Wallentin M, Nielsen AH, Friis-Olivarius M, Vuust C, Vuust P (2010) The musical ear test, a new reliable test for measuring musical competence. Learn Individ Differ 20(3):188–196. https://doi.org/10.1016/j.lindif.2010.02.004CrossRef

36.

Vanden Hannon EE, Bosch der Nederlanden CM, Tichko P (2012) Effects of perceptual experience on children’s and adults’ perception of unfamiliar rhythms. Ann N Y Acad Sci 1252:92–9. https://doi.org/10.1111/j.1749-6632.2012.06466.xADSCrossRef

37.

Trainor LJ, Marie C, Gerry D, Whiskin E, Unrau A (2012) Becoming musically enculturated: effects of music classes for infants on brain and behavior. Ann N Y Acad Sci 1252:129–38. https://doi.org/10.1111/j.1749-6632.2012.06462.xADSCrossRefPubMed

38.

Michaelis K, Wiener M, Thompson JC (2014) Passive listening to preferred motor tempo modulates corticospinal excitability. Front Hum Neurosci 24(8):252. https://doi.org/10.3389/fnhum.2014.00252CrossRef

39.

Drake C, Ben El Heni J (2003) Synchronizing with music: intercultural differences. Ann N Y Acad Sci 999:429–37. https://doi.org/10.1196/annals.1284.053ADSCrossRefPubMed

40.

Drake C, Penel A, Bigand E (2000) Tapping in time with mechanically and expressively performed music. Music Percept 18(1):1–23. https://doi.org/10.2307/40285899CrossRef

41.

Hannon EE, Soley G, Ullal S (2012) Familiarity overrides complexity in rhythm perception: a cross-cultural comparison of American and Turkish listeners. J Exp Psychol Hum Percept Perform 38(3):543–8. https://doi.org/10.1037/a0027225CrossRefPubMed

42.

Cameron DJ, Bentley J, Grahn JA (2015) Cross-cultural influences on rhythm processing: reproduction, discrimination, and beat tapping. Front Psychol 9(6):366. https://doi.org/10.3389/fpsyg.2015.00366CrossRef

43.

Kalender B, Trehub SE, Schellenberg EG (2012) Cross-cultural differences in meter perception. Psychol Res 77(2):196–203. https://doi.org/10.1007/s00426-012-0427-yCrossRefPubMed

44.

Snyder J, Hannon E, Edward L, Christiansen M (2006) Synchronization and continuation tapping to complex meters. Music Percept 24:135–146. https://doi.org/10.1525/mp.2006.24.2.135CrossRef

45.

Morrison SJ, Demorest SM, Stambaugh LA (2008) Enculturation effects in music cognition: the role of age and music complexity. J. Res. Music Educ 56(2):118–129CrossRef

46.

Trainor LJ, Hannon EE (2013) Musical development. In: Deutsch D (ed) The psychology of music. Elsevier Academic Press, Amsterdam, pp 423–497. https://doi.org/10.1016/B978-0-12-381460-9.00011-0CrossRef

47.

Law LNC, Zentner M (2012) Assessing musical abilities objectively: construction and validation of the profile of music perception skills. PLoS one 7(12):e52508. https://doi.org/10.1371/journal.pone.0052508ADSCrossRefPubMedPubMedCentral

48.

Fujii S, Schlau G (2013) The harvard beat assessment test (H-BAT): a battery for assessing beat perception and production and their dissociation. Front Hum Neurosci 7:771. https://doi.org/10.3389/fnhum.2013.00771CrossRefPubMedPubMedCentral

49.

Iversen J, Patel A (2008) The beat alignment test (BAT): surveying beat processing abilities in the general population. In: Proceedings of the 10th international conference on music perception and cognition (ICMPC10)

50.

Dauvergne C, Begel V, Benoit CE, Kotz S, Dalla Bella S (2015) Battery for the assessment of auditory sensorimotor and timing abilities (BAASTA): a rehabilitation perspective. Ann Phys Rehabil Med 58:e72CrossRef

51.

Dalla Bella S, Farrugia N, Benoit CE et al (2017) BAASTA: battery for the assessment of auditory sensorimotor and timing abilities. Behav Res 49:1128–1145. https://doi.org/10.3758/s13428-016-0773-6CrossRef

52.

Gallace A, Tan HZ, Spence C (2007) The body surface as a communication system: the state of the art after 50 years. Presence Teleoperators Virtual Environ 16:655–676. https://doi.org/10.1162/pres.16.6.655CrossRef

53.

Giordano M, Wanderley MM (2015) Follow the tactile metronome: vibrotactile stimulation for tempo synchronization in music performance. In: Proceedings of the SMC conference, Maynooth, Ireland

54.

Giordano M, Sullivan J, Wanderley MM (2018) Design of vibrotactile feedback and stimulation for music performance. In: Papetti S, Saitis C (eds) Musical haptics. Springer, New York, pp 193–214. https://doi.org/10.1007/978-3-319-58316-7_10CrossRef

55.

Gunther E, Davenport G, O’Modhrain S (2002) Cutaneous grooves: composing for the sense of touch. In: Proceedings of the 2002 conference on new instruments for musical expression (NIME-02), Dublin, Ireland, May 24–26, vol 32. pp 37–43

56.

Sakuragi R, Ikeno S, Okazaki R, Kajimoto H (2015) CollarBeat: whole body vibrotactile presentation via the collarbone to enrich music listening experience. In: ICAT-EGVE 2015 - international conference on artificial reality and telexistence and eurographics symposium on virtual environments 6 pages. https://doi.org/10.2312/EGVE.20151321

57.

Benali-Khoudja M, Hafez M, Alexandre JM, Kheddar A (2004) Tactile interfaces: a state-of-the-art survey. Information Systems Research - ISR

58.

Cibrian FL, Pena O, Ortega D, Tentori M (2017) BendableSound: an elastic multisensory surface using touch-based interactions to assist children with severe autism during music therapy. Int J Hum Comput Stud 107:22–37. https://doi.org/10.1016/j.ijhcs.2017.05.003CrossRef

59.

Ifukube T (2017) Tactile stimulation methods for the deaf and/or blind. In: Ifukube T (ed) Sound-based assistive technology support to hearing, speaking and seeing. Springer, New York, pp 111–143. https://doi.org/10.1007/978-3-319-47997-2_4CrossRef

60.

Choi S, Kuchenbecker KJ (2013) Vibrotactile display: perception, technology, and applications. Proc IEEE 101:2093–2104. https://doi.org/10.1109/JPROC.2012.2221071CrossRef

61.

Miura S, Sugimoto M (2005) T-RHYTHM: a system for supporting rhythm learning by using tactile devices. In: IEEE international workshop on wireless and mobile technologies in education (WMTE’05). Tokushima, Japan, pp 5–268. https://doi.org/10.1109/WMTE.2005.63

62.

Baijal A, Kim J, Branje C, Russo F, Fels DI (2012) Composing vibrotactile music: a multisensory experience with the Emoti-chair. In: 2012 IEEE haptics symposium (HAPTICS), pp 509–515. https://doi.org/10.1109/HAPTIC.2012.6183839

63.

Merchel S, Altinsoy ME (2013) Auditory-tactile music perception. J Acoust Soc Am 133(5 Supplement):3256. https://doi.org/10.1121/1.4805254ADSCrossRef

64.

Altinsoy E (2006) Auditory tactile interaction in virtual environments. Ruhr University Bochum, pp 1-166

65.

Merchel S, Altinsoy E (2009) Vibratory and acoustical factors in multimodal reproduction of concert DVDs. In: Altinsoy ME, Jekosch U, Brewster S (eds) Haptic and audio interaction design. Springer, Berlin, pp 119–127CrossRef

66.

Leman M, Maes PJ (2015) The role of embodiment in the perception of music. Empir Musicol Rev. https://doi.org/10.18061/emr.v9i3-4.4498CrossRef

67.

Maes PJ, Leman M, Palmer C, Wanderley MM (2014) Action-based effects on music perception. Front Psychol 3(4):1008. https://doi.org/10.3389/fpsyg.2013.01008CrossRef

68.

Crippen M (2017) Embodied cognition and perception: Dewey. science and skepticism. Contemp Pragmatism 14:112–134. https://doi.org/10.1163/18758185-01401007CrossRef

69.

Macedonia M (2019) Embodied learning: why at school the mind needs the body. Front Psychol 10:2098. https://doi.org/10.3389/fpsyg.2019.02098

70.

Merchel S, Altinsoy ME (2018) Auditory-tactile experience of music. In: Papetti S, Saitis C (eds) Musical haptics. Springer Series on Touch and Haptic Systems. Springer, Cham. https://doi.org/10.1007/978-3-319-58316-7_7

71.

Dimitrov DM, Rumrill PD Jr (2003) Pretest-posttest designs and measurement of change. Work 20(2):159–65PubMed

72.

Juntunen M (2006) Exploring and learning music through embodied experiences. Music and development - challenges for music education : In: The proceedings of the first European conference on developmental psychology of music: 17–19 November 2005, University of Jyvaskyla, Department of Music, Finland, pp 273–276

73.

Kivijärvi S, Sutela K, Ahokas JR (2016) A conceptual discussion of embodiment in special music education: Dalcroze Eurhythmics as a case. Approaches Interdiscip J Music Ther 8:169–178

74.

Parianen Lesemann FH, Reuter EM, Godde B (2015) Tactile stimulation interventions: influence of stimulation parameters on sensorimotor behavior and neurophysiological correlates in healthy and clinical samples. Neurosci Biobehav Rev 51:126–137. https://doi.org/10.1016/j.neubiorev.2015.01.005CrossRefPubMed

75.

Einarson KM, Trainor LJ (2015) The effect of visual information on young children’s perceptual sensitivity to musical beat alignment. Timing Time Percept 3(1–2):88–101. https://doi.org/10.1163/22134468-03002039CrossRef

76.

Provasi J, Anderson DI, Barbu-Roth M (2014) Rhythm perception, production, and synchronization during the perinatal period. Front Psychol 18(5):1048. https://doi.org/10.3389/fpsyg.2014.01048CrossRef

77.

Kirschner S, Tomasello M (2009) Joint drumming: social context facilitates synchronization in preschool children. J Exp Child Psychol 102(3):299–314. https://doi.org/10.1016/j.jecp.2008.07.005CrossRefPubMed

78.

Barros CG, Swardfager W, Moreno S, Bortz G, Ilari B, Jackowski AP, Ploubidis G, Little TD, Lamont A, Cogo-Moreira H (2017) Assessing music perception in young children: evidence for and psychometric features of the M-factor. Front Neurosci 11:18. https://doi.org/10.3389/fnins.2017.00018CrossRefPubMedPubMedCentral

79.

Asztalos K, Csapó B (2017) Development of musical abilities: cross-sectional computer-based assessments in educational contexts. Psychol Music 45(5):682–698. https://doi.org/10.1177/0305735616678055CrossRef

Title: Multimodal exploration in elementary music classroom
Authors: Martha Papadogianni
Ercan Altinsoy
Areti Andreopoulou
Publication date: 18-10-2023
Publisher: Springer International Publishing
Published in: Journal on Multimodal User Interfaces / Issue 1/2024
Print ISSN: 1783-7677
Electronic ISSN: 1783-8738
DOI: https://doi.org/10.1007/s12193-023-00420-x

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 1/2024

A social robot as your reading companion: exploring the relationships between gaze patterns and knowledge gains

Hearing loss prevention at loud music events via real-time visuo-haptic feedback

Review of substitutive assistive tools and technologies for people with visual impairments: recent advancements and prospects

A study on the attention of people with low vision to accessibility guidance signs

Pegasos: a framework for the creation of direct mobile coaching feedback systems

Modelling the “transactive memory system” in multimodal multiparty interactions

Premium Partner