Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben

2011 | Buch

Musical Robots and Interactive Multimodal Systems: An Introduction Kapitel lesen Erstes Kapitel lesen

Musical Robots and Interactive Multimodal Systems

herausgegeben von: Jorge Solis, Kia Ng

Verlag: Springer Berlin Heidelberg

Buchreihe : Springer Tracts in Advanced Robotics

Enthalten in: Springer Professional "Wirtschaft+Technik" , Springer Professional "Technik" , Springer Professional "Wirtschaft"

Einloggen, um Zugang zu erhalten
insite
SUCHEN

Über dieses Buch

Musical robotics is a multi- and trans-disciplinary research area involving a wide range of different domains that contribute to its development, including: computer science, multimodal interfaces and processing, artificial intelligence, electronics, robotics, mechatronics and more. A musical robot requires many different complex systems to work together; integrating musical representation, techniques, expressions, detailed analysis and controls, for both playing and listening. The development of interactive multimodal systems provides advancements which enable enhanced human-machine interaction and novel possibilities for embodied robotic platforms. This volume is focused on this highly exciting interdisciplinary field. This book consists of 14 chapters highlighting different aspects of musical activities and interactions, discussing cutting edge research related to interactive multimodal systems and their integration with robots to further enhance musical understanding, interpretation, performance, education and enjoyment. It is dichotomized into two sections: Section I focuses on understanding elements of musical performance and expression while Section II concentrates on musical robots and automated instruments. Musical Robots and Interactive Multimodal Systems provides an introduction and foundation for researchers, students and practitioners to key achievements and current research trends on interactive multimodal systems and musical robotics.

Inhaltsverzeichnis

Frontmatter

Musical Robots and Interactive Multimodal Systems: An Introduction

Musical Robots and Interactive Multimodal Systems: An Introduction
Context
As early as 17th and 18th centuries different researchers have been interested in studying how humans are capable of playing different kinds of musical instruments as an approach to provide a credible imitation of life. As an early example, in 1738, Jacques de Vaucanson (1709-1782) developed the “Flute Player” automaton that mechanically reproduces the organs required for playing the flute and proposed its development as one way for understanding the human breathing mechanism.
Jorge Solis, Kia Ng

Section I: Understanding Elements of Musical Performance and Expression

Sound-Action Chunks in Music
Abstract
One core issue in music production and perception is the relationship between sound features and action features. From various recent research, it seems reasonable to claim that most people, regardless of levels of musical expertise, have fairly good knowledge of the relationship between sound and sound production, as e.g. manifest in various cases of ’air instrument’ performance and other spontaneous body movements to musical sound. The challenge now is to explore these sound-action links further, in particular at the micro-levels of musical sound such as in timbre and texture, and at the meso-levels of various rhythmical and contoural features. As suggested by the seminal work of Pierre Schaeffer and co-workers on so-called sonic objects several decades ago, perceptually salient features can be found on the chunk-level in music, meaning in fragments of sound-action in the approximately 0.5 to 5 seconds range. In this chapter, research on the emergence of sound-action chunks and their features will be presented together with some ideas for practical applications.
Rolf Inge Godøy
Automatic Music Transcription: From Monophonic to Polyphonic
Abstract
Music understanding from an audio track and performance is a key problem and a challenge for many applications ranging from: automated music transcoding, music education, interactive performance, etc. The transcoding of polyphonic music is a one of the most complex and still open task to be solved in order to become a common tool for the above mentioned applications. Techniques suitable for monophonic transcoding have shown to be largely unsuitable for polyphonic cases. Recently, a range of polyphonic transcoding algorithms and models have been proposed and compared against worldwide accepted test cases such as those adopted in the MIREX competition. Several different approaches are based on techniques such as: pitch trajectory analysis, harmonic clustering, bispectral analysis, event tracking, nonnegative matrix factorization, hidden Markov model. This chapter analyzes the evolution of music understanding algorithms and models from monophonic to polyphonic, showing and comparing the solutions, while analysing them against commonly accepted assessment methods and formal metrics.
Fabrizio Argenti, Paolo Nesi, Gianni Pantaleo
Multimodal Analysis of Expressive Gesture in Music Performance
Abstract
This chapter focuses on systems and interfaces for multimodal analysis of expressive gesture as a key element of music performance. Research on expressive gesture became particularly relevant in recent years. Psychological studies have been a fundamental source for automatic analysis of expressive gesture since their contribution in identifying the most significant features to be analysed. A further relevant source has been research in the humanistic tradition, in particular choreography. As a major example, in his Theory of Effort, choreographer Rudolf Laban describes the most significant qualities of movement. Starting from these sources, several models, systems, and techniques for analysis of expressive gesture were developed. This chapter presents an overview of methods for the analysis, modelling, and understanding of expressive gesture in musical performance. It introduces techniques resulted from the research developed over the years by the authors: from early experiments of human-robot interaction in the context of music performance up to recent set-ups of innovative interfaces and systems for active experience of sound and music content. The chapter ends with an overview of possible future research challenges.
Antonio Camurri, Gualtiero Volpe
Input Devices and Music Interaction
Abstract
This chapter discusses some principles of digital musical instrument design in the context of different goals and constraints.  It shows, through several examples, that a variety of conditions can motivate design choices for sensor interface and mapping, such as robustness and reliability, environmental constraints on sensor technology, or the desire for haptic feedback.  Details of specific hardware and software choices for some DMI designs are discussed in this context.
Joseph Malloch, Stephen Sinclair, Avrum Hollinger, Marcelo M. Wanderley
Capturing Bowing Gesture: Interpreting Individual Technique
Abstract
Virtuosic bowed string performance in many ways exemplifies the incredible potential of human physical performance and expression. Today, a great deal is known about the physics of the violin family and those factors responsible for its sound capabilities. However, there remains much to be discovered about the intricacies of how players control these instruments in order to achieve their characteristic range and nuance of sound. Today, technology offers the ability to study this player control under realistic, unimpeded playing conditions to lead to greater understanding of these performance skills. Presented here is a new methodology for investigation of bowed string performance that uses a playable hardware measurement system to capture the gestures of right hand violin bowing technique. This measurement system (which uses inertial, force, and electric field position sensors) was optimized to be small, lightweight, and portable and was installed on a carbon fiber violin bow and an electric violin to enable study of realistic, unencumbered violin performances. The application of this measurement system to the study of standard bowing techniques, including détaché, martelé, and spiccato, and to the study of individual players themselves, is discussed.
Diana S. Young
Interactive Multimedia for Technology-Enhanced Learning with Multimodal Feedback
Abstract
Musical performances are generally physically demanding with high degree of control (mental and motor) and accuracy. This chapter presents the i-Maestro (www.i-maestro.org) project which explored interactive multimedia environments for technology-enhanced music education. It discusses one of the key requirements for an interactive musical robot which is to analyze and provide feedback/interaction to a “performance”. This Chapter also serves as an example application of a musical robot in educational contexts. Music is not simply playing the right note at the right time. The multitude of interconnecting factors that influence and contribute to the nature of the playing is not easy to monitor nor analyze. Musical instrumentalists often use mirrors to observe themselves practicing. This Chapter briefly introduces the i-Maestro project and focuses on a gesture interface developed under the i-Maestro framework called the 3D Augmented Mirror (AMIR). AMIR captures, analyze and visualizes the performance in 3D. It offers a number of different analyses and feedback to support the learning and teaching of bowing technique and gesture.
Kia Ng
Online Gesture Analysis and Control of Audio Processing
Abstract
This chapter presents a general framework for gesture-controlled audio processing. The gesture parameters are assumed to be multi-dimensional temporal profiles obtained from movement or sound capture systems. The analysis is based on machine learning techniques, comparing the incoming dataflow with stored templates. The mapping procedures between the gesture and the audio processing include a specific method we called temporal mapping. In this case, the temporal evolution of the gesture input is taken into account in the mapping process. We describe an example of a possible use of the framework that we experimented with in various contexts, including music and dance performances, music pedagogy and installations.
Frédéric Bevilacqua, Norbert Schnell, Nicolas Rasamimanana, Bruno Zamborlin, Fabrice Guédy

Section II: Musical Robots and Automated Instruments

Automated Piano: Techniques for Accurate Expression of Piano Playing
Abstract
The challenge of developing an automated piano that accurately produce the soft tones of a desired performance which is a problem encountered by pianists themselves, led to a reconsideration of the touch involved in producing soft piano tones. For this purpose, the behavior of the piano’s action mechanism was measured and observed based on a weight effect which is one of the pianist’s performance techniques, and the accurate expression of soft tones was realized. Furthermore, although double-strikes by the hammer and non-musical sounds occur during the performance of soft tones, such sound effects were cancelable. This Chapter describes the development of a superior automated piano which is capable of reproducing a desired performance with expressive soft tones. The piano’s hardware and software have been developed, and the piano’s action mechanism has been analyzed with reference to the “touch.” This fundamental input waveform can be used to accurately and automatically reproduce a key touch based on performance information for a piece of classical music. The user can accurately create an emotionally expressive performance according to an idea without moving the body of the pianist.
Eiji Hayashi
McBlare: A Robotic Bagpipe Player
Abstract
McBlare is a robotic bagpipe player developed by the Robotics Institute and Computer Science Department at Carnegie Mellon University. This project has taught us some lessons about bagpipe playing and control that are not obvious from subjective human experience with bagpipes. From the artistic perspective, McBlare offers an interesting platform for virtuosic playing and interactive control. McBlare plays a standard set of bagpipes, using a custom air compressor to supply air and electromechanical “fingers” to control the chanter. McBlare is MIDI controlled, allowing for simple interfacing to a keyboard, computer, or hardware sequencer. The control mechanism exceeds the measured speed of expert human performers. McBlare can perform traditional bagpipe music as well as experimental computer-generated music. One characteristic of traditional bagpipe performance is the use of ornaments, or very rapid sequences of up to several notes inserted between longer melody notes. Using a collection of traditional bagpipe pieces as source material, McBlare can automatically discover typical ornaments from examples and insert ornaments into new melodic sequences. Recently, McBlare has been interfaced to control devices to allow non-traditional bagpipe music to be generated with real-time, continuous gestural control.
Roger B. Dannenberg, H. Ben Brown, Ron Lupish
Violin Playing Robot and Kansei
Abstract
Kansei is a Japanese word similar in meaning to “sensibility,” “feeling,” “mood,” etc. Although kansei seems to affect musical instrument playing greatly, many musical instrument playing robots do not utilize kansei very well. Violin playing has been chosen as an example because it is very difficult, and it seems that kansei affects it more clearly than other musical instruments. First, in this chapter, a violin playing-robot is introduced and the sounds produced by the robot are analyzed and discussed. The robot consists of an anthropomorphic right arm offering 7 degrees of freedom (DOF) with a simple hand that grasps a bow. By using the arm, the robot can produce various sounds. The left hand fingers of the robot which are under development are presented next. After that, the information flow of a violin-playing robot from musical notes to sounds considering kansei is proposed. As a first approach, in the flow, timbre is regarded as kansei information, and kansei mainly affects processes that determine sound data from musical notes. Then, human violinists’ kansei is analyzed based on the flow. It has been found that bow force and sounding point play important roles in determining timbre that results from human kansei.
Koji Shibuya
Wind Instrument Playing Humanoid Robots
Abstract
Since the golden era of automata (17th and 18th centuries), the development of mechanical dolls served as a mean to understand how the human brain is able of coordinating multiple degrees of freedom in order to play musical instruments. A particular interest was given to wind instruments as a research approach since this requires the understanding of human breathing mechanisms. Nowadays, different kinds of automated machines and humanoid robots have been developed to play wind instruments. In this chapter, we will detail some issues related to the development of humanoid robots and the challenges in their design, control and system integration for playing wind instruments.
Jorge Solis, Atsuo Takanishi
Multimodal Techniques for Human/Robot Interaction
Abstract
Is it possible for a robot to improvise with a human performer in real-time? This chapter describes a framework for interdisciplinary research geared towards finding solutions to this question. Custom built controllers, influenced by the Human Computer Interaction (HCI) community, serve as new interfaces to gather musical gestures from a performing artist. Designs on how to modify a sitar, the 19-stringed North Indian string instrument, with sensors and electronics are described. Experiments using wearable sensors to capture ancillary gestures of a human performer are also included. A twelve-armed solenoid-based robotic drummer was built to perform on a variety of traditional percussion instruments from around India. The chapter describes experimentation on interfacing a human sitar performer with the robotic drummer. Experiments include automatic tempo tracking and accompaniment methods. This chapter shows contributions in the areas of musical gesture extraction, musical robotics and machine musicianship. However, one of the main novelties was completing the loop and fusing all three of these areas together into a real-time framework.
Ajay Kapur
Interactive Improvisation with a Robotic Marimba Player
Abstract
Shimon is an improvisational robotic marimba player that listens to human co-players and responds musically and choreographically based on analysis of musical input. The paper discusses the robot’s mechanical and motion control and presents a novel interactive improvisation system based on the notion of physical gestures. Our system uses anticipatory action to enable real-time improvised synchronization with the human player. It was implemented on a full-length human-robot Jazz duet, displaying coordinated melodic and rhythmic human-robot joint improvisation. We also describe a study evaluating the effect of visual cues and embodiment on one of our call-and-response improvisation module. Our findings indicate that synchronization is aided by visual contact when uncertainty is high. We find that visual coordination is more effective for synchronization in slow sequences compared to faster sequences, and that occluded physical presence may be less effective than audio-only note generation.
Guy Hoffman, Gil Weinberg
Interactive Musical System for Multimodal Musician-Humanoid Interaction
Abstract
The research on Humanoid Robots designed for playing musical instruments has a long tradition in the research field of robotics. During the past decades, several researches are developing anthropomorphic and automated machines able to create live musical performances for both understanding the human itself and for creating novel ways of musical expression. In particular, Humanoid Robots are being designed to roughly simulate the dexterity of human players and to display higher-level of perceptual capabilities to enhance the interaction with musical partners. In this chapter, the concept and implementation of an interactive musical system for multimodal musician-humanoid interaction is detailed.
Jorge Solis, Klaus Petersen, Atsuo Takanishi
Backmatter
Metadaten
Titel
Musical Robots and Interactive Multimodal Systems
herausgegeben von
Jorge Solis
Kia Ng
Copyright-Jahr
2011
Verlag
Springer Berlin Heidelberg
Electronic ISBN
978-3-642-22291-7
Print ISBN
978-3-642-22290-0
DOI
https://doi.org/10.1007/978-3-642-22291-7

Neuer Inhalt

    Bildnachweise