nach oben

2014 | Buch

Kapitel lesen Erstes Kapitel lesen

Innovative and Creative Developments in Multimodal Interaction Systems

9th IFIP WG 5.5 International Summer Workshop on Multimodal Interfaces, eNTERFACE 2013, Lisbon, Portugal, July 15 – August 9, 2013. Proceedings

herausgegeben von: Yves Rybarczyk, Tiago Cardoso, João Rosas, Luis M. Camarinha-Matos

Verlag: Springer Berlin Heidelberg

Buchreihe : IFIP Advances in Information and Communication Technology

Enthalten in: Springer Professional "Wirtschaft+Technik" , Springer Professional "Technik" , Springer Professional "Wirtschaft"

Einloggen, um Zugang zu erhalten

Über dieses Buch

This book contains the outcome of the 9th IFIP WG 5.5 International Summer Workshop on Multimodal Interfaces, eNTERFACE 2013, held in Lisbon, Portugal, in July/August 2013. The 9 papers included in this book represent the results of a 4-week workshop, where senior and junior researchers worked together on projects tackling new trends in human-machine interaction (HMI). The papers are organized in two topical sections. The first one presents different proposals focused on some fundamental issues regarding multimodal interactions, i.e., telepresence, speech synthesis and interactive modeling. The second is a set of development examples in key areas of HMI applications, i.e., education, entertainment and assistive technologies.

Inhaltsverzeichnis

Frontmatter

Fundamental Issues

Frontmatter

Body Ownership of Virtual Avatars: An Affordance Approach of Telepresence

Abstract

Virtual environments are an increasing trend in today’s society. In this scope, the avatar is the representation of the user in the virtual world. However, that relationship lacks empirical studies regarding the nature of the interaction between avatars and human beings. For that purpose it was studied how the avatar’s modeled morphology and dynamics affect its control by the user. An experiment was conducted to measure telepresence and ownership on participants who used a Kinect Natural User Interface (NUI). The body ownership of different avatars was assessed through a behavioral parameter, based on the concept of affordances, and a questionnaire of presence. The results show that the feelings of telepresence and ownership seem to be greater when the kinematics and the avatar’s proportions are closer to those of the user.

Tiago Coelho, Rita de Oliveira, Tiago Cardoso, Yves Rybarczyk

Reactive Statistical Mapping: Towards the Sketching of Performative Control with Data

Abstract

This paper presents the results of our participation to the ninth eNTERFACE workshop on multimodal user interfaces. Our target for this workshop was to bring some technologies currently used in speech recognition and synthesis to a new level, i.e. being the core of a new HMM-based mapping system. The idea of statistical mapping has been investigated, more precisely how to use Gaussian Mixture Models and Hidden Markov Models for realtime and reactive generation of new trajectories from inputted labels and for realtime regression in a continuous-to-continuous use case. As a result, we have developed several proofs of concept, including an incremental speech synthesiser, a software for exploring stylistic spaces for gait and facial motion in realtime, a reactive audiovisual laughter and a prototype demonstrating the realtime reconstruction of lower body gait motion strictly from upper body motion, with conservation of the stylistic properties. This project has been the opportunity to formalise HMM-based mapping, integrate various of these innovations into the Mage library and explore the development of a realtime gesture recognition tool.

Nicolas d’Alessandro, Joëlle Tilmanne, Maria Astrinaki, Thomas Hueber, Rasmus Dall, Thierry Ravet, Alexis Moinet, Huseyin Cakmak, Onur Babacan, Adela Barbulescu, Valentin Parfait, Victor Huguenin, Emine Sümeyye Kalaycı, Qiong Hu

Laugh When You’re Winning

Abstract

Developing virtual characters with naturalistic game playing capabilities is an increasingly researched topic in Human-Computer Interaction. Possible roles for such characters include virtual teachers, personal care assistants, and companions for children. Laughter is an under-investigated emotional expression both in Human-Human and Human-Computer Interaction. The EU Project ILHAIRE, aims to study this phenomena and endow machines with laughter detection and synthesis capabilities. The Laugh when you’re winning project, developed during the eNTERFACE 2013 Workshop in Lisbon, Portugal, aimed to set up and test a game scenario involving two human participants and one such virtual character. The game chosen, the yes/no game, induces natural verbal and non-verbal interaction between participants, including frequent hilarious events, e.g., one of the participants saying “yes” or “no” and so losing the game. The setup includes software platforms, developed by the ILHAIRE partners, allowing automatic analysis and fusion of human participants’ multimodal data (voice, facial expression, body movements, respiration) in real-time to detect laughter. Further, virtual characters endowed with multimodal skills were synthesised in order to interact with the participants by producing laughter in a natural way.

Maurizio Mancini, Laurent Ach, Emeline Bantegnie, Tobias Baur, Nadia Berthouze, Debajyoti Datta, Yu Ding, Stéphane Dupont, Harry J. Griffin, Florian Lingenfelser, Radoslaw Niewiadomski, Catherine Pelachaud, Olivier Pietquin, Bilal Piot, Jérôme Urbain, Gualtiero Volpe, Johannes Wagner

Tutoring Robots

Multiparty Multimodal Social Dialogue with an Embodied Tutor

Abstract

This project explores a novel experimental setup towards building spoken, multi-modally rich, and human-like multiparty tutoring agent. A setup is developed and a corpus is collected that targets the development of a dialogue system platform to explore verbal and nonverbal tutoring strategies in multiparty spoken interactions with embodied agents. The dialogue task is centered on two participants involved in a dialogue aiming to solve a card-ordering game. With the participants sits a tutor that helps the participants perform the task and organizes and balances their interaction. Different multimodal signals captured and auto-synchronized by different audio-visual capture technologies were coupled with manual annotations to build a situated model of the interaction based on the participants personalities, their temporally-changing state of attention, their conversational engagement and verbal dominance, and the way these are correlated with the verbal and visual feedback, turn-management, and conversation regulatory actions generated by the tutor. At the end of this chapter we discuss the potential areas of research and developments this work opens and some of the challenges that lie in the road ahead.

Samer Al Moubayed, Jonas Beskow, Bajibabu Bollepalli, Ahmed Hussen-Abdelaziz, Martin Johansson, Maria Koutsombogera, José David Lopes, Jekaterina Novikova, Catharine Oertel, Gabriel Skantze, Kalin Stefanov, Gül Varol

Touching Virtual Agents: Embodiment and Mind

Abstract

In this paper we outline the design and development of an embodied conversational agent setup that incorporates an augmented reality screen and tactile sleeve. With this setup the agent can visually and physically touch the user. We provide a literature overview of embodied conversational agents, as well as haptic technologies, and argue for the importance of adding touch to an embodied conversational agent. Finally, we provide guidelines for studies involving the touching virtual agent (TVA) setup.

Gijs Huisman, Merijn Bruijnes, Jan Kolkmeier, Merel Jung, Aduén Darriba Frederiks, Yves Rybarczyk

Key Applications

Frontmatter

Kinect-Sign: Teaching Sign Language to “Listeners” through a Game

Abstract

Sign language is the hearing impaired form of communicating with other people, including listeners. Most cases, impaired people have learned sign language form childhood. The problem arises when a listener comes in contact with an impaired person. For instances, if a couple has a child which is impaired, the parents find a challenge to learn the sign language. In this article, a new playful approach to assist the listeners to learn sign language is proposed. This proposal is a serious game composed of two modes: School-mode and Competition-mode. The first offers a virtual school where the user learns to sign letters and the second offers an environment towards applying the learned letters. Behind the scenes, the proposal contains a sign language recognition system, based on three modules: 1 – the standardization of the Kinect depth camera data; 2 – a gesture library relying on the standardized data; and 3 – the real-time recognition of gestures. A prototype was developed – Kinect-Sign – and tested in a Portuguese Sign-Language school and on eNTERFACE’13 resulting in a joyful acceptance of the approach.

João Gameiro, Tiago Cardoso, Yves Rybarczyk

Hang in There: A Novel Body-Centric Interactive Playground

Abstract

We introduce and evaluate a first version of a novel body-centric playground that aims at increasing bodily exertion and immersion. The concept centers around the player, who is suspended from the ceiling using a rope and climbing harness and stands on a tilted platform. This caused players to assume a body posture that elicits the feeling of flying, which was further enhanced by the flying game that they played. We discuss the choices made in the type of body movements, and how these relate to different aspects such as movement mimicry and exertion. We performed a user study, in which the hanging position was compared to a setting where players stood on the ground. We found no significant differences in the amount of movement and perceived engagement between the two conditions. However, there was a tendency of favoring the hanging position. Moreover, we observed that the placement of game elements affected the movement patterns.

Robby van Delden, Alejandro Moreno, Carlos Ramos, Gonçalo Carrasco, Dennis Reidsma, Ronald Poppe

KINterestTV - Towards Non–invasive Measure of User Interest While Watching TV

Abstract

Is it possible to determine only by observing the behavior of a user what are his interests for a media? The aim of this project is to develop an application that can detect whether or not a user is viewing a content on the TV and use this information to build the user profile and to make it evolve dynamically. Our approach is based on the use of a 3D sensor to study the movements of a user’s head to make an implicit analysis of his behavior. This behavior is synchronized with the TV content (media fragments) and other user interactions (clicks, gestural interaction) to further infer viewer’s interest. Our approach is tested during an experiment simulating the attention changes of a user in a scenario involving second screen (tablet) interaction, a behavior that has become common for spectators and a typical source of attention switches.

Julien Leroy, François Rocca, Matei Mancas, Radhwan Ben Madhkour, Fabien Grisard, Tomas Kliegr, Jaroslav Kuchar, Jakub Vit, Ivan Pirner, Petr Zimmermann

Development of an Ecosystem for Ambient Assisted Living

Abstract

Society is facing big demographic changes. In 2050, it is expected that the number of elders will reach 1500 million (about 16% of the world population). As people age, they become more dependent on assistance services. Care and assistance organizations start failing, as the number of people who need help increases beyond their ability to comply. The creation of an ecosystem for Ambient Assisted Living, facilitating partnerships creation between service providers, is proposed as a strategy to improve care provision and leverage its capacity. The specification of the ecosystem is based on canonical models and verified through simulation.

João Rosas, Luis M. Camarinha-Matos, Gonçalo Carvalho, Ana Inês Oliveira, Filipa Ferrada

Backmatter

Titel: Innovative and Creative Developments in Multimodal Interaction Systems
herausgegeben von: Yves Rybarczyk
Tiago Cardoso
João Rosas
Luis M. Camarinha-Matos
Verlag: Springer Berlin Heidelberg
Electronic ISBN: 978-3-642-55143-7
Print ISBN: 978-3-642-55142-0
DOI: https://doi.org/10.1007/978-3-642-55143-7