Toward Robotic Socially Believable Behaving Systems - Volume I

In this work I am providing comments on some important factors that must be accounted for when robotics comes near to the human body, as in the case of wearable robots. In addition to well-known socio-demographic factors such as sex/gender, age, ethnicity, skin color and emotional and culturally-determined sense of personal space, characteristics of health professionals and caregivers, here I stress the necessity to take into account also fashion and the questions raised by it.

This book aims to investigate the features that are at the core of human interactions to model the involved emotional processes, in order to design and develop autonomous systems and algorithms able to detect early signs of changes, in moods and emotional states. The attention is focused on emotional social features and the human’s ability to decode and encode emotional social cues while interacting. In order to do this, the book will propose a series of investigations that gather behavioral data from speech, handwriting, facial, vocal and gestural expressions. This is done through the definition of behavioral tasks that may serve to produce changes in the perception of emotional social cues. Specific scenarios are designed to assess users’ emphatic and social competencies. The collected data are used to gain knowledge on how behavioral and interactional features are affected by individuals’ moods and emotional states. This information can be exploited to devise multidimensional models of multimodal interactional features that will serve for measuring the degree of empathic relationships developed between individuals and allow the design and development of cost-effective emotion-aware technologies to be used in applicative contexts such as remote health care services and robotic assistance.

We argue that the development of robots that can interact effectively with people requires a special focus on building systems that can perceive and comprehend intentions in other agents. Such a capability is a prerequisite for all pro-social behaviour and in particular underpins the ability to engage in instrumental helping and mutual collaboration. We explore the prospective and intentional nature of action, highlighting the importance of joint action, shared goals, shared intentions, and joint attention in facilitating social interaction between two or more cognitive agents. We discuss the link between reading intentions and theory of mind, noting the role played by internal simulation, especially when inferring higher-level action-focussed intentions. Finally, we highlight that pro-social behaviour in humans is the result of a developmental process and we note the implications of this for the challenge of creating cognitive robots that can read intentions.

Technology is the future, woven into every aspect of our lives, but how are we to interact with all this technology and what happens when problems arise? Artificial agents, such as virtual characters and social robots could offer a realistic solution to help facilitate interactions between humans and machines—if only these agents were better equipped and more informed to hold up their end of an interaction. People and machines can interact to do things together, but in order to get the most out of every interaction, the agent must to be able to make reasonable judgements regarding your intent and goals for the interaction. We explore the concept of engagement from the different perspectives of the human and the agent. More specifically, we study how the agent perceives the engagement state of the other interactant, and how it generates its own representation of engaging behaviour. In this chapter, we discuss the different stages and components of engagement that have been suggested in the literature from the applied perspective of a case study of engagement for social robotics, as well as in the context of another study that was focused on gaze-related engagement with virtual characters.

Recent years have seen a growing interest in applying insights from developmental psychology to build artificial intelligence and robotic systems. This endeavour, called developmental robotics, not only is a novel method of creating artificially intelligent systems, but also offers a new perspective on the development of human cognition. While once cognition was thought to be the product of the embodied brain, we now know that natural and artificial cognition results from the interplay between an adaptive brain, a growing body, the physical environment and a responsive social environment. This chapter gives three examples of how humanoid robots are used to unveil aspects of development, and how we can use development and learning to build better robots. We focus on the domains of word-meaning acquisition, abstract concept acquisition and number acquisition, and show that cognition needs embodiment and a social environment to develop. In addition, we argue that Spiking Neural Networks offer great potential for the implementation of artificial cognition on robots.

This article joins the fields of emotion recognition and human computer interaction. While much work has been done on recognizing emotions, they are hardly used to improve a user’s interaction with a system. Although the fields of affective computing and especially serious games already make use of detected emotions, they tend to provide application and user specific adaptions only on the task level. We present an approach of utilizing recognized emotions to improve the interaction itself, independent of the underlying application at hand. Examining the state of the art in emotion recognition research and based on the architecture of Companion-System, a generic approach for determining the main cause of an emotion within the history of interactions is presented, allowing a specific reaction and adaption. Using such an approach could lead to systems that use emotions to improve not only the outcome of a task but the interaction itself in order to be truly individual and empathic.

The aim of the present study is to use the GRID, online emotions sorting and corpus methodologies to illuminate different types of disgust that an emotion-sensitive socially interacting robot would need to encode and decode in order to competently produce and recognise these and other types of physical, moral and aesthetic types of complex emotions in social settings. We argue that emotions in general, and different types of disgust as an instance of these, differ with respect to the amount of cognitive grounding they need in order to arise and social robots will successfully use such emotions provided they do not only recognise and produce physical, bodily manifestations of emotions, but also have access to large knowledge bases and are able to process situational context clues. The different types of disgust are identified and compared cross-culturally to provide an evaluation of their relative salience. The study also underscores the conceptual viewpoint of emotions as clusters of emotions rather than solitary, individual representations. We argue that such clustering should be at the heart of emotions modelling in social robots. In order to successfully use the emotion of disgust in their interactions with humans, robots need to be sensitive to possible within-culture and cross-culture differences pertaining to such emotions, exemplified by British English and Polish in the present study. Given the centrality of values to the emotion of disgust, robots need to have the capacity to update from a knowledge base and learn from the situational context the set of values for each significant human that they interact with.

Here we report the results of an experiment aimed to investigate the effects of different hand gestures on emotional and attitudinal reactions of receivers through the measure of physiological indexes (facial muscles activity, heart rate, and eye blinking). A videotape was shown to 50 University students, in which an actress presented a speech with a disagreeable verbal content, namely the proposal of increasing University fees. The verbal message included the presentation of four arguments (two strong and two weak) in support of the proposal. During her speech, the actress manipulated “Gesture Type” in order to achieve five conditions (ideational gestures, discursive gestures, object- and self-adaptors, and no gesture as control). ANOVAs reveal that the different type of gestures differently modulate the negative impact of disagreeable verbal content on receivers in different moments of the speech and interact with strong or weak arguments in determining negative reactions to the disagreeable verbal message. In particular, it seems that discursive (conversational and ideational) gestures are more capable to counteract the negative effect of the arguments. These results give a further contribution to a better understanding of the crucial role of gestures in providing characteristics of speech perception, also in terms of persuasion.

Laughter is everywhere. So much so that we often do not even notice it. First, laughter has a strong connection with humour. Most of us seek out laughter and people who make us laugh, and it is what we do when we gather together as groups relaxing and having a good time. But laughter also plays an important role in making sure we interact with each other smoothly. It provides social bonding signals that allow our conversations to flow seamlessly between topics; to help us repair conversations that are breaking down; and to end our conversations on a positive note.

The multimodal HuComTech corpus aims at annotating, studying and publishing data related to a wide spectrum of markers of human behavior in human-human spoken dialogues. By doing so the final goal is to both understand human cognitive behavior in conversational settings and contribute to the enhancement of human-machine interaction systems. One of the main issues still leaving wide spaces for further development is related to speech prosody, the understanding of its association with possible cognitive processes for the expression of emotions as well as the online production of speech utterances. Since the latter often results in incomplete structures, the study of the relation between grammatical incompleteness and prosody can both contribute to a better understanding of human cognition and the enhancement of cognitive infocommunication systems. The data and analyses presented in this paper are intended to serve both these purposes. Two different approaches will be presented as methods of data exploration: the study of static temporal alignments within the ELAN annotation tool, and the discovery of dynamic temporal patterns using the Theme framework.

Speech carries information not only about the lexical content, but also about the age, gender, signature and emotional state of the speaker. Speech in different emotional states is accompanied by distinct changes in the production mechanism. In this chapter, we present a review of analysis methods used for emotional speech. In particular, we focus on the issues in data collection, feature representations and development of automatic emotion recognition systems. The significance of the excitation source component of speech production in emotional states is examined in detail. The derived excitation source features are shown to carry the emotion correlates.