Social Robotics + AI

Inhaltsverzeichnis

1. Frontmatter

2. Emotion and Affective Interaction

   1. Frontmatter

   2. Emotionally Adaptive Conversational Models for Long-Term Human-Robot Interaction Using Proximal Policy Optimization

      Patrick Houman Jair, Chung Hyuk Park

      Abstract

      As socially assistive robots become increasingly integrated into everyday life, their ability to engage in emotionally intelligent interactions is essential for supporting long-term human-robot relationships. This work presents a reinforcement learning-based conversational model that dynamically adapts to the user’s emotional state across multi-turn dialogues. Using Proximal Policy Optimization (PPO), the system selects appropriate response types based on emotion labels extracted from user speech. A pre-trained emotion classifier (DistilRoBERTa) processes speech-to-text input, and the selected response type guides a GPT-based response generator. The model is deployed on Pepper, a humanoid robot, and evaluated through a two-week longitudinal user study comparing static and adaptive dialogue systems. Results demonstrate that the PPO-powered adaptive model significantly improves user engagement and reduces stress levels over time. This study highlights the importance of reinforcement learning for sustained emotional coherence and opens pathways for emotionally adaptive robots in mental health, education, and social support applications.

   3. Individual Differences in Social and Emotional Responses to Robotic Dining Companions: Toward Personalized Interaction Design

      Hunter Fong, Selim Soufargi, Yifei Li, Maurizio Mancini, Radoslaw Niewiadomski

      Abstract

      This study investigates how affective and personality traits shape users’ emotional and social responses to a robotic dining companion in a commensal context. The companion is a NAO robot, equipped with a GPT-based dialogue and a commensal activity visual detection module, able to engage in mealtime conversation and exhibit responsive nonverbal behaviors. Twenty-two participants shared a meal with the robot and completed pre- and post-interaction measures of personality, affect, commensality habits, enjoyment of the interaction, and perceived social connection. Results showed that participants high in openness reported greater enjoyment of the interaction and more positive situational affect, while those high in negative trait affect also reported high enjoyment of the interaction, suggesting that the robot provided value even for users who were not predisposed to feel good. Perceived social connection was predicted by negative affect, frequency of eating with others, and technology use during meals. Traits like extraversion and agreeableness were inversely related to connection—suggesting that artificial social agents may resonate most with emotionally sensitive users, rather than the most sociable ones. These findings suggest that commensal robotic companions may be particularly well-suited to users high in negative affect and openness, but also highlight the importance of adapting dialogue and behavioral strategies to users’ personality traits.

   4. Affective Communication via Haptic Technology: A Usability Study of a Huggable Device with Older Adults

      Eleuda Nunez, Zuzanna Radosz-Knawa, Anna Kołbasa, Paulina Zguda, Alicja Kamińska, Tymon Kukier, Masakazu Hirokawa, Kenji Suzuki, Bipin Indurkhya

      Abstract

      Loneliness among older adults is a growing concern with significant implications for mental and physical health. Although traditional communication technologies often lack the emotional richness of physical touch, recent advances in haptic interfaces offer new possibilities to improve remote social interactions. This study evaluates HugBits, a huggable communication device designed to connect users through shared hugging experiences, with a focus on its usability and emotional impact among older residents in a care facility. Using a mixed methods approach, we conducted a usability study with 16 participants (aged 65+), combining surveys, physiological measures, and qualitative interviews. The results show that while HugBits improved the experience of communication and was generally well received, it did not lead to significant reductions in loneliness or physiological stress markers. Participants valued the simplicity and emotional potential of the device, but highlighted the need for more immersive features, such as warmth or vibration. The findings underscore the promise of haptic communication for emotional support in older adults and underscore the importance of co-design in developing acceptable and meaningful interaction paradigms.

   5. Multimodal Framework for Adaptive HRI via Dynamic Engagement and Affective Feedback

      Gayathri Girijadevi Radhakrishnan, Olga Tveretina, Farshid Amirabdollahian, Diego Resende Faria

      Abstract

      In this work, we propose a multimodal affective communication framework to enhance human-robot interaction (HRI) by dynamically assessing user engagement through biophysiological responses. The system fuses facial expression analysis, speech emotion recognition, and text sentiment analysis to compute an engagement score, which informs adaptive robot behavior. Each modality is processed independently and contributes to a composite score categorized into low, medium, or high engagement levels. A key contribution of this study is a Bayesian-based engagement estimation mechanism, where the weight of each modality is dynamically adjusted based on its reliability, quantified via normalized inverse entropy. This probabilistic approach enables the robot to prioritize the most stable and informative signals, enhancing robustness in real-world environments. Additionally, the robot uses a Bayesian strategy to optimize predefined behavioral stimuli, learning over time which responses most effectively promote user engagement. The proposed framework supports personalized and context-aware real-time adaptation, with promising applications in assistive robotics, socially interactive systems, and long-term HRI.

   6. User Concerns Regarding Social Robots for Mood Regulation: A Case Study on the “Sunday Blues”

      Zhuochao Peng, Jiaxin Xu, Jun Hu, Haian Xue, Laurens A. G. Kolks, Pieter M. A. Desmet

      Abstract

      While recent research highlights the potential of social robots to support mood regulation, little is known about how prospective users view their integration into everyday life. To explore this, we conducted an exploratory case study that used a speculative robot concept—Mora—to provoke reflection and facilitate meaningful discussion about using social robots to manage subtle, day-to-day emotional experiences. We focused on the “Sunday Blues,” a common dip in mood that occurs at the end of the weekend, as a relatable context in which to explore individuals’ insights. Using a video prototype and a co-constructing stories method, we engaged 15 participants in imagining interactions with Mora and discussing their expectations, doubts, and concerns. The study surfaced a range of nuanced reflections around the attributes of social robots like empathy, intervention effectiveness, and ethical boundaries, which we translated into design considerations for future research and development in human-robot interaction.

3. Applications in Real-World Case Studies

   1. Frontmatter

   2. Social Robot Assistive Intervention for Science Students to Prevent Laboratory Accidents

      Mohammad Nehal Hasnine, Yuhuan Wang, Yuya Sato, Bipin Indurkhya, Mahmoud Mohamed Hussien Ahmed

      Abstract

      In science laboratories, accidents happen due to human errors, including carelessness and negligence of laboratory guidelines by the students, resulting in personal injuries and damaged assets. The lab supervisor (hereafter, the teacher) sets certain laboratory protocols for the students to follow to ensure the safety of the laboratory's experimental conditions and avoid potential accidents. One of the first rules students need to follow daily is to wear appropriate personal protective equipment, such as aprons, goggles, and gloves, before conducting an experiment. However, it is time-consuming for the teacher to check them daily. Therefore, this process needs to be automated. To address this research issue, we proposed the Laboratory Safety Assistant (LSA) framework, a robot-assisted learning framework to detect students’ behaviors that could lead to accidents and cause personal injuries during experiments. The LSA framework comprises Misty II Plus, a social robot as a social companion, a Behavior Detection and Analysis (BDA) server, and an intervention decision-making dashboard. We prepared a ‘Lab Objects’ dataset for this study containing 1288 images and 2295 objects grouped into 7 classes. We used this dataset to develop a new object detection model trained on a pre-trained YoloV8 model that could analyze image and video data captured by Misty II Plus. In addition, we developed an educational intervention for the teachers who lead science laboratories. Using this intervention, the teacher assesses students’ in-lab behaviors, identify their areas of support, and guide them in addressing the areas of need.

   3. Remote vs. Presence Laboratories in Human-Robot Interaction with Social Robots: A Study on Task Performance

      Christopher Peters, Kai M. Blum, Ibrahim Al Krad, Pablo Moraes, Ricardo Bedin Grando, Reinhard Gerndt, Tobias Doernbach

      Abstract

      A remote laboratory provides users with remote access to real physical experiments, enhancing accessibility and offering greater flexibility in time management. Yet, the application of remote labs in Human-Robot Interaction (HRI) research and teaching working with social robots introduces unique challenges. This study examines both perceived and objective task performance of a typical undergraduate HRI laboratory task, comparing remote and presence laboratory settings. We developed a remote lab setup that enables users to visually and verbally communicate with a humanoid robot. To investigate potential differences in task performance between these conditions, a user study was conducted at two universities, with 24 participants in total. Our study indicates that perceived task performance and usability are lower in remote laboratory settings compared to in-person setups. In addition, objective performance metrics—such as task duration and the number of questions asked—also favored the presence lab. However, there was no notable difference in grading outcomes. Overall, the observed differences are subtle and only partially statistically significant. We suggest further research into usability and training efforts in order to enable the potential of remote labs for enriching HRI research and teaching between physically distinct sites.

   4. Do Social Robots Motivate Students like Humans?

      Akshara Pande, Bethany Gosala, Manjari Gupta, Deepti Mishra

      Abstract

      The application of artificial intelligence tools provides the opportunity for improvements in several sectors, including education. A companion providing constant motivation can help students towards advanced engagement, learning and performance. In this direction, the present study explores the importance of feedback by considering many factors together, such as feedback provider, feedback type and feedback sequence on students’ visual attention and performance. An experiment was performed with two feedback providers, a social robot Pepper and a human companion; during the task, they provided three types of feedback: negative, neutral and positive. The task was to search for an object in the specified image within a certain time. Furthermore, four scenarios were formed to observe the impact of the feedback provider and the order of feedback provided. The participants were equipped with wearable eye trackers, Tobii Pro glasses 3, to collect their eye movement data. The aim of the paper is to evaluate the effectiveness of Pepper's feedback in comparison to human feedback in motivating users to enhance their visual attention and performance. The findings of the present study indicate the comparable influence of both robot and human feedback on participants’ visual attention and performance, possibly more so with feedback from Pepper. Thus, it suggests that social robots have the potential to inspire participants as human companions and can be utilized as teaching assistants. However, it is important to explore this further with a diverse and increased number of participants in real settings.

   5. More Than a Tool: A Multi Method Exploration of Contextual Social Robot Roles in German Secondary Schools

      Nadine Jansen, Ann-Kathrin Kubullek, Aysegül Dogangün

      Abstract

      Social robots, understood as autonomous, physically embodied systems designed to engage in social interaction with humans, are increasingly explored for use in educational settings, are being explored as support tools in education, yet their roles beyond instruction remain under-researched. This study investigates how students and professionals envision acceptable uses for robots in secondary schools. Using a multi-method approach—including a participatory workshop, student observations, and expert interviews—we identify key expectations and role preferences. Findings suggest that acceptance hinges on role clarity, contextual fit, and non-authoritative behavior. Social robots are most accepted when designed to assist rather than supervise. The study offers design implications for socially embedded robots and highlights the value of participatory development.

   6. Children’s Questions to Robots as an Educational Opportunity - Design Implications

      Alicja Wróbel, Paulina Zguda, Bipin Indurkhya

      Abstract

      We examine the role of spontaneous questions from children directed at or about social robots, specifically NAO and Misty, within educational settings. Focusing on children aged 3 to 6, the study compares different age groups to explore how their inquiries and behaviors reflect their understanding of and relationship with robots. The analysis combines quantitative data, based on the number of questions asked to robots and adults, with qualitative insights into the content of the questions and the broader behavioral trends across age groups. The results show that as children grow older, their interactions become more expressive, both verbally and physically. 6-year-olds, in particular, demonstrated increased physical engagement and more abstract or imaginative questioning. In contrast, younger children tended to initiate conversations grounded in familiar topics such as food or family. These developmental differences suggest a shift toward more complex and socially nuanced interactions with age. Based on these findings, the article proposes age-sensitive design guidelines to support interaction designers in creating more responsive and developmentally appropriate robotic systems for young users.

4. LLMs and Conversational/Verbal Interaction

   1. Frontmatter

   2. AwaR(e)obot: Towards Designing and Generating Context-Aware Companion Robot Behavior Using LLMs

      Eshtiak Ahmed, Juho Hamari, Oğuz ‘Oz’ Buruk

      Abstract

      As robots move beyond industrial and assistive roles into everyday human environments, the ability to communicate naturally and responsively becomes increasingly important. This paper presents a system that enables a quadruped robot, Boston Dynamics’ Spot, to respond to human voice inputs with expressive, dog-like physical behaviors. By integrating voice recognition, large language models (LLMs), and a structured response mapping framework, the robot interprets conversational inputs and generates sequences of behavior markers aligned with its physical capabilities. The system defines a robot persona and prompts the LLM with contextual constraints, including movement affordances and limitations, to ensure realistic and semantically appropriate responses. Our study highlights the potential of using LLMs not only for dialogue generation but also for embodied interaction design. While the limited expressivity and subtlety of robotic movement pose challenges, this work demonstrates a promising step toward more intuitive and engaging human-robot interaction. We discuss the implications for generalizing this approach to different robot morphologies and outline future directions for expanding behavioral nuance, emotional interpretation, and multimodal engagement.

   3. Message for You: Observing the Effects of a Social Robot’s Interruptions During an Office Task

      Elisabeth Ganal, Sophia C. Steinhaeusser, Florian Niebling, Birgit Lugrin

      Abstract

      Social robots and smart assistants are increasingly finding their way into our society. For social robots to be accepted in everyday life, it is essential that they adhere to social norms. The timing of message delivery is crucial, influenced by the message’s urgency and context, as well as the user’s current activity, to minimize distracting or disrupting the user. Vice versa, the timing can also affect how users perceive and evaluate a robotic assistant. In a user study, we investigated different timings of interruptions by a social robot during an office task, and how participants subsequently rate the robot’s social capabilities. For this, we asked participants about the main task’s workload, the stress level before and after the task, the social robot, and additional questions about the notifications. The subjective data is enhanced by objective measurements from log files and eye tracking recordings. The results show that the interruption timing can influence how participants perceive the notifications and the robot. This highlights the importance of context recognition and appropriate timing of interruptions in human-robot interaction.

   4. “Who Ignores You Matters” Asymmetrical Team Dynamics in Human-Robot Collaboration

      • Open Access

      Clarissa Sabrina Arlinghaus, Kennedy Mambilla, Günter W. Maier

      Abstract

      What happens when your robot teammate excludes you? As automation reshapes the workplace, understanding social dynamics in human–robot teams is more important than ever. Grounded in the Temporal Need-Threat Model, this preregistered study examines how coworker behavior (inclusion vs. ostracism-based exclusion) and agent type (human, humanoid robot, industrial robot) affect psychological needs, compensatory behavior, and social reasoning in a manufacturing context.

      Across 117 participants, ostracism significantly threatened core needs (belonging, self-esteem, and meaningful existence) and reduced motivation to engage in compensatory efforts (e.g., becoming more pleasant). These effects were strongest when exclusion came from a human coworker, and weakest when it came from an industrial robot. While compensatory efforts decreased under ostracism, their perceived effectiveness remained highest for human coworkers. Ostracizing humans were also penalized most in perception, receiving lower ratings on likability and intelligence than robot agents.

      Perceived anthropomorphism intensified both need satisfaction and need-threat, depending on context. Open-ended responses revealed distinct attribution patterns: human exclusion was interpreted as personal (e.g., dislike), whereas robot exclusion was interpreted as a mechanical or programming issue.

      These findings challenge the assumption that robots are treated like humans in social settings (CASA) and highlight the double-edged role of anthropomorphism. The study offers novel insights into how team dynamics shift in hybrid human–robot workplaces and underscores the need to manage expectations in collaborative automation design for safeguarding psychological needs in increasingly automated workplaces.

      PDF-Version jetzt herunterladen

   5. Exploring LLM-Generated Culture-Specific Affective Human-Robot Tactile Interaction

      Qiaoqiao Ren, Tony Belpaeme

      Abstract

      As large language models (LLMs) become increasingly integrated into robotic systems, their potential to generate socially and culturally appropriate affective touch remains largely unexplored. This study investigates whether LLMs—specifically GPT-3.5, GPT-4, and GPT-4o—can generate culturally adaptive tactile behaviours to convey emotions in human-robot interaction. We produced text-based touch descriptions for 12 distinct emotions across three cultural contexts (Chinese, Belgian, and unspecified), and examined their interpretability in both robot-to-human and human-to-robot scenarios. A total of 90 participants (36 Chinese, 36 Belgian, and 18 culturally unspecified) evaluated these LLM-generated tactile behaviours for emotional decoding and perceived appropriateness. Results reveal that: (1) under matched cultural conditions, participants successfully decoded six out of twelve emotions—mainly socially oriented emotions such as love and Ekman emotions such as anger, however, self-focused emotions like pride and embarrassment were more difficult to interpret; (2) tactile behaviours were perceived as more appropriate when directed from human to robot than from robot to human, revealing an asymmetry in social expectations based on interaction roles; (3) behaviours interpreted as aggressive (e.g., anger), overly intimate (e.g., love), or emotionally ambiguous (i.e., not clearly decodable) were significantly more likely to be rated as inappropriate; and (4) cultural mismatches reduced decoding accuracy and increased the likelihood of behaviours being judged as inappropriate.

   6. Knowledge-Based Design Requirements for Persuasive Generative Social Robots in Eldercare

      Stephan Vonschallen, Ennio Zumthor, Markus Simon, Theresa Schmiedel, Friederike Eyssel

      Abstract

      Social robots powered by generative AI such as Large Language Models open new possibilities for human-robot interaction by enabling natural, human-like conversations. Consequently, these generative social robots (GSRs) become more capable of influencing user attitudes and behavior through persuasion. In the present research, we conducted qualitative interviews with caregivers and therapists to identify knowledge-based design requirements for persuasive GSRs that promote physiotherapy attendance of residents in a Swiss eldercare facility. Our findings demonstrate that available information about the robot’s role as a caregiver assistant, as well as its assertive and polite personality would increase the robot’s ability to build trust and acceptance. Furthermore, having information on context factors like therapy benefits, facility routines, and news updates would allow the robot to flexibly and adequately adapt its persuasion attempts. Lastly, available information regarding user biography, emotions, and health status, enables the robot to engage in personalized persuasion. Taken together, the present findings highlight the importance of considering self-, context-, and user-related knowledge as key dimensions to integrate into GSRs. This could be realized through prompting techniques, fine-tuning, database access, and the robot’s real-time perception.