Designing Robot Embodiments for Social Interaction: Affordances Topple Realism and Aesthetics

To help ensure the quality of healthcare services in terms of ensuring human-like interaction, social robots should be able to express their own engagement and emotions, show empathy, and have the ability to interpret the human equivalents in the interaction partner [10, 34]. Likewise, realism is an important factor for eliciting social cues tied to these aspects [1‐3] when interacting with virtual agents and robots. Thus far however, media effects research has shown mixed results for the role of perceived realism, which is partly due to how it is defined [5, 46]. Realism manifests itself in form, behavior, and visual fidelity [2, 3, 5, 40, 46].

One of the differences between digital interface agents and electro-mechanical robots is that the latter are physically present in 3D space, whereas interface agents are present on a screen. Lee [23, p. 37] defines presence as “a psychological state in which virtual (para-authentic or artificial) objects are experienced as actual objects in either sensory or nonsensory ways”, which can occur at the physical level, the social level, and the personal (self) level. Interacting with a robot also appears to be more compelling and engaging compared to interacting with an avatar [19]. Several other studies [17, 24] indicated that robots with a physical embodiment have a positive effect on feeling the social presence of the robot, as compared to identical robots with a digital embodiment. Both studies concluded that having a physical embodiment contributes to experiencing social presence of robots. [17, 24]. However, if tactile interaction with the robot is limited, having a physical embodiment could have negative effects on experiencing social presence [17, 24]. Several pilot studies [42, 43] hinted at the positive contribution of physical embodiment to the perceived watchfulness, perceived helpfulness, and enjoyability of a social robot.

Kidd and Breazeal demonstrated that physical presence is not necessarily important for robot engagement, but rather the perception that something exists and is experienced as real [18]. In their study, users perceived physical robots (a physically present robot and a televised robot) as something that exists somewhere in the real world, but a digitally animated avatar as something fictional. If an embodiment is perceived as something that exists in the real world (even when televised), engagement, credibility, informativeness, intelligence, and enjoyment are all positively affected [4, 18]. Additionally, humans employ more social attention when they believe if a (robot) embodiment is controlled by a human compared to being controlled by a machine, regardless of representation [45]. However, for eliciting positive social interactions, having some form of representation is better than not having any representation [46].

The belief that something exists and resembles something ‘real’ is related to the design of the robot embodiment. What the robot represents may be of influence in whether users believe a robot is real, and therefore more engaging and likely to be used. By focusing on form realism (i.e. what the robot embodiment resembles), we can determine whether the embodiment of the robot (regardless of behavior or visual quality) affects the user’s emotional responses or their intention to interact with the robot. We propose the following hypothesis:

To investigate the effects of designed form realism of physical robot embodiments on the user, we applied the model of I-PEFiC, (see Fig.  1). I-PEFiC explains how users interact with fictional characters based on the user’s perception of ethics, affordances, aesthetics, and realism [38]. I-PEFiC divides the process of perceiving and experiencing interactive (fictional) characters into three separate phases: encode, compare, and respond (see Fig. 1).

When interacting with fictional characters (e.g., avatars), humans perceive features of these characters in terms of ethics (how good and evil is the character?), affordances (is the character aiding or obstructing the users goals?), aesthetics (how beautiful and ugly is the character?) and realism (how real and fake is the character?). After encoding, users compare these features to their own characteristics, goals, concerns, and respond accordingly [22, 38], resulting in a felt tendency to use the character or not (use intentions). This is accompanied by a level of engagement with the character. In I-PEFIC, engagement comprises the parallel experience of involvement and distance towards a fictional character [21, 22]. Involvement can be explained as the inclination to affectively approach the character someone is interacting with (e.g., feeling sympathy), and distance is the experienced tendency of wanting to avoid the character (e.g., being annoyed) [22]. Finally, the combination of use intentions together with involvement and distance explain the overall satisfaction with the character [38‐40].

We do realize that (product) affordances usually are defined as the potential action possibilities that a user perceives with a product, in relation to a certain context (cf. ecological affordances, [12]). However, this is difficult to operationalize in the form of a questionnaire and introduces additional test fatigue in the participants, due to the extensive set of factors in I-PEFiC. Therefore, similar to previous I-PEFiC studies [21, 22, 38‐40], we have aggregated affordances into an overall level of competence (e.g., ‘is the robot capable to successfully help you?’), rather than the actual amount of perceived action possibilities.

Previous studies empirically validated PEFiC (at the time without interactivity) in relation to media characters [21, 22] and I-PEFiC with interface agents [39, 40]. However, no studies were conducted with physically embodied robots. Using I-PEFiC as our theoretical framework, we designed an experiment to investigate the effect of designed form realism of social robots on the engagement and use intentions. We specifically focused on I-PEFiC’s tangible design factors: affordances, aesthetics, and realism, and their relation with the interaction and engagement processes. The factors ethics, relevance, and valence were left out because they are less related to (industrial) design. Furthermore, it is important to examine whether I-PEFiC can serve as an appropriate model for predicting engagement with and intentions to use physically embodied social robots. Based on prior studies, we hypothesized the same interrelationships between the factors in the I-PEFiC model as established in previous research (cf. Fig. 1). We propose the following hypotheses:

The study included 29 participants (13 female, 16 male, \(M_{age}\) = 28.8 years, age range 18–56 years) in a within-subjects design. Each participant interacted with three robots that varied in what they resembled (human \(\times \) creature \(\times \) machine). Twenty-seven adults were recruited from a group of university level adults (M.Sc. and Ph.D. students) and two additional participants were recruited from a filing department (university employees). The participants were recruited from the faculty of social sciences and none of them had prior experience with robots. Furthermore, the participants did not receive any compensation for taking part in the experiment.

Three robot designs that differed in their form realism (e.g., what they resembled) were created, resulting in a human-like robot ‘Rob’, a crocodile-like robot ‘Croq’, and a vehicle-like robot ‘Car’. We suspected that varying within a single embodiment design (e.g, three human-like robots) would lead to too subtle responses. Therefore, three different morphologies were chosen in an attempt to maximize the effects they might have on I-PEFiC factors. Rob, Croq, and Car were created from three LEGO Mindstorms NXT 2.0 sets (set #8527), which are toy sets intended for building small, programmable robots. The LEGO Mindstorms enabled us to use the same bricks such that the basic construction elements were similar across the three types. Only the exterior form (different representations) of the robots varied, whilst keeping visual fidelity and aesthetics consistent for each of the three robots. To maintain consistency in behavioral realism as well, all of the robots were static; none of them moved or was animated. The designs of Rob, Croq, and Car were based on the design templates included in the LEGO Mindstorms NXT 2.0 kit and roughly used the same number of LEGO bricks. The ‘Alpha Rex’ design served as a base for Rob, the ‘Robogator’ design for Croq, and the ‘Shooterbot’ design for Car. Each robot design was modified such that they included an ultrasonic sensor, a microphone, a color sensor, and a push button (see Fig. 2, left image).

Rob, Croq, and Car were controlled by identical software and were programmed in C#, using Microsoft Visual Studio 2010 to execute the program. The speech of the robots was synthesized using Loquendo Text To Speech (v7.20.2), with each robot having the same, neutral male voice. The language spoken by the robots was Dutch and was projected via speakers hidden directly behind each robot. The robot provided instructions for four different morning gymnastics exercise. Rob, Croq, and Car were designed as a physiotherapy robots because gymnastics exercises are a type of healthcare interventions that are easy to relate to for participants that are not patients or in direct need of care. The participant activated the exercise routines by presenting colored (red, green, blue, yellow) cardboard cards (85 mm \(\times \) 54 mm) to the color sensor of the robot (see Sect.  2.3).

There was a single session leader overseeing the experiment and was waiting in the hallway during the experiment. Rob, Croq, and Car were placed in similar, adjacent rooms connected to the hallway where the session leader was positioned. Each door was marked with a sign displaying the name of the robot that was present in that specific room. Because every participant had to interact with all three robots, each participant received an instruction envelope from the session leader before the experiment started. The envelop contained an instruction card that stated in which order they had to interact with Rob, Croq, and Car. The orders in which participants had to interact with the three robots were evenly and randomly distributed among the participants.

With this setup, three participants could do the experiment simultaneously with only one participant at the same time in a room interacting with a robot. Participants were given specific time slots to participate in the experiment; no participants were waiting at the hall for their turn while the experiment was in progress. Furthermore, the rooms were blinded so participants could not look inside the room from the hallway. Once the participants entered the appropriate room, the session leader informed them that they could start by pressing the button next to the robot (see Fig. 2, right image). The session leader left the room after giving the instructions, returning to the hallway.

The robot introduced itself as Rob / Croq / Car, the physiotherapy robot, asking the participant to present one of four colored cards to its sensors, each card corresponded with a different gymnastics exercise routine. To reduce potential test fatigue, the participants were given the freedom to determine the sequence in which they presented the cards to the robots. The red card started an exercise to clap as loudly as possible with the arms stretched out. Once completed, the robot asked the participant to clap again, but louder than the first time. The green card started a hand-eye coordination exercise, in which the robot asked the participant to close their left eye and press the button with their right index finger. When the task was completed, the robot asked the participant to press the button again, now with their right eye closed using their left index finger. The blue card started a stretching exercise in which the robot asked the participant to stretch their right arm towards the face of the robot, keeping it stretched. When the participant correctly stretched their right arm, the robot asked to do the same exercise again, now using the left arm. Finally, the yellow card started another stretching exercise in which the robot asked the participant to hold both fists at the chest and stretch them both together towards the face of the robot. The robot counted every stretch out loud and after stretching five times, the exercise was completed. Once all cards were presented and the exercises were completed successfully, the robot thanked the participant and requested to fill out the paper questionnaire (see Table 1 and Sect. 2.4 for an overview of the measures used and reliabilities).

The robots were entirely autonomous; they detected colored cards with a color sensor, correct movements with an ultrasonic sensor, sounds with a microphone, and pressure by means of a push button. Once the tasks and the questionnaire were completed, the participants were instructed to step outside of the room and wait until all of the participants within their group were finished and then to continue with the next robot as indicated on their instruction card. When the participants had interacted with all three robots and completed the questionnaires accordingly, they were debriefed by the session leader and thanked for their participation.

Each questionnaire was divided into five blocks of items, and the order in which the blocks were presented, as well as individual items within the blocks, were pseudo-randomized using an algorithm. No more than two items from the same variable were presented in a row (e.g., aesthetics item 1, aesthetics item 2, affordance item 1, etcetera). The questionnaire in the first condition of each participant included demographic questions. Items were taken from previous I-PEFiC studies [39, 40] and translated for the purpose of the present study. Each factor was measured with 4 to 10 Likert-type items, as described below. Each item was followed by 6-point rating scales, ranging from 0 = ‘totally disagree’ to 5 = ‘totally agree’. Items were positively and negatively formulated; the negative items were recoded after the experiment for composing the scales. To get an indication whether the items were coherent within their corresponding scales and distinguishable from the others, principle component analyses (PCAs) were done. Missing data was imputed using multiple imputation (random seed = 20070525). Because less than 10 % of the data was missing, 5 additional datasets were imputed (\(\gamma \,>\,\) 0.1, m = 5, see [13]). The pooled dataset was used if the analysis allowed it, otherwise the first imputed dataset was used. All resulting scales were reliable according to Cronbach’s \(\alpha \) (\(>\)0.72).

To check whether our manipulation of designed form realism was successful, we tested how participants perceived the realism designed in Rob, Croq, and Car in terms of Resemblance to their reference entities in the real world (cf. [2]). That is, we analyzed to what degree Rob resembled a human being, Croq a real crocodile, and Car a car. A one sample t-test with 0 as the test value revealed that the Resemblance rating of Rob was significantly higher than 0 on a 6-point rating scale (M = 1.28, SE = 1.98), t(28) = 6.45, \(p\,<\,\) 0.001. The Resemblance rating of Croq was also significantly higher than 0 (M = 2.00, SE = 0.258), t(28) = 7.76, \(p\,<\,\) 0.001. Finally, the Resemblance rating of Car was significantly higher than 0 (M = 3.10, SE = 0.269), t(28) = 11.54, \(p\,<\,\) 0.001. In other words, Car was perceived with the highest level of resemblance to the real-life object, followed by Croq and Rob respectively. Thus, our manipulation of designed realism can be considered successful in terms of sufficiently looking like the real-world entity.

Additionally, we looked whether the three representations differed among one another. We performed a one-way repeated measures MANOVA with Designed Form Realism and Resemblance. At the multivariate level, the results revealed a significant main effect of Designed Form Realism (Wilks’ \(\lambda \) = 0.40, F(4, 25) = 9.59, \(p\,<\,\) 0.001, \(\eta _{p}^{2}\) = 0.61). Univariate results confirmed that Designed Form Realism had a significant effect on the level of Resemblance (F(2, 56) = 23.36, \(p\,<\,\) 0.001, \(\eta _{p}^{2}\) = 0.46). Contrasts indicated that the difference between Rob and Car (F(1, 28) = 23.36, \(p <\) 0.001, \(\eta _{p}^{2}\) = 0.57) was the largest, followed by Croq and Car (F(1, 28) = 35.31, \(p\,<\,\) 0.01, \(\eta _{p}^{2}\) = 0.39), and finally Rob and Croq (F(1, 28) = 8.55, \(p\,<\,\) 0.01, \(\eta _{p}^{2}\) = 0.23). Thus, our manipulation of designed realism was successful in that all three representations differed from each other in their degree to which they resembled their corresponding real-world entity.

To test whether a social robot with a greater degree of Designed Form Realism (embodiment) would lead to a higher degree of Perceived Form Realism (H1), we conducted a one-way repeated measures MANOVA. Multivariate results revealed no main effect of Designed Form Realism on the levels of perceived realism across the three designs of Rob, Croq, and Car (Wilks’ \(\lambda \) = 0.94, F(6, 22) = 2.44, p = 0.96, \(\eta _{p}^{2}\) = 0.06). Therefore, H1 had to be rejected (see Table 2). Because no significant differences were found in the perceptions of realism as a function of designed form realism, we collapsed the scores of Rob, Croq, and Car for the analysis of H2 through H6 (see Table 2), thereby increasing the statistical power.

To test the assumed interaction of Perceived Aesthetics and Perceived Affordances on Use Intentions (H2 and H3), we used the SPSS macro PROCESS [14], which is a macro using a regression-based approach to analyze mediation and moderation models together with resampling techniques (bootstrapping). To test the interaction, 5000 bootstrap samples were generated and tested the significance of the interaction effect at 99 % bias-corrected bootstrap confidence intervals, using moderation model 1 [14, p. 218 and 442]. The analysis revealed that the model in its entirety explained 78 % of the variance in Use Intentions (F(3, 25) = 30.29, \(p\,<\,\) 0.001). When looking at individual factors, only the interaction between Perceived Affordances and Perceived Aesthetics on Use Intentions was significant (b(\(SE_b\)) = 0.31 (0.15), p = 0.04). Perceived Affordances (b(\(SE_b\)) = 0.29 (0.34), p = 0.40) and Perceived Aesthetics (b(\(SE_b\)) = \(-\)0.63 (0.42), p = 0.15) did not relate to Use Intentions when this interaction was considered. Therefore, we concluded that Perceived Aesthetics moderated the effect of Perceived Affordances on Use Intentions, which is in support of H3, but not H2 (see Table 2).

To test H4a, H5a, and H6a, we performed a linear regression analysis with Perceived Affordances, Perceived Aesthetics, and Perceived Realism as predictor variables and Involvement as the dependent variable. The analysis revealed that these three Encode factors explained 67 % of the variance in Involvement (F(1, 25) = 37.78, \(p\,<\,\) 0.001). When looking at individual factors, the analysis revealed a relation between Involvement and Perceived Affordances (b (\(SE_b\)) = 0.44 (0.11), \(\beta \) = 0.49, p = 0.001) as well as Perceived Realism (b(\(SE_b\)) = 0.38 (0.17), \(\beta \) = 0.34, p = 0.03), but not with Perceived Aesthetics (b(\(SE_b\)) = 0.20 (0.16), \(\beta \) = 0.20, p = 0.21). These findings support H4a, and H6a, but not H5a (see Table 2).

To test H4b, H5b, and H6b, we performed a linear regression analysis with Perceived Affordances, Perceived Aesthetics, and Perceived Realism as predictors, and Distance as the dependent variable. The analysis revealed that the model in its entirety explained 68 % of the variance in Distance (F(1, 25) = 57.28, \(p\,<\,\) 0.001). When looking at the individual factors, the analysis revealed a direct relationship between Distance and Perceived Affordances (b(\(SE_b\))  = \(-\)0.86 (0.13), \(\beta \) = \(-\)0.84, \(p\,<\,\) 0.001), but not with Perceived Aesthetics (b(\(SE_b\)) = 0.04 (0.18), \(\beta \) = 0.03, p = 0.84) or Perceived Realism (b(\(SE_b\)) = 0.01 (0.19), \(\beta \) = 0.003, p = 0.99). These findings support H4b, but not H5b nor H6b (see Table 2).

Finally, to explore whether age, gender, or prior robot experience affected the results of the main hypotheses, we performed a one-way repeated measures MANCOVA. Gender was treated as a between-subjects factor, and age, profession, and prior robot experience were included as covariates. No significant interaction effects were found between the manipulated Designed Form Realism and gender, age, profession, or prior robot experience (ps \(>\) 0.05).