A Taxonomy of Factors Influencing Perceived Safety in Human–Robot Interaction

There are two different trends to describe perceived safety. The first one is to articulate the existence of perceived safety with positive affective states such as comfortable, assured and stress-free. The second one is to define the term through the lack of safety perception. The pattern in the latter is phrasing perceived safety using negative affective states (emotional responses) such as discomfort, anger, nervous, worried, stressful, fear, and anxiety. The nomenclature for safety perception also varies in the HRI literature: psychological safety [48, 52], sense of safety and security , perceived safety [10], mental safety [61], and sense of security [68, 71].

Lasota et al. [52] described psychological safety as interactions that are stress-free and comfortable. The authors also expressed that to maintain psychological safety, the robot’s motion, appearance, embodiment, gaze, speech, posture, social conduct, or any other attribute should not result in any psychological discomfort or stress [52]. Bartneck et al. [10] proposed a questionnaire to measure perceived safety in HRI, and they defined perceived safety as “the user’s perception of the level of danger when interacting with a robot, and the user’s level of comfort during the interaction” (p. 76). Matsas et al. [61] defined mental safety in the context of human–robot collaboration in industrial settings as “the enhanced users’ vigilance and awareness of the robot motion, that will not cause any unpleasantness such as fear, shock or surprise.” (p. 140). Although Nanoka et al. [68] used both terms, namely mental safety and sense of security, they defined mental safety as: “the motions of the robot do not cause any unpleasantness like fear, shock, surprise to human.” (p. 2770). A review article on human–robot collaboration approaches [87] discussed the importance of mental safety, which described as  “mental stress and anxiety caused by close interaction with the robot” (p. 256).

Context plays a significant role in determining how people will react to a particular situation. Dey [27] provided a definition of context in computing as “any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves” (p. 5). ISO 9241-210:2019 (i.e. Ergonomics of human–system interaction - Part 210: Human-centred design for interactive systems) defines context of use as “combination of users, goals and tasks, resources, and environment” [83].

The context-awareness of a robot is one of the important design principles for increased physical safety in robot architectures [35]. The context of interaction plays a large role in establishing and maintaining perceived safety during HRI. Several studies explored the effect of context in HRI such as the influence of social context on the acceptance of social robots [90] and the influence of context in robot personality preferences [44]. Joosse et al. [44] reported that the users’ preference for a robot’s personality and behaviors depends on the context of the robot’s role and task. Similarly, the context in which the interaction takes place affects people’s perceived safety. As an example, a study that investigated the safety-related factors of pedestrians’ crossing the road process, categorized the influencing factors of perceived safety as demographic factors, contextual factors, and behavioral factors [95]. In HRI, such contextual information is not limited to environmental and social factors, but also includes factors related to the role of the robot in the interaction such as the type of robot’s task, the robot’s competency to accomplish the relevant task, and the robot’s performance. In different contexts, the level of safety perception for the same robot may be different. For example, in our video-based study [1], participants commented that they would feel safer when the robot’s task is finding an object at home than when the robot’s task is reminding about an important medication.

Therefore, how humans perceive the robots depends on the context of use. The aspects influencing context of use are robot’s role, robot’s task type and complexity, robot’s competency for the task, and robot’s task performance.

Perceived safety and human comfort are often mentioned together in HRI literature [10, 23, 52, 54], sometimes used even as synonyms. Therefore, comfort is a criterion that is closely related to perceived safety. Users may experience discomfort that refers to negative feelings due to the occurrence of any intended or unintended situation during the interaction with a robot. To improve the interactions with robots, a robot should avoid any behavior that makes the person uncomfortable regardless of objective safety. For example, an approaching robot can make people feel unsafe and cause discomfort even if the robot controller eliminates all risks of collision [51]. Lasota et al. [54] reported that participants felt safer and more comfortable working with a robot operated with human-aware motion planning compared to a standard robot. To avoid a robot causing discomfort, negative emotional responses, and decreased safety perception, the literature presents different strategies such as maintaining proper distance (proxemics), proper approaching strategy, control strategies to avoid being noisy, and planning to avoid interference [51]. Other aspects that can affect human comfort in HRI are the robot’s response speed, movement trajectory, proximity, object-manipulating fluency, and sociability [88]. Finally, individual characteristics including gender, age, race/nationality, socioeconomic status, and personality traits can affect how people perceive robots [62].

Therefore, the aspects which influence human comfort in HRI are individual characteristics, approaching direction of the robot, proxemics, unintented noise from the robot, robot response speed, robot movement trajectory, motion fluency, and the robot sociability.

Humans seek familiar and known things rather than unfamiliar and unknown things to feel safe [60]. Familiarity refers to “knowledge of a thing or person through long or close association or frequent perception by any of the senses; everyday acquaintance, habituation; an instance of this” [75]. Arai et al. [7] considered familiarity as one of the important factors for mental safety, and familiarity was explained as general preferences for motions and designs of robots. Haring et al. [38] reported that an android robot was perceived significantly less safe in comparison to a humanoid and non-biomimetic robot (i.e., Keepon robot). This could be explained by the uncanny valley. Mori [66] proposed an hypothesis about a person’s reactions to robots that look and act almost human-like. The familiarity with a robot increases with human resemblance until an uncanny valley which is a sudden shift from empathy to revulsion due to a highly realistic appearance.

Interaction experience with robots is another factor that has an impact on perceived safety. Experience is defined as “knowledge resulting from actual observation or from what one has undergone” [74]. Previous experience with a specific technology has a positive effect on different aspects of user experience and acceptance of that technology. People’s exposure to robots will increase their familiarity with robots. The level of familiarity with robots and robot-related experience can be gained by the exposure of robots. A person who has experience and familiarity with robots would have positive perceptions towards acceptance of robots [25], have a positive attitude and less anxiety towards robots [11], and trust in robots [22]. On the other hand, lack of familiarity might lead to negative feelings toward technology and robots [25]. Takayama et al. [85] reported that people who had prior experience and familiarity with robots maintained a closer distance to robots. Similarly, human’s prior experience with robots can influence perceived safety [52].

Therefore, the aspects which influence experience and familiarity are interaction duration, interaction frequency with robots, the robots’ physical appearance and motions.

There is a lack of coherence for the concept of interpretable behaviors for robots/artificial agents. Many different terms, such as explainable, legible, predictable, and transparent have been used to convey intentions that would be ascribed to an agent’s behavior by a human observer [19]. Although predictability and transparency are used interchangeably, we refer to predictability as interpretable future actions and transparency as interpretable present actions. In other words, predictability is understanding a robot’s upcoming actions, and transparency is understanding what the robot is doing and why it is taking the current action. Transparency is sometimes defined as a combination of interpretable present and future actions. As an example, Alonso et al. [5] defined the term as “the observability and predictability of the system behavior, the understanding of what the system is doing, why, and what it will do next”(p. 1). While people feel safe in the predictable cases [76], situations that are unpredictable and uncertain are perceived as unsafe [45] even if there is no threat [16]. Therefore, regardless of the term used, predictable/transparent/legible robot behaviors are important for promoting safety perception in HRI [58]. For example, Koert et al. [49] reported that in a pick and place task, unpredictable robot actions, or robot responses that participants did not understand the reason they were occurring, were found to be uncomfortable and annoying.

Conveying information about the robot’s mode and intention can lead to greater perceived safety. The use of interfaces and natural language explanations are ways of achieving transparent behaviors [5]. For example, equipping AVs with external human–machine interfaces resulted in greater perceived safety, and had a positive effect on acceptance of pedestrians compared to an AV without an interface [36]. Social robots can also facilitate transparent behaviors. As an example, Suvei et al. [84] showed that social gaze cues increased the feeling of safety in a human-sized mobile robot approaching scenario. The robot’s communication that conveys its intent can improve many aspects of HRI, such as communication, reliability, predictability, transparency, and situational awareness [18]. Moreover, transparency of robot behaviors through explanations of its actions was considered to be more trustworthy [50]. However, it is important to use a clear communication to avoid potential ambiguities and thus the robot’s behaviors are easily understandable by its human partner. Therefore, it is important not only what to communicate, but also how to communicate [39].

Therefore, the aspects which influence predictability and transparency are visual interfaces, social gaze, predictable/transparent motions, communication style and intention communication of the robot.

The sense of control or sense of agency refers to the subjective feeling of control over a person’s actions and control through these actions over the external events [6]. Therefore, the sense of control is the degree to which individuals perceive that the consequences of circumstances are under their control. Humans desire explainability and predictability for feeling in control thereby feeling safe [76]. It has also been expressed in gerontology studies that having control over the events leads to safety feelings in older adults [73]. In the context of HRI, we use the term sense of control as the user feels that the user is in charge of the robotic system, and the user has the control over the system, not the vice versa. The user could experience decreased perceived safety when the interaction is demanding, such as the user cannot cope with or control the robot’s behaviors and consequences of those behaviors. The ability to control implies perceived safety [63], on the contrary, lack of sense of control may induce frustration. In the context of HRI, a user’s sense of control is affected not only when the user cannot control a robot’s actions, but also by the social context, as an example being excluded by robots [30]. In a social interaction with a robot, how close the robot is to the person and the approach direction of the robot also affect the sense of control and perceived safety. Young et al. [93] presented a dog-leash interface which was used by participants to lead a robot simply by holding the leash, with three variations: the robot in front of the person, the robot following directly behind the participant, and the robot following the participant behind with 45° angle. Participants felt less in control and less safe when the robot was following behind, and behind at an angle. The level of autonomy of the robot is another aspect that influences sense of control. Chanseau et al. [20] found the participant’s sense of control during the interaction with a robot to be linearly related to the expected level of autonomy.

Therefore, the aspects which influence sense of control are proxemics, the robot’s approach direction, the robot’s follow direction, the level of autonomy and social properties of the robot.

Kok and Soh [50] defined trust in HRI as a “multidimensional latent variable that mediates the relationship between events in the past and the former agent’s subsequent choice of relying on the latter in an uncertain environment” (p. 300). Trust is a cornerstone for having sustainable human social relationships. The use, misuse, disuse, and acceptance of automation depend heavily on trust, especially for complex systems [55]. Trust in autonomous systems and robots is crucial as they begin to appear on the streets, in factories, and in private homes. Therefore, a significant number of studies have been focused on trust in HRI and human–robot collaboration [37, 50, 86]. Hancock et al. [37] reviewed factors affecting trust in HRI. They reported that a robot’s performance-based and attribute-based factors affect trust. Performance-based factors include behavior, dependability, reliability of robot, predictability, level of automation, failure rates, false alarms, and transparency. Attribute-based factors include proximity/ co-location, robot personality, adaptability, robot type and anthropomorphism. Other factors influencing human trust in robots are timing and frequency of faulty robot behaviors [26], type of task [79], and anthropomorphism [67]. Moreover, the ability of a robotic assistant to carry out a prescribed task, occurrence of errors, failures including social norm violation and technical failure, communication style (speech and facial expressions), and behavior transparency have been considered in human–robot trust studies [32]. Robot failures have the greatest impact on loss of trust. These failures include design failures, system failures (hardware, software), and expectation failures (the system acts as it is supposed to, but does not satisfy the user’s expectation) [86]. Perceived safety of robots is essential for building the trust of their human counterparts in HRI and human–robot collaboration [13]. If a robot is perceived as unsafe, people would not trust that robot. Perceived safety plays an important role in building trust in the human interaction partner [70].

Screen-based studies allow researchers to use standardized methodologies (same system behavior in each trial) and reach a large number of participants. However, they lack real-life experience. In screen-based studies, the common approach is to test perceived safety of participants after watching a video or interacting with a simulator. Lichtenthäler et al. [57] used short movie sequences recorded in a virtual environment using a simulator. The scenario was a human crossing the robot’s path in an office environment in which the participants rated the robot’s navigation behavior. The authors reported that human-aware navigation received higher perceived safety ratings. Similarly, another study by the authors [58], used the same human crossing the robot’s path scenario with recorded videos. The authors reported that legible robot behaviors such that the human can predict the next actions of the robot increased perceived safety.

VR has gained popularity in recent years especially for autonomous systems such as flying robots (drones) and AVs. VR environments have the benefit of high experimental control and maintaining real-world realism. One example can be seen in Widdowson et al. [89], where a flying robotic system was explored in a VR test environment for different experimental paradigms. The authors [89] investigated various flight behaviors including different speeds, acceleration, angles of approach, and distances. The aim was to improve the design and control of flying robots for enhanced comfort and perceived safety. The study considered human arousal as an indicator of safety perception. Therefore, several physiological signals (electrodermal activity (EDA), photoplethysmography (PPG), and head motion) were collected and used for predicting human arousal. It is common to use VR to test safety perception during the interactions with AVs [24, 40]. Perceived safety of robots could benefit from the findings of AV studies. As an example, Hollander et al. tested the impact of malfunctioning external car displays during pedestrian crossing on perceived safety and trust [40]. The authors reported that wrong information resulting from actual malfunction decreased perceived safety, trust, and confidence. However, the recovery of participants’ trust was quick.

The WOZ design is an experimental paradigm in which a system is fully or partially operated behind the scenes by a human and participants believe that the system is autonomous. Habibovic et al.  [36] used the WOZ setup in automated driving experiments, where pedestrians felt significantly safer when they encountered an AV with an interface compared to an AV without an interface. This shows that the transparency of the machines can help to improve perceived safety. Chadalavada et al. [18] introduced a bi-directional intention communication system for autonomous mobile robots using spatial augmented reality and eye-tracking glasses. An autonomous forklift projected various patterns onto the common floor area to communicate the robot’s motion intentions. The authors reported that intent communication projection improved participants’ comfort and perceived safety.

To summarize, perceived safety of human–machine systems is still in its infancy; thus, the reviewed studies so far focus on understanding human safety perception under different conditions during HRI. It is very rare utilizing perceived safety for guiding the robot’s behaviors for better interactions. However, maintaining perceived safety throughout the interaction is crucial for long-term interactions and robot acceptance.

The most common evaluation methods in HRI studies are self-assessment subjective evaluations, behavioural measurements, psycho-physiological measures and task performance metrics [81]. The evaluation of perceived safety in HRI is done by monitoring psychological, behavioral, and physiological responses, follow-up questionnaires [52], or handheld devices [24]. Psychological responses are considered as responses that are linked to mental activities. Physiological responses are non-voluntary responses that are related to living organisms’ bodily reactions. Behavioral responses are manners which differ from physiological responses in a way that they can be controlled voluntarily [4]. Facial expressions, eye gaze, and blink variations could be considered as behavioral responses [4]. Monitoring and interpreting humans’ behavioral and physiological responses can provide important information about lower levels of perceived safety.

Psychological responses can be observed through self-assessment subjective evaluations (e.g. questionnaires) or interviews. The former is one of the most widely used ways in the HRI literature, examples are [10, 48, 68]. Bartneck et al. [10] presented a semantic differential questionnaire as an instrument for measuring perceived safety of robots. Kamide et al. [48] presented a questionnaire for measuring the psychological safety of humanoids. In [68], participants rated their emotions using a questionnaire including the items surprise, fear, disgust, and unpleasantness on a scale ranging from 1 (never) to 6 (very much). In our previous work, the participants rated their safety perception using a semantic differential questionnaire.

Physiological responses can provide information regarding the intensity of an individual’s internal state. There are several studies that collected physiological signals data to evaluate perceived safety [3, 8, 47, 68, 89]. As an example, Nonaka et al. [68] also measured the heart rate of participants in addition to questionnaires. However, they did not find a determinate conclusion about the relationship between heart rate and the human sense of security. Arai et al. [8] considered skin potential response to be an indicator of safety perception. Another example of physiological measures can be found in [47], where Kamide et al. collected saliva amylase in their user studies to check the physiological reactions and safety perception.

HRI could benefit from perceived safety measurement methods in automation. Handheld devices have been used in AVs studies to measure perceived safety. In [24], participants continuously reported their safety feelings during their interaction with an AV by holding a handheld button pressed for as long as they felt safe in a pedestrian crossing. The authors reported that the temporal window of feeling safe was wider when the AV was equipped with external human–machine interfaces. Similarly, Rossner et al. [77] used a handset control for the online measurement of perceived safety while driving an AV in a simulator. In Bazilinskyy et al. [12], participants reported their perceived safety while watching AVs equipped with varying parameters of external human–machine interfaces. The participants kept a keyboard key pressed as long as they felt safe to cross the road.

To summarize, there is no standardized way of experimental paradigm or measures for perceived safety in HRI and autonomous systems. The evaluation of perceived safety in HRI could benefit from the combination of several methods. As an example, questionnaires may not always be suitable for detecting the subtle indicators of perceived safety. Therefore, an additional information from different modalities as well as contextual information including the task, robot and environmental related factors could help evaluating perceived safety. The diverse set of complimentary measures can capture different aspects of interactions. It is worth to note that one unique aspect about safety and thereby perceived safety is that we can measure the lack of them, not the existence of them [41]. Thus, lower degrees of perceived safety could be observed through emotional, cognitive, behavioral, and physiological responses.