Meia Chita-Tegmark
ABSTRACT
Trust in human-robot interactions (HRI) is measured in two main ways: through subjective questionnaires and through behavioral tasks. To optimize measurements of trust through questionnaires, the field of HRI faces two challenges: the development of standardized measures that apply to a variety of robots with different capabilities, and the exploration of social and relational dimensions of trust in robots (e.g., benevolence). In this paper we look at how different trust questionnaires (Lyons & Guznov, 2019; Schaefer, 2016; Ullman & Malle, 2018) fare given these challenges that pull in different directions (being general vs. being exploratory) by studying whether people think the items in these questionnaires are applicable to different kinds of robots and interactions. In Study 1 we show that after being presented with a robot (non-humanoid) and an interaction scenario (fire evacuation), participants rated multiple questionnaire items such as "This robot is principled" as "Non-applicable to robots in general" or "Non-applicable to this robot." In Study 2 we show that the frequency of these ratings change (indeed, even for items rated as N/A to robots in general) when a new scenario is presented (game playing with a humanoid robot). Finally, while overall trust scores remained robust to N/A ratings, our results revealed potential fallacies in the way these scores are commonly interpreted. We conclude with recommendations for the development, use and results-reporting of trust questionnaires for future studies, as well as theoretical implications for the field of HRI.
Supplemental Material
Available for Download
- Kathleen Allen, Richard Bergin, and Kenneth Pickar. 2004. Exploring trust, group satisfaction, and performance in geographically dispersed and co-located university technology commercialization teams. In VentureWell. Proceedings of Open, the Annual Conference. National Collegiate Inventors & Innovators Alliance, 201.Google Scholar
- Kurt Byrne and César Marín. 2018. Human Trust in Robots When Performing a Service. In 2018 IEEE 27th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE). IEEE, 9--14.Google ScholarCross Ref
- George Charalambous, Sarah Fletcher, and Philip Webb. 2016. The development of a scale to evaluate trust in industrial human-robot collaboration. International Journal of Social Robotics, Vol. 8, 2 (2016), 193--209.Google ScholarCross Ref
- Filipa Correia, Carla Guerra, Samuel Mascarenhas, Francisco S Melo, and Ana Paiva. 2018a. Exploring the impact of fault justification in human-robot trust. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems. 507--513.Google ScholarDigital Library
- Filipa Correia, Samuel Mascarenhas, Rui Prada, Francisco S Melo, and Ana Paiva. 2018b. Group-based emotions in teams of humans and robots. In Proceedings of the 2018 ACM/IEEE international conference on human-robot interaction. 261--269.Google ScholarDigital Library
- Aimi S Ghazali, Jaap Ham, Emilia I Barakova, and Panos Markopoulos. 2018. Effects of robot facial characteristics and gender in persuasive human-robot interaction. Frontiers in Robotics and AI, Vol. 5 (2018), 73.Google ScholarCross Ref
- Matthew Gombolay, Xi Jessie Yang, Bradley Hayes, Nicole Seo, Zixi Liu, Samir Wadhwania, Tania Yu, Neel Shah, Toni Golen, and Julie Shah. 2018. Robotic assistance in the coordination of patient care. The International Journal of Robotics Research, Vol. 37, 10 (2018), 1300--1316.Google ScholarDigital Library
- PA Hancock, Theresa T Kessler, Alexandra D Kaplan, John C Brill, and James L Szalma. 2020. Evolving Trust in Robots: Specification Through Sequential and Comparative Meta-Analyses. Human Factors (2020), 0018720820922080.Google Scholar
- Peter A Hancock, Deborah R Billings, Kristin E Schaefer, Jessie YC Chen, Ewart J De Visser, and Raja Parasuraman. 2011. A meta-analysis of factors affecting trust in human-robot interaction. Human factors, Vol. 53, 5 (2011), 517--527.Google Scholar
- Marcel Heerink, Ben Krose, Vanessa Evers, and Bob Wielinga. 2009. Measuring acceptance of an assistive social robot: a suggested toolkit. In RO-MAN 2009-The 18th IEEE International Symposium on Robot and Human Interactive Communication. IEEE, 528--533.Google ScholarCross Ref
- Sarita Herse, Jonathan Vitale, Daniel Ebrahimian, Meg Tonkin, Suman Ojha, Sidra Sidra, Benjamin Johnston, Sophie Phillips, Siva Leela Krishna Chand Gudi, Jesse Clark, et almbox. 2018. Bon appetit! robot persuasion for food recommendation. In Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. 125--126.Google ScholarDigital Library
- Jiun-Yin Jian, Ann M Bisantz, and Colin G Drury. 2000. Foundations for an empirically determined scale of trust in automated systems. International journal of cognitive ergonomics, Vol. 4, 1 (2000), 53--71.Google ScholarCross Ref
- Devon Johnson and Kent Grayson. 2005. Cognitive and affective trust in service relationships. Journal of Business research, Vol. 58, 4 (2005), 500--507.Google ScholarCross Ref
- Theresa T Kessler, Cintya Larios, Tiffani Walker, Valarie Yerdon, and PA Hancock. 2017. A comparison of trust measures in human--robot interaction scenarios. In Advances in human factors in robots and unmanned systems. Springer, 353--364.Google Scholar
- Robert E Larzelere and Ted L Huston. 1980. The dyadic trust scale: Toward understanding interpersonal trust in close relationships. Journal of Marriage and the Family (1980), 595--604.Google Scholar
- Theresa Law, Meia Chita-Tegmark, and Matthias Scheutz. 2020. The Interplay Between Emotional Intelligence, Trust, and Gender in Human--Robot Interaction. International Journal of Social Robotics (2020), 1--13.Google ScholarCross Ref
- Theresa Law and Matthias Scheutz. [n.d.]. Trust: Recent concepts and evaluations in human-robot interaction. In Trust in Human-Robot Interaction. Elsevier, 27--57.Google Scholar
- Joseph B Lyons and Svyatoslav Y Guznov. 2019. Individual differences in human--machine trust: A multi-study look at the perfect automation schema. Theoretical Issues in Ergonomics Science, Vol. 20, 4 (2019), 440--458.Google ScholarCross Ref
- Bertram F Malle and Daniel Ullman. 2020. A Multi-Dimensional Conception and Measure of Human-Robot Trust.Google Scholar
- Nikolas Martelaro, Victoria C Nneji, Wendy Ju, and Pamela Hinds. 2016. Tell me more designing HRI to encourage more trust, disclosure, and companionship. In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 181--188.Google Scholar
- Roger C Mayer and James H Davis. 1999. The effect of the performance appraisal system on trust for management: A field quasi-experiment. Journal of applied psychology, Vol. 84, 1 (1999), 123.Google ScholarCross Ref
- Roger C Mayer, James H Davis, and F David Schoorman. 1995. An integrative model of organizational trust. Academy of management review, Vol. 20, 3 (1995), 709--734.Google ScholarCross Ref
- James C McCroskey and Jason J Teven. 1999. Goodwill: A reexamination of the construct and its measurement. Communications Monographs, Vol. 66, 1 (1999), 90--103.Google ScholarCross Ref
- Michael Novitzky, Paul Robinette, Michael R Benjamin, Danielle K Gleason, Caileigh Fitzgerald, and Henrik Schmidt. 2018. Preliminary interactions of human-robot trust, cognitive load, and robot intelligence levels in a competitive game. In Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. 203--204.Google ScholarDigital Library
- Paul Robinette, Wenchen Li, Robert Allen, Ayanna M Howard, and Alan R Wagner. 2016. Overtrust of robots in emergency evacuation scenarios. In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 101--108.Google ScholarCross Ref
- Alessandra Rossi, Kerstin Dautenhahn, Kheng Lee Koay, and Michael L Walters. 2018a. The impact of peoples' personal dispositions and personalities on their trust of robots in an emergency scenario. Paladyn, Journal of Behavioral Robotics, Vol. 9, 1 (2018), 137--154.Google ScholarCross Ref
- Alessandra Rossi, Patrick Holthaus, Kerstin Dautenhahn, Kheng Lee Koay, and Michael L Walters. 2018b. Getting to know Pepper: Effects of people's awareness of a robot's capabilities on their trust in the robot. In Proceedings of the 6th International Conference on Human-Agent Interaction. 246--252.Google ScholarDigital Library
- Tracy Sanders, Alexandra Kaplan, Ryan Koch, Michael Schwartz, and Peter A Hancock. 2019. The relationship between trust and use choice in human-robot interaction. Human factors, Vol. 61, 4 (2019), 614--626.Google Scholar
- Kristin Schaefer. 2013. The perception and measurement of human-robot trust. (2013).Google Scholar
- Kristin E Schaefer. 2016 Measuring trust in human robot interactions: Development of the ?trust perception scale-HRI". In Robust Intelligence and Trust in Autonomous Systems. Springer, 191--218.Google Scholar
- Sarah Strohkorb Sebo, Priyanka Krishnamurthi, and Brian Scassellati. 2019. 'I Don't Believe You": Investigating the Effects of Robot Trust Violation and Repair. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 57--65.Google ScholarCross Ref
- Pan Shu, Chen Min, Indu Bodala, Stefanos Nikolaidis, David Hsu, and Harold Soh. 2018. Human trust in robot capabilities across tasks. In Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. 241--242.Google ScholarDigital Library
- Ja-Young Sung, Lan Guo, Rebecca E Grinter, and Henrik I Christensen. 2007. 'My Roomba is Rambo": intimate home appliances. In International conference on ubiquitous computing. Springer, 145--162.Google ScholarCross Ref
- Benedict Tay, Younbo Jung, and Taezoon Park. 2014. When stereotypes meet robots: the double-edge sword of robot gender and personality in human--robot interaction. Computers in Human Behavior, Vol. 38 (2014), 75--84.Google ScholarDigital Library
- Daniel Ullman and Bertram F Malle. 2018. What does it mean to trust a robot? Steps toward a multidimensional measure of trust. In Companion of the 2018 acm/ieee international conference on human-robot interaction. 263--264.Google Scholar
- Daniel Ullman and Bertram F Malle. 2019. Measuring gains and losses in human-robot trust: evidence for differentiable components of trust. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 618--619.Google ScholarCross Ref
- William G Volante, Janine Sosna, Theresa Kessler, Tracy Sanders, and Peter A Hancock. 2019. Social conformity effects on trust in simulation-based human-robot interaction. Human factors, Vol. 61, 5 (2019), 805--815.Google Scholar
- Ning Wang, David V Pynadath, Ericka Rovira, Michael J Barnes, and Susan G Hill. 2018. Is it my looks? or something i said? the impact of explanations, embodiment, and expectations on trust and performance in human-robot teams. In International Conference on Persuasive Technology. Springer, 56--69.Google ScholarCross Ref
- Lawrence R Wheeless and Janis Grotz. 1977. The measurement of trust and its relationship to self-disclosure. Human Communication Research, Vol. 3, 3 (1977), 250--257.Google ScholarCross Ref
- Yaqi Xie, Indu P Bodala, Desmond C Ong, David Hsu, and Harold Soh. 2019. Robot capability and intention in trust-based decisions across tasks. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 39--47.Google ScholarCross Ref
- Rosemarie E Yagoda and Douglas J Gillan. 2012. You want me to trust a ROBOT? The development of a human--robot interaction trust scale. International Journal of Social Robotics, Vol. 4, 3 (2012), 235--248.Google ScholarCross Ref
- Sangseok You and Lionel P Robert Jr. 2018. Human-robot similarity and willingness to work with a robotic co-worker. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. 251--260.Google ScholarDigital Library
- Roxanne Zolin, Pamela J Hinds, Renate Fruchter, and Raymond E Levitt. 2004. Interpersonal trust in cross-functional, geographically distributed work: A longitudinal study. Information and organization, Vol. 14, 1 (2004), 1--26.Google Scholar
Index Terms
- Can You Trust Your Trust Measure?
Recommendations
Toward an Understanding of Trust Repair in Human-Robot Interaction: Current Research and Future Directions
Gone are the days of robots solely operating in isolation, without direct interaction with people. Rather, robots are increasingly being deployed in environments and roles that require complex social interaction with humans. The implementation of human-...
Do Humans Trust Robots that Violate moral trust?
The increasing use of robots in social applications requires further research on human-robot trust. The research on human-robot trust needs to go beyond the conventional definition that mainly focuses on how human-robot relations are influenced by robot ...
Human-Robot Trust: Just a Button Press Away
HRI '17: Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot InteractionMany of the benefits promised by human-robot interaction require successful continued interaction between a human and a robot; trust is a key component of such interaction. We investigate whether having a person "n the loop" with a robot---i.e., the ...
Comments