6. Conceptualizing Overtrust in Robots: Why Do People Trust a Robot That Previously Failed?

In this chapter, we present work that suggests people tend to be overly trusting and overly forgiving of robots in certain situations. In keeping with the theme of this book where intelligent systems help humans recover from errors, our work so far has focused on robots as guides in emergency situations. Our experiments show that, at best, human participants in our simulated emergencies focus on guidance provided by robots, regardless of a robot’s prior performance or other guidance information, and, at worst, believe that the robot is more capable than other sources of information. Even when the robots do break trust, a properly timed statement can convince a participant to follow it. Based on this evidence, we have conceptualized overtrust of robots using our previous framework of situational trust. We define two mechanisms in which people can overtrust robots: misjudging the abilities or intentions of the robot and misjudging the risk in the scenario. We discuss our prior work in light of this new reconceptualization to attempt to explain our previous results and encourage future work.