The effects of explainability and causability on perception, trust, and acceptance: Implications for explainable AI

Highlights

• This study examines the effect of explainability in AI on user trust and attitudes toward AI.

• Conceptualizes causability as an antecedent of explainability and as a key cue of an algorithm and examines them in relation to trust.

• The dual roles of causability and explainability in terms of its underlying links to trust.

• Causability lends the justification for what and how should be explained.

• Causable explainable AI will help people understand the decision-making process of AI algorithms.

Abstract

Artificial intelligence and algorithmic decision-making processes are increasingly criticized for their black-box nature. Explainable AI approaches to trace human-interpretable decision processes from algorithms have been explored. Yet, little is known about algorithmic explainability from a human factors’ perspective. From the perspective of user interpretability and understandability, this study examines the effect of explainability in AI on user trust and attitudes toward AI. It conceptualizes causability as an antecedent of explainability and as a key cue of an algorithm and examines them in relation to trust by testing how they affect user perceived performance of AI-driven services. The results show the dual roles of causability and explainability in terms of its underlying links to trust and subsequent user behaviors. Explanations of why certain news articles are recommended generate users trust whereas causability of to what extent they can understand the explanations affords users emotional confidence. Causability lends the justification for what and how should be explained as it determines the relative importance of the properties of explainability. The results have implications for the inclusion of causability and explanatory cues in AI systems, which help to increase trust and help users to assess the quality of explanations. Causable explainable AI will help people understand the decision-making process of AI algorithms by bringing transparency and accountability into AI systems.

Introduction

The role of algorithms in our lives is growing rapidly, from simply recommending online content or online search results, to more critical usages, like diagnosing human cancer risk in medical fields (Chazette and Schneider, 2020). Algorithms are widely used for collection, computing, data processing, and automated decision-making. By widely mediating and assisting in human decision-making, algorithms are becoming a ubiquitous part of human lives (Rai, 2020). While algorithms can offer highly personalized and relevant services and content, the effectiveness of artificial intelligence (AI) systems is limited by the algorithm's current inability to explain their decisions and operations to users. Complicated matters, such as fairness, accountability, transparency, and explainability (FATE) are inextricably linked to algorithmic phenomena (Ferrario, Loi, and Viganò, 2020; Shin, Zhong, and Biocca, 2020). Questions regarding how to safeguard the goals, services, and underlying processes of AI, who should be held liable for the consequences of AI, and whether AI is doing things that humans believe are ethical remain unclear and controversial (Dörr and Hollnbuchner, 2017). Thesesubjects, including FATE and ethical concerns regarding how we address and govern issues will be critical to AI development and innovation (Crain, 2018).

The black-box nature of algorithm processes had led to calls for research on explainability in AI (Castelvecchi, 2016; Holzinger, 2016), for example, explore the effects of explainability and transparency in the adoption of personalized news recommendations. Shin (2020) proposes an idea of algorithmic trust in terms of transparency in the content recommendation context. How users interpret algorithmic features and how users understand algorithm-based systems will be important questions to address as AI becomes more widespread (Shin, 2020). Particularly, this topic will be even more critical in news recommendation systems, where fairness, accountability, and credibility are inherent journalistic values (Dörr and Hollnbuchner, 2017). There has been increasing pressure to give the right explanation on how and why a result was provided (Hoeve et al., 2017). Despite their importance, few studies have examined the roles of explainability and interpretability in AI. Recent research on algorithm acceptance (Shin et al., 2020) suggests the heuristic role of explainability in the acceptance of algorithm/AI services. When users interact with an algorithm, they inevitably encounter issues of algorithm functions, which are essentially subjective insofar as they are dependent upon human judgment and context (Shin and Park, 2019). Thus, along with explainability, it is important to examine how users interpret such explanations, how they reason causality and causal inference (Arrieta, 2020), and the process through which people work to understand the issues in algorithms that are ambiguous and uncertain (Vallverdú, 2020). Against the increasing concerns about the opacity of black-box AI, this study operationalizes trust in algorithms by clarifying the role of explainability in reference to causability. It examines FATE in the context of algorithm processing and to clarify its roles and influence in the user interaction with AI. The following research questions (RQ) are formulated based on the research gaps:

• RQ1: How does explainability play out in user heuristics and systematic evaluations for personalized and customized AI news?

• RQ2:How do users perceive/evaluate the given explanations and how do we measure the quality of explanations?

• RQ3:How do explainability combined with causability affect trust and the user experience with a personalized recommender system?

Findings reveal a user dual-process that users go through: a heuristic process by causability and a systematic process through explainability evaluating algorithm features and deciding how and whether to continue to use AI services during their evaluations. Whenever people encounter algorithms, they must make decisions as to whether, how, and to what extent to trustalgorithm-based services (Wölker and Powell, 2020). Heuristically, users evaluateexplanations based on their existing knowledge and beliefs, and partly based on their understanding of the algorithms. Users evaluate the quality of explanation based on their own level of interpretability and understandability (Samek, Binder, Montavon, Lapuschkin, and Muller, 2017). Systematically, users explore AI product information when evaluating algorithmic functionality. In the process, issues of FAT play roles as heuristic cues, triggering user trust. Levels and kinds of FAT are perceived as a function of user appraisal of explainability (Moller, Trilling, Helberger, and van Es, 2018). When such explanations are reasonable and understandable, users begin to accept FAT and trust the AI system.

The causal implications of trust and algorithmic explainability provide important directions for academia and practice. Theoretically, clarifying the role of explainability in AI would make meaningful contributions to the ongoing discussion of human-AI interaction (HAII; Sundar, 2020). Particularly, the human-interpretable heuristic processes of explainable AI (XAI) from a human factor's perspective is useful because they provide new ways of designing and developing causable XAI (Combs, Fendley, and Bihl, 2020). The findings contribute to theformalisation of the field of explainability and causability in HAII by showing how the concepts are conceptualized, by illustrating how they can be implemented in user interfaces, and by examining how the effect and the quality of explainability is measured (Samek et al., 2017). From a practical standpoint, the heuristic role of causability and the systematic dimension of explainability in algorithms lends strategic direction on how to design and develop XAI and user-centered algorithms in order to facilitate algorithm adoption in mainstream services. As the current AI models are increasingly criticized for their black-box nature, the roles of explainability and related causability will surely give insights into user confidence in algorithms (Shin, 2020).