Understanding the Design Elements Affecting User Acceptance of Intelligent Agents: Past, Present and Future

To identify relevant literature as the basis for the state-of-the-art analysis, we conducted a systematic literature review (SLR) following Webster and Watson (2002) and vom Brocke et al. (2015). The overall scope of the conducted SLR can be defined along the dimensions of process, source, coverage, and techniques of the SLR (vom Brocke et al., 2015). Based on a sequential search process, we searched relevant journals and conference proceedings from the field of IS and HCI literature as a source. Thereby, our literature search intends to reach a representative coverage of the design elements reported in the literature. Thus, to establish the basis for our analysis, we used a comprehensive set of techniques (i.e., keyword search, backward and forward search). To reach a high level of reproducibility and transparency of our research, we will describe the three single methodical steps that we undertook.

In the first step, we selected the search strings. Since we aimed to identify a wide range of literature on IAs, the search string was chosen to be rather broad. Based on recent publications, we identified different keywords that researchers used to describe IAs. This resulted in the following search string:

“conversational agent” OR “intelligent agent” OR “chat bot” OR “chatbot” OR “dialogue system” OR “smart personal assistant” OR “smart assistant” OR “intelligent assistant”.

In the SLR, we used all variations of the keywords – singular, plural, hyphenated, or not hyphenated. In the second step, we selected the outlets. As our goal was to identify representative literature samples of different empirical research perspectives on user interaction with IAs, our search covered multiple journals and conference proceedings. We chose this approach because journal acceptance processes take substantially longer than conference proceedings to be processed, which would have led to neglecting some of the most relevant literature since IA research represents a young and nascent topic. For the selection of outlets, we identified two broad areas for deriving design elements of IAs – IS and HCI – as they cover a substantial share of literature on IAs.

Suitable journals and conference proceedings at the intersection of HCI and IS that provided an overview of high-quality and relevant research in the respective research fields were selected using both the AIS Senior Scholars’ Basket, and relevant IS journals and conferences based on the recommendations of the special interest group on human–computer interaction. Moreover, to safeguard the relevance of our results, we discussed our selection of journals and conference proceedings with two senior researchers from the field of interest who were not involved in the writing process of the paper. Based on these inputs and their feedback, we selected 20 journals and proceedings for our keyword search, as seen in Table 2. Finally, in the third step, we selected the papers. We searched in the title, abstract, and keywords of the papers as we assumed that papers that deal with design element of IAs as a focal unit of analysis would exhibit the search strings defined above there. The outlet-based search revealed 383 hits. This number contained literature not relevant to this paper. In an initial screening process, the identified papers were analyzed based on their abstracts. We only included papers that referred to any type of IAs and that provided empirical insights on user interaction with IAs. Papers dealing with this topic trivially or marginally, such as those generally dealing with technology acceptance of IAs, were removed from the sample. This resulted in a selection of 76 publications. Finally, we also performed a forward and backward search to capture papers not covered through the database search. Through screening the references and applying forward searches using Google Scholar, 31 articles were added to the list, resulting in the final number of 107 relevant papers.

We aim to use a theoretically sound frame for guiding the analysis of the retrieved papers. The types and effects of factors affecting user acceptance of IAs differ depending on the theoretical lenses used by the researchers. Hence, we briefly explain our reasoning for selecting our frame of analysis.

First, we present the scaffolding model developed by Wirtz et al. (2018) for the purpose of grounding the analysis of dependent variables on a solid theoretical basis. Then, we elaborate our procedure for the independent variables, for which we seize a taxonomic classification of the social cues of CAs introduced by Feine et al. (2019). This shall allow establishing exhaustive and exclusive paper analysis results benefitting the clarity and structure of the field.

In the vein of dependent variables, most studies share similar theoretical foundations, which are primarily based on the Technology Acceptance Model (TAM) (Davis, 1989) and its subsequent modifications, such as the TAM2 (Venkatesh & Davis, 2000), TAM3 (Venkatesh & Bala, 2008), Unified Theory of Technology Use and Acceptance (UTAUT) (Venkatesh et al., 2003), and UTAUT2 (Venkatesh et al., 2012).

However, although literature often refers to such renowned theories due to their adaptability and accessibility, their utility is largely context-specific (Lowe et al., 2019), and they may not be comprehensive enough to demonstrate the introduction of emerging technologies such as IAs (McLean & Osei-Frimpong, 2019). As a result, introducing them in a new context necessitates a close examination of its fundamental elements as well as empirical confirmation of key relationships. Furthermore, IAs are more personalized, linked, and open than previous technologies (Gummerus et al., 2019). Thereby, user acceptance of IAs shall be determined not only by their practical efficiency but also by their capacity and skills to meet social-emotional and relational needs (Lee et al., 2020; van Doorn, et al., 2017; Wirtz et al., 2018). Wirtz et al. (2018) developed the Service Robot Acceptance Model (sRAM), which expands on the original TAM by including social-emotional and relational variables. The model depicted in Figs. 1 and 2 views service robots as “[…] system-based autonomous and adaptable interfaces that interact, communicate and deliver service to an organization’s customers” (Wirtz et al., 2018, p. 909). This notion can be transferred and adapted to the context of IA acceptance as IAs are described as entities that perceive and respond in a timely manner, are capable of interacting with other agents (i.e., humans), and react to their environment (Rudowsky, 2004).

Per the underlying assumption of sRAM, user acceptance of IAs is determined by how well IAs meet functional, socio-emotional, and relational requirements (Davis, 1989; Fiske et al., 2007; Solomon et al., 1985).

The sub-dimensions of social elements included in the sRAM model are perceived humanness, perceived social presence, and social interactivity. In this regard, social presence can be defined as “the extent to which other beings in the world appear to exist and react to the user” (Heeter, 1992). Whilst the perceived humanness refers to the distinguishability of an IAs from a human (Wuenderlich & Paluch, 2017). Here it must be noted that we did not include the sub-dimension social interactivity within our analysis.

Aside from social aspects, two important relational elements have been identified, i.e., trust and rapport. Trust is usually defined as an expectation that another entity “will perform, a particular action important to the trustor [i.e., user], irrespective of the ability to monitor or control that other party [i.e., IA]” (Mayer et al., 1995, p. 712). The sub-dimension rapport can be defined as the user’s perception of a pleasant encounter with an IA (i.e., IA being friendly, the IA’s ability to stimulate interest, and meeting the user’s needs for fulfillment of a task) as well as a personal relationship between the user and the IA. In general, it can also be described as the personal interplay between two parties (Gremler & Gwinner, 2000).

The functional elements included in sRAM are the original dimensions of the TAM model (Davis, 1989): subjective norms, perceived ease of use, and perceived usefulness. A user’s views about what other (important) users think they should do (or not do) in a particular scenario are referred to as subjective norms (Fishbein & Ajzen, 1977; Venkatesh & Davis, 2000). These social conventions may have a positive relationship with the acceptance of new technologies because people are more likely to respond in a certain way if they assume it is acceptable by society. In previous research, the subjective norm played an inconclusive role and led to divergent views (Schepers & Wetzels, 2007). In addition, IAs are a rather new technology, therefore, we do not know how users’ might adopt it to enhance their social status quo (i.e., McLean & Osei-Frimpong, 2019). Within the sub-dimension, perceived usefulness, we refer to the extent to which a user believes that using a specific system will improve its performance of fulfilling a task (Davis, 1989). Whereas the perceived ease of use refers to “the degree to which a person believes that using a particular system would be free of effort” (Davis, 1989).

As a result, we use this slightly modified version of the sRAM model as a guideline in our coding and analysis of IA design dependent variables.

To facilitate the discussion of the high quantity of independent variables, we categorized them into aggregated dimensions based on a taxonomic classification of the social cues of CAs introduced by Feine et al. (2019), which we extended based on our coding of selected categories, as some design elements did not fit these categories (i.e., interaction). Furthermore, we did exclude some of the original categories, as they were not fitting for our analysis frame (i.e., vocalizations).

Having a look at the verbal elements, we refer to this category for all IA elements that can be expressed by words, either written or spoken (Antaki, 2008). This category includes the sub-categories content, style, and adaptivity. In accordance with Feine et al. (2019) and Collier & Collier (2014), content focuses on what is being said and style on how something is being said. In comparison, adaptivity concentrates on the IA’s verbal adoption to the user’s style (e.g., Akahori et al., 2019). This dimension was added to the original taxonomy by Feine et al. (2019).

Elements that are included within the auditory category refer to design elements that can be perceived via the sense of hearing except the words themselves (Burgoon et al., 2013). The original taxonomy of social cues included two sub-categories, voice qualities, and vocalizations. Whilst the category voice qualities refers to all the elements that represent permanent and adjustable characteristics of the voice, such as pitch, volume, or the rate of the speech (Schmitt et al., 2021). We did not include the subcategory vocalizations in this paper’s analysis because there were not enough findings in this subcategory. However, because voice is becoming an increasingly important channel for IAs (i.e., Kendall et al., 2020), future research should focus on the empirical evidence pertaining to this sub-category.

The design elements in the category interaction allude to the interaction’s underlying structural representation, both in terms of communication medium and turn-taking mechanism. Within this area, we refer to design elements pertaining to mode and degree of freedom. It should be noted that the category of interaction design components is not included in the original taxonomy of social cues (Feine et al., 2019). We added this category since we found a lot of empirical research that focused on the mode of the IA, and the study by Feine et al. (2019) did not include a review of interaction design aspects. In this sense, the sub-category mode refers to the mode of interaction, such as chat or voice. Whereas the degree of freedom includes how free the user is in their engagement with the IA. We hope to gain a better grasp of associated interaction design consequences by expanding the original taxonomy of social cues.

Within the original taxonomy of social cues, the category invisible elements included the sub-categories chronemics and haptics. In the realm of our study, we adapted the original taxonomy and added the sub-category intelligence as well as personality, since both can be described as design elements that cannot be perceived by the sense of hearing or seeing (Knapp et al., 2013). The elements within the four sub-categories are also referred to as the “silent language” (Hall, 1990). In this context, the sub-category chronemics is referring to as timing-related cues in communication and thus are also related to turn-taking. While haptics can be described as tactile communication (Leathers & Eaves, 2015), and include the perception of touches such as high fives, kisses, or slaps. Even though they are visible in the sense of the eye of being able to sense them, they “communicate powerful meanings in the absence of any illumination and […] the decoder relies on cutaneous receptors rather than eyesight to decode them” (Leathers & Eaves, 2015, p. 13). Similar to the sub-category haptics, also the sub-category personality might be “visible” from time to time, however, within this study, it is being classified as an invisible design element. In this sense, the sub-category personality refers to enduring dispositions that are relatively stable over time, e.g., hard-working, calm, emotional (Goldberg, 1990). When it comes to intelligence, we follow as previously described the definition of Dellermann et al. (2019) and define it as the ability to achieve difficult goals, learn, reason, and perform effective behaviors.

Lastly, the category of visual design elements encompasses all non-verbal design elements that are not invisible and can visually be perceived except the words themselves (Leathers & Eaves, 2015). This category can be distinguished into four sub-categories agent appearance, computer-mediated communication (CMC), kinesics, and entrainment. The latter sub-category was named proxemics in the original taxonomy of social cues and was adapted for the means of this study. In accordance with Cauell et al. (2000), we describe entrainment as the adjustments of visual elements to the user, ensuring that the conversation will proceed efficiently. Whilst the agent appearance can be described as the IA’s graphical representation (Burgoon et al., 2013; Feine et al., 2019), kinesics refer to all body movements of an IA in the case of an embodied representation (Burgoon et al., 2013; Feine et al., 2019). The sub-category CMC refers to visual elements that augment or modify written texts, such as emojis or typos (Kalman & Gergle, 2014; Rezabek & Cochenour, 1998; Walther, 2006).

Using the sRAM model and the taxonomy of social cues as a frame for analysis serves as a guiding lens for the subsequent coding of the papers in the paper analysis step.

The 107 relevant papers were analyzed from a concept-centric perspective using an abductive approach. Therefore, we followed an iterative process aggregating the insights from identified studies, which required multiple coding rounds of the identified papers by different researchers. Thereby, the iterative process was started by two of the researchers to independently code a subset of 20 randomly chosen articles. For each of the 20 studies, we listed each dependent and independent variable as named by the author(s), which together formed our initial list of author variables (e.g., delayed responses and more human-like in Gnewuch et al. (2018, p. 11)).

Using the sRAM model and the taxonomy of social cues as a guiding frame for analysis, we carried out selective coding to create a comprehensive allocation of codes to our set of articles (Corbin & Strauss, 2014). In that, the sRAM model and the taxonomy of social cues informed the development of superordinated categories that we used for paper analysis. Moreover, we captured contextual variables such as the application domain and task of the IA.

Next, we re-examined the initial subset set of 20 articles and mapped author variables to our superordinated categories. During the next iteration, two researchers independently coded another subset of 20 articles. Thereby, we coded the dependent, independent, and structural variables and mapped these variables to the superordinated categories (e.g., delayed responses to chronemics and more-human like to perceived humanness in Gnewuch et al. (2018)). Afterward, these researchers discussed their own independent findings. In case the respective findings differed, a third researcher was involved in discussing the differences. This process was concluded once all articles were coded.

Concurrent with the aggregation of the codes from open coding, we also coded for empirical relationships between an independent and a dependent variable in each study. Thereby, following Jeyaraj et al. (2006), we assigned four possible codes to the relationship between independent and dependent variables: ″+1″, ″−1″, ″0″ and “M”. In this process, we coded ″+1″ for a positive relationship, ″−1″ for a negative relationship, and “0″ for relationships that were studied but did not show any significant value in the empirical results. In quantitative studies, we used P < 0.10 as the requirement for a significant positive or negative relationship. In case the study was qualitative, we relied on the authors’ argumentation, signified by a robust theoretical anchoring, which we coded as “M”. All told, we coded 389 relationships between independent and dependent variables (e.g., +1 for the relationship between turn taking and conversation flow in Winkler et al. (2019)).

The publications at hand adopted a wide dispersion of dependent variables. We identified 213 unique dependent variables (DV). We categorized these 213 DV into three broad categories: Social elements, relational elements, and functional elements.

We identified 390 independent variables (IV) used in IA research. To facilitate the discussion of this high quantity of independent variables, we categorized them into five broader categories. Thereby, our allocation into aggregated dimensions was based on a taxonomic classification of the social cues of IAs introduced by Feine et al. (2019), which we extended based on our coding of selected categories, as some design elements did not fit these categories (i.e., interaction). Each category is briefly discussed below.

Within the Auditory (3.3%) category, the elements that related to voice qualities (Schmitt et al., 2021), representing permanent and adjustable characteristics of the voice, were analyzed. In total, these cues were investigated 10 times. For example, Yu et al. (2019) studied the impact of the voice’s gender (female vs. male) on different perceptual outcomes. Although this category hypothetically also included nonlinguistic vocals and sounds, there were no studies in our sample addressing these elements.

Within the Interaction (14.5%) category, we summarize all design elements that refer to the underlying structural representation of the interaction both in regard to its communication mode and its turn-taking mechanism. Overall, the researchers often studied the choice of interaction mode. Moreover, researchers studied the influence of preset answers, which reflects the degree of freedom employed in the conversation. The former category was studied 34 times, whereby most researchers compared chat and voice interfaces (Kim et al., 2019). In comparison, the latter category was studied less but was found equally influential for user perceptions (Behera et al., 2021; Diederich et al., 2019).

Design elements in the Invisible (10.6%) can be divided into four subcategories: Chronemics refers to the role of timing in conversation and is reflected in studies that focused either on the design of conversation flows (e.g., Winkler et al., 2019), system response times (e.g., Gnewuch et al., 2018), or the role of synchronicity (e.g., Park & Sundar, 2015), which in total was studied 11 times. Intelligence refers to elements that express the cleverness of the agent, which was exemplarily studied by Xu et al. (2017). The other two categories, personality (i.e., personality traits) and haptics (i.e., tactile sensations), were comparatively less frequent in the research field. Among personality, e.g., Cafaro et al. (2013) examined the various personality traits.

The Visual (34.3%) category can be distinguished into four subcategories, which were widely studied (104 times). The most prominently researched variable was agent appearance (46 times). The embodiment of the agent gained much attention in our sample (e.g., McBreen & Jack, 2001; Nunamaker et al., 2011). Another studied aspect of the agent’s appearance was gender (e.g., (Pfeuffer et al., 2019; Zhang et al., 2020). Furthermore, kinesics, which refers to body movements such as demeanors (e.g., Krämer et al., 2013) and gaze patterns (e.g., Van Es et al., 2002), was addressed 25 times in total. Computer-mediated communication (CMC) refers to visual elements that augment or modify written texts and was examined 23 times. Here, the effects of using emojis (e.g., Park & Sundar, 2015), typos (e.g., Westerman et al., 2019), or videos and images (Huber et al., 2018) were researched. The other category, entrainment (i.e., the adjustment of visual elements to the user), was studied 20 times in total.

The Verbal (37.3%) category refers to all IA elements that can be expressed by words, either written or spoken (Antaki, 2008). Within this dimension, the conversation style, which refers to how something is being communicated, was the most researched independent variable (53 times). For instance, Mayer et al. (2006) studied the effects of relational strategies. The aspect of content captures all elements that relate to the literal meaning of a message and was researched a total of 22 times. For example, Akahori et al. (2019) looked at the effects of self-disclosure. Similar attention was given to adaptivity (22 times), which refers to the verbal adaptation of the IA to the users. Within this category, researchers studied the use of contextual information (e.g., Vtyurina et al., 2017), user content (e.g., Schuetzler et al., 2014), or the absence of adaptivity (e.g., Engelhardt et al., 2017).

In this section, we summarize our major findings concerning the 49 relationships we coded between 16 IVs and three DVs. At this detailed level, the frequency with which findings were replicated across studies was minimal and did not provide a very coherent or comprehensive picture of IA research. Hence, to study these relationships in a way that would be concise and helpful to researchers, we moved to a higher unit of analysis by reporting the 277 findings using our three categories of DVs and the five categories of IVs. Although precision was reduced when aggregating to the broader categories of DVs, we gained a better overall understanding of the determinants of perceptual and attitudinal outcomes of IAs. Thereby, we also aim to investigate the consistency of the empirical evidence. A detailed table of all included relationships between design Elements (IVs) and DVs can be found in the Appendix of this work. There we specify the IV first and second order constructs, the DV first and second order constructs, and indicate the relationships that could be measured between the IVs and the DVs. In terms of consistency, we looked for variables where at least 60% of the proof was reliable. This minimum threshold was chosen to ensure that more than half of the data yielded the same results. Furthermore, we have to note that findings previously coded with “M”, which referred to a qualitative study, were excluded in the final analysis of the relationships between independent and dependent variables. In the following, we structure our findings along the three DVs and visualize the relationships between the IVs and the DVs in respective figures. As we aim to provide a distinguishable indication on the most reliable results, we created a layered legend. We used the symbol ‘(++)’ to indicate when in more than 70% of investigated IV-DV-relationships, the authors discovered a positively significant relationship. If 51 to 69% of the coded relationships were positively significant, we used a ‘(+)’. Similarly, ‘(− −)’ denotes that in more than 70% of measured relationships between a particular IV and a particular IV were negatively significant, while ‘(−)’ denotes that 51–69% of measured relationships were negatively significant. Clearly, results with greater than 70% consistency are more reliable than those with between 51% and 69% consistency. These cutoff points are determined by the decision rules we used, but since all of the data is in Appendix 1, other researchers are able to re-run studies with different decision rules.

With regard to the relational elements of agent acceptance, prior publications provided insights into design elements that contribute to forming a social bond with the agent. In total, we found 121 relationships involving relational elements within 57 unique studies, which testifies to the high research interest in this aspect of user interaction with IAs. Thereby, several relationships showed consistent and robust evidence of a positive effect on relational elements such as rapport or trust. Overall, the relationships investigated were positive, but no consistent findings were discovered between relational elements and any of the five design elements. As a result, we believe that additional research on how verbal, visual, auditory, interaction, and invisible design elements affect user acceptance of IAs is necessary.

We structure the potentially fruitful future research directions for designing relational elements for IA user acceptance along the five categories of design elements investigated in this study:

Regarding the social elements of agent adoption, we identified 49 findings within 29 unique studies. Thereby, to no surprise, the design elements that represented social cues of IAs, especially, showed a strong and consistent effect on social presence and perceived humanness. However, the findings were especially related to appearance were not as consistent as one would have expected, especially since agent appearance is a well-researched area when considering early research on other agents (Nowak & Biocca, 2003). Thus, future research should incorporate a more fine-grained and configurational view on these design elements since we suspect that the interrelationship could be key in understanding social presence with IAs. Hence, we propose the following prospects for future research:

In the analyzed literature, manifold insights into how design elements contribute to creating functional elements for the user were gained. In this research stream, we identified 90 findings involving functional elements within 41 unique studies. Those with characteristics, especially regarding utility, were concerned with the accessibility or functionality of the interaction. Thus, we found strong evidence for the effects of degree of freedom (interaction), intelligence (invisible), agent appearance, and kinesics (visual). Moreover, we found consistent evidence that adaptivity (verbal), CMC (visual), and mode (interaction) may be positively related to functional elements; however, these results should be corroborated by further studies and replicated in different contexts. In general, the results regarding utility perceptions are already quite profound. However, we see merit, especially in the following research directions: