Designing Personality-Adaptive Conversational Agents for Mental Health Care

The necessity for mental health-related services has increased rapidly (Luxton, 2020). The prevalence of mental health issues (such as anxiety, depression or loneliness) is increasing among people worldwide: One in four people in the world is affected by mental health issues at some point in their lives (WHO, 2017). Access to therapists is limited, resulting in an acute shortage of mental health workers globally (Ta et al., 2020; WHO, 2021). Waiting times to receive treatment, therefore, can profoundly affect a person’s quality of life (Luxton, 2014). Further barriers to treatment access include high costs and unequal access to health care resources, but also attitudinal barriers, such as stigma toward professional treatments and skepticism about the effectiveness of treatment (Wasil et al., 2021). Technological advancements, however, have improved access to mental health care services (Jones et al., 2014). Conversational agents (CAs) represent one emerging technology with the potential to help address the barriers that contribute to the unmet health care needs (Luxton, 2020). CAs are software-based systems designed to interact with humans through natural language (Feine et al., 2019). CA is the overarching and general term for software that interacts with users via written or spoken natural language, and includes systems such as chatbots, which provide text-based communication, and virtual assistants, which rely primarily on voice-based interaction (Diederich et al., 2022). Applying Artificial intelligence (AI)-based methods, such as machine learning and natural language processing (NLP), CAs have rapidly become more efficient and enable broad and often cross-topic conversations (Diederich et al., 2022; Nißen et al., 2021). A benefit of CAs is that they are accessible anywhere and available at any time to provide counseling and deliver therapeutic interventions, and they may therefore be a promising source of support helping people cope with mental health issues (Luxton, 2020; Ta et al., 2020). A recent review by Abd-Alrazaq et al. (2021) demonstrates that the usefulness level of mental health care CAs is perceived as high by patients and that they overall have positive perceptions of and opinions about the CAs. Though AI-based CAs are increasingly capable of handling highly complex tasks, such as social support and therapeutic decision-making in a human-like way, currently available CAs, however, do not live up to their full potential yet (Fiske et al., 2019; Graham et al., 2019). Capturing dynamic human behavior to an adequate extent, and providing responses and reactions tailored to users’ individual contexts, special interaction dynamics and personalities, still pose a challenge in designing CAs (Grudin & Jacques, 2019; McTear et al., 2016). The field of tailored health communication has long established the need to personalize conversation (Abd-Alrazaq et al., 2021; Smith et al., 2015), since language is a primary tool to understand patients’ experiences and express therapeutic interventions (Laranjo et al., 2018). Psychotherapy, in particular, is highly patient-centered in clinical practice, requiring skills such as observing patient behavior and adapting to their individual personalities and needs accordingly (Graham et al., 2019; Laranjo et al., 2018). In their interaction with individuals, contemporary CAs lack the ability to dynamically change their own personality to adapt to the user’s personality. Provided a CA has features normally associated with humans, such as the use of natural language or human-like appearance (Nass & Moon, 2000), human beings tend to treat computer systems as social entities and ascribe different personality traits to them (Nass et al., 1994). Thus, as individuals’ interactions with computers have a fundamental social nature, users feel more appreciated and comfortable interacting with the machines when they perceive CAs as more human-like (Moon & Nass, 1996; Nass et al., 1993). One way to ensure that a CA is perceived as human-like, is to provide the CA with the ability to dynamically change its personality traits. Since personality preferences differ from person to person (McCrae & Costa Jr, 1997), the personalization of CA conversation is highly important and required (Abd-alrazaq et al., 2019). Based on an established personality model, and with advances in NLP, we propose the concept of a personality-adaptive conversational agent (PACA). PACAs are able to recognize and express personality by automatically inferring personality traits from users, giving them the ability to adapt to the changing needs and states of users when establishing a personalized interaction with them. As personality differences are manifested in language use, engagement with users can be further enhanced through tailored conversation styles (Kocaballi et al., 2020). While there is a large body of descriptive knowledge on design elements or cues that can be adapted, there is a lack of prescriptive knowledge on ways in which to actually design PACAs. Therefore, we pose the following research question (RQ):

To answer our RQ, we conduct a research project following a design science research (DSR) approach (Hevner et al., 2004). Our design approach is particularly anchored in two existing kernel theories, namely the five factor model of personality (McCrae & John, 1992) and the ‘computers are social actors’ paradigm (Nass et al., 1994). On the basis of these theories, we believe that PACAs have the potential to improve interaction with users by personalizing their health care (Luxton, 2014). To the best of our knowledge, there is no study that rigorously derives requirements from both literature issues and user stories to develop design principles, and that further evaluates the preliminary design principles by means of expert interviews, in order to propose an expository instantiation that translates the abstract knowledge, captured in the design principles, into applicable knowledge. Our results indicate that practitioners (e.g., designers, developers, etc.) who instantiate our design principles to create a PACA, receive a promising level of guidance to improve interaction with users in mental health care.

The remainder of this paper is structured as follows: In the section on related work, we give a brief overview of the history and current developments of mental health care CAs. We also provide an overview of current research and highlight the critical aspects in the use of mental health care CAs. In section three, we illustrate the theoretical foundations of our DSR project. Section four presents our research methodology in more detail. Section five contains our derived and evaluated design principles for PACAs, as well as a demonstration of an expository instantiation. Finally, we discuss our results, current limitations, and contributions of our work, and close with a conclusion.

Efforts to develop software-based systems within the health care environment have a long history. In fact, ELIZA – widely considered the first CA in history – first appeared in 1966 in a psychotherapeutic context (Weizenbaum, 1966). Though only developed for demonstration and not commercial purposes, the simple computer program was able to mimic a Rogerian psychotherapist communicating with people in an empathic manner (Weizenbaum, 1966). The interaction resulted in people ascribing human characteristics to it, and some psychiatrists seeing ELIZA’s potential computer-based therapy as a “form of psychological treatment” (Kerr, 2003, p. 305). The underlying technology of ELIZA was rather simple: By searching the textual input of its conversation partner for relevant keywords, the machine produced appropriate responses according to rules and directions based on scripts hand-crafted by the programmers (Natale, 2018; Peters, 2013). PARRY, another early example of a prototype CA developed in 1979, was designed to simulate and behave like a person with paranoid schizophrenia (Shum et al., 2018). The developers’ intention was to find out if other psychiatrists could determine a real paranoid patient from their computer model (Shah et al., 2016). PARRY was a rule-based CA and worked in a similar way as ELIZA, though with better language understanding capabilities (Shum et al., 2018).

Current interest in CAs for mental health care is also nascent, as can be seen in the growing number of online services offered by health care providers (Bendig et al., 2019). More than one-fourth of 15,000 mobile health apps focus on mental health diagnosis or support, according to the World Health Organization (Abd-Alrazaq et al., 2021). Over the last few years, three particularly prominent therapeutic mental health CAs based on AI technologies have emerged: Woebot (Woebot Health, 2021), Wysa (Wysa, 2021) and Tess (X2, 2021). These CAs are publicly available mobile phone applications aimed at helping people to manage symptoms of anxiety and depression by providing counseling services. Woebot is a CA that is built to assess, monitor and respond to users dealing with mental health issues (Woebot Health, 2021). The CA has a responsive way to intervene with in-the-moment help and provide targeted therapies based on cognitive behavioral therapy (Woebot Health, 2021). The AI chatbot Wysa is also based on cognitive behavioral therapy, but employs several other methods, such as behavioral reinforcement and mindfulness, to help clients with depression (D’Alfonso, 2020; Wysa, 2021). According to its developers, Wysa provides 24/7 high-quality mental health support (Wysa, 2021). The mental health chatbot Tess pursues a similar approach by being available 24/7 and delivering conversations in the same way that “a coach or friend would” (X2, 2021). Preliminary studies of the efficacy of all three applications have shown a significant reduction in depression and anxiety levels in the group of participants using the CAs (D’Alfonso, 2020). Though the presented CAs were developed to have specific personalities, they are not able to dynamically change these or to infer user personality in order to be more adaptive towards the users’ needs. Therefore, all three CAs represent a rather “one-size-fits-all” solution, by not adequately adapting to the specificities of their users.

Ever since research in the field of human-machine interaction stressed the importance of avoiding “one-size-fits-all” interactions, the customization or personalization of CAs to individual users have become an important research topic (Abd-Alrazaq et al., 2021). Though this topic is still in its infancy (Kocaballi et al., 2019), there has been an increased interest in studies addressing the aspect of personality adaptivity in CAs. Studies from Yorita et al. (2019), Kampman et al. (2019) and Ahmad et al. (2020a, b) show that it is technically feasible to develop a CA that adapts its own personality traits to match the identified traits of the user. Yorita et al. (2019) developed their CA specifically with the purpose of providing emotional support for the user. While the studies by Yorita et al. (2019), Kampman et al. (2019) and Ahmad et al. (2020a, b) primarily focus on text-based human-machine interaction, the work of Völkel et al. presents approaches to adapt a voice assistant’s personality to the user, in order to improve interaction experience (Völkel et al., 2020, 2021). Further studies by Ranjbartabar et al. (2018) and Zalake (2020) particularly deal with CAs that, among other factors, aim to adapt to user personality to reduce study stress (Ranjbartabar et al., 2018) or to promote anxiety coping strategies among college students (Zalake, 2020). The research of Wibhowo and Sanjaya (2021) concentrate on CAs for use in clinical psychology and psychotherapy: The authors developed a CA as a warning system to prevent individuals with borderline personality disorder from committing suicide. The previous studies mainly employ empirical methods to describe behaviors in interaction with these systems and consequently generate descriptive knowledge about the use and effectiveness of personality-adaptivity CAs. In addition, these studies all have different focuses, so there is still a research gap in providing prescriptive knowledge about how to design a PACA to improve interactions with users in mental health care.

Due to technological advancements, AI-based CAs have become increasingly capable of handling highly complex tasks with human qualities such as a higher autonomy of decision-making (Brendel et al., 2021) or expressing human-like feelings (Porra et al., 2019). Consequently, the application of CAs can have diverse impacts on individuals – both positive and negative. The field of mental health care raises ethical considerations by its very nature (D’Alfonso, 2020). Voices have therefore emerged from the research field asking for a reassessment of the potential “dark sides” of AI and the ethical responsibilities of developers and designers (Brendel et al., 2021; Porra et al., 2019). Critics specifically stress the caveats of creating and perfecting human-like CAs with simulated feelings, without considering long-term consequences for human beings, such as deep, emotional attachments (Ahmad et al., 2021a). However, human beings treat computer systems as social entities and ascribe different personality traits to them (Nass et al., 1994); hence, they feel more appreciated and comfortable interacting with the machines when they perceive CAs as more human-like (Moon & Nass, 1996; Nass et al., 1993). To ensure an authentic interaction experience, CAs therefore should be imbued with some degree of human-like features (Gnewuch et al., 2017). While advanced CAs are able to simulate conversations employing therapeutic techniques, it is “not on the near horizon” (D’Alfonso, 2020, p. 113) for CAs to replicate human therapists. In fact, researchers agree that mental health care CAs should be used primarily as support systems, since the interaction experience and relationship that develops between a therapist and a patient is considered a significant factor in the outcome of psychological therapy (D’Alfonso, 2020) and cannot easily be substituted by a machine. The role of CAs in mental health care, rather, is to address individuals in need of treatment who are not receiving any treatment at all due to various barriers (Bendig et al., 2019; Stieger et al., 2018). In this way CAs could provide low-threshold access to mental health care but also bridge the waiting time before approval of psychotherapy (Bendig et al., 2019; Grünzig et al., 2018). Mental health care CAs have the potential to create their own form of interaction experience with users, provided the CA responses to users are tailored to their individual personalities. As a result, a personalized interaction would improve health outcomes among care seekers (Luxton, 2014). Given the widespread interest in mental health care CAs and the lack of prescriptive knowledge on how to design PACAs, it is important to address this research gap for researchers, designers, developers and clinicians.

To generate design knowledge about the design of PACAs, we follow the design science research (DSR) approach proposed by Hevner et al. (2004). The overall goal of DSR projects is to generate rigorously derived and relevant design knowledge. Design knowledge thereby covers all aspects of the relationship between the problem and the solution space (Venable, 2006; Vom Brocke et al., 2020), and aims to create prescriptive knowledge that contributes to both the theory and practice of solving real-world problems (Hevner, 2020). By providing prescriptive solution-oriented design knowledge, DSR therefore represents a complement to descriptive research that explains the nature of a phenomenon or problem. Gregor and Hevner (2013) define descriptive knowledge as omega-knowledge, which includes “descriptions of natural, artificial, and human-related phenomena” and is “composed of observations, classifications, measurements, and the cataloging of these descriptions into accessible forms” (p. A2). In contrast, DSR also creates prescriptive knowledge about artifacts that address the phenomenon or problem (Gregor et al., 2007), which Gregor and Hevner (2013) define as lambda-knowledge. DSR should not only generate original solutions to existing problems, but also demonstrate the beneficial application of such solutions to the problem and show their broad implications (Baskerville & Pries-Heje, 2010; Vom Brocke et al., 2020), thus generating both omega and lambda-knowledge (Hevner, 2020). Prescriptive design knowledge (lambda-knowledge) can be represented in different ways, such as design principles (DPs), design features, or instantiations (Hevner, 2020; Vom Brocke et al., 2020), whereas descriptive knowledge (omega-knowledge) describes observations, measurements or classifications of phenomena and sense-making, such as patterns or natural laws (Gregor & Hevner, 2013). Hence, DSR projects involve an interplay of synergies between descriptive and prescriptive knowledge, as well as the use of existing knowledge and the production of new knowledge (Hevner, 2020).

DPs as a form of formalized knowledge are gaining popularity in the field of information systems (IS) because they allow researchers to capture abstract or meta-knowledge that addresses a class of problems, rather than a single problem (Gregor et al., 2020; Iivari et al., 2021; Purao et al., 2020). To ensure practical relevance and applicability, DPs should imply accessibility, effectiveness, and, most importantly, guidance for action (Iivari et al., 2021). In our research, we follow an iterative and multi-step process that is fundamentally embedded in the three-cycle view of DSR (i.e., relevance cycle, rigor cycle and design cycle) (Hevner, 2007), to generate design knowledge for the design of PACAs for mental health care. The relevance cycle incorporates issues from the problem space in the study and places outcomes from the design cycle in evaluation and practice. The rigor cycle involves the use of kernel theories and scientific methods, as well as research experience and expertise, in the study, while also adding new knowledge created by the researchers to the growing knowledge base. The design cycle, as the core of DSR, includes construction activities for the development and evaluation of artifacts and design processes (Hevner, 2007). In our DSR project, we generate design knowledge in the form of DPs and an expository instantiation, by involving all three cycles multiple times. We follow the first strategy as proposed by Iivari (2015), in which we construct a meta-artefact (i.e., DPs) as a general solution concept, and then further adapt and follow the methodological components proposed by Möller et al. (2020) for the development of DPs in IS. These authors propose a supportive approach as well as a reflective approach that differ within “the point of artifact design and the logic of generating design principles” (p. 214). The two approaches of Möller et al. (2020) are similar to the two general strategies proposed by Iivari (2015) in that either generalizable design knowledge is created first, which is then concretized, or concrete design knowledge (e.g., design features or instantiations) is created first, which is then abstracted and generalized. However, the approach of Möller et al. (2020) focuses specifically on the construction of DPs and proposes a methodological procedure. We follow the supportive approach, in which DPs are “the provision of design knowledge in advance to support the design of an artifact before the design process takes place” (p. 214). Figure 1 shows our DSR approach based on Hevner (2007), Iivari (2015) and Möller et al. (2020).

Our DSR approach is divided into two main phases, addressing the problem space and the solution space (including the evaluation of the solution), which in turn consist of three sub-steps, respectively. To generate and evaluate a solution in the form of DPs for the identified problems, we use a mapping diagram. Mapping diagrams help to visualize the connection and derivation logic between DPs and meta-requirements, as well as between the problem space and the solution space (Möller et al., 2020). In this way, connections between and derivations of the individual aspects become clearer. The mapping diagram of the derivation and construction of the MRs and DPs can be found in the Appendix (see Fig. 4). An interplay of synergies between the rigor cycle, relevance cycle, and design cycle is reflected in each sub-step.

An important prerequisite for effective and practical design knowledge is a good understanding and description of the underlying problem space. To conceptualize the problem space, Maedche et al. (2019) propose considering and analyzing four key concepts of the problem space; stakeholders, needs, goals, and requirements. In our approach, the process of understanding and defining the problem space consists of three steps addressing the needs of the stakeholders and issues from the application domain, which we subsequently captured in requirements. To conceptualize the problem space, we followed three steps that build upon each other:

We approached the problem space by reviewing extant literature and identifying current problems in mental health care. In a next step, and based on our kernel theories, we conducted a user survey to capture the perspective of potential users and to derive meta-requirements together with the literature issues. We followed an explorative approach and conducted a qualitative study (Babbie, 2020) using an open questionnaire with the aim to capture comprehensive user stories about PACAs in the context of mental health care. Our open questionnaire consisted first of an extensive explanation of the functionality and nature of a PACA to make sure participants from all backgrounds understood the concept. After that, we used an open questionnaire to capture user stories about the design of PACAs in the context of mental health care. These open questions asked for general requirements (e.g., mental health support, safety and privacy) and design-related requirements (e.g., behavior, communication style) (see Table 1 for an excerpt from the survey with sample responses). To help participants visualize what a conversation between a patient and a PACA might look like, we created a predefined chat record. For our simulated dialogue, we used the conversational design tool Botsociety, which allows prototyping and visualizing CAs. The conversation was provided in the form of a video. Figure 2 shows a mock-up of the simulated interaction. The video with the entire conversation can be viewed here: https://​youtu.​be/​-sfSNJwCCI0

The survey was distributed via our private network and the crowdsourcing platform Mechanical Turk (mTurk) and was carried out in December 2020. A total of 60 respondents participated in the study, producing more than 6865 words of qualitative data answering the open questions, which took roughly between 25 and 40 min to complete. Table 1 shows an excerpt from the survey with four open questions and example responses that were used for qualitative content analysis.

Participants (32 male, 28 female) were between 23 and 71 years old, with an average age of 36 years. In order to analyze the data, we followed a qualitative content analysis by coding the answers of the participants, which consisted mainly of inductive category forming (Mayring, 2014). The authors conducted the coding process independently, and whenever the results differed, we discussed the points of disagreement until we reached a consensus. In the last sub-step, we constructed meta-requirements from the captured user stories, the issues from the literature and the application domain.

Our proposed solution to the problem space identified in step 1 is a PACA for mental health care, for which we present design knowledge in the form of DPs. We constructed our DPs based on the meta-requirements, and then evaluated them with experts from the application domain. For the systematic derivation and evaluation of our solution space, we followed the following steps:

Our DPs are formulated based on the approach proposed by Gregor et al. (2020, p. 2), who defined the anatomy of design principles so that DPs are “understandable and useful in real-world design contexts.” The authors point out the importance of including the concerned actors when formulating the DPs to complement the anatomy, with the aim that DPs should be “prescriptive statements that show how to do something to achieve a goal” (Gregor et al., 2020, p. 2). The anatomy of a DP consists of the aim, implementer, and user; context; mechanism; and rationale, and is presented in the form of heuristic rules. The evaluation of constructed artifacts such as DPs is an essential step in the design cycle to generate rigorous design knowledge. To evaluate our constructed DPs, we chose a qualitative approach in the form of expert interviews. To ensure relevance to the application domain, we selected experts, such as psychologists and psychiatrists, and had them evaluate the constructed DPs from a psychotherapist’s perspective. The criteria for selecting experts were a minimum of at least two years of professional experience in the care or therapy of mentally ill people, as well as a completed apprenticeship or studies in medicine, psychology, or another related field. The experts were selected and approached from our personal network. The work of EX1 (a psychologist) and EX6 (a social worker and therapist) particularly focus on youth care, while EX2 to EX5 are all psychiatrists who trained as psychotherapists and work with patients of all ages. EX2 deals specifically with geriatric psychiatry. The six experts were interviewed between March and April 2021. Each interview took between 50 and 80 min. Table 2 shows an overview of the interview panel, including the experts’ education and professional background.

The interviews were conducted with the support of a semi-structured interview guideline (Mayring, 2014). Interviews that follow a guideline are based on pre-defined questions that help orient the interviewer and thus ensure that all critical aspects of the scope of the interview are included. In addition, queries and deviations by the experts are possible (Mayring, 2014). The interview guide started with general questions about the profession of the experts, their professional experience, and their experience in dealing with CAs. This was followed by a detailed explanation of the concept of a PACA and follow-up questions by the interviewer to ensure that the concept was understood. Thereupon, all DPs were examined individually and evaluated by the experts, who were asked to recapitulate each DP in their own words, to again check whether they understood each DP. The experts were then asked to comment on each DP in terms of its relevance and to add possible missing aspects or highlight particularly relevant aspects. Subsequently, the audio recordings were transcribed and coded using MaxQDA (version 2020) qualitative data analysis software. The results were incorporated in the further development of the DPs. Despite the anatomy of a DP and different frameworks for the accumulation of design knowledge (Rothe et al., 2020), the reusability and use of DPs for practitioners is often challenging. Iivari et al. (2021) address this problem and specifically propose strengthening the actability and guidance for the implementation of DPs to maintain the practical relevance of DSR. Since DPs represent formalized design knowledge, we design an expository instantiation (Gregor et al., 2007) that translates the abstract knowledge captured in DPs into applicable knowledge. The next section presents the result of the two steps and their sub-steps.

Though being a fruitful area with large practical potential, the adoption of CAs for mental health care is associated with some challenges. Literature issues involve the inability of current CAs to capture and adapt dynamically to user personality, as well as to provide responses and reactions tailored in accordance with users’ individual characteristics to an adequate extent. In addition, CAs do not live up to their full potential yet, as they have been shown to have limited conversational abilities, particularly when it comes to longer and more complex interactions. These issues, however, are important factors that need to be taken into consideration when addressing primary issues from the application domain, such as the increased necessity for health-related services, the shortage of health care providers, or the paramount importance of patient safety. Motivated by the lack of design knowledge for PACAs, we proposed and evaluated DPs for designing PACAs that improve interaction, specifically for users who seek support for their mental health. We focused on two steps to answer our research question: Based on our kernel theories we first identified current issues from the AD and LIs, as well as derived USs, through a qualitative study. Within this first step of our problem space, we then derived MRs. On the basis of step one, and as part of our solution space, we proposed DPs for PACAs in step two. We then conducted expert interviews with psychologists and psychiatrists to evaluate the derived DPs, and adjusted and refined our final DPs based on their feedback. In a last step, we transferred our defined DPs to an expository instantiation for the purpose of better visualization.

According to the DSR contribution framework proposed by Gregor and Hevner (2013), our work can be classified as an improvement, as we address a known problem with a new solution. We provide prescriptive design knowledge by deriving and evaluating DPs for PACAs in mental health care. Our DPs contribute to the body of design knowledge for CAs and provide guidance for practitioners, such as designers, developers, and mental health organizations, on how to design PACAs that can better support their users. Instantiating these principles may improve interaction with users who seek support for mental health issues. We believe that our design approach could also be a valuable starting point for the design of PACAs in other domains.