Enhancing workplace competence acquisition through a persuasive system

The concept of competence was first introduced in the human resources field by David McClelland (1973) who argued that competences should be assessed instead of intelligence to better account for job success. He also provided six principles to test competences more accurately.

Boyatzis (2008, p. 6) defines a competence as “a capability or ability”. But there is no convergent definition of the term “competence” in the human resources management literature (Le Deist and Winterton 2005). For example, Draganidis and Mentzas (2006) list a dozen of definitions of the term in the academic and practitioner literature in order to provide their own definition of competency: "a combination of tacit and explicit knowledge, behaviour and skills, that gives someone the potential for effectiveness in task performance” (p. 53). Actually, competence is often referred to as a combination of different factors such as knowledge, skills, and/or individual characteristics. Le Deist and Winterton (2005) propose a holistic competence model composed of three fundamental dimensions: cognitive (knowledge and understanding), functional (skills) and social (behaviour and attitude). While the distinction of the three fundamental dimensions could be analytically identified, in practice, the competence of an individual is the result of the combination of these three fundamental dimensions. Those three dimensions are also identified by Boyatzis and Ratti (2009, p. 824): “Competencies are a behavioural approach to emotional, social, and cognitive intelligence”. In addition, competencies may be fostered in an organization through training and development (Boyatzis 2008; McClelland 1998), aiming to improve the performance of specific tasks. The extent to which the training is effective may be measured on the basis of pre-established criteria (Boyatzis 1982). Research indicates that competencies can be used to distinguish outstanding top executives and managers, as both positions tend to exhibit distinct social, emotional, and cognitive intelligence competences (Boyatzis and Ratti 2009).

Moreover, in this paper we focus specifically on the concept of atomic competence (Baldoni et al. 2011). Atomic competences (also known as micro competences) can be understood as the smallest element of a competence (Pedraza-Jimenez et al. 2004; Valverde-Albacete et al. 2003). Once a competence is identified, it should be verified if it corresponds to an atomic competence (Baldoni et al. 2011) or if it is composed by a set of atomic competences. In the second case, the competence is further broken up until a single atomic competence is reached. For each micro competence, it is necessary to define its fundamentals characteristics that can be translated into simple tasks.

We adopt Howell's (1982) five-stage competence model as it provides a quite comprehensive framework that covers a wide variety of competences valuable in an organizational setting—from behaviours related to IS security (Thomson et al. 2006) to skills required for cross-cultural team working (Barnes et al. 2000; Tung 1993). The key assumption underlying the model is that people will respond to training when they are aware of its value and need (Thomson et al. 2006). The five stages of competences are described below:

In order to foster competences development, companies tend to rely increasingly on e-learning systems (Capece and Campisi 2013; Cheng et al. 2014; Costello and McNaughton 2016; Mohammadyari and Singh 2015). The rapid pace of mainstream adoption of new forms of ubiquitous digital devices such as smartphones and tablets are enabling an ubiquitous learning environment that can be paramount for the development of competence acquisition tools (Chu and Lo 2011; Kaganer et al. 2013; Lyytinen et al. 2004). As users are able to accomplish a multitude of tasks and interact fluidly in an ubiquitous ecosystem (Carillo et al. 2014), they become empowered of their own individual learning process. For instance, an interesting example of innovative design and delivery of competence development is Google’s Primer mobile app. It aims at training employees in a fast and easy way with a 5-min module. It allows people to get free training on marketing basics anywhere and at their own pace.

Capece and Campisi (2013) demonstrated that e-learning training programs can generate positive attitudes towards technology-mediated education. They conducted a case study in a multinational company where they collected 5083 questionnaires. Respondents had to train themselves on the Sarbanes–Oxley Act through a handful of e-learning modules and, overall, the results indicate that perceived usefulness and ease of use of the modules lead to employee satisfaction. More recently, Dachner et al. (2021) identified 5 firm- and job-related trends that require rethinking employee development in the workplace. First, employees have to deal with more complex and dynamic tasks that necessitate soft skills (such as creativity) and problem-solving strategies. They also often face time constraints because of the imperative of new competency requirements. Second, employees must deal with increasing work demands, very often longer hours of work, more responsibilities and travelling. Third, they have less opportunities for upward mobility. Fourth, technologies have become pervasive in the workplace and central in employees’ communications and interactions. Five, since employees change jobs more regularly, they have to continuously learn new knowledge and skills.

These five trends allow us to conclude that employee development should be more focused on the acquisition of soft skills and that it needs to take into account time and space constraints that employees face. Dachner et al. (2021) also recommend proactivity in competence development programs so that employees feel in personal control of what they learn. Proactive learning should also help employees “feel capable, sufficiently motivated, and energized” (ibid, p. 7).

Prior research has mainly focused on “learning strategies, barriers, facilitators and outcomes” (Crouse et al. 2011), thus providing very little attention to design issues. This lack of understanding on how to best design and develop mobile systems for employee training in the workplace represents an important research gap which this study aims to contribute to (McPherson and Nunes 2006; Wong 2007; Zhang et al. 2004). In the next section, we suggest a renewed approach, based on persuasive systems, to develop e-learning modules for workplace training that acknowledge the five trends identified by Dachner et al. (2021). The objective is to develop a system for supporting the acquisition (learning process) of a specific competence—exploiting the ubiquity nature of digital technologies. As such, a mobile app could represent an adequate solution (Pereira and Rodrigues 2013; Pimmer and Pachler 2014).

Persuasive systems have increasingly become the object of industry and scholarly research (Byrnes 2015). Persuasive systems respond to a critical issue of making users go beyond simply adopting technologies (Venkatesh and Davis 1996) to actually engage with them over an extended span of time in order to develop desirable behaviours and organizational practices (Peng et al. 2019).

The underlying mechanism of persuasive information technologies is persuasion. It is known to be an effective method to imply a voluntary change of behaviour in individuals (Berdichevsky and Neunschwander 1999). It differs from coercion or deception since persuasion does not force a change of behaviour nor stipulate the use of misinformation. Media and digital technologies (from billboards to television and the Internet) play a relevant role in “facilitating the delivery of persuasive messages to purchase, donate, vote, concede, or act” (IJsselsteijn et al. 2006, p. 1).

Usually, the process of persuading individuals to perform a target behaviour requires the simultaneous presence of three factors: motivation, ability (or simplicity), and triggers (Fogg 2009a). Figure 1 presents the adapted Fogg’s conceptual framework illustrating the relationships among these three factors.

High motivation (the reason to carry out the target behaviour) and high ability (often related to “simplicity” to accomplish a specific task) are necessary to reach the target behaviour but are not enough. Trigger is the third fundamental factor that is necessary as well and at the same time: indeed, “without an appropriate trigger, behaviour will not occur even if motivation and ability are high” (Fogg 2009a). A trigger can have different forms, such as an alarm or a notification on a digital technology. The main objective is to draw the individual’s attention to perform a specific task.

The emergence of “persuasive technologies”—interactive computing systems designed to influence people's attitudes and behaviours towards a specific target—also represent a unique opportunity for improving the process of employee competence acquisition as they have the capacity to foster user interest over time (Fogg 2003).

Those technologies exert power on individuals because they incorporate social cues such as physical and psychological cues, language, social dynamics, and social roles. For instance, Fogg’s research has shown that simulators, avatars or video game consoles can be considered as persuasive systems for their physical cues: the attractive interfaces of these devices especially contribute to increase their persuasive power. Indeed, “a computing technology that is visually attractive to target users is likely to be more persuasive as well.” (Fogg 2003, p. 93). Smartphones also qualify as persuasive technologies because they possess all of the social cues identified by Fogg (2003). In addition, the ubiquity and multiple functionalities of the devices contributes to social presence at a large scale (Cui et al. 2021).

The Elaboration Likelihood Model—ELM—(Petty et al. 1983) is the most common framework for developing persuasion and it is applicable to various sources and context variables (Kitchen et al. 2014). According to the ELM, persuasion may be induced through a central route or a peripheral route: the former represents the content and it is related to the interest stimulated by the content of the message received by the users, the latter represents the format and it is related to cues such as attractiveness of the message source (Petty and Cacioppo 1986; SanJosé-Cabezudo et al. 2009). The state of “elaboration likelihood” (EL) is the extent to which individuals choose to process the information provided by a source. If the EL is high, individuals are likely to process the message received, since they are interested in the arguments of the message (central route). On the other hand, if the EL is low users need to be motivated by using extra stimuli (peripheral route) (Cyr et al. 2018). The state of EL determines the route through which persuasion may occur (Petty et al. 1983; Petty and Cacioppo 1986). Furthermore, the combination of both routes can “act jointly and significantly to impact attitudes and intentions in individuals’ behavior” (SanJosé-Cabezudo et al. 2009, p. 306), where one route—usually the peripheral—might enhance the effects of the other one—often the central (Cyr et al. 2018; SanJosé-Cabezudo et al. 2009).

One way to foster user engagement is to incorporate persuasive elements in the design of a digital solution aimed at fostering behavioral competency acquisition (Oinas-Kukkonen and Harjumaa 2009). Specifically, the artefact should incorporate persuasive elements for further stimulating the user engagement in the digital learning process (Chatterjee et al. 2018; Filippou et al. 2016) and for influencing human behaviour in a positive way, motivating users to engage in self-management and triggering behavioural changes (Asbjørnsen et al. 2019). For instance, Filippou et al. (2016) created two statistical models that help understanding how students and alumni manage their academic performance. Those models are presented as the basis for the development of a persuasive system to improve students’ learning. While the authors did not develop a persuasive system per se, they rely on Persuasive Systems Design to influence students’ behaviour in order to improve their academic performance. Along similar lines, Widyasari et al. (2019) suggests the application of Web 2.0-based e-learning system with persuasive features can be an effective mean to positively change learners’ behaviour. More precisely, they conducted a survey in a class of 30 students to uncover if persuasive features could change users’ acceptance of new learning methods. Their findings showed mixed feelings regarding user involvement and intention to use online learning. However, both studies did not provide any technological artifact or precise features to impact learning, nor do they provide measures of engagement or effectiveness, gaps we try to fill in with that study. According to Fogg (2009a, b), persuasive systems should target a single behaviour to avoid flooding users with excessive information and to ensure its effectiveness. Therefore, selecting one atomic competence for our DSR approach aligns with Fogg’s recommendation and ensures the persuasiveness of the training program.

The first step necessary for creating the set of specific exercises for the STM was to identify a specific competence to be acquired by employees. As stated in the introduction, among the top 10 skills that learning and development experts deemed important in 2021, 9 out of 10 were soft skills (Linkedin Learning 2021). Soft skills are “intangible, nontechnical, personality-specific skills that determine one’s strengths as a leader, facilitator, mediator, and negotiator” and are broadly applicable independently of one’s profession (Robles 2012, p. 457). The author points out that the most important soft skills perceived by business executives are integrity and communication. As a result, it was decided to develop a STM focused on the skill of how to communicate an “improvement feedback” when an employee or colleague behaves in an undesirable manner.

The result of the first two phases of the framework guiding the production of the tiny exercises (Fig. 2) is the definition of this specific atomic competence, specifically how to communicate an improvement feedback. An “improvement feedback” is not something to be done intuitively, but it represents a skill that needs to be acquired especially by managers who supervise a team of workers. It should be provided in four sequential steps (Aharon Tziner and Latham 1989; Blanchard and Johnson 1983; Pearce and Porter 1986; Shute 2008): (i) the description of undesirable behaviour (that is the reason for providing the “improvement feedback”), (ii) its possible negative consequences (disadvantages), (iii) the description of the desirable behaviour (the alternative), and (iv) its possible positive consequences (advantages). The feedback must be referred to the behaviour and not to the person, and it must be free by the influence of personal opinion. It also should not be based on a comparison with what is done by other people and must be based on objective information or direct observation. Finally, the improvement feedback must be clear and shall not contain contradictions.

These factors were grouped in two specific categories: objectiveness and morphology. Morphology refers to users’ understanding of improvement feedback nature and structure (completeness, order, focus on behaviour and specificity), while objectiveness refers to the impartiality of the manager (judgment based on direct experience and observation, and not based on a comparison with other people, not subjective or does not contain contradictions).

Based on this definition, around 70 use cases were developed. From each use case it was possible to produce 7 or 8 different simple questions. Finally, approximately 500 simple exercises were developed for this specific atomic competence. Most of the exercises provide a description of a simple real situation in which a hypothetical manager needs to give feedback to an employee or a colleague. Few exercises provide theoretical short definitions on the “improvement feedback” used for the first part of the training process (Fig. 3). The exercises were presented in the following formats:

The effectiveness of the mobile app was pre-tested by a group of six people (3 training experts, 2 researchers and one developer). This test group simulated an entire learning process for the specific “soft skill” and thus gave us the possibility to evaluate the functionalities of this method and to refine some aspects about the usability of the app (e.g. default values settings), bugs on the app (user interface) as well as the server side (administration interface).

The study was based on a real-life scenario and was developed to evaluate the overall effectiveness and efficiency of the app. Twenty employees (middle or project managers) of an IT consulting firm were invited to take part in the pilot study. The participants were divided in two groups: a test group consisting of 12 members and a control group with the 8 remaining participants. The participants of the test group were on average 46 years old, while the participants of the control group were 43 years old. The majority of the participants were male, having only one female in each group. While this could potentially represent a gender balance issue of the sample, it is actually a fair representation of gender distribution in the field of software development (https://​survey.​stackoverflow.​co/​2022/​#developer-profile-demographics). As shown on Fig. 6, both groups started the learning path together in an introductory class, as generally done with blended learning where there is a combination of online and face to face experiences (Means et al. 2013). This class introduced the basic concepts on “how to provide improvement feedback”. During the lesson, several examples were provided based on common scenarios—strongly linking with the work activities that the group was involved in. At the end of the class both groups performed the test for measuring their entrance level (test in). During the following three weeks, both groups received by email multimedia material (e.g. slides or small videos) with messages that reinforced specific concepts covered in class. These messages also encouraged them to apply those concepts in their work environment. In addition to the emails, the test group members had access to the mobile app. On average, half of the app users completed the daily micro-quest twice (aiming to improve their scores). In total, more than 85% of the available poll of “micro-quests” were utilized during the experiment.

At the end of this follow-up period, all employees involved in the pilot study performed a test (test out) in order to assess if there was any improvement of their competence level. Both test-in and test-out are based on three open-ended cases. Each case provides the description of a hypothetical scenario where an employee produces undesirable behaviour. The user is challenged to provide an improvement feedback for each of the three cases (providing feedback to distinct hierarchical levels—e.g. a peer, a subordinate and a boss).

The eight factors defined in the training development phase (e.g. structure, nature and objectiveness of the feedback) were used to assess the degree of competence acquisition by the sample. The test-in and test-out results are reported in Fig. 7 for both groups.

The efficacy of the two post-lesson training approaches (with and without the mobile app) was notable by an overall improvement in the scores of the test-in and test-out results. It was also noticeable that the level of improvement on the results of the group using the app was significantly higher. Indeed, the average improvement for the control group was 15.6% while for the test group was 40.6%. This divergence is quite salient regarding the categories measuring feedback objectiveness. In addition, members of the test group achieved a final score of 100% in four categories. On the other hand, the control group did not achieve a score 100% and in one of the criteria (“Directly Observed”) the actual test out score was lower than the test in.

A further follow up test was done six months after the conclusion of the experiment—aiming to evaluate the longitudinal efficacy of the training method. The structure of the assessment was the same used in the test-in and test-out phases, however, different case scenarios were deployed. The results of the follow-up are in line with the outcome of the previous test-out phase, which favoured the test group. In addition, it was noted that some discrepancies between the two groups have increased over time. In particular, the performance gap of the feedback structure parameters had significantly increased between the two groups over time. Overall, the control group presented an average performance decrease of -4.2%, while the test group achieved an average improvement of 2.3%.

Finally, in order to have a more wholesome understanding on the efficacy of this training method, a short feedback was asked to all members of the experiment, aiming to capture their perceptions regarding feedback competence level, efficacy of the training process as well as frequency of use of the competence acquired.

Most of the respondents from the test group noticed positive changes in the feedbacks transmitted by their boss, while most of the control group did not notice any change. Some test group members also observed an improvement in the way the feedback is communicated. One commented: “I acknowledge positive changes regarding the communication aspects and attention to feedback.” Another positive result concerns the relationships between feedback providers and feedback receivers. One employee mentioned: “I saw him (the company owner) more available to answer my questions, especially without creating those situations of embarrassment when the answer to my question could be ‘this is something you should already know’”. Finally, another respondent noticed a more constructive feedback approach adopted by his managers: “During the last few weeks, I’ve noticed a more constructive attitude in the way feedback was given to us”.

As revealed by the participant feedbacks, the perception of the competence level as well as the efficacy of the training method, were increased for all members of both groups—even though it was higher for the test group. The test group feels quite capable of correctly performing the acquired competence, while the control group revealed a certain hesitation on their perceived capabilities to deliver feedback. In addition, it was noted that employees perceived improvement in the feedback given by their managers.

The next section describes the full-scale deployment of the mobile app with a partner organization.