Introduction

We are very pleased to have the opportunity to review our paper “Implementation of Motivational Tactics in Tutoring Systems” some 20 years after its initial publication (del Soldato and du Boulay 1995). This paper arose from the PhD work of the second author of the current paper (del Soldato 1994), and the first author is proud to have been able to assist in the development of the ideas. The work involved the design, development and evaluation of a system called MORE (Motivational Reactive Plan) to teach Prolog debugging. The focus of the design of the system was to express its pedagogical strategy and tactics, particularly those relating to motivation, in explicit representations that could, in principle, be adjusted in the light of experience or developments in motivational theory.

Motivation

The importance of the learner’s motivation to learn has long been acknowledged as a crucial factor in education, not just in terms of how it affects how much is learned, but also in terms of the quality of that learning and the consequences for future learning opportunities. Motivation can be understood in terms of the learner’s willingness to engage in the learning in the first place, to set appropriate goals for it, to devote sufficient practical, cognitive and emotional resources to it, and to persevere when obstacles and disappointments occur. A major role for teachers is to ignite the motivation of their students in the first place, especially if they are unmotivated, and re-motivate them if they become demotivated. For example, as Lepper and his colleagues report, expert human teachers include among their goals “first, to sustain and enhance their students’ motivation and interest in learning …… and second, to maintain their pupils’ feelings of self-esteem and self-efficacy, even in the face of difficult or impossible problems” (Lepper et al. 1990, Page 219).

Within the tradition of Artificial Intelligence in Education much of the work on motivation in 1990, when del Soldato’s PhD work started, had been concerned with macro-adaptive adjustments to learning systems, though the work of Keller was a notable exception. His ARCS model of motivational tactics was based on a psychological model of motivation from which he derived a number of ways that the learner’s motivation might be managed and improved in a dynamic (micro-adaptive) way (Keller 1979, 1983, 1987). Keller’s work took account of the fact that the learner’s degree of motivation would fluctuate while learning depending on how the learning process progressed. For example, failing to solve several problems in succession might well decrease motivation for some learners, while successfully solving a problem would normally increase motivation for all learners, unless boredom had set in because the problems had become too easy as a result of (over)-practice. It ought to be possible to track these minute-by-minute changes and, where there was a deterioration in motivational state, to try to repair it by adjusting some aspect of the learning conditions, such as making the problems easier, or more interesting for some students, or more challenging for others. However it was not until 2001 that a dynamically adaptive system directly based on ARCS appeared (Song and Keller 2001). This system was designed to teach genetics.

So the motivation for our work on motivation was to develop and evaluate a prototype system that could respond to short-term changes in the learner’s motivation over the course of a single lesson. This involved three sub-problems that can be expressed in the following questions:

  • What is motivation: establishing a way of characterising learners’ motivation?

  • How can motivation be measured: establishing a way of detecting a particular learner’s motivation?

  • How can motivation be changed in principle: finding a pedagogic theory that enumerates how specific pedagogic tactics and learner behaviours affect the motivational states of learners?

Approach

The approach taken was to design a teaching system, called MORE, for problem solving that incorporated two kinds of pedagogic reasoning. The first, a domain planner, was concerned with helping the learner master the domain itself. It based its decisions on what to do next in terms of offering the learner problems that traversed the domain, respecting conceptual prerequisite links and the individual difficulties of each available problem. In this respect MORE broke no new ground. The other pedagogical reasoner, a production system, considered the motivational state of the learner. Its rules were designed to maintain the learner’s motivational state if satisfactory, or otherwise improve it, see Table 1.

Table 1 Motivational strategy production system, adapted from del Soldato and du Boulay (1995)
Full size table

Sometimes the two planners would agree exactly about what was the next best thing for the system to do, e.g., offer some help when the learner seemed to need it. Sometimes the two planners were in broad agreement but differed in detail, e.g., that help was needed and should be given, or that help was needed but should simply be offered. And sometimes the two planners disagreed about what to do next, e.g., offering a harder problem than the last vs offering a problem of the same difficulty as the last. In order to cope with these disparities, a third mechanism (a set of rules) was implemented which arbitrated about how to deal with such disparities.

An important underlying feature of the approach was to separate out issues of how to traverse the domain from issues about how to manage the learner’s motivation. Plenty of systems had implicitly designed-in ways of maintaining the learner’s motivation (e.g., by presenting the material in an interesting way), but the goal for us was to make the micro-adaptive motivational aspects of the pedagogy explicit within the system. Of course, we had to choose particular rules for this when designing and implementing the system, but the important issue was not whether these were the best rules to choose, but the fact that the rules were explicit and could be adjusted in the light of new theoretical insights or empirical evidence of what worked and what did not.

What is Motivation?

We took a pragmatic and indirect approach to understanding motivation by characterising it in terms of three factors that are associated with it, namely effort, confidence and independence. The idea was that students who were well motivated would put effort into their learning, would have confidence in their ability to learn the material, and would seek to be able to succeed independently if at all possible. By contrast unmotivated or demotivated learners would typically not exert themselves, or would not believe that they could succeed, or would feel that they could not succeed independently from the teacher, or were bored because the material was too easy and they needed more challenge, or indeed might mix these factors in various combinations.

Note that we were interested in short time-scale changes over the course of a single lesson, so “confidence” referred to the learner’s degree of confidence in tackling the next task, rather some more stable notion of self-efficacy for that kind of domain, though clearly the two are linked. Likewise independence and effort were monitored continuously rather than only being measured at the start of the lesson and those values being used once and for all to adjust the pedagogy.

How Can Motivation Be Measured? Establishing a Way of Detecting a Particular Learner’s Motivation

Effort could be measured relatively easily in terms of the learner’s perseverance, particularly when faced with difficulties and obstacles. Confidence and independence were measured by the amount of help that the student requested and by asking the learner to choose from a menu of possibilities at various points in the interaction. For example, on first being shown a new problem, the learner would indicate, prior to attempting it, the degree of difficulty it seemed to pose. Independence was estimated additionally in terms of the system’s behaviour towards the learner. Offering or indeed providing help were assumed, in principle, to decrease the learner’s independence.

Each of the three measures, effort, confidence and independence, were conceived as linear scales and the learner’s motivational state was characterised as a triple made up of the three current values on those scales. Each scale for a learner was seeded with an initial value (halfway along the scale) and thereafter each of the learner’s and system’s activities would increment or decrement the values on one or more of the three scales. Each scale had a threshold value that divided it into two subscales. Above that threshold, the system regarded matters as satisfactory, below that threshold the system would switch on tactics to attempt to raise the value. So the model was essentially a motivational thermostat. Thus if confidence dropped below threshold, the system attempted to raise confidence, and the same for the other two measures.

Some system actions such as providing help might have the effect of incrementing on one scale, say effort, following the consequential success of the learner, but decrementing on the independence scale because help had been offered. So there were various interactions between the scales.

How Can Motivation Be Changed in Principle?

We scanned the pedagogic literature of the time to find theories and empirical evidence about how learner behaviours and learner motivational states were affected by system behaviours and by outcomes (see e.g., Keller 1979; Wood et al. 1976). As already mentioned, while we hoped that we had chosen wisely from the literature, this was not the main issue. What we wanted was a system in which it was possible to express these issues at all, as opposed to getting the rules right.

The main methods for helping the learner maintain motivation were by adjusting the difficulty of the problems posed and adjusting feedback and access to help in such a way as build up confidence while respecting independence and praising effort, see Table 1. The rules in the table were ordered and the first one to match the student model/history was fired. The student model and history included items relating to individual aspects of the learner’s motivation (e.g., “effort-valu < medium”), to the learner’s actions (e.g., help rejected: “help-state = rejected”) and to the success or otherwise with the problem in hand (e.g., “problem-state = failed”). The rightmost column, “Possible Action” gives a few examples of actions that the system might take, following a reconciliation of suggested actions between the domain planner and the motivational planner.

Core Contributions

The core contribution of the work was to argue for the importance of motivational issues in an AIED context by showing that it was possible to build a system that reasoned about the dynamic motivational state of a learner and adjust its behavior accordingly. Since then the search for ways to measure and react effectively to the learner’s motivational state has become hugely popular within the AIED community, with many papers on different aspects of these issues appearing in this Journal and in the AIED and ITS conferences.

We conducted only a brief formative evaluation of the approach embodied in MORE and gathered comments from a small number of participants about their experiences with it. This showed that the system did indeed adjust its behavior, but that some of the rules were not appreciated: such as when the system refused to give help when requested if it thought that the learner did not need help, i.e., when confidence was OK but independence was low. While this is certainly a tactic employed by human teachers, having a system do it raised issues about the social rules of felicitous engagement between a learner and a system.

Practical Impact

The work had limited practical impact as the system, MORE, was not developed and maintained, as is so often the case with PhD work. Our hope that the motivational reasoning component might be integrated within a large-scale domain-based tutor was never realized. This meant that we were not able to undertake an evaluation either of the potential educational benefits of adding in the extra complexity required by motivational reasoning or of the efficacy of the particular rules we had chosen.

While the work on MORE was not followed up directly, the first author has continued to work on motivational issues himself and with various PhD students and colleagues until the present day. This work included the development of systems that included a motivational component (du Boulay 2011a; Hull and du Boulay 2015; Rebolledo-Mendez et al. 2005, 2011, 2013; Uresti and du Boulay 2004), as well as theoretical work attempting to clarify the nature of motivation (Avramides and du Boulay 2009; du Boulay 2011a, b; du Boulay et al. 2007, 2008, 2010).

Core Limitations

There were three core limitations in the work. First was that neither MORE’s general approach of separating out the motivational reasoning from the domain focused pedagogy, nor its particular choice of rules to express the motivational reasoning were fully evaluated. Second, the notion of motivation as characterised by effort, confidence and independence was a huge simplification. These days one would want to bring into the frame academic emotions (see e.g., Pekrun 2006), academic engagement (see e.g., Christenson et al. 2012) and goal setting as well as the relationship of the learner’s values, prior educational experiences and beliefs about their own learning in framing how motivation “works” (see e.g., Schunk et al. 2008). Re-reading our earlier paper now, we see that there is some ambiguity between goals and tactics and examples of actions in support of tactics in the planning process as expressed in Table 1. Rather than fix these in Table 1, we have left them in place. The ambiguity arises from using the same term to express both a pedagogical goal as well as a tactic used to attempt to achieve that goal. We have also slightly simplified Table 1 from the original paper to remove what look like negative goals, but in fact were temporary suspensions of what would otherwise be overriding goals, such as increase confidence, in certain circumstances, such as when the overriding need was to stimulate curiosity. Finally, an aspect of the above is that in MORE the rules for incrementing and decrementing the values of the learner’s degree of effort, confidence and independence were the same for all learners. One would expect that different personality types as well as learner’s with different histories might well react differently (see e.g., Robison et al. 2010). Also the initial values of these variables were set rather arbitrarily at the halfway point along the scale. This could be improved using data from either past performance or a pre-lesson questionnaire.

Progress Since 1995

Much work in recent years within the field of AIED has been devoted to reacting to learners’ motivational and emotional states detected via sensors, machine vision and human observation (Arroyo et al. 2009; Beck 2005; Burleson 2006; Conati and Maclaren 2009; D’Mello and Graesser 2010; D’Mello et al. 2009). While much has been achieved since 1995, there is as yet no clear consensus as to how best to model a learner’s motivational state, with most work concentrating on the affective aspect of motivation via a limited set of emotions, such as boredom, engagement, confusion, and frustration associated with learning (D’Mello et al. 2008). Academic emotions (Pekrun 2011) that incorporate some notion of the value that the learner ascribes to the learning process or its outcome, such as pride, satisfaction, shame, disappointment and hope have as yet played a small role in AIED systems. Our own work has tried to explore the importance taking the learner’s values into account, see below (Avramides and du Boulay 2009)..

Within educational theory there have been corresponding advances in our understanding of motivation. These range from a more refined articulation of the role that academic achievement emotions play in learning (Pekrun 2011) to the integrative work of Pintrich on bringing together different types of motivational theory, such as expectancy-value theory, achievement goals and self-efficacy theory (Pintrich 2003). More recent work on learner goal setting (Hulleman et al. 2008) and on developing meta-motivational skills in general (Maehr 2012) as well as specific skills such as resilience in particular (Yeager and Dweck 2012) point to the importance of helping the learner to focus on learning processes as well as outcomes. Yeager and Dweck’s work underlines the value of helping to refine learners’ theories, often incorrect, about the nature of their own learning and the interactions between effort, perseverance, ability and outcomes as well as how they should more reasonably interpret success and failures along the way (Dweck 2002).

However the core AIED issue that still needs much work is mapping out a motivational pedagogy more precisely: in other words, developing strategies and tactics for intelligent tutoring systems and intelligent learning environments operating at a sufficiently fine level of granularity to be implementable. This would apply to ‘average’ students as well as taking individual differences of various kinds into account. There have been some successes, see for example (Boekaerts 2007; D’Mello et al. 2011; Porayska-Pomsta et al. 2008) but much needs to be done.

In summary, even if we could characterize a particular learner’s motivational state at a particular instant with high accuracy, the question still remains as to how best the system should behave in order to improve that state, both in terms of short term benefits for that lesson, as well as in terms of any longer term benefits towards “Encouraging a Continuing Personal Development in Learning” (Maehr 2012). In this regard it is interesting to compare the similarity of the effort-based strategy developed by Arroyo and her colleagues (2014), see Table 2, with that developed in MORE, see Table 1. Table 1 and Table 2 are similar, particularly in terms of including feedback based on the effort expended by the learner, and also in terms of adjusting problem difficulty according to performance. Of course both tables assume that the learner is broadly motivated at the start to study the material of the lesson, though may temporarily suffer from reduced motivation during the lesson. Dealing with learners who are unmotivated or blasé about the work from the beginning needs a wider range of tactics (du Boulay 2011b).

Table 2 Strategy in Wayang Outpost, adapted from Arroyo et al. (2014)
Full size table

It is normal now to see systems displaying a pedagogical agent whose comments, facial expressions, and demeanour are designed to keep the student working despite setbacks. There is also increasing interest in spotting and reacting to students who are disengaging (Forbes-Riley and Litman 2013), or gaming the system (Baker et al. 2008) or betraying evidence of unconstructive negative feelings such as boredom that are likely to badly impact motivation (Baker et al. 2010).

In MORE we took it for granted that our learners, in some sense, wanted to learn (how to debug Prolog programs), but might get discouraged or bored during the process. However we now acknowledge more clearly that motivation is crucially affected by the learner’s value system, as well as by their expected and actual experience and feelings prior to, during and following learning (Pintrich 2003). For a motivational strategy to have a real chance of success learners have to personally value what may be gained by their effort and fear what might be lost by their lack of it. Thus if the intrinsic and extrinsic rewards of the experience and the outcome of the learning effort are not attractive (or not believed) or the losses for lack of effort are of little consequence (or not believed), such a strategy is unlikely to have much leverage.

As a consequence some human motivational strategies are not simply straightforward appeals to what is already valued or expected by the learner but are also attempts to change the value system of the learner; or change their expectations of what the learning process or its outcome will in fact be like (see the tactic “Deemphasize importance of immediate success” in Table 2; or, possibly, even change their preferences for certain kinds of experience or outcome). For example, unsurprisingly many teachers value the act of understanding and learning for its own sake, unlike some of their students, and so attempt to inculcate such an ethos accordingly.

Following Pintrich (2003) we now divide the space of work on motivation into three: values, expectancies and feelings. In this paper we indicate our more recent thinking by dividing the space of motivational strategies that address these three areas also into three broad areas: appeals to the positive, appeals to the fear of the negative, and attempts to change some aspect of the learner’s assessment of the situation, see Table 3. While much of the work on motivation, include MORE, has concentrated on reacting to the learners’ actions, it is also important to frame the learning activity at the start (see e.g., Rosiek 2003), not least to help the learner set appropriate and realistic goals. So Table 3 contains examples of the kinds of statement that might form part of a motivational strategy in advance of action by the learner. A similar table can be formulated to characterise the different kinds of feedback after action that might be offered in different situations as in MORE. Each cell in the table is intended to illustrate the broad focus of a motivational tactic rather than showing the actual words used by the tutor, or indeed some non-verbal reaction by the tutor such as switching topic, offering a more interesting example or changing the nature of the learning task.

Table 3 Motivation tactics in advance of learner action in nine sub-spaces
Full size table

Ideally an intelligent motivational strategy would choose which kind of tactic was most likely to yield a desired result. That is, when faced with an unmotivated student, it would need to ascertain the specific causes of the lack of motivation, since different causes ideally need different approaches (Baker et al. 2008; du Boulay 2011b). In terms of MORE, this would require modelling how the learner reacted to tactics drawn from the different cells, not least as one needs to distinguish the first use of a tactic from its n’th use.

The nine cells have the following intentions:

  1. 1.

    Urging effort so that the learner achieves a valued goal. Of course this implies a discussion of the relative value of different learning goals and of the choice of goal, see Tactic 3.

  2. 2.

    Urging effort so that the learner avoids an unwelcome outcome.

  3. 3.

    Seeking to change the learner’s value system so as to make a goal become valued, e.g., so that Tactics 1 and 2 may be applied in the future.

  4. 4.

    Urging effort in order to improve the learner’s capability and make his or her self-efficacy judgements more accurate.

  5. 5.

    Urging effort in order to help the learner potentially dispel the fear that he or she may be incapable in this area.

  6. 6.

    Seeking to change the nature of the learner’s beliefs about learning, e.g., so that Tactics 4 and 5 may be applied in the future.

  7. 7.

    Urging effort so that the learner has learning experiences that produce pleasant feelings.

  8. 8.

    Urging effort so that the learner avoids learning experiences that produce unpleasant affective outcomes.

  9. 9.

    Seeking to change the learner’s understanding about the affective aspects of learning, e.g., so that Tactics 7 & 8 may be applied in the future and negative feelings not so much avoided as understood and mastered.

Future Needs

The design of future Intelligent Tutoring systems and Intelligent Learning Environments could be improved in several ways. One might be to pay greater attention to the role of values in motivation as indicated above. Another might be to develop modelling to include meta-motivational issues.

An initial line of development might include paying greater attention to the learner’s values, see Table 3. While it is possible in many circumstances to estimate what a learner’s values might be and to act on that basis, we would need greater developments in natural language understanding to conduct an interactive conversation with a learner to establish their actual values. However there are still immediate possibilities that can be achieved:

  • Helping teachers better understand how and why a system embedded within a normal classroom might assist the learners develop their investment in learning so that the work with the system is appropriately introduced and followed up (see for example, Koedinger et al. 1997).

  • Getting the system to prompt discussion between the learner and others—learners or teachers, or indeed via internal reflection—on the purpose and likely consequences of the interaction with the system. It does not need to observe and understand this discussion; it only needs to provoke it (see for example, Avramides and du Boulay 2009).

  • Engaging the learner in a ‘conversation’ about purpose and outcomes and process, perhaps using menus and keywords (see for example, Puntambekar and du Boulay 1999). Such a ‘conversation’ would naturally occur at the start and at the end of an educational interaction much as has already been implemented for helping learners develop metacognitive awareness (see for example, Gama 2004). In the longer term it may be able to take learners’ cultures into account as this will have an intimate effect on their values (see for example, Benford et al. 2012; Rosiek 2003).

Meta-motivational strategies are aimed to increase the learner’s insight into the motivational aspects of their own learning: for example, the kinds of tactics that they might apply to motivate themselves when not in the mood for learning; or their understanding of their own reactions to success and failure, praise and criticism in learning and the methods that work for them in dealing with these events. Maehr argues that motivation is a process that the learner can develop and deploy in new learning contexts, given appropriate prior educational experiences that have focused on “Encouraging a continuing personal investment in learning” (Maehr 2012). In this view the main aim of formal education is not just learning stuff — knowledge and skills — but learning how to learn. And this means not just learning how to find out, how to systematise and collate, how to self-question, how to problem-solve and so on, but also how, in a new context, to develop a sense of “direction, persistence and continuing motivation”: these three being evidenced by the choices and preferences that the learner exhibits, by the length of time devoted to tasks and the kind of tasks accomplished as well as by “later and ‘spontaneous’, return to [the] task/activity” (Maehr 2012, Page 3).

The work of Yeager and Dweck (2012) mentioned earlier is a good example of the positive outcome of training a particular meta-motivational skill, namely resilience. They show that “students who believe (or are taught) that intellectual abilities are qualities that can be developed (as opposed to qualities that are fixed) tend to show higher achievement across challenging school transitions and greater course completion rates in challenging math courses [my emphasis].”

The aim of Intelligent Tutoring systems and Intelligent Learning Environments has largely been that their learners learn more ‘stuff’, or learn it faster or deeper than by other means — taking the skill of an expert human tutor as their yardstick. The work on MORE and subsequent motivational systems has continued to subscribe to that aim by attempting to ensure that the learner exerts effort in a productive way. Maehr argues that schools, and by implication expert human tutors, need to go beyond this and the same arguments should also apply to Intelligent Tutoring systems and Intelligent Learning Environments. Thus a long-term aim for motivationally intelligent systems should be to assist learners to develop their learning skills per se, and their insight into them.