Motivation

The prime motivation for this work was involving the learner in diagnosis by allowing him/her to be an active participant in the construction of the learner model (LM) that an intelligent learning system builds. This was underpinned by both computational and pedagogical rationale. Involving the learner in diagnosis can address key learner modelling problems, such as accuracy (ensuring that the learner model is a correct representation of the learner) and dynamicity (maintaining the learner model to capture changes in the learner’s cognitive state). From a pedagogical perspective, involving a learner in diagnosis can promote meta-cognitive processes such as self-awareness and reflection, which in turn can motivate learning.

The research stream dealing with involving the learner in diagnosis is known as Open Learner Modelling. Back around 1990–2000, this research stream was gaining a momentum based on several key developments. John Self had argued that it would be worthwhile to involve learners in diagnosis since this could accommodate the dynamics of the student’s behaviour and make student modelling more tractable (Self 1990). He pointed out that the diagnosis should be made interactive and collaborative, which required that the learner model be made accessible to the learners with the aim of placing the system and the learner in a collaborative role. This approach, naturally, goes beyond merely showing the model to the student. Cumming and Self (1991) suggested a “switch from ITS-as-instructor” to “ITS-as-collaborator” and argued that learner modelling should be a shared activity with a learner model even being open to inspection and change by the learner. Paiva and Self (1995) had developed TAGUS, a workbench in which it was possible to externalise the learner model to the learner. This was seen as providing the means to explore ways of motivating reflection about the learner’s problem solving strategies. Bull et al. (1995) described a system, Mr. Collins, which allowed students to view their student model and even change it. Kay and her colleagues (e.g. Kay 1995) were exploring scrutable student models with fairly rudimentary facilities for the student to edit the model. In parallel, skillometers were developed for the Cognitive Tutors allowing the learners to know how close they were to finishing a unit (Corbett and Anderson 1994) - a development that has often been used in open learning modelling systems.

Artificial intelligence in education research was showing strong interest in open learner modelling, and it was becoming apparent that existing approaches would not fully realise its potential for both reducing the difficulty of student diagnosing and promoting meta-cognitive and reflective behaviours in pursuit of pedagogical goals. Driven by this gap, our research proposed to investigate the complex issues surrounding the interaction of the learner with the system’s model of themselves. The idea was very simple – it was based on the premise that people, willing to be understood, would engage in an interaction to make sure the other party (the learning system in this case) would get a better understanding of them. Consequently, an interactive dialogue between the system and the learner was structured with the learner model being the focal point for such dialogue. The approach, which was called Interactive Open Learner Modelling, had a twofold aim: on the one hand, it would allow the learner to inspect and change their learner model in a more intuitive and engaging way; and on the other hand, it would promote learner reflection and awareness of their knowledge.

STyLE-OLM: Interactive Open Learner Modelling

The work developed a computational framework for interactive open learner modelling (Dimitrova 2001), embodied in the STyLE-OLM (Scientific Terminology Learning Environment - Open Learner Model) demonstrator, which illustrate the implementation of the approach and allowed examining its computational and pedagogical benefits (Fig. 1).

Fig. 1
figure 1

Interactive Open Learner Modelling Overview. STyLE-OLM was innovative in incorporating an interaction model (Fig. 1) for managing the ongoing interaction between a learner and a “diagnoser” (the learning system) in a manner that provided a common communication means and symmetrical power in jointly maintaining the learner model

Full size image

Interactive Open Learner Modelling required tackling several research challenges:

  • Challenge 1. How to design an appropriate communication means which allows both inspection of the learner model and engagement in a dialogue about it?

  • Challenge 2. How to plan an interactive open learner modelling dialogue which gives the learner and the system a symmetric power to negotiate and construct the learner model?

  • Challenge 3. How to accommodate different beliefs about the learner’s domain knowledge in a jointly maintained learner model?

  • Challenge 4. How to assess the hypotheses that interactive open learner modelling can both lead to improved LM accuracy and support learner reflection?

The above challenges were addressed by relying on existing research in dialogue systems, external representations and communication languages, and logical representation and reasoning.

Addressing Challenge 1

We adopted a knowledge-centric approach whereby the focus was on modelling a learner’s domain knowledge, and utilised Sowa’s Conceptual Graphs theory (1994) as the logical underpinning for the representation and maintenance of the learner model. Adopting Conceptual Graphs brought crucial advantages. On the one hand, we could exploit a robust reasoning mechanism which allowed checking the validity of domain statements and inferring domain facts as domain model entailments (used in the dialogue planning – see dialogue example below). On the other hand, there was already a diagrammatic language for presenting knowledge statements formulated as Conceptual Graphs (Sowa 1994) which allowed computationally tractable way to express domain statements. Consequently, we exploited the diagrammatic external representation of Conceptual Graphs for designing a communication medium which provided a structured interface combining a graphical construction area for propositions representing beliefs from the LM. In addition to the domain statements, the illocutionary acts were expressed by utilising sentence openers. The STyLE-OLM communication means is illustrated in Fig. 2.

Fig. 2
figure 2

STyLE-OLM communication medium (Dimitrova 2003). The diagrammatic representation shows the domain statement, the illocutions are indicated as sentence openers corresponding to the dialogue moves selected by the buttons on the right

Full size image

Reflection on Addressing Challenge 1

The combination of graphically rendered statements with propositions about the LM and sentence openers for representing the illocutionary force of the dialogue utterances was a suitable way for structuring the communication medium. Specifically, the diagrammatic way allowed the learners to articulate their knowledge while the sentence openers provided a scaffold for the discourse structure of the interaction (Dimitrova 2003). The communication medium was easily replicable in other domains for declarative knowledge driven diagnostic interactions, e.g. the OWL-OLM system (Denaux et al. 2005) implemented the same communication medium based on a LM represented in the Web Ontology Language OWL (as illustrated in Fig. 3). The main drawback of the communication means was the rather limited way of constructing domain propositions with the graphical apparatus provided (see Figs. 2 and 3). However, using a controlled way for expressing domain statements enabled dealing with the complexity of natural language statements and provided an easy way for converting propositions into logical axioms. We extended the approach in follow-on work on controlled natural languages for capturing domain knowledge (Denaux et al. 2011, 2012). Another way to extend the communication means could be to explore various ontology visualisation languages (c.f. Katifori et al. 2007).

Fig. 3
figure 3

OWL-OLM communication medium (Denaux et al. 2005). It adopts the STyLE-OLM communication means, using UML-like diagrammatic expressions of domain statements and sentence templates for a pre-defined set of dialogue moves

Full size image

Addressing Challenge 2

Based on research in dialogue games (Levin & Moore 1977; Walton 1984, Pilkington et al. 1992), a dialogue framework for managing diagnostic interactions was defined. The dialogue framework itself combined both linguistic dialogue games to organise the dialogue episodes and logical dialogue games to enable the participants’ symmetrical involvement and to collect their commitments. The framework included two main components – a communication act analyser and a dialogue game planner. The dialogue-game based planner incorporated pre-defined strategies (e.g. explore domain knowledge, clarify confusion, provide explanation) which guided the dialogue episodes and provided macro-level planning. In addition, diagnostic tactics, which spanned multiple turns, allowed micro-level planning and maintaining the turn taking in a simple, yet flexible way. The dialogue framework and its implementation were presented in more detail in Dimitrova et al. (2001a). The original STyLE-OLM paper (Dimitrova 2003)Footnote 1 presents elaborated examples that show the STyLE-OLM framework in action. The examples there are taken from a user study with the system, illustrating interactions of a more knowledgeable learner and a less knowledgeable learner discussing their learner models with STyLE-OLM. We include here a short example with a fragment of interaction with STyLE-OLM where the system challenges the robustness of a learner’s conceptualisation, which triggers reflection leading to changes in the changes in the conceptual knowledge.

The dialogue extract below gives a sense for the kinds of arguments the system would entertain from students regarding their own knowledge. Initially, the learner model contains information that the learner knows some aspects of money market and capital market but often mixes these domain concepts. STyLE-OLM initiates a dialogue to probe the learner’s knowledge about these terms and to help learner identify possible erroneous beliefs leading to this confusion.

  • [d0]STyLE-OLM[inquire] DO YOU THINK: money market operates with short term investments .

  • [d1]Learner[agree] I AGREE THAT: money market operates with short term investments .

  • [d2]STyLE-OLM[inquire] WHAT IS: capital market raises capital and operates with what ?

  • [d3]Learner[inform] I THINK THAT: capital market operates with short term investments .

At the beginning, the system follows a confuse dialogue game, initiated by a particular situation (confusion of money market and capital market). The dialogue follows a game tactic that elaborates on the similarities and differences between the two concepts (as inferred by the conceptual graphs reasoning engine). Dialogue move [d3], however, reveals an erroneous belief in the learner’s conceptual model - the statement ‘C apital market operates with short term investments’, which the learner is assumed to believe, is not confirmed by the system’s domain knowledge (as inferred by the conceptual graphs reasoning engine). The strategy rules suspend the current dialogue game and initiate a new game aimed at finding explanations for the learner’s erroneous beliefs.

  • [d4]STyLE-OLM[challenge] WHY DO YOU THINK: capital market operates with short term investments ?

  • [d5]Learner[justify] IT IS TRUE BECAUSE: capital market operates with treasury bills and treasury bills are short term investments .

The system now registers a possible misconception, misattribution in this case, as the reason for believing wrongly that ‘C apitalmarket operates with short term investments’ might be that the learner believes that ‘C apital market operates with treasury bills and treasury bills are short term investments’. Since the fact in [d5] has not been confirmed by the domain knowledge base, the dialogue strategies initiate a new game for explaining this error.

  • [d6]STyLE-OLM[challenge] WHY DO YOU THINK: capital market operates with treasury bills and treasury bills are short term investments ?

At this time, the learner challenges the robustness of his domain beliefs, and inspects the learner model (the beliefs in the learner model are rendered as conceptual graphs, using the same diagrammatic representation, and indicated as correct or wrong depending on whether the system’s domain model supports these beliefs). The learner can see that the belief ‘C apital market operates with treasury bills and treasury bills are short term investments’ is indicated as wrong. He can initiate a dialogue with the system to check this statement.

  • [d7]Learner[inquire] IS IT TRUE THAT: treasury bills are short term investments ?

In order to generate a following system move, the dialogue module checks the dialogue game strategies. In this case, the strategies indicate that the current tactic should be disregarded because when the learner asks a question, the highest priority is given to the tactic to answer this question by checking in the domain knowledge base.

  • [d8]STyLE-OLM[inform] I KNOW THAT: treasury bills are short term investments .

The learner realises that this part of his statement in [d5] is correct and checks the other part.

  • [d9]Learner[inquire] IS IT TRUE THAT: capital market operates with treasury bills ?

  • [d10]STyLE-OLM[inform] I DO NOT KNOW THAT: capital market operates with treasury bills .

The learner has clarified the wrong part of his beliefs and withdraws his claim that ‘C apitalmarket operates with short term investments’.

  • [d11]Learner[disagree] I DISAGREE THAT: capital market operates with short term investments .

The belief ‘C apitalmarket operates with short term investments’ is now deleted from the learner’s commitment store. Since the learner uses a [disagree] move, which shows a strong commitment that the fact is not true, the corresponding claim is deleted from the system’s commitment store too. The learner model is updated at the end of the interaction, and the potential misconception registered after move [d5] is not included as it has not been confirmed in the dialogue. After move [d11], the system will terminate the confuse dialogue game (initiated at the start of the dialogue) as the misconception has been clarified. The dialogue will continue with further comparison between capital market and moneymarket, see examples of such dialogue episodes presented in (Dimitrova 2003).

Reflection on Addressing Challenge 2

The integrative dialogue game based approach including both linguistic discourse management and logical negotiation planning was a feasible way of handling learner modelling interactions. This allowed dealing with accuracy and dynamicity through structured negotiation of the learner model, as well as promoting learners’ reflection on the domain being studied. This flexible model was easily extendable by adding further dialogue games which enabled addressing the main dialogue deficiencies observed during the STyLE-OLM evaluation (Dimitrova 2003). including broken dialogue coherence which gave the sense that the dialogue was a series of unconnected episodes, lack of control in promoting reflection, and inability to handle limitations and noise in the system’s domain knowledge. For example, OWL-OLM adopted the STyLE-OLM dialogue framework for interactive open learner modelling and further extended this with meta-games that provided explanations when dialogue episodes started or ended – this improved the dialogue coherence (Denaux et al. 2005). Furthermore, the STyLE-OLM dialogue planner was extended in the dialogue agent Perico (Denaux et al. 2014) to deal with uncertainty in the system’s statements and possible noise in the system’s domain knowledge. Although the evidence gathered of reflective activities (learners rendering statements about their beliefs, revisiting claims about their beliefs, changing domain beliefs, and formulating arguments to justify domain beliefs) was encouraging, these activities were somehow a side effect of the interaction and not explicit goals of the dialogue episodes (see also the reflection on Challenge 4). The STyLE-OLM dialogue framework did not explicitly include dialogue games that aimed at promoting reflection. This was addressed in (Eamsinvattana 2011) where the STyLE-OLM dialogue mechanism was extended to embed dialogue games promoting learner’s reflection in a reflective dialogue for on-the-job training in emergency services.

Addressing Challenge 3

STyLE-OLM utilised a rigorous mechanism for maintaining different views about the learner’s beliefs accumulated throughout the interaction in a jointly constructed learner model. This mechanism adapted a belief modal operator to maintain a joint model of the domain knowledge of the learner incorporating belief statements made during the interaction between the computer system and the learner (Dimitrova et al. 2001b). Given that the reasoning of both interaction agents (i.e. the learner and the computer system) may not be complete and may not necessarily be sound, we developed a mechanism for finding agreements and conflicts between the system’s and the learner’s views about the learner’s knowledge. Agreements were indicated for those system’s beliefs about the learner’s domain knowledge that the learner accepted (i.e. was informed about the system’s belief and did not object). In contrast, when the learner objected the system’s beliefs or when the system did not agree with the learner’s statements, conflicts were detected. In these cases, clarification dialogue games were triggered to resolve the conflict. In the resultant learner model, the most recent statements from the clarification dialogue overrode past contradictory statements about the learner’s beliefs.

Reflection on Addressing Challenge 3

Adopting a formal approach adapted to human reasoning is in general beneficial for practical implementations that are independent from the specific domain and application contexts. This allowed us to adopt the approach in two independent domains instantiations of STyLE-OLM – for learning key concepts (i) from programming and (ii) from financial markets. Furthermore, the mechanism was implemented in the interactive open learner modelling system OWL-OLM (Denaux et al. 2005). and was adapted for a schema theory-based user-adapted dialogue to support children’s conceptual understanding (Ibrahim 2006). We showed with an illustrative example that the formal approach could allow managing jointly constructed knowledge in dyadic interactions. However, this was not implemented/validated in a computer system. It would indeed be interesting to further examine how the formal mechanism for modelling agreements and conflicts between the beliefs of two agents could be extended to manage jointly constructed knowledge (e.g. in group discussions).

Addressing Challenge 4

The last challenge is crucial part of any research in Artificial Intelligence in Education. To validate the usefulness of the computational framework, it was implemented and evaluated in a user study as part of the EU project LARFLAST. The chosen domain was knowledge of finance concepts (e.g. stocks, shares, bonds). The study was conducted with university students whose knowledge of finance concepts varied. STyLE-OLM was invoked after an initial learner model was obtained. The resultant learner model was shown to the learners who were asked to confirm or reject the beliefs in the LM. The interaction scripts and the resultant LM were also examined by a finance terminology teacher. The study showed that, compared to the initial learner models (before the learners engaged in interactive open leaner modelling dialogue), the resultant learner models included more beliefs about the learner, and most of these beliefs were approved by both the learners and the teacher. The presence of reflection was assessed by adopting discourse analysis – the interaction transcripts were examined looking for evidence of reflection, in terms of: (a) the learner renders statements about their beliefs; (b) the learner changes their beliefs as result of the interaction; (c) the learned develops an argument to justify their belief. The study showed that STyLE-OLM would be beneficial both for more knowledgeable and less knowledgeable learners - the former were engaged in reflective interactions about the domain, while the later were provoked to inspect their models and challenge the robustness of these models.

Reflection on Addressing Challenge 4

Finding evidence about the pedagogical benefit of interactive open learner modelling, and open leaner modelling in general, is challenging. Most of the evidence is collected via carefully designed user studies, usually within university conditions focusing on selected aspects. The evaluation of STyLE-OLM, and some of its extensions, e.g. Onto-AIMS (Denaux et al. 2005) and PORML (Eamsinvattana 2011 Footnote 2), shows a typical example of this approach. Such studies inherit key limitations with regard to broader applicability of the findings in other domains and contexts. The generic framework developed by Bull and Kay (2007) provided a systematic way to analyse and describe open learner models, as well as to compare the features of open learner models in different systems. This was followed by a series of studies comparing open learner models across domains and applications (Bull et al. 2010). as well as investigating features that may influence the learners’ decisions to use an independent open learner model (Bull 2010). These studies promoted the uptake of open learner models in a broad range of learning environments. Nevertheless, the experimental pedagogical validation, especially the effect of meta-cognition, is a key challenge that deserves further investigation. Crucially, further work is needed to develop comparative evaluations across domains and learner contexts (differences in age, experience, cultural background). This should also seek evidence that these reflective moments in interactions with the system correlate with learning outcomes or other benefits from the instruction.

STyLE-OLM Adoption and Extension

The main contribution of this work is the holistic framework for interactive open learner modelling which addresses the three main issues – providing a suitable communication means, managing a diagnostic dialogue, and maintaining a jointly constructed learner model. The framework was deployed in STyLE-OLM instantiations in two domains (programming and finance), and was used as the learner modelling component in the intelligent learning environment for foreign language terminology learning developed by the EU project LARFLAST.

The STyLE-OLM framework provided a formal description of the main components for interactive open learner modelling which enabled its easy adoption in other applications. Future work of the authors in collaboration with their research students and colleagues adopted STyLE-OLM framework in several domains and applications.

  • The framework’s independence from the knowledge representation language allowed its fairly quick implementation in the learner modelling component - OWL-OLM - of the semantic web learning environment Onto-AIMS (Denaux et al. 2005). It was trialled in the domain of learning Linux concepts. This showed that the interactive open learner modelling approach was suitable for dealing with cold start in user-adaptive systems. Furthermore, reusability was facilitated by the implementation in the Web Ontology Language OWL, used as standard for semantic web applications.

  • The dialogue planner provided the backbone for implementing other interactive agents by extending the dialogue game set. Eamsinvattana expanded the dialogue mechanism to include reflective games as part of the user diagnosis interactions (Eamsinvattana 2011). The mechanism was implemented in a personalised mobile interactive agent (PORML) promoting reflection-on-action. The agent was trialled in the context of emergency services training focusing on fire risk assessment. The feedback from fire fighters indicated that the interaction with the dialogue agent could enhance the current training practice by facilitating fire risk assessment debrief. Furthermore, the dialogue provided effective means for capturing individual job experiences and collecting a repository of working practices.

  • While PORML adopted the STyLE-OLM dialogue mechanism for another user diagnosis context, the SAIC (Schema Activation and Interpersonal Communication) agent extended the dialogue mechanism to include the set of dialogue games to support cognitive tasks occurring when learning new concepts (Ibrahim 2006 Footnote 3). The Schema theory, which explains how meaning-making occurs and stresses the importance of prior knowledge, was used as the underpinning pedagogical framework. Consequently, dialogue games that diagnose a learner’s knowledge (as in STyLE-OLM) were combined with additional games for schema activation, tuning and modification. The SAIC agent was embedded within a multimedia educational system to support children’s conceptual understanding while reading text in an unfamiliar domain (astronomy in this case). A controlled user study in an after school reading club compared the learning effects of two versions of a multimedia learning system with and without the SAIC agent using schema activation pre- and post-test. The results showed that SAIC facilitated more improvements of students’ schematic knowledge in terms of developing more complete and more specific schemas of domain concepts (difference between control and experimental group was statistically significant, U-test). Despite this controlled study, we have not examined empirical evidence showing that an interactive open learner model leads to better learning, compared say to an open learner model that is not negotiable, or to not having an open learner model at all.

  • The STyLE-OLM framework was adopted in the EU project ImREAL to develop the interactive agent Perico that built a model of a learner’s knowledge of different cultures (Denaux et al. 2014). Differently from the dialogue approaches described above which utilised bespoke domain model knowledge, Perico exploited a pool of crowdsourced knowledge extracted from DBpediaFootnote 4 that provides structured access to information content from Wikipedia. Crowdsourced knowledge can provide an efficient way to deal with diversity of domain knowledge that drives user modelling dialogues. A series of evaluation studies with Perico (Denaux et al. 2014; Thakker et al. 2014) confirmed the feasibility of the approach for generating interactive questionnaires to obtain profiles of users for adaptation. This is specifically beneficial in domains where diverse content at a scale is required, like a variety of cultures and situations which somebody may have to be aware of.

Follow on Research

Research in user-adaptive learning environments developed further frameworks and extended the scope of open learner modelling. Work that cited STyLE-OLM is outlined below. In some cases any influence of STyLE-OLM is indirect at best. It shows similar research with which STyLE-OLM (and the extensions discussed above) share similarity, mainly at conceptual level and ideas development.

Developing Enhanced Methods for Interaction and LM Inspection

The importance of interaction when inspecting and changing the LM was embraced in a series of projects. Van Labeke et al. (2007) developed the xOLM – the extended Open Learner Model which permitted learners to interact with it. The underlying mechanism for maintaining a model of the learner accumulated evidence of the learner’s state in relation to knowledge of the domain being learned and assessment of the learner’s competence. xOLM also included motivational and affective factors, as well as metacognitive information and knowledge of conceptual and performance errors. Zapata-Rivera et al. (2007) developed an evidence-based approach to interacting with open student models. This extended the interaction means utilising arguments and evidence-centred design, and following assessment-driven ways. Similarly, Perez-Marin et al. (2007) also developed an extension of the interaction to make the open learner modelling approach more suitable for use in assessment contexts. Their automatically generated inspectable learning models were based on conceptual models and aimed at providing formative assessment to the students so that they can get more practise and feedback before their exams. The conceptual model (defined as a set of interconnected concepts associated with an estimation value indicating how well these concepts were used by the students) shared similarities with the conceptual graphs based LM in STyLE-OLM. The interaction with the learner was further extended by Kerly et al. (2008) to enable natural language negotiation of the open LM, which was inspired by research in chatbots and their potential for extending the interactive means in learning applications. A conversational agent, or “chatbot” was developed to allow the learner to negotiate over the representations held about them using natural language. Lloyd and Bull (2006) took a different angle on the interaction comparing the use of textual and haptic means. They investigated whether learners can understand a haptic open learner model, and whether they find it useful. The findings of the study suggested the need for a variety of methods of accessing the model to be available. These approaches made important contribution to knowledge, beyond the work by STyLE-OLM, providing novel frameworks (extending the interaction means and the communication media) for engaging the learner in interactions around the LM to support metacognition and self-regulation.

Wide Adoption of Open Learner Models

The Artificial Intelligence in Education research community embraced the broad open learner modelling approach (interaction, addressed by STyLE-OLM and its extensions, is only one aspect of the broad open learner modelling approach). This interest was driven by the potential computational and pedagogical benefits - see for examples the IJAIED special issue on open learner modelling (Bull et al. 2007; Dimitrova et al. 2007). The wide adoption of the open leaner modelling approach in a broad range of domains and applications was facilitated by the generic open learner modelling framework developed by Bull and Kay (2007). as well as the various example systems. There was a series of open learner models in constraint-based tutors (Mitrovic et al. 2007). SQL-Tutor and KERMIT provided students with means to inspect a high-level abstraction of the detailed information contained in their student models (Mitrovic and Martin 2007). Mathews et al. (2012) embedded open student models in constraint-based tutors allowing students to see their own student model to facilitate the development of metacognitive strategies. Thomson and Mitrovic (2010) adopted negotiable student models in an established constraint-based tutoring system, and showed that the approach was both useful for learning and enjoyable to use. The suitability of open learner models in web-based testing and assessment was also illustrated by Lazarinis and Retalis (2007). Opening the learner profiles supported learner and tutor involvement in decision making creating a more credible learner model. Another adoption of open learner models in web-based assessment was explored by Pérez-Marín and Pascual-Nieto (2010) whereby automatic evaluation of free-text students’ answers was utilised for generating students’ conceptual models which were shown to both students and teachers in an e-assessment system. In a different context, Xu and Bull (2010) utilised open learner models to encourage second language speakers to recognise their language difficulties by comparing with proficient speakers. Kay led a research stream exploiting the use of pervasive and ubiquitous open learner models to realise the vision for a lifelong user model whereby the LM was seen as a first class citizen, existing independently of the specific application and controlled by the learner (Kay 2008). Ubiquitous open user models were also explored in Vernero et al. (2009) to allow users to understand how a user-adapted social recommender system makes adaptation decisions, enabling users to inspect and modify the values stored in their user model.

Open Learner Modelling Approaches That Utilise Semantic web Technologies

One of the main advantages of the STyLE-OLM framework was its reusability in a wider context, which was also enabled by utilising semantic web technologies in a follow on extension of the framework. Similar line of research was adopted by other work in semantic web and user modelling. Dolog and Schäfer (2005) developed a framework for browsing, manipulating and maintaining interoperable learner profiles which provided a common base for the exchange of learner profiles between several sources. Ontological technologies for user modelling were adopted by Sosnovsky and Dicheva (2010) to translate overlay student models for relative domains by means of ontology mapping. Furthermore, semantic web technologies were also linked to open user models. Bakalov et al. (2010) developed an interface exploiting introspective views for scrutinising semantic user models. Within this research stream, the closest to STyLE-OLM was the work by Cena and Aroyo (2007) who developed a computational framework to enable a semantics-based dialogue for interoperability of user-adaptive systems in an ubiquitous environment. They modelled the interactions between systems as a semantics-based dialogue for exchanging parts of the user model and the corresponding context data. With the growing popularity of semantic web and the adoption of semantic web technologies in educational contexts, the works mentioned above provided foundations for further research in semantic web, user modelling and user-adaptive intelligent learning environments.

Future Direction: Towards Intelligent Mentoring

The research on STyLE-OLM was driven by the need for designing and developing intelligent learning environments for adult learners. The STyLE-OLM framework and the follow on work outlined above showed the interactive open learning modelling potential to develop effective means to help learners understand themselves. While this is necessary for effective adult learning systems, it is not sufficient. Intelligent learning environments have to consider the way adults learn: self-directed, experienced-based, goal- and relevancy oriented (Hetzner et al. 2011). Furthermore, it is critical to focus on the 21st century skills demanded by the job market. The recent turn towards seeking to support learning in so-called ill-defined domains is crucial in adult learning (Lynch et al. 2009). Embarking on domains in which ill-definedness plays a part, such as interpersonal communication, decision making, problem solving, team management, cultural awareness, together with utilising approaches that support personal development, self-regulation and meta-cognition, opens the door to providing the kind of individual support suited for adult learners. We see this leading to a new breed of intelligent learning systems that provide mentor-like features.

Taking advantage of recent advances in learner modelling, natural language interactions, data/text mining, pervasive interfaces, and robust agent architectures, it becomes possible to design and develop a radically new kind of adaptive interactive tutoring agents which act as mentors to promote ‘self-actualisation’. That is, such an intelligent mentor seeks to help the learner: develop an understanding of his/her strengths and abilities; become able to see and interpret different perspectives and opportunities; make sense of personal experiences; develop strategies and set goals; and develop confidence and self-esteem. Intelligent mentoring is taken here to be the ideas and techniques used/developed by the Artificial Intelligence in Education community to provide mentoring for learners/trainees. Mentoring itself has numerous definitions (see Jacobi 1991) but, we suggest that for now it is sufficient to suggest that mentoring implies emotional and psychological support, direct assistance with career and professional development, and role modelling (Jacobi 1991).

We believe that the time is ripe to embark on the challenge to provide intelligent tools to support mentoring, and, in the long run, to automate and scale mentoring features. Our strong belief is that we will see more and more personalised learning assistants which exhibit coaching behaviour and are seamlessly embedded in existing virtual learning environments. The design and development of such agents will require tackling key challenges, such as: (i) understanding real world experience and building corresponding learner models; (ii) designing new interaction models that promote reflection and self-awareness, and take this further to self-actualisation (iii) identifying new ways to seamlessly embed mentoring-like functionality in traditional educational environments. Major advancements in user modelling and intelligent learning environments, have prepared the foundation for a move towards ‘intelligent mentoring’.

Crucial for intelligent mentors will be the ability to help learners connect their learning with the real world, which brings forth the key challenge of modelling real world experiences. This challenge was addressed in the recently completed EU project ImREALFootnote 5 by exploiting crowdwisdom approaches. It was shown that digital traces left in social media can offer a cost-effective, scalable and reusable way to sense the real world (Karanasios et al. 2013). Using semantic technologies, we can get deeper insights into social content and to understand different experiences and model diversity of viewpoints (Despotakis et al. 2013). This can provide a feasible way to capture and maintain a dynamic ‘domain knowledge’ model that is based on approaches for taming collective intelligence (Thakker et al. 2012).

Mentors who are to facilitate self-actualisation require a broad (yet shallow) understanding of the learner and the current situation in order to select appropriate pedagogical strategies and respond in a motivational, emotionally-aware way. Accordingly, intelligent mentors will need to develop a multi-faceted learner experience modelling mechanism to get sufficient understanding of the learner, his/her current situation, and relevance to past experiences by the same learner (or by other people). Recent research developed augmented user modelling approaches (Abel et al. 2012). i.e. enriching existing user models with additional information mined from other data sources not considered previously, is perceived as an approach to effectively help in aligning the learning experience within virtual environments with the real world context in which the learner engages with their day-to-day job. A major, if not key, advantage is that the user models become enriched with a range of aspects that cannot be captured from merely analysing the learner’s interaction with the learning application. The field is in an embryonic state with promising results shown in detecting user interests from twitter, modelling social contexts, modelling learner identities. There is a need for new approaches that exploit the synergy of linguistic and semantic analysis to sense various aspects of the learners.

There have been impressive achievements in the development of knowledge-centric dialogue planners, such as those adopted in BEETLE (Dzikovska et al. 2010). ATLAS (Rose et al. 2001). ANDES (VanLehn et al. 2005). CIRCSIM-Tutor (Khuwaja et al. 1994) and LeActiveMath (Callaway et al. 2006). which enable sophisticated dialogue solutions. Apart from the extensive effort needed to create and maintain up-to-date, reliable domain models, knowledge-centric dialogue planners are not flexible. Whilst highly effective in so-called well-structured domains (e.g. Science, Maths, Engineering), the traditional knowledge-centric dialogue models do not respond to the changing demands for moving to ill-defined domains that are often subject to different interpretations and perspectives. In such domains, open approaches that exploit text mining and shallow domain modelling appear appropriate, e.g. AUTOTUTOR (Graesser et al. 2004). Writing Pal (McNamara et al. 2012). WILLOW (Pérez-Marín and Pascual-Nieto 2010). Intelligent Mentoring systems are likely to need to adopt a shallow domain modelling approach where various aspects of the domain are sensed from different sources using existing semantic repositories and information extraction techniques from text corpora. We expect that further research will explore this notion and assess its practical utility.

The requirement – at least, initially – for a broad understanding of the learner is important: we hypothesis that for intelligent mentors ‘wide and shallow’ learner models would be preferred to ‘narrow and deep’. We believe that the ‘wide’ approach is likely to have significantly higher payoff when compared with the more cautious traditional approach of adding a single human tutor-like competence one at a time. Our hypothesis, of course, needs to be validated when exemplars of intelligent mentoring are implemented and evaluated. Furthermore, there is a need to have a sufficient spread of mentor-like skills with an adequate depth of development to provide the necessary flexibility in handling learning in a wide range of situations and various personal contexts. This would call for the adoption of appropriate pedagogical models to develop new conceptual frameworks promoting self-regulation, connecting to real-world experience, and allowing domain-independent skills to be deepened and broadened.