Intelligent Tutoring Systems

Students build cognitive models for solving a crypt-arithmetic task in a learning environment that enables them to formally describe various types of procedural knowledge in a group learning setting in which each student is allowed to refer to the procedural rules described by the other group members. Experimental evaluation showed that: (1) three-quarters of participants successfully constructed valid models with the system, and (2) participants learned to describe procedural knowledge more precisely not only for the training task (crypt-arithmetic task) but also for a transfer task (bug identification for a multi-column subtraction problem).

Learning from Problem Solving (PS), Worked Examples (WE) and Erroneous Examples (ErrEx) have all proven to be effective learning strategies. However, there is still no agreement on what kind of assistance (in terms of different learning activities) should be provided to students in Intelligent Tutoring Systems (ITSs) to optimize learning. A previous study [1] found that alternating worked examples and problem solving (AEP) was superior to using just one type of learning tasks. In this paper, we compare AEP to a new instructional strategy which, in addition to PS and WEs, additionally offers erroneous examples to students. The results indicate that erroneous examples prepare students better for problem solving in comparison to worked examples. Explaining and correcting erroneous examples also leads to improved debugging and problem-solving skills.

Questioning has been shown to improve learning outcomes, and automatic question generation can greatly facilitate the inclusion of questions in learning technologies such as intelligent tutoring systems. The majority of prior QG systems use parsing software and transformation algorithms to create questions. In contrast, the approach described here infuses natural language understanding (NLU) into the natural language generation (NLG) process by first analyzing the central semantic content of each independent clause in each sentence. Then question templates are matched to what the sentence is communicating in order to generate higher quality questions. This approach generated a higher percentage of acceptable questions than prior state-of-the-art systems.

Recent research on self-regulated learning (SRL) includes multi-channel data, such as eye-tracking, to measure the deployment of key cognitive and metacognitive SRL processes during learning with adaptive hypermedia systems. In this study we investigated how 147 college students’ proportional learning gains (PLGs), proportion of time spent on areas of interest (AOIs), and frequency of fixations on AOI-pairs, differed based on their prior knowledge of the overall science content, and of specific content related to sub-goals, as they learned with MetaTutor. Results indicated that students with low prior sub-goal knowledge had significantly higher PLGs, and spent a significantly larger proportion of time fixating on diagrams compared to students with high prior sub-goal knowledge. In addition, students with low prior knowledge had significantly higher frequencies of fixations on some AOI-pairs, compared to students with high prior knowledge. The results have implications for using eye-tracking (and other process data) to understand the behavioral patterns associated with underlying cognitive and metacognitive SRL processes and provide real-time adaptive instruction, to ensure effective learning.

Among other factors, student behavior during learning activities is affected by the pedagogical content they are interacting with. In this paper, we analyze this effect in the context of a problem-solving based online Physics course. We use a representation of the content in terms of its position, composition and visual layout to identify eight content types that correspond to problem solving sub-tasks. Canonical examples as well as a sequence model of these tasks are presented. Student behaviors, measured in terms of activity, help-requests, mistakes and time on task, are compared across each content type. Students request more help while working through complex computational tasks and make more mistakes on tasks that apply conceptual knowledge. We discuss how these findings can inform the design of pedagogical content and authoring tools.

Game-based practice within Intelligent Tutoring Systems (ITSs) can be optimized by examining how properties of practice activities influence learning outcomes and motivation. In the current study, we manipulated when game-based practice was available to students. All students (n = 149) first completed lesson videos in iSTART-2, an ITS focusing on reading comprehension strategies. They then practiced with iSTART-2 for two 2-hour sessions. Students’ first session was either in a game or nongame practice environment. In the second session, they either switched to the alternate environment or remained in the same environment. Students’ comprehension was tested at pretest and posttest, and motivational measures were collected. Overall, students’ comprehension increased from pretest to posttest. Effect sizes of the pretest to posttest gain suggested that switching from the game to nongame environment was least effective, while switching from a nongame to game environment or remaining in the game environment was more effective. However, these differences between the practice conditions were not statistically significant, either on comprehension or motivation measures, suggesting that for iSTART-2, the timing of game-based practice availability does not substantially impact students’ experience in the system.

In this paper, we report results of an evaluation study that investigate the applicability and usefulness of the formal models of the Socratic Method. Nelson suggested that the Socratic Method, which is employed in teaching consists of three phases: searching for examples, searching for attributes and generalizing the attributes. These formal models are intended to serve in a computerized learning environment where users can train with a chatbot to stimulate their critical thinking. This paper demonstrates the applicability and the usefulness of the formal models and shows its effectiveness in group discussion where the chatbot acts as a discussion leader who applies the Socratic Method. The contribution of this paper is two-fold. First, in the dialogue models, we integrated critical questions using the question taxonomy of Paul and Elder in the three phases of the Socratic Method. Second, the formalization of the three phases of the Socratic Method using state diagrams is a new innovation.

Intelligent tutoring systems are adapting the curriculum to the needs of the student. The integration of stealth assessments of student traits into tutoring systems, i.e. the automatic detection of student characteristics has the potential to refine this adaptation. We present a pipeline for integrating automatic assessment seamlessly into a tutoring system and apply the method to the case of developmental dyscalculia (DD). The proposed classifier is based on user inputs only, allowing non-intrusive and unsupervised, universal screening of children. We demonstrate that interaction logs provide enough information to identify children at risk of DD with high accuracy and validity and reliability comparable to traditional assessments. Our model is able to adapt the duration of the screening test to the individual child and can classify a child at risk of DD with an accuracy of 91 % after 11 min on average.

Making effective problem selection decisions is a challenging Self-Regulated Learning skill. Students need to learn effective problem-selection strategies but also develop the motivation to use them. A mastery-approach orientation is generally associated with positive problem selection behaviors such as willingness to work on new materials. We conducted a classroom experiment with 200 6th – 8th graders to investigate the effectiveness of shared control over problem selection with mastery-oriented features (i.e., features that aim at fostering a mastery-approach orientation that simulates effective problem-selection behaviors) on students’ domain-level learning outcomes, problem-selection skills, enjoyment, future learning and future problem selection. The results show that shared control over problem selection accompanied by mastery-oriented features leads to significantly better learning outcomes, as compared to fully system-controlled problem selection, as well as better declarative knowledge of a key problem-selection strategy. Nevertheless, there was no effect on future problem selection and future learning. Our experiment contributes to prior literature by demonstrating that with tutor features to foster a mastery-approach orientation, shared control over problem selection can lead to significantly better learning outcomes than full system control.

Providing choice is known to intrinsically motivate learners and support self-regulated learning. In order to study the effect of providing the choice to skip feedback in an online tutor traditionally used in-natura, we conducted a controlled study in Fall 2015. Experimental group was given the choice to skip the worked example provided as feedback after the student had solved a problem incorrectly, whereas control group was not. We found that providing the choice did not lead to greater learning. Experimental group students needed marginally more problems to learn each concept, and their pre-post improvement was marginally less. When we analyzed skipping behavior, we found that neither the grade on a problem nor the grade on the prior problem on the same concept affected a student’s decision to read or skip feedback. Novelty of the concept on the other hand may prompt students not to skip feedback. Whether or not students skipped feedback on a problem did not affect their grade on the next problem on the same concept. Students were just as likely to skip as not skip feedback on the various problems. Some students tended to skip far more than others.

In this paper we study the effect of adaptive scaffolding to learning by teaching. We hypothesize that learning by teaching is facilitated if (1) students receive adaptive scaffolding on how to teach and how to prepare for teaching (the metacognitive hypothesis), (2) students receive adaptive scaffolding on how to solve problems (the cognitive hypothesis), or (3) both (the hybrid hypothesis). We conducted a classroom study to test these hypotheses in the context of learning to solve equations by teaching a synthetic peer, SimStudent. The results show that the metacognitive scaffolding facilitated tutor learning (regardless of the presence of the cognitive scaffolding), whereas cognitive scaffolding had virtually no effect. The same pattern was confirmed by two additional datasets collected from two previous school studies we conducted.

While the use of autograders for code correctness is widespread, less effort has focused on automating feedback for good programming style: the tasteful use of language features and idioms to produce code that is not only correct, but also concise, elegant, and revealing of design intent. We present a system that can provide real-time actionable code style feedback to students in large introductory computer science classes. We demonstrate that in a randomized controlled trial, 70 % of students using our system achieved the best style solution to a coding problem in less than an hour, while only 13 % of students in the control group achieved the same. Students using our system also showed a statistically-significant greater improvement in code style than students in the control group.

Past studies have shown that Bayesian Knowledge Tracing (BKT) can predict student performance and implement Cognitive Mastery successfully. Standard BKT individualizes parameter estimates for skills, also referred to as knowledge components (KCs), but not for students. Studies deriving individual student parameters from the data logs of student tutor performance have shown improvements to the standard BKT model fits, and result in different practice recommendations for students. This study investigates whether individual student parameters, specifically individual difference weights (IDWs) [1], can be derived from student activities prior to tutor use. We find that student performance measures in reading instructional text and in a conceptual knowledge pretest can be employed to predict IDWs. Further, we find that a model incorporating these predicted IDWs performs well, in terms of model fit and learning efficiency, when compared to a standard BKT model and a model with best-fitting IDWs derived from tutor performance.

The Pedagogical Model is one of the main components of an Intelligent Tutoring System. It is exploited to select a suitable action (e.g., feedback, hint) that the intelligent tutor provides to the learner in order to react to her interaction with the system. Such selection depends on the implemented pedagogical strategy and, typically, takes care of several aspects such as correctness and delay of the learner’s response, learner’s profile, context and so on. The main idea of this paper is to exploit Formal Concept Analysis to automatically learn pedagogical models from data representing human tutoring behaviours. The paper describes the proposed approach by applying it to an early case study.

Modeling student learning during tutorial interaction is a central problem in intelligent tutoring systems. While many modeling techniques have been developed to address this problem, most of them focus on cognitive models in conjunction with often-complex domain models. This paper presents an analysis suggesting that observing students’ multimodal behaviors may provide deep insight into student learning at critical moments in a tutorial session. In particular, this work examines student facial expression, electrodermal activity, posture, and gesture immediately following inference questions posed by human tutors. The findings show that for human-human task-oriented tutorial dialogue, facial expression and skin conductance response following tutor inference questions are highly predictive of student learning gains. These findings suggest that with multimodal behavior data, intelligent tutoring systems can make more informed adaptive decisions to support students effectively.

Interactively modeling science phenomena enables students to develop rich conceptual understanding of science. While this understanding is often assessed through summative, multiple-choice instruments, science notebooks have been used extensively in elementary and secondary grades as a mechanism to promote and reveal reflection through both drawing and writing. Although each modality has been studied individually, obtaining a comprehensive view of a student’s conceptual understanding requires analyses of knowledge represented across both modalities. Evidence-centered design (ECD) provides a framework for diagnostic measurement of data collected from student interactions with complex learning environments. This work utilizes ECD to analyze a corpus of elementary student writings and drawings collected with a digital science notebook. First, a competency model representing the core concepts of each exercise, as well as the curricular unit as a whole, was constructed. Then, evidence models were created to map between student written and drawn artifacts and the shared competency model. Finally, the scores obtained using the evidence models were used to train a deep-learning based model for automated writing assessment, as well as to develop an automated drawing assessment model using topological abstraction. The findings reveal that ECD provides an expressive unified framework for multimodal assessment of science learning with accurate predictions of student learning.

Everything in life has a bright and a dark side; and gamification is not an exception. Although there is an increasing number of publications discussing the benefits of gamification in learning environments, i.e. looking into the bright side of it, several issues can hinder learning because of gamification. Nevertheless, it seems that only few researchers are discussing the dark side of using gamification in learning environments and how to overcome it. Thus, in this paper, we discuss some of the problems of gamification, namely, addiction, undesired competition, and off-task behavior. Furthermore, to deal with both bright and dark sides of gamification at the same time, we propose a framework for intelligent gamification (FIG) that can offer the necessary infrastructure for ITS to personalize the use of gamification by monitoring risk behavior, exploring how best use game design elements to avoid their overuse and finally supporting “fading” mechanisms that gradually reduces the use of gamification and help students to concentrate on learning and not only on extrinsic motivators.

Open-ended computer-based learning environments (OELEs) can be powerful learning tools in that they help students develop effective self-regulated learning (SRL) and problem solving skills. In this study, middle school students used the SimSelf OELE to build causal models to learn about climate science. We study their learning and model building approaches by calculating a suite of behavioral metrics derived using coherence analysis (CA) that are used as features on which to group students by their type of learning behavior. We also analyze changes in these metrics over time, and compare these results to results from other studies with a different OELE to see determine generalizable their findings are across different OELE systems.

In this study we tested whether external regulation provided by artificial pedagogical agents (PAs) was effective in facilitating learners’ self-regulated learning (SRL) and can therefore foster complex learning with a hypermedia-based intelligent tutoring system. One hundred twenty (N = 120) college students learned about the human circulatory system with MetaTutor during a 2-hour session under one of two conditions: adaptive scaffolding (AS) or a control (C) condition. The AS condition received timely prompts from four PAs to deploy various cognitive and metacognitive SRL processes, and received immediate directive feedback concerning the deployment of the processes. By contrast, the C condition learned without assistance from the PAs. Results indicated that those in the AS condition gained significantly more knowledge about the science topic than those in the C condition. In addition, log-file data provided evidence of the effectiveness of the PAs’ scaffolding and feedback in facilitating learners’ (in the AS condition) metacognitive monitoring and regulation during learning. We discuss implications for the design of external regulation by PAs necessary to accurately detect, track, model, and foster learners’ SRL by providing more accurate and intelligent prompting, scaffolding, and feedback regarding SRL processes.

Bayesian Knowledge Tracing (BKT) is one of the most widely adopted student modeling methods in Intelligent Tutoring Systems (ITSs). Conventional BKT mainly leverages sequences of observations (e.g. correct, incorrect) from student-system interaction log files to infer student latent knowledge states (e.g. unlearned, learned). However, the model does not take into account the instructional interventions that generate those observations. On the other hand, we hypothesized that various types of instructional interventions can impact student’s latent states differently. Therefore, we proposed a new student model called Intervention-Bayesian Knowledge Tracing (Intervention-BKT). Our results showed the new model outperforms conventional BKT and two factor analysis based alternatives: Additive Factor Model (AFM) and Instructional Factor Model (IFM); moreover, the learned parameters of Intervention-BKT can recommend adaptive pedagogical policies.

Many students suffer of online reading difficulties because of their low abilities of text comprehension. Several educators tried to set strategies to support learners during their online reading. In current work, we present an online reading environment where students can enroll in virtual reading class, to read and annotate their documents. Based on students’ annotation traces, we build their personality profiles which reflect their level of reading performance. Given the students’ reading abilities, we share the annotations of skilled readers with those having problems of text comprehension. The experimental results show the efficiency of the proposed approach to support learners with low reading abilities.

Calls for widespread Computer Science (CS) education have been issued from the White House down and have been met with increased enrollment in CS undergraduate programs. Yet, these programs often suffer from high attrition rates. One successful approach to addressing the problem of low retention has been a focus on group work and collaboration. This paper details the design of a collaborative ITS (CIT) for foundational CS concepts including basic data structures and algorithms. We investigate the benefit of collaboration to student learning while using the CIT. We compare learning gains of our prior work in a non-collaborative system versus two methods of supporting collaboration in the collaborative-ITS. In our study of 60 students, we found significant learning gains for students using both versions. We also discovered notable differences related to student perception of tutor helpfulness which we will investigate in subsequent work.

We study wheel-spinning behavior among students using an educational game for physics. We attempted to determine whether students wheel-spin, and to build a wheel-spinning detector. We found that about 30 to 40 % of students are unable to successfully complete a level when attempting it 8 times or more, or when working on it for more than 160 s. We also found that past performance is predictive of wheel-spinning, and that persistence increases both the likelihood of success and of wheel-spinning. Finally, we found that wheel-spinning in this context is different from wheel-spinning exhibited in prior work in that it is relatively easy to detect and does not suffer from cold starts.

Studies investigating the effectiveness of game-based learning environments (GBLEs) have reported the effectiveness of these environments on learning and retention. However, there is limited research on using eye-tracking data to investigate metacognitive monitoring with GBLEs. We report on a study that investigated how college students’ eye tracking behavior (n = 25) predicted performance on embedded assessments within the Crystal Island GBLE. Results revealed that the number of books, proportion of fixations on book and article content, and proportion of fixations on concept matrices—embedded assessments associated with each in-game book and article—significantly predicted the number of concept matrix attempts. These findings suggest that participants strategized when reading book and article content and completing assessments, which led to better performance. Implications for designing adaptive GBLEs include adapting to individual student needs based on eye-tracking behavior in order to foster efficient completion of in-game embedded assessments.

The effectiveness of Intelligent Tutoring Systems (ITS) research is enhanced by tools that allow researchers to quickly bridge the divide between theoretical and applied work. By providing a common infrastructure to test cognitive and learning science theories in authentic contexts with real students, the Mobile Fact and Concept Training System (MoFaCTS) can aid in accelerating ITS research and real world implementation. MoFaCTS is run from a web browser and allows the teacher or administrator to set up a sequence of units of content. Because the “optimal practice” module is interchangeable, the system allows for the comparison of alternative methods of adaptive practice. To foster faster research progress, data export supports the DataShop transaction format, which allows quick analysis of data using the DataShop tools. Integration with Amazon Turk allows quick and efficient data collection from this source.

This study investigates how pedagogical conversational agents can facilitate learner-learner collaborative learning during a knowledge integration task. The study focuses on (1) how knowledge integration activity can be facilitated by using multiple Pedagogical Conversational Agents (PCAs) with gaze gestures and (2) how dyad coordination influences learners’ perspective-changing processes and understanding. In a controlled experiment, dyads were accompanied by multiple PCAs programmed to facilitate learning. Two eye-trackers were used to detect the learner’s learning process and coordination. The results show that learners who received facilitation from the PCAs about integrating different perspectives performed better on the task, and if they received gaze gestures, they tended to focus on the relationship of different knowledge as well. Recurrence analysis of gaze patterns show that those who performed well using PCAs synchronized their gaze.

This work examines the efficiency of an agent intervention mode, aiming to stimulate productive conversational interactions and encourage students to explicate their historical reasoning about important domain concepts. The findings of a pilot study, conducted in the context of primary school class in Modern History, (a) suggest a favorable student opinion of the conversational agent, (b) indicate that agent interventions can help students to engage in a transactive form of dialogue, where peers build on each other’s reasoning, and (c) reveal a series of interaction patterns emerging from the display of the agent interventions.

We study the impact of question difficulty on learners’ engagement and learning using an experiment with an open online educational system for adaptive practice of geography. The experiment shows that easy questions are better for short term engagement, whereas difficult questions are better for long term engagement and learning. These results stress the necessity of careful formalization of goals and optimization criteria of open online education systems. We also present disaggregation of overall results into specific contexts of practice.

This study examined the proportional learning gains attained by 165 college students as they learned about the human circulatory system over two sessions with the intelligent tutoring system, MetaTutor. Results indicated that learners in the prompt and feedback condition, which were afforded the full capabilities of the four pedagogical agents (PAs), attained significantly greater proportional learning gains than learners in the control condition who did not receive the same scaffolding. In addition, we also found that the amount of time spent with each PA produced different types of impacts on the learners, with Sam the Strategizer having the most influence on proportional learning gains. Lastly, results from the revised Agent Persona Inventory (API), administered following the learning session with MetaTutor, revealed key findings regarding learners’ overall retrospective affective reactions towards each individual PA. These results have implications for the design of future PAs capable of offering real-time and adaptive pedagogical instruction within Intelligent Tutoring Systems (ITSs).

Research on the impact of peer rating (PR) has provided encouraging results, as a method to foster collaborative learning and improve its outcomes. The scope of this paper is to discuss peer rating towards two specific directions that usually are neglected in the CSCL field, namely: (a) coaching of objective anonymous peer rating through a rubric, and (b) provision of peer rating summary information during collaboration. The case study utilized an asynchronous CSCL tool with the two aforementioned capabilities. Initial results showed that peer rating, when anonymous, and guided, can be as reliable as off-line expert/teacher rating, with indications that this process can foster collaboration.

Peer review is the main mechanism for quality evaluation and peer-mentoring in the research community. Yet, it has been criticized with respect to its summative function, as being prone to bias and inconsistency and approaches had been proposed to improve it (e.g. double blind review). However, relatively less attention has been paid on how well it meets its formative objective, i.e. providing useful feedback to help the authors improve their quality of work. In our previous work we proposed a modified peer review process, which involved a back-evaluation of reviews by the authors. This paper reports the results of a study of the application of this peer review process to support a group of teachers in Chile engage in group peer mentorship in the context of a summer continuing education course. The objectives are to find out if authors reciprocate their reviews feedback in the back-evaluation given to their reviewers, and if the review length affects the helpfulness and authors’ satisfaction with the reviews. Our results showed that peers did not reciprocate their ratings and review length did not affect peers’ satisfaction with the reviews.

This study uses gamification as the carrier of understanding the motivational benefits of applying the Self-Determination Theory (SDT) in social adaptive e-learning, by proposing motivational gamification strategies rooted in SDT, as well as developing and testing these strategies. Results show high perceived motivation amongst the students, and identify a high usability of the implementation, which supports the applicability of the proposed approach.

This paper investigates the effects of training the metacognitive skill of knowledge monitoring when metacognitive instruction is adapted to the characteristics of students in intelligent tutoring systems. An animated pedagogical agent that trains knowledge monitoring was developed and integrated into a step-based tutoring system that helps students in solving algebraic equations. The training provided by the agent encourages learners to reflect on their knowledge and has its content and frequency of intervention adapted to the characteristics of the student. Related work has not adapted the metacognitive instruction to the characteristics of the student, nor has it aimed at investigating the effects of knowledge monitoring training specifically. Results of a classroom study suggest that students who received metacognitive training improved their knowledge monitoring skill and performed better on tests.

The LEA’s Box open learner model (OLM) allows learners to try to persuade the system to make changes to their learner model by challenging evidence or providing justifications. This aims to help make the OLM more accurate, and provides a means for learners to satisfy themselves that the model does indeed reflect their current state of learning. We report an exploratory study with 15 university students, with learner model data coming from quizzes in a Learning Management System. Students generally claimed to understand the approach of learner model persuasion, how it is useful, how it relates to their learning, and identified cases when they could use persuasion.

This work addresses students’ open responses on causal attributions of their self-reported affective states. We use qualitative thematic data analysis techniques to develop a coding scheme by identifying common themes in students’ self-reported attributions. We then applied this scheme to a larger set of student reports. Analysis shows that students’ reasons for reporting a certain affect do not always align with researchers’ expectations. In particular, we discovered that a sizable group of students externalize their affect, attributing perceived difficulty of the problem and their own negativity as lying outside of themselves.

Creating a domain model (expert behavior) is a key component of every tutoring system. Whether the process is manual or semi-automatic, the construction of the rules of expert behavior requires substantial effort. Once finished, the domain model is treated as a fixed entity that does not change based on scope, sequence modifications, or student learning parameters. In this paper, we propose a framework for automatic learning and optimization of the domain model (expressed as condition-action rules) based on designer-provided learning criteria that include aspects of scope, progression sequence, efficiency of learned solutions, and working memory capacity. We present a proof-of-concept implementation based on program synthesis for the domain of linear algebra, and we evaluate this framework through preliminary illustrative scenarios of objective learning criteria.

We hypothesize that when cognitive tutors are integrated into online courseware, the online courseware can provide a new type of adaptive instructions, such as impasse-driven adaptive remediation and need-based assessments. As a proof of concept, we have developed an adaptive online course on the Open Learning Initiative (OLI) platform by integrating four new instances of cognitive tutors into an existing OLI course. Cognitive tutors were created with an innovative cognitive tutor authoring system called Watson. To evaluate the effectiveness of the adaptive online course, a quasi-experiment was conducted in a gateway course at Carnegie Mellon University. The results show that the proposed adaptive online course technology is robust enough to be used in actual classroom with mixed effect for learning.

Cognitive load theory states that improper cognitive loads may negatively affect learning. By identifying students’ working memory capacity (WMC), personalized scaffolding techniques can be used, either by teachers or adaptive systems to offer students individual recommendations of learning activities based on their individual cognitive load. WMC has been identified traditionally by dedicated tests. However, these tests have certain drawbacks (e.g., students have to spend additional time on them, etc.). Therefore, recent research aims at automatically detecting WMC from students’ behavior in learning systems. This paper introduces an automatic approach to identify WMC in learning systems using a genetic algorithm. An evaluation of this approach using data from 63 students shows it outperforms the existing leading approach with an accuracy of 85.1 %. By increasing the accuracy of automatic WMC identification, more accurate interventions can be made to better support students and ensure that their working memory is balanced properly while learning.

Educational standards put a renewed focus on strengthening students’ abilities to construct scientific explanations and engage in scientific arguments. Evaluating student explanatory writing is extremely time-intensive, so we are developing techniques to automatically analyze the causal structure in student essays so that effective feedback may be provided. These techniques rely on a significant training corpus of annotated essays. Because one of our long-term goals is to make it easier to establish this approach in new subject domains, we are keenly interested in the question of how much training data is enough to support this. This paper describes our analysis of that question, and looks at one mechanism for reducing that data requirement which uses student scores on a related multiple choice test.

Previous research has shown that worked examples can increase learning efficiency during computer-aided instruction, especially when alternatively offered with problem solving opportunities. In this study, we investigate whether these results are consistent in a complex, open-ended problem solving domain, where students are presented with randomly ordered sets of worked examples and required problem solving. Our results show that worked examples benefits students early in tutoring sessions, but are comparable to hint-based systems for scaffolding domain concepts. Later in tutoring sessions, worked examples are less beneficial, and can decrease performance for lower-proficiency students.

In this paper we discuss the findings related to our research on the paradigm of Negotiation-Driven Learning (NDL). Fully-negotiated OLMs have employed different negotiation mechanisms to support learner learning and reflection. In NDL research we are trying to combine and extend the best practices of previous OLMs to enhance the role of negotiations and promote cognitive and metacognitive learning in the context of fully-negotiated OLMs. This paper describes the findings of our research and introduces the NDLtutor, which is the realization of the NDL paradigm.

Concept maps represent a significant tool in education, used to plan and guide learning activities and to help teachers in some endeavors such as analyzing and refining their teaching strategies, retrieving suitable learning material, and supporting the provision of adaptive guidance in adaptive learning environments. Here we propose seven measures of similarity among concept maps, representing course modules. They deal with both structural and didactic aspects of the maps, to find out educational similarities among their associated course modules. The performance of the proposed similarity measures are analyzed and evaluated by means of some significant case studies.

This study examines whether an ITS that fosters the use of metacognitive strategies can benefit from variations in its prompts based on learners’ self-regulatory behaviors. We use log files and questionnaire data from 116 participants who interacted with MetaTutor, an advanced multi-agent learning environment that helps learners to develop their self-regulated learning (SRL) skills, in 3 conditions: one without adaptive prompting (NP), one with fading prompts based on learners’ deployment SRL processes (FP), and one where prompts can also increase if learners fail to deploy SRL processes adequately (FQP). Results indicated that an initially more frequent but progressively fading prompting strategy is beneficial to learners’ deployment of SRL processes once the scaffolding is faded, and has no negative impact on learners’ perception of the system’s usefulness. We also found that increasing the frequency of prompting was not sufficient to have a positive impact on the use of SRL processes, when compared to FP. These results provide insights on parameters relevant to prompting adaptation strategies to ensure transfer of metacognitive skills beyond the learning session.

Identifying the pre-requisite relationships among learning objects is a crucial step for faculty and instructional designers when they try to adapt them for delivery in their general education distance courses. We propose a general-purpose content-based approach for facilitating this step by means of semantic analysis techniques: the learning objects are associated to WikiPedia pages (topics), and their dependency is obtained using the classification of those topics supported by Wikipedia Miner.

In this paper we present a serious game, Lewispace, where we focus on measuring and using Electroencephalograms in order to detect how the learner reasons in the game. We track learner’s reasoning according to different regions of the brain. Four standard lobes were taken into consideration: frontal, parietal, occipital and temporal. Each lobe was measured for each participant. We also studied the lobes measures distribution for all the participants. We found that some regions are more related to learner’s vision and reflexion during the game and this could be an indice that the learner follows the correct reasoning process. Primary results show that our game enhance learners’ performance. Moreover, the learners use almost occipital lobe to visualize the task presented in the game and the frontal lobe for the reasoning process.

Worked-out examples have been shown to increase learning gains over problem solving alone. These increases are even greater in novices and those who are learning algorithmic topics, such as those in Computer Science. We have integrated this strategy into our Intelligent Tutoring System and evaluated it on undergraduate students learning the linked list data structure. Although promising, we have identified behavioral differences between high and low gainers - spending less time on an example, and prematurely quitting them led to greater learning.

For cross-pollination between ITS authoring tools, it may be useful to know the prevalence of tutoring behaviors crafted with these tools. As a case study, we analyze the problem units of Mathtutor, a web-based intelligent tutor for middle-school mathematics built, as an example-tracing tutor, with the Cognitive Tutor Authoring Tools (CTAT). We focus on tutoring behaviors that are relevant to a wide range of tutoring systems, not just example-tracing tutors, including behaviors not found in VanLehn’s (2006) taxonomy of tutor behaviors. Our analysis reveals that several tutor behaviors not typically highlighted in the ITS literature were used extensively, sometimes in unanticipated ways. Others were less prevalent than expected. This novel insight into the prevalence of tutor behaviors may provide practical guidance to ITS authoring tool developers. At a theoretical level, it extends VanLehn’s taxonomy of tutor behavior, potentially expanding how the field conceptualizes ITS behavior.

This research contributes to the advancement of intelligent tutoring systems by proposing an affective intelligent tutoring system in the field of specialized education. The Integrated Specialized Learning Application (ISLA) helps autistic children manage their emotions by analyzing the learning trace and considering the learner’s current performance to respond accordingly to it during a mathematical learning situation. We have conducted an experiment to validate the support provided by Jessie based on our accompaniment model. The results showed significant improvement in learning by the test group.

Intelligent tutoring systems (ITS) and MOOCs tend to have complementary pedagogical approaches, but their combination is rarely (if ever) seen. A key obstacle may be technical integration. We present a generalizable case study of extending ITS authoring technology to make tutors easily embeddable into a variety of MOOC/e-learning platforms and run on a range of web-enabled devices. We enhanced the domain-independent Cognitive Tutor Authoring Tools (CTAT) to enable integration of CTAT tutors into multiple environments. A salient lesson learned is that use of widely-used web-based technologies (HTML and JavaScript) may be a major factor in ITS uptake. Also, we found that embedding tutors into existing LMS is challenging, but environment-specific changes can be isolated in a generalizable manner.

In order to increase the engagement of learners, we incorporated cloze procedure questions into the worked-example-style feedback provided by problem-solving tutors currently used by introductory programming students unsupervised. We conducted a multi-institution controlled study to evaluate the effectiveness of this intervention from fall 2012 through spring 2014. The results of the study were mixed. We found that when students had to answer cloze procedure questions embedded in the feedback, they did spend significantly more time per problem and they learned concepts with significantly fewer practice problems. However, they did not learn significantly more concepts and their change in score from pretest to post-test was not any different on the learned concepts from that of control group. Finally, the increased time on task due to the intervention may benefit different demographic subgroups differently.

Intelligent Tutoring Systems that employ a teachable agent or reciprocal tutoring agent are designed to elicit the beneficial effects of tutoring, known as the tutor learning effect. However, untrained tutors do not spontaneously use beneficial tutoring strategies, and in a reciprocal format, it is unclear how the tutor learning effect affects those tutors’ future problem-solving. Here, we examine the effect that the relationship between tutor and tutee has on their likelihood to use various tutoring and learning strategies, and the impact those strategies have on tutees’ future problem-solving in a reciprocal format. We find that among friends, tutees tend towards more verbalization of their problem-solving, with their tutors adopting a more questioning tutoring style, while among strangers, tutees use more shallow questions, with more procedural instruction from their tutor.

Intelligent learning environments based on interactions within the digital world are increasingly popular as they provide mechanisms for interactive and adaptive learning, but learners find it difficult to transfer this to real world tasks. We present the initial development stages of CRISTAL, an intelligent simulator targeted at trainee radiologists which enhances the learning experience by enabling the virtual environment to adapt according to their real world experiences. Our system design has been influenced by feedback from trainees, and allows them to practice their reporting skills by writing freeform reports in natural language. This has the potential to be expanded to other areas such as short-form journalism and legal document drafting.