Artificial Intelligence in Education

Recent years have seen a growing recognition of the central role of affect and motivation in learning. In particular, nonverbal behaviors such as posture and gesture provide key channels signaling affective and motivational states. Developing a clear understanding of these mechanisms will inform the development of personalized learning environments that promote successful affective and motivational outcomes. This paper investigates posture and gesture in computer-mediated tutorial dialogue using automated techniques to track posture and hand-to-face gestures. Annotated dialogue transcripts were analyzed to identify the relationships between student posture, student gesture, and tutor and student dialogue. The results indicate that posture and hand-to-face gestures are significantly associated with particular tutorial dialogue moves. Additionally, two-hands-to-face gestures occurred significantly more frequently among students with low self-efficacy. The results shed light on the cognitive-affective mechanisms that underlie these nonverbal behaviors. Collectively, the findings provide insight into the interdependencies among tutorial dialogue, posture, and gesture, revealing a new avenue for automated tracking of embodied affect during learning.

We conducted a study to track the emotions, their behavioral correlates, and relationship with performance when novice programmers learned the basics of computer programming in the Python language. Twenty-nine participants without prior programming experience completed the study, which consisted of a 25 minute scaffolding phase (with explanations and hints) and a 15 minute fadeout phase (no explanations or hints) with a computerized learning environment. Emotional states were tracked via retrospective self-reports in which learners viewed videos of their faces and computer screens recorded during the learning session and made judgments about their emotions at approximately 100 points. The results indicated that flow/engaged (23%), confusion (22%), frustration (14%), and boredom (12%) were the major emotions students experienced, while curiosity, happiness, anxiety, surprise, anger, disgust, fear, and sadness were comparatively rare. The emotions varied as a function of instructional scaffolds and were systematically linked to different student behaviors (idling, constructing code, running code). Boredom, flow/engaged, and confusion were also correlated with performance outcomes. Implications of our findings for affect-sensitive learning interventions are discussed.

Research in affective computing and educational technology has shown the potential of affective interventions to increase student’s self-concept and motivation while learning. Our project aims to investigate whether the use of affective interventions in a meta-cognitive tutor can help students achieve deeper modeling of dynamic systems by being persistent in their use of meta-cognitive strategies during and after tutoring. This article is an experience report on how we designed and implemented the affective intervention. (The meta-tutor is described in a separate paper.) We briefly describe the theories of affect underlying the design and how the agent’s affective behavior is defined and implemented. Finally, the evaluation of a detector-driven categorization of student behavior, that guides the agent’s affective interventions, against a categorization performed by human coders, is presented.

Recent research has shown that differences in software design and content are associated with differences in how much students game the system and go off-task. In particular the design features of a tutor have found to predict substantial amounts of variance in gaming and off-task behavior. However, it is not yet understood how this influence takes place. In this paper we investigate the relationship between a student’s affective state, their tendency to engage in disengaged behavior, and the design aspects of the learning environments, towards understanding the role that affect plays in this process. To investigate this question, we integrate an existing taxonomy of the features of tutor lessons [3] with automated detectors of affect [8]. We find that confusion and frustration are significantly associated with lesson features which were found to be associated with disengaged behavior in past research. At the same time, we find that the affective state of engaged concentration is significantly associated with features associated with lower frequencies of disengaged behavior. This analysis suggests that simple re-designs of tutors along these lines may lead to both better affect and less disengaged behavior.

Csikszentmihalyi’s Flow theory states that a balance between challenge and skill leads to high engagement, overwhelming challenge leads to anxiety or frustration, and insufficient challenge leads to boredom. In this paper, we test this theory within the context of student interaction with an intelligent tutoring system. Automated detectors of student affect and knowledge were developed, validated, and applied to a large data set. The results did not match Flow theory: boredom was more common for poorly-known material, and frustration was common both for very difficult material and very easy material. These results suggest that design for optimal engagement within online learning may require further study of the factors leading students to become bored on difficult material, and frustrated on very well-known material.

Recent research has indicated that learning environments that intentionally induce confusion to promote deep inquiry can be beneficial for learning if students engage in confusion resolution processes and if relevant scaffolds are provided. However, it is unlikely that these environments will benefit all students, so it is necessary to identify the student profiles that most benefit from confusion induction. We investigated how individual differences (e.g., prior knowledge, interest, attributional complexity) impacted confusion and learning outcomes in an environment that induced confusion via false system feedback (e.g., negative feedback after a correct response). A

k

-means cluster analysis revealed four clusters that varied on cognitive ability and cognitive drive. We found that students in the high cognitive ability + high cognitive drive cluster reported more confusion after receiving false feedback compared to the other clusters. These students also performed better on tasks requiring knowledge transfer, but only when they were meaningfully confused.

In this study we aligned and compared self-report and on-line emotions data on 67 college students’ emotions at five different points in time over the course of their interactions with MetaTutor. Self-reported emotion data as well as facial expression data were converged and analyzed. Results across channels revealed that neutral and positively-valenced basic and learner-centered emotional states represented the majority of emotional states experienced with MetaTutor. The self-report results revealed a decline in the intensity of positively-valenced and neutral states across the learning session. The facial expression results revealed a substantial decrease in the number of learners’ with neutral facial expressions from time one to time two, but a fairly stable pattern for the remainder of the session, with participants who experienced other basic emotional states, transitioning back to a state of neutral between self-reports. Agreement between channels was 75.6%.

Maintaining learner engagement is critical for all types of learning technologies. This study investigated how choice over a learning topic and the difficulty of the materials influenced mind wandering, engagement, and learning during a computerized learning task. 59 participants were randomly assigned to a text difficulty and choice condition (i.e., self-selected or experimenter-selected topic) and measures of mind wandering and engagement were collected during learning. Participants who studied the difficult version of the texts reported significantly higher rates of mind wandering (

d

= .41) and lower arousal both during (

d

= .52) and after the learning session (

d

= .48). Mind wandering and arousal were not affected by choice. However, participants who were assigned to study the topic they selected reported significantly more positive valence during (

d

= .57) but not after learning. These participants also scored substantially higher on a subsequent knowledge test (

d

= 1.27). These results suggest that choice and text difficulty differentially impact mind wandering, engagement, and learning and provide important considerations for the design of ITSs and serious games with a reading component.

Automatic question generation can support instruction and learning. However, work to date has produced mostly “shallow” questions that fall short of supporting deep learning and discussion. We propose an extension to a state-of-the-art question generation system that allows it to produce deep, subjective questions suitable for group discussion. We evaluate the questions generated by this system against a panel of experienced judges, and find that our approach fares significantly better than the baseline system.

Peer-review systems such as SWoRD lack intelligence for detecting and responding to problems with students’ reviewing performance. While prior work has demonstrated the feasibility of automatically identifying desirable feedback features in free-text reviews of student papers, similar methods have not yet been developed for feedback regarding argument diagrams. One desirable feedback feature is problem localization, which has been shown to positively correlate with feedback implementation in both student papers and argument diagrams. In this paper we demonstrate that features previously developed for identifying localization in paper reviews do not work well when applied to peer reviews of argument diagrams. We develop a novel algorithm tailored for reviews of argument diagrams, and demonstrate significant performance improvements in identifying problem localization in an experimental evaluation.

Learning to collaborate is important. But how does one learn to collaborate face-to-face? What are the actions and strategies to follow for a group of students who start a task? We analyse aspects of students’ collaboration when working around a multi-touch tabletop enriched with sensors for identifying users, their actions and their verbal interactions. We provide a technological infrastructure to help understand how highly collaborative groups work compared to less collaborative ones. The contributions of this paper are (1) an

automatic approach

to distinguish, discover and distil salient common patterns of interaction within groups, by mining the logs of students’ tabletop touches and detected speech; and (2) the

instantiation

of this approach in a particular study. We use three data mining techniques: a classification model, sequence mining, and hierarchical clustering. We validated our approach in a study of 20 triads building solutions to a posed question at an interactive tabletop. We demonstrate that our approach can be used to discover patterns that may be associated with strategies that differentiate high and low collaboration groups.

We developed a learning environment to combine problem-posing and problem-solving activities. The participants learned a formal logic system, natural deduction, by alternating between the problem-posing and problem-solving phases. In the problem posing-phase, the participants posed original problems and presented them on a shared problem database called “Forum,” which was accessible to other group members. During the problem-solving phase, the participants solved the problems presented on Forum. This first round of problem posing and solving was followed by a second round of problem posing. We performed two practices for evaluation. The results showed that the participants successfully posed more advanced problems in the second round of problem posing as compared to the first. The empirical data gathered from the two practices indicated a significant relationship between problem-solving and problem-posing abilities.

Social user-generated content (e.g. comments, blogs) will play a key role in learning environments providing a rich source for capturing diverse viewpoints; and is particularly beneficial in ill-defined domains that encompass diverse interpretations. This paper presents ViewS - a framework for capturing viewpoints from user-generated textual content following a semantic sensing approach. It performs semantic augmentation using existing ontologies and presents the resultant semantic spaces in a visual way. ViewS was instantiated for interpersonal communication and validated in a study with comments on job interview videos, achieving over 82% precision. The potential of ViewS for enriching learning environments is illustrated in an exploratory study by analysing micro-blogging content collected within a learning simulator for interpersonal communication. A group interview with simulator designers evinced benefits for gaining insights into learner reactions and further simulator improvement.

We explore the potential of bringing together the advantages of computer games and the physical world to increase engagement, collaboration and learning. We introduce EarthShake: A tangible interface and mixed-reality game consisting of an interactive multimodal earthquake table, block towers and a computer game synchronized with the physical world via depth camera sensing. EarthShake helps kids discover physics principles while experimenting with real blocks in a physical environment supported with audio and visual feedback. Students interactively make predictions, see results, grapple with disconfirming evidence and formulate explanations in forms of general principles. We report on a preliminary user study with 12 children, ages 4-8, indicating that EarthShake produces large and significant learning gains, improvement in explanation of physics concepts, and clear signs of productive collaboration and high engagement.

The personalization of learning remains a major challenge for research in Intelligent Tutoring Systems (ITS). We report in this article how we used the Adapte tool to make AMBRE-add adaptive. AMBRE-add is an ITS designed to teach a problem solving method. This ITS includes a module that analyzes the learner’s activity traces in order to compute a learner profile. Furthermore a problem generator enables us to specify activities proposed to the student. In order to design an automated process of personalizing activities according to the learner profile, we used the Adapte system. This is a generic system enabling the definition of a personalization strategy and its application to an external ITS. In this article we present how this tool provides real assistance to an ITS designer wishing to make his/her system adaptive.

For decades, intelligent tutoring systems researchers have been developing various methods of student modeling. Most of the models, including two of the most popular approaches: Knowledge Tracing model and Performance Factor Analysis, all have similar assumption: the information needed to model the student is the student’s performance. However, there are other sources of information that are not utilized, such as the performance on other students in same class. This paper extends the Student-Skill extension of Knowledge Tracing, to take into account the class information, and learns four parameters: prior knowledge, learn, guess and slip for each class of students enrolled in the system. The paper then compares the accuracy using the four parameters for each class versus the four parameters for each student to find out which parameter set works better in predicting student performance. The result shows that modeling at coarser grain sizes can actually result in higher predictive accuracy, and data about classmates’ performance is results in a higher predictive accuracy on unseen test data.

We describe a method to evaluate how student models affect ITS decision quality – their raison d’être. Given logs of randomized tutorial decisions and ensuing student performance, we train a classifier to predict tutor decision outcomes (success or failure) based on situation features, such as student and task. We define a decision policy that selects whichever tutor action the trained classifier predicts in the current situation is likeliest to lead to a successful outcome. The ideal but costly way to evaluate such a policy is to implement it in the tutor and collect new data, which may require months of tutor use by hundreds of students. Instead, we use historical data to simulate a policy by extrapolating its effects from the subset of randomized decisions that happened to follow the policy. We then compare policies based on alternative student models by their simulated impact on the success rate of tutorial decisions. We test the method on data logged by Project LISTEN’s Reading Tutor, which chooses randomly which type of help to give on a word. We report the cross-validated accuracy of predictions based on four types of student models, and compare the resulting policies’ expected success and coverage. The method provides a utility-relevant metric to compare student models expressed in different formalisms.

Bayesian Knowledge Tracing (BKT)[1] is a user modeling method extensively used in the area of Intelligent Tutoring Systems. In the standard BKT implementation, there are only skill-specific parameters. However, a large body of research strongly suggests that student-specific variability in the data, when accounted for, could enhance model accuracy [5,6,8]. In this work, we revisit the problem of introducing student-specific parameters into BKT on a larger scale. We show that student-specific parameters lead to a tangible improvement when predicting the data of unseen students, and that parameterizing students’ speed of learning is more beneficial than parameterizing a priori knowledge.

Both Knowledge Tracing and Performance Factors Analysis, are examples of student modeling frameworks commonly used in AIED systems (i.e., Intelligent Tutoring Systems). Both of them use student correctness as a binary input, but student performance on a question might better be represented with a continuous value representing a type of partial credit. Intuitively, a student who has to make more attempts, or has to ask for more hints, deserves a score closer to zero, while students who asks for no hints and just needs to make a second attempt on a question should get a score close to one. In this work, we present a simple change to the Knowledge Tracing model and a simple (non-optimized) method for assigning partial credit. We report our real data experiment result in which we compared the original Knowledge Tracing (OKT) model with this new Knowledge Tracing model that uses partial credit as input (KTPC). The new model outperforms the traditional model reliably. The practical implication of this work is that this new technique can be widely used easily, as it is a small change from the traditional way of fitting KT models.

Semantic network theories of knowledge organization support the idea that recall of organized information depends on how well a learner encodes the connections between the items in the semantic network. However, there is need for more research into what this implies for configuring instruction so that strong semantic network learning is supported with the goal of creating an integrated mental model in the student’s mind. We investigate this question in the context of map learning, where country names are encoded relative to geographic border, internal features, or external features. The main hypothesis was that external features as cues would encourage transfer, since students would practice a network of relationships. The results primarily supported a theory of “cue reinstatement”, where transfer occurred when cues present at learning were present at testing. These effects were analyzed with a mixed effects logistic regression learner model of trial-by-trial learning.

This paper introduces the Next-TELL independent open learner model which is constructed based on data from a range of sources. An example is presented for a university course, with the learner model built from the main activities undertaken during the course. Use of the Next-TELL open learner model over a five week period is described for this group of students, suggesting that independent open learner models built from multiple sources of data may have much to offer in supporting students’ understanding of their learning, and could potentially be used to encourage greater peer interaction.

Inquiry-based learning has been proposed as a natural and authentic way for students to engage with science. Inquiry-based learning environments typically require students to guide their own learning and inquiry processes as they gather data, make and test hypotheses and draw conclusions. Some students are highly self-regulated learners and are able to guide and monitor their own learning activities effectively. Unfortunately, many students lack these skills and are consequently less successful in open-ended, inquiry-based environments. This work examines differences in inquiry behavior patterns in an open-ended, game-based learning environment,

Crystal Island

. Differential sequence mining is used to identify meaningful behavior patterns utilized by Low, Medium, and High self-regulated learners. Results indicate that self-regulated learners engage in more effective problem solving behaviors and demonstrate different patterns of use of the provided cognitive tools. The identified patterns help provide further insight into the role of SRL in inquiry-based learning and inform future approaches for scaffolding.

Self-assessment and study choice are two important metacognitive processes involved in Self-Regulated Learning. Yet not much empirical work has been conducted in ITSs to investigate how we can best support these two processes and improve students’ learning outcomes. The present work redesigned an Open Learner Model (OLM) with three features aimed at supporting self-assessment (self-assessment prompts, delaying the update of the skill bars and progress information on the problem type level). We also added a problem selection feature. A 2x2 experiment with 62 7

th

graders using variations of an ITS for linear equation solving found that students who had access to the OLM performed significantly better on the post-test. To the best of our knowledge, the study is the first experimental study that shows an OLM enhances students’ learning outcomes with an ITS. It also helps establish that self-assessment has key influence on student learning of problem solving tasks.

In this paper, we explore the potential of gaze data as a source of information to predict learning as students interact with MetaTutor, an ITS that scaffolds self-regulated learning. Using data from 47 college students, we show that a classifier using a variety of gaze features achieves considerable accuracy in predicting student learning after seeing gaze data from the complete interaction. We also show promising results on the classifier ability to detect learning in real-time during interaction.

This paper describes a method for improving students’ help-seeking behavior by creating a teammate relationship between intelligent tutors and students. Help seeking in intelligent tutors involves student self-regulation as described in learning theory and can be explored from the perspective of social psychology. We describe an experiment in which ninety-seven students were randomly assigned to treatment and control conditions and students in the treatment group were supported to relate to the Wayang Math Tutor as teammates by providing the help button named “Work Together”. The result suggests that students who treated the tutor as teammates saw more hints (asked for more hints), exhibited reduced quick-guessing behavior and did not abuse hints while working together to solve math problems.

According to Self-Regulated Learning theories, self-assessment by students can facilitate in-depth reflection and help direct effective self-regulated learning. Yet, not much work has investigated the relation between students’ self-assessment and learning outcomes in Intelligent Tutoring Systems (ITSs). This paper investigates this relation with classrooms using the Geometry Cognitive Tutor. We designed a paper-based skill diary that helps students take advantage of the tutor’s Open Learner Model to self-assess their problem-solving skills periodically, and investigated whether it can support students’ self-assessment and learning. In an experiment with 122 high school students, students in the experimental group were prompted periodically to fill out the skill diaries, whereas the control group answered general questions that did not involve active self-assessment. The experimental group performed better on the post-test, and the skill diaries helped lower-performing students to significantly improve their learning outcomes and self-assessment accuracy. This work is among the first empirical studies that successfully establish the beneficial role of self-assessment in students’ learning of problem-solving tasks in ITSs.

This study investigates students’ essay revising in the context of an intelligent tutoring system called

Writing Pal

(W-Pal), which combines strategy instruction, game-based practice, essay writing practice, and automated formative feedback. We examine how high school students use W-Pal feedback to revise essays in two different contexts: a typical approach that emphasizes intensive writing practice, and an alternative approach that offers less writing practice with more direct strategy instruction. Results indicate that students who wrote fewer essays, but received W-Pal strategy instruction, were more likely to make substantive revisions that implemented specific recommendations conveyed by the automated feedback. Additional analyses consider the role of motivation and perceived learning on students’ revising behaviors.

We present an evaluation of the Writing Pal (W-Pal) intelligent tutoring system (ITS) and the W-Pal automated writing evaluation (AWE) system through the use ofcomputational indices related to text cohesion. Sixty-four students participated in this study. Each student was assigned to either the W-Pal ITS condition or the W-Pal AWE condition. The W-Pal ITS includes strategy instruction, game-based practice, and essay-based practice with automated feedback. In the ITS condition, students received strategy training and wrote and revised one essay in each of the 8 training sessions. In the AWE condition, students only interacted with the essay writing and feedback tools. These students wrote and revised two essays in each of the 8 sessions. Indices of local and global cohesion reported by the computational tools Coh-Metrix and the Writing Assessment Tool (WAT) were used to investigate pretest and posttest writing gains. For both the ITS and the AWE systems, training led to the increased use of global cohesion features in essay writing. This study demonstrates that automated indices of text cohesion can be used to evaluate the effects of ITSs and AWE systems and further demonstrates how text cohesion develops as a result of instruction, writing, and automated feedback.

We present an approach for combining symbolic interpretation and statistical classification in the natural language processing (NLP) component of a tutorial dialogue system. Symbolic NLP approaches support dynamic generation of context-adaptive natural language feedback, but lack robustness. In contrast, statistical classification approaches are robust to ill-formed input but provide less detail for context-specific feedback generation. We describe a system design that combines symbolic interpretation with statistical classification to support context-adaptive, dynamically generated natural language feedback, and show that the combined system significantly improves interpretation quality while retaining the adaptivity benefits of a symbolic interpreter.

A large number of studies carried out on pupils aged 8–14 have shown that teachable agent (TA) based games are beneficial for learning. The present pioneering study aimed to initiate research looking at whether TA based games can be used as far down as preschool age. Around the age of four, theory of mind (ToM) is under development and it is not unlikely that a fully developed ToM is necessary to benefit from a TA’s socially engaging characteristics. 10 preschool children participated in an experiment of playing a mathematics game. The participants playing a TA-version of the game engaged socially with the TA and were not disturbed by his presence. Thus, this study unveils exciting possibilities for further research of the hypothesised educational benefits in store for preschoolers with regard to play-and-learn games employing TAs.

To date, the majority of learning technologies only afford virtual interactions on desktops or tablets, despite evidence that students learn through physical manipulation of their environment. We implemented a tangible system that allows students to solve coordinate geometry problems by interacting in a physical space with digitally augmented devices, using a teachable agent framing. We describe our system and the results from a pilot involving students using our system to teach a virtual agent. Students used a variety of strategies to solve problems that included embodied behaviors, and the majority did feel they were teaching their agent. We discuss the implications of our findings with respect to the design of adaptive tangible teachable systems.

We describe Coach Mike, an animated pedagogical agent for informal computer science education, and report findings from two experiments that provide initial evidence for the efficacy of the system. In the first study, we found that Coach Mike’s presence led to 20% longer holding times, increased acceptance of programming challenges, and reduced misuse of the exhibit, but had limited cumulative impact on attitudes, awareness, and knowledge beyond what the host exhibit already achieved. In the second study, we compared two different versions of Coach Mike and found that the use of enthusiasm and self-regulatory feedback led to greater self-efficacy for programming.

This paper describes two types of Conceptually Grounded Learning Activities designed to foster more robust learning in the Genetics Cognitive Tutor: interleaved worked examples and genetic-process reasoning scaffolds. We report three empirical studies that evaluate the impact of these learning activities on three diverse genetics problem-solving topics in the tutor. We found that interleaved worked examples yielded less basic-skill learning than conventional problem solving, unlike many prior ITS studies of worked examples. We also found preliminary evidence that scaffolded reasoning tasks in conjunction with conventional problem solving leads to more robust understanding than conventional problem solving alone. Implications for the use of contextually grounded learning activities are discussed.

Multiple graphical representations can significantly improve students’ learning. To acquire robust knowledge of the domain, students need to make connections between the different graphical representations. In doing so, students need to engage in two crucial learning processes: sense-making processes to build up conceptual understanding of the connections, and fluency-building processes to fast and effortlessly make use of perceptual properties in making connections. We present an experimental study which contrasts two hypotheses on how these learning processes interact. Does understanding facilitate fluency-building processes, or does fluency enhance sense-making processes? And consequently, which learning process should intelligent tutoring systems support first? Our results based on test data and tutor logs show an advantage for providing support for sense-making processes before fluency-building processes. To enhance students’ robust learning of domain knowledge, ITSs should ensure that students have adequate conceptual understanding of connections between graphical representations before providing fluency-building support for connection making.

Learning from worked examples has been shown to be superior to unsupported problem solving in numerous studies. Examples reduce the cognitive load on the learner’s working memory, thus helping the student to learn faster or deal with more complex questions. Only recently researchers started investigating the worked example effect in Intelligent Tutoring Systems (ITSs). We conducted a study to investigate the effect of using worked examples in combination with supported problem-solving in SQL-Tutor. We had three conditions: Examples Only (EO), Problems Only (PO), and Alternating Examples/Problems (AEP). After completing a problem, students received a self-explanation prompt that focused on the concepts used in the problem, to make sure that students acquire conceptual knowledge. On the other hand, examples were followed by self-explanation prompts that focused on procedural knowledge. The study showed that the AEP and PO conditions outperformed EO in learning gain, while AEP outperformed PO in conceptual knowledge acquisition. Therefore, interleaving examples with supported problems is an optimal choice compared to using examples or supported problems only in SQL-Tutor.

We have created a generalized algorithm for automatically constructing domain level knowledge bases from student input. This method has demonstrated greater efficiencies than when knowledge is hand crafted by subject matter experts (SMEs). This paper presents two related methods for improving automated knowledge acquisition by leveraging the properties of games and simulations. First, we discuss game mechanics that, when added to our intelligent tutor Rashi, lead to higher quantity and quality of student input. In a separate but related analysis, we present a novel game type called a knowledge refinement game (KRG) to improve the knowledge in an expert knowledge base. This game motivates SMEs to refine the generated knowledge base, especially for data in which the system has low confidence. Utilizing an anonymous agreement policy ensures the quality of SME responses and results show that small amounts of KRG activity leads to noticeable improvements in the quality of the knowledge base. We assert that these two results in unison provide evidence that gaming has a powerful potential role in improving artificial intelligence techniques for education.

This study investigates how students’ prior expectations of technology affect overall learning outcomes across two adaptive systems, one game-based (iSTART-ME) and one non-game based (iSTART-Regular). The current study (n=83) is part of a larger study (n=124) intended to teach reading comprehension strategies to high school students. Results revealed that students’ prior expectations impacted learning outcomes, but only for students who had engaged in the game-based system. Students who reported positive expectations of computer helpfulness at pretest showed significantly higher learning outcomes in the game-based system compared to students who had low expectations of computer helpfulness. The authors discuss how the incorporation of game-based features in an adaptive system may negatively impact the learning outcomes of students with low technology expectations.

A key challenge posed by narrative-centered learning environments is dynamically tailoring story events to individual students. This paper investigates techniques for sequencing story-centric embedded assessments—a particular type of story event that simultaneously evaluates a student’s knowledge and advances an interactive narrative’s plot—in narrative-centered learning environments. We present an approach for personalizing embedded assessment sequences that is based on collaborative filtering. We examine personalized event sequencing in an edition of the

Crystal Island

narrative-centered learning environment for literacy education. Using data from a multi-week classroom study with 850 students, we compare two model-based collaborative filtering methods, including probabilistic principal component analysis (PPCA) and non-negative matrix factorization (NMF), to a memory-based baseline model,

k

-nearest neighbor. Results suggest that PPCA provides the most accurate predictions on average, but NMF provides a better balance between accuracy and run-time efficiency for predicting student performance on story-centric embedded assessment sequences.

ReaderBench

is a multi-purpose, multi-lingual and flexible environment that enables the assessment of a wide range of learners’ productions and their manipulation by the teacher.

ReaderBench

allows the assessment of three main textual features: cohesion-based assessment, reading strategies identification and textual complexity evaluation, which have been subject to empirical validations.

ReaderBench

covers a complete cycle, from the initial complexity assessment of reading materials, the assignment of texts to learners, the capture of metacognitions reflected in one’s textual verbalizations and comprehension evaluation, therefore fostering learner’s self-regulation process.

This paper introduces a method to predict and analyse students’ mathematical performance by detecting distinguishable subgroups of children who share similar learning patterns. We employ pairwise clustering to analyse a comprehensive dataset of user interactions obtained from a computer-based training system. The available data consist of multiple learning trajectories measured from children with developmental dyscalculia, as well as from control children. Our online classification algorithm allows accurate assignment of children to clusters early in the training, enabling prediction of learning characteristics. The included results demonstrate the high predictive power of assignments of children to subgroups, and the significant improvement in prediction accuracy for short- and long-term performance, knowledge gaps, overall training achievements, and scores of further external assessments.

Systems for smart authoring of automated tutors, like SimStudent, have been mostly applied in well-defined problem-solving domains where little real-world background knowledge is needed, like math. Here we explore the generality of these methods by considering a very different task, article selection in English, where little problem-solving is done, but where complex prior perceptual skills and large amounts of background knowledge are needed. This background knowledge includes the ability to parse text and the extensive understanding of semantics of English words and phrases. We show that good performance can be obtained by coupling SimStudent with appropriate broad-coverage linguistic tools. Performance can be improved further on this task by extending one of the learning mechanisms used by SimStudent so that it will accept less-accurate production rule conditions, and prioritize learned production rules by accuracy. Experimental results show that the extended SimStudent successfully learns the tutored article selection grammar rules, and can be used to discover a student model that predicts human student behavior as well as the human-generated model.

Simulated pedagogical agents have a long history in AIED research. We are interested in simulation from another, less well explored perspective: simulating the entire learning environment (including learners) to inform the system design process. An AIED system designer can carry out experiments in the simulation environment that would otherwise be too costly (or time consuming) with real learners using a real system. We suggest that an architecture called the “ecological approach (EA)”[1] can form the basis for creating such simulations. To demonstrate, we describe how to develop a proof-of-concept simulated ITS prototype, modelled in the EA architecture. We also show how to factor in data from two human subject studies (done for other purposes) to gain a degree of cognitive fidelity. An experiment is carried out with the prototype. The approach is general and can apply to learning systems with a wide variety of “pedagogical styles” (not just ITSs) at various stages of their life cycle. We conclude that simulation is a critically needed methodology in AIED.

Deep analysis of domain content yields novel insights and can be used to produce better courses. Aspects of such analysis can be performed by applying AI and statistical algorithms to student data collected from educational technology and better cognitive models can be discovered and empirically validated in terms of more accurate predictions of student learning. However, can such improved models yield improved student learning? This paper reports positively on progress in closing this loop. We demonstrate that a tutor unit, redesigned based on data-driven cognitive model improvements, helped students reach mastery more efficiently. In particular, it produced better learning on the problem-decomposition planning skills that were the focus of the cognitive model improvements.

The concept of mastery learning is powerful: rather than a fixed number of practices, students continue to practice a skill until they have mastered it. However, an implicit assumption in this formulation is that students are capable of mastering the skill. Such an assumption is crucial in computer tutors, as their repertoire of teaching actions may not be as effective as commonly believed. What if a student lacks sufficient knowledge to solve problems involving the skill, and the computer tutor is not capable of providing sufficient instruction? This paper introduces the concept of “wheel-spinning;” that is, students who do not succeed in mastering a skill in a timely manner. We show that if a student does not master a skill in ASSISTments or the Cognitive Tutor quickly, the student is likely to struggle and will probably never master the skill. We discuss connections between such lack of learning and negative student behaviors such as gaming and disengagement, and discuss alterations to ITS design to overcome this issue.

Uncovering the right skills behind question items is a difficult task. It requires a thorough understanding of the subject matter and of the cognitive factors that determine student performance. The skills definition, and the mapping of item to skills, require the involvement of experts. We investigate means to assist experts for this task by using a data driven, matrix factorization approach. The two mappings of items to skills, the expert on one side and the matrix factorization on the other, are compared in terms of discrepancies, and in terms of their performance when used in a linear model of skills assessment and item outcome prediction. Visual analysis shows a relatively similar pattern between the expert and the factorized mappings, although differences arise. The prediction comparison shows the factorization approach performs slightly better than the original expert Q-matrix, giving supporting evidence to the belief that the factorization mapping is valid. Implications for the use of the factorization to design better item to skills mapping are discussed.

Educational assessments occasionally require “uniform test forms” for which each test form consists of a different set of items, but the forms meet equivalent test specifications (i.e., qualities indicated by test information functions based on item response theory). We propose two maximum clique algorithms (MCA) for uniform test forms assembly. The proposed methods can assemble uniform test forms with allowance of overlapping items among uniform test forms. First, we propose an exact method that maximizes the number of uniform test forms from an item pool. However, the exact method presents computational cost problems. To relax those problems, we propose an approximate method that maximizes the number of uniform test forms asymptotically. Accordingly, the proposed methods can use the item pool more efficiently than traditional methods can. We demonstrate the efficiency of the proposed methods using simulated and actual data.

Intelligent Tutoring Systems (ITSs) have proven their effectiveness in many instructional domains, ranging in the complexity of domain theories and tasks students are to perform. The typical effect sizes achieved by ITSs are around 1SD, which are still low in comparison to the effectiveness of expert human tutors. Recently there have been several analyses done in order to identify the factors that contribute to success of human tutors, and to replicate it in ITSs. VanLehn [6] proposes that the crucial factor is the

granularity of interaction

: the lower the level of discussions between the (human or artificial) tutor and the student, the higher the effectiveness. We investigated the effect of interaction granularity in the context of NORMIT, a constraint-based tutor that teaches data normalization. Our study compared the standard version of NORMIT, which provided hints in response to errors, to a version which used adaptive tutorial dialogues instead. The results show that the interaction granularity hypothesis holds in our experimental situation, and that the effect size achieved is consistent with other reported studies of a similar nature.

Most work on learning curves for ITSs has focused on the knowledge components (or

skills

) included in the curves, aggregated across students. But an aggregate learning curve need not have the same form as subsets of its underlying data, so learning curves for subpopulations of students may take different forms. We show that disaggregating a skill’s aggregate learning curve into separate learning curves for different student subpopulations can reveal learning: 70% of the skills that did not show learning and were identified as candidates for improvement did show learning when disaggregated. This phenomenon appears to be in part a characteristic of mastery learning. Disaggregated learning curves can reconcile an apparent mismatch between the tutor’s

runtime

assessment of student knowledge and the

post hoc

assessment provided by the aggregate learning curve. More precise learning curves can be used to refine Bayesian knowledge tracing parameters and to improve skill model assessment metrics.

This paper describes an exploratory study of system-side errors (i.e. expectation- or rule-violations) in a

virtual environment

(VE), and the subsequent reactions of young children with

autism spectrum conditions

(ASC). Analysis of existing video from 8 participants interacting with the ECHOES VE showed that they frequently detected and reacted to system-side errors, engaging in social and communicative behaviours targeted by ECHOES. Detecting errors requires children to compare the VE’s state to their “mental model” of its behaviour, determining where the two are discrepant. This is equivalent to learners identifying mistakes in their

own

knowledge and then re-aligning with the system-as-expert. This paper explores the implications of these results, proposing a taxonomy of discrepant event types, and discussing their location with respect to the learner and/or system. In addition to considering these results’ significance for this user group and context, it relates the research to existing work that uses erroneous examples.

Dialectal differences are one explanation for the systematically reduced test scores of children of color compared to their Euro-American peers. In this work, we explore the relationship between academic performance and dialect differences exhibited in a learning environment by assessing 3

rd

grade students’ science performance after interacting with a “distant peer” technology that employed one of three dialect use patterns. We found that our participants, all native speakers of African American Vernacular English (AAVE), demonstrated the strongest science performance when the technology used AAVE features consistently throughout the interaction. These results call for a re-examination of the cultural assumptions underlying the design of educational technologies, with a specific emphasis on the way in which we present information to culturally-underrepresented groups.

This paper analyzes the state of current intelligent tutoring systems (ITS) research for applications in the developing world. Recent data shows a rapidly narrowing digital divide, with internet and computing device access rising sharply in less developed countries. Tutoring systems could be a transformative technology in these areas, where shortages of teachers and materials are persistent problems. However, the unique challenges and opportunities for ITS in this context are not well-explored. This paper identifies barriers to adoption distinct to the developing world, then presents the results of a systematic mapping study of recent ITS literature (2009-2012) that looks at the level of focus given to each barrier. This study finds that only a small percentage of peer-reviewed publications and architectures address even one of the barriers preventing adoption in these contexts. Implications and strategies being used to target these barriers are discussed.

The translation of word problems into equations is one of the major difficulties for students regarding problem solving. This paper describes both a domain-specific knowledge representation and an inference engine based on hypergraphs that permits intelligent student supervision of this stage of the solving process. The framework presented makes it possible to simultaneously: a) represent all potential algebraic solutions to a given word problem; b) keep track of the student’s actions; c) provide automatic remediation; and d) determine the current state of the resolution process univocally. Starting from these ideas, we have designed an intelligent tutoring system (ITS). An experimental evaluation supports the use of this ITS in practice.

Because feedback affects learning, it is central to many educational technologies. We analyze properties of hint feedback in an intelligent tutoring system for high school geometry. First, we examine whether feedback content or feedback sequence is a better predictor of student performance after feedback. Second, we investigate whether linguistic features of hints affect performance. We find that students respond to different hint types differently even after accounting for student proficiency, skill difficulty, and prior practice. We also find that hint content, but not linguistic features affects performance. The findings suggest that tutoring system developers should focus on individual learner differences and feedback content.

While open-ended learning environments (OELEs) offer powerful learning opportunities, many students struggle to learn in them. Without proper support, these learners use system tools incorrectly and adopt suboptimal learning strategies. Typically, OELEs support students by providing hints: suggestions for how to proceed combined with information relevant to the learner’s situation. However, students often ignore or fail to understand such hints. To address this problem, we present an alternative approach to supporting students in OELEs that combines suggestions and assertions with guided skill practice. We demonstrate the feasibility of our approach through an experimental study that compares students who receive suggestions, assertions, and guided skill practice to students who receive no such support. Findings indicate that learners who received the scaffolds approached their tasks more systematically.

We investigate the combination of Augmented Reality (AR) with Intelligent Tutoring Systems (ITS) to assist with training for manual assembly tasks. Our approach combines AR graphics with adaptive guidance from the ITS to provide a more effective learning experience. We have developed a modular software framework for intelligent AR training systems, and a prototype based on this framework that teaches novice users how to assemble a computer motherboard. An evaluation found that our intelligent AR system improved test scores by 25% and that task performance was 30% faster compared to the same AR training system without intelligent support. We conclude that using intelligent AR tutor can significantly improve learning compared to traditional AR training.

Designing interactive learning experiences for informal educational settings, such as museums, presents challenges due to the particularities of context. In this presentation, the implications of applying user modeling and human computer interaction methods in the design of informal digital learning experiences will be highlighted. The discussion will be based on the example of the CHESS project and its formative and summative evaluation effort in two museums.

A central goal of education involves helping students develop deep understandings of complex models at the heart of core learning goals. Interestingly, an analogous goal of commercial recreational digital games involves helping players develop deep understandings of the models at the heart of those games. Given that games can motivate players to engage voluntarily over extended periods of time in developing understandings of complex game models, one may ask whether and how one might foster similar engagement with educational concepts and models. Much fanfare has accompanied claims about games’ potential for engagement and motivation, but many of those claims have focused on a shallow idea of “fun”. This talk takes a deeper view of motivation and learning by considering motivation and games through the lens of research on motivation to learn in classrooms. The talk then considers how research from the learning sciences, psychology, and science education can expand this motivation framework to scaffold the integration of intuitive and formal understanding through games for learning. Discussion of these ideas is framed in terms of examples from commercial game design and from our ongoing research and development of games to support physics learning. This talk builds on a submitted paper [3].

For 20+ years, Project LISTEN (www.cs.cmu.edu/~listen) has made computers listen to children read aloud, and help them learn to read. Along the way we have learned lessons about children, reading, speech technology, intelligent tutors, educational data mining, and doing AIED research in schools.

The new Industry and Innovation Track of the AIED 2013 conference includes submissions from commercial and entrepreneurial organizations that are putting AIED technologies into practice. As digital tutors enter the main stream, and demand increases for advanced capabilities such as automated assessment and personalized learning, there is increasing interest in learning products that incorporate artificial intelligence technologies. The Industry and Innovation Track is intended to attract innovators, practitioners, and technology adopters to the AIED conference to share lessons learned and best practices, and draw on emerging technologies and methods. It includes regular papers and posters, as well as late-breaking reports from fast-moving efforts.

The acquisition of procedural skills requires

learning by doing

. Ideally, a student would receive real-time assessment and feedback as he attempts practice problems designed to exercise the targeted skills. This paper describes an automated assessment and feedback capability that has been applied to training for a complex software system in widespread use throughout the U.S. Army. The automated assessment capability uses soft graph matching to align a trace of student actions to a predefined gold standard of allowed solutions, providing a flexible basis to evaluate student performance, identify problems, give hints, and suggest pointers to relevant tutorial documentation. Collectively, these capabilities facilitate self-directed learning of the training curriculum.

Instructor-mediated

training systems give end users direct control over content, increasing acceptance but introducing new technical challenges. Decreased opportunity for parameter estimation limits the utility of item-response or Bayesian approaches to adaptive assessment. We present four adaptive assessment algorithms that require little data about test item characteristics. Two algorithms present about half as many items as random selection before producing accurate skill estimates. These algorithms enable adaptive assessment in training settings where calibration data is sparse.

We compared two versions of an affect-sensitive embodied conversational agent for Aplusix, an intelligent tutoring system for algebra. The initial agent, Grimace v.1, was able to detect and respond to user affect, but it responded too quickly and too frequently. The second version of the agent, Grimace v.2 was less sensitive compared to the first version, in that it provided fewer interventions to engaged students, more evaluations of engagement, fewer evaluations of boredom. In a field test of the agent, students generally preferred version 2 over version 1.

This study investigated the learning and assessment efficacy of a physics video game we developed called Newton’s Playground. 165 8th and 9th graders played Newton’s Playground for roughly five hours. Findings include significant pre-post physics gains and notable correlations between performance in Newton’s Playground and physics pretest knowledge. Suggestions are given on how to develop assessments in video games to enhance learning.

Teaching introductory programming has challenged educators through the years. Although Intelligent Tutoring Systems that teach programming have been developed to try to reduce the problem, none have been developed to teach web programming. This paper describes the design and evaluation of the PHP Intelligent Tutoring System (PHP ITS) which addresses this problem. The evaluation process showed that students who used the PHP ITS showed a significant improvement in test scores.

Affect has been hypothesized to play a significant role in triggering engagement/disengagement during learning. In this paper, we study the inter-relationships between students’ affect (boredom, confusion, frustration, engaged concentration) and their engaged and disengaged behaviors (off-task, on-task solitary, on-task conversation, gaming the system). We study these relationships in the context of four different software programs, involving students of different ages, in order to increase confidence in the generalizability of the findings. Understanding these relationships might assist in maintaining students’ engagement over time.

This paper presents the approach to automated analysis of student argument diagrams to be used in the Genetics Argumentation Inquiry Learning (GAIL) system. Student arguments are compared to expert arguments automatically generated using an existing argument generator developed previously for the GenIE Assistant project. A prototype argument analyzer was implemented for GAIL. Weaknesses in student arguments are identified using non-domain-specific, non-content-specific rules that recognize common error types.

Intelligent tutoring systems need to know a mapping between particular problems and general domain concepts. Such mapping can be constructed manually by an expert, but that is time consuming and error prone. Our aim is to detect concepts automatically from problem solving times. We propose and evaluate two approaches: a model of problem solving times with multidimensional skill and an application of spectral clustering. The results show that it is feasible to construct a problem-concept mapping from solely the problem solving times and that the results of the analysis can bring an interesting insight.

We report a study on student’s attitudes to a visually androgynous in comparison to a male and a female Teachable Agent (TA). Results were that overall the androgynous agent was preferred over the female and male agents. A visually androgynous agent does not embody categorical gender attributes. At the same time it does not have to be

genderless

but instead represent

both

maleness and femaleness so that students can chose for themselves. Androgyny, in this sense, is potentially a way to have femaleness and maleness represented, with corresponding educational benefits such as role modelling and identification, without risking negative reinforcement of gender stereotypes.

Exploratory Learning Environments (ELE) provide a rich educational environment for students, but challenge teachers to keep track of students’ progress and to assess their performance. This paper proposes an algorithm that decomposes students complete interaction histories to create hierarchies of interdependent tasks that describe their activities in ELEs. It matches students’ actions to a predefined grammar in a way that reflects students’ typical use of ELEs, namely that students solve problems in a modular fashion but may still interleave between their activities. The algorithm was empirically evaluated on peoples interaction with two separate ELEs for simulating a chemistry laboratory and for statistics education. It was separately compared to the state-of-the-art recognition algorithm for each of the ELEs. The results show that the algorithm was able to correctly infer students’ activities significantly more often than the state-of-the-art, and was able to generalize to both of the ELEs with no intervention. These results demonstrate the benefit of using AI techniques towards augmenting existing ELEs with tools for analyzing and assessing students’ performance.

We created an application for the Apple iPad that families at a children’s museum used as they toured the museum. The application provided activities and explanation at various exhibit areas along with adaptive quizzes. We investigated their retention of the museum content and their attitudes toward the intervention. We found that content that provides an over-arching narrative to the museum experience along with the adaptive quizzes resulted in families enjoying the activities more, staying longer at the museum, and the children learning more information.

Model-tracing tutors were shown to be effective for the tutoring of problem solving tasks, but they usually lack the capability to provide feedback on learners’ off-path steps. In this paper, we define a method, inspired by Sierra, to diagnose many of the learners’ errors from their off-path steps. This method is implemented in Astus, a model-tracing tutor authoring framework. We show how Astus diagnose errors from off-path steps and use the resulting diagnostic to generate negative feedback.

Difficult materials overwhelm learners whereas easy materials deter advanced knowledge acquisition. Toward the goal of automatic assessment of learning materials, we conducted a laboratory experiment involving 50 college students recruited from two universities in Korea using 115 PowerPoint files. On the basis of the qualitative analysis results, we propose a model of learning difficulty, distinguishing measurable factors from non-measurable factors. The most influential factors for the easiest and the hardest learning materials are also identified and compared. The study findings have implications for educational service providers who need to automatically classify learning materials based on their innate difficulties.

We have developed a mobile testing system using computerized adaptive testing for assessing learning at museums, parks, and other sites in the field. Computerized adaptive testing is a form of computer-based testing that progressively estimates an examinee’s ability from his/her answer history and uses that ability to present test items making ability estimation even more accurate. Field-testing, however, requires activities such as observing and searching at specific positions within a site, which requires the learner to move about to get to those positions. Moreover, the time that can be spent taking such an on-site test is usually limited, which means that the test may end before a sufficient number of test items can be answered thereby decreasing the accuracy of ability estimation. In response to these issues, we formalize for field-testing purposes an optimization problem called the traveling purchaser problem (TPP) that incorporates graph theory and propose an computerized adaptive testing system using TPP.

This study presents Quantitative Field Observations (QFOs) of educationally relevant affect and behavior among students at three schools using Reasoning Mind, a game-based software system designed to teach elementary-level mathematics. High levels of engagement are observed. Possible causes for these high levels of engagement are considered, including the interactive pedagogical agent and other design elements.

MONSAKUN is software for learning by problem-posing in arithmetical word problems where a learner poses a problem by selecting and combining sentence cards from a given set of sentence cards. It is not easy task to prepare the sets of the sentence cards manually because it is necessary to evaluate all combinations. This paper describes an analyzer of a set of sentence cards. Experimental evaluation of the analyzer is also reported.

BioWorld is a computer-based learning environment that supports medical students in their clinical reasoning about virtual cases. We model the regulatory processes sudents use in the context of BioWorld in an effort to see when they ask for tutorial guidance and how guidance can be improved. BioWorld provides assistance using an artificial physician to deliver hints when students request a consult. We analyzed the concurrent think aloud protocols and log-file trace data collected from 30 students who solved 3 cases with BioWorld. Our findings highlight the antecedents and functions of regulatory activities involved in help-seeking. We discuss the implications for tailoring the content of the hints provided by the consult tool to the specific needs of different students.

This poster describes Rimac, a natural-language tutoring system that engages students in dialogues that address physics concepts and principles, after students have solved quantitative physics problems. We summarize our approach to deriving decision rules that simulate the highly interactive nature of human tutoring, and describe a pilot test that compares two versions of Rimac: an experimental version that deliberately executes these decision rules within a Knowledge Construction Dialogue (KCD) framework, and a control KCD system that does not intentionally execute these rules.

We have developed a methodology for constructing domain-level expert knowledge bases automatically through crowdsourcing. This approach involves collecting and analyzing the work of numerous students within an intelligent tutor and using an intelligent algorithm to coalesce data to construct the domain model. This evolving expert knowledge base (EEKB) is then utilized to provide expert coaching and tutoring with future students. We can compare the knowledge created in human crafted expert knowledge bases (HEKB) with knowledge resulting from our knowledge acquisition algorithm to judge quality. We find that our EEKB models have qualities that rival that of the human crafted knowledge bases and can be generated in significantly less time. We have built four unique knowledge bases using this methodology. This paper provides a pithy high-level overview of our approach along with some findings.

In this paper, we propose the provision of feedback in Intelligent Tutoring Systems in absence of a formalized domain model. In a Wizard of Oz experiment, a human tutor gave feedback to students based on sample solutions applying two strategies which aimed to encourage learners’ self-reflection. We discuss possibilities to automate the methods of feedback provision using domain-independent proximity measures.

This paper presents a computer-supported approach for providing ‘enhanced’ discovery learning in informal settings like museums. It is grounded on a combination of gesture-based interactions and artwork-embedded AIED paradigms, and is implemented through a distributed architecture.

In this paper, a new data-driven model to measure procedural knowledge is described. The model is based on Item Response Theory. The main idea behind this new model is to establish an analogy between the testing and the problem solving environment. For this purpose, we model each problem (or exercise) solution path as a directed graph where nodes are states of the problem and edges, transitions between states (i.e. the actions accomplished by the student). We can match this model with testing by seeing each node as a question and each edge as choices within the questions.

Alternate reality games (ARGs) are a promising new approach for increasing student engagement; however, automated methods for analyzing and optimizing game play are non-existent. We captured the player communication generated by a recent STEM-focused ARG that we piloted in a Los Angeles charter high school. We used shallow sentiment analysis to gauge the levels of various emotions experienced by the players during the course of the game. Pre/post-game surveys gauged whether the game narratives had any effect on student engagement and interest in STEM topics.

This paper reports a hybrid method for data-driven instructional design, a method that combines Principle Components Analysis (PCA), Hidden Markov Models (HMM), and Item Response Theory (IRT). PCA is used to identify instructional objectives as well as potential student states, HMMs are used to identify dynamics between states, and IRT is used to construct measurements of state. We report on the architecture of the system along with preliminary results.

While modeling dynamic systems in an efficient manner is an important skill to acquire for a scientist, it is a difficult skill to acquire. A simple step-based tutoring system, called AMT, was designed to help students learn how to construct models of dynamic systems using deep modeling practices. In order to increase the frequency of deep modeling and reduce the amount of guessing/gaming, a meta-tutor coaching students to follow a deep modeling strategy was added to the original modeling tool. This paper presents the results of two experiments investigating the effectiveness of the meta-tutor when compared to the original software. The results indicate that students who studied with the meta-tutor did indeed engage more in deep modeling practices.

Self-regulated learners have been shown to learn more effectively. However, it is not easy to become self-regulated because learners have to be capable of observing and evaluating their thoughts, actions and behaviors while learning. In this work, we used Q-learning to reveal the effectiveness or ineffectiveness of a learning behavior that carries over learning episodes. We also showed different types of effective learning behavior discovered and how they were differentiated. Providing learners with knowledge about learning behavior effectiveness can help them observe how strategy selection affects their performance and will help them select more appropriate strategies in succeeding learning episodes for better future performance.

Online discussion board has become increasingly popular in higher education. As a step towards analyzing the role that students and instructors play during the discussion process and assessing students’ learning from discussions, we model different types of contributions made by instructors and students with a dialogue-state model. By analyzing frequent Q&A discussion patterns, we have developed a graphic model of dialogue states that captures the information role that each message plays, and used the model in analyzing student discussions, presenting several viable ap-proaches including CRF, SVM, and decision tree for the state classification. Such analyses can give us a new insight on how students interact in online discussions and kind of assistance needed by the students.

In this article we propose a semi-automatic generator of self-assessment exercises. This work is part of the CLAIRE project the aim of which is to design a collaborative authoring platform for pedagogic content. The proposed generator of exercises allows the author (usually a teacher) to create a model of exercise according to his/her pedagogic objectives. This model is automatically instantiated to produce several different exercises that evaluate the same skills. The learner’s answer is automatically and instantly evaluated by the system. He/she thus receives immediate feedback on his/her skills. The distinctive feature of this generator is that the proposed types of exercise are independent of the domain, which allows them to be used for many different subjects and levels. In addition, domain knowledge is used to facilitate the author’s task when the model of exercises and the diagnostic are designed.

Brainstorming is a widely-used group technique to enhance creativity. Interactive tabletops have the potential to support brainstorming and, by exploiting learners’ trace data, they can provide Open Learner Models (OLMs) to support reflection on a brainstorming session. We describe our design of such OLMs to enable an individual to answer core questions: C1) how much did I contribute? C2) at what times was the group or an individual stuck? and C3) where did group members seem to ‘spark’ off each other? We conducted 24 brainstorming sessions and analysed them to create brainstorming models underlying the OLMs. Results indicate the OLM’s were effective. Our contributions are: i) the first OLMs supporting reflection on brainstorming; ii) models of brainstorming that underlie the OLMs; and iii) a user study demonstrating that learners can use the OLMs to answer core reflection questions.

Our interest in this work is to automatically predict whether peer ratings have high or low agreement in terms of disparity with instructor ratings, using solely features extracted from quantitative peer ratings and text-based peer comments. Experimental results suggest that our model can indeed outperform a majority baseline in predicting low versus high rating disparity. Furthermore, the reliability of both peer ratings and comments (in terms of peer disagreement) shows little correlation to disparity.

This study investigates methods to automatically assess the features of content texts within an intelligent tutoring system (ITS). Coh-Metrix was used to calculate linguistic indices for texts (n = 66) within the reading strategy ITS, iSTART. Coh-Metrix indices for the system texts were compared to students’ (n = 126) self-explanation scores to examine the degree to which linguistic indices predicted students’ self-explanation quality. Initial analyses indicated no relation between self-explanation scores on a given text and its linguistic properties. However, subsequent analyses indicated the presence of robust text effects when analyses were separated for high and low reading ability students.

Motivated by disturbing national educational statistics, the newly adopted Common Core State Standards [1] prioritize reading instruction across the content areas. This will significantly increase students’ exposure to informational texts that are notorious for low comprehension rates and less than engaging content. Given the substantial literature supporting the positive relationship between situational interest and reading comprehension [2,3], this study will address whether game-based learning environments generate situational interest and, more importantly, whether the produced situational interest increases students’ reading comprehension for informational texts. Using an explanatory sequential mixed methods design, eighth-grade students’ situational interest and comprehension of texts embedded within a science game-based learning environment will be measured. Implications for this research include the design of intelligent game-based learning environments, the extent to which game elements generate situational interest, and techniques for capitalizing on this situational interest by intelligently and automatically integrating texts to challenge each reader.

Web brings about a lot of opportunities for learners to investigate a topic with Web resources to learn. Such investigative learning process involves creating a scenario that explains what to and how to investigate with Web resources. However, it is quite difficult for the learners to create their own learning scenario concurrent with knowledge construction from the contents of the resources. The main issue addressed in this paper is how to scaffold learning scenario creation. This paper presents a model of investigative learning, which induces learners to create the learning scenario by decomposing the topic into sub-topics to be learned while searching and learning the Web resources. This paper also demonstrates an interactive learning scenario builder, which provides a scaffold for the learners to build their own scenario in learning with Web resources.

Causal reasoning is difficult for middle school students to grasp. In this research, we wanted to test the possibility of using machine learning for modeling students’ causal reasoning in a virtual environment designed to assess this skill. Our findings suggest it is possible to use machine learning to emulate student pathways that are able to predict their causal understanding.

This paper presents a quantitative study on the use of Topolor - a prototype that introduces Web 2.0 tools and Facebook-like appearance into an adaptive educational hypermedia system. We present the system design and its evaluation using system usability scale questionnaire and learning behavior data analysis. The results indicate high level of student satisfaction with the learning experience and the diversity of learning activities.

To propose a new CAT(Computerized Adaptive Testing) algorithm, we regard selecting an item from an item bank as a decision making in Bayesian theory and propose a new item selection criterion we call “expected value of test information” (EVTI). The unique features of EVTI are that it 1) maximizes the prediction utility of an examinee’s ability estimation and 2) generates a decision tree with an item selection order based on the examinee’s responses. The CAT references the tree and then instantaneously selects and presents the optimal item from an item bank. Simulation results showed that the proposed method performed better than conventional methods.

Social reference management systems, such as Mendeley, Zotero or CiteUlike offer many services to their users: finding and managing references, finding other users, grouping users with similar research interests. Harnessing these systems to build personalized recommendations could be useful both for novice researchers (graduate students) and for experienced researchers to keep them updated in their areas. We propose a trust-based hybrid recommender system that infers the user’s ratings of papers and builds a social trust network for an area of recent research interest. We will evaluate the accuracy of predicting the most relevant papers for the current interest and experience level of the researcher and the user satisfaction of the system.

To accurately model and represent student knowledge is a challenging task, and it is especially difficult for ill-defined domains, characterized by uncertainty and ambiguity. We propose a way to represent students’ positions as they analyze case studies in the Professional Ethics domain. We designed our representation with the goal not only to model students’ knowledge, but also to encourage positive behaviour in students, and increase the quality of their case analyses. As our experiment demonstrates our representation was successful in stimulating certain desired actions in students, but didn’t seem to significantly affect the quality of students’ case analyses.

It is common practice to compare gain scores in order to determine the effectiveness of adding features to a training system. Here we argue that relying on one measure of overall system effectiveness may result in overlooking valuable lessons available from a comparison of different versions of a system. To illustrate our point, we present the results of comparing a Natural Language Processing (NLP) based adaptive feedback system to a system that does not utilize NLP capabilities. We show that, while there were no learning gain differences between the two systems, the correlates to gain were different. In the non-NLP system, only student performance during the training was correlated to learning gain. In the adaptive system, more variables correlated with learning, including measures of system capability and student satisfaction. This level of analysis suggests that the two systems are not equivalent and points us towards modifications that may improve effectiveness.

This paper presents a domain independent question generation and interaction procedure that automatically generates multiple-choice questions for conceptual models created with Qualitative Reasoning vocabulary. A Bayesian Network is deployed that captures the learning progress based on the answers provided by the learner. The likelihood of concepts being known or unknown on behalf of the learner determines the focus, and the question generator adjusts the contents of its questions accordingly. As a use case, the Quiz mode is introduced.

Building on existing work on artificial tutors with human-like capabilities, we describe the EMOTE project approach to harnessing benefits of an artificial embodied tutor in a shared physical space. Embodied in robotic platforms or through virtual agents, EMOTE aims to capture some of the empathic and human elements characterising a traditional teacher. As such, empathy and engagement, abilities key to influencing student learning, are at the core of the EMOTE approach. We present non-verbal and adaptive dialogue challenges for such embodied tutors as a foundation for researchers investigating the potential for empathic tutors that will be accepted by students and teachers.

As we are entering the age of open social e-learning environments, group (peer) mentorship becomes an increasingly important mode of learning. The academic peer review system can be viewed as a group mentorship system. Peer reviews have been used for over a century by the research community to provide not only quality control for publishing new research contributions, but also as a way to provide constructive feedback to the authors and help them to improve their work. There are two critical questions that need to be addressed in both peer-review and group peer mentorship: 1) how to motivate reviewers (mentors) to give serious, detailed and constructive feedback, 2) how to find good reviewers (mentors) for a particular author (mentee). This research addresses the above questions in the context of a group online peer-mentorship system aimed at improving the writing skills of university students using a conference peer review model.

Emotions detection and their management are key issues to provide personalize support in educational scenarios. Literature suggests that combining several input sources can improve the performance of affect recognition. To gain a better understanding of this issue, we carried out a large scale experiment in our laboratory where about 100 participants performed several mathematical exercises while emotional information was gathered from different input sources, including a written emotional report. As a first step, we have explored emotions detection from traditional methods by combining analysis of user behavior when typing this report with sentiment analysis on the text. Moreover, an expert labeled these reports. All these data were used to feed several machine learning algorithms to infer user’s emotions. Preliminary results are not conclusive, but lead some light on how to proceed with the analysis.

We examine whether there are differences between students regarding the utility of learning from visual representations (illustrations or graphs) within the context of a typed natural language-based conceptual physics tutoring system. Showing half of the students only illustrations and the other half only graphs, we found that novices benefited from illustrations, whereas non-novices showed no difference.

This study investigates the relationships between student entrainment to a tutoring dialogue system and learning. By finding the features of prosodic entrainment which correlate with learning, we hope to inform educational dialogue systems aiming to leverage entrainment. We propose a novel method to measure prosodic entrainment and find specific features which correlate with user learning. We also find differences in user entrainment with respect to tutor voice and user gender.

We propose a method and a first authoring tool to assist the design and implementation of diagnostic techniques. This method is independent from the domain and allows building more than one technique at once. The method is based on knowledge representation and a semi-automatic machine learning algorithm. We tested the method in two domains, surgery and reading English. Techniques built with our method beat the majority class in terms of accuracy.

Technological maturity allows, nowadays, to plan increasingly complex applications. However, on the one hand, such complexity increases the difficulty to propose simultaneous, pedagogical and narrative control as well as some freedom of actions. On the other hand, that complexity makes difficult the tracking of a learner’s path. To overcome this limitation, we propose in this paper

1

) a tracking system of learners’ actions along with analysis and automatic diagnosis tools of learners’ performances and

2

) a scripting model for training in virtual environments combining both a pedagogical control and the emergence of pertinent learning situations.

Multiple graphical representations are ubiquitous in educational materials because they serve complementary roles in emphasizing conceptual aspects of the domain. Yet, to benefit robust learning, students have to understand each representation and make connections between them. We describe research-based principles for the use of multiple graphical representations within intelligent tutoring systems (ITSs). These principles are the outcome of a series of iterative classroom experiments with the Fractions Tutor with over 3,000 students. The implementation of these principles into the Fractions Tutor results in robust conceptual learning. To our knowledge, the Fractions Tutor is the first ITS to use multiple graphical representations by implementing research-based principles to support conceptual learning. The instructional design principles we established apply to ITSs across domains.

User modeling in AIED has been extended in the past decades to include affective and motivational aspects of learner’s interaction in intelligent tutoring systems. In order to study those factors, various detectors have been created that classify episodes in log data as gaming, high/low effort on task, robust learning, etc. In this article, we present our method for creating a detector of shallow modeling practices within a meta-tutor instructional system. The detector was defined using HCI (human-computer interaction) task modeling as well as a coding scheme defined by human coders from past users’ screen recordings of software use. The detector produced classifications of student behavior that were highly similar to classifications produced by human coders with a kappa of .925.

Many students have difficulty with inquiry and difficulty with interpreting data, in particular. Of interest here is confirmation bias, i.e., when students won’t discard a hypothesis based on disconfirming results, which is in direct contrast to when students make a discovery, having originally made a scientifically inaccurate hypothesis. The goal of the present study is to better understand these two data interpretation patterns and autoscore them. 145 eighth grade students engaged in inquiry with a state change microworld. Production rules were written to produce model-tracing in order to identify when students either made a discovery or engaged in confirmation bias. Interesting to note was an emerging pattern wherein many of the same students made discoveries across the four inquiry tasks. These data are important for performance assessment of inquiry and suggest that students may need adaptive scaffolding support while engaging in data interpretation.

Nowadays, teachers tend to build their own didactic local repository composed by learning objects retrieved from web repositories or, in most cases, by self-made didactic material. In this way they do not share their teaching experience, loosing a precious shortcut to a fast professional update and to an improvement of their teaching activity. In this paper we address the problem of helping teachers to retrieve didactic material from a repository through a didactic social network where teachers with similar Teaching Styles, can help each other in retrieving educational material. To this aim a teaching-styles based social network is built following the Grasha

TS

paradigm. We present a first evaluation of the network embedded in a web application.

Analysis of turn-taking in tutoring dialogues can be helpful to understand the procedure of tutoring and also the influence with regard to demographics between students and the tutor. In this research, we analyze turn-taking behavior between students in a human-human spoken tutoring system. Our approach is to learn turn-taking models using dialog activity state sequences and then we measure the association of these models with students’ demographic features (gender and education). The experimental results show that female students speak simultaneously longer with the tutor than male students, female activities are less than male activities and also the tutor speaks longer with students who have lower pre-test score.

In this paper, we present a web-based automatic marking system that aims to assist the tutor in assessing the performance of students in interactive exercises related to breadth-first search (BFS) and depth-first search (DFS) algorithms. The system has been tested on a number exercises for BFS and DFS search algorithms and its performance has been compared against that of an expert tutor. The experimental results are quite promising.

Penalties are frequently used in games and rarely in tutors, creating a dilemma for designers seeking to combine games and tutors to maximize fun and learning. On the one hand, penalties can be frustrating and waste instructional time, on the other, they may increase excitement and prevent gaming. This study tested the effects of penalties on learning and interest. In a randomized, controlled experiment with a two-group, between subjects design, 100 University students played two versions of a game with an embedded tutor, with and without penalties that forced students to replay parts of the game. Results showed that penalties decreased learning and interest. These findings suggest a minimize penalties principle for designing cognitive games.

The present study evaluates the effectiveness of

Just-In-Time

Hints

(

JITs

) by testing two competing hypotheses about learning from errors. The

tutor-remediation hypothesis

predicts that students learn best when a tutoring system immediately explains why an entry is incorrect. The

self-remediation hypothesis

predicts that learning is maximized when learners attempt to correct their own errors. The

Cognitive Tutor

was used to test these hypotheses because it offers both JITs, which map onto the tutor-remediation hypothesis, and flag feedback, which maps onto the self-remediation hypothesis. To evaluate the effectiveness of JITs, we conducted a naturalistic experiment where learning from older versions of the software, which did not include specific JITs, was contrasted with a later version that included the JITs. The results suggest JITs reduced the frequency of errors; however, this observation was qualified by an aptitude-treatment interaction whereby high- and low-prior knowledge students differentially benefited from JIT availability.

We report on two studies that suggest that showing reports of student progress at key moments of deactivating negative emotions (boredom or lack of excitement) can help improve students’ affective state and learning behavior while using an adaptive math tutoring system. The studies involved 160 middle-school students in public schools in Arizona and California who reported higher levels of interest and excitement and also demonstrated more positive engagement behavior when using the intervention progress pages.

We present a study that addressed if providing students with scaffolding about how to “integrate” science text and animations impacts content learning. Scaffolding was delivered by a pedagogical agent and driven by student’s eye gaze movements (compared to controls).We hypothesized that students in the pedagogical agent condition would engage in richer learning as evidence by a more “integrated” pattern from text to animation and back, etc. In addition to eye gazes we collected pre- and post test knowledge about the domain, and open responses to explanation-type questions. We are currently analyzing these data.

Erroneous examples, an unusual and challenging form of learning material, are arguably a type of desirable difficulty for students that could lead to deeper learning. In a series of studies we have done over the past three years involving web-based math instruction, the learning benefits of erroneous examples we have observed occured on delayed tests, as occurs in the desirable difficulties literature. This short paper briefly reviews the literature, summarizes our results, and speculates on how an adaptive version of our materials could better leverage desirable difficulties theory and lead to deeper student learning.

Successfully promoting engagement within learning environments is a subject of increasing attention within the AI in Education community. Evidence is mounting that game-based learning environments hold great potential to engage students, but disengaged behavior is still observed. Devising adaptive strategies to re-engage students in the learning task is a key open research question. Toward that end, this paper examines the collaborative behavior of pairs of middle school students solving game-based computer science problems. We examine the dialogue moves that were used by a more engaged learner to repair a partner’s disengagement and consider the implications that these strategies may have for designing collaborative game-based learning environments.

We report on text processing and machine learning methods with the goal of building classifiers for social deliberative skill, i.e. the capacity to deal productively with heterogeneous goals, values, or perspectives. Our corpus includes online deliberative dialogue from three diverse domain contexts. We use the LIWC and CohMetrix linquistic analysis tools to generate feature sets for machine learning. We report on our evaluation of various machine learning algorythms, feature selection methods, and cross-domain training methods.

Extended interactions with a pedagogical agent (PA) assisting students to enact cognitive and metacognitive self-regulated processes requires the system to adapt the types and frequency of scaffolding. We compared learners’ perception of PAs’ prompts with MetaTutor, a hypermedia adaptive learning environment, with 40 undergraduates randomly assigned to one of three conditions: non-adaptive prompting (NP), frequency-based adaptive prompting (FP) and frequency and quality-based adaptive prompting (FQP). Results indicate learners are unable to reliably perceive differences in the number of prompts received, though these differences are reflected in positive outcomes in terms of SRL processes enacted and learning gains, and negative outcomes in terms of self-reported satisfaction. Preliminary results indicated that more frequent, but adaptive prompting is an efficient scaffolding strategy, despite negatively impacting learners’ satisfaction.

This paper describes our experiments and analysis of utilizing class-level features to predict student performance for retention tests. There are two aspects that make this paper interesting. First, instead of focusing on short-team performance, we investigated student performance after a delay of at least 7 days. Second, we explored several class-level features that can be captured in intelligent tutoring systems (ITS), and we showed that some of them have encouraging predictive power. With the help of class-level features, the prediction result indicated an improvement from an R² of 0.183 with a normal feature set to an R² value of 0.224.

Traditional studies of intelligent tutoring systems have focused on their use in the classroom. Few have explored the advantage of using ITS as a web-based homework (WBH) system, providing correctness-only feedback to students. A second underappreciated aspect of WBH is that teachers can use the data to more efficiently review homework. Universities across the world are employing these WBH systems but there are no known comparisons of this in K12. In this work we randomly assigned 63 thirteen and fourteen year olds to either a traditional homework condition (TH) involving practice without feedback or a WBH condition that added correctness feedback at the end of a problem and the ability to try again. All students used ASSISTments, an ITS, to do their homework but we ablated all of the intelligent tutoring aspects of hints, feedback messages and mastery learning as appropriate to the two practice conditions. We found that students learned reliably more in the web-based homework condition and with an effect size of 0.56. Additionally, teacher use of the homework data lead to a more robust and systematic review of the homework. Future work will further examine modifications to WBH to further improve learning from homework and the role of WBH in formative assessment.

Designing dialogue systems that engage in rich tutorial dialogue has long been a goal of the intelligent tutoring systems community. A key challenge for these systems is determining when to intervene during student problem solving. Although intervention strategies have historically been hand-authored, utilizing machine learning to automatically acquire corpus-based intervention policies that maximize student learning holds great promise. To this end, this paper presents a Markov Decision Process (MDP) framework to learn an intervention policy capturing the most effective tutor turn-taking behaviors in a task-oriented learning environment with textual dialogue. The model and its learned policy highlight important design considerations, including maintaining tutor engagement during student problem solving and avoiding multiple consecutive interventions.

Operation ARA is a serious game that teaches scientific inquiry using natural language conversations. Within the context of the game, students completed up to two distinct training modules that teach either didactic or applied conceptual information about research methodology (e.g., validity of dependent variables, need for control groups). An experiment using a 4-condition between-subjects pretest-interaction-posttest design was conducted in which 81 undergraduate college students interacted with varying modules of Operation ARA. The four conditions were designed to test the impact of the two distinct modules on different types of learning measured by multiple-choice, short answer, and case-based assessment questions. Results revealed significant differences on training condition and learning gains on two of the three types of questions.

One of the most popular methods for modeling students’ knowledge is Corbett and Anderson’s [1] Bayesian Knowledge Tracing (KT) model. The original Knowledge Tracing model does not allow for individualization. In this work, we focus on comparing two different individualized models: the Student Skill model and the two-phase model, to find out which is the best for formulating the individualization problem within a Bayesian networks framework.

This study proposes a first step toward the automated realization of student tracking, i.e., dividing a class of students into several streams according to criteria such as overall strength, specific abilities, etc. Our study is based on a database of 214 students who took a 64-question multiple choice exam. We examine a family of tracking schemes based on the k means algorithm but differing in feature selection and attribute weighting. We compare these schemes to a naïve scheme based solely on overall grades and a human-based scheme that applies k means to content-based features assigned by experienced teachers.

Introductory computer science courses are a valuable resource to students of all disciplines. While we often look at students’ end products to judge their proficiency, little analysis is done on the most integral aspect of learning to programming, the process. We also have a hard time quantifying how students’ programming changes over the course of a semester. In order to address these we show how a process-oriented analysis can identify meaningful trends in how programmers develop proficiency across various assignments.

To support introductory Java programming students in preparing for their exams, we developed Knowledge Maximizer as a concept-based problem sequencing tool that considers a fine-grained concept-level model of student knowledge accumulated over the semester and attempts to bridge the possible knowledge gaps in the most efficient way. This paper presents the sequencing approach behind the Knowledge Maximizer and its classroom evaluation.

We annotated and analyzed Worked Out Examples (WOEs) in a corpus of tutoring dialogues on Computer Science data structures. We found that some dialogue moves that occur within WOEs, or sequences thereof, correlate with learning. Features of WOEs such as length also correlate with learning for some data structures. These results will be used to augment the tutorial tactics available to iList, an ITS that helps student learn linked lists.

Recent research in CS education has leveraged machine learning techniques to capture students’ progressions through assignments in programming courses based on their code submissions [1, 2]. With this in mind, we present a methodology for creating a set of descriptors of the students’ progression based on their coding styles as captured by different non-semantic and semantic features of their code submissions. Preliminary findings show that these descriptors extracted from a single assignment can be used to predict whether or not a student got help throughout the entire quarter. Based on these findings, we plan on developing a model of the impact of teacher intervention on a student’s pathway through homework assignments.

The research question addressed in this paper is: Given a problem, can we automatically predict how difficult the problem will be to solve by humans? We focus our investigation on problems in which the difficulty arises from the combinatorial complexity of problems. We propose a measure of difficulty that is based on modeling the problem solving effort as search among alternatives and the relations among alternative solutions. In experiments in the chess domain, using data obtained from very strong human players, this measure was shown at a high level of statistical significance to be adequate as a genuine measure of difficulty for humans.

We present in this paper CALM (ContextuAlized Learning through Mobility), an ubiquitous learning environment for museum visits. This environment uses semantic proximities over a semantic model of the domain (cultural heritage) and context (e.g. position in the museum, activity) to offer contextualized activities. Our proposal aims to provide learners with situated interactions, while giving teachers the opportunity to integrate learning objectives that will influence the proposed interactions. To that end, we propose to use semantic rules that enables a loosely-based control of learning activities by the teacher.

Mobile augmented reality applications are increasingly utilized as a medium for enhancing learning and engagement in history education. Although these digital devices facilitate learning through immersive and appealing experiences, their design should be driven by theories of learning and instruction. We provide an overview of an evidence-based approach to optimize the development of mobile augmented reality applications that teaches students about history. Our research aims to evaluate and model the impacts of design parameters towards learning and engagement. The research program is interdisciplinary in that we apply techniques derived from design-based experiments and educational data mining. We outline the methodological and analytical techniques as well as discuss the implications of the anticipated findings.

Although several Intelligent Tutoring Systems (ITS) have been built to teach students how to write programs, few focus on teaching students the skills required to debug faulty code. Indeed, outside of general debugging advice, it is also a skill seldom outright taught in the classroom. This paper discusses a web-based ITS to teach introductory level Computer Science students debugging skills, using and teaching case-based reasoning.

Language learners are often isolated because of their inability to communicate. Adaptive mobile communication support tools could be used to scaffold both their interaction with others and their vocabulary acquisition. I propose the exploration of a new tool that is designed to meet this need.

Concept mapping is a tool used in many classrooms and highly researched in the field of education. However, there are fewer concept mapping studies in the field of artificial intelligence in education, specifically within intelligent tutoring systems. Two studies highlight the important roles that concept maps and other non-linear organizers play in learning. Concept maps provide students with a macrostructure view of the information as well as allow students to easily see relationships between concepts. Students generating material for a concept map has shown high learning gains; however, students creating maps from scratch or students being provided a completed map has not seen such positive effects. The proposed study looks at the importance of the links, or relationships between concepts, within concept maps. We plan to provide students with partially filled in concept maps as note-taking devices to investigate how much and what kind of assistance or scaffolding is needed.

This PhD project lays the groundwork for a future VLE that adaptively introduces

discrepancies

(i.e. novel or rule-violating occurrences) in order to support young children with

autism spectrum conditions

(ASC) in practicing foundational social skills. This paper suggests a taxonomy of discrepancy types and briefly summarises a completed analysis of discrepancy-detection in existing video data from 8 children with ASC using the ECHOES VLE. It then describes planned future work, which will explore possible types of discrepancies for exploratory social content (as present in ECHOES) and address other key questions about how they might impact this group of learners, and be incorporated into the design of a future VLE. It also considers how the current work relates to existing literature on metacognition and use of erroneous worked examples in tutoring systems.

Applied Behavior Analysis-based early interventions are evidence based, efficacious therapies for autism. They are, however, labor intensive and often inaccessible at the recommended levels. In this paper we present ongoing doctoral research aimed at development of the formal, computational representation for Applied Behavior Analysis (ABA) that could serve as a reasoning foundation for intelligent-agent mediated ABA therapies. Our approach is to formulate the representation of ABA dynamics and concepts as a process ontology expressed in a controlled natural language (CNL). As an ontology language, CNL is not only a machine interpretable, logically sound reasoning foundation, but also understandable and editable by human users.

Open-ended learning environments (OELEs) are learner-centered, and they offer students opportunities to take part in authentic and complex problem-solving tasks. However, learners typically struggle to learn with OELEs without proper adaptive scaffolds. This paper describes research and development related to designing real-time algorithms for diagnosing students’ needs in OELEs and responding with appropriate adaptive scaffolds.

Mind wandering is the attentional shift from task-related thought to task-unrelated thoughts and can have disastrous effects on learning. Previous research has found that mind wandering is detrimental to comprehension during reading. However, to our knowledge, no research has investigated mind wandering during interactive educational learning environments. This paper discusses preliminary studies and a proposed line of research that aims to investigate models of mind wandering in the context of an interactive computerized learning environment. The proposed three-phase plan will develop a deep understanding of the factors that influence mind wandering with an eye towards developing intelligent learning environments that automatically detect and respond to minds when they begin to wander.

Collaborative and individual instruction may support different types of knowledge. Optimal instruction for a subject domain may therefore need to combine these two modes of instruction. There has not been much research, however, on combining individual and collaborative learning with Intelligent Tutoring Systems (ITSs). A first step is to expand ITSs for collaborative learning. This paper investigates the expansion of the Cognitive Tutor Authoring Tools to include collaborative components for example-tracing tutors. The tools were enhanced to support flexible use of collaboration scripts so different learning goals can be supported. We introduce the collaboration features supported and describe an initial pilot study using the new features in a fractions ITS.

This project focuses on developing methods to automatically detect and respond to emotions that students experience while developing writing proficiency with computerized environments. We describe progress that we have already made toward detecting affect during writing using keystroke analysis, stable traits, and task appraisals. We were able to distinguish boredom from engagement with an accuracy of 38% above random guessing. Our next goal is to improve the accuracy of our classifier. We plan to accomplish this through an exploration of higher level features such as sequences of character types. Ultimately we hope to develop a system capable of both detecting affect and influencing affect through interventions and experimentally testing this system.

Students learning computer programming must learn difficult concepts via complex problem-solving activities which elicit strong emotional responses. In this research we explore the affective states that occur while learning computer programming, the events that precede them, and the outcomes that are influenced by them. The data collected in current and future research will be used to create an affect-sensitive intelligent tutoring system which will be better able to maximize learning gains in novice computer programmers and improve their perception of computer science via intelligent handling of emotion.

Access to and effective use of relevant information and continuously learning is an integral part of graduate students’ daily lives. However, when searching for learning materials, students face challenges selecting relevant information because of the tremendous increase of learning resources over the last few years. This research proposes a novel methodology that aids graduate students to find appropriate sources of information in their lifelong learning endeavors by using people-to-people recommender system (RS) techniques. The people-to-people RS aims to help graduate students by suggesting persons (peers/experts) to contact about the problems they are facing when the problems are not easily identifiable from static fact sheets (a.k.a, question and answer or frequently asked questions).

Our tutoring system for fraction addition uses dynamic pictorial representations that reflect student-inputted quantities. However, students had difficulty interpreting the pictorial feedback. Surprisingly, we found that including symbolic numbers with the pictures decreased performance. We hypothesize that students’ difficulty may stem from insufficient domain knowledge, or insufficient metacognitive skills to use conceptual knowledge to check their work.

Computational Thinking (CT) defines a domain-general, analytic approach to problem solving, combining computer science concepts with practices central to modeling and reasoning in STEM (Science, Technology, Engineering and Mathematics) domains. In our research, we exploit this synergy to develop CTSiM (Computational Thinking in Simulation and Modeling) - a cross-domain, visual programming and agent based, scaffolded environment for learning CT and science concepts simultaneously. CTSiM allows students to conceptualize and build computational models of scientific phenomena, execute the models as simulations, conduct experiments to verify the simulation behaviors against ‘expert behavior’, and use the models to solve real world problems.

Educational games intend to make learning more enjoyable, but potentially compromise learning by consuming both instructional time and student cognitive resources. Therefore, instead of creating an educational game, we are exploring different ways of integrating game-like elements in a computer tutor. We are experimenting with cognitive, metacognitive and affective modes of such game-like interventions. We are also exploring causal mechanisms of how different interventions lead to the desired learning outcomes.

Rimac is a natural-language intelligent tutoring system that engages students in dialogues that address physics concepts and principles, after they have solved quantitative physics problems. Much research has been devoted to identifying features of tutorial dialogue that can explain its effectiveness (e.g., [1]), so that these features can be simulated in natural-language tutoring systems. One hypothesis is that the highly interactive nature of tutoring itself promotes learning. Several studies indicate that our understanding of interactivty needs refinement because it cannot be defined simply by the amount of interaction nor the granularity of the interaction but must also take into consideration how well the interaction is carried out (e.g., [2]).

AutoTuto 2013 is an advanced version of the intelligent tutoring system, proven to be effective in empirical tests. AutoTutor 2013 is an agent-based online system with rich media among multiple agents and learners. AutoTutor delivers knowledge by means of multi-turns of conversions with the assist of the comprehensive media technology, including images, diagrams, audios, videos and other interactive presentations developed by Media Semantics Character Builder program.

We have developed an adaptive communication support tool that also supports vocabulary acquisition. This tool is called VocabNomad; it is one of the few mobile assisted language learning tools that aims to support the call for activities that are fundamentally different than those provided by paper and pencil or computer assisted language learning [1]. VocabNomad meets this call by trying to support the communication of immigrants who are isolated from their surrounding environment because of their limited English language proficiency. In the US, these English language learners (ELL) make up more than 20 percent of the population [2, 3].

Problem-solving or step-based ITS have been proven successful for well-defined domains, particularly in well-defined tasks, but their success is mitigated by their cost. Typically, the main factor behind the cost is the efforts needed to model the task domain. Different approaches have been investigated to reduce these efforts: Model-Tracing Tutors (e.g. Cognitive Tutors [1], Andes [2]), Constraint-Based Tutors (e.g. SQL-Tutor [3], ASPIRE [4]) and Example-Tracing Tutors (e.g. CTAT [5], ASSISTment [6]).

With the range of educational tools available it is now realistic for learner models to take account of broader information, and there are strong arguments for placing open learner models in the centre of environments with diverse sources of data [1],[2],[3]. This Interactive Event will demonstrate the Next-TELL approach to facilitating teachers’ use of data from a variety of sources, and will allow participants to interact at all stages of this process. The Interactive Event will comprise three parts:

Going to the Chatterdale village: an OpenSim mystery for language learners;

Interaction with ProNIFA (probabilistic non-invasive formative assessment) to help teachers transform Chatterdale log data for an open learner model;

Interaction with the Next-TELL Open Learner Model to explore learner model visualisations from automated and manual sources.

Exploratory Learning Environments (ELEs) are open-ended software in which students build scientific models and examine properties of the models [1,4]. Such software are generally used in classes too large for teachers to monitor all students and provide assistance when needed, and are becoming increasingly prevalent in developing countries where access to teachers and other educational resources is limited [6]. Thus, there is a need to develop tools of support for teachers’ understanding of students’ activities. Such tools can provide support for teachers and education researchers in analyzing and assessing students’ use of ELEs.

AutoMentor is an artificial intelligent mentor who guides groups of players to accomplish tasks through online interaction including chats and E-mails in a serious game called “Land Science”. The architecture of AutoMentor consists of such analysis modules as speech act classifier, newness, relevance, epistemic network analysis and state transition network. The analyses of these modules make human mentor to be replaced by automated mentor agent. The forms of conversation among mentor agent and groups of students involve multi-logues and mutli-turns.

Intelligent tutoring systems have shown promise as personalized learning assistants that can increase learning by as much as a standard deviation over classroom teaching. However, typically, they are expensive to build, requiring extensive technical and educational expertise. They are often difficult to develop and deploy with simple modifications often requiring weeks of time to develop and favorable deployments requiring months of negotiations. This interactive event presents an alternative to such traditional large systems that we call Practical Ultra-Portable Intelligent Tutoring Systems(PUPITS).

Technological advances in the use of Artificial Intelligence for Educational (AIEd) applications over the past two decades have enabled the development of highly effective, deployable learning technologies that support learners across a wide-range of domains and age-groups. Alongside, mass access and adoption of revolutionary communication technologies have made it possible to bridge learners and educators across spatiotemporal divides.

This workshop provides the AIED community with an in-depth exploration of the Army Research Laboratory’s effort to develop tools, methods and standards for Intelligent Tutoring Systems (ITS) as part of their Generalized Intelligent Framework for Tutoring (GIFT) research project. GIFT is a modular, service-oriented architecture developed to address authoring, instructional strategies, and analysis constraints currently limiting the use and reuse of ITS today. Such constraints include high development costs; lack of standards; and inadequate adaptability to support tailored needs of the learner. GIFT’s three primary objectives are to provide: (1) authoring tools for developing new ITS, ITS components (e.g., learner models, pedagogical models, user interfaces, sensor interfaces), tools, and methods based on authoring standards that support reuse and leverage external training environments; (2) an instructional manager that encompasses best tutoring principles, strategies, and tactics for use in ITS; and (3) an experimental testbed for analyzing the effect of ITS components, tools, and methods. GIFT is based on a learner-centric approach with the goal of improving linkages in the adaptive tutoring learning effect chain in Figure 1.

Educators and researchers have long recognized the importance of formative feedback for learning. Formative feedback helps learners understand where they are in a learning process, what the goal is, and how to reach that goal. While experimental and observational research has illuminated many aspects of feedback, modern interactive learning environments provide new tools to understand feedback and its relation to various learning outcomes.

The global economy increasingly depends upon Computer Science and Information Technology professionals to maintain and expand the infrastructure on which business, education, governments, and social networks rely. Demand is growing for a global workforce that is well versed and can easily adapt ever-increasing technology. For these reasons, there is increased recognition that computer science and informatics are becoming, and should become, part of a well-rounded education for every student. However, along with an increased number and diversity of students studying computing comes the need for more supported instruction and an expansion in pedagogical tools to be used with novices. The study of computer science often requires a large element of practice, often self-guided as homework or lab work. Practice as a significant component of the learning process calls for AI-supported tools to become an integral part of current course practices.

The 4

th

international workshop on Culturally Aware Tutoring Systems (CATS2013) is a follow-up to the three previously successful CATS workshop editions, organized in conjunction with ITS2008, AIED2009, and ITS2010. It discusses the place of culture in AIED research. Considering culture in this field is important because it is known to have a strong impact on many cognitive and affective processes including those related to learning. Furthermore, people with different cultural backgrounds develop alternative interpretations and strategies and do not similarly appraise their environment, which naturally reflects in their interactions with AIED systems.

The moocshop will survey the rapidly expanding ecosystem of Massive Open Online Courses (MOOCs). We will foster a cross-institutional and cross-platform dialogue in order to articulate and synthesize the plurality of challenges that arise when evaluating and designing MOOCs. While the forms and functions of MOOCs are currently evolving, we aim to develop a shared foundation for an interdisciplinary field of inquiry moving forward. Researchers, technologists, and course designers from universities and multiple platforms will share their approaches and perspectives on key topics, including analytics and data mining, assessment, pedagogy, platform design, data standards, and privacy for open datasets.

The LT4D workshop aims to provide a forum for a discussion of cross-cultural differences regarding the immersion of AIED systems and the rational introduction of learning technologies in the developing world. Focus of the workshop is to explicitly explore the economic, social, political and cultural constraints that shape affordances for learning technologies in the developing world.

Open-ended learning environments offer students opportunities to take part in authentic and complex problem-solving and inquiry tasks by providing a learning context and a set of tools for exploring, hypothesizing, and building their own solutions to problems. Also referred to as exploratory environments, examples include hypermedia learning environments, modelling and simulation environments, microworlds, scientific inquiry environments, and educational games featuring open worlds. By the very nature of the choices they provide for learning, exploration and problem solving, OELEs offer opportunities for students to exercise higher-order skills that include: (i)

cognitive processes

for accessing and interpreting information, constructing problem solutions, and assessing constructed solutions; (ii)

metacognitive and self-regulation processes

for coordinating the use of cognitive processes and reflecting on the outcome of solution assessments; and (iii)

emotional and motivational regulatory processes

, such as curiosity and persistence. This presents significant challenges to novice learners; they may have neither the proficiency for using the system’s tools nor the experience and understanding necessary for explicitly monitoring and regulating their learning behaviours. Not surprisingly, research has shown that novices often struggle to succeed in OELEs. Without adaptive scaffolds, these learners typically use tools incorrectly, adopt sub-optimal learning strategies, and fail to regulate key cognitive, motivational, and emotional processes. Adaptive scaffolds in OELEs refer to actions taken by the learning environment, based on the learner’s interactions, intended to support the learner in completing a task and understanding the topic.

In their landmark paper VanLehn, Ohlsson and Nason [1] delineate three roles for simulated learners in learning systems: (i) to provide an environment in which human teachers can practise; (ii) to embed simulated learners as part of the learning environment; (iii) to provide an environment for exploring and testing learning system design issues. The second of these roles has been much explored in AIED, with the development of pedagogical agents [2] that can serve, for example, as learning companions [3] or disturbing agents, or even as tutors. In contrast, there is a paucity of research into either the first or third role for simulated learners. The main research touching on the first role is the development of teachable agents in a reciprocal learning context [4], but this is more of a pedagogical strategy for learners than it is a practice environment for teachers. As to the third role, even though VanLehn et al strongly argued that simulated learners could be used to provide both quick and deep insights about learners and pedagogy at the formative evaluation stage of the design of a learning system, there has not been much subsequent research into this role for simulated learners. There has been recent interest in opening up this third line of research again.

Learners need to acquire insight into their own learning as well as developing the skill to manage and regulate it. A key question for this workshop is whether instructional technology can be as effective in fostering such metacognitive skills as it is in teaching domain-specific skills and knowledge. This workshop will focus on modeling metacognitive and SRL skills, evaluating metacognitive and SRL behaviours, fostering metacognitive and SRL skills as well as exploring the relationships between metacognition and domain level learning and between metacognition, motivation and affect.