Adaptive Instructional Systems. Design and Evaluation

Adaptive instructional systems (AIS) hold great promise for increasing the effectiveness and efficiency of educational systems due to their ability to tailor instruction to the specific needs of learners. In addition, because AIS necessarily elicit evidence of learning from students to drive adaptive decisions, there has long been interest in mining data from these systems for broader assessment purposes such as program evaluation and policy making. Recently, this desire was accelerated by the novel Coronavirus pandemic. It became difficult and in some cases impossible to bring groups of students together to take standardized assessments capable of providing comparable data points suitable for aggregation at district and state levels. Simultaneously, online AIS became a part of daily instruction for many students. This paper examines theories of change that have been proposed for AIS and theories of change commonly associated with assessment and accountability in K-12 education in the United States. It then proposes a conceptual model for hybrid adaptive instructional and assessment systems (AIAS) with the unique contribution of explicitly representing the role of the AIS within the broader assessment eco-system. Finally, the paper reflects on research that will be necessary to realize the benefits of the hybrid AIAS.

The United States Marine Corps Training Command is responsible for preparing Marines to succeed in their Military Occupational Specialty (MOS) but faces increasing challenges including limited number of instructors, instructor time in billet, and classroom time available. Additionally, an institution-wide shift is taking place to better prepare Marines for tomorrow’s increasingly uncertain, complex, and decentralized operating environment, by transitioning from passive, instructor-focused training towards an active, student-centered, adult learning model [1]. Making this shift requires efficiencies in how students learn the foundations of their MOS in order to increase time spent on problem solving and practical application exercises in the classroom. The purpose of the Adaptive Blended Learning Experience (ABLE) effort was to assess the outcomes of self-paced, adaptive online learning blended with classroom instruction, and develop a model to facilitate learning of MOS training concepts in a self-paced, adaptive format that enhances student learning and creates efficiencies for instructor time. This paper describes the development, implementation, and findings from an experimental study that compared the outcomes of an adaptive blended learning design to those of traditional classroom teaching practices. The results of this effort inform recommendations for best practices in implementing adaptive, blended learning designs throughout Training Command.

Adaptive instructional systems (AISs) hold tremendous promise for addressing learner variability at scale. Many AISs are grounded in Benjamin Bloom’s (1971) Mastery Learning approach, which delivers differentiated instruction, appropriate scaffolding, and feedback to ensure each child masters each concept or skill before moving on. (Bloom’s 1984) framework for learning went beyond the immediate interactions of learners and the AIS. He described “four objects of the change process” that must be addressed to significantly improve student learning: the learner, the materials, the teacher, and the learner’s environment, where parents/caretakers are a critical component, especially for young children. This paper describes a learning engineering approach to craft a Personalized Mastery-Based Learning Ecosystem (PMLE) that uses all people, processes, data, and networked connections to create new capabilities, richer experiences, and unprecedented educational opportunities for children and their families. This ecosystem treats all individuals within the system as learners (child, parent, teacher, etc.) whose knowledge and expertise can be enhanced to benefit the child’s learning. The PMLE enables parents and teachers to become empowered “agents” of change by providing them with knowledge, tools, and evidence-based strategies to support meaningful and effective interactions with the child, all driven by real-time data about the readiness of the child. This paper presents a vision of how AISs can move beyond working solely with the child to become more robust ecosystems that empower all agents of change to optimize personalization and ensure long-term success of all children at scale.

Since the advent of gamification, organizations have begun transitioning away from a “learning by listening” approach towards training programs that incorporate game-like elements with the hope that by doing so improvements in user performance, engagement, and motivation would be immediately realized. However, these outcomes are difficult to achieve in practice. Part of the difficulty encountered when designing and implementing a gamified training program can be traced back to the unknown relationships between game elements, user psychology, and pedagogical theory. This difficulty is encountered because the cause and effect relationships between the implemented game elements and the desired training outcomes remain unknown. Prior efforts have been made to construct best-practice guides to support the researchers and developers of gamified training programs; however, these guides rarely outline the relationship between specific game elements and student psychology. As a result, several models and definitions of gamification have been developed in concurrent isolation. Doing so has caused confusion within the domain of pedagogical gamification research. In an effort to reconcile the relationships between specific game elements, user performance, engagement, and motivation within Gamified Training Programs (GTP), we have made an initial step towards a model that unifies the extant understanding of gamification and its relationship with motivational psychology through the development of the Unified Gamification and Motivation (UGM) Model.

Today, various actors are exploiting and misusing online social media to spread disinformation and to create false narratives. This paper summarizes an education and training approach targeted to help people think more critically about potential disinformation. The approach we outline emphasizes the development and maturation of general critical-thinking skills, in contrast to technical skills (e.g., social network analysis). However, it also offers opportunity to apply these skills in a scaffolded, adaptive environment that supports the learner in putting concepts into use. The approach draws on the situated-learning paradigm to support skill development and reflects empirically-based best practices for pedagogy for critical thinking. This analysis and review provides context to inform the design of a learning environment to enable targeted practice of critical thinking skills. The paper outlines the high-level design, describes several specific “experiential lessons” and overviews a few technical challenges that remain to be overcome to make the training feasible for wide-scale use.

The Cognitive Expertise through Repetition-Enhanced Simulation (CERES) project is aimed to provide a novel approach to accelerating the development of expertise in topographic map reading skill. Practice at this complex cognitive skill is accomplished through rapid-paced, simulation-based training events with a large procedurally generated database of training content items to provide continuing novel challenges. On each short training event, participants are presented with simultaneous views of a novel topographic map and a video render of first-person perspective movement through the terrain. The trainee goal is to identify what position on the map is reflected by the movement clip. Trainee performance level was continually assessed during training and the difficulty of training content was adjusted adaptively to enhance learning and maintain engagement and motivation with the training task. Data are reported from a sample of naïve online participants (n = 37, 169 h of total training) who acquired topographic map reading skill based on their experience with the CERES protocol. Participants without any prior topographic map reading training or experience nevertheless exhibited improved performance over training. In addition, a substantial subgroup also demonstrated very high levels of engagement and motivation with the training protocol. We conclude that the approach of rapid-paced training to induce practice with a complex cognitive skill can be administered in an individualized adaptive manner to accelerate the development of skilled expertise.

The advances in technology continually push at the boundary of what is possible in online learning environments. Digital learning generates data at scale that can be analyzed to gain new insights into the learning process, which in turn sparks further changes in technology. Adaptive instructional systems have been growing in type and complexity, which also increases the scope of the technology and the teams who work to develop them. In this time of intense exploration and innovation in learning science and technology, it is also imperative to put in place a system of ethics to center this innovative spirit on the intended user: the learner. I propose that learning engineering provides a purpose to advocate for the best interests of the learner as the Learning Engineering Process is carried out. This practitioner purpose can help the learning engineer develop an ethical voice and engage in a dialogic ethic as new technology is being developed. I use a case study on the development and improvement of adaptive activities to illustrate both the Learning Engineering Process as well as how this process supports an ethical practice. In this paper I situate learning engineering in an ethical framework and provide a contextual example to spark discussion on the role of ethics in an increasingly complex learning ecosystem.

Traditionally, adaptive instructional systems (AISs) are built to instruct human students. However, they are not the only students that might benefit from an AIS. The field of reinforcement learning (RL), a subfield of machine learning, studies the instruction of synthetic students called agents, by means of various algorithms. In this paper, we advocate the use of an AIS as a conceptual framework to design and teach RL agents. We form our argument by deconstructing what it means to build and use an AIS for a human student, and discuss how the various concepts and relationships may apply to RL agents. We illustrate our findings by means of examples from the reinforcement learning literature and show a domain implementation of an AIS for RL agents.

The advent of E-learning has allowed students to have access to a massive group of educators and learning resources. However, the concept of online learning still lacks a quality that deems it inferior to classroom education and that is the ability to understand the needs of individual students. With reference to online learning, the complexity of different online resources plays a crucial role in determining the usefulness of that resource for a given user. As a result, students get intimidated by these divergences in explanations, making the effectiveness of e-learning subject to a user's psychology and self-motivation. Thus, there is a need to understand the dynamics of a student's learning behavior before suggesting resources. In order to address this need, in this research paper, we present an Adaptive Educational Hypermedia System (AEHS) called SQLearn which assesses the performance of students with an assessment as they study a topic and consequently assists their learning experience. SQLearn consists of two main components, the Testing Platform, and the Web Browser Extension which works in unison to understand students learning behavior. After analyzing a student's learning behavior, the designed system is capable of suggesting them online resources to help them grasp concepts they is weak at. The system is also capable of making inferences based on the students answering behavior to help them maintain an optimum learning and answering speed. In order to test the efficacy of the designed system, a pilot study was conducted with 11 undergraduate students. This study helped bolster claims regarding the usefulness of the system while also motivating the creation of a more accurate system.

This paper presents the initial design and mechanics of two proposed trackers – the Presence Card (PC) and the Trust Rating (TR) – to define an alternative way of checking attendance, and to challenge the spread of formal and informal outsourced work in the conduct of online classes. The PC and TR mechanics were implemented in various undergraduate and graduate classes to demonstrate how it may be used and to collect insights from the students about such an intervention. Using the metrics we have identified during testing together with an analysis of the survey results, we conclude the paper with directions and ideas towards adaptive versions of the PC and TR mechanics.

Ethical dilemmas are complex scenarios involving a decision between conflicting choices related to ethical principles. While considering a case of an ethical dilemma in education presented in [17], it can be seen how, in these situations, it might be needed to take into consideration the student’s needs, preferences, and potentially conflicting goals, as well as their personal and social contexts. Due to this, planning and foreseeing ethically challenging situations in advance, which would be how ethical design is normally used in technological artifacts, is not enough. As AI systems become more autonomous, the amount of possible situations, choices and effects their actions can have grow exponentially. In this paper, we bring together the analysis of ethical dilemmas in education and the need to incorporate moral reasoning into the AI systems’ decision procedures. We argue how ethical design, although necessary, is not sufficient for that task and that artificial morality, or equivalent tools, are needed in order to integrate some sort of “ethical sensor” into autonomous systems taking a deeper role in an educational settings in order to enable them to, if not resolve, at least identify new ethically-relevant scenarios they are faced with.

Building reliable interactive systems has been identified as an important and difficult task from the late ‘60s. One approach to augment the reliability of interactive systems is to use formal models during system development. Formal methods have received attention for the design and analysis of human-computer interaction (HCI) for thirty years. The field of adaptive instructional systems (AIS) in general, and intelligent tutoring systems in particulars, have been mostly relying on empirical methods for training systems validation (the system supports learning), rather than formal methods for verification (the system meets its specifications). Empirical methods focus on the validity of pedagogical interventions at the individual task and problem sequence levels, using learning analytic methods such as Bayesian knowledge tracing, additive factors models, or machine learning models of human performance. The purpose of the paper is to explore some parallel and the applicability of HCI formal models to AIS. The paper: a) presents key concepts related to HCI formal models using semi-formal representations (workflow graphs), b) gives examples of formal properties to be verified, c) discuss briefly formal notations, and d) defines adaptive human-computer interaction. The last section of the paper discuss the similarity between HCI formal models and AIS standard modules, and identifies some area of applicability of HCI formal models to AIS design, recognizing the central value of AIS empirical methods at the foundation of AIS iterative design.

Team training in online, simulated environments can improve teamwork skills and task performance skills in a team setting. Teamwork assessment often relies on human observers. Instructors, team leaders, or other observers typically assess complex team competencies using checklists of observed behavior markers to infer performance. Automation can reduce training bottlenecks, provide evidence for objective assessment, and increase the impact of team training. A software capability is being developed to automate team assessments in dynamic online simulations. The simulations are dynamic to the extent that team actions and performance can change the progression of simulation events, assessment context, and the expected behavior of individuals contributing to team performance. A goal of automation design is to enhance usability for non-technical personnel to select, configure, reuse, and interpret team assessments in dynamic simulations. As a result of the reusable design, the assessments can generalize across different simulation software, settings, and scenarios. This paper describes work in progress on the research and development of an automated team assessment capability for the US Army’s Generalized Intelligent Framework for Tutoring (GIFT), an open source adaptive instructional architecture.

With the COVID-19 pandemic postponing face-to-face classes and closing down the doors of educational institutions worldwide, online learning is one of the alternatives which these institutions have been adopting. With the advent of these online learning systems, students face many barriers which include lack of time and motivation. To help address these barriers in online learning, this paper presents the design and development of a gamified task management web application which aims to increase student engagement and motivation. In addition, this paper also aims to determine how these implemented gamified features can further be developed for adaptive learning. The application was developed incorporating design elements from two gamification frameworks which aim to improve users’ motivation and engagement while catering to as wide an audience as possible. In addition, data which can be gathered from the application may prove helpful towards the design and development of further adaptive gamified features. Future work on the application includes testing its effectiveness with student audiences and implementation of further adaptive features.

“Learning” is a means to an end; the end is to perform. Tools aid human performance. Before electronic computers became commercially available, the term “computer” meant “one who computes”: a person performing mathematical calculations. Humans no longer need to do parts of a task that computers or machines can do well. In many industries, intelligent machines with advanced hardware or software have been exploited to augment human performance. In comparison, education and training industries have yet to realize the potential of intelligence augmentation. In this paper, we review case studies of Intelligence Augmentation in industry and in educational institutions, considering common barriers to adoption in the latter. We also explore the possibilities of collaboration between humans and intelligent machines based on Industry 4.0 tools and techniques. Based on industry examples, we propose a model for building collaboration between humans and intelligent machines for improved performance in the development and delivery of teaching and learning.

Intelligent Tutoring Systems (ITSs) are considered among the most effective and efficient learning systems. The difficulty of authoring content limits the use of ITSs and creates a bottleneck for most ITS researchers and related industries. AutoTutor, a conversation-based ITS, faces the same problem. This paper will introduce the new improvement of AutoTutor Lite using xAPI and its potential as a generic CbITS authoring tool.

Today’s classrooms can be remarkably different from those of yesteryear. In place of individual students responding to the teacher from neat rows of desks, today’s innovative schools have students working in groups on projects, with a teacher circulating among groups. AI applications in learning have been slow to catch up; most available technologies are described as personalizing or adapting instruction to needs of individual learners. Meanwhile, an established science of Computer Supported Collaborative Learning has come to prominence, with clear implications for how collaborative learning can be supported. In this contribution, I consider how intelligence augmentation could evolve to support collaborative learning. A focus on AI role in automating, adding to awareness, assisting and augmenting is suggested, extending the field’s prior tendency to focus on assessing, assigning and adapting.

Adaptive Instructional Systems (AIS) have the potential to provide students with a flexible, dynamic learning environment in a manner that might not be possible with the limited resources of human instructors. In addition to technical knowledge learning engineering also requires considering the values and ethics associated with the creation, development, and implementation of instruction and assessment techniques such as fairness, accountability, transparency, and ethics (FATE). Following a review of the ethical dimensions of psychometrics, I will consider specific ethical dimensions associated with AIS (e.g., cybersecurity and privacy issues, invidious selection processes) and techniques that can be adopted to address these concerns (e.g., differential item function, l-diversity). By selectively introducing quantitative methods that align with principles of ethical design, I argue that AIS can be afforded a minimal ethical agency.

This paper examines the role of participatory codesign in the creation of parent, teacher, and administrator dashboards for a game-based adaptive instructional system designed to teach math to young children. Taking a learning engineering approach, our team of researchers, curriculum specialists, and UX designers engaged in an iterative design process with six primary school teachers and 4 school administrators with the goal of understanding the appropriateness and effectiveness of the dashboards for different role types. Using ongoing interviews and a participatory codesign workshop, we engaged with teachers and administrators over several months and worked with them to understand how the dashboards could serve and be used by role types at varying levels (parent, teacher, principal, administrator, superintendent). We found that the effectiveness and appropriateness of the dashboard stemmed from its ability to communicate information across systems, like allowing teachers to communicate with parents on how to help their child, allowing principals to check-in on teachers on student progress, and allowing superintendents to review school-wide learning goals with principals. In sum, the participatory codesign process was highly successful, leading to a rich understanding of how the dashboards can be better designed to connect information across systems to better serve different role types.

This paper reviews horizontal and vertical scaling methodologies for adaptive instructional system (AIS) software architectures. The term AIS refers to any instructional approach that accommodates individual differences to facilitate and optimize the acquisition of knowledge and/or skills. The authors propose a variety of scaling methods to enhance the interaction between AISs and low-adaptive training ecosystems with the goal of increasing adaptivity and thereby increasing learning and performance. Typically, low-adaptive training systems only accommodate differences in the learner’s in-situ performance during training and do not consider the impact of other factors (e.g., emotions, prior knowledge, goal-orientation, or motivation) that influence learning. AIS architectures such as the Generalize Intelligent Framework for Tutoring (GIFT) can accommodate individual differences and interact with low-adaptive training ecosystems to model a common operational picture of the training relative. These capabilities enable AISs to track progress toward learning objectives and to intervene and adapt the training ecosystem to needs and capabilities of each learner. Finding new methods to interface AISs with a greater number of low-adaptive training ecosystems will result in more efficient and effective instruction.

HyWorM is an approach and implementation for guiding analytic sensemaking processes using the HyGene model of human hypothesis generation. It is an evolution of the RAMPAGE Workflow Monitor (WorM) that monitors and guides analysts in the production of counterfactual forecasts, dynamically adapting work prompts and the revelation of new evidence to broaden and narrow analyst attention, then controlling the schedule of specific forecast problems. WorM also monitors and controls the timing of workflow steps to ensure that attention is distributed effectively across counterfactual problems and other analysis tasks. The inclusion of HyGene theory in WorM to yield the HyWorM process shows potential to broaden analysts’ attention to a variety of evidence by using results from the HyGene simulation. Based on previous studies with HyGene, we hypothesize that this will improve the quality of counterfactual forecasts.

Courseware as an adaptive instructional system is a complex environment to develop. The student will encounter lessons of content with integrated formative practice, adaptive activities, and assessments in their learning path. The alignment of all course features, including the scaffolding structure of the adaptive activities, may vary between courses and the teams who created them. In a previous analysis of adaptive activities [1], these activities had net positive effects on student learning estimates and summative assessment scores. In this paper, we will analyze three additional non-STEM courses that had less effective adaptive activities using the same methods as the original study, and further investigate course features that could be influencing their effectiveness, such as alignment, difficulty, and amount of practice. The results of this analysis can provide guidance on how to best create content for adaptive courseware and provide an example of the critical role data analysis has in the evaluation and iterative improvement of student learning environments.

In this article we present an knowledge extraction approach that can be used in systems that implement teaching in a fully automated manner. These systems are called Intelligent Tutoring Systems (ITS) and are conceived around the idea of one-to-one teaching. Many such systems use natural language processing to improve the communication interface between student and the system. These techniques can be also used on the content creator side to semi-automate or fully automate the task of teaching content creation. In such systems the knowledge representation plays a crucial role to successfully implement teaching and encourage learning. The output of the knowledge extraction phase is a knowledge in the form of a hyper graph that can be used for adaption to the students current knowledge level. We present a deep neural network architecture for precise POS tagging of words written in languages that are morphologically rich. Using sparse representations for words in this task increases the vector space and makes learning more complex. This problem can be solved to some extent by using traditional vector representations but there is also the problem with representing words that are ambiguous. Proposed architecture uses a Bidirectional Encoder Representations from Transformers (BERT) model that is pre-trained on Croatian language to achieve state-of-the-art accuracy for POS tagging.

Many people in the education sector are aware of the benefits students derive from adaptive learning systems. Unfortunately, what is not well known is that an adaptive learning system can also augment the instructor’s understanding of what learning has occurred. Valid and relevant questions used by the adaptive learning system can provide an instructor with real-time insight into the level of knowledge attained by each individual student. Similarly, the same sort of knowledge can also indicate learning deficits that might affect an entire class and allow for an instructor to accurately respond to the student’s learning needs in an informed way.The key to this is an understanding and confidence on the part of the instructor for each assessment delivered by the adaptive learning system.In this paper, we will demonstrate how to use information gained through the implementation of an adaptive learning system to augment knowledge. In it, we will propose a methodology for evaluating adaptive learning questions given as assessments.The intent is to produce a tool for determining the validity, reliability, and standardization of the questions asked by the system in a way that gives the instructor specific knowledge about student learning. This tool will approach a question’s evaluation, both before the question is asked and then after, using basic psychometric principles. While this paper will focus on the use of such a tool by the instructor, it should be understood it would serve very well to evaluate questions by instructional designers and curriculum developers.

Despite decades of research and significant current investment, AI-based applications in education have not gotten traction at scale in a way that transforms learning. The most common learning and assessment applications are intelligent tutoring systems that adjust content based on a student profile and automated essay scoring systems that apply “learned” models of scoring to score written assignments. Among the challenges facing these applications in achieving classroom implementation are: trust, existing systems of teacher and student roles and responsibilities, and fairness. This paper discusses these issues and then examines a case study of the use and subsequent removal of artificial intelligence in Khan Academy offerings.

Advanced technologies for individualized learning present intuitively appealing opportunities to provide students with exactly what they need, when they need it. However, observed challenges with implementations of adaptive instructional systems in K–12 education to date point to a need to better articulate the theory of change for the incorporation of individualized pathways and teacher-led grade-level learning, removing incongruities. This paper first examines separate exemplar theories of change underlying each of the adaptive instructional system and blended learning models. It then discusses associated pedagogical and setting considerations, identifying several areas of incongruity to be addressed when bringing these models together. An exemplar theory of change for integrating the adaptive instructional system as a component core to the learning day is proposed. Finally, the paper reflects on research required to test the proposed model and changes in interactions, pedagogy, and curriculum that may be required.

The present study aims to explore the effectiveness and learner characteristics of Chinese Mathematical intelligent tutoring system in multiple mathematics subjects. Data were collected from primary school, secondary school, and college. The students’ interactions were recorded in database as log files. The data were coded to fit the format of Datashop, which is a data repository and web application for researchers analyzing learning sequence of data. A learning curve visualizes changes in student learning state over interactions on different mathematics subjects. The results of learning characteristics of students were discussed. Furthermore, the study compared students’ learning patterns in three different score groups (high, medium, and low). The results found that students with lower pre-test score received more prompting in their interaction comparing students with higher pre-test score. More learning patterns were discussed.

Artificial intelligence offers great opportunities for the future, including for teaching and learning. Applications such as personalized recommendations and learning paths based on learning analytics [i.e. 1], the integration of serious games in intelligent tutoring systems [2], intelligent agents in the form of chatbots [3], and other emerging applications promise great benefits for individualized digital learning. However, what value do these applications really add and how can these benefits be measured?With this article, we would like to give a brief overview of AI-supported functionalities for learning management system as well as their possible benefits for future learning environments. Furthermore, we outline methods for a comprehensive evaluation that meets the users’ needs and concretizes the actual benefit of an AI-supported LMS.

This paper is a critical examination of four factors that affect widespread adoption of computer-based interventions in general and adaptive instructional systems (AISs) in particular within the Philippine context: Despite government and private sector efforts, hardware diffusion in schools has not reached critical mass. The poorest sectors of society have very limited Internet access. The curriculum tends to focus on computer literacy skills and is generally unable to use information and communication technologies to support specific subjects. Finally, teachers lack the preparation and, as a consequence, the confidence to use technology in innovative ways. The study-from-home necessitated by the COVID-19 pandemic would have been an ideal opportunity to maximize the impact of computer-based interventions such as AISs as these technologies have been shown to compensate for weak teacher-led instruction. However, AISs demand the very same resources that the Philippines educational system does not have, making such deployments impossible. This paper ends with a challenge to design AISs to operate under these difficult circumstances in order to maximize the benefits that they bring to education.

The present work explores the effectiveness of goal setting in the context of adaptive training. Previous research has demonstrated that adaptive training approaches that tailor feedback and difficulty based on task performance lead to better learning outcomes than non-adaptive approaches. Likewise, decades of research on goal setting as an instructional technique has shown that setting achievement goals for trainees to improve also increases learning outcomes. In particular, challenging, specific goals have been found to be more effective than generic “try your best” goals. Bridging these techniques together presents an interesting opportunity to examine the effects of goal setting on performance in a training system that adapts both feedback and difficulty. For example, when a scenario’s difficulty is adapted up, a specific goal may no longer be attainable, which begs the question – do challenging goals improve performance compared to generic goals in a training system that adapts difficulty? In this experiment, 45 college students were trained to perform a complex radar detection task under two goal setting conditions, specific (“try to improve by 25%”) or general (“try your best”). We evaluated performance using a pre- to post-test design across several task measures. Overall, results were mixed, showing advantages for the specific goal condition on some accuracy measures but disadvantages on some timeliness measures compared to the general goal group. Implications for goal setting theory and practical applications for adaptive training are discussed.

Adoption of e-learning for those with special needs lags that for mainstream learners. Not much is known about barriers and facilitators that drive this disparity. The present study used focus groups and interviews to collect the views of 21 teachers taking part in preliminary evaluations of an adaptive learning system based on multimodal affect recognition for students with learning disabilities and autism. The system uses multimodal detection of affective state and scoring of performance to drive its adaptive selection of learning material. Five themes captured the teachers’ views of the system’s potential impact, especially regarding learning and engagement but also on factors that might influence adoption. These were: the potential of the system to transform their teaching practice; the ability of the system to impact on learning outcomes; the potential impact on teacher-student/peer to peer relationships; usability issues; and organisational challenges. Despite being highly motivated as volunteer testers, teachers highlighted barriers to adoption, which will need addressing. This finding underscores the importance of involving teachers and students in the design and development process.

Students entering the doctor of chiropractic program at Northwestern Health Sciences University were offered three adaptive units on chemistry concepts deemed foundational for the two-course biochemistry series offered in the first two terms of the program. The effects of this remedial intervention offered to 3 incoming cohorts were assessed in a retrospective case-control approach by comparing several outcomes with a control group of similar size who started the program before the implementation of the adaptive units. Our calculations suggest that there is a positive effect of these adaptive units, in that the odds ratio for students to end the course with a grade of D or F decreased. The biggest impact on performance among 4 summative exams in the course was observed on the final exam, with an odds ratio of 2.3 to earn an A or B on the final, indicating that students who had access to the adaptive units had a substantially higher chance to earn a good grade. The odds ratio for students to earn an F or D on the final was 0.5, indicating a 50% lower risk for a low or failing score.

The U.S. military is interested in modernizing its training and educational technology resources to support tailored, learner-centric experiences more consistent with the digital-age. The goal of this research was to explore the benefits of adaptive training in the context of a USMC course. To this end, we describe the development of an adaptive flashcard system to help Marines study during an automotive maintenance course. Specifically, the system incorporated techniques, such as adaptive sequencing and the use of mastery criteria to drop flashcards from study. These techniques have been effective in the context of short-term laboratory studies but not for longer-term course performance. Therefore, we conducted an initial pilot study to compare course performance outcomes between cohorts before and after the addition of the adaptive training system in the course. Overall, there was a 50% reduction in exam learning objective failures in the adaptive training cohort compared to the cohort without adaptive training. Furthermore, we found that greater usage of the system was related to higher course GPA. Taken together, these results demonstrate a promising use-case for an adaptive training solution implemented successfully “in the wild,” and future research plans for this system are discussed.

Self-regulated learning (SRL) with advanced learning technologies has shown to significantly augment learners’ performance across contexts. Yet studies find learners lack sufficient SRL skills to successfully implement strategies (e.g., judgments of learning, note taking, self-testing, etc.). Current research does not fully explain how and why this failure of effective strategy deployment occurs. We used principle component analysis (PCA) on process data (i.e., log files) from 190 undergraduates learning with MetaTutor, a hypermedia-based intelligent tutoring system, to explore underlying patterns in the frequency of strategy deployment occurring with and without pedagogical agent scaffolding to better understand any underlying structures of system- and learner-initiated cognitive and metacognitive SRL strategy use. Results showed that the system’s underlying architecture deploys processes corresponding to both the phases of learning and type of effort allocation according to Winne’s (2018) Information Processing Theory of SRL. However, learner-initiated processes for those who received scaffolding only displayed strategy deployment that corresponded to the type of effort allocation required of the processes (i.e., more effortful constructionist processes like note-taking versus short canned responses for judgements of learning). Additionally, results suggest all learners deploy strategies based on the familiarity of processes. Regression models using these principle components outperformed raw frequency models for capturing post-test learning performance across all participants.

3A refers to content aware, context aware, and learner aware intelligent tutoring system (ITS) [2]. The idea behind this is that any ITS should be delivering content intelligently by knowing about the state of the user . The state of the user could be emotional or even physical.Almost all the ITS are more or less intelligent to deliver content. But less intelligent to know whether the learner is accepting the content. In addition, the context consists of two components: context of the content and context of the environment. It is easy for an ITS to be aware of the context of the content (e.g., calculus in case of integrals) but very few ITS take into account the context of the environment. For example, a learner is accessing content in a crowded environment from her cell phone and a learner is accessing content inside a library where it is calm. Contexts of these two learners are different. Moreover, the learner awareness includes emotional states as well as physical states of a learner.In this research our focus is to collect data by enabling a 3A learning system in AutoTutor [11]. AutoTutor is a conversation-based ITS that uses an expectation-misconception tailored dialogue to promote learning. Several questions involved in designing 3A enabled AutoTutor. How to collect 3A data without violating learners’ privacy is the most important one. All other design questions revolve around this.

Conversation-based learning technology is playing important role in adaptive instructional systems (AIS). As a part of the adaptivity of an instructional system it would be ideal to incorporate a human tutor to deal with conversation that is beyond the capability of a chat-bot or virtual tutor. Moreover, it is possible to answer many research questions if experiments are performed with a blended human tutor. In this research we have implemented a prototype that blends a human tutor with a virtual tutor in a typical conversation-based tutoring system (i.e., AutoTutor). We performed R&D with server-based and serverless implementations. Additionally, we have implemented audio-visual blending through WebRTC so that the conversation between students and teachers can take place through spoken language with video. We found that the serverless chat blending with AutoTutor is fast, easy to implement, and reliable. We made the so-called serverless implementation possible by using some very powerful features of a learning record store (LRS).

Adaptive training can take place across multiple learning contexts, such as in-person or online class types, and in classroom or simulation environments [1]. Given the COVID-19 pandemic, institutes of higher education were forced to adapt and make instruction available both online and in-person; providing the opportunity to examine how adaptive learning strategies faired in different class types. When examining adaptive strategies across class types, of particular interest are individual learner characteristics and preferences, as students who typically would not take online classes found themselves taking classes in this medium [2]. We sought to evaluate the effectiveness of adaptive learning strategies across preferred and non-preferred class types. A significant interaction was found between instructional strategy presence and preferred class type, with students primarily attending classes in their preferred class type self-reporting significantly lower stress and workload levels than those who did not receive the adaptive instructional strategies. For students attending class primarily in their non-preferred learning environment, the opposite was true. Additionally, students in their preferred class types achieved higher exam scores. The methods, results, discussion and implications are discussed in the context of how to effectively conduct adaptive training.

The attraction of competency-based learning as a progressive model for newer learning systems has increased significantly over the past several years. It is built around the principle that learner outcomes, defined as what the learner is able to do and master after their learning experiences, are the ultimate measure for evaluation. This improves on earlier learning frameworks which focused on content authoring to develop curricula and instructor delivery techniques that relied heavily on standard grading systems, emphasizing testing only in terms of how much content was consumed and could be recalled by the learner. Thus, the focus was on instructor efficacy instead of on learner needs. A competency-based learning approach encourages tailoring the learning experiences to the learner and using evidence of learning to improve and adapt the learning components, which aligns well with adaptive instructional system approaches. The purpose of this paper is to further expand on competency-based learning, system and student evaluation, the assessment process, and how competency-based learning approaches may be applied to enhance and tailor adaptive instructional systems to achieve higher levels of learning based on individual outcomes. Additionally, it will be shown that there is a close connection between competency-based learning, learning analytics, and adaptive learning systems.

An essential component to intelligence analysis is inferring an explanation for uncertain, contradictory, and incomplete data. In order to arrive at the best explanation, effective analysts in any discipline conduct an iterative, convergent broadening and narrowing hypothesis assessment using their own tradecraft. Based on this observation, we developed an adaptive framework to support intelligence analysis while being tradecraft agnostic. The Reasoning About Multiple Paths and Alternatives to Generate Effective Forecasts (RAMPAGE) process framework provides a structure to organize and order analysis methods to maximize the number and quality of hypotheses generated, helping to improve final forecasts. The framework consists of five stages of analysis: (1) Information Gathering and Evaluation; (2) Multi-Path Generation; and (3) Problem Visualization; (4) Multi-Path Reasoning; and (5) Forecast Generation. As part of IARPA’s FOCUS program, we demonstrated the flexibility of this framework by developing five versions of the process to answer five different sets of counter-factual forecasting challenges. While the FOCUS program concentrated on counter-factual forecasting, this framework was designed to support hypothesis generation and assessment, which is a critical component of analysis across the intelligence domain.

The design of physiologically adaptive systems entails several complex steps from acquiring human body signals to create responsive adaptive behaviors that can be used to enhance conventional communication pathways between human and technological systems. Categorizing and classifying the computing techniques used to create intelligent adaptation via physiological metrics is an important step towards creating a body of knowledge that allows the field to develop and mature accordingly. This paper proposes the creation of a taxonomy that groups several physiologically adaptive (also called biocybernetic) systems that have been previously designed and reported. The taxonomy proposes two subcategories of adaptive techniques: control theoretics and machine learning, which have multiple sub-categories that we illustrate with systems created in the last decades. Based on the proposed taxonomy, we also propose a design framework that considers four fundamental aspects that should be defined when designing physiologically adaptive systems: the medium, the application area, the psychophysiological target state, and the adaptation technique. We conclude the paper by discussing the importance of the proposed taxonomy and design framework as well as suggesting research areas and applications where we envision biocybernetic systems will evolve in the following years.

Students can be placed in more than one category at the start, middle, and end of their educational journey. These categories can be based on demographics (age, gender, sex, minority, disability, ethnicity), on behavior (procrastination, struggle, frustrated guessing, pathological re-reading), on individual attributes (help-seeking, locus of control, time management, optimism), on community (internet access, setting, average education and income), and on academic factors (previous grades and degrees). These categories are frequently used by faculty, designers, and leadership to seek a better understanding of students and their needs with the goal to personalize or adapt the learning environment in the hopes of leading to more effective learning and more successful student outcomes. In these analyses we seek to determine the relative value of different student categories – and how these can be combined to result in the most effective educational process. We find ourselves asking what attributes matter the most – and which interact with each other to increase or reduce the amount of relative value. It is worth noting that several of the categories – while they play a large role in our students’ holistic selves – are both highly sensitive (frequently protected) and static. If we can approach or match their value (educationally) in other categories which are less sensitive or more changeable, that will be a positive result. Because, while these attributes play a role in who the students are, they need not play a role in how the students are taught.

Adult learners with low literacy skills compose a highly heterogeneous population in terms of demographic variables, educational backgrounds, knowledge and skills in reading, self-efficacy, motivation etc. They also face various difficulties in consistently attending offline literacy programs, such as unstable worktime, transportation difficulties, and childcare issues. AutoTutor for Adult Reading Comprehension (AT-ARC), as an online conversation-based intelligent tutoring system that incorporated a theoretical model of reading comprehension, was developed with great efforts to meet adult learners’ needs and be adaptive to their knowledge, skills, self-efficacy, and motivation. In this paper, we introduced the adaptive features of AT-ARC from four aspects: learning material selection, adaptive branching, trialogues, and interface, as well as the rationale behind these designs. In the end, we suggested further research on improving the adaptivity of AT-ARC.

This work proposes a conceptual framework that captures cultural considerations in providing options in the way learners interact with an instructional system. It revisits the longstanding relationship between culture and learning behavior and how cultural background influences the design and implementations of instructional systems. The literature presents some approaches including the use of cultural references embedded into the learning material which draws from the concept of shared meanings and cultural familiarity; and through the development of immersive environments using virtual reality to provide cultural experiences. Aside from domain limitations and technological requirements, these approaches are usually focused on the content or environment rather than the interaction. In order to identify culturally-biased behaviors and preferences, the paper looks at the contextualization of Hofstede’s cultural dimensions in educational software linked to interaction concepts in the Cultural Artefacts in Education (CAE) framework. These adaptation rules are then anchored to the Universal Design for Learning (UDL) Guidelines to provide a more cohesive pedagogical structure and as an attempt to make them applicable across disciplines and domains. Future work includes the implementation and validation of the proposed framework.

Adaptive Training Protocols (ATP) is a collection of algorithms and software to apply principals of intelligent tutoring to physical fitness training. To obtain norming data for ATP, we examined exercise performance from 34 participants under an adaptive workout regimen lasting 13 weeks. The goal of the regimen was to train to pass the performance criteria of the US Marine Corps Initial Strength Test (IST; a 1.5-mile run, sits-ups, pull-ups, and push-ups). The weekly regimen comprised an IST, an interval workout, and a maximum workout. Adaptation was accomplished via two algorithms: maximum-day reps were double those accomplished on the prior IST and maximum-day and interval-day runs were performed at specified rates of perceived exertion. Starting capabilities for run, sit-ups, and push-ups negatively correlated with progression rates; participants who exhibited lower performance at the start of the study made steeper gains in performance. Individual logistic curve fitting found decelerating, inflecting, and accelerating progression profiles. Participants showed considerable variation in their profiles both across individuals in each exercise and within individuals across exercises. Progression profiles can be used to forecast the performance that a person can attain in a given timeframe under a given training regimen. This knowledge can be used to adapt the workout to provide more time to reach a goal if needed or to focus on exercises that are in jeopardy of not achieving the goal in time. ATP will help the Marine Corps plan for when intended recruits may be physically ready to ship out to boot camp.