Virtual, Augmented, and Mixed Realities in Education

Virtual reality (VR) typically results in the illusion of presence. The participant in a VR scenario typically has the illusion of being in the virtual place, and under the right conditions the further illusion that events that are occurring there are really occurring. We review how these properties are useful for the application of VR in education. We present a further illusion that can be triggered in VR referred to as body ownership. This can occur when the participant sees a life-sized virtual body substituting her or his own, from first person perspective. This virtual body can be programmed to move synchronously with the participant’s real body movements, thus leading to the perceptual illusion that the virtual body is her or his actual body. We survey various experiments that show that the form of the virtual body can result in implicit changes in attitudes, perception and cognition, and changes in behavior. We compare this with the process of implicit learning and conclude that virtual body ownership and its consequences may be used as a form of implicit learning. We conclude by suggesting how the study of the relationship between body ownership and implicit learning might be taken forward.

Authenticity, is a concept found in both media design and educational design, usually as a quality needed for success. Here, we develop a theory of authenticity for educational experiences with immersive media (VR, MR, MUVEs, etc.) to help educators and authors in this new field. In our framework, authenticity refers to the relationship between a truth and its representation, guided by a purpose. By truth, we refer to a fact, concept, or procedure, about something in the world or in the body of human knowledge, something we want to learn. To scaffold the learning process, students require a representation of the thing. It may be a written article (for concepts), an image (e.g., a photograph), or maybe an exemplar (an idealized example of a category). A representation or an experience is said to be authentic, when it successfully captures the fundamental truth of what we are learning. The immersive media have unique capabilities and just in the last few years have become available to the public on a large scale. Our theory is not a comprehensive style guide, but a practical way to look at one key dimension of good educational design.

The chapter reviews new technologies and their impact on learning and students’ motivation. The main argument is that in order to achieve immersion, social interactions should be fostered. Therefore, three technologies are discussed which either inherently draw on social interactions (pedagogical agents, transformed social interaction) or can be enriched by including collaborative learning elements (augmented reality). For each of the three realms, a short overview on the state of current developments as well as on empirical studies and results is given. Also, it is discussed to what extent they built on social interaction, how this might be extended and whether beneficial outcomes can be expected from this.

Immersive Environments (IEs) hold many promises for learning. They represent an active approach to learning and are intended to facilitate better, deeper learning of competencies relevant for success in today’s complex, interconnected world. To harness the power of these environments for educational purposes (i.e., to support learning), we need valid assessments of the targeted competencies. In this chapter we focus on how to design and develop such valid assessments, particularly those providing an ongoing, unobtrusive collection and analysis of data as students interact within IEs. The accumulated evidence on learning thus provides increasingly reliable and valid inferences about what students know and can do across multiple contexts. This type of assessment is called “stealth assessment” and is applied toward the real-time measurement and support of learning in IEs—of cognitive and non-cognitive variables. The steps toward building a stealth assessment in an IE are presented through a worked example in this chapter, and we conclude with a discussion about future stealth assessment research, to move this work into classrooms for adaptivity and personalization.

VR, AR, and MR are becoming ubiquitous in consumer gaming, military applications, and office environments. These successes are driving emerging efforts to integrate these immersive media into the K-12 classroom. In this chapter, first we summarize the distribution and availability of the infrastructure needed for using VR and MR in the schools. Using immersive media requires a technology infrastructure consisting of dependable high-speed Internet connectivity to the classroom, a ratio of at least one-to-one computer to student, an interactive white board, and curriculum materials that can be monitored and controlled by the teacher. This infrastructure is quickly becoming a reality. However, a larger and more complex barrier remains: integrating the new technologies with existing classroom systems and with existing and emerging pedagogical practice. I argue that the Digital Teaching Platform serves as a model for classroom practice. The evolving nature of digital curricula, formative assessment, and classroom practice impact how teachers will be able to integrate these new technologies. Finally, I examine how immersive media such as virtual reality, augmented reality, mixed reality, and multi-user virtual reality can work as supplemental digital materials for instruction and assessment. In particular, I focus on the sensory comfort and fidelity of interaction as these issues impact the viability of these technologies in the classroom.

Virtual reality has gained worldwide interest among the researchers in the field of educational technology recently. This chapter presents an overview of virtual reality research in education and also a bibliometric analysis was performed to evaluate the publications on virtual reality from 1995 to 2016, based on the Thomson Reuters’s Web of Science (WoS). A total of 975 related documents were analyzed based on their publication patterns (documents types and languages, major journals and their publications, most prolific authors, most productive journals and their publications, and international collaborations). Bibliometric results show that the number of article has been increasing since 1995 exponentially. USA, UK and Chinese Taipei are the top 3 most productive countries/regions which are involved in virtual reality research in education. The findings would help the researchers to understand current developments and barriers in applications of virtual reality in education.

This chapter describes a design strategy for blending virtual reality (VR) with an immersive multi-user virtual environment (MUVE) curriculum developed by the EcoLearn design team at Harvard University for middle school students to learn ecosystems science. The EcoMUVE Pond middle grades curriculum focuses on the potential of immersive authentic simulations for teaching ecosystems science concepts, scientific inquiry (collaborative and individual), and complex causality. The curriculum is inquiry-based; students investigate research questions by exploring the virtual ecosystem and collecting data from a variety of sources over time, assuming roles as ecosystems scientists. The implications of blending in VR for EcoMUVE’s technical characteristics, user-interface, learning objectives, and classroom implementation are discussed. Then, research questions for comparisons between the VR version and the “Classic” version are described. The chapter concludes with generalizable design heuristics for blending MUVE-based curricula with head-mounted display immersion.

This chapter examines the use of mixed-reality technologies for teaching and learning, particularly for more active and collaborative learning activities. The basis for this work was the creation of the MiRTLE platform—the Mixed Reality Teaching and Learning Environment. We report on some of the lessons learnt from using this platform on a range of different courses and describe how different active/collaborative approaches were used. We also provide evidence of the effect of these different approaches on the overall student attainment and discuss the implications on the use of this technology. We then consider some of the technological research being done to develop these mixed reality learning spaces and the affordances offered by this approach. Finally we reflect on the tensions between the pedagogy and technology and consider the implications for the wider systems that support teaching and learning and co-creative collaboration in mixed-reality environments.

The places where Virtual Reality (VR) can really make a difference in learning are those in which the VR can bring a truly unique experience to students. The simulated online world of games is an ideal way to take advantage of the capabilities of this new technology. Games provide a set of structures that not only scaffold learners in solving complex problems but also provide a great deal of freedom to explore personally interesting pathways. In particular, Massively Multiplayer Online Role Playing Games (MMOs) offer an environment that supports social learning and exploration around increasingly challenging problems. VR can greatly enhance MMOs through opportunities for more natural and expressive communication and collaboration as well as ways to visualize the complex information resulting from interactions in this space. When this approach is applied in an educational context, learners can be presented with challenging problems, requiring participation from multiple players around realistic scientific concepts. As this genre moves forward it can explore interesting hybrid approaches that combine VR with Augmented Reality (AR) and traditional displays to meet the needs of schools, teachers, and learners.

This chapter provides a summary of some of this lab’s immersive media and embodied STEM learning research. It focuses on the integration of gesture in learning, and a new gesture-based assessment. A taxonomy for embodiment in education is included. The chapter concludes with several design principles that the Embodied Games Lab has culled over the years while creating educational content that maximizes the affordances of virtual and mixed reality technologies and meshes those with best pedagogical practices.

In this chapter, I explore how new learning environments, such as mixed reality interfaces (i.e., interfaces that combine physical and virtual information), can prepare students for future learning. I describe four controlled experiments that I have conducted over the years where students learned complex concepts in STEM and where a Tangible User Interface created a “Time for Telling”. This is followed by a summary the findings, a discussion of the possible mechanisms for the effect found in those studies, and a suggestion of design guidelines for creating this type of constructivist activities. I conclude by discussing the potential of mixed reality interfaces for preparing students for future learning.

This chapter articulates a Grand Challenge of applied, collective research. In our judgment, we must establish educational research laboratories with access to the next generations of tools and complementary research agendas to rapidly accumulate knowledge. Immersive technologies cannot be examined in the hope of changing one parameter and observing the effects. Rather, immersion, particularly VR and MUVEs, by their very nature alter perspective, context, and even the participants’ sense of self, while immersion in the form of MR and AR alters our sense of participation in the external environment. Design strategies for this scholarly work include applied, collective research; studies on what works when for whom, it what context; and research balanced between design and evaluation. Our illustrative research agenda includes thematic areas: authenticity, representation of the self, social immersion, and technological support and infrastructure. We also suggest developing implementation testbeds that provide authentic contexts to judge the educational value of immersive learning experiences.