Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning

Highlights

• Engagement in the game has a positive effect on learning.

• Immersion in the game does not have a significant effect on learning.

• Challenge affected learning both directly and via the increased engagement.

• Skill did not affect learning directly but only via the increased engagement.

• Perceived challenge was an especially strong predictor of learning outcomes.

Abstract

In this paper, we investigate the impact of flow (operationalized as heightened challenge and skill), engagement, and immersion on learning in game-based learning environments. The data was gathered through a survey from players (N = 173) of two learning games (Quantum Spectre: N = 134 and Spumone: N = 40). The results show that engagement in the game has a clear positive effect on learning, however, we did not find a significant effect between immersion in the game and learning. Challenge of the game had a positive effect on learning both directly and via the increased engagement. Being skilled in the game did not affect learning directly but by increasing engagement in the game. Both the challenge of the game and being skilled in the game had a positive effect on both being engaged and immersed in the game. The challenge in the game was an especially strong predictor of learning outcomes. For the design of educational games, the results suggest that the challenge of the game should be able to keep up with the learners growing abilities and learning in order to endorse continued learning in game-based learning environments.

Introduction

Pervasive student disengagement is both a national and an international problem, with 20–25% of students in 28 OECD (Organisation for Economic Co-operation and Development) countries classified as having low participation and/or a low sense of belonging (Drigas et al., 2014, Willms, 2003). A promising strategy for increasing engagement in a meaningful way has been thought to stem from video games (Connolly et al., 2012, Gee, 2007, Steinkuehler et al., 2012) and gamification (Hamari, Koivisto, & Sarsa, 2014) as observed by educational scholars for several decades.

In an ideal educational game setting, students learn how to solve complex problems. The problems within a game typically start off easy and then progressively become more difficult as players' skills develop. Players are motivated to learn, in part, because learning is situated and occurs through a process of hypothesizing, probing, and reflecting upon the simulated world within the game. In addition, the goals are clear, and information becomes available to players at just the time that it is needed to reach each goal. Making sense of that information becomes a goal intrinsic to gameplay. As McGonigal (2011) observed:

“In a good computer or video game you're always playing on the very edge of your skill level, always on the brink of falling off. When you do fall off, you feel the urge to climb back on. That's because there is virtually nothing as engaging as this state of working at the very limits of your ability. (p. 24)”

Computer games have been observed to scaffold learning in ways that keeps players at the edge of their seats fostering continued interest in the game for hours, weeks, and even years. Players hone their skills and build knowledge as long as they continue to play. In some rare cases game developers, such as Valve (see Valve Corporation, 2007, Valve Corporation, 2011), have described their effective design framework of “layered learning” which attempts to optimize learning elements consistent with interrelated principles of challenge, skills, engagement and immersion. In this framework, engagement and learning are necessary to keep players progressing in the game, and visa-versa. Entertainment game developers, however, are less concerned with how the learning may transfer to the outside world.

This study contributes to the current body of literature on learning by investigating the above mentioned psychological factors of challenges, skills, engagement and immersion that have been commonly believed to be characteristic to a good game and learning experiences. In the study, we investigated the relationship among these variables, and the extent to which they predict learning, in physics-based video games. The study also allowed us to explore the extent to which engagement and immersion may mediate the effect of challenges and skills on learning, as predicted by a theory of flow experiences (Csikszentmihalyi, 1990). Drawing on flow theory, perceived challenge and skills (the main two elements of flow) are hypothesized to predict engagement and immersion, which in turn are believed to predict perceived learning. We utilized a psychometric survey (see e.g. Nunnally, 1978) asking participants about their subjective learning experience after playing two video games designed by two research teams in the U.S. We then employed structural equation modeling in order to investigate these direct and mediated effects among flow (skill and challenge), engagement, immersion, and learning outcomes. The following research questions were examined:

• Do challenge and skills predict engagement and immersion in game-based learning?

• Do engagement and immersion predict perceived learning in game-based learning?

• How engagement and immersion mediate the effect of challenge and skills on perceived learning in game-based learning?

Serious games, gamification and game-based learning are distinct from entertainment-oriented games in that, while they are often also enjoyable, they are designed for primary end purposes other than entertainment and leisure (Davidson, 2008, Hamari and Koivisto, 2015b). Educational games, the focus of this study, are developed for the primary purpose of educating or training. Serious and educational games often combine the concentration demanded by challenging activities and the enjoyment experienced when maximally utilizing one's skills, as in “serious play” or “playful work” (Csikszentmihalyi & Schneider, 2000).

The integration of work and play characterizes the psychological state that Csikszentmihalyi (1990) has called “flow.” Flow refers to a state of mind characterized by focused concentration and elevated enjoyment during intrinsically interesting activities (Shernoff, Csikszentmihalyi, Schneider, & Shernoff, 2003).

Research on flow in general has found that utilizing high degrees of skills in challenging tasks results in deep concentration, absorption, or immersion. Flow has also been related to learning, talent-development, academic achievement, and creative accomplishment in a profession (Csikszentmihalyi, 1996, Csikszentmihalyi et al., 1993). In the game-based learning and gamification contexts specifically, studies predict that learning and gamified curricula will become more and more commonplace as a method to invoke engagement and flow in students (Crisp, 2014). Moreover, studies have found these technologies do indeed invoke flow experiences (Hamari and Koivisto, 2014, Procci et al., 2012) and have the potential to affect consequent learning outcomes (Barzilai and Blau, 2014, Brom et al., 2014a, Chang et al., 2012, Hung et al., 2015, Liu et al., 2011, Sabourin and Lester, 2014) although there are also studies that find no significant association (e.g. Brom, Bromová, Děchtěrenko, Buchtová, & Pergel, 2014b). Therefore, investigating the structure of this phenomenon is essential. Specifically, what are conditions theorized as essential to flow experienced by players of games, to what extent do such conditions lead to greater engagement and immersion, and to what extent are conditions for flow and the engagement or immersion that these conditions may engender relate to learning through the game.

The subjective experience of flow, according to Csikszentmihalyi's (1990) theory, is enhanced by certain experiential conditions or properties of the task. The most central condition for flow experiences to occur is that the individual uses a high level of skill to meet a significant challenge. The activity is therefore not too easy for one's skills, nor is it impossibly difficult. Reaching the goal is doable: one has a reasonable chance of success with sincere and concerted effort. Typically, the challenge and skill are high and in balance—individuals stretch their skills to their limits in pursuit of a challenging goal. The various combinations of high or low challenges and skills predict distinct psychological states: (a) apathy, resulting from low challenge and low skill; (b) relaxation, resulting from high skill but low challenge; (c) anxiety, resulting from high challenge but low skill; and (d) flow, resulting from high challenge combined with high skill. This model later evolved into one with eight flow channels including four intermediary or transitional states between these four quadrants (Strati, Shernoff, & Kackar, 2012).

According to both Csikszentmihalyi's (1990) and Bronfenbrenner's (1979) theory, more cognitively complex and challenging classwork engages students more deeply. Research corroborates this theoretical stance, demonstrating that students are significantly more engaged and concentrate much harder when challenged in classrooms. The challenge-skill dynamic has also been found to increases motivation while extending players' capacities (Fullagar, Knight, & Sovern, 2013). When invited to engage in complex problem solving instead of confronting topics only superficially, students see more connections, becoming more intrinsically interested, and thus also pay better attention. Newmann (1992) referred to curriculum that fosters higher order thinking skills and is perceived as relevant as “authentic,” but found authentic curricula in schools to be rare. Although some students might perceive being challenged as arduous and unpleasant, most students state that they like challenging work, value cognitive complexity, and are willing to work hard to complete schoolwork that challenges them (Newmann, Wehlage, & Lamborn, 1992). Conversely, national studies have repeatedly found that lack of challenge is a common reason for disengagement (Shernoff, 2010, Shernoff, 2013, Yazzie-Mintz, 2007).

Research has also shown that students have higher motivation, via greater self-efficacy and self-worth, when they perceived themselves to be competent (Covington, 1985). Perceptions of skill and competence have long been considered one of the most important determinants of achievement expectations, motivation, and behavior (Nicholls, 1979, White, 1959). Because success is positively valued and failure is negatively valued, people are inherently motivated and engaged to produce the feeling of competency. Some have argued that the perception of their competence and how it relates to perceived chances of success is a fundamental motivator for learning (Thomas, 1980), contributing to continuing motivation and global self-worth. Conversely, many students may feel at least somewhat uncomfortable or insecure as a function of perceived incompetence, resulting in a reluctance to take risks or take on new challenges that might increase competencies.

Engagement resembling flow experiences reflect a state of complete absorption in a challenging activity with no psychic energy left for distractions. All attention is focused on relevant stimuli. For example, composers have described a shift in consciousness when music is “flowing” from the depth of their souls, stirred by inspiration, like being part of a river (Custodero, 2005). The high level of focus is often accompanied with a feeling that the activity is going well, that one is being successful, and often with feelings of inner peace, joy, or wonder. Csikszentmihalyi (1990) observes that when one loses self-consciousness during flow, this may lead to self-transcendence, a sense of expanding the boundaries of the self towards merging with one's environment. This description seems consistent with gamer's accounts of immersion or “being there” in the game. Research also suggests that the higher the challenge, the greater the engagement or sense of immersion (Shernoff, 2010).

This challenge-skill dynamic introduces a growth principle that is also inherently related to learning. When learning a new skill, the challenge of even a basic task may exceed a student's beginning level of ability, and hence one may feel overwhelmed. To reach flow, the level of skill must increase to match the challenge. Sufficient practice may be needed until the skill is mastered. Once mastered, a higher level of challenge is needed for one's skill level to increase yet again. Thus, individuals may progress through increasingly difficult challenges at ever-higher levels of skill. Because most video games allow the player to adjust the level of challenge as skills are increasing, the continuing cycle of new challenges results in the gradual building of increased competencies targeted by the game (Fullagar et al., 2013); and because the flow experience is so enjoyable, players are intrinsically motivated to improve their skills in order to meet the raised challenge and re-enter flow.

Generally, literature in the game-based context reflects similar understanding of the phenomenon that the challenge in games may drive a players' sense of flow and engagement (e.g. Wang and Chen, 2010, Hwang et al., 2012). Prior research on challenges in game-based environments has indeed showed that challenge and skill are salient factors leading to the overall flow experience (Hamari and Koivisto, 2014, Hung et al., 2015, Procci et al., 2012, Wang and Chen, 2010). Furthermore, Hung et al., 2015 found that challenge in the game-based learning increased flow and learning outcomes as well as satisfaction. However, Ronimus, Kujala, Tolvanen, & Lyytinen, 2014 found no significant relationship between challenge and children's engagement in a game-based reading platform. While the factors and conditions related to flow and immersion in learning activities are often implied, overall prior studies systematically investigating the relationships among challenge and skill, engagement, immersion and further learning in game-based learning are scarce. This study attempts to contribute to the literature in this area by investigating both the antecedents of engagement and immersion as well as their impact on learning in meaningful game-based challenges.

In this study, engagement is conceptualized as the simultaneous occurrence of elevated concentration, interest, and enjoyment encapsulating the experience of flow. All three phenomena are inherently related to learning (Shernoff, 2013). Concentration or absorption, which is central to flow (Csikszentmihalyi, 1990), is related to meaningful learning (Montessori, 1967), including depth of cognitive processing and academic performance (Corno & Mandinach, 1983). Interest directs attention, reflects intrinsic motivation, stimulates the desire to continue engagement in an activity, and is related to school achievement (Schiefele, Krapp, & Winteler, 1992). Enjoyment is a positive feeling related to the demonstration of competencies, creative accomplishment, and school performance (Csikszentmihalyi et al., 1993). In this conceptualization, engagement in learning is highest when all three components are simultaneously stimulated.

Engagement has been a canonical concept in game-based learning research. However, there are surprisingly few studies that actually measure psychological engagement in the game-based learning context. Engagement has been separated into three types of engagement: behavioral, cognitive and emotional (see e.g. Fredricks, Blumenfeld, & Paris, 2004). Pellas (2014) found that these three dimensions of engagement were correlated in a game-based learning environment. Coller and Shernoff (2009) found that those student who did homework and labs for an undergraduate engineering course in a game-based format were clearly more engaged in the activity than those who completed homework normally. Akkerman, Admiraal, and Huizenga (2009) found that ‘storifying’ history using mobile games had a positive effect on the student engagement. Previous studies have also found that voiceovers in a game can also have a positive effect on engagement in game-based learning environment (Byun & Loh, 2014). However, engagement into an educational game has also been observed to be moderated by gaming experience (Deater-Deckard, El Mallah, Chang, Evans, & Norton, 2014) and the nature of the learning tasks (Eseryel, Law, Ifenthaler, Ge, & Miller, 2013).

Previous studies have also found a positive association between engagement and learning (e.g. Hsu, Tsai, & Wang,2012; Huizenga, Admiraal, Akkerman, & Ten Dam, 2009) and that engagement in game can redirect unwarranted focus on grades to learning (Tüzün, Yilmaz-Soylu, Karakuş, Inal, & Kizilkaya, 2009). For example, Sabourin and Lester (2014) found that a game-based learning environment was able to both support learning and promote engagement. Hou (2015) and Brom et al. (2014a), however, establish a positive relationship between flow and learning. Admiraal, Huizenga, Akkerman, & Dam, 2011 found that flow had a positive effect on student performance in the game but did not have an effect on learning outcomes; however, if the students were engaged in a group competition, the more the students learned. Other studies have found that while games lead to learning gains, engagement remained unaffected (van der Spek, van Oostendorp, & Meyer, 2013).

Similarly, gamification settings have been found to influence engagement. For example, in the domain of commerce, Bittner and Shipper (2014) found that the effect of gamification on behavioral engagement was mediated by flow and enjoyment. Similarly, Hamari, 2013, Hamari, 2015) found that gamification increased trading activity but it was deemed that the results greatly depend on how engaged and interested the users are toward the gamification features in a service. In learning context, Huizenga et al. (2009) have similarly concluded that in order for the game-based solution to have an effect on learning, students should first actually be engaged within the game.

The sense of immersion characterizing flow experiences is also related to learning and related emotions (e.g. Fassbender, Richards, Bilgin, Thompson, & Heiden, 2012). For example, recent experiments in neuroscience have demonstrated that when a reader is fully engrossed in a novel, the human brain is activated not only in areas responsible for attention; it also dramatically “lights up” in areas controlling affect and emotion (Thompson & Vedantam, 2012). Flow theory has been a primary theoretical base for exploring the implications of learning through immersion or “being enveloped” by a virtual learning environment because the emotional composition of these experiences resemble flow and precipitate a deeper engagement with learning. Research has explicitly related the sense of “presence,” “being there,” “immersion,” or “flow” in different virtual reality interfaces with positive learning outcomes (e.g., Abrantes and Gouveia, 2012, Fassbender et al., 2012). In addition, there is evidence that fantasy through simulations and games promotes intrinsic motivation and can enhance learning compared to instruction without fantasy elements (Lepper and Hodell, 1989, Parker and Lepper, 1992), in part by focusing the learner's attention on relevant features of the learning environment (Lepper & Molone, 1987). However, currently there is a dearth of studies that investigate the relationship between immersion and learning in game-based learning environments. The only study (as far as we know) that does so, by Cheng, She, and Annetta (2015), found that immersion has a positive impact on learning outcomes especially when the players gaming performance was high.

Overall, according to larger theoretical developments as well as the body of empirical literature there is reason to believe that flow (challenge and skills), engagement, and immersion have a positive impact on learning. Research suggests that increases in challenge and skills relate to higher degrees of engagement and immersion; and that challenges, skills, engagement, and immersion may also relate to increased learning directly in addition to the mediated effects.

Thus far, there has also been little research that has applied structural research models among variables surrounding engagement and immersion to investigate their interdependencies and pathways to predicting learning. Most studies either investigate the relationship between game features and learning directly without measuring mediating psychological factors, or else investigate the relationship between game features and psychological factors but do not extended the measurement to further learning outcomes. In the present study, we investigate mediations and direct effects in order to gauge the phenomenon more reliably.

Based on the larger theoretical developments as well as the body of empirical literature, we hypothesize that increased challenge, skill, engagement, and immersion in a game or gamified experience will have a beneficial effect on learning. The specific hypotheses stemming from the three research questions outlined in Section 1 are described below and represented in Fig. 1.