Studying the effects of educational games on cultivating computational thinking skills to primary school students: a systematic literature review

To accomplish the main goal highlighted in Sect. “Related work” , the SLR was steered by the subsequent research questions, aligned with the research’s primary objective:

To find empirical studies regarding educational games designed for teaching CT and programming to young students, the subsequent search term was applied:

("serious game*" OR "educational game*") AND ("primary school children" OR "young children" OR "K-12" OR "primary education") AND ("teaching programming" OR "computational thinking" OR "learning programming" OR "algorithmic thinking")

The following digital research databases were searched initially between April 2021 and September 2022: Scopus, ACM Digital Library, SpringerLink, Science Direct and IEEE Xplore. Afterward, additional searches were carried out on Google Scholar to discover published and unpublished material that had not been indexed by the databases. The process of retrieving data from the studies was performed using a data extraction form created in Microsoft Excel. Subsequently, the selection process consisted of three phases. During the first phase, we looked over each paper’s title, abstract, and keywords. At this stage, we saved the potentially relevant papers and eliminated the duplicate or irrelevant studies. By means of this methodology, this study also excluded literature reviews. The second phase involved eliminating papers that were not published in English, papers with unavailable full texts, and publications that were not complete articles (such as extended abstracts).

Figure 1 depicts the full research selection procedure as well as the actual number of articles in each phase. The two researchers that carried out the whole study rigorously applied Kitchenham's (2004) recommendations for conducting SLRs and agreed on the research protocol and the paper selection process prior paper selection. Both researchers had experience in conducting reviews and SLRs. After retrieving the records returned by applying the search query, the two researchers independently carried out each one of the three phases of the selection process and compared the results after each phase in order to agree on a common pool of records for the subsequent phase.

After completing the second phase, the papers were screened according to a set of inclusion and exclusion criteria based on the SLR methodology proposed by Kitchenham (2004), as presented in Table 2, to finalize the study selection process.

After completing the third phase, the complete text of each potential eligible study was read and the studies were analyzed. During this phase, the studies were scrutinized based on the traits presented in Table 3.

The data from the studies was extracted using a Microsoft Excel-based form created for this purpose. Table 3 shows the characteristics that were considered to extract data.

The initial search yielded 1410 results. The total number of papers gathered throughout the first phase was 175, with the remainder discarded as irrelevant or duplicate and after the second phase 150 studies remained as possible included papers. Next, the papers were filtered and after applying the inclusion and exclusion criteria, 61 main studies were included in the SLR. The distribution of the studies between 2010 and September 2022 is represented in Fig. 2.

According to the yearly distribution of the 61 articles represented in Fig. 2, a rising trend in the publication of empirical studies on the use of SGs to teach young students CT skills and fundamental programming concepts has been seen in recent years, which is consistent with the growing popularity and importance of CT. In particular, 48 relevant empirical studies were discovered after 2017, representing about 80% of all empirical research conducted after 2010.

The majority of the papers published are from conferences (33), with the remaining 25 coming from journals. Three (3) empirical studies are also included that were part of master’s (2) and Ph.D. theses (1).

Figure 3 shows the main subject areas of journal and conference publications. As illustrated in Fig. 3, the primary subject area of journal publication is Educational Computing (24). Computer Science or Information Technology (14) and Serious Games (10) are two more prominent subjects. According to the results of the subject areas of the publications, the majority of empirical investigations involve Information Technology, Computer Science and Education. Taking this into consideration, we can infer that collaboration between researchers and those involved in the educational process (educators, students) is critical for developing games aimed at cultivating CT in young children.

The main objective of this study was to identify the most recent empirical research on the application of educational games for teaching young students’ CT skills and basic programming concepts, as well as to explore the effects of these games on the cultivation of the aforementioned skills. To that end, we examined 61 empirical studies concentrating on the primary school age group that were chosen after searching the literature over the previous twelve years (2010 until September 2022).

In terms of the programming concepts covered by the identified studies, it was discovered that the most extensively covered notions by academics and game developers are fundamental programming concepts such as sequence, selection, and iteration, whilst OOP concepts and recursion are the least covered. Maybe this is because these notions need a high degree of abstraction, which makes it challenging to teach them to elementary school students who have not yet mastered abstract thought (Zaharija et al., 2013). As highlighted by Relkin et al. (2021), taking into account these challenges and other developmental aspects is of utmost importance when developing educational programs specifically targeted at teaching CT to young children (Papadakis, 2021). In addition, regarding the CT skills that games seek to cultivate in learners, it appears that the majority of them aim to develop algorithmic thinking, decomposition, abstraction, as well as pattern recognition and modularity, which are skills recognized by numerous researchers (Brennan & Resnick, 2012; Atmazidou & Demetriadis, 2016; Shute et al., 2017; Grover & Pea, 2018), as fundamental elements of CT. Additionally, in terms of debugging, it has been observed that there is a scarcity of empirical studies focusing specifically on this aspect of CT.

Moreover, academics have used a range of tools, frequently in tandem, to collect data and assess the effectiveness of games designed for inexperienced programmers. Most often, perceptions/attitudes questionnaires are used to investigate typical emotional learning results, especially for the purpose of studying the incentives and perspectives related to learning techniques that utilize games. It was also found that in-game evaluations utilizing learning analytics, or game logs, was another popular technique to measure the efficacy of games in learners' acquisition of new knowledge, while researchers in several studies preferred employing pre-test and post-test procedures. Observations during game play, interviews with players, and even relevant worksheets constitute some of the extra data gathering methods employed by the researchers. Additionally, Learning Analytics approaches which may have the potential to provide students with personalized feedback on their performance and compare it with that of the entire group can help alleviate the burden on teachers when managing a substantial number of students, particularly in obtaining data that would be strenuous to gather manually (Guenaga et al., 2021). Although it appears that learning analytics techniques is a useful way to recognize the acquisition of CT, some details are unlikely to be captured using this approach to gather data alone (Forquesato, 2018). Therefore, it is necessary to use additional techniques like observing and conducting interviews, which all together could form “integrated assessment systems” providing an overall view of student understanding (Guenaga et al., 2021). Consequently, we believe that in order to improve our understanding of the educational influence of games that aim to cultivate CT skills, it is essential to utilize a wide variety of tools to measure their effectiveness.

Shifting our focus from the measurement tools to the evaluation methods employed in the analyzed studies, we observe the utilization of both formative and summative assessments. The selection of these approaches is contingent upon the main objectives of the teaching intervention and the particular goals set for each study. Moreover, in relation to the methodology employed, it was observed that the majority of studies have a primary focus on outcomes, adopting an outcome-oriented approach. On the other hand, a smaller number of studies center their attention on the process itself, adopting a process-oriented perspective.

The majority of the studies that looked into the efficiency or impact of games on the acquisition of the suggested topics (39 out of 47) reported positive results, demonstrating that educational programming games can assist elementary school students in developing CT abilities or understanding fundamental programming concepts. Moreover, no study showed a negative impact, whereas 8 investigations reported no appreciable change following the instructional intervention. The results of this research unequivocally demonstrated a strong positive impact of using an educational game as a teaching tool for CT skills and basic programming concepts to elementary school kids. Students established a good attitude toward programming through game-based activities, and the utilization of programming games proved to be a powerful motivator for young children in engaging them in CT activities. Furthermore, it is advised that in future studies of a similar nature, which examine the educational impact of games specifically designed to foster CT skills, researchers should make an effort to involve a broader range of participants. This would facilitate a more dependable generalization of the learning outcomes.

Nevertheless, one aspect that appears to be absent from the relevant research included in this study is player collaboration during the game. There was just one publication that examined the aspect of collaboration, indicating that future research may look into how participants communicate during the game and how this impacts their efficacy in obtaining new knowledge. Additionally, the research findings show that another area that has not been thoroughly examined thus far is the adaptation of the game’s level to the player’s profile, as well as the manner in which the educational information is delivered to the player. Taking these factors into account, we can infer that there is still opportunity for progress in these domains and that further research is necessary to tap into these possibilities.

Although this study has some limitations that are presented in the next subsection, it also has a distinct contribution in the field. Specifically, this study could be used as a basis for further experimental research on the impact of games on programming and CT skills among students. This could also include exploring the acceptance of using such games for teaching programming and acquiring CT skills, among both teachers and students. Also, this study may be helpful to educators who often express a lack of knowledge on effectively integrating a game-based learning strategy into their lessons or to policymakers looking for methods to incorporate CT education into the curriculum. Finally, the SLR recorded various important approaches, such as collaboration and adaptation, which are missing from existing games and has highlighted areas of future work that can have a unique impact on the design and more effective application of educational games for cultivating CT skills.

There are some limitations in the provided literature review that need to be pointed out. To start with, it is possible that there are research papers that were not incorporated into the articles we have selected because some digital research databases could not be accessed. As a result, certain publications could not be collected or analyzed. Furthermore, a considerable number of empirical studies involved a limited number of individuals in the intervention process, thus making it challenging to extend the efficacy of these investigations to a broader population.

In line with the research questions of this paper, emphasis is placed on investigating CT skills and programming concepts, as well as the methodology and the results of the relevant studies. In the analysis and discussion of results, the games on which the studies were based are listed, as we believe that the type and the elements of a game may have an effect on learning outcomes. However, the study of the specific characteristics of the games was not included in the objectives of this SLR. Although the majority of the related studies demonstrated positive cognitive and affective results, it is crucial for future research to conduct an SLR that explores what types of educational games for programming and CT exist, what their distinct characteristics are, as well as their suitability as tools for cultivating CT skills considering children's age.

Additionally, it would be intriguing for researchers to look into unresolved concerns and missing characteristics from the games being examined, such as collaborative play, adaptation to learner profiles and assistance for developing personalized lesson plans. Afterwards, they can integrate these characteristics into an improved educational game platform, and conduct additional experiments to investigate how these features affect the learning process.