The Promises and Challenges of Artificial Intelligence for Teachers: a Systematic Review of Research

There have been several waves of emerging educational technologies over the past few decades, and now, there is artificial intelligence (AI; Bonk & Wiley, 2020). The term artificial intelligence was first mentioned in 1956 by John McCarthy (Russel & Norvig, 2010). Baker and Smith (2019) pointed out that AI does not refer to a single technology but is defined as “computers [that] perform cognitive tasks, usually associated with human minds, particularly learning and problem-solving” (p. 10). AI is a general term that refers to diverse analytical methods. These methods can be classified as machine learning, neural networks, and deep learning (Aggarwal, 2018). Machine learning is defined as the capacity of a computer algorithm learning from the data to make decisions without being programmed (Popenici & Kerr, 2017). Although numerous machine learning models exist, the two most used models are supervised and unsupervised learning models (Alloghani et al., 2020). Supervised machine learning algorithms build a model based on the sample data (or training data), while unsupervised machine learning algorithms are created from untagged data (Alenezi & Faisal, 2020). In other words, the unsupervised model performs on its own to explore patterns that were formerly undetected by humans.

AI is used in education in different ways. For instance, AI is integrated into several instructional technologies such as chatbots (Clark, 2020), intelligent tutoring, and automated grading systems (Heffernan & Heffernan, 2014). These AI-based systems offer several opportunities to all stakeholders throughout the learning and instructional process (Chen et al., 2020). Previous research conducted on the educational use of AI presented AI’s support for student collaboration and personalization of learning experiences (Luckin et al., 2016), scheduling of learning activities and adaptive feedback on learning processes (Koedinger et al., 2012), reducing teachers’ workload in collaborative knowledge construction (Roll & Wylie, 2016), predicting the probability of learners dropping out of school or being admitted into school (Popenici & Kerr, 2017), profiling students’ backgrounds (Cohen et al., 2017), monitoring student progress (Gaudioso et al., 2012; Swiecki et al., 2019), and summative assessment such as automated essay scoring (Okada et al., 2019; Vij et al., 2020; Yuan et al., 2020). Despite these opportunities, the educational use of AI is more behind what is expected, unlike in other sectors (e.g., finance and health). To achieve successful AI implementation in education, various stakeholders, specifically, teachers, should participate in AI creation, development, and integration (Langran et al., 2020; Qin et al., 2020).

The evolution of education towards digital education does not imply that people will need less teachers in the future (Dillenbourg, 2016). Instead of speculating if AI will replace teachers, understanding the advantages that AI offers teachers and how these advantages can change teachers’ roles in the classroom is more reasonable (Hrastinski et al., 2019). Salomon (1996) demonstrated this during the early stages of development of educational technology by pointing out the need to consider how learning occurs through and with computers. As for AI, Holstein et al. (2019) suggested that in the future, AI-based machines can help teachers perform what Dillenbourg (2013) emphasized as their orchestrator role in the learning and teaching process. For AI to be able to truly help teachers in this way, however, it must first learn effective orchestration of learning and teaching from teachers’ data. This is because effective teaching depends on teachers’ capability to implement appropriate pedagogical methods in their instruction (Tondeur et al., 2020), and their pedagogically meaningful and productive teaching incidents can serve as models for AI-based educational systems (Prieto et al., 2018). That is, the data collected from the learning setting orchestrated by teachers form the foundation of AI-based teaching. For example, the data may help researchers to understand when and how teaching is effectively progressing (Luckin & Cukurova, 2019; Luckin et al., 2016). To prove that the role of teachers in providing the data on features of effective learning is crucial for the development of AI algorithms, we investigated the kind of data collected from teachers and teachers’ roles in the creation of AI algorithms.

To effectively integrate AI-based education in schools, teachers must be empowered to implement such integration by endowing them with the requisite knowledge, skills, and attitudes (Häkkinen et al., 2017; Kirschner, 2015; Seufert et al., 2020). However, teachers’ AI-related skills have not yet been sufficiently defined because the potential of AI in education has not yet been fully exploited (Luckin et al., 2016). To explore teachers’ AI-related knowledge, skills, and attitudes, their engagement with AI-based systems within their teaching setting has to be investigated in detail (Dillenbourg, 2016; Seufert et al., 2020). Therefore, in this study, we reviewed empirical research on how teachers interacted with AI-based systems and how they participated in the development of AI-based education systems. We believe that our synthesis of empirical research on the topic will contribute to the identification of AI-related teaching skills and the effective implementation of AI-based education in schools with the support of teachers.

This study explored the perspective and roles of teachers in AI-based research through a systematic review of the latest research on the topic. Our specific research questions (RQs) are as follows:

Table 1 below lists these RQs with their corresponding rationales.

In reviews of research, several methods are used to select the studies that will be reviewed. Studies published in important journals of a given domain are selected from databases such as ProQuest (Heitink et al., 2016), Education Resources Information Center (ERIC), and the Social Science Citation Index (SSCI) (Akçayır & Akçayır, 2017; Kucuk et al., 2013). For this review, we selected English-language scientific studies on teachers’ AI use that were published in journals from the Web of Science (WoS) database within the last 20 years until 14 September 2020. We used this method because the field tags (e.g., the topic and research area) of the studies were easy to access from the WoS database (Luor et al., 2008). We used the following search string: “artificial intelligence,” “deep learning,” “reinforcement learning,” “supervised learning,” “unsupervised learning,” “neural network,” “ANN,” “natural language processing,” “fuzzy logic,” “decision trees,” “ensemble,” “Bayesian,” “clustering,” and “regularization.” To narrow our search, we used “teacher,” “teacher education,” “teacher professional development,” “K-12,” “middle school*,” “high school*,” “elementary school*,” and “kindergarten*.” We selected the search strings based on the main concepts of AI in education in past studies and literature reviews (Baran, 2014; Zawacki-Richter et al., 2019). Figure 1 presents our study search procedure.

In our first search, we found 751 studies. Next, we checked them to see if they met our inclusion and exclusion criteria. Our inclusion criteria were as follows: (a) empirical studies on AI in pre-service and in-service teacher education and on in-service teachers’ use of AI; (b) studies on AI applications and algorithms (e.g., personal tutors, automated scoring, personal assistant; decision trees, and artificial neural networks) for teaching or analyzing teachers’ data; and (c) studies on data collected from in-service K-12 teachers or pre-service teachers. We excluded editorials, reviews, and studies conducted at the higher education level. After we applied the criteria, 44 articles remained suitable for inclusion in this study.

The publication year of the articles was noted to determine the distribution of the studies over time (RQ1). For RQ2, the following categories and category numbers were assigned to the data collected from teachers in previous AI-based research: self-report (1), video (2), interview (3), observation (4), feedback/discourse (5), grading (6), eye tracking (7), audiovisual/accelerometry (8), and log file (9). We qualitatively analyzed the content of the 44 articles to determine the advantages and challenges of AI for teachers (RQ4 and RQ5, respectively) and teachers’ roles in AI-based instruction as found in research (RQ3). We coded the studies not with the preliminary or template coding scheme, which would have unnecessarily limited them by fitting them into a pre-determined coding scheme (Şimşek & Yıldırım, 2011), but with the open coding process (Akçayır & Akçayır, 2017; Williamson, 2015), which followed these steps: (1) Familiarize with the whole set of articles; (2) Choose a document randomly, consider its primary meaning, and write down your thought on such meaning on the margin of the document; (3) List all your thoughts on the subject, combine similar thoughts, create three columns for key, unique, and leftover thoughts, and put each thought in the appropriate column; (4) Code the text; (5) Find the most illustrative phrases for your thoughts and turn them into categories; (6) Decide on an abbreviation for each category and alphabetize these abbreviations; (7) Incorporate the final codes and perform the initial analysis; and (8) Recode the studies if needed. To classify the AI methods (RQ6), we used previous literature reviews of AI use in diverse areas such as higher education, medicine, and business (Borges et al., 2020; Contreras & Vehi, 2018). We performed the investigator triangulation method to ensure the reliability of the coding process (Denzin, 2017). Accordingly, the first author coded the articles separately and then shared the codes with the second author. We negotiated disagreements by checking the code list and the relevant studies, and we updated and renamed some categories. Finally, we recoded the studies using the final code list.

Our analysis indicated that the first study on teachers’ AI use was published in 2004. Of the 44 studies we reviewed, 22 were published in 2018 and the following years. It has been forecasted that the usage of educational AI applications will increase (Qin et al., 2020; Zawacki-Richter et al., 2019). Such increase is implied in our finding that the publication of studies on AI-based teaching increased after 2017. Figure 2 presents the research trend on AI and teachers.

Figure 2 further indicates that research on teachers’ AI use in education intensified in the last four years. This implies that AI-based instruction by teachers is most likely to become more common in the near future. Supporting this, our review of literature on the topics “AI” and “education” showed that studies published between 2015 and 2019 accounted for 70% of all the studies from Web of Science and Google Scholar since 2010 (Chen et al., 2020). The availability of AI technologies and of educational software companies to create AI-based applications is increasing rapidly all over the world (Renz & Hilbig, 2020). Accordingly, it seems likely that teachers’ use of AI in the teaching process will grow and more studies will be conducted on this topic.

On the other hand, there are still fewer studies on AI use in education than in other areas such as medicine and business (Borges et al., 2020; Luckin & Cukurova, 2019). The educational technology (EdTech) market is growing much more slowly than other markets with respect to the dynamics of digital transformation. One of the reasons for this is the resistance of decision-makers such as educators, teachers, and traditional textbook publishers to the use of AI (EdTechXGlobal Report, 2016). Considering this resistance, it can be argued that more AI research is needed to show the pedagogical uses of AI in instructional processes and to speed up the uptake of AI technologies in education.

Self-reported data were the most common data collected from teachers in the AI-based education studies. The researchers collected self-reported data to predict teacher-related variables such as engagement, performance, and teaching quality. In these studies, machine learning algorithms were used instead of conventional regression analysis to reveal nonlinear relationships between variables of teaching practice. For instance, Wang et al. (2020) collected data from 165 early childhood teachers to better understand indicators of quality teacher–child interaction. Similarly, in Yoo and Rho (2020), teachers’ self-reported job satisfaction was predicted by a machine learning technique. In some AI studies, teacher grades of student assignments or essays were used to train AI algorithms. For example, Yuan et al. (2020), in developing an automated scoring approach, needed expert teachers’ grades to validate their AI-based scoring system. A notable finding from our review is that self-reported grades accounted for nearly 44% of all data obtained from teachers (Fig. 3).

In 11 of the studies that we reviewed, teachers provided more than one type of data. The data were mostly collected during or after teachers’ instruction. Our review findings highlight the crucial role of teachers in the instructional process (e.g., Huang et al., 2010; Lu, 2019; McCarthy et al., 2016; Pelham et al., 2020). For example, Schwarz et al. (2018) presented an online learning environment that uses machine learning to inform teachers about learners’ critical moments in collaborative learning by sending the teachers warnings. In their study, they observed how the teacher guided several groups at different times in a mathematics classroom. In addition to observations, they collected interview data from the teachers about the effectiveness of the online environment. Our review indicates that there is a significant gap in physiological data collection in AI studies with teachers. Only one of the studies we reviewed collected physiological data, that is, data on eye tracking and audiovisual/accelerometry data from sensors worn by the teachers (Prieto et al., 2018). In fact, physiological data can be considered relevant and useful for providing process-oriented, objective metrics regarding the critical moments that impact the quality of teaching or learning in an educational activity (Järvelä et al., 2021).

Our findings from our open-coding analysis indicate that teachers have seven roles in AI research. These roles and their descriptions are shown in Table 2. As seen from the table, teachers participated in AI research as models to train AI algorithms. This role was found to be the most common role of teachers in AI-based instruction (f = 18). This finding underlines the pivotal role of teachers in the development of AI-based education systems. For instance, Kelly et al. (2018) conducted a study to train AI algorithms to automatically detect teachers’ authentic questions in real-life classrooms. During the training of the AI algorithms, the teachers’ effective authentic questions were fed to the AI system as features. Following the AI training, the researchers tested AI in a different classroom and found that AI successfully identified authentic questions.

Another role that teachers were observed to have in AI research was providing big data to AI systems to enable them to forecast teachers’ professional development. In this line of research, teachers mostly provided data to AI systems for the latter’s prediction of different variables of the professional development of teachers such as their job satisfaction, performance, and engagement. For example, in one study, 10,642 teachers answered a survey (Buddhtha et al., 2019). Then, using AI, predictors of teacher engagement were determined. Similar to other areas, big data have played an important role in education, and teachers are considered among the most important sources of big data (Ruiz-Palmero et al., 2020). Our findings imply that AI can effectively inform teachers of their professional development.

This study also found that teachers involved in AI research provided input information on students’ characteristics for the AI-based implementation. For example, Nikiforos et al. (2020) investigated automatic detection of learners’ aggressive behavior in a virtual learning community. The AI system utilized teacher observations of students’ behavioral characteristics to predict the students who were more likely to bully others in the online community. Our review further revealed that teachers have taken on the role of grading assignments and essays to test the accuracy of AI algorithms in grading student performance. In such studies, the accuracy rate of the AI-based assessment was determined with the help of experienced teacher assessments (Bonneton-Botté et al., 2020; Gaudioso et al., 2012; McCarthy et al., 2016; Yuan et al., 2020).

In some AI-based education studies, teachers determined the criteria for some components of AI-based systems and assessments. For example, Huang et al. (2010) investigated the effect of the learning assistance tool ICT Literacy. The tool used machine learning. In their study, experienced teachers guided the AI system by defining the criteria for effective and timely feedback. In some studies, teachers also provided pedagogical guidance on the selection of materials for AI-based implementation. For example, Fitzgerald et al. (2015) utilized AI to present learning content with varying degrees of text complexity to early-grade students. They attempted to explore early-grade text complexity features. Text complexity in the AI system was determined based on teachers’ pedagogical guidance. Furthermore, teachers commented on the usability and design of AI-based technologies (Burstein et al., 2004). Finally, our results revealed a notable absence of pre-service teachers as participants in AI use studies. That is, there were no studies in which pre-service teachers actively participated or interacted with AI technologies.

We found several advantages of AI from our review of selected empirical studies on teachers’ AI use. The open coding revealed three categories of AI advantages: planning, implementation, and assessment (see Table 3).

The challenges in teachers’ use of AI are summarized in Table 3. One of the most observed challenges is the limited technical capacity of AI. For example, AI may not be efficient for scoring graphics or figures and text. Fitzgerald et al. (2015) reported that an AI-based system failed to assess the complexity of texts when they included images. The limited reliability of the AI algorithm was found to be another considerable challenge. Therefore, automated writing evaluation technologies that use AI algorithms have to be improved to provide trustworthy evaluations for teachers (Qian et al., 2020). Inefficiency of AI systems in assessment and evaluation is related more to validity than to reliability. AI-based scoring may sometimes improperly evaluate performance (Lu, 2019). Our review further indicated that AI systems may be too context-dependent such that using them in varying educational settings can be challenging. For example, an AI algorithm designed to detect specific behavior in a specific online learning environment cannot work in different languages (Nikiforos et al., 2020). In other words, this limitation can stem from cultural differences.

The lack of technological knowledge of teachers (Chiu & Chai, 2020) and the lack of technical infrastructure in schools (McCarthy et al., 2016) are two other challenges in integrating AI into education. It has also been reported that AI-based feedback is sometimes slow. This can lead to teacher boredom in using AI (McCarthy et al., 2016). Although adaptive and personalized feedback is important for teachers to reduce their workload, AI systems are not always capable of giving different kinds of feedback based on students’ needs (Burstein et al., 2004). Therefore, AI systems currently fall short of meeting the needs of teachers for effective feedback (Fig. 5).

We coded AI methods in the studies, following previous reviews (Borges et al., 2020; Contreras & Vehi, 2018; Saa et al., 2019). Artificial neural networks (ANN) appeared to have been the most used (f = 16) AI method in the education studies involving teachers. ANN is a machine learning method that is widely used in business, economics, engineering, and higher education (Musso et al., 2013). According to our review, ANN also processes common data sourced from teachers. For example, Alzahrani and his colleagues (Alzahrani et al., 2020) investigated the relationship between thermal comfort and teacher performance. Through ANN analysis, they analyzed the data related to teachers’ productivity and the classroom temperature. Decision trees, another machine learning algorithm, were frequently utilized in our reviewed studies. For instance, Gaudioso et al. (2012) used decision tree algorithms on data to support teachers in detecting moments in which students were having problems in an adaptive educational system. Similar to our findings, a review of predictive machine learning methods for university students’ academic performance found that the decision tree algorithm was the most commonly used (Saa et al., 2019).

In our review, we also investigated the subject domains of teachers’ AI-based instruction. The studies with teachers from various domains accounted for 16% of all research (see Fig. 6). These studies generally had a larger sample size than the studies with teachers from a single domain (e.g., Buddhtha et al., 2019). Primary education and the English language appeared to be the domains where teachers use AI the most. Studies on automated essay scoring and adaptive feedback were conducted in English language courses. We found that 46% of all the studies we reviewed were performed in fields related to science, technology, engineering, and mathematics (STEM), and a much smaller percentage of studies were performed in the social science and early childhood fields together. These might have been because teachers in STEM fields are more accustomed to technology use (Chai et al., 2020).