The Educational Affordances and Challenges of ChatGPT: State of the Field

From the 210 studies, 20 duplicates were removed. This reduced the number of studies to 190. Those remaining articles were then examined against the inclusion/exclusion criteria in Table 1.

Two researchers independently reviewed each article against the inclusion and exclusion criteria to determine inter-rater reliability. Calculated using percentage agreement (Belur et al., 2021), researchers arrived at a 97% inter-rater agreement. In examining and discussing the misaligned articles, a 100% agreement was achieved. The inclusion and exclusion criteria resulted in 166 studies being removed, leaving 44 articles for final inclusion in the systematic review. Figure 1 provides a chart of the numbers following the PRISMA principles.

Three axial codes emerged from the coding of how ChatGPT can support teaching. These include 1) content creation/lesson planning, 2) personalizing student learning experiences, and 3) assessing student learning. ChatGPT can assist teachers in the creation of content, lesson plans, and learning activities (Lodge et al., 2023; Trust et al., 2023). From a holistic perspective, ChatGPT could be used to create an entire course syllabus (Kasneci et al., 2023). At a more granular level, it can draw on its vast knowledge of educational fields and aid in the generation of specific lesson plans or specific learning activities (Bonner et al., 2023). Topsakal and Topsakal (2022) suggest using ChatGPT to generate dialogues that students can use in learning a foreign language Cooper (2023) involves generating ideas for science units using the 5Es model. Across disciplines Cotton et al., (2023) posited the use of the tool to create interactive games and use chatbots or virtual assistants that can challenge students to solve problems or answer questions through natural language interaction.

One of the challenges all teachers have is personalizing student learning and adapting their instruction to meet the varying needs of their students. The data show that ChatGPT can assist in helping teachers to provide personalized learning experiences for their students in a variety of ways. ChatGPT can level student texts topically, lexically, and syntactically (Bonner et al., 2023). By providing ChatGPT with specific instructions about the desired grade level, length of the passage, and a specific topic, teachers can create passages that are well-suited to the needs of the learners. ChatGPT has the potential to provide learning tasks at different levels of complexity (Farrokhnia et al., 2023). Kasneci et al. (2023) suggest that teachers can use ChatGPT to analyze student’s writing and responses and provide tailored feedback and suggest materials that align with the student’s specific learning needs. They also suggest that teachers can generate questions and prompts that encourage participation at different knowledge and ability levels and elicit critical thinking and problem-solving.

Trust et al. (2023) state that ChatGPT can be used by teachers as an individual tutor for a student in any subject, providing research support, directions, and explanations about complex topics at a more accessible level. ChatGPT can serve as a teachable agent that encourages students to learn by teaching ChatGPT a concept through a text-based conversation. Finally, teachers can personalize learning by having ChatGPT provide recommendations for resources, such as books tailored to the student’s needs and interests (Cotton et al., 2023).

Assessment of student learning plays an important role in teaching. A well-developed assessment can help teachers identify students’ academic understandings and issues, and then provide appropriate assistance or guidance (William et al., 2004). It appears from the data that ChatGPT can assist in developing a variety of methods of assessing student learning. Some of these methods include generating targeted and personalized practice problems and quizzes, which can help to ensure that students are mastering the material (Kasneci et al., 2023; Qadir, 2022; Mizumoto & Eguchi, 2023). Other methods include creating rubrics that students can use to self-assess their learning (Cooper, 2023) and creating different forms of tests such as multiple choice, and constructed response (Zhai, 2022).

The ability of ChatGPT to automate numerous tasks holds great potential to considerably decrease teachers’ busy workload (Farrokhnia et al., 2023). The coding of the articles revealed three specific axial codes that involve task automation: 1) communicating, 2) summarizing documents, and 3) writing support. As teachers spend significant time communicating with a variety of constituencies, ChatGPT can assist in automating these tasks. Trust et al. (2023) suggest using ChatGPT to draft emails, write text messages in different languages to send to students’ family members, or provide text for a classroom newsletter or website to keep students and family members up to date.

Halaweh (2023) states that ChatGPT serves as an effective tool to aggregate and summarize information. This can be beneficial to teachers in saving time and effort that may have otherwise been spent on searching through hundreds of web pages and databases, downloading files, and filtering them. Data also show ChatGPT supporting as a proofreader and editor. It can quickly and efficiently identify and rectify grammatical errors, incorrect spelling or punctuation, awkward sentence structures, and stilted syntax. In addition, it can assist with word choice, and suggest alternative phrasing (Sun & Hoelscher, 2023).

The final axial code for task automation is professional development. This describes the way ChatGPT can provide information on a wide range of topics, from general research to specific assignments to support educators gain new information (Yorio, 2023). This access to information can allow educators to stay current regarding new technologies, pedagogies, and content area-specific information. This immediate access to the most current resources can contribute to teaching effectiveness. Kasneci et al. (2023) state that these resources can be used to improve the clarity of teaching materials, locate information or resources that educators may need as they learn on the job, as well as used them for on-the-job training.

As ChatGPT and other GenAI systems continue to become incorporated into teachers’ practice, researchers need to continue to explore further uses of these systems. This exploration should expand to investigate use with specific student populations and with specific subject matter content, as uses will vary depending on the context in which ChatGPT is used.

The data show that as ChatGPT is available around the clock 24 h every day, students can access ChatGPT as a tool for learning at any moment in time. Sun and Hoelscher (2023) state that this unlimited access provides students a unique opportunity to receive immediate answers to questions and be promptly directed toward relevant resources. Farrokhnia et al. (2023) maintain that ChatGPT can offer students easy access to information through a variety of platforms (e.g., a website or a smartphone app) and in a variety of subjects. They posit that is a more efficient tool compared with traditional search engines as ChatGPT offers written answers rather than just a list of sources.

Accessibility for learning opportunities for students with disabilities is greatly enhanced by ChatGPT. ChatGPT can empower learners with visual impairments by providing speech-to-text or text-to-speech solutions (Kuzdeuov et al., 2023). Additionally, it can adapt writing, translate, and highlight important content in various formats (Trust et al., 2023).

ChatGPT can explain difficult or complex topics assisting students in their understanding of difficult content. ChatGPT can summarize text at the appropriate language level and provide a written summary of the content to help facilitate student understanding (Bonner et al., 2023). Sun and Hoelscher (2023) explain that the use of sophisticated algorithms for keyword extraction, sentence scoring, text compression, and advanced language comprehension capabilities allows the generation of succinct yet comprehensive summaries of lengthy articles, texts, or documents. This can be a useful tool for students who need to quickly navigate through extensive materials. They also state that ChatGPT can enhance students’ grasp of complex concepts by efficiently outlining rationales explaining those concepts.

ChatGPT has the capability to serve as a conversation partner with students. Extant studies show that this is a particularly effective tool for learning foreign languages. Designing conversations as a teaching activity is a very important, yet challenging, and time-consuming task. Topsakal and Topsakal (2022) state that ChatGPT is particularly valuable in producing high-quality, customized, and personalized conversations for students to use. Although the use of conversation partners is most evident in language learning, this can be used in a variety of different content areas. A study by Bayat et al. (2022) described how ChatGPT can provide students with valuable opportunities to improve their argumentation skills through low-stakes practices. Students can take one side of the debate and ask ChatGPT to take the other side, presenting their points and having the ChatGPT rebut them (Bozkurt et al., 2023). As ChatGPT functions by taking in a user’s input and using its language model to generate a response, it can serve as a conversational partner in any educational context.

ChatGPT operates on a one-to-one basis with the user. It appears that this can be used to personalize individual students’ learning experiences. These personalized learning experiences include providing feedback and generating questions and providing practice problems, explanations, and assessments that are tailored to the student’s level of knowledge so that they can learn at their own pace. (Trust et al. 2023). Others include creating appropriately leveled topical texts when provided with specific instructions about the desired level, length of the passage, and a specific topic (Bonner et al., 2023). ChatGPT can also recommend books and other materials based on a student’s reading level, interests, and reading history. Finally, ChatGPT can provide language assistance for non-native speakers (Yorio, 2023). This allows students to focus on the learning content and not have to navigate translating the content beforehand.

As the ability to communicate in writing is a skill that transcends all content areas, ChatGPT’s capacity to assist students in developing their writing skills is an important benefit. Across disciplines, Sun and Hoelscher (2023) posit that ChatGPT can efficiently and effectively identify and correct grammatical errors, awkward sentence structures, stilted syntax, and incorrect punctuation or spelling. For students needing help starting writing, ChatGPT can be used to brainstorm writing topics and content to help facilitate the writing process (Zhai, 2022). For those studying writing, Anders and Sahakyan (2023) posited that ChatGPT can help students with each of the seven steps of the writing process: (a) choosing a topic, (b) brainstorming, (c) outlining, (d) drafting, (e) soliciting feedback, (f) revising, and (g) proofreading. ChatGPT can assist with word choice, suggest alternative phrasing, and help maintain a consistent style and tone in the student’s writing, ensuring a preferred voice and presentation.

Students’ ability to assess their level of understanding regarding specific content is an effective learning strategy, and the data show that ChatGPT could assist in this process. Self-assessment strategies provided by Sun and Hoelscher (2023) include generating outlines, study guides, and study questions. They also suggest having ChatGPT create open-ended questions for critical thinking and diagnostic reasoning, as well as providing detailed responses with rationale for each question. Finally, they suggest that ChatGPT can produce multiple-choice questions with correct answers and an accompanying rationale. These features can be valuable tools for students to help reinforce their understanding of specific content and better prepare them for exams.

It was interesting to see student engagement emerge as a trend in the data, specifically how ChatGPT can be used to improve student engagement and participation. In Trust et al.’s (2023) paper on ChatGPT, they note that ChatGPT can facilitate group discussions and debates by providing a discussion structure, real-time feedback, and personalized guidance for students during discussions. They also suggested collaborative writing activities, where multiple students work together to write a document or a project. Yorio (2023) provides an additional example where ChatGPT can be used to create interactive learning experiences, such as virtual book clubs or author visits.

These types of learning opportunities and interactions can help students feel more connected to the material and can lead to a deeper understanding of the subject concepts (Nikolic et al., 2023). The conversational style of ChatGPT can make the learning experience more engaging and enjoyable for students, reducing the boredom and disinterest that can arise in traditional lecture-style teaching.

The grounded coding revealed a trend of ChatGPT and self-determination. This is a unique find as it has not appeared in prior systematic reviews that used a pre-determined set of criteria that the researchers were looking for. This is an important find as we understand the human technology interactions. The coding showed that while using ChatGPT it was supporting students’ basic psychological needs of autonomy, competence, and relatedness which lead to helping facilitate self-determination (Geary et al., 2023). Crawford et al. (2023) maintain that ChatGPT can provide students with a sense of control and psychological autonomy by providing them with more control over their learning. ChatGPT-3 can do so by placing autonomy in the hands of the students and supporting students by bolstering their feelings of competence by equipping them with the skills and resources needed to be successful learners.

What is new in this study is the emergence of two new areas as AI is supporting students with student engagement and self-determination. Both areas can play powerful roles in increasing student learning (Erdogdu, 2019; Tian & Park, 2022). Exploring how ChatGPT can facilitate student engagement and self-determination are important for researchers to explore further. Additionally, continued investigation into the how ChatGPT can be used as a tool to support student learning specific populations and subject matter. Different contexts will necessitate different uses and strategies for optimal success.

A significant limitation of ChatGPT is found in inaccuracies and hallucinations in the output provided. The term hallucinations is often used when ChatGPT generates output that provides many details that appears that ChatGPT understands what is being asked and can provide an answer; however, the information is wrong. From the studies, authors found both inaccuracies and hallucinations.

When Naidoo (2023) asked ChatGPT to generate a paragraph about research on employee recognition, with a reference list for the citations, it generated relevant text with citations that looked plausible but did not exist. Specifically, four out of the five references it generated were fake. Wang (2023) quizzed ChatGPT on a nonformulaic conceptual question in quantum mechanics. The answer given was wrong and ChatGPT cited a nonsensical reason for the answer. Dennings (2023) described attempts by people to request self-biographies and were provided with a description of a person that did not match who they were.

Another limitation from the data appears to show ChatGPT perpetuating biases. As a machine learning model, ChatGPT has been trained on a large body of data, and any biases present in that data will be reflected in the output created by the model. If the training data contains stereotypical gender roles or political, cultural, or racial biases, ChatGPT’s output will most likely reflect these biases (Nikolic et al., 2023). Therefore, it is important to ensure that the training data or the data used for updating the model is representative of different groups of people and diverse. Regular monitoring and testing of the model’s performance on different groups of people could help identify and address any biases early on. Human oversight in the process is indispensable and critical for the mitigation of bias of large language models in education (Kasneci et al., 2023). Nonetheless, humans perhaps need to remain cognizant of their own bias each time they attempt to ameliorate that bias as humans introduced bias in the first instance.

The review of the literature revealed four tool limitations regarding the use of ChatGPT. 1) database currency 2) word count, 3) lack of transparency, and 4) lack of higher-level thinking skills. The first limitation is database currency. The current iteration of ChatGPT has a knowledge base cutoff date of 2021, which limits its capability to provide, summarize, or synthesize accurate and current information. Sun and Hoelscher (2023) share that this is a particular concern for educational settings that rely on the most current data, such as, medical environments. The second limitation is word limits. ChatGPT has a word limit for the input it can process, and while this is often increasing, it limits current users from being able to input entire articles or book chapters. To navigate this issue, Sun and Hoelscher (2023) recommend break large text into smaller sections and inputting a section at a time. The third limitation is the perceived lack of transparency of how ChatGPT creates and provides output (Masters, 2023). When AI developers are asked to explain their algorithms, and they do not, it is not because they do not wish to; it is because they cannot. In this ‘black-box scenario, no-one knows to the full extend how the algorithm is functioning. Haggart (2023) cautions that unlike a book, which provides information about the publisher, the author, and the author’s sources that can be reviewed to determine its trustworthiness, ChatGPT is a set of algorithms that lack transparency and traceability.

A fourth limitation uncovered from the literature is that it appears ChatGPT is less competent when it comes to content that requires higher-order thinking skills, such as critical and analytical thinking. Farrokhnia et al. (2023) explain that this is the result of the high dependency of AI tools on the data that are trained without a deep understanding of context, emotions, and common sense, which are critical for higher-order thinking. This lack of the ability to process higher-order thinking skills results in ChatGPT being better at correlations than understanding causations. ChatGPT breaks words, sentences, paragraphs, and texts into data and looks for patterns of words and sentences that tend to appear together in certain situations. In Wang’s (2023) study of the use of ChatGPT in physics, he found that ChatGPT can interpret simple physics problems, assume relevant parameters, and write correct codes; however, it was wrong when answering conceptual questions. Haggart (2023) cautions that is possible that LLMs like ChatGPT could create a completely new form of knowledge, one in which correlations confer legitimacy, and in which the evaluation of the truthfulness of these correlations occurs behind the scenes, embedded in programming decisions and hidden labor.

The review of the research in this study revealed that the most often misuse cited regarding ChatGPT was plagiarism or cheating, consequently creating threats to academic integrity. In a poll of more than 4,000 Stanford students, 5% of the respondents reported submitting materials directly from ChatGPT for assignments without any additional editing (Cu & Hochman, 2023). Although cheating and academic integrity are not new considerations in education, it is important to note that ChatGPT has lowered the barriers to engaging in cheating behavior and the ability to detect cheating has become significantly more difficult, if not impossible (Lodge et al., 2023). Garg and Goel (2022) share the example of cheating occurring in general online assessment security and specifically cheating on online exams. Another opportunity for plagiarism is the ability to have ChatGPT create text and then submit the text as one’s own. In a survey featuring data from more than 1,000 students who were older than 18, 53% reported using ChatGPT to write an essay for class (Study.com, 2023).

While the challenge of easily accessible opportunities to cheat is available by using ChatGPT, researchers are not calling for restricting its use, but rather they are calling for the need to educate students and provide guidelines to ensure the proper use of ChatGPT (Cotton et al., 2023). Another option would be a call to educators to rethinking how students are assessed. Some options include assessment approaches that develop students’ skills in presentation and defense, evaluating information, correcting information, referencing work, and developing new creative ideas (Lodge et al., 2023).

Users’ privacy when accessing ChatGPT is a concern for possible misuse. OpenAI’s privacy policy for ChatGPT indicates that the company collects information from the user, including log data, usage data, cookies, device information, IP address, interactions with the site, and date and time of use. The policy states that this information may be shared with vendors and service providers, law enforcement, affiliates, and other users. While users can request to have their data deleted, OpenAI will not delete any prompts a user inputs. If a user inputs a sensitive prompt, such as giving personal details about a medical condition or mental health issue, OpenAI keeps a permanent record of that input (Trust et al., 2023). There is the potential that this information could be used to compromise users’ privacy.

Another misuse of ChatGPT is its ability to spread misinformation quickly and easily. When asked to produce false narratives for 100 controversial topics, ChatGPT obliged 80% of the time (Brewster et al., 2023). This presence of false information can create both confusion and exhaustion by saturating resources with material that overwhelms the truth or at least drowns out more balanced perspectives (Bell, 2023).

This study provides a robust look at the potential limitations and misuses of ChatGPT. The presence of concerns regarding the use of ChatGPT serves as a caution to both users and researchers. As with all technologies, humans need to be the primary determiners of practical and ethical applications. This is particularly true with ChatGPT, as its use is in the nascent stage and there is still much to be learned about its power and potential pitfalls.