AI in the Genetics Classroom: A Useful Tool but not a Replacement for Creative Writing

This study was conducted at a large research-forward land grant university on the East Coast. For this study we recruited students in the BSCI222 Principles of Genetics class as subjects. Principles of Genetics is an entry level genetics class required for progression in the biology major and is taken primarily by sophomores, juniors, and seniors from biology and biology-adjacent majors. Only a small number of freshmen were enrolled, since in order to take the class, students must complete two biology and two chemistry courses as pre-requisites. This class has previously had a creative writing assignment designed to prompt students to engage with the ethics surrounding genetics outside of traditional lectures. This 4-credit class generally has 150–200 students enrolled. The class meets twice a week for 75 min lectures as well as in multiple sections once a week for a 120 min recitation/discussion section (in groups of 20–25 students) led by a teaching assistant (TA). For the semester in which this study took place, initial enrollment in the class was 194 students. Data from the registrar on the class at the beginning of the semester showed that it was predominantly female (78%, 151 students), and the most common ethnicities were white or Caucasian ethnicity (43%, 83 students), Asian (27%, 53 students), and Black or African American (16%, 31 students) (Table 1).

The students were asked to participate in a pre-survey at the beginning of the semester after the drop/add period closed, to avoid recruiting students who would not remain in the class. The students were then asked to generate a first draft of their creative writing assignment using ChatGPT (or an AI like it, which will be inferred for the rest of this study). Once they generated their first draft, they were required to complete a self-reflection worksheet (Supplemental Material Section IV) designed to encourage them to think critically about the output from the chat-bot. This reflection worksheet was designed to have two purposes. The first was to guide student interactions with the AI by directing them to provide it multiple prompts, and to adjust these prompts to get different results. The second was to have students interrogate the AI output in terms of factual accuracy, quality of writing, ability to generate legitimate sources, and evidence of biases in the AI. Students then exchanged their writing during recitation and provided feedback to their peers using a guided worksheet (Supplemental Material Section V). This worksheet was similar to the self-reflection and focused on further assessing the quality of the output. These two worksheets served the purpose of guiding student use of AI as well as directing them to focus on areas where AI might fall short. During this time period, students were encouraged but not required to complete an informative online module on AI developed by the university’s teaching center and library. This module informed students on AI ethics issues, how to cite AI, and potential uses of AI tools.

After the writing was critiqued both by themselves and a peer, the students were then asked to re-write their writing assignment without using ChatGPT based on the feedback they received. This was then considered the final draft and graded by the course TAs. After the writing assignment was turned in, the students took the post-survey. Both the pre- and post-surveys were administered online using Qualtrics during their recitation section to encourage participation and to ensure that the data remained confidential. The study concluded with an in-class discussion with the students on the assignment and AI. The timeline and progression of the study are summarized in Fig. 1.

Full size image

The data collected for this study was in the form of two surveys that included 12 or 13 questions each (Supplemental Material Sections I and II) and were administered before and after the creative writing assignment. Face validity of the novel survey was established using a science education faculty member (2nd author), a graduate research assistant (first author), the senior graduate teaching assistant, an education development specialist, and the instructor of the course (senior author). Survey questions were negotiated among these parties until one hundred percent agreement was reached. The pre-survey started with a five-sentence explanation of AI to ensure that students had at least some minimal background on the topic before proceeding with the survey. The surveys utilized both Likert-style and open-ended questions. Some questions overlapped between the two surveys to allow change in perception and knowledge to be assessed. For most questions students were prompted to provide further detail as to why they selected the Likert-style response. Other questions did not overlap and were designed to understand how the students interact with the course assignment as well as prompt further reflection. We compared change in perception in the class to see if education and hands on experience had an effect on students. In addition to the student surveys, we also had the opportunity to formally interview the graduate-level TAs for this class to assess student perceptions.

The pre-survey was completed by 191 students (98% of the students in class at the time of delivery) and the post-survey was completed by 173 students (93% of the students in class at the time of delivery, n = 186 students). This represented the majority of the class, and the distribution of students across majors for pre and post survey participants can be found in Table 2. As most students took both surveys, this is a representative sampling of the class demographics. The class is almost evenly split between biology majors that are broken into specific concentration areas, and public health science majors. The majority of the participants in both surveys were juniors (Table 3). Additionally, there are a small number of freshmen also enrolled in the class. The post-survey had slightly less participation that the pre-survey. The decrease in participation likely can be accounted for by students dropping the class, and student fatigue as this survey took place later in the semester.

The TAs were interviewed at the end of the course to gain further insights into how well the assignments worked. As some TAs have taught the course with this instructor in previous years, they were able to speak to quality of writing assignments previously in comparison with the study’s version of the assignment. TAs graded the draft and final versions, and thus they have the best understanding of how using the AI on the first draft impacted final versions. Additionally, they had the most communication with students around the assignment and can illuminate any other challenges that arose.

The first three questions were used to collect demographic information on students, such as student year and major, as well as an identifier that could be used to connect an individual’s pre- and post-survey. Mean and standard deviation were calculated for the Likert-style questions. To calculate a mean, choices were converted to a numerical scale ranging of 1–3, 1–4, or 1–5, where the lowest number is the strongest disagreement, and the highest number is strongest agreement. Yes/No questions were measured by calculating percentages of the total number of responses.

To determine change in student perception, several questions were included in both the pre- and post-survey. Responses for students who participated in both surveys were matched using the identifier, where any duplication of the identifier was resolved either with student major and/or student year data, or else it was removed from this part of the study. Once the dataset was cleaned and responses matched, Wilcoxon-Signed Rank tests assuming paired data were calculated for each question to determine if the distributions were significantly different. A Kruskal–Wallis test was used to compare the Likert-style results by student year and by student major where majors were grouped into Public Health Science or Non-Public Health Science as this is nearly an even split of the class. Year and major were considered the predictors of the response in this test.

The open-ended responses were utilized to illustrate the themes in the qualitative results. One author developed the codes for each question based on a subset of the responses. A second author then reviewed the codes and responses. The codes were additionally reviewed by a graduate assistant who was not involved in the study. Each set of codes was unique to the question, though for questions used in both surveys the same set of codes was used. These codes were developed to best represent the students’ responses. With this coding framework, it was possible for a response to be coded as falling under multiple codes. An example of this can be seen in Table 4. The total number of instances of each code were then summed for each question and percentage out of total responses to the question was assessed. From these responses the authors together also identified quotes that represent the data. Quotes have been lightly edited for conciseness and clarity; verbatim quotations are available upon request.

Interviewing the TAs served the purpose of illuminating trends or shortcomings that may not have been apparent from student-reported data. The TAs interviewed were all experienced graduate TAs who have taught more than one semester. For the purposes of this study, the undergraduate course TAs were not interviewed as they were first-time TAs. All three graduate TAs were asked the same set of questions, with the exception of the TA who has taught the course previously. This TA was asked an additional question based on their previous experiences. The answers to these questions were transcribed and compared between all three TAs. As with the open-ended responses, quotes from these interviews were lightly edited for clarity.

The first two questions in the pre-survey asked students to report about their previous experience with AI in a classroom setting. In response to the first question, 69 students (36%) reported that they had learned about AI in one of their classes. To distinguish if there was an effect by student year, this was broken out further by percentage of each group that responded “yes” to learning about AI in one of their classes (one freshman student [33% of freshmen], 21 sophomores [43%], 37 juniors [38%], 10 seniors [24%]). This does not reflect the normal expectation, where seniors should have had the most opportunities for exposure, likely because AI has only been relevant in the class setting for approximately a year. Thus, this has not left much time for differences between student years to develop. In response to the second question, only 23 students (12%) reported that they had been prompted to use it by an instructor in the classroom.

When asked specifically about their use of ChatGPT (or AI like it) and Grammarly prior to this class, most students (118 students, 61.8%) reported that they had never used ChatGPT for an academic assignment (Fig. 2). A small subset (21 students, 11%) reported that they had only used ChatGPT once, and another subset (51 students, 26.7%) reported using it sometimes. Only one student reported that they used ChatGPT for almost every written assignment. The distribution of Grammarly use was slightly different. Students either reported never using it (43 students, 23%) or used it fairly frequently (reported as sometimes [77 students, 40%] or for almost every assignment [58 students, 30%]) to assist with their writing. This suggests a precedence for students using tools to improve their writing. Grammarly use has been generally accepted as meeting academic integrity policies, and for the authors’ institution is included in the university software catalog. However, Grammarly now has its own AI functionality and additionally has a feature called GrammarlyGO (Grammarly, 2023) that is classified as a generative AI.

Full size image

Students were also asked in the pre-survey to report the ethical issues related to AI that they were aware of. The primary issue reported was academic integrity (71 students, 47.3%), which is unsurprising considering that this is also one of their main concerns. They also reported being concerned about copyright issues (36 students, 24%), biases or errors in AI training datasets (29 students, 19.3%), privacy and data use issues (18 students, 12%), the impact of AI on jobs (13 students, 8.7%), an impact on learning (9 students, 6%), and impacts on humans at a social level (19 students, 12.7%). A student expressed more in-depth knowledge of a social impact of AI, stating, “It’s powered by underpaid workers in 3rd world countries.” This appears to be a lesser-known ethical concern of AI, alongside the environmental impact. A thoughtful exploration of this issue can be found in the “Labor” chapter of Kate Crawford’s book Atlas of AI (Crawford, 2021).

At the beginning of this study, students reported they had little knowledge of AI (mean = 1.5 out of 3, SD = 0.55, Table 5). This was not statistically different between major or year, likely because AI is still very new. As AI becomes more prevalent, it is likely that students will have a greater baseline knowledge of AI, however the fact that AI technology keeps changing means that this may not be consistent. Many instructors fear that students are using AI frequently, which is not supported by these reported data. As this class consists almost solely of STEM students, it is recommended to conduct similar studies on other majors in order to address the prevalence of AI use in the student body as a whole. However, perhaps this will provide insight to instructors on the likelihood of student AI use in their own classrooms.

In the pre-survey, students felt fairly positively about the use of AI in the classroom (mean = 3.78 out of 5, SD = 0.96). This persisted in the post-survey, where student perception indicated a significant increase in favorability about AI as an assistant to learning (mean = 4.19 out of 5, SD = 0.85, p-value = 0.03). In regard to using AI as a learning tool, students stated, “It provides valuable explanations and help understanding material.” Another student provided multiple ways that AI could be helpful to them, “ChatGPT could help a student develop an outline for a writing assignment, stay organized, find basic background information, or could answer simple calculations.”

When asked to describe how they felt about use of AI in the classroom in the post-survey, a subset of students felt that AI should be used in the classroom, but only if used “right” (45 students, 31% of short-answer responses), and 38% (52 students) felt as if AI was helpful to them as a learning or writing tool. A prevailing theme is that students were positive about the use of AI but required that guidelines or restrictions be placed on its use. The vocabulary they used to describe this utilized the following words: right, proper, appropriate, and moderation. This terminology was frequently used, however not defined by any of the students. Students have a sense of what they want but were not able to explicitly define how it might work in practice. Some students (11, 7%) felt that use of AI in the classroom should be class or assignment dependent, indicating that there are instances where it is appropriate to use, and others where it is not. One student summarized several of these issues.

I think there is no harm to using chat GPT in the classroom as long as everyone knows how to use it appropriately and fact check. I also think it depends on the class. In a writing/discussion/opinion-based class it could definitely help but in a math or physics class that’s mostly factual it may be less helpful, but I am sure there are ways it could be used that I’m not considering.

Other students were still not convinced, “I don’t think it would be useful to use in class.”

The responses to this survey show that students are aware of the barriers to education that some of their peers face.

ChatGPT can provide students with instant access to information and feedback on various topics and subjects. Students can ask questions, request explanations, or seek guidance from ChatGPT at any time and place, without relying on teachers, tutors, or peers. This can be especially helpful for students who have limited resources, face barriers to education, or need extra support.

The vast array of responses to this question show that students are thinking deeply about how AI affects them and their learning.

Overall, students felt as if this assignment helped them become more informed about AI, and the ethical issues surrounding it (mean = 4.14 out of 5, SD = 0.8, Table 5). Many students had not used AI previous to this assignment, and thus had a more dramatic change in knowledge than students who were more informed at the start of this study. This was reflected in their short responses.

I never used ChatGPT before this assignment and never really had an idea how this sort of AI worked, this was a great opportunity to learn about this tool, and though I don’t completely trust it, I know I will be using it in the future.

Data collected from the pre-survey showed that slightly more than half (53%) of the students (102 out of 191) reported that they were concerned about AI’s impact on them as students. Students were again asked in the post-survey if they had concerns about the impact of AI on them as students, and 72% (124) responded that they were concerned, which is an increase from the pre-survey by 19%. When they were prompted to explain what those concerns were, 98 students responded to the open-ended question in the pre-survey and 121 responded to the post survey. The following four major themes emerged and were consistent for both the pre- and post-survey.

In the pre-survey, 27 (27%) students raised concerns about the accuracy of AI, demonstrating that they had some knowledge of the weakness of LLMs. Responses included the following statement: “Asking it [the AI tool] specific questions may not have accurate answers,” “It makes stuff up,” and “The information on there could be false and misinformed.” In the post survey, 63 students (52%) expressed concerns about the accuracy of what AI produces. There was an increased concern specifically in regard to the citations produced by the AI, which is something they learned from the self-reflection worksheet, “Some concerns that came up while completing the assignment were the fake citations.” This increased concern from the pre-survey suggests that completing the assignment increased students’ awareness of possible inaccuracies in the AI outputs, as well as the challenges ChatGPT faces in generating sources.

Twenty-six students (27%) expressed concern in the pre-survey that AI would be detrimental to their learning. Several students referred to writing and reading skills. As one wrote:

“… I also think reading comprehension and writing skills are important to practice, even if annoying or hard in the moment, because they give you the tools to be well-informed and less persuaded by misinformation. Using ChatGPT could make us worse at those necessary skills.”

Students also raised concerns that using ChatGPT “minimizes creativity” and makes you “rely on [the AI tool] completely.” Others were worried that students will become too dependent on AI and would not “struggle on their own to come up with new ideas” or make them “lazy in problem solving.” In the post-survey, 23 students (19%) expressed this same concern. “…ChatGPT could make gaps in the knowledge that I might need in my career.”

A related concern is that students felt strongly that their peers would cheat, and that this was unfair.

“I am concerned that my fellow students are using/would use it to be academically dishonest, which is concerning because I am always academically honest and want to be graded fairly against other people’s work, not an AI.”

In the pre-survey, 20 students (20%) expressed concerns about their peers cheating, and 24 (20%) had the same concerns in the post-survey. As a whole, students are against their peers receiving better grades for work they did not truly do.

Twenty students (20%) were concerned in the pre-survey that using AI would result in being accused of plagiarism, cheating, or some other aspect of academic dishonesty that would involve censure from the university. One student reported, “I am worried about plagiarism and being flagged within my classes,” while another indicated that they had already been accused of using an AI in their academic work.

“I have been reported for using an AI on exam before, when I did not use it. The only reason I was reported was because other students in the class were using AI. This caused the entire class to have to go through the Office of Student Conduct, which is a very stressful situation to be in when you did not use the AI or go against academic integrity.”

In the post-survey, 15 students (12%) expressed persisting concerns about academic honesty, and potential accusations.

The concerns students voiced in response to this question align with those expressed by instructors. Students are aware of the potential impact on their learning, which may prove helpful to instructors and facilitate cooperation.

We targeted the change in student perceptions in two different ways. First, we prompted students to self-report how they felt their opinions changed over the course of the study, and second, we connected pre- and post-survey responses of individual students. For self-reporting, students were asked if they felt that this assignment resulted in a change in their opinion on AI. The mean response was 3.3 out of 5 (Table 5); however, in the short answer, 54% (78 students) stated that they felt their opinion had not changed. “No, I still think that I’ll stay pretty clear of any AI or ChatGPT. The cons outweigh the benefits to me.” Interestingly, 30% (43 students) reported that this assignment gave them a more positive opinion on AI. For example, “Previously, I thought AI was just used to cheat but now I can see how it can be helpful!” An additional 10% (12 students) reported either feeling more negative about AI or being more critical of its outputs.

A total of 143 students responded to both the pre- and post-surveys. Six questions were shared between the two surveys to assess change in student perception. Five of these questions showed statistically significant change in the distribution of student response between the two surveys when assessed with the Wilcoxon Signed-Rank Test (Table 6). In the post-survey, students appeared more inclined to think that AI could help them academically than they did in the pre-survey. Additionally, they became somewhat more favorable to AI being used in a classroom setting. This was accompanied by short-answer feedback as to how this might be done, with a focus on a desire to be taught how to use AI in a way that is appropriate for a given class. For example, one student wrote, “Yes, instructors should teach students about the implications and proper use of ChatGPT and other AI learning platforms.” They also expressed a desire for clearer instructions and boundaries. After the assignment, students reported less inclination to use AI to write an entire assignment; however, their willingness to use AI as a writing tool/assistant did not change significantly. There was no significant difference between predictors such as major or year for any of the questions.

Finally, students reported overall more knowledge of AI and its ethical issues; however, this mean was still relatively low (1.79 out of 3, Table 5). Thus, while the purpose of this assignment was to give students an introductory level of knowledge on AI, students still felt that there was more they could learn. This is compounded by the fact that AI technologies are changing at a rapid rate, and thus underwent advancements even over the course of this study. In order to stay abreast of the advancements, both students and instructors must invest time into keeping up to date. Unfortunately, this means that some assignment types may not maintain relevance the same way they have historically and could require year-to-year or semester-to-semester updates.

As reported in the post-survey results (Table 5), more than half of students found this assignment to be helpful on educating them on AI (89 students, 62%) as well as prompting them to think more deeply about ethical issues in genetics (88 students, 63%). They reported that integrating AI into the assignment forced them to be critical of the ChatGPT outputs and do their own extensive research to validate sources. One student commented, “ChatGPT provided some reasons, but during my edits I had to think of more. Thus, I thought of it on my own as well as understood the reasons ChatGPT provided.” A small subset of students (8) reported that because they used ChatGPT for this assignment, they did not do as much research on their own as they might have if they had done the assignment in a traditional manner. As mentioned by one of the students, “I didn’t do much of my own research on the issue because ChatGPT did the bulk of it for me so I didn’t get as much out of it as I probably could have.”

The assignment also helped students further investigate ethical and social implications of genetics. Fifty-seven percent of the students who answered to the open-ended question (80 of 140) reported that the assignment itself helped improve their understanding of these issues. Some students reported that the assignment prompted them to think more in depth “It allowed me the space to consider an ethical genetics debate more in depth,” and prompted them to be more engaged because the assignment was something interesting to them. “[The assignment] allowed us to get creative on any topic related to genetics and I believe that gave us free reign to investigate issues that are interesting to ourselves and kept us engaged.” This aligns with what has reported previously for assignments like this, where creative freedom encourages a higher investment from students (Nicholes, 2020; Ostrom et al., 2020). It would be interesting to further disentangle how much this creative aspect motivates investment in the topic itself in comparison to their investment in their own writing. It seems possible that creative writing may be achieving both of these things.

Students found that having to address an ethical dilemma from a story-like or creative approach changed their perspective. One student reported, “[The assignment] allowed me to think about both sides of a story.” Another student described something similar, “I had to think of both viewpoints and think about how each person thinks/views a discussion.”

Creative writing provides students the opportunity to interrogate a problem from multiple perspectives, which in turn can change their own perspective on an issue (Nicholes, 2020; Watkins & Tehrani, 2020). The ability to evaluate a problem from multiple perspectives is an important skill that students need, regardless of major or desired profession.

Students expressed a desire for clearer directions or a rubric for this assignment should it be conducted again. For this assignment, it was not defined how much of the final draft should be original writing versus writing generated by the AI. Students found this confusing and were unsure how much they were supposed to edit or re-write the AI-generated writing. It is challenging to set a threshold for this that can be easily verified, and for this assignment the researchers did not want to be too stringent and thus restrict student creativity. One student suggested having AI-generated text and human generated text in different fonts or colors which might be one way to accomplish this. In another iteration of this assignment, it would be beneficial to be more explicit about how different the re-written assignment is expected to be from the AI-generated version. Overall, this reflects how critical it is for instructors to define clear boundaries or guidelines for the use of AI in the classroom.

In addition to the student responses, the course TAs provided valuable insight into the effectiveness of the assignment. When asked if significant improvement occurred between the draft and the final paper, one TA primarily reported, “I saw mostly either students that really re-worked it or did very little work with it.” Another TA further elaborated on this.

I think there were a couple students that made an effort, I think there were students who I hypothesize are stronger writers and felt bothered by the style that was output and, in their feedback and comments and stuff, they indicated that. They were the most likely to go in and really pare down what it was saying because first they had a more advanced understanding of writing style and what makes good writing, and that more words aren’t always better.

This suggests an expected spectrum of writing ability across students, which impacted their willingness or ability to revise the AI-generated content.

The suggestions made by students (Table 7) for alternate versions or expansions on this assignment were useful and reflected that they understand how to challenge AI tools. These suggestions primarily broke out into three categories: Further comparison of AI draft vs. human draft, further challenge or explore the AI’s responses, or change the type of paper written. These three categories represent areas where instructors can develop AI-based assignments. Adjustments to this assignment could be class-dependent and reflect the overall goals of that class. For example, a STEM-based course might want to emphasize problem-solving skills, whereas an English course might want to focus on how to hone writing or reading comprehension skills. Framing exercises in AI around course goals will result in assignments that are beneficial and relevant to students. This study shows that students can be a valuable resource for creating these assignments, as they have well-thought-out suggestions for how to utilize AI in the classroom in a meaningful way. Thus, the onus of coming up with these assignments is not solely on the instructors. Especially while AI is still new, expanding discussions to include students can improve AI-based assignments.

The TAs over all reported that they had little difficulty implementing the assignment. The biggest challenges they faced were students failing to understand that the assignment was meant to be creative, or students who did not have a good grasp on what an ethical problem was. In their recitations or via email, TAs were generally able to address these issues. TAs reported that students found the most challenging part of the assignment to be the peer review. Two of three TAs reflected that students were not as engaged in this task. “Some students were kind of sluggish with the peer review. I think some of them just weren’t as interested in the peer review questions, so they were like “yes” or “no” [rather than elaborating more],” while the third TA stated, “I have never seen, in the entire semester, such concentration and hard work from every single student in the room.” A suggestion for improving the peer review was as follows,

I think there could have been fewer peer review questions, but then if they also had to provide a statement responding to the peer review as well. Like ‘here’s what I felt the strengths and weaknesses were from my initial draft so here’s what I changed in the revised copy.’

Alongside this suggestion for improvement, another TA recommended to reduce the number of evaluation tasks (the self-evaluation or the peer review) as students disliked busy work. This represents a primary challenge of this sort of assignment. The evaluation portions of the assignment were designed to guide students’ thinking; however, this may not be something they find to be interesting or engaging. In the future, these worksheets could be streamlined to be less arduous for students while still accomplishing the same goals. Overall, the TAs were positive about the assignment.

I think [the AI portion of the assignment] was fantastic, I think they understood how to use it, they whined and complained about [the informative module] before they did it, but when they did it, they were proud of their learning. They feel in the know about this thing that their peers are talking about.

While both students and TAs encountered some challenges, it seems the assignment accomplished the goals outlined at the beginning of the study.

This study is limited by being conducted only once in this course. Ideally, more than 1 year of data would be collected; however, with the urgency of adapting to AI in the classroom, we felt this data was most useful to other educators sooner rather than later. The other limitation of this study is that it is based on student reported data collected through voluntary surveys, with some additional data contributed from interviews of the course TAs. This could be further expanded on by interviewing students as well, though this is more challenging to recruit for. However, in the pre-survey, 99% of students enrolled in the class responded, and this did not drop off dramatically in the post-survey considering approximately 10 students dropped the class after the initial drop/add period.