Using Computational Grounded Theory to Analyze Pre-service Chemistry Teachers’ Reflective Practice Regarding Technology Integration in Classrooms Within a Service-Learning–Oriented Seminar

The TPACK (Technological Pedagogical Content Knowledge) model (Koehler & Mishra, 2009; Mishra & Koehler, 2006) was employed as a theoretical foundation to enhance our understanding of teachers’ professional knowledge, building upon the seminal work of Shulman (1986). It complements the body of literature by adding a specific technological dimension to more generic frameworks (Baumert & Kunter, 2013; Fischer et al., 2012).

TPACK conceptualizes teacher knowledge as comprising three primary dimensions integrated to address the demands of technology-enhanced teaching. Despite critiques regarding the empirical validity of TPACK’s structural specifics (Scherer et al., 2017), its general explanatory utility has been acknowledged for illuminating pre-service teachers’ professional knowledge growth (Stinken-Rösner et al., 2023). The core dimensions of TPACK and their intersections are depicted in Fig. 1.

Full size image

We highlight the critical role of pedagogical content knowledge (PCK), situated at the intersection of content knowledge (CK) and pedagogical knowledge (PK). PCK, regarded as transformative knowledge, merges the specifics of a subject, in this case, chemistry, with universal strategies for instructional practice. It is transformative because it means that teachers shape content and pedagogy to specific situations, and that PCK is not a static body of literature (Fischer et al., 2012). In the sense of the refined consensus model of PCK (RCM; Carlson et al., 2019), we differentiate between personal PCK (pPCK) and enacted PCK (ePCK). The former means theoretical knowledge acquired in pre-service teacher training, and the latter means the implementation of what a teacher knows in a specific teaching situation at a specific time and place. In short, having high PCK means successfully connecting subject matter, devising various representations, and customizing instructional materials to align with students’ misconceptions and pre-existing knowledge.

However, understanding the nuances of pre-service teachers’ subject-specific lesson planning cannot be solely attributed to the knowledge dimensions’ interplay. Simply enhancing pre-service teachers’ knowledge base is not sufficient for equipping them for their forthcoming roles. Consequently, teacher education programs must design and assess learning environments that promote collaboration among lecturers, in-service teachers, and pre-service teachers to meld knowledge with reflective practice.

Moreover, using technology in classrooms and schools effectively is no abstract knowledge but a materialized effort of actual people doing actual things in the sense of ePCK. Enabling students to use an app to capture measurements during an experiment is very different from providing learning materials via an online platform. We address this by choosing a broader analytical framework comprising five design spaces that are essential for merging professional knowledge with the comprehensive demands of classroom activities (Mishra & Warr, 2021; Warr et al., 2020):

Choosing this framework underscores that the complexity inherent in teaching and learning extends beyond mere knowledge transmission for the effective preparation of future-oriented chemistry teachers.

The SLA in teacher education seeks to connect theory and school practice by giving pre-service teachers the opportunity both to participate in an organized service activity outside the university (here: implementing technology-focused chemistry lessons) and to reflect on the experience within their coursework at university in order to gain a deeper understanding of the course content (here: combining professional content, technology, and pedagogical content knowledge in chemistry; Bringle et al., 2006). Applying an organized service activity in schools, meaning a direct service for pupils, parents, teachers, or other professionals in schools, leads to an enhanced sense of civic engagement (Chambers & Lavery, 2012). A technology-focused, direct service was chosen in the [blinded course], on which this study is based. While the SLA has been widely practiced, studied, and accepted in Anglo-American, Canadian, and Australian educational contexts, it has yet to establish itself fully in the German-speaking world (Reinders, 2016). In Austria, the higher education sector is said to be autonomous to develop without financial, political, or labor market restrictions (Resch & Fellner, 2022). Hence, knowledge production is oftentimes initiated without influences from practice fields (here: schools). Applying the SLA—for this reason—requires engaged scholarship and additional effort by educators. Educators who promote service-learning predominantly belong to the academic mid-level, according to a recent national study in Austria among educators (postdoc phase without habilitation; Fahrenwald et al., 2023). This is also true for the authors of this study. Their engaged scholarship means building trust relationships with schools, maintaining these relationships, and regularly cooperating in different forms of campus-community partnerships (Johnson Butterfield & Soska, 2004). The service-learning approach supports students in making practical experiences in their discipline (here: chemistry) and, in parallel, developing academic, technological, and civic competencies by engaging in practice while studying.

The course evaluated in our study is part of the elective Bachelor curriculum, offering 3 ECTS credits to pre-service chemistry teachers at the University of Vienna. Conducted by the Austrian Educational Competence Centre Chemistry (AECC Chemistry), this course is available every semester, running weekly for two hours. It was deliberately redesigned to align with the SLA, aiming to merge professional knowledge acquisition with practical preparation for secondary school chemistry teaching in Austria. To introduce the course participants to the TPACK model, they received a theoretical introduction and then categorized their initial ideas of technology implementations within the dimensions of the model. In-service teachers, collaborating through University of Vienna’s partnerships with approximately 58 schools under the school cooperation program of the Centre for Teacher Education, contributed to the course design by providing authentic classroom experiences to the participants.

To evaluate and compare the content knowledge (CK) and pedagogical content knowledge (PCK) of students (N = 6), we administered knowledge tests (Tepner et al., 2012) at the beginning and end of the course. Students were also asked to provide sociodemographic information, including their chosen subjects; details on participant demographics are available in the results section. Additionally, an adapted TPACK questionnaire (Schmidt et al., 2009) was used to assess students’ self-reported knowledge gains across the dimensions outlined by Mishra and Koehler (2006).

Students were further instructed to document their professional knowledge development through reflective diaries, incorporating insights from theory-focused lectures and their experiences in cooperating schools. These reflections aimed at elucidating students’ personal interpretations of how their knowledge gains could impact their future technology-integrated chemistry teaching. Each student was expected to make five diary entries ranging from 300 to 400 words. The final diary entry was part of a larger graded assignment. This assignment required students to demonstrate their lesson planning skills and provide a practical-theoretical analysis of their teaching experiences. The students knew that the diary entry, in particular, was not graded.

The (translated) prompt for their first diary entry was the following:

The follow-up prompts for the remaining four entries were:

Table 1 Shows a summary of our data sources.

Grounded Theory (Glaser & Strauss, 2017) is our starting point for reading and interpreting our students’ texts. This approach facilitates the identification of observational patterns, potentially leading to hypotheses or generalizations. For example, consistent high performance in summative assessments by students taught by a specific teacher may suggest a causal relationship with the teacher’s methods. However, human analysis of observational or textual data often lacks reliability in inter-personal and intra-personal contexts over time (Williams et al., 2013), limiting our confidence in such inferences without comparative analysis involving different teachers.

Grounded Theory addresses these limitations through methodologies such as applying diverse theoretical perspectives, iterative coding, and interpretation, along with inter-subjective discussions. These practices may enable a deep understanding of pre-service chemistry teachers’ learning growth when rigorously applied. Nonetheless, the thorough application of Grounded Theory is notably time-intensive, posing challenges in terms of resource allocation.

We employed Computational Grounded Theory (CGT; Nelson, 2020; Nelson et al., 2021) to enhance our analysis of participants’ reflections, utilizing various data sources. CGT provides a structured framework to augment traditional hermeneutic methods with computational efficiency, allowing for consistent and reproducible data analysis.

CGT involves three main steps: The first, pattern detection, leverages computational tools to identify patterns within natural language texts, setting the stage for a more focused manual review. In the refinement phase, researchers engage in deep hermeneutic reading and discussions, or qualitative content analysis (QCA), refining insights based on the patterns identified earlier. This phase may also include the integration of additional data sources to enrich the analysis. The final step, confirmation, connects the insights back to the original patterns and the overarching research question, ensuring alignment and relevance. In chemistry education,

All participants in the course (\({N}_{\text{female}}=3,{N}_{\text{male}}=3\)) were pre-service secondary science teachers specializing in chemistry alongside a second subject within their Bachelor’s program at the University of Vienna. The majority (four students) selected mathematics as their second subject, while one opted for biology and another for Spanish. These students were in the advanced stages of their programs, with their study experience ranging from the 5 th to the 8 th semester. They identified as white and middle class, aligning with the common demographic profile of pre-service teachers in Austria. Figure 4 shows the pre-post comparison in terms their knowledge gains, assessed through summative knowledge tests by Tepner et al. (2012). Due to the absence of one student at the post-test session, their data were omitted from the pre-post analysis.

Full size image

Our findings reveal a strong variability among students for content knowledge but not for pedagogical content knowledge. There was no substantial increase from pre to post, except for one student’s CK.

In terms of self-reported knowledge, there was an average increase in perceived CK and overall TPACK scores. However, responses related to PK and TK measures displayed diverse outcomes. The variations in individual progress are illustrated in Fig. 5, highlighting the within-person changes.

Full size image

Additionally, we report the relationship between our students’ self-reported PCK and CK with our summative questionnaire, effectively revealing no substantial correlation (Fig. 6).

Full size image

Our objective was to understand how students contextualized their perceived knowledge gains to their experiences in an SLA-based course on utilizing technology in chemistry education, focusing on CK and PCK. CK and PCK measures showed no systematic improvement across the learning group, with perceived knowledge gains lacking a clear correlation to assessed capabilities. This finding aligns with existing research in pre-service teacher education, indicating that unsystematic PCK patterns underscore the absence of a definitive link between the course and summative knowledge assessments (Holtz & Gnambs, 2017; Krauskopf & Forssell, 2018). The self-reported measures increased descriptively from pre to post, but we could not find a meaningful relationship with the summative scores, limiting reliable insights over a broader range of the TPACK dimensions. We interpret the lack of a relationship such that our students contextualize their knowledge gains very diversely. We support this interpretative claim via our text analyses.

We observed distinct patterns in students’ diary entries, notably one student deviating significantly in text length and content perspective.

Across the diaries, students’ reflections lacked a clear link between classroom activities, planning, and professional knowledge. This disconnect is supported by a documented inverse relationship between gaining professional knowledge and in-classroom experience time (Kulgemeyer et al., 2020, 2021), strengthening a hypothesis that the complex task of integrating digital technologies in a single classroom session overwhelms students, subsequently limiting their ability to apply their resources effectively. In our case, this resulted in reliance on general explanatory frameworks without reflecting the specifics of chemistry education, despite possibly having the necessary knowledge.

The overwhelming emergence of the “Culture” design space across diaries from the qualitative content analysis, with one student particularly emphasizing the role of digital technologies in education and the relevance of chemistry in daily life, indicates a holistic view rather than concrete professional development strategies. We observed a shift from cultural to more specific tech implementation discussions only as students detailed their thoughts, pointing to a nuanced understanding of digital media’s role in education yet lacking explicit connections to professional knowledge. In contrast, the SSLDA brought about a more nuanced perspective on our students’ writings. We see clear differences between the texts on the sentence level, reflecting the manifold contextualization of students when they report on what they learned.

We want to present and discuss our study’s technical and conceptual limitations. While we are going to discuss our limitations with a focus on the computational text analyses, we point out that the mismatch between the categorization by humans and the topic model could also mark the deductive human coding as insufficiently capturing the rich contextualization of our students’ self-reported knowledge gains. From a technical perspective, we are strongly limited by a low sample size that is only partially compensated by multiple data collections per student, i.e., five diary entries over time. Even though we extracted interpretable patterns, statistical noise is inevitably bound to sample size and, therefore, generalizing conclusions from our sparse data must be handled with due care. This relates to a potential loss of important information due to using only two dimensions in DBSCAN. However, when we tested our workflow with more dimensions, we were left with even more noise, making the link between human interpretation and (semi-)automatic pattern detection fuzzier.

Within NLP and ML applications, data leakage can become problematic (Tschisgale et al., 2025). Data leakage metaphorically refers to detrimentally informing a predictive model, e.g., classification of unknown data, with known data from the training set. This gives rise to phenomena such as overfitting and, among other biases, hurts generalizing research results. A full-sized data-split strategy to prevent data leakage in this study, including testing, training, and validating as well as separately embedding, analyzing, and evaluating stratified texts, led to unstable and uninterpretable clusters due to the low amount of data. We tackled that with our cross-validation approach, yet due to the joint word embedding of all texts, the analysis still risks leakage from the test data to the train data. The same goes for the naïve Bayes classifier, where we tried to trade off accuracy and train-test separation with a stratification by topics rather than participants.

Also, choosing an uncased BERT variant for the word embeddings could have negatively influenced further semantic differentiation of the German texts, and it could have amplified biases already discussed within research driven by machine learning (Nwafor, 2021; Traag & Waltman, 2022). For example, Fuster-Baggetto and Fresno (2022) discuss the anisotropy of BERT models in this regard. Their demonstration of how choosing the a priori boundary conditions for text models and interpreting the respective semantics is intertwined motivates us to point out a major conceptual limitation of our study: Establishing a causal relationship between word embeddings, semantics, and human interpretation of texts and text models is unsurprisingly and incredibly difficult (Feder et al., 2022). Addressing this limitation here extensively is well beyond the study’s scope. Yet, we emphasize that asking questions about causality and its relationship to theory, data, and data interpretation should be our next step in exploring natural language processing and machine learning applications in science education research. This challenge is not even specific to technology implementation in science education research (Weidlich et al., 2023) but rather to all scientific domains (Cinelli et al., 2022; McElreath, 2020; McGowan et al., 2023; Pearl, 2009) and, consequently, a mandatory part of future NLP applications in the social sciences.

Despite explicit seminar parts on TPACK and explicit instructions to concentrate on professional knowledge gains in their diaries, students’ written reflections were eclectic and often lacked an explicit focus on professional development. This approach to reflection seemed simplistic but, through NLP, revealed more profound insights into their learning processes, extending beyond mere declarative knowledge gains as envisaged by the SLA. This adds to our understanding of how students contextualize their professional knowledge by further enriching the toolbox for processing formative assessments. Students discussed their knowledge gains in the context of the broader implications of becoming a chemistry teacher, emphasizing a multitude of contexts for technology integration within chemistry classrooms. The low resolution of our self-report questionnaires can be seen as a consequence of this multitude. That is, a multiple-choice self-report questionnaire is not able to capture students’ diverse views on professional knowledge growth through practical in-class technology implementation. Such a perspective aligns with recent scholarly discussions on incorporating sociocultural considerations (Thyssen et al., 2023) into technology implementation in science education or looking at the importance of non-cognitive dimensions in chemistry education (Flaherty, 2020). If science education research aims to understand these dimensions, it needs to adapt more sophisticated approaches to students’ professional development. We suggest that CGT and NLP can help to reveal and quantify the finer-grained entanglements of, e.g., learners’ cognition, affects, and attitudes. The results reflect a chemistry-specific interconnectedness of knowledge dimensions from the RCM (Carlson et al., 2019): regarding the questionnaires, our students may have had no systematic increase in pPCK. But their reflective diaries allow a better understanding of their ePCK, i.e., what they actually did with what they know. In particular, our study extracted the empirical coincidence of the five design spaces (Mishra & Warr, 2021) and perceived knowledge gains of individual students over multiple points in time. This way, we combined the technical application of CGT, research on pre-service chemistry teachers’ professional knowledge, and the development of a seminar with respect to SLA.

Regarding future pre-service teacher training at the University of Vienna, our findings underscore the SLA’s effectiveness in encouraging students to contemplate the broader questions of their future profession. This suggests that, similar to addressing students’ preconceptions in learning, SLA seminars could start from broader educational questions. Here, Mishra and Warr’s (2021) design spaces can be starting points to frame seminars addressing pre-service science teachers’ professional development of digital teaching tools. Our application of CGT within the SLA-driven seminar effectively analyzed pre-service teachers’ diaries, linking their school experiences with digital technology use in chemistry classes. However, the challenge of systematically integrating professional knowledge persists—our seminar effectively had no impact on summative knowledge measures. Nonetheless, professional knowledge is crucial for both reflective practice and effective teaching. We suggest incorporating direct instruction phases about the broader educational context into higher education teaching could mitigate these shortcomings without diminishing opportunities for reflection. If students show interest in a broader range of questions regarding technologies in teaching and learning, such themes should be conceived of as entry points. This way, teacher educators could enclose and focus on the multi-faceted affordances of digitalization and digitality. This would call for a stronger individualization of teacher education present within the SLA framework, or, in other words, to appreciate and leverage the messiness in education as a fruitful and diverse resource.