A multi-feedback system integrated simulation-based teacher training to scaffold pre-service teachers’ teaching skills: A phenomenological approach

In Türkiye, PTs’ practical teaching experience occurs during their final year in the Teaching Practice course. This course is mandatory for all PTs and each subject education program provides opportunities for domain-specific practice. PTs observe teaching methods and techniques relevant to their respective fields, engage in individual and group micro-teaching practices using domain-specific teaching methods, create domain-specific teaching activities and materials, arrange teaching environments, develop classroom management skills, and assess and evaluate their teaching in a real classroom environment (Council of Higher Education, 2007). The course involves designing and implementing lessons with MTs at schools for six hours per week. Meanwhile, TEs observe and evaluate these lessons with a MT in the faculties for two hours. Despite the substantial positive effects of the teaching practicum, the literature reports that this ideal school-university cooperation is jeopardized for several reasons, for example, negative attitudes and interests of MTs, subjective evaluation, lack of communication of MTs and TEs, and lack of guidance of TEs (Cengiz, 2021; Yıldırım Yakar et al., 2021).

The current study involved novices (PTs) and experts (MTs and TEs) from four different fields who voluntarily took part in a simulation-based teacher training course. In this way, the sample was intended to represent maximum diversity. The literature demonstrates that phenomenological studies can be conducted with different numbers of participants under different conditions (Low, 2019; Yalçın, 2022).

PTs were purposefully selected from senior class PTs as the simulation-based teacher training was conducted within the teaching practice course. Despite the participants having different fields of specialization, it is important to note that they had all undergone relevant coursework to qualify for their participation in the research. Over a period of seven semesters, these PTs completed courses such as Computer and Communication Technologies, Research Methods in Education, Assessment and Evaluation, Classroom Management, and Content Teaching. The study began with 81 PTs, although 11 were unable to complete the training for various reasons. As a result, 70 senior PTs took part in the study. Eleven participants were from the department of Elementary Education, 13 from English Language Teaching, five from Computer and Instructional Technologies Education, and 41 from Mathematics Education.

The study also involved 12 mentor teachers. Two of these were primary school teachers, three English teachers, six mathematics teachers and one computer and instructional technology teacher. The MTs had successfully completed the in-service training, Teaching Practice Mentorship Training Course, provided by the Turkish Ministry of National Education and were teaching in the same field of education in which the PTs were trained. The TEs consisted of 6 faculty members with PhD degrees: one from the Department of Computer and Instructional Technology, two from the Department of English Language Teaching, two from Mathematics Education and one from Elementary Education. The MTs, who served as guides and supervisors to the PTs during their teaching practice, provided valuable insights and feedback based on their extensive classroom experience. The TEs also contributed to PTs’ experience in teacher education and pedagogy, providing guidance and support during their teaching practicum. The involvement of these experts ensured a comprehensive and well-rounded perspective in the study. Consequently, by choosing participants from this specific group, a direct connection was established between the researchers conducting the study and the PTs. This close association allowed for a more intimate understanding of the participants' experiences, as well as facilitating effective communication and collaboration throughout the research process.

The SimInClass simulation is an immersive virtual classroom simulation. It provides a simplified and structured virtual learning setting that incorporates the key elements of a real classroom setting, while also addressing the comprehensive processes of instructional planning and implementation (SimInClass, 2022). In the "With Colleagues" mode, illustrated in Fig. 1, PTs interacted with virtual students and practiced classroom management, teaching methods, and lesson planning (Kelleci & Aksoy, 2021). The interaction begins with an exploration of the profiles of the students in the virtual classroom and progresses towards the implementation of pedagogical strategies to deal with their behaviors. Furthermore, in the virtual setting, PTs can establish eye contact with students and walk around the classroom. PTs also have the opportunity to evaluate the effect of employing technological tools like interactive whiteboards, tablets, and e-books on classroom technology integration, depending on the chosen methods, techniques, and learning objectives. During a practice session, TEs and MTs can observe the PT’s virtual classroom performance, assign misbehaviors to virtual students, and provide summary and instant feedback. In addition, PTs are able to observe and provide feedback on their peers' virtual classroom experiences (Fig. 1a and c).

The simulation-based teaching training consisted of a 14-week curriculum designed to enhance the teaching skills of participating PTs and was conducted in the Teaching Practice course (Kelleci Alkan et al., manuscript submitted for publication). The curriculum comprised four modules, each lasting three or four weeks, focusing on areas such as lesson planning, technology usage, and classroom management. PTs firstly learned theoretical knowledge about the module content from training videos and supplementary materials tailored to each module's topic. This process was also to remind them of their previous pedagogical knowledge and skills. Then they virtually practiced using the "With Colleagues" mode of SimInClass simulation.

PTs then applied their simulation-based knowledge and skills in real classrooms, comparing and transferring their experiences. The TEs and MTs observed the PTs in the real classroom environment and evaluated them according to a rubric. Afterwards, feedback sessions (details provided in SimInTeach-FBS System) were held with the participation of the stakeholders by associating both the simulation performances and the teaching performances in the real classroom. PTs then transferred what they learned from the real classroom and feedback by practicing again in the simulation. The connection between simulation and real classroom context facilitated the transfer and structuring process of teaching knowledge and skills.

A system where MTs, TEs and peers can give feedback was integrated into SimInClass. The components of the multiple feedback system are called SimInTeach-FBS. The interfaces, given in Fig. 1 in SimInClass, are as follows:

 The first component of the SimInTeach-FBS was provided by the simulation itself. With SimInClass, users were furnished with a practice report encapsulating their virtual classroom experience (Fig. 1b). This report encompassed an evaluation of PTs’ teaching performance, scores, and feedback. Criteria for evaluation covered multiple aspects such as lesson planning and teaching aligned with Gagne's instructional model, employing pedagogical strategies referenced in the literature that deal with misbehaviors, effectively integrating technology, and engaging with students by accurately assessing their profiles. Each virtual classroom scenario involved the creation of a task list for evaluating teachers categorized based on the TPACK classification (Dehghan, 2015). In practice, the evaluation was conducted in accordance with predefined criteria, and feedback was provided accordingly. Additionally, the report featured a graph illustrating students' levels of knowledge, concentration, and engagement throughout the virtual classroom experience. This graphical representation allowed for a comparison with feedback of strategies for addressing misbehaviors, tasks related to TPACK components, and adherence to Gagne’s instructional model. Subsequently, the practice report served as a valuable tool for PTs to reflect on their teaching strategies and assess the impact of their actions on students' engagement and experiences. PTs could observe the effects of their instructional methods and the utilization of technological tools on students' knowledge retention, attentiveness, and enjoyment. The practice report functioned as a concise evaluation and feedback mechanism, empowering PTs to refine their teaching methodologies and enhance students' learning experiences.

The second component of the feedback system was carried out by the MTs and TEs. Within this framework, an interface integrated into the administrator module of SimInClass allowed MTs and TEs to monitor the virtual classroom interactions of PTs (Fig. 1d). This administrator module encompassed PTs' lesson plans, strategies for managing misbehavior in the virtual classroom, execution of TPACK tasks, and statistical data within SimInClass. Through this interface, MTs and TEs had the opportunity to comprehensively observe PTs' experiences and offer them valuable feedback. After completing each module, both MTs and TEs provided feedback to PTs based on a rubric developed by the researchers and integrated into the SimInClass simulation (Fig. 1c). This rubric consisted of 14 items (Appendix 1) and utilized the practice reports accessible in the administrator module of SimInClass as a reference. The items within the rubric assessed the effectiveness of training modules in various content areas, including lesson planning, TPACK skills, analysis of student characteristics, classroom layout, and classroom management. Through this rubric, the feedback aimed to evaluate how well these components were integrated within SimInClass. The feedback provided by MTs and TEs facilitated the improvement of PTs competencies within the SimInClass platform. By receiving feedback based on the rubric's evaluation, PTs were guided in refining their teaching strategies, enhancing their execution of TPACK tasks, analyzing student characteristics more effectively, optimizing classroom layout, and managing the classroom environment. Overall, the feedback process played a crucial role in supporting PTs' development and ensuring their competence in utilizing SimInClass effectively.

Development of the rubric incorporated the criteria established by the Ministry of National Education (Teacher Training and Development General Directorate, 2022) for the Teaching Practice course. Subsequently, the rubric was reviewed by three field experts, one assessment and evaluation expert, and one language expert. Their expertise and insights were taken into consideration, and the rubric was refined based on their opinions and suggestions. This collaborative approach ensured that the rubric aligns effectively with the objectives of the Teaching Practice course and incorporates valuable input from a diverse range of experts in the field.

The final component of the SimInTeach-FBS involved the peers of the PTs. Similar to the feedback mechanism involving MTs and TEs, each PT received feedback from fellow PTs who were also participating in the research. Peer feedback adhered to the same rubric (Appendix 1), ensuring consistency in the assessment process (Fig. 1a and c). Peer-to-peer communication was actively encouraged and facilitated to promote constructive exchange of feedback among the PTs. This phase was designed to cultivate a collaborative learning environment where PTs could mutually learn from and support each other's development as future educators. Peer feedback was integrated into the assessment process, following the same rubric utilized by MTs and TEs. PTs engaged in meaningful interactions with their peers, providing and receiving feedback based on the established evaluation criteria. The feedback exchange among peers served as a valuable resource for PTs, offering diverse perspectives and insights into their teaching practices.

To gain a deeper understanding of the participants' experiences, various tools were employed to collect data. One of these tools involved asking the PTs to reflect on their virtual classroom experiences and draw connections between the simulated environment and real classroom settings. The PTs were prompted to consider how they could apply their virtual classroom experiences to the real classroom. They were also asked to identify any differences they observed between these environments. To capture the PTs’ reflections and thoughts, they were asked to keep reflective journals (RJ). These journals served as the main data collection tool for the PTs as they documented their experiences, thoughts, and insights throughout the research process.

Open-ended feedback forms (OEFF) were used to collect additional qualitative data to complement the insights gained from the reflective journal. At the end of each of the four modules in the implementation process, OEFF was implemented to the participants consisting of the grand-tour questions. The questions of the grand tour can prompt participants to verbally narrate significant aspects of their experiences (Spradley, 2016). The grand tour questions included inquiries about how the feedback PTs received from various sources (simulation, MTs, TEs, and peer PTs) influenced their classroom behaviors and teaching skills.

In the SimInClass, the progress and performance of the PTs were closely monitored by both MTs and TEs. Regular meetings were conducted between MTs and TEs to discuss and provide feedback to the PTs. To further enhance the feedback process, MTs and TEs documented their experiences as video and text formats before and after the meetings held for feedback sessions. These data sources formed observation notes (ON) of MTs and TEs. This qualitative data source added depth and richness to the research findings, providing valuable insights into the effectiveness of the feedback system and its influence on the PTs' development.

This study adopted a holistic approach to data analysis and employed a content analysis method to analyze the data. Content analysis is an inductive approach aiming to derive descriptive concepts and expressions from data (Miles et al., 2014; Yıldırım & Şimşek, 2013).

The data analysis process began with the analyses of RJs and OEFFs of PTs. The written data was transformed to an Excel spreadsheet by two researchers. Data analysis continued with the coding of approximately 30 percent of the data in Excel. Subsequently, these code tables were independently coded by two different researchers who are experts in teacher education and qualitative research. These four researchers then discussed the similarities and differences in the codes with the aim of reaching consensus (Miles et al., 2014).

Two researchers with the greatest familiarity with the data proceeded to code the remaining data. While coding the transcripts on the Excel, they checked whether the data reached saturation. As the study was a part of a larger project, all participants’ data was included in the analysis in an attempt to avoid early proclamation of saturation and aiming for maximum diversity (Charmaz, 2006). This facilitated and contributed to the gaining of new insights and strengthening the data saturation. Through an iterative process of discussing the coding tables, an agreed-upon code book was finally formed.

After the coding process was completed, researchers established the themes and subthemes based on the coded data. Additionally, researchers used observation notes and notes derived from video recordings of MTs and PTs, which served as supplementary information to support and enrich the overall data analysis.

Preservice teachers (PT; e.g. PT1), mentor teachers (MT; e.g. MT1), and teacher educators (TE; e.g. TE1) were represented by abbreviations in the findings section during the interpretation of the participants' quotations.

Trustworthiness includes four evaluation criteria: credibility, transferability, dependability, and confirmability (Lincoln & Guba, 1985). A triangulation approach was employed to enhance the credibility of the findings by combining multiple data sources, methods, and perspectives. By utilizing RJs, OEFFs, and ONs, diverse insights were captured regarding PTs’ experiences in the SimInTeach-FBS. The trustworthiness of this study is also grounded in its methodological approach. Thick descriptions (Cresswell & Creswell, 2018) were provided with the elaboration of the study context, contributing to the transferability as well as purposeful sampling. The inclusion of PTs from diverse educational backgrounds and the MTs and TEs in monitoring PTs' performances contributed to the credibility of the study. Furthermore, these three types of participants share their experiences regarding the same phenomena and also serve as triangulation evidence.

The detailed implementation process description underscored the study's trustworthiness by emphasizing transferability. Also, the involvement of two researchers who were not included in the study as TEs and only participated in the data analysis, contribute to the confirmability, credibility and objectivity of the findings (Lincoln & Guba, 1985).

The iterative data analysis process also enhanced the confirmability and dependability of the study. Implementing an inter-coder check further strengthened the credibility of the analysis by minimizing coder bias and enhancing inter-rater reliability. Additionally, developing a codebook provided a standardized framework for data coding, ensuring consistency and coherence in the analysis process.

PTs usually mentioned that the feedback they received helped them improve their classroom management skills and transfer these skills to the real classroom under the categories presented in Table 1. PTs indicated that SimInTeach-FBS raised their awareness and improved their knowledge in dealing with misbehavior. As PT18 said: “My MT and TE told me that I should move around the classroom more and that I made very little eye contact. They also told me that I should deal more with misbehaviors. I learned how to be more successful in dealing with misbehaviors by taking these (feedback) into consideration at the end of the lesson” (from OEFF). Another pre-service teacher, P4, claimed that “Thanks to the feedback from my peers in the simulation…I gained insights into dealing with misbehaviors.’ Moreover, PTs stated that SimInTeach-FBS helped them transfer skills such as eye contact and walking around classroom to the real classroom during their teaching practices. As PT17 said: “I benefited from the feedback I received (via SimInTeach-FBS), such as making eye contact with students and walking around the classroom” (from OEFF). Another participant, PT37, stated that thanks to SimInTeach-FBS she was able to improve her skills in both awareness and transfer context, saying: “These feedbacks helped me realize the issues I need to focus on and develop during both virtual and real classroom” (from OEFF). In addition, participants mentioned the relationship between different codes under the CM theme, for example, P63 talked about how giving importance to student personality affects the occurrence of misbehaviors in the classroom:

I was able to transfer the things I learned (via SimInTeach-FBS) in the real classroom. I can say that preparing the seating arrangement according to the personality traits of the students is a very important issue. When two compatible students sit next to each other, there is no problem, but when two incompatible students sit next to each other, we may have to deal with many misbehaviors (from RJ).

In addition to the positive statements, the PTs also expressed negative opinions, for example, by stating that it was not realistic to transfer the simulation practices to real life. For example, PT25 said: “I don't think there are too many parallels with real life for undesirable situations. I don't think I will see an excessive effect of the feedback on my teaching life, because I don't think I had a realistic experience” (in simulation) (from OEFF). In addition, TEs and MTs observed that PTs gained experience in classroom management (walking around the classroom, eye contact, dealing with misbehaviors, time management, effective communication) through SimInTeach-FBS and transferred their experience to the real classroom environment. A representative observation regarding this is as follows:

“I observed that the higher scores in the simulation by the PTs walking in the virtual classroom and making eye contact with the students increased the number of steps they took in the classroom environment and the eye contact they established with the students” (TE3).

On the other hand, MT2 said that:

“As a result of the feedback I gave after each module, they made up for their deficiencies by applying what they could not do in the classroom in simulation. In this way, I observed that they developed methods of dealing with undesirable behaviors.”

MT1 stated that she observed the positive effects of PTs in the real classroom environment such as “Using time more efficiently,” and “Ensuring active participation of students in the lesson by establishing a more effective communication” (from ON).

The PTs stated that they gained experience in lesson planning from the feedback they received through SimInTeach-FBS during their teaching practice (see Table 1 for details). For example, PT2 stated that the feedbacks contributed to her lesson plan preparation and that she transferred this to the real classroom environment: “I have experienced that the lesson plans I have created also affect students' knowledge, fun and concentration levels (in teaching practice)” (from OEFF). PT18 stated the awareness she gained from the feedback in terms of LP: “I learned how to correct my small mistakes while preparing the lesson plan. I learned what to do when there is a misbehavior in the classroom, and how to keep the students' concentration high in the lesson. I learned how to maximize their level of knowledge” (from OEFF). Another pre-service teacher, PT38, spoke of her awareness in lesson planning with the help of the feedback: “I think that the feedbacks are very useful in terms of planning the lesson and how much time I should allocate to which teaching method” (from OEFF). PT23 said, “Thanks to this feedback, we saw what we did wrong or incomplete. We learned to prepare the lesson plan better, to intervene more accurately in misbehaviors, to use technological equipment more accurately” (from OEFF).

Participants believed the feedback they received in SimInTeach-FBS helped them improve their TPACK and contributed to their practices. For example, P5 stated that in addition to her classroom management and lesson planning experience, the feedback supported her TPACK. saying, “I realized the importance of using technology in lesson planning thanks to the simulation, therefore, I made use of technology as much as possible in my teaching practice courses” (from OEFF). PT3 indicated the contribution of the SimInTeach-FBS:

"...I transferred what I learned about TPACK to my teaching practice courses… I tried to teach the vocabulary items with a video and music on the smartboard... In short, I think that a technology-oriented lesson plan as in the simulation is very useful for students (from OEFF).

In the process, TEs and MTs also observed that PTs gained experience in the process of Effective use of technology and transferred their experience they gained from SimInTeach-FBS to the real classroom environment. TE6 mentioned said: “I was monitoring that PTs used technological tools and applications more frequently and effectively in the real classroom environment by including them in the lesson plan during the implementation process.”. Also, MT3 said: “As a result of the feedback I gave to the PTs, I saw that they included technological tools more frequently in the real classroom environment, both in their lesson plans and in practice” (from ON).

PTs emphasized that the CM, LP and TPACK experiences they obtained through SimInTeach-FBS were interrelated. Figure 2 demonstrates the relationship between these themes based on the interpretation of the data.

As Fig. 2 shows, two themes were categorized under teaching skills-based experiences of preservice teachers’ on SimInTeach-FBS, teaching experience and learning transfer. These two themes included three common themes: classroom management, lesson planning and TPACK. The arrows in Fig. 2 show the directions of the relationship of each theme with other themes. In other words, lesson planning affects both classroom management and TPACK, classroom management affects TPACK and lesson planning, and TPACK affects lesson planning and classroom management.

Regarding this relationship, PT9 said: “I try to apply what I learned in simulation in my teaching practice courses… I still try to integrate technological tools into the lesson plan while teaching. Simulation has an incredible contribution to real-life lessons” (RJ). Similarly, PT2 said: “I now pay a lot of attention to utilizing technology, daily life, and visuals while preparing the lesson plan” (OEFF). PTs also referred to the influence of SimInTeach-FBS on the relationship between CM and LP, for instance, PT47, who stated: “I have learned that to be able to teach the lesson effectively it is crucial to prepare the lesson plan by considering the students’ personality characteristics and planning a seating arrangement based on the LP” (from OEFF).

In addition, there is a relationship between teaching experience and transfer of learning. In other words, it is possible to say that PTs’ awareness of teaching skills is instrumental in the transfer of their experiences and learning and vice versa. Similarly, PT50 mentioned that he was able to see that what he learned through SimInTeach-FBS did actually work in a real classroom: “After I started playing simulation, I realized that in the real classroom I was moving around the classroom more and paying attention to making an eye contact with students” (RJ). PT70, talked about being aware of the possible outcomes of activity in a real classroom thanks to simulation: “when we switched to group work with hands-on materials, misbehaviors emerged in the simulation, I thought that I should be a little more dominant in classroom management because I thought that such a situation could occur in the real classroom” (RJ).