1 Introduction
1.1 Problem statement
1.2 Research questions
-
RQ1: What characteristics of role-playing did the students exhibit?
-
RQ2: What are the patterns of social presence, cognitive presence, and teaching presence for each group?
-
RQ3: How are the students’ perceptions of the role-playing learning experience and their learning satisfaction?
2 Relevant literature review
2.1 Role-playing
2.2 The CoI framework
3 Context of the study
3.1 CAI class description
3.2 Online role-playing and its procedures
Preparation | Orientation | Implementation | Internalisation |
---|---|---|---|
Lecturers’ activities | |||
• Determination of role-playing scenarios and rules of the game • Uploading reading materials for independent study • Group formation and role assignment | • Content orientation: explaining briefly about the CoI framework • Process orientation: briefing about role-playing process in a discussion forum, learning strategies and expectations | • Triggering each group based on individual roles • Monitoring the process and facilitating discourse with minimum intervention • Fading away as students become more engaged | • Initiating class discussion after completion of the role-playing to articulate the learning experience. Facilitating each group to carry out self-evaluation • Providing feedback |
Students’ activities | |||
• Joining a group and getting to know other group members | • Attending the orientation • Reading the material about the CoI framework | • Playing the role in asynchronous online discussion forums | • Offering feedback to other students • Engaging in group self-evaluation • Composing individual reflection to deepen understanding and organise newly learned knowledge |
Group | Person Played | |
---|---|---|
Self | Other | |
Students | √ | |
Lecturers | √ | |
Academic Secretariat and IT division | √ | |
Parents of The Students | √ | |
Association of Graduate Users in the field of IT | √ | |
Faculty Management | √ |
4 The method
4.1 Participants
4.2 Data collection
Type of data | Sources | Semester | Analysis | Measurement |
---|---|---|---|---|
Qualitative | Discussion Transcript | 2nd of 2019/2020 | Thematic content analysis | Cognitive presence, teaching presence, and social presence |
2nd of 2019/2020 | Thematic content analysis | Characteristics of the role-playing | ||
Reflection | 2nd of 2019/2020 | Thematic content analysis | Obstacles and students’ opinion | |
Lecturer evaluation by students (open-ended questions) | 2nd of 2019/2020 | Thematic content analysis | Suggestions, criticisms, ideas, and comments | |
Quantitative | Discussion forums | 2nd of 2018/2019, 1st of 2019/2020, and 2nd of 2019/2020 | Number of postings and average sentences per message | Students’ involvement |
Lecturer evaluation by students (6-Likert Scale Questionnaires) | 2nd of 2019/2020 | Descriptive statistics | Students’ learning experience and satisfaction |
4.3 Data analysis
5 Findings
5.1 Students involvement in role-playing
Group | Number of Postings | Average | ||
---|---|---|---|---|
2nd semester 2018/2019 | 1st semester 2019/2020 | 2nd semester 2019/2020 | ||
Students | 25 | 64 | 46 | 45 |
Lecturers | 24 | 57 | 67 | 49.3 |
Academic Secretariat and IT Division | 19 | 50 | 38 | 35.7 |
Parents of the Students | 23 | 56 | 42 | 40.3 |
Association of Graduates Users in the Field of IT | 19 | 68 | 56 | 47.7 |
Faculty Management | 27 | 57 | 57 | 47 |
The average numbers of posts | 22.8
| 58.7 | 51 | 44.2 |
The average numbers posts per person | 2.2 | 5.87 | 5.67 | 4.58 |
Group | Average Number of Sentences per Message | Average | ||
---|---|---|---|---|
2nd semester 2018/2019 | 1st semester 2019/2020 | 2nd semester 2019/2020 | ||
Students | 6.84 | 7.29 | 6.63 | 6.92 |
Lecturers | 6.92 | 7.61 | 4.92 | 6.48 |
Academic Secretariat and IT Division | 6.89 | 7.34 | 5.74 | 6.66 |
Parents of the Students | 5.30 | 8.32 | 6.98 | 6.87 |
Association of Graduates Users in the Field of IT | 7.16 | 8.54 | 5.52 | 7.07 |
Faculty Management | 6.40 | 9.56 | 5.86 | 7.27 |
The Average of Each Semesters | 6.58 | 8.11 | 5.94 | 6.88 |
5.2 The characteristics of the role-playing
Themes/ indicators | Group | Exemplary messages |
---|---|---|
Playing the role accordingly: • identifies themselves as a member of the group • mentions other members using specific title • presents points of view according to the role | All groups | Good afternoon, Dean Based on the Dean's statement regarding Online Collaborative Learning (OCL), are there any other advantages in implementing collaborative learning compared to regular online teaching during this distance learning session? Thank you, Academic Manager [Faculty management] |
Connecting the discussion to the current issue: the COVID-19 pandemic | Lecturers Academic secretariat and IT Division Association of Graduates Users in the field of IT Faculty management | Good afternoon, Dear Professors, I agree with Ms. Lulu's opinion, that the COVID-19 pandemic is an extraordinary event that requires faculty to take action regarding the implementation of teaching and learning activities [Lecturers] |
Conveyed obstacles and strengths of the OCL process | Students Lecturers Academic secretariat and IT Division Parents of students Faculty management | Good afternoon Mr XX I think the obstacles my children Shelly and X, Mr XX’s son, are quite similar, the lack of interaction in discussion. From the system's view, I think OCL is good for students, because OCL requires discussion among participants, increasing interaction while practicing our kids’ critical thinking skills. In my opinion, the role of the faculties should be able to be a facilitator for OCL activities, and parents can be a partner for kids’ discussion. At least, just being a listener, like my daughter to me (haha: D) is enough to help stimulate her confidence Regards, (Shelly's father)[Parents of students] |
Explicitly mentioning the CoI framework or its components | Lecturers Academic secretariat and IT Division | "Thank you, Prof. A What are the good strategies to create a social presence with online learning like this? Of course, we also want students to feel "present" in class and be able to express, discuss, ask questions and so on. Do you think there are ways we can make our students more comfortable in class? What are your opinion, Professors?” [Lecturers] Good afternoon friends, Apart from communication between students and lecturers, we also provide communication facilities aimed at the secretariat and lecturers who can help convey information to students. We understand that communication is an important element of the OCL as a model of Community of Inquiry theory. Therefore, we always try to improve the quality of communication between elements of the faculty. Regards, IT Division [Academic secretariat and IT Division] |
-
When participating in group discussions, students related to the current situation and their learning experiences based on the perspective of the roles they played.
-
Very few students explicitly mentioned the CoI framework or its components. No students reminded their group of the learning objective: to improve their skills by implementing the CoI framework in online discussion. All groups placed greater focus on the challenges of implementing online collaborative learning.
-
At the end of the discussion period, all groups produced sets of agreed ideas and concepts, but these were not compiled in the form of recommendation documents. However, the two groups summarised the results of their discussions. Only one group provided an in-depth rationale behind their recommendations and the arguments to support the recommendation (resolution).
-
Students raised the subject of the COVID-19 pandemic concerning the conditions associated with the application of distance learning. Students conveyed experiences as online learners (particularly those experiences that were less enjoyable). The experiences were expressed according to their role. For instance, parents of students shared their child’s difficulties in learning in the current pandemic situation. Students presented their real experiences from the perspective of their role-play group.
-
Students conveyed expectations of distance learning (from a student perspective). The expectations were expressed in the discussions as those of the role-play group. Moreover, they explained the situation that triggered the expectation. In the joint discussion and reflection session (following the role-playing activity), students were asked to share their experiences of implementing CoI. All students applied all of the CoI components. However, only a small number of students stated that they had consciously implemented the CoI framework explicitly.
5.3 The pattern of the social, cognitive, and teaching presences
The CoI components | Frequency |
---|---|
Social Presence (SP) | 253 |
Teaching Presence (TP) | 136 |
Cognitive Presence (CP) | 225 |
The CoI components | Categories | Frequency |
---|---|---|
Cognitive Presence (CP) | Triggering Event | 26 |
Exploration | 124 | |
Integration | 99 | |
Resolution | 1 |
-
Four groups showed a similar pattern in the presentation of particular indicators of CP: they conducted fast triggering events, extremely intensive exploration, followed by less intensive integration. Two groups demonstrated a higher intensity of integration compared to exploration. The group of lecturers displayed a process of fast triggering events and exploration before proceeding to integration and synthesising of ideas.
-
All groups tended to interpret problems rapidly and carried out considerable analysis and integration of ideas. Whether agreeing or disagreeing during discussions, students often provided arguments to support their thinking.
-
The most challenging indicator to achieve is that of the resolution, as it requires the ability to apply solutions to different contexts or defend proposed solutions. As a generalisation, groups only proposed solutions in the form of recommendations and brief background information. Only one group achieved resolution; this group proposed a detailed background-supported solution, complemented by the impact of implementing the proposed recommendations and the consequences should they not be implemented.
-
The explicit mention of CoI or its components is limited, occurring in less than five messages, and only two groups mentioned it: lecturers and academic secretariat, and IT Division. A lecturer discussed the use of CP in solving specific problems. An IT Division member argued the application of the CoI framework to improve the quality of communication in the faculty environment.
-
Although it does not explicitly discuss the CoI framework, the application of SP, TP, and CP were performed by all groups; however, the essential sub-indicators of these presences have not all been applied. For example, students still lack TP in terms of reminding each other of the learning objectives, assessing the discussion progress, and summarising the discussion results. The dominant sub-indicator of TP is the directing of the discussion through asking questions.
5.4 Students’ learning experience and satisfaction
5.4.1 Reflection
5.4.2 Obstacles
5.4.3 Students’ opinion
5.4.4 Lecturer evaluation by students
The components | Questions | Result | Average |
---|---|---|---|
Learning Content | Lecturers convey the teaching design (Lecture Unit / Syllabus / Teaching Design Book) clearly at the beginning of the lecture | 5.66 | 5.70 |
Learning materials are arranged systematically to make it easier to understand the interrelationships between the materials | 5.75 | ||
Lecturers encourage students to actively participate in the learning process | 5.68 | ||
Teaching–Learning Process | Lecturers convey lecture material clearly | 5.68 | 5.65 |
Lecturers are able to create a classroom atmosphere that is conducive to learning | 5.62 | ||
Lecturers are able to provide examples that help to understand difficult concepts | 5.58 | ||
Lecturers are able to answer questions clearly | 5.66 | ||
Evaluation of the learning given (for example: quizzes, Midterm Test, Final Test, assignments, etc.) in accordance with the teaching material | 5.71 | ||
The weight of each component of the assessment is in accordance with the workload | 5.66 | ||
The grades that I get represent my true abilities | 5.68 | ||
Class Management | Lectures are held on time | 5.42 | 5.59 |
Lecturers are open to receiving input from students | 5.75 | ||
Lecturers apply teaching contract consistently | 5.60 | ||
Learning Evaluation | Evaluation of the learning given (for example: quizzes, Midterm Test, Final Examination, assignments, etc.) in accordance with the teaching material | 5.69 | 5.66 |
The weight of each component of the assessment is in accordance with the workload | 5.66 | ||
The values I get represent my true abilities | 5.63 | ||
Overall Average Score | 5.65 |
Suggestions, criticisms, ideas, and comments |
---|
• One of the best courses and instructors |
• Thank you, Mr. A and Mrs. B |
• Thank you, Mrs. A., Mr. B for the learning experience |
• Thank you |
• Please note to the time the lecture ends so as not to exceed the time it should be |
• I am very happy in learning this CAI course. I can learn a lot about what if in the future I want to build a website for computer-aided learning but also put forward existing learning theories. Here I have learned a lot from Mr. A and Mrs. about how to communicate well, improve critical thinking, etc. Hopefully Mr. A and Mrs. B are always healthy so they can still teach fun CAI classes again and maybe each meeting can be started with an interactive mini game to trigger students' pre-existing knowledge about what they have learned in the previous meeting:) |
• Thank you very much for appreciating even out small contribution:)) |
• Thank you Mr. A and Mrs. B, cheer up, as always! |
• Mr. A and Mrs. B are very good at teaching |
• The best |
• Students are required to be active in the CAI classes |
• Very Good |
• Thank you very much for the knowledge, Mrs. B and Mr. A. I am very grateful to be guided by extraordinary lecturers like both of you. Hopefully the knowledge gained in the class will give blessings for today, the following days and the hereafter. Thank you |
• The learning methods applied to the CAI course are very representative of the learning theories we learn in this course, so that we experience the real implementation in our daily life. Mr. A and Mrs. B as lecturers always provide direction and feedback to us so that we know where the mistakes are and what should be improved. Thank you Mr. A and Mrs. B, I hope you will always be healthy |
-
Learning Content: Lecturers encourage students to participate in the learning process actively.
-
Teaching–Learning Process: Lecturers can create a classroom atmosphere that is conducive to learning
-
Class Management: Lecturers apply the teaching contract consistently.
6 Discussion
The challenges | Proposed strategies to overcome the challenges |
---|---|
At the beginning of the discussion, group members are inactive Note: Possibly because they do not realise the learning objectives and the benefits of role-playing, or they do not see a direct impact on learning attainment | • Dissemination and modelling of social, teaching and cognitive presences • Remind students of the relevance and purpose of learning and the reasons for selecting the role-playing method • Explicitly convey expectations • Provision of learning incentives, such as demonstrating appreciation for student contributions and the value added to discussions, based on the process and results of the discussion |
Asynchronous discussion is flexible. As a result, the learning process takes longer than scheduled for face-to-face. Role-playing requires a process of simulating the asynchronous online discussion and carrying out an assigned role | • Taking advantage of the flexibility of asynchronous discussion, such as giving more time to think deeply and prepare quality contributions |
Students enjoy playing the assigned role; this results in the focus being drawn away from the learning goals, namely, applying the CoI framework Note: Students may be confused between the purpose of the role-playing activity and the expected output of the discussion: recommendations for faculty management in implementing OCL. It is possible that students do not fully understand the CoI framework, in particular, the sub-indicators and their important functions | • In the preparation stage of the role-playing, it is necessary to ensure that students understand the CoI framework, its sub-components and their important functions • Explain to students the purpose of the discussion and the expected outcome(s) • Remind students of the learning objective during the discussion • Explicitly remind students to apply every component and sub-indicator of social, teaching and cognitive presences, in accordance with the dynamics of the discussion |
It is difficult to evaluate individual and group performance in online discussions, both in general and in role-playing. The provision of meaningful and timely feedback and maintaining learning motivation presents further challenges | • Provide opportunities for students to perform peer assessment (students assess the performance of their peers) • Provide rubric and its explanation |
The success of individuals and groups depends on individual contributions and the ability of individuals to interpret and carry out their roles Groups whose discussions go well are those that have one or more active members. This dominant member indirectly inspires other members to respond. Groups that do not have a prominent member tend to be inactive | • Heterogeneous group formation • Encourage each group to select a leader, timekeeper and other possible roles needed • Encourage the sharing responsibility |