Impact of remote experimentation, interactivity and platform effectiveness on laboratory learning outcomes

Moore’s Transactional Distance Theory (TDT) has proven to be an immensely useful theoretical framework to study the effectiveness of remote laboratories in engineering education (Moore, 1973, 1991; Garrison, 2000; Jung, 2001; Murphy and Collins 1997; Goel et al., 2012; Delgaty, 2018). The distance between the instructor and the student is substantial in remote education (Tirado-Morueta et al., 2018) and TDT explains and quantifies the learning relationship between instructor and student in the remote learning situation (Delgaty, 2018). The high transactional distance between teacher and students may contribute to feelings of isolation and reduced motivation and engagement in students (Moore, 1991). According to Moore, the three constructs of TDT are (1) Structure, (2) Interaction (or Dialogue), and (3) Learner Autonomy (Moore and Kearsley 2011). Structure represents the rigidity or flexibility of the instructional materials and methods, interaction represents the interaction between the instructor, students and machine (or equipment) (Moore, 1989; Lowe et al., 2008; Sher, 2009), and autonomy represents the nature and degree of self-directedness of the students. Identifying the level of structure required, facilitating interaction and encouraging individual learner autonomy is demanding as the greater the structure and the lower the interaction, the more autonomy the student must demonstrate. The interactions that are part of the learning process (Goel et al., 2012) help to improve the conceptual understanding of the students (Wei et al., 2019). Successful transactional distance environments depend on the teacher providing opportunities for interaction and ‘appropriately’ (Moore, 1993) structured learning materials. The greater, and faster, and more involved the level of interaction, the lower the level of psychological feeling of separation there would be (Wei et al., 2019; Moore & William, 2007). Efficiently structured content that utilizes latest technologies and increased interaction promote effective online learning (Sun & Chen, 2016). Literature suggests that transactional distance is directly proportional to structure and inversely proportional to interaction (Saba, 2012; Demir Kaymak & Horzum, 2013). Higher learner autonomy implies less structure required to reduce the transactional distance (Kearsley & Moore, 2012). Compared to traditional learning, online learning happens through viewing pre-recorded videos via the internet. Online platforms are highly flexible (low structure) and allow interactive learning environments (Jung, 2000; Jung, IS 2000; Pauls, 2003). The key objectives of this work included assessing and comparing the PL-UTM and RT-UTM features with respect to the transactional distance, which is inversely proportional to the effectiveness of the remote laboratory (Lindsay et al., 2007). The TDT framework for RT-UTM depicted in Fig. 1 included multiple structural components such as detailed theoretical descriptions, self-evaluation questions, remote user-interface for defining experimental parameters, live result observation windows with calculator to monitor output parameter evolution, and instant data-set export function. The in-built virtual lab learning management system (VL-LMS) and discussion forum aids student-instructor (S-I) and student-student (S-S) interactions. Finally, the conceptual understanding analysis gauges the flexibility of the laboratory platform’s structure in aiding student learning.

The focus of this study was experimentation with UTM, which is used for load application and deformation measurement involving tension, shear, bending, and compression. The bouquet of experiments within UTM included determination of Young’s modulus, determination of Poisson’s ratio, study of stress concentrations on a plate with a hole and notch, and study of Saint Venant’s principle. A sample specimen setup in the UTM is shown in Fig. 2.

The UTM that is predominantly used in the mechanics of solids undergraduate laboratory course is utilized by batches of students to perform experiments physically (hence termed PL-UTM). Typically students spend between 2–3 h in the physical lab. An hour is spent on instruction of theory and procedure followed by 30 min to perform the initial measurements of length, diameter of specimen, and strain gauge wiring. This is followed by physical experimentation for about an hour. Most of the doubts regarding theory and procedure are raised and interactively addressed during the experimentation when students see the actual test and come to terms with what is happening. This takes time away from focusing on the actual test data during the live experiment. They perform the analysis asynchronously after laboratory hours. Thus physical laboratory courses taught in STEM (Science, Technology, Engineering, and Mathematics) education are rigid in their structures due to various constraints such as limited time to complete experiments, safety protocols that are required to be followed and in some cases insufficient infrastructure or resources for personalized learning, which lead to inadequate hardware interaction (student-machine, S-M) which limits the learning effectiveness (Lal et al., 2019; Lowe et al., 2008).

The design of educational activities in the Remotely triggerable UTM (i.e., RT-UTM) are shown in Fig. 3. The features, flexibility and procedural elements of the platform were prepared to help learners exercise increased autonomy. Concepts are introduced highlighting the underlying theory prior to the start of virtual experimentation. A set of pre-assessment questions were provided in the RT-UTM to emphasize and gauge understanding of key concepts. Through an interactive graphical user interface (GUI) learners are not only able to, in real time, control the experiment but also view the live streaming of the experimentation. Additionally they also have the facility to download and analyze the data. Learners are able to repeat experiments, imbibe the educational content and analyze data in a far more synchronous fashion compared to PL-UTM. The details of the remote experimentation set up and user interface of the experiment are described in the following sections.

The learning platform is comprehensive in that it not only allows remote experimentation, but also provides detailed preparatory material to the users that helps them with (1) understanding the theory and concepts behind the experiments (2) a detailed step-by-step experimental procedure, (3) video tutorials of how the experiments are performed in physical laboratory, (4) self-evaluation quizzes to evaluate themselves on conceptual knowledge prior to start of the experiment, (5) assignment questions on computing parameters through RT-UTM. Immediate feedback from the self-evaluation helps users to correct their understanding before performing the experimentation. The assignment questions cover the complete aspects of UTM experiment and were prepared by mechanical engineering faculty members who have not only taught UTM concepts but are also subject matter experts. These assignments enhance student learning by facilitating application of knowledge gained through RT-UTM Laboratory to real world problems. Since Remote triggered experiments are restricted to one user at a time, a scheduler is integrated and that helps to accommodate multiple requests sequentially on the RT-UTM.

On the host side, the specimens are changed manually once a day according to the schedule of the experiment. The scheduler in the procedure tab helps to arrange one experiment every day to minimize human intervention. No further human presence is allowed in the remote lab at the time of experimentation and the performance of the hardware is also monitored remotely. The experiments performed are described as follows.

The benefits of the RT-UTM design in terms of structure can be seen in Table 1. The rigidity or flexibility of the RT-UTM structure can be assessed in terms of variety, individualization, formality, media use, cognitive load, visualization, functionality and usability (Huang et al., 2015). Variety is enhanced in RT-UTM by multiple content formats (audio, video, animation) and modes of interaction (S-I, S-S, S-M). Individualization is also improved in RT-UTM by the self-paced nature of the platform which allows multiple experiment repetitions under different load rates and two-way communication (learning management system), while maintaining the same structural formality in terms of course outlines and objectives. The ease of media use (interface and experiment controls) provided by RT-UTM enriches the learning experience by promoting better S-M interaction (Lowe et al., 2008; Lal et al., 2019, 2018). The cognitive load on RT-UTM is expected to be lower with the use of ICT and absence of physical hazards. Step-by-step procedure is provided for the instrumentation and controls which, if not followed sequentially, will prevent activation of the controls for the next stage of the experiment, and guide the user to revisit the procedural steps until the correct operational sequence is performed. This feature is typically absent in physical labs, allowing erroneous test initiation and controls, which lead to incorrect measurements, safety concerns, and higher cognitive load requirements. Visualization is characterized by a spatially well-designed RT-UTM interface while ensuring the same functionality as a physical lab through the interaction afforded by the learning management system (LMS). Usability is improved in RT-UTM through easy navigation of the online platform.

Enhanced interaction between students and teachers is aided by the integration of LMS within the Virtual Lab framework (Nedungadi et al., 2011). The primary users of VL-LMS are either the students or the instructors with each having specific access privileges. Figure 11 shows the schematic representation of the VL-LMS with user privileges. Instructors can prepare and manage the assessment questions, and assign them to groups of students, schedule the assessment time and track individual student responses. On the other hand, student privileges include: entering into group or individualized chats with peers or instructors, viewing and submitting assignments, performing RT experimentation, self evaluation tests, and providing feedback. VL-LMS stores student interactivity data in the server and this includes their responses to questions, the number of attempts taken to answer questions, total time spent online and so on. The student-student (S-S) and student-instructor (S-I) interactions were captured while performing the experiments in physical and RT-UTM laboratories as shown in (Fig. 12). In RT-UTM, interactions were captured using VL-LMS whereas, in the physical lab the interactions were captured from the observations of students and feedback from multiple instructors.

For the assessment of the PL-UTM and RT-UTM in meeting the learning outcomes, an experimental study (Fig. 12) was conducted on two groups of students, group A (GA, N = 50) and group B (GB, N = 50), from the second year undergraduate (UG) mechanical engineering course. The participants were grouped randomly to avoid any bias and both groups had undergone an UG level theory course on Mechanics of Solids. Two types of assessments were conducted for this study. One was conceptual understanding (CU) of governing principles behind experiments through a conceptual understanding questionnaire (CUQ) and the second assessment was the effectiveness of use of the learning platform through a survey instrument. Ethics approval was gained for the data obtained from the Institutional review board.

VL-LMS platform allows students to have interactions with instructors (S-I) as well as peers (S-S). A comparison of such interactivity in RT-UTM and Physical-UTM laboratory sessions are tabulated in Table 2.

The data shows remote students worked and re-worked on experiments three times more than those in physical labs (PLs). There was also significant difference in the time taken for completion of assignments between the the two modes with remote students taking 30% lesser time. The S-S interactions were more in the PLs and this corroborates with earlier findings that physical proximity promotes interaction (Corter et al., 2011; Lindsay et al., 2007; Messman and Jones-Corley 2001; Park et al., 2017; Fila and Loui 2014). On the other hand, the S-I interactions increased with RT-UTM (Lal et al., 2019). On analysis of the interactions and questions, it was found that students who were learning remotely had more questions overall on the theory, experimentation and procedural processes. This is indicative of the impact of an immersive environment that allows students to experiment more frequently, think independently and seek answers quickly. These results corroborate with a prior study on usage of LMS to capture student-instructor interactions in remote laboratories (Tirado et. al 2018). In addition, S-M interaction also increased in RT-UTM due to one-on-one experimentation on the hardware. Thus, RT-UTM has enabled more S-I and S-M interactions in comparison to PL-UTM, leading to lower TD with RT-UTM.

Assessment of learning through design and development of conceptual understanding questionnaire (CUQ) helped quantify understanding of governing scientific phenomena behind the experiments. A pre- and post-CUQ assessment was conducted before and after the laboratory experimentation (Fig. 12). An independent sample t-test is used to compare the results of Group A (GA) and Group B (GB). The pre-test results in the Table 3 shows that there is no significant difference (p = 0.39) in the score of GA-pre (M = 3.80, SD = 1.85) with GB-pre (M = 3.50, SD = 1.66), implying student groups GA and GB were similar in their CU prior to the experimentation.

GA students were assessed using the CUQ questionnaire after performing the experiment in PL (intermediate-CUQ) to understand the level of CU of students (called GA-inter). The results show that there is a significant difference (p < 0.05) when compared to the average score of GA-pre (CU score of GA before performing PL) as a result of physical lab experimentation. Similarly, the average score of GA-inter is significantly different from GB-pre (CU score of GB before performing RT-UTM).

The students were again assessed using the same questionnaire after performing the experiment in PL+RT (GA) and RT-UTM (GB) (Fig. 12). The results suggest a significant improvement in the conceptual understanding of both GA-post and GB-post students after performing RT-UTM experiments compared to the pre-experiment (GA-pre and GB-pre) data and the GA-inter data post-physical lab only, although no significant difference was found between the two post-experiment groups (GA-post and GB-post). This illustrates the benefit of incorporating RT-UTM intervention in Mechanics of Solids education to aid physical lab conceptual understanding (Corter et al., 2011; Lowe et al., 2013). Additionally, we compared the average CUQ scores of both groups before and after performing the experiment as shown in Fig. 13A. These box plots show the range of scores of students from minimum to maximum, the average score (solid horizontal line), four interquartile ranges, data points and distribution curves of both groups. Group A shows 133.2% increase in the average score after performing experiment in PL. A further increase of 30.2% in the average score of CUQ after performing the same experiment in RT platform is observed (A total of 203.7% increase in the average score from 3.80 to 11.54), confirming the t-test observation that RT-UTM intervention results in statistically significant improvement in conceptual understanding due to its self-paced personalized nature. Compared to GB-pre, GB-post shows 219.4% increase in the average score of CUQ after performing experiment in RT platform directly without physical lab, confirming the t-test observation that the benefit of RT-UTM intervention in physical lab education (RT + PL) is statistically indistinguishable from RT-only. This shows that RT + PL incorporates all the benefits of RT platform, like self-paced learning and personalized test visualization, which help in improving conceptual understanding.

The percentage of students who got correct answers for each MCQ is shown in Fig. 13B. Stress concentration due to the presence of notch on the specimen (Q9, Q10) and Poisson’s ratio (Q13, Q14) were the concepts students found most difficult to comprehend. These gaps in conceptual understanding were corrected after using RT-UTM platform due to more S-I and S-M interactions resulting in lower TD. Figure 13B shows only 40–60% of students were able to identify the correct answer for each question after the physical laboratory experimentation. But approximately 80%-90% of students were able to identify the correct answers for each question after the integration of RT-UTM into the study.

The effectiveness of learning platform feedback assesses the following usage-related critical aspects of UTM experimentation: (1) comprehension without supervision, (2) a well-defined procedure (3) user interactivity, (4) user convenience and (5) impact on learning. Figure 14 shows the students’ feedback after performing PL-UTM (Group A) and RT-UTM (Group B). The responses from students indicate that majority of students either strongly agreed or agreed that RT-UTM is effective in all of above five aspects. From the feedback, 84% of the students agreed or strongly agreed that RT-UTM helped them comprehend the concepts and run the experiments without supervision and 60% of the participants reported that physical lab helped them comprehend the concepts and run the experiments without supervision. In the physical lab, students need to interact with the instructor to gain knowledge about the experiment. But in RT-UTM, contents like animated videos, simulated experimental setup, and video lectures add variety and individualization to help students understand the concepts without supervision. Students are able to review the RT-UTM learning material multiple times without time constraints until adequate comprehension is exhibited in self-assessment. This increases the flexibility of the structure and lowers the transactional distance. More than 80% of the students agreed that the RT-UTM had useful step-by-step procedure that aided in the execution of experiments. When comparing with RT-UTM, only 37% of the participants responded that physical lab has well-defined procedure. The rigid time-bound structure of physical lab does not allow self-paced learning prior to self-assessment of conceptual and procedural clarity. A significant number of users (i.e., over 90%) felt that RT-UTM had good user interactivity resulting in satisfactory user experience. The interactive graphical user interface of RT-UTM allows students to control the loading and unloading speed of the experimentation and allows real-time data visualization and capture. More than 79% of the students agreed that RT-UTM is convenient to perform the experiments. This can be explained by the higher functionality, media richness, and usability of RT-UTM platform. Impact of learning score shows the average of all the above factors, which represents how much the structure of RT/PL helps students to attain the goal of the experimentation. About 83% of the students either strongly agreed or agreed that RT-UTM had an impact on their learning outcomes where as, in physical lab, only 52% of the students strongly agreed or agreed that the structure of PL had an impact on their learning outcomes. RT-UTM allows students to perform and repeat the experiments at their convenience, enhancing flexibility in structure in addition to higher S-I and S-M interaction. This enhances autonomy and reduces the transactional distance, resulting in effective learning.