To explore engagement, disengagement, the relation between them and the extent to which upper secondary school students perceive their digital skills to meet their educational requirements, we proposed three hypotheses:
4.1 Hypothesis 1 (H1)
We hypothesised that students reporting high levels of general engagement in learning are more likely than students reporting low levels of engagement in learning, to experience higher levels of engagement when learning with technologies. To test this hypothesis, we performed an independent samples t-test comparing students’ general engagement in learning, using high and low levels of engagement as a grouping variable and a combined variable of engagement in TEL.
Table
2 reveals that the students reporting high (M = 4.23, SD = 0.64) and low (M = 4.02, SD = 0.64) levels of general engagement in learning and TEL differ significantly (t(410) = 0.88,
p < 0.05). This means that students who generally display high levels of engagement also display higher levels of engagement in TEL. Thus, we found this hypothesis to be supported.
Table 2Comparison of high and low levels of general engagement in learning and engagement in TEL using a t-test
Engagement in technology-enhanced learning | 129 | 4.02 | 0.64 | 281 | 4.23 | 0.64 | 0.88 | 0.01 |
4.2 Hypothesis 2 (H2)
We hypothesised that students' reported level of digital skills is related to how they engage or disengage when learning with technologies; that students with higher levels of digital skills engage more, and that students with low levels of digital skills disengage more than students with high and low levels of digital skills respectively. To test this hypothesis, we first used engagement indicators to explore if students with high levels of digital skills engage more in TEL than students with low levels. We then used disengagement indicators to explore if students with low levels of digital skills disengage more.
We had asked the students to report the extent to which they were competent in managing the learning technologies at school to support their learning. We conducted two independent sample t-tests to test our hypothesis, one using a dummy coded variable of digital skills, and variables reflecting engagement in TEL. We also conducted a bivariate correlation test to further identify the relationship between students’ digital skills and their engagement in TEL.
Using an independent sample t
-test (see Table
3), we found a significant difference between students that reported low levels of digital skills (M = 4.00, SD = 0.66) and students that reported high levels (M = 4.26, SD = 0.61) in terms of their engagement in TEL (t(410) = 0.42,
p < 0.01). Thus, the t-test supports our hypothesis that students with strong digital skills engage more in TEL. We conducted an additional t-test to study the hypothesis with separate variables (see Table
4).
Table 3Comparison between digital skills and overall engagement in TEL using a t-test
Engagement in technology-enhanced learning | 140 | 4.00 | 0.66 | 270 | 4.26 | 0.61 | 0.42 | 0.01 |
Table 4Comparison between digital skills and separate indicators of engagement in TEL using a t-test
I use IT to support my learning | 140 | 4.91 | 1.01 | 270 | 5.19 | 0.84 | 0.06 | 0.01 |
I routinely search the Internet | | 4.93 | 1.01 | | 4.98 | 1.04 | 0.29 | 0.62 |
I routinely use published materials | | 4.16 | 1.28 | | 4.50 | 1.30 | 0.82 | 0.01 |
Autonomously switching between different technologies when doing schoolwork increases my engagement | | 4.60 | 1.27 | | 4.89 | 1.08 | 0.01 | 0.02 |
I concentrate easily when working with digital technologies | | 4.11 | 1.17 | | 4.53 | 1.17 | 0.51 | 0.01 |
I take the initiative to use IT | | 3.79 | 1.42 | | 4.44 | 1.35 | 0.52 | 0.01 |
I need IT to maximise my learning | | 4.01 | 1.48 | | 4.29 | 1.38 | 0.93 | 0.06 |
I use IT as a cognitive enhancement | | 4.03 | 1.47 | | 4.35 | 1.35 | 0.55 | 0.03 |
I want the school to use more IT | | 3.44 | 1.28 | | 3.87 | 1.38 | 0.82 | 0.01 |
More, and more varied, IT increases my engagement | | 3.95 | 1.20 | | 4.11 | 1.29 | 0.37 | 0.21 |
IT increases my engagement (general) | | 3.70 | 1.44 | | 4.23 | 1.31 | 0.20 | 0.01 |
It is important that the school’s IT works | | 4.89 | 1.17 | | 5.11 | 1.03 | 0.10 | 0.06 |
Digital creation enhances my interest in learning | | 3.06 | 1.30 | | 3.78 | 1.35 | 0.61 | 0.01 |
My absenteeism would decrease if I could attend class from elsewhere | | 3.96 | 1.85 | | 3.72 | 1.81 | 0.96 | 0.20 |
Personalisation increases my engagement | | 3.95 | 1.25 | | 3.99 | 1.31 | 0.78 | 0.79 |
My teachers have insight into my learning process | | 3.76 | 1.32 | | 3.99 | 1.41 | 0.55 | 0.11 |
I prefer digital feedback | | 3.47 | 1.64 | | 3.87 | 1.66 | 0.49 | 0.02 |
I am satisfied with the teachers’ use of digital technologies to provide feedback | | 4.04 | 1.25 | | 4.52 | 1.26 | 0.28 | 0.01 |
IT is used to prevent social exclusion | | 2.85 | 1.37 | | 3.04 | 1.37 | 0.37 | 0.17 |
Our group instructions mean that everyone participates | | 3.69 | 1.19 | | 3.82 | 1.23 | 0.57 | 0.31 |
Table
4 reveals that all but one variable displays a relation between digital skills and engagement in TEL. Students who reported higher digital skills than students with lower digital skills to greater extent:
1)took the initiative to use digital technologies,
2)concentrated easily when working with technologies;
3)experienced that switching between digital technologies increased their engagement,
4)experienced that technologies were used as cognitive enhancements,
5)created using technologies,
6)found asynchronous technology-mediated communication between themselves and the teacher sufficient and satisfying,
7)preferred digital feedback, and
8)used learning materials their teacher had published.
While most of the indicators were significantly related to high levels of digital skills, students with low levels of reported digital skills (M = 3.96, SD = 1.85) reported to a higher extent than students with higher digital skills (M = 3.72, SD = 1.81) that their absenteeism would decrease if they could attend class from elsewhere (t(410) = 0.96,
p < 0.05). To further investigate the relation between students’ digital skills and engagement we performed bivariate correlation tests. Using a combined variable of student engagement in TEL, we found a significant correlation with reported digital skills (r = 0.22,
p < 0.01).
Table 5Correlation between digital skills and separate indicators of engagement in TEL
1. Students’ reported digital skills | 1.00 | | | | | | | | | | | |
2. I use IT to support my learning | 0.15** | 1.00 | | | | | | | | | | |
3. I routinely use published materials | 0.15** | .020** | 1.00 | | | | | | | | | |
4. Autonomously switching between different technologies when doing schoolwork increases my engagement | .012* | 0.47** | 0.13** | 1.00 | | | | | | | | |
5. I can concentrate easily when working with digital technologies | 0.20** | 0.31** | 0.22** | 0.29** | 1.00 | | | | | | | |
6. I use my own initiative in using IT | 0.23** | 0.32** | 0.13** | 0.38** | 0.25** | 1.00 | | | | | | |
7. I want the school to use more IT | 0.17** | 0.20** | 0.07 | 0.27** | 0.25** | 0.42** | 1.00 | | | | | |
8. IT increases my engagement | 0.20** | 0.40** | 0.08 | 0.36** | 0.37** | 0.38** | 0.49** | 1.00 | | | | |
9. More, and more varied, IT increases my engagement | 0.13** | 0.24** | 0.09 | .032** | 0.26** | 0.35** | 0.36** | 0.39** | 1.00 | | | |
10. Participating in digital creation enhances my interest in learning | 0.26** | 0.23** | 0.06 | 0.23** | 0.19** | 0.31** | 0.34** | 0.50** | 0.26** | 1.00 | | |
11. I prefer digital feedback | 0.12* | 0.25** | 0.21** | 0.27** | 0.18** | 0.27** | 0.30** | 0.34** | 0.23** | 0.18** | 1.00 | |
12. I am satisfied with how teachers use digital technologies to provide feedback | 0.19** | 0.27** | 0.26** | 0.14** | 0.29** | 0.18** | 0.18** | 0.33** | 0.21** | .018** | 0.29** | 1.00 |
Table
5 presents the correlation between digital skills and specific variables. Several significant (albeit weak) correlations between different engagement variables in TEL, that include the use of IT and digital skills, were identified. For example, higher digital skills were associated with engaging in digital creation (r = 0.26,
p < 0.01), taking initiative with IT use (r = 0.23, p < 0.01), and satisfaction with teachers use of IT (r = 0.19, p < 0.01). It is noteworthy that students who expressed a general increase in engagement when using digital technologies also reported using digital technologies to support learning (r = 0.40, p < 0.01), that they took more initiative with technology (r = 0.38, p < 0.01), that they could concentrate more easily (r = 0.37, p < 0.01), and that they wanted the school to use more digital technologies (r = 0.49, p < 0.01).
Since digitalisation is the norm in many schools today, it is essential that schools not only focus on motivating their students but also ensure that students have the digital skills to succeed. Having hypothesised that students with high levels of digital skills engage more in TEL than students with low levels, our findings reveal that engagement in TEL is indeed related to students’ level of digital skills, thus supporting the second hypothesis.
Using the disengagement indicators, we then explored the second relationship in H2 (that students with low levels of digital skills disengaged more from TEL than students with high levels of digital skills) (Table
6).
Table 6Correlation between digital skills and disengagement in TEL
Disengagement in technology-enhanced learning | 140 | 3.51 | 0.59 | 270 | 3.44 | 0.59 | 0.83 | 0.21 |
The results of the t-test reveal that there is no significant difference between students reporting low levels (M = 3.51, SD = 0.59) and high levels (M = 3.44, SD = 0.59) of digital skills in terms of disengagement in TEL (t(410) = 0.83, p > 0.05). To complement this analysis, we conducted a bivariate correlation test.
Table
7 reveals that variables reflecting disengagement in TEL correlated weakly with low and high levels of digital skills. Two items of disengagement were related to high levels of digital skills: (i) “We are more knowledgeable about IT than the teacher, and are therefore left to decide how technologies should be used for learning” (r = 0.21,
p < 0.01), indicating that students with high levels of digital skills ask for the teacher’s knowledge and support when learning with technologies, but might not always receive it; (ii) “It upsets me that one student often does more than the rest in a group” (r = 0.13, p < 0.01), which indicates that when working with technologies, some students do the work on behalf of the others in a group, which affects engagement negatively for both groups. Thus, considering the results of the t-test and the fact that the correlations were weak, we H2 was only partly verified, as we did not find that low levels of digital skills are associated with disengagement in TEL.
Table 7Correlation between digital skills and disengagement in TEL
Students’ reported digital skills | 1.00 | | | | | | | |
Using IT makes me hand in my homework late | −0.10* | 1.00 | | | | | | |
I feel frustrated because information is constantly changing | −0.15** | 0.18** | 1.00 | | | | | |
I feel frustrated by unclear expectations | −0.12* | 0.22** | 0.41** | 1.00 | | | | |
I don’t want to read everything from a laptop | −0.12* | 0.11* | 0.10* | 0.19** | 1.00 | | | |
I don’t want to write everything using a laptop | −.018** | 0.13** | 0.16** | 0.15** | 0.56** | 1.00 | | |
We are more knowledgeable about IT than the teacher, and are therefore left to decide how technologies should be used for learning | 0.21** | 0.13** | 0.16** | 0.12* | 0.09 | 0.09 | 1.00 | |
It upsets me that one student often does more than the rest in a group | 0.13** | −0.05 | 0.02 | 0.03 | 0.14** | 0.08 | .021** | 1.00 |
4.3 Hypothesis 3 (H3)
Finally, we hypothesised that student engagement and disengagement in TEL could predict student grades. We conducted a stepwise regression analysis with combined engagement and disengagement variables as independent factors. Before commencing the regression analysis, the homoscedasticity and normal distribution were checked to confirm that the regression analysis was appropriate.
The final model could explain 17% of the variance in the students’ final grades (F(4, 409) = 7.99,
p < 0.001, R
2 = 0.17, R
2 Adjusted = 0.15). The following predictors were identified:
1.)My absenteeism would decrease if I could attend class from elsewhere (β = −0.19, p < 0.01),
2.)Working alone decreases my engagement (β = −0.15, p < 0.01),
3.)It upsets me that one student often does more than the rest in the group (β = 0.10, p < 0.01),
4.)The most capable student does the work on behalf of the group (β = 0.11, p < 0.05),
5.)I routinely search the Internet (β = 0.10, p < 0.05)
6.)I don’t want to read everything from a laptop (β = 0.14, p < 0.01)
7.)My teacher takes part within applications that I use (β = 0.12, p < 0.05),
8.)I use IT to support my learning (β = 0.19, p < 0.01),
9.)I often switch between playing games/browsing/social media/streaming YouTube (or similar) and learning in class (β = −0.10, p < 0.05),
10.)Autonomously switching between different technologies when doing schoolwork increases my engagement (β = −0.11, p < 0.05).
In light of the second hypothesis, we examined the structural relationships between student grades and student engagement and disengagement in TEL, and hypothesised that this engagement and disengagement could predict student grades. The best regression model could explain a total of 17% of the variance in the student’s final grades, and identified ten significant predictors (five engagement variables and five disengagement variables). Thus, H3 is supported, since variables reflecting both engagement and disengagement in TEL can predict students’ final grades.