Abstract
The measurement of learning engagement is a major research theme, both in the learning analytics community and the broader area of educational research. The complexity of conceptualizing as well as operationalizing the construct of engagement generates a wide range of instruments, such as self-report surveys, log data from technology-enhanced learning systems, think-aloud, and tests. In this empirical work, we investigate the alignment of behavioural traces of engagement with self-report measures and their impact on academic performance. The unique contribution of this study is the integration of temporal, behavioural, affective, and cognitive dimensions of engagement by combining digital traces at three different learning phases with self-report, formative as well as summative assessments. Using a two-step cluster analysis based on data from 1027 undergraduate students in a first-year 8-week statistics course, we identified four distinct temporal engagement patterns (i.e. nonactive, active before tutorial, active before quiz, and active before exams). Our analysis showed that early engagement (i.e. before tutorial) was significantly associated with course performance and self-report measures, while late engagement patterns had weaker correlations. This study shed further lights on a potential source of heterogeneity and collinearity in engagement measures (i.e. timing of engagement) that should be accounted for in learning analytics model. In order to design effective intervention, it is crucial to consider different profiles of learners based on their engagement patterns as well as the temporal relation between trace data, self-report, and academic performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aleven, V., McLaren, B., Roll, I., & Koedinger, K. (2004). Toward tutoring help seeking. In J. C. Lester, R. M. Vicari, & F. Paraguaçu (Eds.), Intelligent tutoring systems. ITS 2004 (Lecture notes in computer science) (Vol. 3220). Berlin, Heidelberg: Springer.
Aleven, V., McLaren, B. M., & Koedinger, K. R. (2006). Towards computer-based tutoring of help-seeking skills. In S. Karabenick & R. Newman (Eds.), Help seeking in academic settings: Goals, groups, and contexts (pp. 259–296). Mahwah, NJ: Erlbaum.
Aleven, V., Roll, I., McLaren, B. M., & Koedinger, K. R. (2016). Help helps, but only so much: Research on help seeking with intelligent tutoring systems. International Journal of Artificial Intelligence in Education, 26(1), 205–223.
Azevedo, R. (2015). Defining and measuring engagement and learning in science: Conceptual, theoretical, methodological, and analytical issues. Educational Psychologist, 50(1), 84–94.
Christenson, S. L., Reschly, A. L., & Wylie, C. (Eds.). (2006). Handbook of research on student engagement. New York: Springer.
Ifenthaler, D. (2012). Intelligent model-based feedback: Helping students to monitor their individual learning progress. In S. Graf, F. Lin, Kinshuk, & R. McGreal (Eds.), Intelligent and adaptive learning systems: Technology enhanced support for learners and teachers (pp. 88–100). Hershey, PA: IGI Global.
Ifenthaler, D. (2015). Learning analytics. In J. M. Spector (Ed.), The SAGE encyclopedia of educational technology (Vol. 2, pp. 447–451). Thousand Oaks, CA: Sage.
Ifenthaler, D., Gibson, D. C., & Zheng, L. (2018a). Attributes of engagement in challenge-based digital learning environments. In Proceedings of the 15th International Conference on Cognition and Exploratory Learning Technologies, CELDA 2018, pp. 225–232. Lisbon, Portugal: IADIS Press.
Ifenthaler, D., Gibson, D. C., & Zheng, L. (2018b). The Dynamics of Learning Engagement in Challenge-Based Online Learning, in Proceedings 2018 IEEE 18th International Conference on Advanced Learning in the Digital Age, ICALT 2018, pp. 178–182. Los Alamitos, California: IEEE.
Ifenthaler, D., Yau, J. Y.-K., & Mah, D.-K. (Eds.). (2019). Utilizing learning analytics to support study success. New York: Springer.
Martin, A. J. (2007). Examining a multidimensional model of student motivation and engagement using a construct validation approach. British Journal of Educational Psychology, 77(2), 413–440.
McLaren, B. M., van Gog, T., Ganoe, C., Karabinos, M., & Yaron, D. (2016). The efficiency of worked examples compared to erroneous examples, tutored problem solving, and problem solving in computer-based learning environments. Computers in Human Behavior, 55, 87–99.
Nguyen, Q., Tempelaar, D.T., Rienties, B., & Giesbers, B. (2016). What learning analytics based prediction models tell us about feedback preferences of students. In R. Amirault, & Y. Visser (Eds.), e-Learners and their data, Part 1: Conceptual, research, and exploratory perspectives. Quarterly Review of Distance Education. 17(3).
Rienties, B., Cross, S., & Zdrahal, Z. (2017). Implementing a learning analytics intervention and evaluation framework: What works? In B. Kei Daniel (Ed.), Big data and learning analytics: Current theory and practice in higher education (pp. 147–166). Cham, Switzerland: Springer International Publishing.
Rienties, B., Tempelaar, D., Nguyen, Q., & Littlejohn, A. (2019). Unpacking the intertemporal impact of self-regulation in a blended mathematics environment. Computers in Human Behavior. https://doi.org/10.1016/j.chb.2019.07.007
Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015). The challenges of defining and measuring student engagement in science. Educational Psychologist, 50(1), 1–13.
Tempelaar, D., Rienties, B., & Nguyen, Q. (2017). Towards actionable learning analytics using dispositions. IEEE Transactions on Learning Technologies, 10(1), 6–16.
Tempelaar, D., Rienties, B., & Nguyen, Q. (2018). Investigating learning strategies in a dispositional learning analytics context: The case of worked examples. In Proceedings of the International Conference on Learning Analytics and Knowledge, Sydney, Australia, March 2018 (LAK’18), pp. 201–205.
Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: Learning analytics in a data-rich context. Computers in Human Behavior, 47, 157–167.
Tempelaar, D. T., Rienties, B., Mittelmeier, J., & Nguyen, Q. (2018). Student profiling in a dispositional learning analytics application using formative assessment. Computers in Human Behavior, 78, 408–420.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Tempelaar, D., Nguyen, Q., Rienties, B. (2020). Learning Analytics and the Measurement of Learning Engagement. In: Ifenthaler, D., Gibson, D. (eds) Adoption of Data Analytics in Higher Education Learning and Teaching. Advances in Analytics for Learning and Teaching. Springer, Cham. https://doi.org/10.1007/978-3-030-47392-1_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-47392-1_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-47391-4
Online ISBN: 978-3-030-47392-1
eBook Packages: EducationEducation (R0)