nach oben

Journal of Computing in Higher Education

Erschienen in:

25.07.2017

Exploring the potential of LMS log data as a proxy measure of student engagement

verfasst von: Curtis R. Henrie, Robert Bodily, Ross Larsen, Charles R. Graham

Erschienen in: Journal of Computing in Higher Education | Ausgabe 2/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

This study examines the relationship between log data of student activity in learning management systems and self-reported student engagement survey scores. Log data has the potential to serve as a meaningful proxy for survey scores. Should this be the case, log data could be used as a minimally disruptive and scalable approach to quickly identify who needs help, evaluate design, and personalize instruction. We correlated LMS log data variables to student engagement survey scores to study the relationship between these two sources of data. Overall, log data was not a statistically significant proxy measure of students’ self-reported cognitive and emotional engagement. Our results underscore the complexity of learning and the relationship between observed and reported cognitive and emotional states. Future educational research using log data will need to account for other factors that help explain trends in student engagement. Exploring the Potential of LMS Log Data as a Proxy Measure of Student Engagement.

Vorheriger Artikel The effects of online glossary quizzes and student autonomy on domain vocabulary learning in business law

Nächster Artikel Comparative effect of project-based learning and electronic project-based learning on the development and sustained development of english idiom knowledge

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nur mit Berechtigung zugänglich

Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006). Measuring cognitive and psychological engagement: Validation of the student engagement instrument. Journal of School Psychology, 44, 427–445. doi:10.1016/j.jsp.2006.04.002.CrossRef

Baker, R. S. J., Gowda, S. M., Wixon, M., Kalka, J., Wagner, A. Z., Aleven, V., Rossi, L. et al. (2012). Towards sensor-free affect detection in cognitive tutor algebra. In K. Yacef, O. Zaïane, H. Hershkovitz, M. Yudelson, & J. Stamper (Eds.), Proceedings of the 5th international conference on educational data mining (pp. 126–133), Chania. Retrieved April 27, 2015 from http://educationaldatamining.org/EDM2012/index.php?page=proceedings.

Baumeister, R. F., Vohs, K. D., DeWall, C. N., & Zhang, L. (2007). How emotion shapes behavior: Feedback, anticipation, and reflection, rather than direct causation. Personality and Social Psychology Review, 11(2), 167–203. doi:10.1177/1088868307301033.CrossRef

Beer, C., Clark, K., & Jones, D. (2010). Indicators of engagement. In Proceedings of ASCILITE 2010 (pp. 75–86), Sydney. Retrieved November 15, 2015 from http://ascilite.org.au/conferences/sydney10/procs/Beer-full.pdf.

Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57(1), 289–300. Retrieved from http://www.jstor.org.byui.idm.oclc.org/stable/2346101.

Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief. Washington, DC: SRI International. Retrieved from http://www.ed.gov/edblogs/technology/files/2012/03/edm-la-brief.pdf.

Bodily, R. G., Graham, C. R., & Bush, M. D. (2017). Online learner engagement: Opportunities and challenges with using data analytics. Educational Technology, 57(1), 10–18.

Borup, J., Graham, C., & Davies, R. (2013). The nature of parental interactions in an online charter school. American Journal of Distance Education, 27, 40–55. doi:10.1080/08923647.2013.754271.CrossRef

Cocea, M., & Weibelzahl, S. (2011). Disengagement detection in online learning: Validation studies and perspectives. IEEE Transactions on Learning Technologies, 4, 114–124. doi:10.1109/TLT.2010.14.CrossRef

Cooley, R., Mobasher, B., & Srivastava, J. (2013). Data preparation for mining world wide web browsing patterns. Knowledge and Information Systems, 1(1), 5–32. doi:10.1007/BF03325089.CrossRef

D’Mello, S. K., & Graesser, A. (2011). The half-life of cognitive-affective states during complex learning. Cognition and Emotion, 25, 1299–1308. doi:10.1080/02699931.2011.613668.CrossRef

D’Mello, S., & Graesser, A. (2012). AutoTutor and affective AutoTutor. ACM Transactions on Interactive Intelligent Systems, 2(4), 1–39. doi:10.1145/2395123.2395128.CrossRef

Dixson, M. D. (2010). Creating effective student engagement in online courses: What do students find engaging? Journal of the Scholarship of Teaching and Learning, 10(2), 1–13. Retrieved November 15, 2015 from http://josotl.indiana.edu/.

Drutsa, A., & Serdyukov, P. (2015). Future user engagement prediction and its application to improve the sensitivity of online experiments. In WWW 2015 (pp. 256–266), Florence. doi:10.1145/2736277.2741116.

Elliot, J. (2004). Multimethod approaches in educational research. International Journal of Disability, Development and Education, 51, 135–149. doi:10.1080/10349120410001687364.CrossRef

Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6), 304–317. doi:10.1504/IJTEL.2012.051816.CrossRef

Finn, J. D., & Owings, J. (2006). The adult lives of at-risk students: The roles of attainment and engagement in high school. Statistical Analysis Report (NCES 2006-328). Washington, DC: U.S. Department of Education, National Center for Education Statistics. Retrieved November 15, 2015 from http://eric.ed.gov/?id=ED491285.

Finn, J. D., & Zimmer, K. S. (2012). Student engagement: What is it? Why does it matter? In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 97–131). Boston, MA: Springer. doi:10.1007/978-1-4614-2018-7_5.CrossRef

Fredricks, J. A., Blumenfeld, P., Friedel, J., & Paris, A. (2005). School engagement. In K. A. Moore & L. Lippman (Eds.), What do children need to flourish? Conceptualizing and measuring indicators of positive development (pp. 305–321). New York, NY: Kluwer Academic/Plenum Press. doi:10.1007/0-387-23823-9_19.CrossRef

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74, 59–109. doi:10.3102/00346543074001059.CrossRef

Fredricks, J. A., & McColskey, W. (2012). The measurement of student engagement: A comparative analysis of various methods and student self-report instruments. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 763–782). Boston, MA: Springer. doi:10.1007/978-1-4614-2018-7_37.CrossRef

Fredricks, J., McColskey, W., Meli, J., Mordica, J., Montrosse, B., & Mooney, K. (2011). Measuring student engagement in upper elementary through high school: A description of 21 instruments. (Issues & Answers Report, REL 2011–No. 098). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southeast. Retrieved April 27, 2015 from http://ies.ed.gov/ncee/edlabs/regions/southeast/pdf/REL_2011098.pdf.

Fulmer, S. M., & Frijters, J. C. (2009). A review of self-report and alternative approaches in the measurement of student motivation. Educational Psychology Review, 21, 219–246. doi:10.1007/s10648-009-9107-x.CrossRef

Gobert, J. D., Baker, R. S., & Wixon, M. B. (2015). Operationalizing and detecting disengagement within online science microworlds. Educational Psychologist, 50, 43–57. doi:10.1080/00461520.2014.999919.CrossRef

Halverson, L.R., & Graham, C. R. (2015). Learner engagement in blended learning environments (Submitted for publication).

Halverson, L., Spring, K., Huyett, S., Henrie, C., & Graham, C. (2016). Blended learning research in higher education and K-12 settings. In J. M. Spector, B. B. Lockee, & M. D. Childress (Eds.), Learning, design, and technology: An international compendium of theory, research, practice and policy. Berlin: Springer. doi:10.1007/978-3-319-17727-4_31-1.

Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance, course completion rates, and student perception of the quality and frequency of interaction in a virtual high school. Distance Education, 34, 64–83. doi:10.1080/01587919.2013.770430.CrossRef

Heinemann, M. H. (2005). Teacher-student interaction and learning in online theological education. Part II: Additional theoretical frameworks. Christian Higher Education, 4(4), 277–297. doi:10.1080/15363750500182794.CrossRef

Henrie, C., Bodily, R., Manwaring, K., & Graham, C. (2015). Exploring intensive longitudinal measures of student engagement in blended learning. The International Review of Research in Open and Distributed Learning, 16(3), 131–155. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/2015.

Henrie, C. R., Halverson, L. R., & Graham, C. R. (2015b). Measuring student engagement in technology-mediated learning: A review. Computers & Education, 90, 36–53. doi:10.1016/j.compedu.2015.09.005.CrossRef

Henrie, C. R., Larsen, R., Manwaring, K., Halverson, L. R., & Graham, C. R. (2016). Validation of a longitudinal activity-level measure of student engagement (Submitted for publication).

Hollands, F., & Bakir, I. (2015). Efficiency of automated detectors of learner engagement and affect compared with traditional observation methods. New York, NY: Center for Benefit-Cost Studies of Education, Teachers College, Columbia University. Retrieved November 15, 2015 from http://cbcse.org/

Janosz, M. (2012). Part IV commentary: Outcomes of engagement and engagement as an outcome: Some consensus, divergences, and unanswered questions. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 695–703). Boston, MA: Springer. doi:10.1007/978-1-4614-2018-7_33.CrossRef

Jordan, K. (2014). Initial trends in enrollment and completion of massive open online courses. The International Review of Research in Open and Distance Learning, 15(1), 133–159. Retrieved April 27, 2015 from http://www.irrodl.org/index.php/irrodl/article/view/1651.

Kuh, G. D. (2001). Assessing what really matters to student learning: Inside the national survey of student engagement. Change: The Magazine of Higher Learning, 33(3), 10–17. Retrieved April 27, 2015 from http://www.jstor.org/stable/40165768.

Kuh, G. D., Kinzie, J., Buckley, J. A., Bridges, B. K., & Hayek, J. C. (2007). Piecing together the student success puzzle: Research, propositions, and recommendations. ASHE Higher Education Report, 32(5), 1–182. doi:10.1002/aehe.3205.CrossRef

Lawson, M. A., & Lawson, H. A. (2013). New conceptual frameworks for student engagement research, policy, and practice. Review of Educational Research, 83, 432–479. doi:10.3102/0034654313480891.CrossRef

Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54, 588–599. doi:10.1016/j.compedu.2009.09.008.CrossRef

Morris, L. V., Finnegan, C., & Wu, S.-S. (2005). Tracking student behavior, persistence, and achievement in online courses. The Internet and Higher Education, 8, 221–231. doi:10.1016/j.iheduc.2005.06.009.CrossRef

Oncu, S., & Cakir, H. (2011). Research in online learning environments: Priorities and methodologies. Computers & Education, 57, 1098–1108. doi:10.1016/j.compedu.2010.12.009.CrossRef

Park, S. (2015). Examining learning experience in two online courses using web logs and experience sampling method (ESM). In B. Hokanson, G. Clinton, & M. W. Tracey (Eds.), The design of learning experience (pp. 269–287). Boston, MA: Springer. doi:10.1007/978-3-319-16504-2_18.CrossRef

Prince, S. A., Adamo, K. B., Hamel, M. E., Hardt, J., Gorber, S. C., & Tremblay, M. (2008). A comparison of direct versus self-report measures for physical activity in adults: A systematic review. International Journal of Behavioral Nutrition and Physical Activity. doi:10.1186/1479-5868-5-56.

Reeve, J., & Tseng, C.-M. (2011). Agency as a fourth aspect of students’ engagement during learning activities. Contemporary Educational Psychology, 36, 257–267. doi:10.1016/j.cedpsych.2011.05.002.CrossRef

Reschly, A. L., & Christenson, S. L. (2012). Jingle, jangle, and conceptual haziness: Evolution and future directions of the engagement construct. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 3–20). New York, NY: Springer. doi:10.1007/978-1-4614-2018-7_1.CrossRef

Russell, J., Ainley, M., & Frydenberg, E. (2005). Schooling issues digest: Student motivation and engagement (pp. 1–16), Canberra, AU. Retrieved April 27, 2015 from http://docs.education.gov.au

Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57, 1380–1400. doi:10.1177/0002764213498851.CrossRef

Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015). The challenges of defining and measuring student engagement. Educational Psychologist, 55, 1–13. doi:10.1080/00461520.2014.1002924.CrossRef

Skinner, E. A., Kindermann, T. A., & Furrer, C. J. (2009). A motivational perspective on engagement and disaffection: Conceptualization and assessment of children’s behavioral and emotional participation in academic activities in the classroom. Educational and Psychological Measurement, 69, 493–525. doi:10.1177/0013164408323233.CrossRef

Skinner, E. A., & Pitzer, J. R. (2012). Developmental dynamics of student engagement, coping, and everyday resilience. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 21–44). Boston, MA: Springer. doi:10.1007/978-1-4614-2018-7_2.CrossRef

Wang, Z., Bergin, C., & Bergin, D. A. (2014). Measuring engagement in fourth to twelfth grade classrooms: The classroom engagement inventory. School Psychology Quarterly, 29, 517–535. doi:10.1037/spq0000050.CrossRef

Watson, J., Pape, L., Murin, A., Gemin, B., & Vashaw, L. (2014). Keeping pace with K-12 digital learning. Retrieved April 27, 2015 from http://www.kpk12.com/reports/.

Titel: Exploring the potential of LMS log data as a proxy measure of student engagement
verfasst von: Curtis R. Henrie
Robert Bodily
Ross Larsen
Charles R. Graham
Publikationsdatum: 25.07.2017
Verlag: Springer US
Erschienen in: Journal of Computing in Higher Education / Ausgabe 2/2018
Print ISSN: 1042-1726
Elektronische ISSN: 1867-1233
DOI: https://doi.org/10.1007/s12528-017-9161-1

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 2/2018

The role of scaffold interactivity in supporting self-regulated learning in a community college online composition course

Transforming undergraduate biology learning with inquiry-based instruction

E-textbook as object and mediator: interactions between instructor and student activity systems

EFL medical students’ metacognitive strategy use for hypertext reading comprehension

Learning experience assessment of flipped courses

An application of corresponding fields model for understanding exclusion in online social networks