nach oben

Journal of Computers in Education

Erschienen in:

01.12.2015

Mining learner–system interaction data: implications for modeling learner behaviors and improving overlay models

verfasst von: Tenzin Doleck, Ram B. Basnet, Eric G. Poitras, Susanne P. Lajoie

Erschienen in: Journal of Computers in Education | Ausgabe 4/2015

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

A growing body of empirical evidence suggests that the adaptive capabilities of computer-based learning environments can be improved through the use of educational data mining techniques. Log-file trace data provides a wealth of information about learner behaviors that can be captured, monitored, and mined for the purposes of discovering new knowledge and detecting patterns of interest. This study aims to leverage these analytical techniques to mine learner behaviors in relation to both diagnostic reasoning processes and outcomes in BioWorld, a computer-based learning environment that support learners to practice solving medical problems and receive formative feedback. In doing so, hidden Markov models are used to model behavioral indicators of proficiency during problem solving, while an ensemble of text classification algorithms are applied to written case summaries that learners’ write as an outcome of solving a case in BioWorld. The application of these algorithms characterize learner behaviors at different phases of problem solving which provides corroborating evidence in support of where revisions can be made to provide design guidelines of the system. We conclude by discussing the instructional design and pedagogical implications for the novice–expert overlay system in BioWorld, and how the findings inform the delivery of feedback to learners by highlighting similarities and differences between the novice and expert trajectory toward solving problems.

Vorheriger Artikel A systematic literature review of design-based research from 2004 to 2013

Nächster Artikel Seamless flipped learning: a mobile technology-enhanced flipped classroom with effective learning strategies

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Aggarwal, C. C., & Zhai, C. (2012). A survey of text classification algorithms. In C. C. Aggarwal & C. Zhai (Eds.), Mining text data (pp. 77–128). New York: Springer.CrossRef

Anderson, J., Corbett, A., Koedinger, K., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), 167–207.CrossRef

Anderson, J., & Gluck, K. (2001). What role do cognitive architectures play in intelligent tutoring systems? In D. Klahr & S. Carver (Eds.), Cognition & Instruction: 25 years of progress (pp. 227–262). Mahwah, NJ: Lawrence Erlbaum.

Baker, R. S. J. D. (2007). Modeling and understanding students’ off-task behavior in intelligent tutoring systems. Proceedings of ACM CHI 2007: computer-human interaction, 1059–1068.

Baker, R. S. J. D., Corbett, A. T., Roll, I., Koedinger, K. R., Aleven, V., Cocea, M., et al. (2013). Modeling and studying gaming the system with educational data mining. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 97–116). New York: Springer.CrossRef

Baker, R. S. J. D., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3–17.

Beal, C. R., Mitra, S., & Cohen, P. (2007a). Modeling learning patterns of students with a tutoring system using hidden Markov models. In R. Lucking, K. R. Koedinger, & J. Greer (Eds.), Artificial intelligence in education (pp. 238–245). Amsterdam: IOS Press.

Beal, C. R., Walles, R., Arroyo, I., & Woolf, B. P. (2007b). On-line tutoring for math achievement testing: A controlled evaluation. Journal of Interactive Online Learning, 6(1), 43–55.

Biswas, G., Jeong, H., Kinnebrew, J., Sulcer, B., & Roscoe, R. (2010). Measuring self-regulated learning skills through social interactions in a teachable agent environment. Research and Practice in Technology-Enhanced Learning, 5(2), 123–152.CrossRef

Biswas, G., Kinnebrew, J. S., & Segedy, J. R. (2014). Using a Cognitive/Metacognitive Task Model to analyze Students Learning Behaviors. In Proceedings of the 16th International conference on human-computer interaction. Crete, Greece.

Blikstein, P. (2011). Using learning analytics to assess students’ behavior in open-ended programming tasks. Proceedings of the 1st international conference on learning analytics and knowledge (lak ‘11) (pp. 110–116). New York,: ACM.CrossRef

Bloom, B. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13(6), 4–16. doi:10.3102/0013189x013006004.CrossRef

Chang, C., & Lin, C. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3), 1–27.CrossRef

Chapelle, O., & Vapnik, V. (2000). Model selection for support vector machines. Advances in neural information processing systems (Vol. 12). Cambridge, MA: MIT Press.

Cocea, M., & Weibelzahl, S. (2009). Log file analysis for disengagement detection in e-Learning environments. User Model and User-Adapted Interaction, 19(4), 341–385.CrossRef

Collins, A. (2006). Cognitive apprenticeship. In K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 47–60). NY: Cambridge University Press.

Craig, S., D’Mello, S., Witherspoon, A., & Graesser, A. (2008). Emote aloud during learning with Autotutor: Applying the facial action coding system to cognitive–affective states during learning. Cognition and Emotion, 22(5), 777–788.CrossRef

Dodds, P., & Fletcher, J. D. (2004). Opportunities for new “smart” learning environments enabled by next-generation web capabilities. Journal of Educational Multimedia and Hypermedia, 13(4), 391–404.

Doleck, T., Jarrell, A., Chaouachi, M., Poitras, E., & Lajoie, S. (in prep). A tale of three cases: Examining accuracy, efficiency, and process differences in diagnosing virtual patient cases.

Doleck, T., Basnet, R. B., Poitras, E., & Lajoie, S. (2014). BioWorldParser: A suite of parsers for leveraging educational data mining techniques. In Proceedings of 2nd IEEE International Conference on MOOCs, Innovation & Technology in Education (MITE), (pp. 32–35), India: IEEE. doi: 10.1109/MITE.2014.7020236

Doleck, T., Basnet, R. B., Poitras, E., & Lajoie, S. (2014). Augmenting the novice-expert overlay model in an intelligent tutoring system: Using confidence-weighted linear classifiers. In Proceedings of IEEE International Conference on Computational Intelligence & Computing Research (IEEE ICCIC), (pp. 87–90), India: IEEE

Doleck, T., Jarrell, A., Poitras, E., & Lajoie, S. (2015). Towards investigating performance differences in clinical reasoning in a technology rich learning environment. In Conati, C., Heffernan, N., Mitrovic, A., Verdejo, M. F. (Eds.), Artificial intelligence in education (pp. 567–570). Lugano: Springer International Publishing. doi: 10.1007/978-3-319-19773-9_63

Durlach, P. J. & Ray, J. M. (2011). Designing adaptive instructional environments: Insights from empirical evidence. Army Research Institute Technical Report 1297. U.S. Army Research Institute, Arlington, VA.

Fitzgerald, J., Wolf, F., Davis, W., Barclay, M., Bozynski, M., Chamberlain, K., et al. (1994). A preliminary study of the impact of case specificity on computer-based assessment of medical student clinical performance. Evaluation and the Health Professions, 17(3), 307–321. doi:10.1177/016327879401700304.CrossRef

Frank, E., & Bouckaert, R. R. (2006). Naive Bayes for text classification with unbalanced classes. In Proceedings of 10th European Conference on Principles and Practice of Knowledge Discovery in Databases, pp. 503-510. Berlin: Springer.

Gauthier, G., & Lajoie, P. S. (2014). Do expert clinical teachers have a shared understanding of what constitutes competent case-based reasoning? Instructional Science, 42(4), 579–594.CrossRef

Gauthier, G., Lajoie, P. S., Naismith, L., & Wiseman, J. (2008). Using expert decision maps to promote reflection and self-assessment in medical case-based instruction. In Proceedings of Workshop on the Assessment and Feedback in Ill-Defined Domains at ITS, Montréal, Canada.

Graesser, A. C., Chipman, P., Haynes, B. C., & Olney, A. (2005). AutoTutor: An intelligent tutoring system with mixed-initiative dialogue. IEEE Transactions in Education, 48, 612–618.CrossRef

Graesser, A., Lu, S., Jackson, G., Mitchell, H., Ventura, M., Olney, A., & Louwerse, M. (2004). Autotutor: A tutor with dialogue in natural language. Behavior Research Methods, Instruments, & Computers, 36(2), 180–192.CrossRef

Hall, M. A. (1999). Correlation-based feature subset selection for machine learning. (doctoral dissertation). Department of Computer Science, University of Waikato, New Zealand.

Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009). The WEKA data mining software: An update. SIGKDD Explorations, 11(1), 10–18.CrossRef

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.CrossRef

Jeong, H., Gupta, A., Roscoe, R., Wagster, J., Biswas, G., & Schwartz, D. (2008). Using hidden Markov models to characterize student behaviors in learning-by-teaching environments. In Proceedings of Intelligent Tutoring Systems: Vol. 5091. Lecture Notes in Computer Science (pp. 614–625). Montreal: Springer.

Joachims, T. (1998). Text Categorization with support vector machines: Learning with many relevant features. In Proceedings of ECML-98, 10th European Conference on Machine Learning.

Kibriya, A. M., Frank, E., Pfahringer, B., & Holmes, G. (2004). Multinomial naïve bayes for text categorization revisited. In G. I. Webb & X. Yu (Eds.), Advances in artificial intelligence (pp. 488–499). Berlin, Heidelberg: Springer.

Koedinger, K., & Corbett, A. (2006). Cognitive tutors: Technology bringing learning science to the classroom. In K. Sawyer (Ed.), The cambridge handbook of the learning sciences (pp. 61–78). Cambridge: Cambridge University Press.

Lajoie, S. P. (2003). Transitions and trajectories for studies of expertise. Educational Researcher, 32(8), 21–25.CrossRef

Lajoie, S. P. (2009). Developing professional expertise with a cognitive apprenticeship model: Examples from avionics and medicine. In K. A. Ericsson (Ed.), Development of professional expertise: Toward measurement of expert performance and design of optimal learning environments (pp. 61–83). Cambridge: Cambridge University Press.CrossRef

Lajoie, S. P., Naismith, L., Hong, Y. J., Poitras, E., Cruz-Panesso, I., Ranellucci, J., et al. (2013). Technology rich tools to support self-regulated learning and performance in medicine. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 229–242). New York: Springer.CrossRef

Lajoie, S. P., Poitras, E. G., Doleck, T., & Jarrell, A. (2015). Modeling metacognitive activities in medical problem-solving with bioworld. In A. Peña-Ayala (Ed.), Metacognition: Fundaments, applications, and trends (pp. 323–343). New York: Springer Series: Intelligent Systems Reference Library.

Matsuda, N., Yarzebinski, E., Keiser, V., Raizada, R., William, W. C., Stylianides, G. J., & Koedinger, K. R. (2013). Cognitive anatomy of tutor learning: Lessons learned with SimStudent. Journal of Educational Psychology, 105(4), 1152–1163.CrossRef

McNamara, D. S. (2007). IIS: A marriage of computational linguistics, psychology, and educational technologies. In D. Wilson & G. Sutcliffe (Eds.), Proceedings of the twentieth international florida artificial intelligence research society conference (pp. 15–20). Menlo Park, California: The AAAI Press.

Merrill, M. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43–59. doi:10.1007/bf02505024.CrossRef

Mitrovic, A. (2003). An intelligent SQL tutor on the web. International Journal of Artificial Intelligence in Education, 13(2), 173–197.

Naismith, L. (2013). Examining motivational and emotional influences on medical students’ attention to feedback in a technology-rich environment for learning clinical reasoning (Unpublished doctoral dissertation). Montreal: McGill University.

Naismith, L., & Lajoie, S. P. (2010). Using expert models to provide feedback on clinical reasoning skills. In proceedings of the International Conference on Intelligent Tutoring Systems (pp. 242–244).

Park, O. C., & Lee, J. (2004). Adaptive instructional systems. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (2nd ed., pp. 651–684). Mahwah, NJ: Lawrence Erlbaum.

Platt, J. C. (1998). A fast algorithm for training support vector machines. Technical Report MSR-TR-98-14.

Poitras, E. G., Lajoie, S. P., Doleck, T., & Jarrell, A. (in press). Subgroup discovery with user interaction data: An empirically guided approach to improving intelligent tutoring systems. Educational Technology & Society.

Poitras, E., Lajoie, S., & Hong, Y.-J. (2012). The design of technology-rich learning environments as metacognitive tools in history education. Instructional Science, 40(6), 1033–1061.CrossRef

Rabiner, L. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286. doi:10.1109/5.18626.CrossRef

Rabiner, L., & Juang, B. (1986). An introduction to hidden Markov models. IEEE ASSP Magazine, 3(1), 4–16.CrossRef

Romero, C., & Ventura, S. (2010). Educational data mining: A review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(6), 601–618.

Sebastiani, F. (2002). Machine learning in automated text categorization. ACM Computing Surveys, 34(1), 1–47.CrossRef

Shute, V. J., & Zapata-Rivera, D. (2008). Adaptive technologies. In J. M. Spector, D. Merrill, J. van Merriënboer, & M. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 277–294). New York, NY: Lawrence Erlbaum Associates, Taylor & Francis Group.

Shute, V. J., & Zapata-Rivera, D. (2012). Adaptive educational systems. In P. Durlach & A. Lesgold (Eds.), Adaptive technologies for training and education. New York, NY: Cambridge University Press.

van der Kleij, F., Eggen, T., Timmers, C., & Veldkamp, B. (2012). Effects of feedback in a computer-based assessment for learning. Computers & Education, 58(1), 263–272.CrossRef

van der Vleuten, C., & Swanson, D. (1990). Assessment of clinical skills with standardized patients: State of the art. Teaching and Learning in Medicine, 2(2), 58–76. doi:10.1080/10401339009539432.CrossRef

Vanlehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221.CrossRef

Vanlehn, K., Lynch, C., Schulze, K., Shapiro, J. A., Shelby, R., Taylor, L., et al. (2005). The Andes physics tutoring system: Lessons learned. International Journal of Artificial Intelligence in Education, 15, 147–204.

Zapata-Rivera, D., & Greer, J. (2000). Inspecting and visualizing distributed Bayesian student models. In Proceedings of Intelligent Tutoring Systems (pp. 544–553).

Titel: Mining learner–system interaction data: implications for modeling learner behaviors and improving overlay models
verfasst von: Tenzin Doleck
Ram B. Basnet
Eric G. Poitras
Susanne P. Lajoie
Publikationsdatum: 01.12.2015
Verlag: Springer Berlin Heidelberg
Erschienen in: Journal of Computers in Education / Ausgabe 4/2015
Print ISSN: 2197-9987
Elektronische ISSN: 2197-9995
DOI: https://doi.org/10.1007/s40692-015-0040-3

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 4/2015

Exploring the nature of teacher-student interaction in small-group discussions in a Chinese university setting

Digital badges in undergraduate composition courses: effects on intrinsic motivation

Use of serious games for creating awareness about social enterprises

Seamless flipped learning: a mobile technology-enhanced flipped classroom with effective learning strategies

A systematic literature review of design-based research from 2004 to 2013