Abstract
This article presents a methodology for modelling the development of self-regulated learning skills in the context of computer-based learning environments using a combination of tracing techniques. The user-modelling techniques combine statistical and computational approaches to assess skill acquisition, practice, and refinement with the MetaHistoReasoning tool, a single-agent system that supports inquiry-based learning in the domain of history. Data were collected from twenty-two undergraduate students during a 4-h session where user interactions were logged by the system. A logistic regression model predicted user performance in relation to a skill categorization task with 75 % accuracy. The manner in which users apply the skills that are acquired is then assessed through a rule-based reasoning system that allows the pedagogical agent to adapt instruction. The results show that the model allows the agent to detect instances when skills are inappropriately applied as well as what type of goal that is pursued by students. We discuss the implications of these user-modelling techniques in terms of sequencing instructional content and using the tutoring agent to deliver several types of discourse moves in order to enhance learning.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aleven, V., Roll, I., McLaren, B. M., & Koedinger, K. R. (2010). Automated, unobtrusive, action-by-action assessment of self-regulation during learning with an intelligent tutoring system. Educational Psychologist, 45(4), 224–233.
Alexander, P. A., Dinsmore, D. L., Parkinson, M. M., & Winters, F. I. (2011). Self-regulated learning in academic domains. In B. Zimmerman & D. Schunk (Eds.), Handbook of self-regulation of learning and performance. New York: Routledge.
Azevedo, R. (2005). Computer environments as metacognitive tools for enhancing learning. Educational Psychologist, 40(4), 193–197.
Azevedo, R., Johnson, A., Chauncey, A., & Graesser, A. (2011). Use of hypermedia to convey and assess self-regulated learning. In B. Zimmerman & D. Schunk (Eds.), Handbook of self-regulation of learning and performance (pp. 102–121). New York: Routledge.
Azevedo, R., Moos, D. C., Johnson, A. M., & Chauncey, A. D. (2010). Measuring cognitive and metacognitive regulatory processes during hypermedia learning: Issues and challenges. Educational Psychologist, 45(4), 210–223.
Baylor, A. L., & Kim, Y. (2005). Simulating instructional roles through pedagogical agents. International Journal of Artificial Intelligence in Education, 15(1), 95–115.
Brusilovsky, P. (2012). Adaptive hypermedia for education and training. In P. Durlach & A. Lesgold (Eds.), Adaptive Technologies for Training and Education (pp. 46–68). Cambridge: Cambridge University Press.
Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65, 245–281.
Baker, R. S. J. d., Corbett, A. T., & Aleven, V. (2008a). More accurate student modeling through contextual estimation of slip and guess probabilities in bayesian knowledge tracing. In Proceedings of the 9th international conference on intelligent tutoring systems (pp. 406–415). Montreal, Canada.
Baker, R. S. J. d., Corbett, A. T., & Aleven, V. (2008b). Improving contextual models of guessing and slipping with a truncated training set. In Proceedings of the 1st international conference on educational data mining (pp. 67–76). Montreal, Canada.
Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. Journal of the Learning Sciences, 13(1), 15–42.
D’Mello, S. K., Picard, R. W., & Graesser, A. C. (2007). Toward an affect-sensitive AutoTutor. IEEE Intelligent Systems, 22(4), 53–61.
Dietterich, T. G. (2000). Ensemble methods in machine learning. In J. Kittler & F. Roli (Eds.), First International Workshop on Multiple Classifier Systems (pp. 1–15). New York, NY: Springer Verlag.
Faragher, J. M. (2005). A great and noble scheme: The tragic story of the expulsion of the french Acadians from their American homeland. New York, NY: W. W. Norton.
Gong, Y., Beck, J. E., & Heffernan, N. T. (2011). How to construct more accurate student models: Comparing and optimizing knowledge tracing and performance factor analysis. International Journal of Artificial Intelligence in Education, 21(1–2), 27–46.
Graesser, A. C. (2011). Learning, thinking, and emoting with discourse technologies. American Psychologist, 66(8), 746–757.
Graesser, A. C., Jeon, M., & Dufty, D. (2008). Agent technologies designed to facilitate interactive knowledge construction. Discourse Processes, 45, 298–322.
Graesser, A. C., & McNamara, D. (2010). Self-regulated learning in learning environments with pedagogical agents that interact in natural language. Educational Psychologist, 45(4), 234–244.
Graesser, A. C., McNamara, D. S., & VanLehn, K. (2005). Scaffolding deep comprehension strategies through Point & Query, AutoTutor, and iSTART. Educational Psychologist, 40(4), 225–234.
Greene, J. A., Bolick, C. M., & Robertson, J. (2010). Fostering historical knowledge and thinking skills using hypermedia learning environments: The role of self-regulated learning. Computers & Education, 54(1), 230–243.
Gulz, A., Haake, M., Silvervarg, A., Sjödén, B., & Veletsianos, G. (2011). Building a social conversational pedagogical agent—Design challenges and methodological approaches. In D. Perez-Marin & I. Pascual-Nieto (Eds.), Conversational agents and natural language interaction: Techniques and effective practices. IGI Global.
Hämäläinen, W., & Vinni, M. (2010). Classifiers for educational technology. In C. Romero, S. Ventura, M. Pechenizkiy, & R. S. J. Baker (Eds.), Handbook on educational data mining (pp. 54–74). Chapman & Hall/CRC Data Mining and Knowledge Discovery Series, CRC Press.
Hicks, D., & Doolittle, P. E. (2009). Multimedia-based historical inquiry strategy instruction. Do size and form really matter? In J. Lee & A. Friedman (Eds.), Research on technology in social studies education. Greenwich, CT: Information Age Publishing.
Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42(2), 99–107.
Ingrand, F. F., Georgeff, M. P., & Rao, A. S. (1992). An architecture for real time reasoning and system control. IEEE Expert, 7(6), 34–44.
Jackson, G. T., Boonthum, C., & McNamara, D. S. (2010). The efficacy of iSTART extended practice: Low ability students catch up. In V. Aleven, J. Kay, & J. Mostow (Eds.), Proceedings of the Tenth International Conference on Intelligent Tutoring Systems (ITS 2010) (pp. 349–351). Berlin: Springer-Verlag.
Jacott, L., López-Manjón, A., & Carretero, M. (1998). Generating explanations in history. In J. F. Voss & M. Carretero (Eds.), Learning and reasoning in history. International review of history education (Vol. 2, pp. 294–306). London: Woburn.
Jacques, P. A., & Vicari, R. (2007). A BDI approach to infer students’ emotions in an intelligent learning environment. Computers & Education, 49(2), 360–384.
Kinnebrew, J., Biswas, G., Sulcer, B., & Taylor, R. (2013). Investigating self-regulated learning in teachable agent environments. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies. Berlin: Springer.
Krajcik, J. S., & Blumenfeld, P. (2006). Project-based learning. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (pp. 317–334). Cambridge: New York, NY.
Kuncheva, L. (2004). Classifier ensembles for changing environments. In Proceedings of the 5th international workshop on multiple classifier systems (pp. 1–15). Springer.
Lehman, B., D’Mello, S. K., & Graesser, A. C. (2012). Confusion and complex learning during interactions with computer learning environments. The Internet and Higher Education, 15(3), 184–194.
Levstik, L. S. (2011). Learning history. In R. E. Mayer & P. A. Alexander (Eds.), Handbook of research on learning and instruction (pp. 108–126). New York, NY: Routledge.
Liu, Z., Jin, H., & Fang, Z. (2006). Collaborative learning in e-learning based on multi-agent systems. In Proceedings of the 10th international conference on computer supported cooperative work in design (pp. 1–5). Nanjing.
Loyens, S. M. M., & Rikers, R. M. J. P. (2011). Instruction based on inquiry. In R. Mayer & P. Alexander (Eds.), Handbook of research on learning and instruction. Routledge Press.
Manouselis, N., Drachsler, H., Vuorikari, R., Hummel, H., & Koper, R. (2011). Recommender systems in technology enhanced learning. In L. Rokach, B. Shapira, P. Kantor, & F. Ricci, (Eds.), Recommender systems handbook (pp. 387–415). Springer.
McNamara, D. S., Boonthum, C., Levinstein, I. B., & Millis, K. (2007). Evaluating self-explanations in iSTART: Comparing word-based and LSA systems. In T. Landauer, D. S. McNamara, S. Dennis, & W. Kintsch (Eds.), LSA: A Road to Meaning (pp. 227–241). Mahwah, NJ: Erlbaum.
McNamara, D. S., O’Reilly, T., Best, R., & Ozuru, Y. (2006). Improving adolescent students’ reading comprehension with iSTART. Journal of Educational Computing Research, 34, 147–171.
Meijer, J., Veenman, M. V. J., & van Hout-Wolters, B. H. A. M. (2006). Metacognitive activities in text-studying and problem-solving: Development of a taxonomy. Educational Research and Evaluation, 12, 209–237.
Montanero, M., & Lucero, M. (2011). Causal discourse and the teaching of history. How do teachers explain historical causality? Instructional Science, 39(2), 109–136.
Nokes, J. D., Dole, J. A., & Hacker, D. J. (2007). Teaching high school students to use heuristics while reading historical texts. Journal of Educational Psychology, 99, 492–504.
O’Rourke, S. T., & Calvo, R. A. (2009). Analysing semantic flow in academic writing. In Proceedings of the international conference on artificial intelligence in education (AIED 2009). Brighton, UK.
Pavlik, P. I., Cen, H., & Koedinger, K. R. (2009). Performance factors analysis—A new alternative to knowledge tracing. In Proceedings of the 14th international conference on artificial intelligence in education (pp. 531–538). Brighton, UK.
Pintrich, P. R. (2000). Multiple goals, multiple pathways: The role of goal orientations in learning and achievement. Journal of Educational Psychology, 92, 544–555.
Pintrich, P. R. (2004). A conceptual framework for assessing motivation and self-regulated learning in college students. Educational Psychology Review, 16, 385–407.
Poitras, E. G., & Lajoie, S. P. (2012). The MetaHistoReasoning tool: A metacognitive tool for fostering domain-specific metacognitive activities in learning through historical inquiry. In A. Weerasinghe, R. Azevedo, I. Roll, & B. du Boulay (Eds.), Proceedings of the Workshop on Self-Regulated Learning in Educational Technologies: Supporting, Modelling, Evaluating, and Fostering Metacognition with Computer-Based Learning Environments (pp. 41–44). Greece: Crete.
Poitras, E. G., & Lajoie, S. P. (2013). A domain-specific account of self-regulated learning: The cognitive and metacognitive activities involved in learning through historical inquiry. Metacognition & Learning, 8(3), 213–234.
Poitras, E. G., Lajoie, S. P., & Hong, Y.-J. (2012). The design of technology-rich learning environments as metacognitive tools in history education. Instructional Science, 40(6), 1033–1061.
Polikar, R. (2006). Ensemble based systems in decision making. IEEE Circuits and Systems Magazine, 6, 21–45.
Rao, A. S., & Georgeff, M. P. (1995). BDI agents: From theory to practice. In Proceedings of the first international conference on multi-agent systems (ICMAS-95) (pp. 312–319). San Francisco, CA.
Reiser, B. J. (2004). Scaffolding complex learning: The mechanisms of structuring and problematizing student work. Journal of the Learning Sciences, 13(3), 273–304.
Renkl, A. (2010). Instruction based on examples. In R. E. Mayer & P. A. Alexander (Eds.), Handbook of research on learning and instruction. New York, NY: Routledge.
Renkl, A., Hilbert, T., & Schworm, S. (2009). Example-based learning in heuristic domains: A cognitive load theory account. Educational Psychology Review, 21(1), 67–78.
Rokach, L. (2010). Ensemble-based classifiers. Artificial Intelligence Review, 33(1–2), 1–39.
Roscoe, R. D., Segedy, J. R., Sulcer, B., Jeong, H., & Biswas, G. (2013). Shallow strategy development in a teachable agent environment designed to support self-regulated learning. Computers & Education, 62, 286–297.
Royer, J. M., Carlo, M. S., Dufresne, R. J., & Mestre, J. P. (1996). The assessment of levels of domain expertise while reading. Cognition and Instruction, 14(3), 373–408.
Schunk, D. H. (2005). Self-regulated learning: The educational legacy of Paul R. Pintrich. Educational Psychologist, 40, 85–94.
Shute, V. J., & Zapata-Rivera, D. (2012). Adaptive educational systems. In P. Durlach & A. Lesgold (Eds.), Adaptive Technologies for Training and Education (pp. 7–27). Cambridge: Cambridge University Press.
Sun, S., & Joy, M. (2005). An autonomous multi-agent solution for adaptive education. In Proceedings of the 23rd IASTED international multi-conference artificial intelligence and applications. Innsbruck, Austria.
Van der Stel, M., & Veenman, M. V. J. (2008). Relation between intellectual ability and meta-cognitive skillfulness as predictors of learning performance of young students performing tasks in different domains. Learning and Individual Differences, 18, 128–134.
Wiley, J., & Voss, J. F. (1999). Constructing arguments from multiple sources: Tasks that promote understanding and not just memory for text. Journal of Educational Psychology, 91, 301–311.
Winne, P. H. (2001). Self-regulated learning viewed from models of information processing. In B. Zimmerman & D. Schunk (Eds.), Self-regulated learning and academic achievement: Theoretical perspectives (pp. 153–189). Mahwah, NJ: Erlbaum.
Winne, P., & Hadwin, A. (1998). Studying as self-regulated learning. In D. Hacker, J. Dunlosky, & A. Graesser (Eds.), Metacognition in Educational Theory and Practice (pp. 279–306). Hillsdale, NJ: Erlbaum.
Winne, P., & Hadwin, A. (2008). The weave of motivation and self-regulated learning. In D. Schunk & B. Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and applications (pp. 297–314). Mahwah: Erlbaum.
Winne, P. H., & Perry, N. E. (2000). Measuring self-regulated learning. In M. Boekaerts, P. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 531–566). San Diego: Academic.
Winters, F. I., Greene, J. A., & Costich, C. M. (2008). Self-regulation of learning within computer-based learning environments: A critical analysis. Educational Psychology Review, 20, 429–444.
Zimmerman, B. J. (2000). Attainment of self-regulation: A social cognitive perspective. In M. Boekaerts, P. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation research, and applications (pp. 13–39). Orlando, FL: Academic Press.
Zimmerman, B. J. (2001). Achieving academic excellence: A self-regulatory perspective. In M. Ferrari (Ed.), The pursuit of excellence through education (pp. 85–110). Mahwah, NJ: Erlbaum.
Zimmerman, B. (2006). Development and adaptation of expertise: The role of self-regulatory processes and beliefs. In K. Ericsson, N. Charness, P. Feltovich, & R. Hoffman (Eds.), The Cambridge handbook of expertise and expert performance (pp. 705–722). New York: Cambridge University Press.
Zimmerman, B. (2008). Investigating self-regulation and motivation: Historical background, methodological developments, and future prospects. American Educational Research Journal, 45, 166–183.
Zimmerman, B., & Schunk, D. (Eds.). (2011). Handbook of self-regulation of learning and performance. New York, NY: Routledge.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix A
Sentence verification task
Original item
“8. Having received by the deputies, a letter from the French inhabitants of this province, dated August 1st, newstyle, in which they ask us to grant them priests, and the free and public exercise of their religion, and also that they shall not be obliged to take up arms in case of war, even should the province be attacked.”
Paraphrase item
“14. We hereby warn that by command and in the name of the King, that His Majesty does not want any of his subjects, who benefit from the rights and rewards of his Government, and who own habitations and lands in this region, shall be excused from a total commitment or from the expected duty to preserve themselves, their habitations, their lands, and the government under which they enjoy so many rewards.”
Meaning change item
“6. Accordingly, in order to execute the orders of his Majesty, we will send, at the earliest opportunity, some officers of the King to the French settlements, viz. to the Annapolis River, to Grand Pré, and to Chicanecto who shall swear to the oath of allegiance to the said King; and we command all those who wish to enjoy their possessions under the happy government of his Majesty, to present themselves in order to administer the oath of allegiance before the 15/26 October, which will be the last day granted by us.”
Distractor item
“4. It is His Excellency’s order that this decree [oath of allegiance] be published in a small subset of departments as soon as possible, that no person may pretend ignorance of the same.”
Inference verification task
Near inference, true item
“23. The dispute between the Acadians and Edward Cornwallis concerns the obligation, mentioned in the oath of allegiance, to take up arms in case of war, should the province be attacked.”
Near inference, false item
“5. Edward Cornwallis would allow the Acadians to refuse the oath should they consent to remain neutral and not fight alongside the French army.”
Far inference, true item
“3. The Acadians refused to swear to the oath of allegiance that was administered by the officers of the king at Annapolis River, Grand Pré, and Chicanecto.”
Far inference, false item
“15. The order to swear the oath of allegiance, which was published across all of the French settlements, was ignored by the French deputies.”
Appendix B
Elaborate argumentative essay
The Acadian Deportation of 1755 was a complicated event that involved many factors and motivations, thus it can be difficult to identify the most important cause. Nevertheless, I believe that the primary explanation for the Deportation was that the British suspected that the Acadians may not have been as neutral as initially thought and in fact were supporting the French in the war. This suspicion was based on the fact that the Acadian deputies and communities refused to swear the unconditional oath of allegiance, and furthermore, some of the Acadians were found to be secretly supporting the French. When the British first took control of the province of Nova Scotia, occupied by French descendents, they wanted the Acadians to take an unconditional oath, however the Acadians proposed a compromise, whereas they would not need to take up arms in the time of war; this was known as the unconditional oath. Later, when the 7 year war was beginning and there was a new Governor, Charles Lawrence, the British again asked the Acadians to take the unconditional oath, but the Acadians refused and wished to remain neutral in the war. In Doc 3 (Extract from a Letter of Governor Lawrence to Lords of Trade, 1754), the Governor argues that a large number of Acadians were working for the French and giving them supplies. I initially believed that Lawerence was only saying this so that he could get the approval of the Lords of Trade, however Doc 5 (The Deportation of the Acadians, 2011), confirms this fact by acknowledging that there were 270 Acadians found at the French Fort Beausejour when it fell. In conclusion, I believe that the best explanation for the Acadian Deportation was that the British suspected that the Acadians were loyal to the French and felt threatened by this. This suspicion was based off of the fact that the Acadians would not swear the unconditional oath of allegiance and that there were several instances where they were found to be in fact supporting the French.
Shallow argumentative essay
Given that it is a recurring theme in many of the documents concerning the Acadians, especially those written by persons in positions of authority at the time that the events were unfolding, I believe that one of the main reasons the Acadians were deported or imprisoned was due to their refusal to swear the oath of allegiance to the British monarchy.
Rights and permissions
About this article
Cite this article
Poitras, E.G., Lajoie, S.P. Developing an agent-based adaptive system for scaffolding self-regulated inquiry learning in history education. Education Tech Research Dev 62, 335–366 (2014). https://doi.org/10.1007/s11423-014-9338-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11423-014-9338-5