Skip to main content
SpringerLink
Log in

Using Bayesian Networks to Manage Uncertainty in Student Modeling

  • Published:
User Modeling and User-Adapted Interaction Aims and scope Submit manuscript

Abstract

When a tutoring system aims to provide students with interactive help, it needs to know what knowledge the student has and what goals the student is currently trying to achieve. That is, it must do both assessment and plan recognition. These modeling tasks involve a high level of uncertainty when students are allowed to follow various lines of reasoning and are not required to show all their reasoning explicitly. We use Bayesian networks as a comprehensive, sound formalism to handle this uncertainty. Using Bayesian networks, we have devised the probabilistic student models for Andes, a tutoring system for Newtonian physics whose philosophy is to maximize student initiative and freedom during the pedagogical interaction. Andes’ models provide long-term knowledge assessment, plan recognition, and prediction of students’ actions during problem solving, as well as assessment of students’ knowledge and understanding as students read and explain worked out examples. In this paper, we describe the basic mechanisms that allow Andes’ student models to soundly perform assessment and plan recognition, as well as the Bayesian network solutions to issues that arose in scaling up the model to a full-scale, field evaluated application. We also summarize the results of several evaluations of Andes which provide evidence on the accuracy of its student models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Albacete, P. and Van Lehn, K.: 2000, The conceptual helper: An intelligent tutoring system for teaching fundamental physics concepts. Proceedings of Intelligent Tutoring Systems, 5th International Conference, ITS2000, Montreal, Canada, Lecture Notes in Computer Science 1839, Springer, pp. 564-573.

  • Albrecht, D.W., Zukerman, I. and Nicholson, A. E.: 1999, Bayesian Models for Keyhole Plan Recognition in an Adventure Game. User Modeling and User-Adapted Interaction, 8(1-2), 5-47.

    Article  Google Scholar 

  • Anderson, J. R., Corbett, A. T., Koedinger, K. R. and Pelletier, R.: 1995, Cognitive Tutors: Lessons Learned. The Journal of the Learning Sciences, 4(2), 167-207.

    Article  Google Scholar 

  • Bauer, M.: 1995, A Dempster-Shafer approach to modeling agents preferences in plan recognition. User Modeling and User-Adapted Interaction, 5(3-4), 317-348.

    Article  Google Scholar 

  • Breese, J., Goldman, R. and Wellman, P.: 1994, Introduction to the special section on knowledge-based construction of probabilistic and decision models. IEEE Transactions on Systems, Man, and Cybernetics, 24, pp. 1577-1579.

    Google Scholar 

  • Bunt, A., Conati, C., Hugget, M. and Muldner, K.: 2001, On Improving the Effectiveness of Open Learning Environments through Tailored Support for Exploration. Proceedings of AIED 2001, 10thWorld Conference of Artificial Intelligence and Education, SanAntonio, TX, U.S.A., pp. 365-376.

  • Calistri-Yeh, R. J.: 1991, An {A*} approach to robust plan recognition for intelligent interfaces. N. G. Bourbakis (ed.): Applications of Learning and Planning Methods. World Scientific Publishing Co., pp. 227-251.

  • Carberrry, S.: 2001, Techniques for Plan Recognition. User Modeling and User-Adapted Interaction, 11, 31-48.

    Article  Google Scholar 

  • Carberry, S.: 1990, Incorporating default inferences into plan recognition. Proceedings of the Eighth National Conference on Artificial Intelligence, Menlo Park, CA, MIT Press, pp. 471-478.

    Google Scholar 

  • Charniak, E. and Goldman, R. P.: 1992, A Bayesian model of plan recognition. Artificial Intelligence, 64, 53-79.

    Article  Google Scholar 

  • Chi, M. T. H.: in press, Self-Explaining: The dual process of generating inferences and repairing mental models. Advances in Instructional Psychology.

  • Conati, C. and Carenini, G.: 2001, Generating Tailored Examples to Support Learning via Self-explanation. Proceedings of IJCAI’01, 17th International Joint Conference on Artificial Intelligence, Seattle, WA, U.S.A., pp. 1301-1306.

  • Conati, C. and VanLehn, K.: 1996a, POLA: A student modeling framework for probabilistic on-line assessment of problem solving performance. Proceedings of UM-96, 5th International Conference on User Modeling, Kailua-Kona, Hawaii, U.S.A., UserModeling, Inc., pp. 75-82.

    Google Scholar 

  • Conati, C. and VanLehn, K.: 1996b, Probabilistic plan recognition for cognitive apprenticeship. Proceedings of the 18th Annual Meeting of the Cognitive Science Society, San Diego, CA. U.S.A., Erlbaum, pp. 403-408.

    Google Scholar 

  • Conati, C., Larkin, J., VanLehn, K.: 1997, A computer framework to support self-explanation. In duBouley B, Mizoguchi R. (eds) Artificial Intelligence in Education: Proceedings of the 8th World WNF, IOS Press, Ohmsha, pp. 279-286.

    Google Scholar 

  • Conati, C. and VanLehn, K.: 2000, Toward Computer-based Support of Meta-cognitive Skills: A Computational Framework to Coach Self-Explanation. International Journal of Artificial Intelligence in Education, 11(4), 389-415.

    Google Scholar 

  • Corbett, A., McLaughlin, M. and Scarpinatto, K. C.: 2000, Modeling Student Knowledge: Cognitive Tutors in High School and College. User Modeling and User-Adapted Interaction, 10(2-3), pp. 81-108.

    Article  Google Scholar 

  • Corbett, A. T. and Anderson, J. R.: 1995, Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4), 253-278.

    Article  Google Scholar 

  • Corbett, A. T. and Bhatnagar, A.: 1997, Student modeling in the ACT programming tutor: Adjusting a procedural learning model with declarative knowledge. User Modeling: Proceedings of the Sixth International Conference, UM97, Chia Laguna, Italy, Springer, Wien, New York, pp. 243-254.

    Google Scholar 

  • Dean, T. and Kanazawa, K.: 1989, A Model forReason ing about Persistence and Causation. Computational Intelligence, 5(3), 142-150.

    Google Scholar 

  • Gertner, A., Conati, C. and VanLehn, K.: 1998, Procedural help in Andes: Generating hints using a Bayesian network student model. Proceedings of the 15th National Conference on Artificial Intelligence, Madison, Wisconsin, U.S.A., pp. 106-111.

  • Gertner, A. and VanLehn, K.: 2000, Andes: a coached problem solving environment for physics. Proceedings of Intelligent Tutoring Systems, 5th International Conference, ITS2000, Montreal, Canada, Lecture Notes in Computer Science 1839, Springer, pp. 131-142.

    Google Scholar 

  • Henrion, M.: 1989, Some practical issues in constructing belief networks. Proceedings of the 3rd Conference on Uncertainty in Artificial Intelligence, Elsevier Scien ce, pp. 161-173.

  • Horvitz, E. and Barry, M.: 1995, Display of Information for Time-Critical Decision Making. Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence, Montreal, Canada, Morgan Kaufmann: San Francisco, pp. 296-305.

    Google Scholar 

  • Horvitz, E., Breese, J., Heckerman, D., Hovel, D. and Rommelse, R.: 1998, The Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users. Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence, Madison, WI, U.S.A., Morgan Kaufmann: San Francisco, pp. 256-265.

    Google Scholar 

  • Huber, M. J., Durfee, E. H. and Wellman, M. P.: 1994, The automated mapping of plans for plan recognition. Proceedings of the 10th Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann, pp. 344-351.

  • Jameson, A.: 1996, Numerical uncertainty management in user and student modeling: An overview of systems and issues. User Modeling and User-Adapted Interaction, 5(3-4), 193-251.

    Article  Google Scholar 

  • Just, M. and Carpenter, P.: 1986, The Psychology of Reading and Language Comprehension. Boston.

  • Katz, S., Lesgold, A., Eggan, G. and Gordin, M.: 1992, Modelling the student in Sherlock II. Journal of Artificial Intelligence in Education, 3(4), 495-518.

    Google Scholar 

  • Koedinger, K. and Anderson, J. R.: 1993, Reifying implicit planning ingeometry: Guidelines for model-based intelligent tutoring system design. S. P. Lajoie, and S. J. Derry (eds.). Computers as cognitive tools. Hillsdale, NJ, Lawrence Erlbaum Associates, pp. 15-46.

    Google Scholar 

  • Koedinger, K. R., Anderson, J. R., Hadley, W. H. and Mark, M. A.: 1995, Intelligent tutoring goes to school in the big city. Proceedings of the 7thWorld Conference on Artificial Intelligence and Education, Charlottesville, NC, AACE, pp. 421-428.

    Google Scholar 

  • Mahoney, S. M. and Laskey, K. B.: 1998, Constructing situation-specific Belief networks. Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence, S. Francisco, CA, U.S.A., Morgan Kaufmann, pp. 370-378.

    Google Scholar 

  • Martin, J. and VanLehn, K.: 1993, OLAE: Progress toward a multi-activity, Bayesian student modeller. Artificial Intelligence in Education, 1993: Proceedings of AI-ED 93, Charlottesville, VA, Association for the Advancement of Computing in Education, pp. 410-417.

    Google Scholar 

  • Martin, J. and VanLehn, K.: 1994, Discrete factor analysis: Learning hidden variables in Bayesian networks, LRDC, University of Pittsburgh: Technical report.

  • Martin, J. and VanLehn, K.: 1995, Student assessment using Bayesian nets. International Journal of Human-Computer Studies, 42, 575-591.

    Article  Google Scholar 

  • Mislevy, R.: 1995, Probability-based inference in cognitive diagnosis. P. Nichols, S. Chipman, and R. Brennan (eds.). Cognitive Diagnostic Assessment. Hillsdale, NJ., Erlbaum, pp. 43-71.

    Google Scholar 

  • Mislevy, R. J. and Gitomer, D. H.: 1996, The Role of Probability-Based Inference in an Intelligent Tutoring System. User Modeling and User-Adapted Interaction, 5(3-4), 253-282.

    Article  Google Scholar 

  • Murray, C. and VanLehn, K.: 2000, DT Tutor: A decision-theoretic dynamic approach for optimal selection of tutorial actions. Proceedings of Intelligent Tutoring Systems, 5th International Conference, ITS2000, Montreal, Canada, Lecture Notes in Computer Science 1839, Springer, pp. 153-162.

    Google Scholar 

  • Norman, D. A.: 1981, Categorization of action slips. Psychological Review, 88(1), 1-15.

    Article  Google Scholar 

  • Pearl, J.: 1988, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA, Morgan-Kaufmann.

    Google Scholar 

  • Petrushin, V. A., Sinitsa, K.M. and Zherdienko, V.: 1995, Probabilistic approach to adaptive student knowledge assessment: methodology and experiment. Artificial Intelligence in Education: Proceedings of AI-ED ‘95, Washington, DC, U.S.A., pp. 51-58.

  • Polk, T. A., VanLehn, K. and Kalp, D.: 1995, ASPM2: Progress toward the analysis of symbolic parameter models. P. D. Nichols, S. F. Chipman, and R. L. Brennan (eds.). Cognitively Diagnostic Assessment. Mahwah, NH, Erlbaum, pp. 127-141.

    Google Scholar 

  • Reggia, J. A. and D’Autrechy, C. L.: 1990, Parsimonious covering theory in cognitive diagnosis and adaptive instruction. N. Frederiksen, R. Glaser, A. Lesgold and M. G. Shafto (eds.). Diagnostic Monitoring of Skill and Knowledge Acquisition. Hillsdale, NJ, Lawrence Erlbaum Associates, pp. 191-216.

    Google Scholar 

  • Renkl, A.: 1997, Learning from worked-examples: A study on individual differences. Cognitive Science, 21(1), 1-30.

    Article  Google Scholar 

  • Reye, J.: 1998, Two-phase updating of student models based on dynamic belief networks. Proceedings of the 4th International Conference on Intelligent Tutoring Systems, ITS ‘98, San Antonio, TX, U.S.A., Lecture Notes in Computer Science 1452, Springer, pp. 274-283.

    Google Scholar 

  • Russell, S. and Norvig, P.: 1995, Artificial Intelligence: A Modern Approach. Los Altos, CA, Morgan-Kaufman.

    Google Scholar 

  • Schulze, K.G., Correll, D., Shelby, R.N., Wintersgill, M. C. and Gertner, A.: 1998, ACLIPS problem solver for Newtonian physics force problems. C. Giarratano, and G. Riley (eds.). Expert Systems Principles and Programming. Boston, MA, PWS Publishing Company.

    Google Scholar 

  • Schulze, K. G., Shelby, R. N., Treacy, D. J., Wintersgill, M. C., VanLehn, K. and Gertner, A.: 2000, Andes: An intelligent tutorforclassica l physics. The Journal of Electronic Publishing 6(1), The University of Michigan Press.

  • Shelby, R. N., Schulze, K. G., Treacy, D. J., Wintersgill, M. C. and VanLehn, K.: 2001, The Andes Intelligent Tutor: an Evaluation. Proceedings of the Physics Education Research Conference, Rochester, NY.

  • Shute, V. J.: 1995, SMART: Student Modeling Approach for Responsive Tutoring. User Modeling and User-Adapted Interaction, 5(1), 1-44.

    Article  Google Scholar 

  • Shute, V. J. and Glaser, R.: 1990, A large-scale evaluation of an intelligent discovery world. Interactive Learning Environments, 1, 51-76.

    Google Scholar 

  • Shute, V. J. and Psotka, J.: 1996, Intelligent Tutoring Systems: Past, Present and Future. D. Jonassen (ed.) Handbook of Research on Educational Communications and Technology. Scholastic Publications.

  • Singley, M. K. and Anderson, J. R.: 1989, Transfer of Cognitive Skill. Cambridge, MA., Harvard University Press.

    Google Scholar 

  • Tulving, E. and Thomson, D.M.: 1973, Encoding specificity and retrieval processes in episodic memory. Psychological Review, 80, 352-373.

    Google Scholar 

  • Van Mulken, S.: 1996, Reasoning about the user’s decoding of presentations in an intelligent multimedia presentation system. Proceedings of UM ‘96, 5th International Conference on User Modeling, Kailua Kona, HW, U.S.A., pp. 67-74.

  • VanLehn, K.: 1988, Student modeling. M. Polson and M. Richardson (eds.). Foundations of Intelligent Tutoring Systems. Hillsdale, NJ, Lawrence Erlbaum Associates, pp. 55-78.

    Google Scholar 

  • VanLehn, K.: 1996, Conceptual and meta learning during coached problem solving. Proceedings of Intelligent Tutoring Systems, 3rd International Conference, ITS ‘96, Montreal, Canada, Lecture Notes in Computer Science 1086, Springer, pp. 29-47.

    Google Scholar 

  • VanLehn, K., Freedman, R., Jordan, P., Murray, C., Osan, R., Ringenberg, M., Rose, C. P., Shultze, K., Shelby, R., Treacy, D., Weinstein, A. and Wintersgill, M.: 2000, Fading and deepening: The next steps forAndes and othermodel-tracing tutors. Proceedings of Intelligent Tutoring Systems, 5th International Conference, ITS2000, Montreal, Canada, Lecture Notes in Computer Science 1839, Springer, pp. 474-483.

    Google Scholar 

  • VanLehn, K. and Martin, J.: 1998, Evaluation of an assessment system based on Bayesian student modeling. International Journal of Artificial Intelligence in Education, 8(2), 179-221.

    Google Scholar 

  • VanLehn, K. and Niu, Z. 2001, Bayesian student modeling, useri nterfaces and feedback: A sensitivity analysis. International Journal of Artificial Intelligence in Education, 12, 154-184.

    Google Scholar 

  • VanLehn, K., Niu, Z., Siler, S. and Gertner, A. S.: 1998, Student modeling from conventional test data: A Bayesian approach without priors. Proceedings of the 4th International Conference on Intelligent Tutoring Systems, ITS ‘98, San Antonio, TX, U.S.A., LectureNotes in Computer Science 1452, Springer Verlag, pp. 434-443.

    Google Scholar 

  • Zukerman, I. and Albrecht, D. V.: 2001, Predictive Statistical Models for User Modeling. User-Modeling and User-Adapted Interaction, 11(1-2), 5-18.

    Article  Google Scholar 

  • Zukerman, I., Albrecht, D. and Nicholson, A.: 1999, Predicting Users’ Requests on the WWW. Proceedings of UM’99, the 7th International Conference on User Modeling, Banff, Canada, Springer-Verlag, pp. 275-284.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of British Columbia, Vancouver, BC, V6T1Z4, Canada

    Cristina Conati

  2. The MITRE Corporation, 202 Burlington Road, Bedford, MA, 01730, USA

    Abigail Gertner

  3. Department of Computer Science and Learning and Research Development Center, University of Pittsburgh, Pittsburgh, PA, 15260, USA

    Kurt VanLehn

Authors
  1. Cristina Conati
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abigail Gertner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kurt VanLehn
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Conati, C., Gertner, A. & VanLehn, K. Using Bayesian Networks to Manage Uncertainty in Student Modeling. User Model User-Adap Inter 12, 371–417 (2002). https://doi.org/10.1023/A:1021258506583

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1021258506583

Search

Navigation