This chapter describes a new idea for the design and development of assessments for mental models using “flexible belief networks” (FBNs). The idea involves joining and extending two assessment approaches—evidence-centered design (ECD) and concept mapping (CM). ECD will be extended beyond single, static proficiency models to dynamic models of learning over time. CM will be extended to include belief networks, which may be accomplished by overlaying concept maps with Bayesian networks. Our goal is to derive a methodology to better assess mental models as they evolve over time, with valid inferences regarding both syntactic (structural) and semantic (conceptual) similarities to reference models. This work leverages the seminal research conducted in the area of assess- ing mental models by Norbert M. Seel.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Almond, R. G. (2007). Cognitive modeling to represent growth (learning) using Markov Decision Processes. Princeton, NJ: ETS Research Report.
Almond, R. G., & Mislevy, R. J. (1999). Graphical models and computerized adaptive testing, Applied Psychological Measurement, 23, 223-238.
Almond, R. G., Mislevy, R. J., Steinberg, L. S., Williamson, D. M., & Yan, D. (in press). Bayesian networks in educational assessment, manuscript in preparation to be published by Springer.
Bangert-Drowns, R. L., Kulik, C. C., Kulik, J. A., & Morgan, M. T. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61(2), 213-238.
Black, P., & Wiliam, D. (1998a). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-71.
Black, P., & Wiliam, D. (1998b). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2), 139-148.
Cimolino, L., Kay, J., & Miller, A. (2004). Concept mapping for eliciting verified personal ontologies, Special Issue on concepts and ontologies in WBES of the International Journal of Continuing Engineering Education and Lifelong Learning, 14(3), 212-228.
Clarke, S. (2001). Unlocking formative assessment—Practical strategies for enhancing pupils’ learning in the primary classroom. London: Hodder and Stoughton.
deKleer, J. & Brown, J. S. (1981). Mental models of physical mechanisms and their acquisition. In J. R. Anderson (Ed.), Cognitive Skills and their Acquisition. Hillsdale, NJ: Erlbaum.
Dimitrova, V. (2003). StyLE-OLM: Interactive open learner modelling, International Journal of Artificial Intelligence in Education, 13(1), 35-78.
Forbus, K., Carney, K., Harris, R., & Sherin, B. (2001). Modeling environment for middle-school students: A progress report, in Biswas, G. (Ed.): Papers from the 2001 Qualitative Reasoning Workshop, The Printing House, Stoughton, WI.
Gentner, D., & Stevens, A. (1983). Mental models. Hillsdale, NJ: Erlbaum.
Greenwald, A. G., & Banaji, M. R. (1995). Implicit social cognition: Attitudes, self-esteem, and stereotypes. Psychological Review, 102(1), 4-27.
Griffin, T., & Ohlsson, S. (2001). Beliefs versus knowledge: A necessary distinction for explaining, predicting, and assessing conceptual change. In J. D. Moore and K. Stenning (Eds.), proceedings of the Twenty-Third Annual Conference of the Cognitive Science Society (pp. 364-369). Mahwah, NJ: Lawrence Erlbaum.
Hoska, D. M. (1993). Motivating learners through CBI feedback: Developing a positive learner perspective. In V. Dempsey & G. C. Sales (Eds.), Interactive instruction and feedback (pp. 105-132). Englewood Cliffs, N.J.: Educational Technology Publications.
Ifenthaler, D., Pirnay-Dummer, P., & Seel, N. M. (2007). The role of cognitive learning strategies and intellectual abilities in mental model building processes. Technology, Instruction, Cognition and Learning, Special Issue on Applying Cognitive Modeling.
Ifenthaler, D., & Seel, N. M. (2005). The measurement of change. Learning-dependent progression of mental models. Technology, Instruction, Cognition and Learning, 2(4), 317-336.
Jeong, A. & Juong, S. (2007). Scaffolding collaborative argumentation in asynchronous discussions with message constraints and message labels. Computers & Education, 48, 427-445.
Johnson-Laird, P. N. (1983). Mental models: Towards a cognitive science of language, inference, and consciousness. Cambridge: Cambridge University Press.
Jonassen, D., Beissner, K., & Yacci, M. (1993). Structural knowledge techniques for representing, conveying, and acquiring structural knowledge. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers Hove & London.
Mayer, R. E. (1989). Models for understanding. Review of Educational Research, 59(1), 43-64.
Mayer, R. E., Moreno, R., Boire, M., & Vagge, S. (1999). Maximizing constructivist learning from multimedia communication by minimizing cognitive load. Journal of Educational Psychology, 91(4), 638-643.
Mislevy, R. J. (1994). Evidence and inference in educational assessment, Psychometrika, 12, pp. 341-369.
Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessment. Measurement: Interdisciplinary Research and Perspective, 1 (1), 3-62.
Novak, J. D. (1995). Concept mapping: A strategy for organizing knowledge. In S. M. Glynn, & R. Duit (Eds.), Learning Science in the Schools: Research Reforming Practice (pp. 229-245), Mahwah, NJ: Erlbaum Associates.
Novak, J. D., & Gowin, D. R. (1984). Learning how to learn. New York: Cambridge Press.
Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. San Mateo, CA: Kaufmann.
Seel, N. M. (1999). Educational diagnosis of mental models: Assessment problems and technology-based solutions. Journal of Structural Learning and Intelligent Systems, 14 (2), 153-185.
Seel, N. M. (2003). Model-centered learning and instruction. Technology, Instruction, Cognition and Learning, 1(1), 59-85.
Shute, V. J. (in press). Tensions, trends, tools, and technologies: Time for an educational sea change. To appear in C. Dwyer (Ed.), The future of assessment: Shaping teaching and learning. Mahwah, NJ: Erlbaum Associates.
Shute, V. J. (2007). Focus on formative feedback. ETS Research Report, RR-07-11 (pp. 1-55), Princeton, NJ.
Shute, V. J., Hansen, E. G., & Almond, R. G. (2007). An assessment for learning system called ACED: Designing for learning effectiveness and accessibility, ETS Research Report, RR-0726 (pp. 1-54), Princeton, NJ.
Shute, V. J., Ventura, M., Bauer, M. I., & Zapata-Rivera, D. (in press). Melding the power of serious games and embedded assessment to monitor and foster learning: Flow and grow. To appear in U. Ritterfeld, M. J. Cody, & P. Vorderer (Eds.), The social science of serious games: Theories and applications. Philadelphia, PA: Routledge/LEA.
Spector, J. M., Dennen, V. P., & Koszalka, T. (2006). Causal maps, mental models and assessing acquisition of expertise. Technology, Instruction, Cognition and Learning, 3, 167-183.
Spiro, R. J., Feltovich, P. J., Jacobson, M. J., & Coulson, R. L. (1991). Cognitive flexibility, constructivism, and hypertext: Random access instruction for advanced knowledge acquisition in ill-structured domains. Educational Technology, 24-33.
Sterman, J. D. (1994). Learning in and about complex systems. System Dynamics Review, 10(2-3), 291-330.
Stiggins, R. J. (2002). Assessment crisis: The absence of assessment for learning, Phi Delta Kappan Professional Journal, 83(10), pp. 758-765, 2002.
Vo, H. V., Poole, M. S., & Courtney, J. F. (2005). An empirical comparison of causal mapping approaches. In D. J. Armstrong (Ed.), Causal Mapping for Information Systems and Technology Research: Approaches, Advances and Illustrations (pp. 142-173), Battle Creek, MI: Idea Group Publishing.
Weinberger, A. & Mandl, H. (2003). Computer-mediated knowledge communication, Studies in Communication Sciences, 81-105.
White, B. & Frederiksen, J. (1985). Qualitative models and intelligent learning environments. In R. Lawler & M. Yazdani (Eds.), Artifical Intelligence and Education. Norwood, NJ: Ablex.
Zapata-Rivera, J. D., & Greer, J. (2004). Interacting with Bayesian student models. International Journal of Artificial Intelligence in Education, 14(2), 127-163.
Zapata-Rivera, J. D., Greer, J., & Cooke, J. (2000). An XML-based tool for building and using concept maps in education and training environments. Proceedings of the 8th Conference on Computers in Education, ICCE 2000, 755-762.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Shute, V.J., Zapata-Rivera, D. (2008). Using an Evidence-Based Approach to Assess Mental Models. In: Ifenthaler, D., Pirnay-Dummer, P., Spector, J.M. (eds) Understanding Models for Learning and Instruction. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-76898-4_2
Download citation
DOI: https://doi.org/10.1007/978-0-387-76898-4_2
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-76897-7
Online ISBN: 978-0-387-76898-4
eBook Packages: Humanities, Social Sciences and LawEducation (R0)