Abstract
This chapter serves three purposes. First, we briefly review knowledge representations to stress the implications of different knowledge types on instruction and assessment. Second, we describe a novel cognitive tool, DNA (Decompose, Network, Assess), designed to aid knowledge elicitation and organization for instruction — specifically geared to increase the efficiency of creating the domain model used within intelligent instructional systems. Third, we present an exploratory test of the tool’s efficacy. Three statistical experts used DNA to explicate their knowledge related to measures of central tendency in statistics. DNA was able to effectively elicit relevant information, commensurate with a benchmark system, generating a starting curriculum upon which to build instruction, and did so in hours compared to months for conventional elicitation procedures.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Almond, R., Steinberg, L. & Mislevy, R. (in press). A four-process architecture for assessment delivery, with connections to assessment design. Journal of Technology, Learning, and Assessment.
Anderson, J. R. (1987). Skill acquisition: Compilation of weak-method problem solutions. Psychological Review, 94, 192–210.
Anderson, J. R. (1988). The expert module. In M. C. Polson & J. J. Richardson (Eds.), Foundations of intelligent tutoring systems (pp. 21–50 ). Hillsdale, NJ: Lawrence Erlbaum Associates.
Anderson, J. R. (1993). Rules of the mind. Hillsdale, NJ: Lawrence Erlbaum Associates.
Chipman, S., Shalin, V., & Schraagen, J. (Eds.) (2000). Cognitive Task Analysis. Hillsdale, NJ: Erlbaum Associates.
Collins, A. M., & Loftus, E. F. (1975). A spreading activation theory of semantic processing. Psychological Review, 82, 407–428.
Collins, A. M., & Quillian, M. R. (1969). Retrieval time from semantic memory. Journal of Verbal Learning and Verbal Behavior, 8, 240–248.
Gagné, R. (1985). The Conditions of Learning and Theory of Instruction. Holt, Rinehart, and Winston. New York.
Gordon, S. E., Schmierer, K. A., & Gill, R. T. (1993). Conceptual graph analysis: Knowledge acquisition for instructional system design. Human Factors, 35, 459–481.
Hubert, L., Arabie, P. & Meulman, J. (2001). Combinatorial data analysis: Optimization by dynamic programming (Vol. 1 ). Philadelphia, PA: SIAM/ Monograph on Discrete mathematics and applications.
Johnson-Laird, P., N. (1983). Mental models. Cambridge, MA: Harvard University Press.
Lajoie, S. P., & Derry, S. J. (1993). Computers as cognitive tools. Hillsdale, NJ: Lawrence Erlbaum Associates.
Merrill, M. D. (1994). Instructional Design Theory. Englewood Cliffs: Educational Technology Publications.
Merrill, M. D. (2000). Knowledge objects and mental models. In D. A. Wiley (Ed.), The Instructional Use of Learning Objects: Online Version. Retrieved March 6, 2002, from the World Wide Web: http://reusability.org/read/chapters/merrill.docf
Minstrell, J. (2000). Student thinking and related instruction: Creating a facet-based learning environment. In J. Pellegrino, L. Jones, & K. Mitchell (Eds.) Grading the Nation’s Report Card: Research for the Evaluation of NAEP. Committee on the Evaluation of NAEP, Board on Testing and Assessment. Washington, DC: National Academy Press.
Mislevy, R. J., Almond, R. G., Yan, D., & Steinberg, L. S. ( 2000, March). Bayes nets in educational assessment: Where do the numbers come from? (CSE Technical Report 518). Retrieved from
Mislevy, R. J., Steinberg, L. S., & Almond, R. G. ( 1999, January). On the roles of task model variables in assessment design (CSE Technical Report 500). Retrieved from
Mislevy, R. J., Steinberg, L. S., Almond, R. G., Haertel, G. D., & Penuel, W. R. ( 2001, February). Leverage points for improving educational assessment (CSE Technical Report 534). Retrieved from http://www.cse.ucla.edu/CRESST/Reports/newTR534.pdf
Murray, T. (1998). Authoring Knowledge Based Tutors: Tools for Content, Instructional Strategy, Student Model, and Interface Design. Journal of the Learning Sciences (Special Issue on Authoring Tools for Interactive Learning Environments), 7, No. 1, pp. 5–64.
Polson, M. C., & Richardson, J. J. (1988). Foundations of Intelligent Tutoring Systems. Hillsdale, NJ: Lawrence Erlbaum Associates.
Schank, R. (1999). Dynamic memory revisited. Cambridge, England: Cambridge University Press.
Shraagen, J. M. C., Chipman, S. E., Shute, V. J., Annett, J., Strub, M., Sheppard, C., Ruisseau, J. Y., & Graff, N. (1997). State-of-the-art review of cognitive task analysis techniques. Deliverable Report of RSG.27 on Cognitive Task Analysis NATO Defense Research Group (Panel 8/RSG. 27 ). TNO Human Factors Research Institute Group: Information Processing.
Shute, V. J. (1995). SMART: Student modeling approach for responsive tutoring. User Modeling and User -Adapted Interaction, 5, 1–44.
Shute, V. J., & Gluck, K. A. (1994). Stat Lady: Descriptive Statistics Module. [Unpublished computer program]. Brooks Air Force Base, TX: Armstrong Laboratory.
Shute, V. J., & Psotka, J. (1996). Intelligent tutoring systems: Past, present, and future. In D. Jonassen (Ed.), Handbook of Research on Educational Communications and Technology, (pp. 570–600 ). New York, NY: Macmillan.
Shute, V. J. & Towle, B. (in press). Adaptive e-Learning. Paper to appear in special issue of Educational Psychologist, in honor of Dr. Richard Snow.
Shute, V. J., Gawlick, L. A., & Lefort, N. K. (1996). Stat Lady (DS-2 module) [Unpublished computer program]. Brooks Air Force Base, TX: Armstrong Laboratory.
Shute, V. J., Torreano, L. A., and Willis, R. E. (1999). Exploratory test of an automated knowledge elicitation and organization tool. International Journal of AI and Education, 10 (3–4), 365–384.
Shute, V. J. & Torreano, L., & Willis, R. (2000). DNA: Towards an automated knowledge elicitation and organization tool. In S. P. Lajoie (Ed.) Computers as Cognitive Tools, Volume 2. Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 309–335.
Sleeman, D. H., & Brown, J. S. (1982). Intelligent tutoring systems. London, England: Academic Press.
VanLehn, K. (1990). Mind bugs: The origins of procedural misconceptions. Cambridge, MA: MIT Press.
White, B. Y. & J. R. Frederiksen (1987). Qualitative models and intelligent learning environments. In R. Lawler & M. Yazdani (Eds.), AI and education, (pp. 281–305 ). Norwood, NJ: Ablex Publishing.
Williams, K. E. (1993). The development of an automated cognitive task analysis and modeling processfor intelligent tutoring system development Contract final report on N00014–97-J-5–1500. ManpowerPersonnel and Training Program, Office of Naval Research.
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Shute, V.J., Torreano, L.A. (2003). Formative Evaluation of an Automated Knowledge Elicitation and Organization Tool. In: Murray, T., Blessing, S.B., Ainsworth, S. (eds) Authoring Tools for Advanced Technology Learning Environments. Springer, Dordrecht. https://doi.org/10.1007/978-94-017-0819-7_6
Download citation
DOI: https://doi.org/10.1007/978-94-017-0819-7_6
Publisher Name: Springer, Dordrecht
Print ISBN: 978-90-481-6499-8
Online ISBN: 978-94-017-0819-7
eBook Packages: Springer Book Archive