Abstract
Essays are an important measure of complex learning, but pronouns can confound an author’s intended meaning for both readers and text analysis software. This descriptive investigation considers the effect of pronouns on a computer-based text analysis approach, ALA-Reader, which uses students’ essays as the data source for deriving individual and group knowledge representations. Participants in an undergraduate business course (n = 45) completed an essay as part of the course final examination. The investigators edited the essays to replace the most common pronouns (their, it, and they) with the appropriate referent. The original unedited and the edited essays were processed with ALA-Reader using two different approaches, sentence and linear aggregate. These data were then analyzed using a Pathfinder network approach. The average group network similarity values comparing the original to the edited essays were large (i.e., about 90% overlap) but the linear aggregate approach obtained larger values than the sentence aggregate approach. The linear aggregate approach also provided a better measure of individual essay scores (e.g., r = 0.74 with composite rater scores). This data provides some support that the ALA-Reader linear approach is adequate for capturing group knowledge structure representations from essays. Further development of the ALA-Reader approach is warranted.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cernusca, D. (2007). A design-based research approach to the implementation and examination of a cognitive flexibility hypertext. Unpublished doctoral dissertation, University of Missouri, Columbia. Retrieved November 11, 2008, from http://edt.missouri.edu/Summer2007/Dissertation/CernuscaD-071907-D8036/research.pdf.
Chen, C. (1999). Visualizing semantic spaces and author co-citation networks in digital libraries. Information Processing and Management, 35, 401–420.
Clariana, R. B., & Koul, R. (2004). A computer-based approach for translating text into concept map-like representations. In A. J. Canas, J. D. Novak, & F. M. Gonzales (Eds.), Proceedings of the first international conference on concept mapping: Vol. 2. Concept maps: Theory, methodology, technology (pp. 131–134). Pensacola, FL: Institute for Human and Machine Cognition. Retrieved November 11, 2008, from http://cmc.ihmc.us/papers/cmc2004-045.Pdf.
Clariana, R. B., & Wallace, P. E. (2007). A computer-based approach for deriving and measuring individual and team knowledge structure from essay questions. Journal of Educational Computing Research, 37, 209–225.
Cooke, N. M. (1992). Predicting judgment time from measures of psychological proximity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 640–653.
Craik, K. (1943). The nature of explanation. Cambridge, UK: Cambridge University Press.
Goldsmith, T. E., Johnson, P. J., & Acton, W. H. (1991). Assessing knowledge structure. Journal of Educational Psychology, 83, 88–96.
Gonzalvo, P., Canas, J. J., & Bajo, M. (1994). Structural representations in knowledge acquisition. Journal of Educational Psychology, 86, 601–616.
Harper, M. E., Hoeft, R. M., Evans, A. W. III, & Jentsch, F. G. (2004). Scoring concepts maps: Can a practical method of scoring concept maps be used to assess trainee’s knowledge structures? Human Factors and Ergonomics Society Annual Meeting Proceedings, 48, 2599–2603.
Johnson, P. J., Goldsmith, T. E., & Teague, K. W. (1994). Locus of the predictive advantage in Pathfinder-based representations of classroom knowledge. Journal of Educational Psychology, 86, 617–626.
Johnson, T. E., O’Connor, D. L., Pirnay-Dummer, P. N., Ifenthaler, D., Spector, J. M., & Seel, N. (2006). Comparative study of mental model research methods: relationships among ACSMM, SMD, MITOCAR and DEEP methodologies. In A. J. Canas & J. D. Novak (Eds.), Proceedings of the second international conference on concept mapping: Vol. 1. Concept maps: Theory, methodology, technology (pp. 177–184). Pensacola, FL: Institute for Human and Machine Cognition. Retrieved November 11, 2008, from http://cmc.ihmc.us/cmc2006Papers/cmc2006-p177.pdf.
Johnson-Laird, P. N., Girotto, V., & Legrenzi, P. (1998). Mental models: A gentle guide for outsiders. Retrieved November 29, 2008, from http://www.si.umich.edu/ICOS/gentleintro.html.
KNOT. (1998). Knowledge network organizing tool. Retrieved June 11, 2004, from http://interlinkinc.net/.
Koul, R., Clariana, R. B., & Salehi, R. (2005). Comparing several human and computer-based methods for scoring concept maps and essays. Journal of Educational Computing Research, 32, 261–273.
Poindexter, M. T., & Clariana, R. B. (2006). The influence of relational and proposition-specific processing on structural knowledge and traditional learning outcomes. International Journal of Instructional Media, 33, 177–184.
Seel, N. M. (1999). Educational diagnosis of mental models: Assessment problems and technology-based solutions. Journal of Structural Learning and Intelligent Systems, 14, 153–185.
Shavelson, R. J. (1974). Some methods for examining content structure and cognitive structure in instruction. Educational Psychologist, 11, 110–122.
Taricani, E. M., & Clariana, R. B. (2003). Semantic map automated assessment techniques. Paper presented at the meeting of Association for Educational Communications and Technology (AECT), Anaheim, CA, October, 2003.
Taricani, E. M., & Clariana, R. B. (2006). A technique for automatically scoring open-ended concept maps. Educational Technology Research and Development, 54, 61–78.
Author information
Authors and Affiliations
Corresponding author
Additional information
This is an expanded version of the paper: Clariana, R. B., Wallace, P. E., & Godshalk, V. M. (2008). Deriving and measuring group knowledge structure via computer-based analysis of essay questions: the effects of controlling anaphoric reference. In Kinshuk, D. G. Sampson, J. M. Spector, P. Isaías, & D. Ifenthaler (Eds.), Proceedings of the IADIS international conference on cognition and exploratory learning in the digital age (88–95). Freiburg, Germany: International Association for Development of the Information Society.
Rights and permissions
About this article
Cite this article
Clariana, R.B., Wallace, P.E. & Godshalk, V.M. Deriving and measuring group knowledge structure from essays: The effects of anaphoric reference. Education Tech Research Dev 57, 725–737 (2009). https://doi.org/10.1007/s11423-009-9115-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11423-009-9115-z