nach oben

Education and Information Technologies

Erschienen in:

01.12.2015

Data science in educational assessment

verfasst von: David C. Gibson, Mary E. Webb

Erschienen in: Education and Information Technologies | Ausgabe 4/2015

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

This article is the second of two articles in this special issue that were developed following discussions of the Assessment Working Group at EDUsummIT 2013. The article extends the analysis of assessments of collaborative problem solving (CPS) to examine the significance of the data concerning this complex assessment problem and then for educational assessment more broadly. The article discusses four measurement challenges of data science or ‘big data’ in educational assessments that are enabled by technology: 1. Dealing with change over time via time-based data. 2. How a digital performance space’s relationships interact with learner actions, communications and products. 3. How layers of interpretation are formed from translations of atomistic data into meaningful larger units suitable for making inferences about what someone knows and can do. 4. How to represent the dynamics of interactions between and among learners who are being assessed by their interactions with each other as well as with digital resources and agents in digital performance spaces. Because of the movement from paper-based tests to online learning, and in order to make progress on these challenges, the authors call for the restructuring of training of the next generation of researchers and psychometricians to specialize in data science in technology enabled assessments.

Vorheriger Artikel Technology enhanced assessment in complex collaborative settings

Nächster Artikel Computational thinking in compulsory education: Towards an agenda for research and practice

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Al-diban, S., & Ifenthaler, D. (2011). Comparison of two analysis approaches for measuring externalized mental models. Educational Technology and Society, 14(2), 16–30.

Aldrich, C. (2005). Learning by doing : the essential guide to simulations, computer games, and pedagogy in e-learning and other educational experiences. San Francisco: Jossey-Bass.

Bar-Yam, Y. (1997). Dynamics of complex systems. Reading: Addison-Wesley.MATH

Bates, D., & Watts, D. (1988). Nonlinear regression analysis and its applications. John Wiley & Sons (Vol. 32). doi:10.2307/2289810.

Bennett, R. (2010). Cognitively based assessment of, for, and as learning (CBAL): a preliminary theory of action for summative and formative assessment. Measurement: Interdisciplinary Research and Perspective, 8(2–3), 70–91. Retrieved from http://www.tandfonline.com/doi/abs/10.1080/15366367.2010.508686..

Black, P., Harrison, C., Hodgen, J., Marshall, B., & Serret, N. (2010). Validity in teachers’ summative assessments. Assessment in Education: Principles, Policy & Practice, 17(2), 215–232.

Bollen, J., Van de Sompel, H., Hagberg, A., Bettencourt, L., Chute, R., Rodriguez, M. A., & Balakireva, L. (2009). Clickstream data yields high-resolution maps of science. PloS One, 4, e4803. doi:10.1371/journal.pone.0004803.

Borgatti, S. P., & Halgin, D. S. (2011). On network theory. Organization Science, 22, 1168–1181. doi:10.1287/orsc.1100.0641.CrossRef

Brandes, U., & Erlebach, T. (2005). Network Analysis: Methodological Foundations. Lecture Notes in Computer Science (Vol. 3418). doi:10.1007/b106453.

Choi, Y., Rupp, A., Gushta, M., & Sweet, S. (2010). Modeling learning trajectories with epistemic network analysis: An investigation of a novel analytic method for learning progressions in epistemic games. In National Council on Measurement in Education (pp. 1–39).

Clarke-Midura, J., & Dede, C. (2010). Assessment, technology, and change. Journal of Research in Teacher Education, 42(3), 309–328.CrossRef

Clarke-Midura, J., Code, J., Dede, C., Mayrath, M., & Zap, N. (2012). Thinking outside the bubble: Virtual performance assessments for measuring complex learning. In Technology-based assessments for 21st Century skills: Theoretical and practical implications from modern research (pp. 125–148). Charlotte: Information Age Publishers.

Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302. doi:10.1037/h0040957.CrossRef

De Freitas, S. (2014). Education in computer generated environments. London: Routledge.

DeFreitas, S., Gibson, D., & Morgan, J. (2015). Will MOOCS transform teaching and learning in higher education? British Journal of Educational Technology, in press.

Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007). Grit: perseverance and passion for long-term goals. Journal of Personality and Social Psychology, 92(6), 1087–1101. doi:10.1037/0022-3514.92.6.1087.CrossRef

Gibson, D. (2012). Game changers for transforming learning environments. In F. Miller (Ed.), Transforming Learning Environments: Strategies to Shape the Next Generation (Advances in Educational Administration, Volume 16) (pp. 215 – 235). Emerald Group Publishing Ltd. doi:10.1108/S1479-3660(2012)0000016014

Gibson, D., & Clarke-Midura, J. (2013). Some Psychometric and Design Implications of Game-Based Learning Analytics. In D. Ifenthaler, J. Spector, P. Isaias, & D. Sampson (Eds.), E-Learning Systems, Environments and Approaches: Theory and Implementation. London: Springer.

Gibson, D., & de Freitas, S. (2015). Exploratory analysis in learning analytics. Technology, Knowledge and Learning. doi:10.1007/s10758-015-9249-5.

Gibson, D., & Jakl, P. (2013). Data challenges of leveraging a simulation to assess learning. CA: West Lake Village. Retrieved from http://www.curveshift.com/images/Gibson_Jakl_data_challenges.pdf.

Gibson, D., & Knezek, G. (2011). Game Changers for Teacher Education. In P. Mishra & M. Koehler (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2011 (pp. 929–942). Chesapeake: VA: AACE.: AACE.

Gijbels, D., Dochy, F., Van den Bossche, P., & Segers, M. (2005). Effects of problem-based learning: a meta-analysis from the angle of assessment. Review of Educational Research, 75, 27–61.CrossRef

Han, J., Cheng, H., Xin, D., & Yan, X. (2007). Frequent pattern mining: current status and future directions. Data Mining and Knowledge Discovery, 15(1), 55–86. doi:10.1007/s10618-006-0059-1.MathSciNetCrossRef

IBM. (2015). Big Data. Retrieved from http://www-01.ibm.com/software/au/data/bigdata/.

Ifenthaler, D., Eseryel, D., Ge, X., & Ge, X. (2012). In D. Ifenthaler & D. Eseryel (Eds.), Assessment in game-based learning. Lobdon: Springer.CrossRef

Jonassen, D. H. (1997). Instructional design models for well-structured and ill-structured problem-solving learning outcomes. Educational Technology Research and Development, 45, 65–90. doi:10.1007/BF02299613.CrossRef

Khan, S. (2011). Khan Academy. Educational Technology. Retrieved from http://www.khanacademy.org/.

Kline, P. (1998). The new psychometrics: Science, psychology, and measurement. London: Routledge.

Kopainsky, B., Pirnay-dummer, P., & Alessi, S. M. (2010). Automated assessment of learner’s understanding in complex dynamic systems. In 28th International Conference of the System Dynamics Society (pp. 1–39). Seoul.

Kozleski, E., Gibson, D., & Hynds, A. (2012). Changing complex educational systems: Frameworks for collaborative social justice leadership. In C. Gersti-Pepin & J. Aiken (Eds.), Defining social justice leadership in a global context (pp. 263–286). Charlotte: Information Age Publishing.

Mansell, W., James, M., & the Assessment Reform Group. (2009). Assessment in schools. Fit for purpose? A Commentary by the Teaching and Learning Research Programme. London. Retrieved from http://www.tlrp.org/pub/documents/assessment.pdf.

Margetts, H., & Sutcliffe, D. (2013). Addressing the policy challenges and opportunities of “big data”. Policy and Internet, 5(2), 139–146. doi:10.1002/1944-2866.POI326.CrossRef

Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13–23.CrossRef

Mislevy, R., Steinberg, L., & Almond, R. (1999). Evidence-centered assessment design. Educational Testing Service. Retrieved from http://www.education.umd.edu/EDMS/mislevy/papers/ECD_overview.html

Mislevy, R., Steinberg, L., & Almond, R. (2003a). On the structure of educational assessments. Russell the Journal of the Bertrand Russell Archives, 1(1), 3–62.

Mislevy, R., Wilson, M., Ercikan, K., & Chudowsky, N. (2003). Psychometric Principles in Student Assessment. In T. Kellaghan & D. Stufflebeam (Eds.), International Handbook of Educational Evaluation (pp. 489–531). Kluwer Int.

Newell, A., & Simon, H. (1972). Human problem solving. Englewood Cliffs: Prentice-Hall.

Patton, M. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York: The Guilford Press.

Pirnay-Dummer, P., Ifenthaler, D., & Spector, M. (2010). Highly integrated model assessment technology and tools. Educational Technology Research and Development, 58(1), 3–18.CrossRef

PISA. (2013). Draft collaborative problem solving framework. OECD: Organization for Economic and Community Development.

Prensky, M. (2001). Digital game-based learning. New York: McGraw-Hill.

Psychometrics. (2014). In American Heritage Dictionary. Houghton-Mifflin Company

Quellmalz, E., Timms, M., & Schneider, S. (2009). Assessment of Student Learning in Science Simulations and Games. DC: National Research Council.

Quellmalz, E., Timms, M., Buckley, B., Davenport, J., Loveland, M., & Silberglitt, M. (2012). 21st century dynamic assessment. In M. Mayrath, J. Clarke-Midura, & D. Robinson (Eds.), Technology-based assessments for 21st Century skills: Theoretical and practical implications from modern research (pp. 55–90). Charlotte: Information Age Publishers.

Scheffel, M., Niemann, K., & Leony, D. (2012). Key action extraction for learning analytics. 21st Century Learning (pp.320–333). Retrieved from http://link.springer.com/chapter/10.1007/978-3-642-33263-0_25.

Schmidt, M., & Lipson, H. (2009). Symbolic regression of implicit equations. Genetic Programming Theory and Practice, 7, 73–85.

Shaffer, D., Hatfield, D., Svarovsky, G., Nash, P., Nulty, A., Bagley, E., & Mislevy, R. (2009). Epistemic network analysis: a prototype for 21st-century assessment of learning. International Journal of Learning and Media, 1(2), 33–53. Retrieved from http://www.mitpressjournals.org/doi/abs/10.1162/ijlm.2009.0013.

Sporns, O. (2011). Networks of the brain. Cambridge: MIT Press.

Sterman, J. D. (1994). Learning in and about complex systems. System Dynamics Review, 10(2 / 3), 291.CrossRef

Stevens, R. (2006). Machine learning assessment systems for modeling patterns of student learning. In D. Gibson, C. Aldrich, & M. Prensky (Eds.), Games and simulations in online learning: Research & development frameworks (pp. 349–365). Hershey: Idea Group.

Stevens, R., & Palacio-Cayetano, J. (2003). Design and performance frameworks for constructing problem-solving simulations. Cell Biology Education, 2(Fall), 162–179.CrossRef

Stevens, R., Johnson, D., & Soller, A. (2005). Probabilities and predictions: modeling the development of scientific problem-solving skills. Cell Biology Education, 4(1), 42–57. doi:10.1187/cbe.04-03-0036.CrossRef

Stiggins, R. J. (1995). Assessment Literacy for the 21st Century. Phi Delta Kappan, 77.

Sugihara, G., May, R., Ye, H., Hsieh, C., Deyle, E., Fogarty, M., & Munch, S. (2012). Detecting causality in complex ecosystems. Science (New York, N.Y.), 338(6106), 496–500. doi:10.1126/science.1227079.CrossRef

Tashakkori, A., & Teddlie, . (2003). Handbook of Mixed Methods in Social & Behavioral Research. SAGE. Retrieved from http://books.google.com/books?id=F8BFOM8DCKoC&pgis=1.

Thagard, P. (2010). How brains make mental models. Retrieved from cogsci.uwaterloo.ca/Articles/Thagard.brains-models.2010.pdf.

Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning Analytics Dashboard Applications. American Behavioral Scientist. doi:10.1177/0002764213479363.

Voogt, J., & Knezek, G. (2008). International Handbook of Information Technology in Primary and Secondary Education. New York: Springer.CrossRef

Webb, M., & Gibson, D. (2015). Technology enhanced assessment in complex collaborative settings. Education and Information Technologies. doi:10.1007/s10639-015-9413-5.

Webb, M., Gibson, D., & Forkosh-Baruch, A. (2013). Challenges for information technology supporting educational assessment. Journal of Computer Assisted Learning, 29(5), 451–462. doi:10.1111/jcal.12033.CrossRef

Titel: Data science in educational assessment
verfasst von: David C. Gibson
Mary E. Webb
Publikationsdatum: 01.12.2015
Verlag: Springer US
Erschienen in: Education and Information Technologies / Ausgabe 4/2015
Print ISSN: 1360-2357
Elektronische ISSN: 1573-7608
DOI: https://doi.org/10.1007/s10639-015-9411-7

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Weitere Artikel der Ausgabe 4/2015

Digital citizenship within global contexts

Technology enhanced assessment in complex collaborative settings

Research-informed strategies to address educational challenges in a digitally networked world

A model driven framework to address challenges in a mobile learning environment

Teachers’ professional development for ICT integration: Towards a reciprocal relationship between research and practice

Computational thinking in compulsory education: Towards an agenda for research and practice