Skip to main content
Erschienen in: Education and Information Technologies 3/2016

20.08.2014

Performance assessment in serious games: Compensating for the effects of randomness

verfasst von: Wim Westera

Erschienen in: Education and Information Technologies | Ausgabe 3/2016

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper is about performance assessment in serious games. We conceive serious gaming as a process of player-lead decision taking. Starting from combinatorics and item-response theory we provide an analytical model that makes explicit to what extent observed player performances (decisions) are blurred by chance processes (guessing behaviors). We found large effects both theoretically and practically. In two existing serious games random guess scores were found to explain up to 41 % of total scores. Monte Carlo simulation of random game play confirmed the substantial impact of randomness on performance. For valid performance assessments, be it in-game or post-game, the effects of randomness should be included to produce re-calibrated scores that can reasonably be interpreted as the players’ achievements.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Abt, C. (1970). Serious games. New York: Viking. Abt, C. (1970). Serious games. New York: Viking.
Zurück zum Zitat Aldrich, C. (2005). Learning by doing: the essential guide to simulations, computer games, and pedagogy E-learning and other educational experiences. San Francisco: Wiley. Aldrich, C. (2005). Learning by doing: the essential guide to simulations, computer games, and pedagogy E-learning and other educational experiences. San Francisco: Wiley.
Zurück zum Zitat Becker, K., & Parker, J. R. (2011). The guide to computer simulations and games. Indianapolis: Wiley. Becker, K., & Parker, J. R. (2011). The guide to computer simulations and games. Indianapolis: Wiley.
Zurück zum Zitat Bente, G., & Breuer, J. (2009). Making the implicit explicit: Embedded measurement in serious games. In U. Ritterfield, M. J. Cody, & P. Vorderer (Eds.), Serious Games: Mechanisms and Effects (pp. 322–343). New York: Routledge. Bente, G., & Breuer, J. (2009). Making the implicit explicit: Embedded measurement in serious games. In U. Ritterfield, M. J. Cody, & P. Vorderer (Eds.), Serious Games: Mechanisms and Effects (pp. 322–343). New York: Routledge.
Zurück zum Zitat Brennan, R. L. (2006). Educational measurement (4th ed.). Lanham: Rowman & Littlefield Publishers. Brennan, R. L. (2006). Educational measurement (4th ed.). Lanham: Rowman & Littlefield Publishers.
Zurück zum Zitat Chin, J., Dukes, R., & Gamson, W. (2009). Assessment in simulation and gaming: a review of the last 40 years. Simulation & Gaming, 40(4), 553–568.CrossRef Chin, J., Dukes, R., & Gamson, W. (2009). Assessment in simulation and gaming: a review of the last 40 years. Simulation & Gaming, 40(4), 553–568.CrossRef
Zurück zum Zitat Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic literature review of the empirical evidence on computer games and serious games. Computers and Education, 59, 661–686.CrossRef Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic literature review of the empirical evidence on computer games and serious games. Computers and Education, 59, 661–686.CrossRef
Zurück zum Zitat Dweck, C. S. (1986). Motivational processes affecting learning. American Psychologist, 41, 1040–1048.CrossRef Dweck, C. S. (1986). Motivational processes affecting learning. American Psychologist, 41, 1040–1048.CrossRef
Zurück zum Zitat Ebel, R. L., & Frisbie, D. A. (1991). Essentials of educational measurement (Vol. 5). New Jersey: Prentice Hall. Ebel, R. L., & Frisbie, D. A. (1991). Essentials of educational measurement (Vol. 5). New Jersey: Prentice Hall.
Zurück zum Zitat Farr, J. L., Hofmann, D. A., & Ringenbach, K. L. (1993). Goal orientation and action control theory: Implications for industrial and organizational psychology. In C. L. Cooper & I. T. Robertson (Eds.), International review of industrial and organizational psychology (pp. 193–232). New York: John Wiley. Farr, J. L., Hofmann, D. A., & Ringenbach, K. L. (1993). Goal orientation and action control theory: Implications for industrial and organizational psychology. In C. L. Cooper & I. T. Robertson (Eds.), International review of industrial and organizational psychology (pp. 193–232). New York: John Wiley.
Zurück zum Zitat Kolb, D. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs: Prentice Hall. Kolb, D. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs: Prentice Hall.
Zurück zum Zitat Redeker, C., Punie, Y., & Ferrari, A. (2012). eAssessment for 21st century learning and skills. In A. Ravenscroft, S. Lindsteadt, C. D. Kloos, & D. Hernandez-Leo (Eds.), 21st Century Learning for 21st Century Skills. Proceedings of the 7th European Conference on technology-enhanced learning EC-TEL, Saarbrücken, 2012 (pp. 292–305). Heidelberg: Springer. Redeker, C., Punie, Y., & Ferrari, A. (2012). eAssessment for 21st century learning and skills. In A. Ravenscroft, S. Lindsteadt, C. D. Kloos, & D. Hernandez-Leo (Eds.), 21st Century Learning for 21st Century Skills. Proceedings of the 7th European Conference on technology-enhanced learning EC-TEL, Saarbrücken, 2012 (pp. 292–305). Heidelberg: Springer.
Zurück zum Zitat Schank, R. C. (1995). Engines for education. New York: Lawrence Erlbaum. Schank, R. C. (1995). Engines for education. New York: Lawrence Erlbaum.
Zurück zum Zitat Schank, R. C., Berman, T. R., & Macpherson, K. A. (1999). Learning by doing. In C. M. Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory (Vol. II, pp. 161–181). Mahwah: Lawrence Erlbaum Associates. Schank, R. C., Berman, T. R., & Macpherson, K. A. (1999). Learning by doing. In C. M. Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory (Vol. II, pp. 161–181). Mahwah: Lawrence Erlbaum Associates.
Zurück zum Zitat Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books. Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books.
Zurück zum Zitat Shute, V. J., Ventura, M., Bauer, M., & Zapata-Rivera, D. (2009). Melding the power of serious games and embedded assessment to monitor and foster learning: flow and grow. In U. Ritterfeld, M. Cody, & M. Vorderer (Eds.), Serious games: Mechanisms and effects (pp. 295–321). New York: Routledge. Shute, V. J., Ventura, M., Bauer, M., & Zapata-Rivera, D. (2009). Melding the power of serious games and embedded assessment to monitor and foster learning: flow and grow. In U. Ritterfeld, M. Cody, & M. Vorderer (Eds.), Serious games: Mechanisms and effects (pp. 295–321). New York: Routledge.
Zurück zum Zitat Vargas, J. S. (1986). Instructional Design Flaws in Computer-Assisted Instruction. The Phi Delta Kappan, 67(10), 738–744. Retrieved from http://www.jstor.org/stable/20403230. Vargas, J. S. (1986). Instructional Design Flaws in Computer-Assisted Instruction. The Phi Delta Kappan, 67(10), 738–744. Retrieved from http://​www.​jstor.​org/​stable/​20403230.​
Zurück zum Zitat Westera, W., Hommes, M. A., Houtmans, M., & Kurvers, H. J. (2003). Computer-supported training of psycho-diagnostic skills. Interactive Learning Environments, 11(3), 215–231.CrossRef Westera, W., Hommes, M. A., Houtmans, M., & Kurvers, H. J. (2003). Computer-supported training of psycho-diagnostic skills. Interactive Learning Environments, 11(3), 215–231.CrossRef
Zurück zum Zitat Westera, W., Nadolski, R., Hummel, H., & Wopereis, I. (2008). Serious games for higher education: a framework for reducing design complexity. Journal of Computer-Assisted Learning, 24(5), 420–432.CrossRef Westera, W., Nadolski, R., Hummel, H., & Wopereis, I. (2008). Serious games for higher education: a framework for reducing design complexity. Journal of Computer-Assisted Learning, 24(5), 420–432.CrossRef
Metadaten
Titel
Performance assessment in serious games: Compensating for the effects of randomness
verfasst von
Wim Westera
Publikationsdatum
20.08.2014
Verlag
Springer US
Erschienen in
Education and Information Technologies / Ausgabe 3/2016
Print ISSN: 1360-2357
Elektronische ISSN: 1573-7608
DOI
https://doi.org/10.1007/s10639-014-9347-3

Weitere Artikel der Ausgabe 3/2016

Education and Information Technologies 3/2016 Zur Ausgabe