Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
Erschienen in:
Introduction to the GI-Dagstuhl Book on Entertainment Computing and Serious Games Kapitel lesen Erstes Kapitel lesen

2016 | OriginalPaper | Buchkapitel

Operationalization and Measurement of Evaluation Constructs

verfasst von : Katharina Emmerich, Natalya Bogacheva, Mareike Bockholt, Viktor Wendel

Erschienen in: Entertainment Computing and Serious Games

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This chapter deals with the operationalization and measurement of evaluation constructs, an important and challenging part of the serious games evaluation process. Hereby, advices will be given on what has to be measured and how to quantify an abstract concept. Thus, the chapter makes two main contributions. First, general data gathering methods are described and discussed in terms of advantages and disadvantages. Second, main psychological concepts and evaluation constructs relevant in the context of serious games, as well as their theoretical foundations, are introduced. In order to support the reader on planning future serious game evaluations, a list and description of concrete techniques and questionnaires addressing concepts like motivation, player experience, learning outcomes, health, well-being, and attitudes are compiled.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Vorheriges Kapitel The Experimental Method as an Evaluation Tool in Serious Games Research and Development
Nächstes Kapitel Personalized and Adaptive Serious Games
Fußnoten
Literatur
1.
Aronson, E., Wilson, T.D., Akert, R.M.: Social Psychology, 7th edn. Pearson, Boston (2010). Global edn
2.
Bandura, A.: Guide for constructing self-efficacy scales. In: Pajares, F., Urdan, T.C. (eds.) Self-Efficacy Beliefs of Adolescents, pp. 307–337. Information Age Publishing, Charlotte (2006)
3.
4.
Bartle, R.A.: Designing Virtual Worlds. New Riders Pub., Indianapolis (2004)
5.
Bellotti, F., Kapralos, B., Lee, K., Moreno-Ger, P., Berta, R.: Assessment in and of serious games: an overview. Adv. Hum. Comput. Interact. 2013, 1–11 (2013)
6.
Bernhaupt, R. (ed.): Evaluating User Experience in Games: Concepts and Methods. Human-Computer Interaction Series. Springer, London, New York (2010)
7.
Bernhaupt, R.: User experience evaluation in entertainment. In: Bernhaupt, R. (ed.) Evaluating User Experience in Games. Human-ComputerInteraction Series, pp. 3–7. Springer, London, New York (2010)CrossRef
8.
Biocca, F., Harms, C., Burgoon, J.: Towards a more robust theory and measure of social presence: Review and suggested criteria. Presence Teleoper. Virtual Environ. 12(5), 456–480 (2003)CrossRef
9.
10.
Campbell, D.T., Stanley, J.C., Gage, N.L.: Experimental and Quasi-Experimental Designs for Research. Ravenio books, New York (2015)
11.
Chin, J., Dukes, R., Gamson, W.: Assessment in simulation and gaming a review of the last 40 years. Simul. Gaming 40(4), 553–568 (2009)CrossRef
12.
Connolly, T., Stansfield, M., Hainey, T.: Towards the development of a games-based learning evaluation framework. In: Games-Based Learning Advancements for Multisensory Human Computer Interfaces: Techniques and Effective Practices (2009)
13.
Csikszentmihalyi, M.: Flow: The Psychology of Optimal Experience. Harper Perennial, New York (1990)
14.
De Kort, Y.A.W., Ijsselsteijn, W.A.: People, places, and play: player experience in a socio-spatial context. Comput. Entertain. 6(2), 18 (2008)
15.
De Kort, Y.A.W., Ijsselsteijn, W.A., Poels, K.: Digital games as social presence technology: development of the social presence in gaming questionnaire (SPGQ). In: Proceedings of the 10th Annual International Workshop on Presence, pp. 195–203 (2007)
16.
Dignan, A.: Game frame: Using Games as a Strategy for Success. Free Press, New York (2011). 1st free press hardcover edn
17.
Dillon, R.: On The Way to Fun: An Emotion-Based Approach to Successful Game Design. A K Peters, Natick (2010)CrossRef
18.
Dodge, R., Daly, A., Huyton, J., Sanders, L.: The challenge of defining wellbeing. Int. J. Wellbeing 2(3), 222–235 (2012)CrossRef
19.
Dugard, P., Todman, J.: Analysis of pre-test-post-test control group designs in educational research. Educ. Psychol. 15(2), 181–198 (1995)CrossRef
20.
21.
Fernandez, A.: Fun experience with digital games: a model proposition. In: Leino, O., Wirman, H., Fernandez, A. (eds.) Extending Experiences, pp. 181–190. Lapland University Press, Rovaniemi (2008)
22.
Ferrara, J.: Playful Design: Creating Game Experiences in Everyday Interfaces. Rosenfeld Media, Brooklyn (2012)
23.
Garfield, S., Clifford, S., Eliasson, L., Barber, N., Willson, A.: Suitability of measures of self-reported medication adherence for routine clinical use: a systematic review. BMC Med. Res. Methodol. 11, 149 (2011)CrossRef
24.
Garris, R., Ahlers, R., Driskell, J.E.: Games, motivation, and learning: a research and practice model. Simul. Gaming 33(4), 441–467 (2002)CrossRef
25.
Gee, J.P.: What video games have to teach us about learning and literacy. Comput. Entertain. 1(1), 20 (2003)CrossRef
26.
27.
Gill, P., Stewart, K., Treasure, E., Chadwick, B.: Methods of data collection in qualitative research: interviews and focus groups. Br. Dent. J. 204(6), 291–295 (2008)CrossRef
28.
Girard, C., Ecalle, J., Magnan, A.: Serious games as new educational tools: how effective are they? a meta-analysis of recent studies. J. Comput. Assist. Learn. 29(3), 207–219 (2013)CrossRef
29.
Goldstein, G., Hersen, M.: Handbook of Psychological Assessment. Elsevier, Hoboken (2000)
30.
Goodwin, C.J., Goodwin, K.A.: Research in Psychology: Methods and Design, 7th edn. Wiley, Hoboken (2012)
31.
Greenwald, A.G., McGhee, D.E., Schwartz, J.L.: Measuring individual differences in implicit cognition: the implicit association test. J. Person. Soc. Psychol. 74(6), 1464–1480 (1998)CrossRef
32.
Guilford, J.P.: Psychometric Methods, 2nd edn. McGraw-Hill, New York (1954)
33.
Hudson, M., Cairns, P.: Measuring social presence in team-based digital games. In: Riva, G., Waterworth, J., Murray, D. (eds.) Interacting with Presence: HCI and the Sense of Presence in Computer-Mediated Environments. DE GRUYTER OPEN, Warsaw, Poland (2014)
34.
Igroup Project Consortium: Igroup presence questionnaire (ipq)
35.
IJsselsteijn, W., de Kort, Y., Poels, K., Jurgelionis, A., Bellotti, F: Characterising and measuring user experiences in digital games. In: ACE 2007 International Conference on Advances in Computer Entertainment Technology, Workshop ‘Methods for Evaluating Games - How to measure Usability and User Experience in Games’ (2007)
36.
Ijsselsteijn, W.A., De Kort, Y.A.W., Poels, K.: The game experience questionnaire: Development of a self-report measure to assess the psychological impact of digital games. Manuscript in preparation
37.
Kahneman, D., Krueger, A.B.: Developments in the measurement of subjective well-being. J. Econ. Perspect. 20(1), 3–24 (2006)CrossRef
38.
Kato, P.M., Cole, S.W., Bradlyn, A.S., Pollock, B.H.: A video game improves behavioral outcomes in adolescents and young adults with cancer: a randomized trial. Pediatrics 122(2), e305–e317 (2008). 18676516CrossRef
39.
Kirkpatrick, D.L.: Evaluating Training Programs. Tata McGraw-Hill Education, San Francisco (1975)
40.
Kivikangas, J.M., Chanel, G., Cowley, B., Ekman, I., Salminen, M., Järvelä, S., Ravaja, N.: A review of the use of psychophysiological methods in game research. J. Gaming Virtual World 3(3), 181–199 (2011)CrossRef
41.
Kline, P.: Psychometrics and Psychology. Academy Press, Washington, D.C. (1979)
42.
Koster, R.: A Theory of Fun for Game Design, 2nd edn. O’Reilly Media Inc., Sebastopol (2014)
43.
Krosnick, J.A., Presser, S.: Question and questionnaire design. In: Marsden, P.V. (ed.) Handbook of Survey Research. Emerald Group Publishing, Bingley (2010)
44.
Landers, R.N., Bauer, K.N.: Quantitative methods and analyses for the study of players and their behaviour. In: Lankoski, P., Björk, S. (eds.) Game Research Methods, pp. 151–174. ETC Press, Pittsburgh (2015)
45.
Law, E.L.C., Roto, V., Hassenzahl, M., Vermeeren, A.P., Kort, J.: Understanding, scoping and defining user experience. In: Olsen, D.R., Arthur, R.B., Hinckley, K., Morris, M.R., Hudson, S., Greenberg, S. (eds.) The SIGCHI Conference, p. 719 (2009)
46.
Lazzaro, N.: Why we play games: Four keys to more emotion without story (2004)
47.
Loh, C.S., Anantachai, A., Byun, J., Lenox, J.: Assessing what players learned in serious games: in situ data collection, information trails, and quantitative analysis. In: 10th International Conference on Computer Games: AI, Animation, Mobile, Educational & Serious Games (CGAMES 2007), pp. 25–28 (2007)
48.
Loh, C.S., Sheng, Y., Ifenthaler, D. (eds.): Serious Games Analytics: Methodologies for Performance Measurement, Assessment, and Improvement. Springer, Heidelberg (2015)
49.
Maki, P.L.: Assessing for Learning: Building a Sustainable Commitment Across the Institution. Stylus Publishing, LLC, Menlo Park (2012)
50.
Mandryk, R.L., Inkpen, K.M.: Physiological indicators for the evaluation of co-located collaborative play. In: Herbsleb, J., Olson, G. (eds.) The 2004 ACM Conference, p. 102 (2004)
51.
McAuley, E., Duncan, T., Tammen, V.V.: Psychometric properties of the intrinsic motivation inventory in a competitive sport setting: a confirmatory factor analysis. Res. Q. Exerc. Sport 60(1), 48–58 (1989)CrossRef
52.
Nacke, L.E., Bateman, C., Mandryk, R.L.: BrainHex: preliminary results from a neurobiological gamer typology survey. In: Anacleto, J.C., Fels, S., Graham, N., Kapralos, B., Saif El-Nasr, M., Stanley, K. (eds.) ICEC 2011. LNCS, vol. 6972, pp. 288–293. Springer, Heidelberg (2011). doi:10.​1007/​978-3-642-24500-8_​31 CrossRef
53.
Nacke, L.E.: Affective ludology: scientific measurement of user experience in interactive entertainment, Blekinge Institute of Technology doctoral dissertation series, vol. 2009: 04. School of Computing, Blekinge Institute of Technology, Karlskrona (2010)
54.
O’Neil, H.F., Wainess, R., Baker, E.L.: Classification of learning outcomes: evidence from the computer games literature. Cirriculum J. 16(4), 455–474 (2005)CrossRef
55.
Plass, J.L., Homer, B.D., Kinzer, C.K., Chang, Y.K., Frye, J., Kaczetow, W., Isbister, K., Perlin, K.: Metrics in simulations and games for learning. In: El-Nasr, M.S., Drachen, A., Canossa, A. (eds.) Game Analytics, pp. 697–729. Springer, Heidelberg (2013)CrossRef
56.
Poels, K., de Kort, Y., Ijsselsteijn, W.: It is always a lot of fun! In: Kapralos, B., Katchabaw, M., Rajnovich, J. (eds.) The 2007 Conference, p. 83 (2007)
57.
Rammstedt, B., Kemper, C.J., Klein, M.C., Beierlein, C., Kovaleva, A.: A short scale for assessing the big five dimensions of personality
58.
Ravaja, N., Saari, T., Turpeinen, M., Laarni, J., Salminen, M., Kivikangas, M.: Spatial presence and emotions during video game playing: does it matter with whom you play? Presence Teleoper. Virtual Environ. 15(4), 381–392 (2006)CrossRef
59.
Rheinberg, F.: Motivation, Kohlhammer-Urban-Taschenbücher, vol. 555. Kohlhammer, Stuttgart, 7, aktualisierte aufl. edn. (2008)
60.
Ryan, D.: Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp. Educ. Psychol. 25(1), 54–67 (2000)CrossRef
61.
Ryan, R.M., Rigby, C.S., Przybylski, A.: The motivational pull of video games: a self-determination theory approach. Motiv. Emot. 30(4), 344–360 (2006)CrossRef
62.
63.
Sano, A., Picard, R.W.: Stress recognition using wearable sensors and mobile phones. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII), pp. 671–676 (2013)
64.
Shute, V.J., Ventura, M., Bauer, M., Zapata-Rivera, D.: Melding the power of serious games and embedded assessment to monitor and foster learning. Serious Games Mech. Effects 2, 295–321 (2009)
65.
Slater, M.: A note on presence terminology. Presence Connect 3(3), 1–5 (2003)MathSciNet
66.
Spielberger, C.D.: State-trait anxiety inventory. In: Weiner, I.B., Craighead, W.E. (eds.) The Corsini Encyclopedia of Psychology. Wiley, Hoboken (2010)
67.
Strube, M.J., Newman, L.C.: Psychometrics. In: Cacioppo, J., Tassinary, L.G., Berntson, G.G. (eds.) The Handbook of Psychophysiology, pp. 789–811. Cambridge University Press, Cambridge (2007)
68.
Sweetser, P., Wyeth, P.: Gameflow: a model for evaluating player enjoyment in games. Comput. Entertain. 3(3), 3 (2005)CrossRef
69.
Takatalo, J., Häkkinen, J., Kaistinen, J., Nyman, G.: Presence, involvement, and flow in digital games. In: Bernhaupt, R. (ed.) Evaluating User Experience in Games, pp. 23–46. Human-Computer Interaction Series, Springer, London, New York (2010)CrossRef
70.
Takatalo, J., Häkkinen, J., Komulainen, J., Särkelä, H., Nyman, G.: Involvement and presence in digital gaming. In: Mørch, A., Morgan, K., Bratteteig, T., Ghosh, G., Svanaes, D. (eds.) The 4th Nordic Conference, pp. 393–396 (2006)
71.
UQO Cyberpsychology Lab: Immersive tendencies questionnaire (itq) (2004)
72.
Witmer, B.G., Singer, M.J.: Measuring presence in virtual environments: a presence questionnaire. Presence Teleoper. Virtual Environ. 7(3), 225–240 (1998)CrossRef
73.
Yee, N.: Motivations for play in online games. Cyberpsychol. Behav. 9(6), 772–775 (2006). The impact of the Internet, multimedia and virtual reality on behavior and societyCrossRef
Metadaten
Titel
Operationalization and Measurement of Evaluation Constructs
verfasst von
Katharina Emmerich
Natalya Bogacheva
Mareike Bockholt
Viktor Wendel
Copyright-Jahr
2016
Buch
Entertainment Computing and Serious Games

Print ISBN: 978-3-319-46151-9

Electronic ISBN: 978-3-319-46152-6

Copyright-Jahr: 2016

DOI
https://doi.org/10.1007/978-3-319-46152-6_13