Britton Horn
ABSTRACT
A significant issue in research on educational games lies in evaluating their educational impact. Although game analytics is often leveraged in the game industry, it can also provide insight into player actions, strategy development, and the learning process in educational games separate from external evaluation measures. This paper explores the potential of game analytics for learning by analyzing player strategies of an educational game that is designed to support algorithmic thinking. We analyze player strategies from nine cases in our data, combining quantitative and qualitative game analysis techniques: hierarchical player clustering, game progression visualizations, playtraces, and think-aloud data. Results suggest that this combination of data analysis techniques provides insights into level progression and learning strategies that may have been otherwise overlooked.
- Mike Ambinder. 2009. Valve's approach to playtesting: The application of empiricism.Google Scholar
- Erik Andersen, Sumit Gulwani, and Zoran Popović. 2013. A Trace-Based Framework for Analyzing and Synthesizing Educational Progressions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 773--782. Google ScholarDigital Library
- Erik Andersen, Yun-En Liu, Ethan Apter, Francois Boucher-Genesse, and Zoran Popović. 2010. Gameplay Analysis Through State Projection. ACM, 1--8. Google ScholarDigital Library
- CodeCombat. 2016. CodeCombat: Learn to Code by Playing a Game. (2016).Google Scholar
- Adele Cutler and Leo Breiman. 1994. Archetypal analysis. Technometrics 36, 4 (1994).Google Scholar
- Phillip DeRosa. 2007. Tracking player feedback to improve game design. Gamasutra (2007).Google Scholar
- Nicholas Duchenaut, Nick Yee, Eric Nickel, and Robert J. Moore. 2006. Building an MMO with mass appeal: A look at gameplay in World of Warcraft. Game Developer Magazine 1 (2006), 281--317.Google Scholar
- Magy Seif El-Nasr, Anders Drachen, and Alessandro Canossa. 2013. Game analytics: Maximizing the value of player data. Springer Science & Business Media. Google ScholarDigital Library
- Yetunde Folajimi, Britton Horn, Jacqueline Barnes, Amy Hoover, Gillian Smith, and Casper Harteveld. 2016. A Cross-Cultural Evaluation of a Computer Science Teaching Game. In Proceedings of Games+Learning+Society. ETC Press, Pittsburgh, PA.Google Scholar
- Tracy Fullerton. 2008. Game Design Workshop: A playcentric Approach to Creating Innovative Games (2nd ed.). Morgan Kaufmann.Google Scholar
- Bradley S Greenberg, John Sherry, Kenneth Lachlan, Kristen Lucas, and Amanda Holmstrom. 2010. Orientations to video games among gender and age groups. Simulation & Gaming 41 (2010), 238--259. Google ScholarDigital Library
- Erik Harpstead and Vincent Aleven. Using Empirical Learning Curve Analysis to Inform Design in an Educational Game. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY 2015). ACM Press, 197--207. Google ScholarDigital Library
- Erik Harpstead, Christopher J MacLellan, Vincent Aleven, and Brad A Myers. 2015. Replay Analysis in Open-Ended Educational Games. In Serious Games Analytics, Christian Sebastian Loh, Yanyan Sheng, and Dirk Ifenthaler (Eds.). Springer International Publishing, 381--399.Google Scholar
- Casper Harteveld. 2011. Triadic game design: Balancing reality, meaning and play. Springer Science & Business Media. Google ScholarDigital Library
- Casper Harteveld. 2012. Making Sense of Virtual Risks. IOS Press.Google Scholar
- Casper Harteveld, Gillian Smith, Gail Carmichael, Elisabeth Gee, and Carolee Stewart-Gardiner. 2014. A Design-Focused Analysis of Games Teaching Computer Science. In Proceedings of Games+Learning+Society 10. Madison, WI.Google Scholar
- Casper Harteveld and Steven Sutherland. 2015. The goal of scoring: Exploring the role of game performance.. In Proceedings of the 2015 ACM Conference on Computer-Human Interaction. Google ScholarDigital Library
- Eric Hazan. 2013. Contextualizing Data. In Game Analytics. Springer, 477--496.Google Scholar
- David Hilbert and David Redmiles. 2000. Extracting usability information from user interface events. Comput. Surveys 32, 4 (2000), 384--421. Google ScholarDigital Library
- Britton Horn, Christopher Clark, Oskar Strom, Hilery Chao, Amy J. Stahl, Casper Harteveld, and Gillian Smith. 2016. Design insights into the creation and evaluation of a computer science educational game. In Proceedings of the 47th ACM Technical Symposium on Computer Science Education (SIGCSE). ACM Press, Memphis, TN. Google ScholarDigital Library
- Ioanna Iacovides, Anna L Cox, Ara Avakian, and Thomas Knoll. Player Strategies: Achieving Breakthroughs and Progressing in Single-player and Cooperative Games. In Proceedings of the First ACM SIGCHI Annual Symposium on Computer-Human Interaction in Play (CHI PLAY 2014). ACM Press, 131--140. Google ScholarDigital Library
- Amanda Lenhart and Pew Internet & American Life Project. 2008. Teens, Video Gaming and Civics. Technical Report.Google Scholar
- Vladimir I Levenshtein. 1966. Binary Codes Capable of Correcting Deletions, Insertions and Reversals. Soviet Physics Doklady 10, 8 (1966), 707--710.Google Scholar
- Frederick WB Li and Christopher Watson. 2011. Game-based concept visualization for learning programming. In Proceedings of the third international ACM workshop on Multimedia technologies for distance learning. ACM, 37--42. Google ScholarDigital Library
- Conor Linehan, George Bellord, Ben Kirman, Zachary H Morford, and Bryan Roche. Learning Curves: Analysing Pace and Challenge in Four Successful Puzzle Games. In Proceedings of the First ACM SIGCHI Annual Symposium on Computer-Human Interaction in Play (CHI PLAY 2014). ACM Press, 181--190. Google ScholarDigital Library
- Yun-En Liu, Erik Andersen, Richard Snider, Seth Cooper, and Zoran Popović. 2011. Feature-based Projections for Effective Playtrace Analysis. In Proceedings of the 6th International Conference on Foundations of Digital Games (FDG '11). ACM, New York, NY, USA, 69--76. Google ScholarDigital Library
- Christian Sebastian Loh, Yanyan Sheng, and Dirk Ifenthaler. 2015. Serious Games Analytics: Methodologies for Performance Measurement, Assessment, and Improvement. Springer.Google Scholar
- Truong-Huy Dinh Nguyen, Magy Seif El-Nasr, and Alessandro Canossa. 2015. Glyph: Visualization Tool for Understanding Problem Solving Strategies in Puzzle Games. Foundations of Digital Games (FDG) (2015).Google Scholar
- Jackie O'Kelly and J Paul Gibson. 2006. RoboCode & problem-based learning: a non-prescriptive approach to teaching programming. ACM SIGCSE Bulletin 38, 3 (2006), 217--221. Google ScholarDigital Library
- Joseph C Osborn, Ben Samuel, Joshua Allen McCoy, and Michael Mateas. 2014. Evaluating play trace (dis)similarity metrics. In 10th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment.Google Scholar
- Mikki H Phan, Jo R Jardina, Sloane Hoyle, and Barbara S Chaparro. 2012. Examining the Role of Gender in Video Game Usage, Preference, and Behavior. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting. SAGE Publications, 1496--1500.Google ScholarCross Ref
- Robert C. Prim. 1957. Shortest Connection Networks And Some Generalizations. Bell System Technical Journal 36 (Nov. 1957), 1389--1401.Google Scholar
- Ramon Romero. 2008. Successful instrumentation: Tracking attitudes and behviors to improve games. In Game Developers' Conference.Google Scholar
- Angel Serrano-Laguna, Javier Torrente, Pablo Moreno-Ger, and Baltasar Fernández-Man. 2014. Application of Learning Analytics in Educational Videogames. Entertainment Computing 5 (2014), 313--322.Google ScholarCross Ref
- David Williamson Shaffer and James Paul Gee. 2012. The Right Kind of GATE: Computer games and the future of assessment. Technology-based assessments for 21st century skills: Theoretical and practical implications from modern research. Charlotte, NC: Information Age Publishing (2012).Google Scholar
- Valerie Shute and Matthew Ventura. 2013. Stealth assessment: Measuring and supporting learning in video games. MIT Press, Cambridge, MA. Google ScholarDigital Library
- George Siemens and Ryan S J d Baker. 2012. Learning Analytics and Educational Data Mining: Towards Communication and Collaboration. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge. ACM, 252--254. Google ScholarDigital Library
- Catherine A Sugar and Gareth M James. 2003. Finding the Number of Clusters in a Dataset: An Information-Theoretic Approach. J. Amer. Statist. Assoc. (2003), 750--763.Google Scholar
- Robert L Thorndike. 1953. Who belongs in the family? Psychometrika 18 (Dec. 1953), 267--276.Google Scholar
- Allen Tucker, Fadi Deek, Jill Jones, Dennis McCowan, Chris Stephenson, and Anita Verno. 2006. ACM K-12 CS Model Curriculum. Computer Science Teachers Association.Google Scholar
Index Terms
- Opening the Black Box of Play: Strategy Analysis of an Educational Game
Recommendations
A Gateway Game to Make Computational Problem Solving Accessible and Engaging (Abstract Only)
SIGCSE '15: Proceedings of the 46th ACM Technical Symposium on Computer Science EducationIn this work, we seek to better understand how game design principles impact the effectiveness of a video game learning environment for computational problem solving. Video games naturally encourage problem solving and incremental learning of their game ...
Clashroom: A Game to Enhance the Classroom Experience (Abstract Only)
SIGCSE '16: Proceedings of the 47th ACM Technical Symposium on Computing Science EducationClashroom is a web-based fantasy game that was created to enhance the motivation and learning of a traditional classroom. It was created to address several drawbacks of other educational games by being broadly applicable to any course, requiring no ...
Evaluation of the user experience and intrinsic motivation with educational and mainstream digital games
CLIHC '15: Proceedings of the Latin American Conference on Human Computer InteractionMany studies of digital games have been developed to investigate a series of attributes to improve the games industry and to know the impacts of the use of digital games on users. Many arguments exist stating that the appeal of mainstream digital games ...
Comments