Kristin Siu
Mark O. Riedl
ABSTRACT
Human computation games (HCGs) can provide novel solutions to intractable computational problems, help enable scientific breakthroughs, and provide datasets for artificial intelligence. However, our knowledge about how to design and deploy HCGs that appeal to players and solve problems effectively is incomplete. We present an investigatory HCG based on Super Mario Bros. We used this game in a human subjects study to investigate how different social conditions---singleplayer and multiplayer---and scoring mechanics---collaborative and competitive---affect players' subjective experiences, accuracy at the task, and the completion rate. In doing so, we demonstrate a novel design approach for HCGs, and discuss the benefits and tradeoffs of these mechanics in HCG design.
- Jonathan Barone, Colin Bayer, Rowan Copley, Nova Barlow, Matthew Burns, Sundipta Rao, Georg Seelig, Zoran Popović, Seth Cooper, and Nanocrafter Players. 2015. Nanocrafter: Design and Evaluation of a DNA Nanotechnology Game. In Proceedings of the 10th International Conference on the Foundations of Digital Games.Google Scholar
- Luke Barrington, Douglas Turnbull, and Gert Lanckriet. 2012. Game-powered machine learning. Proceedings of the National Academy of Sciences 109, 17 (2012), 6411--6416. Google ScholarCross Ref
- Seth Cooper, Firas Khatib, Adrien Treuille, Janos Barbero, Jeehyung Lee, Michael Beenen, Andrew Leaver-Fay, David Baker, Zoran Popović, and others. 2010. Predicting protein structures with a multiplayer online game. Nature 466, 7307 (2010), 756--760. Google ScholarCross Ref
- Seth Cooper, Adrien Treuille, Janos Barbero, Andrew Leaver-Fay, Kathleen Tuite, Firas Khatib, Alex Cho Snyder, Michael Beenen, David Salesin, David Baker, Zoran Popović, and Foldit Players. 2010. The Challenge of Designing Scientific Discovery Games. In 5th International Conference on the Foundations of Digital Games. Google ScholarDigital Library
- Werner Dietl, Stephanie Dietzel, Michael D. Ernst, Nathaniel Mote, Brian Walker, Seth Cooper, Timothy Pavlik, and Zoran Popović. 2012. Verification Games: Making Verification Fun. In 14th Workshop on Formal Techniques for Java-like Programs. Google ScholarDigital Library
- Julie S. Downs, Mandy B. Holbrook, Steve Sheng, and Lorrie Faith Cranor. 2010. Are Your Participants Gaming the System?: Screening Mechanical Turk Workers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 2399--2402. Google ScholarDigital Library
- Katharina Emmerich and Maic Masuch. 2013. Helping friends or fighting foes: The influence of collaboration and competition on player experience.. In Proceedings of the 8th International Conference on the Foundations of Digital Games.Google Scholar
- Dion Hoe-Lian Goh, Rebecca P. Ang, Chei Sian Lee, and Alton Y. K. Chua. 2011. Fight or Unite: Investigating Game Genres for Image Tagging. J. Am. Soc. Inf. Sci. Technol. 62, 7 (July 2011), 1311--1324. Google ScholarDigital Library
- Matthew Guzdial and Mark Riedl. 2016. Conceptually Blended Levels in a Unity Engine. In Playable Experiences at Twelfth Artificial Intelligence and Interactive Digital Entertainment Conference.Google Scholar
- Chien-Ju Ho, Tao-Hsuan Chang, Jong-Chuan Lee, Jane Yung-jen Hsu, and Kuan-Ta Chen. 2010. KissKissBan: A Competitive Human Computation Game for Image Annotation. SIGKDD Explor. Newsl. 12, 1 (Nov. 2010), 21--24. Google ScholarDigital Library
- Peter Jamieson, Lindsay Grace, and Jack Hall. 2012. Research Directions for Pushing Harnessing Human Computation to Mainstream Video Games. In Meaningful Play.Google Scholar
- Fengfeng Ke and Barbara L. Grabowski. 2007. Gameplaying for maths learning: cooperative or not? British Journal of Educational Technology 38 (2007), 249--259. Google ScholarCross Ref
- Markus Krause and René Kizilcec. 2015. To Play or Not to Play: Interactions between Response Quality and Task Complexity in Games and Paid Crowdsourcing. In Third AAAI Conference on Human Computation and Crowdsourcing.Google ScholarCross Ref
- Markus Krause and Jan Smeddinck. 2011. Human computation games: A survey. In 2011 19th European Signal Processing Conference. IEEE, 754--758.Google Scholar
- Markus Krause, Aneta Takhtamysheva, Marion Wittstock, and Rainer Malaka. 2010. Frontiers of a Paradigm: Exploring Human Computation with Digital Games. In Proceedings of the ACM SIGKDD Workshop on Human Computation (HCOMP '10). ACM, New York, NY, USA, 22--25. Google ScholarDigital Library
- Chris J Lintott, Kevin Schawinski, Anže Slosar, Kate Land, Steven Bamford, Daniel Thomas, M Jordan Raddick, Robert C Nichol, Alex Szalay, Dan Andreescu, Phil Murray, and Jan Vandenberg. 2008. Galaxy Zoo: morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey. Monthly Notices of the Royal Astronomical Society 389, 3 (2008), 1179--1189.Google ScholarCross Ref
- Heather Logas, Jim Whitehead, Michael Mateas, Richard Vallejos, Lauren Scott, Dan Shapiro, John Murray, Kate Compton, Joseph Osborn, Orlando Salvatore, Zhongpeng Lin, Huascar Sanchez, Michael Shavlovsky, Daniel Cetina, Shayne Clementi, and Chris Lewis. 2014. Software Verification Games: Designing Xylem, The Code of Plants. In Proceedings of the 9th International Conference on the Foundations of Digital Games.Google Scholar
- Regan L. Mandryk and Kori M. Inkpen. 2004. Physiological Indicators for the Evaluation of Co-located Collaborative Play. In Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work (CSCW '04). ACM, New York, NY, USA, 102--111. Google ScholarDigital Library
- Ei Pa Pa Pe-Than, Dion Hoe-Lian Goh, and Chei Sian Lee. 2013. A typology of human computation games: an analysis and a review of current games. Behaviour & Information Technology (2013).Google Scholar
- Wei Peng and Gary Hsieh. 2012. The influence of competition, cooperation, and player relationship in a motor performance centered computer game. Computers in Human Behavior 28, 6 (2012), 2100--2106. Google ScholarDigital Library
- Mark Peplow. 2016. Citizen science lures gamers into Sweden's Human Protein Atlas. Nature Biotechnology 34, 5 (2016), 452--452. Google ScholarCross Ref
- Jan L Plass, Paul A Ofikeefe, Bruce D Homer, Jennifer Case, Elizabeth O Hayward, Murphy Stein, and Ken Perlin. 2013. The impact of individual, competitive, and collaborative mathematics game play on learning, performance, and motivation. Journal of Educational Psychology 105, 4 (2013), 1050.Google ScholarCross Ref
- Norsaremah Salleh, Emilia Mendes, and John Grundy. 2011. Empirical studies of pair programming for CS/SE teaching in higher education: A systematic literature review. IEEE Transactions on Software Engineering 37, 4 (2011), 509--525. Google ScholarDigital Library
- Magy Seif El-Nasr, Bardia Aghabeigi, David Milam, Mona Erfani, Beth Lameman, Hamid Maygoli, and Sang Mah. 2010. Understanding and Evaluating Cooperative Games. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 253--262. Google ScholarDigital Library
- Katharina Siorpaes and Martin Hepp. 2008. Games with a Purpose for the Semantic Web. IEEE Intelligent Systems 23, 3 (2008), 50--60. Google ScholarDigital Library
- Kristin Siu, Alexander Zook, and Mark O. Riedl. 2014. Collaboration versus Competition: Design and Evaluation of Mechanics for Games with a Purpose. In 9th International Conference on the Foundations of Digital Games.Google Scholar
- Julian Togelius, Sergey Karakovskiy, and Noor Shaker. 2012. 2012 Mario AI Championship. http://www.marioai.org/. (2012).Google Scholar
- Kathleen Tuite. 2014. GWAPs: Games with a Problem. In Proceedings of the 9th International Conference on the Foundations of Digital Games.Google Scholar
- Kathleen Tuite, Noah Snavely, Dun-yu Hsiao, Nadine Tabing, and Zoran Popović. 2011. PhotoCity: Training Experts at Large-scale Image Acquisition Through a Competitive Game. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 1383--1392. Google ScholarDigital Library
- Luis von Ahn and Laura Dabbish. 2004. Labeling Images with a Computer Game. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04). ACM, New York, NY, USA, 319--326. Google ScholarDigital Library
- Luis von Ahn and Laura Dabbish. 2008. Designing Games with a Purpose. Commun. ACM 51, 8 (2008), 58--67. Google ScholarDigital Library
- Rina R. Wehbe and Lennart E. Nacke. 2015. Towards Understanding the Importance of Co-Located Gameplay. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '15). ACM, New York, NY, USA, 733--738. Google ScholarDigital Library
Index Terms
- Evaluating singleplayer and multiplayer in human computation games
Recommendations
Reward Systems in Human Computation Games
CHI PLAY '16: Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in PlayHuman computation games (HCGs) are games in which player interaction is used to solve problems intractable for computers. Most HCGs use simple reward mechanisms such as points or leaderboards, but in contrast, many mainstream games use more complex, and ...
A framework for exploring and evaluating mechanics in human computation games
FDG '17: Proceedings of the 12th International Conference on the Foundations of Digital GamesHuman computation games (HCGs) are a crowdsourcing approach to solving computationally-intractable tasks using games. We outline a formal representation of the mechanics in HCGs, providing a structural breakdown to visualize, compare, and explore the ...
Evaluating GameFlow in a Multiplayer Online Strategy Game Under Development
ACSW '20: Proceedings of the Australasian Computer Science Week MulticonferenceGameFlow is a widely used model of player enjoyment, with hundreds of applications to designing and evaluating games and game-like experiences since its first publication. Derived from a general set of heuristics for creating enjoyable player ...
Comments