Top

Empirical Software Engineering

Published in:

01-11-2021

An empirical study of Q&A websites for game developers

Authors: Arthur Kamienski, Cor-Paul Bezemer

Published in: Empirical Software Engineering | Issue 6/2021

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

The game development industry is growing, and training new developers in game development-specific abilities is essential to satisfying its need for skilled game developers. These developers require effective learning resources to acquire the information they need and improve their game development skills. Question and Answer (Q&A) websites stand out as some of the most used online learning resources in software development. Many studies have investigated how Q&A websites help software developers become more experienced. However, no studies have explored Q&A websites aimed at game development, and there is little information about how game developers use and interact with these websites. In this paper, we study four Q&A communities by analyzing game development data we collected from their websites and the 347 responses received on a survey we ran with game developers. We observe that the communities have declined over the past few years and identify factors that correlate to these changes. Using a Latent Dirichlet Allocation (LDA) model, we characterize the topics discussed in the communities. We also analyze how topics differ across communities and identify the most discussed topics. Furthermore, we find that survey respondents have a mostly negative view of the communities and tended to stop using the websites once they became more experienced. Finally, we provide recommendations on where game developers should post their questions, which can help mitigate the websites’ declines and improve their effectiveness.

previous article Predicting unstable software benchmarks using static source code features

next article The forgotten role of search queries in IR-based bug localization: an empirical study

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

https://stackoverflow.com/, accessed February 3, 2021.

https://answers.unity.com/, accessed February 3, 2021.

https://answers.unrealengine.com/, accessed February 3, 2021.

https://gamedev.stackexchange.com/, accessed February 3, 2021.

https://answers.yahoo.com/, accessed February 5, 2021.

https://www.quora.com/, accessed February 5, 2021.

http://doi.org/10.5281/zenodo.5047790.

https://archive.org/details/stackexchange, accessed February 5, 2021.

https://stackoverflow.com/tags, accessed February 5, 2021.

https://www.crummy.com/software/BeautifulSoup/bs4/doc/, accessed February 5, 2021.

https://radimrehurek.com/gensim/, accessed February 5, 2021.

Survey available at https://figshare.com/s/a8fe3851e409967e2d39

https://unity.com/our-company/newsroom/unity-technologies-releases-unity-3-5, accessed June 28th, 2021.

https://data.stackexchange.com/stackoverflow/query/1284342/answered-and-resolved-questions#graph, accessed February 5, 2021.

About Stack Overflow (2020) https://stackoverflow.com/company. Accessed: August 14, 2020

Adamic LA, Zhang J, Bakshy E, Ackerman MS (2008) Knowledge sharing and yahoo answers: everyone knows something. In: Proceedings of the 17th international conference on World Wide Web. pp 665–674

Ahmad A, Feng C, Ge S, Yousif A (2018) A survey on mining Stack Overflow: Question and answering (Q&A) community. Data Technol Appl

Ahmed S, Bagherzadeh M (2018) What do concurrency developers ask about? A large-scale study using Stack Overflow. In: Proceedings of the 12th ACM/IEEE international symposium on empirical software engineering and measurement. pp 1–10

Allamanis M, Sutton C (2013) Why, when, and what: analyzing Stack Overflow questions by topic, type, and code. In: 2013 10th working conference on mining software repositories (MSR). IEEE, pp 53–56

An L, Mlouki O, Khomh F, Antoniol G (2017) Stack Overflow: A code laundering platform?. In: 2017 IEEE 24th International conference on software analysis, evolution and reengineering (SANER). IEEE, pp 283–293

Anderson A, Huttenlocher D, Kleinberg J, Leskovec J (2012) Discovering value from community activity on focused question answering sites: A case study of Stack Overflow. In: Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining. pp 850–858

Asaduzzaman M, Mashiyat AS, Roy CK, Schneider KA (2013) Answering questions about unanswered questions of Stack Overflow. In: 2013 10th Working conference on mining software repositories (MSR). IEEE, pp 97–100

Bajaj K, Pattabiraman K, Mesbah A (2014) Mining questions asked by web developers. In: Proceedings of the 11th Working conference on mining software repositories, pp 112–121

Bangash AA, Sahar H, Chowdhury S, Wong AW, Hindle A, Ali K (2019) What do developers know about machine learning: a study of ml discussions on stackoverflow. In: 2019 IEEE/ACM 16th International conference on mining software repositories (MSR). IEEE, pp 260–264

Barua A, Thomas SW, Hassan AE (2014) What are developers talking about? An analysis of topics and trends in Stack Overflow. Empir Softw Eng 19 (3):619–654CrossRef

Beyer S, Macho C, Di Penta M, Pinzger M (2020) What kind of questions do developers ask on Stack Overflow? A comparison of automated approaches to classify posts into question categories. Empir Softw Eng 25(3):2258–2301CrossRef

Blei DM, Ng AY, Jordan MI (2003) Latent dirichlet allocation. J Mach Learn Res 3(Jan):993–1022MATH

Blodgett BM, Salter A (2013) Hearing ‘lady game creators’ tweet: #1reasonwhy, women and online discourse in the game development community. In: 14th Annual conference for the association of internet researchers (AoIR)

Blodgett BM, Salter A (2014) # 1reasonwhy: Game communities and the invisible woman. In: Foundations of digital games

Brookes J, Warburton M, Alghadier M, Mon-Williams M, Mushtaq F (2019) Studying human behavior with virtual reality: The Unity experiment framework. Behav Res Methods :1–9

Budhiraja A, Reddy R, Shrivastava M (2018) Lwe: Lda refined word embeddings for duplicate bug report detection. In: Proceedings of the 40th International conference on software engineering: companion proceeedings. pp 165–166

Cavusoglu H, Li Z, Huang KW (2015) Can gamification motivate voluntary contributions? The case of StackOverflow Q&A community. In: Proceedings of the 18th ACM conference companion on computer supported cooperative work & social computing. pp 171–174

Chen L, Baird A, Straub D (2019) Why do participants continue to contribute? Evaluation of usefulness voting and commenting ootivational affordances within an online knowledge community. Decis Support Syst 118:21–32CrossRef

Chen TH, Thomas SW, Hassan AE (2016) A survey on the use of topic models when mining software repositories. Empir Softw Eng 21(5):1843–1919CrossRef

Christopoulou E, Xinogalos S (2017) Overview and comparative analysis of game engines for desktop and mobile devices. Int J Serious Games 4(4)

Cliff N (1993) Dominance statistics: Ordinal analyses to answer ordinal questions. Psychol Bull 114(3):494CrossRef

Cowan B, Kapralos B (2014) A survey of frameworks and game engines for serious game development. In: 2014 IEEE 14th International conference on advanced learning technologies. IEEE, pp 662–664

Cox DR, Stuart A (1955) Some quick sign tests for trend in location and dispersion. Biometrika 42(1/2):80–95MathSciNetCrossRef

Cristie V, Berger M (2017) Game engines for urban exploration: Bridging science narrative for broader participants. In: Playable Cities. Springer, pp 87–107

Cummaudo A, Vasa R, Barnett S, Grundy J, Abdelrazek M (2020) Interpreting cloud computer vision pain-points: A mining study of Stack Overflow. arXiv:2001.10130

Deng S, Tong J, Lin Y, Li H, Liu Y (2019) Motivating scholars’ responses in academic social networking sites: An empirical study on ResearchGate Q&A behavior. Inf Process Manag 56(6):102082CrossRef

Epp R, Lin D, Bezemer CP (2021) An empirical study of trends of popular virtual reality games and their complaints. IEEE Trans Games :1–12

Fang C, Zhang J (2019) Users’ continued participation behavior in social Q&A communities: A motivation perspective. Comput Hum Behav 92:87–109CrossRef

Fischer F, Böttinger K., Xiao H, Stransky C, Acar Y, Backes M, Fahl S (2017) Stack Overflow considered harmful? The impact of copy&paste on android application security. In: 2017 IEEE symposium on security and privacy (SP). IEEE, pp 121–136

Fisher SJ, Harvey A (2013) Intervention for inclusivity: Gender politics and indie game development. Loading 7(11)

Ford D, Lustig K, Banks J, Parnin C (2018) “we don’t do that here” how collaborative editing with mentors improves engagement in social Q&A communities. In: Proceedings of the 2018 CHI conference on human factors in computing systems, pp. 1–12

Foxman M (2019) United we stand: Platforms, tools and innovation with the Unity game engine. Soc Media+ Soc 5(4):2056305119880177

Fu H, Oh S (2019) Quality assessment of answers with user-identified criteria and data-driven features in social Q&A. Inf Process Manag 56(1):14–28CrossRef

Guan T, Wang L, Jin J, Song X (2018) Knowledge contribution behavior in online Q&A communities: An empirical investigation. Comput Hum Behav 81:137–147CrossRef

Han J, Shihab E, Wan Z, Deng S, Xia X (2020) What do programmers discuss about deep learning frameworks. Empir Softw Eng 25(4):2694–2747CrossRef

Harper FM, Raban D, Rafaeli S, Konstan JA (2008) Predictors of answer quality in online q&a sites. In: Proceedings of the SIGCHI Conference on human factors in computing systems. pp 865–874

Harrap R, Hutchinson D, Sala Z, Ondercin M, DiFrancesco P (2019) Our GIS is a game engine: Bringing Unity to spatial simulation of rockfalls. In: GeoComputation

Harvey A (2019) Becoming gamesworkers: Diversity, higher education, and the future of the game industry. Television & New Media 20(8):756–766CrossRef

Hong Z, Deng Z, Evans R, Wu H (2020) Patient questions and physician responses in a Chinese health Q&A website: Content analysis. J Med Internet Res 22(4):e13071CrossRef

Hu H, Wang S, Bezemer CP, Hassan AE (2019) Studying the consistency of star ratings and reviews of popular free hybrid android and ios apps. Empir Softw Eng 24(1):7–32CrossRef

Jelodar H, Wang Y, Yuan C, Feng X, Jiang X, Li Y, Zhao L (2019) Latent dirichlet allocation (lda) and topic modeling: models, applications, a survey. Multimed Tools Appl 78(11):15169–15211CrossRef

Jin J, Li Y, Zhong X, Zhai L (2015) Why users contribute knowledge to online communities: An empirical study of an online social Q&A community. Inf Manag 52(7):840–849CrossRef

Kanode CM, Haddad HM (2009) Software engineering challenges in game development. In: 2009 Sixth international conference on information technology: new generations. IEEE, pp 260–265

Koksal I (2020) Video gaming industry & its revenue shift. https://www.forbes.com/sites/ilkerkoksal/2019/11/08/video-gaming-industry--its-revenue-shift/#12d74894663e. Accessed: August 14, 2020

Lin B, Zampetti F, Bavota G, Di Penta M, Lanza M (2019) Pattern-based mining of opinions in Q&A websites. In: 2019 IEEE/ACM 41st International conference on software engineering (ICSE). IEEE, pp 548–559

Linares-Vásquez M, Bavota G, Di Penta M, Oliveto R, Poshyvanyk D (2014) How do API changes trigger Stack Overflow discussions? A study on the Android SDK. In: proceedings of the 22nd International conference on program comprehension. pp 83–94

Linares-Vásquez M, Dit B, Poshyvanyk D (2013) An exploratory analysis of mobile development issues using stack overflow. In: 2013 10th Working conference on mining software repositories (MSR). IEEE, pp 93–96

Lukins SK, Kraft NA, Etzkorn LH (2010) Bug localization using latent dirichlet allocation. Inf Softw Technol 52(9):972–990CrossRef

Mamykina L, Manoim B, Mittal M, Hripcsak G, Hartmann B (2011) Design lessons from the fastest Q&A site in the west. In: Proceedings of the SIGCHI conference on Human factors in computing systems. pp 2857–2866

Mann HB, Whitney DR (1947) On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat :50–60

Messaoudi F, Ksentini A, Simon G, Bertin P (2017) Performance analysis of game engines on mobile and fixed devices. ACM Trans Multimed Comput Commun Appl (TOMM) 13(4):1–28CrossRef

Morris MR, Teevan J, Panovich K (2010) What do people ask their social networks, and why? a survey study of status message q&a behavior. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp 1739–1748

Movshovitz-Attias D, Movshovitz-Attias Y, Steenkiste P, Faloutsos C (2013) Analysis of the reputation system and user contributions on a question answering website: StackOverflow. In: 2013 IEEE/ACM International conference on advances in social networks analysis and mining (ASONAM 2013). IEEE, pp 886–893

Nadi S, Treude C (2020) Essential sentences for navigating Stack Overflow answers. In: 2020 IEEE 27th International conference on software analysis, evolution and reengineering (SANER). IEEE, pp 229–239

Parnin C, Treude C, Grammel L, Storey MA (2012) Crowd documentation: Exploring the coverage and the dynamics of API discussions on Stack Overflow. Georgia Institute of Technology, Tech. Rep 11

Pascarella L, Palomba F, Di Penta M, Bacchelli A (2018) How is video game development different from software development in open source?. In: 2018 IEEE/ACM 15th international conference on mining software repositories (MSR). IEEE, pp 392–402

Porter MF, et al. (1980) An algorithm for suffix stripping. Program 14(3):130–137CrossRef

Procaci TB, Nunes BP, Nurmikko-Fuller T, Siqueira SW (2016) Finding topical experts in question & answer communities. In: 2016 IEEE 16th international conference on advanced learning technologies (ICALT), pp. 407–411. IEEE

Procaci TB, Siqueira SW, Nunes BP, Nurmikko-Fuller T (2017) Modelling experts behaviour in Q&A communities to predict worthy discussions. In: 2017 IEEE 17th international conference on advanced learning technologies (ICALT). IEEE, pp 291–295

Ravi S, Pang B, Rastogi V, Kumar R (2014) Great question! question quality in community q&a. In: Proceedings of the international AAAI conference on web and social media, vol 8

Ray B, Posnett D, Filkov V, Devanbu P (2014) A large scale study of programming languages and code quality in github. In: Proceedings of the 22nd ACM SIGSOFT international symposium on foundations of software engineering. pp 155–165

Romano J, Kromrey JD, Coraggio J, Skowronek J, Devine L (2006) Exploring methods for evaluating group differences on the nsse and other surveys: Are the t-test and cohen’sd indices the most appropriate choices. In: Annual meeting of the Southern Association for institutional research. Citeseer, pp 1–51

Rosen C, Shihab E (2016) What are mobile developers asking about? A large scale study using Stack Overflow. Empir Softw Eng 21(3):1192–1223CrossRef

Sadowski C, Stolee KT, Elbaum S (2015) How developers search for code: A case study. In: Proceedings of the 2015 10th joint meeting on foundations of software engineering. pp 191–201

Santos T, Burghardt K, Lerman K, Helic D (2020) Can badges foster a more welcoming culture on Q&A boards?. In: Proceedings of the international AAAI conference on web and social Media, vol 14. pp 969–973

Shen X, Jia AL, Shen S, Dou Y (2020) Helping the ineloquent farmers: Finding experts for questions with limited text in agricultural Q&A communities. IEEE Access 8:62238–62247CrossRef

Somasundaram K, Murphy GC (2012) Automatic categorization of bug reports using latent dirichlet allocation. In: Proceedings of the 5th India software engineering conference. pp 125–130

Stack Overflow’s 2020 developer survey (2020) https://insights.stackoverflow.com/survey/2020. Accessed: August 14, 2020

Treude C, Barzilay O, Storey MA (2011) How do programmers ask and answer questions on the web? (NIER track). In: Proceedings of the 33rd international conference on software engineering. pp 804–807

Treude C, Robillard MP (2016) Augmenting API documentation with insights from Stack Overflow. In: 2016 IEEE/ACM 38th International conference on software engineering (ICSE). IEEE, pp 392–403

Wang S, Chen TH, Hassan AE (2018) Understanding the factors for fast answers in technical Q&A websites. Empir Softw Eng 23(3):1552–1593CrossRef

Wang S, Lo D, Jiang L (2013) An empirical study on developer interactions in StackOverflow. In: Proceedings of the 28th annual ACM symposium on applied computing, pp 1019–1024

Wang Y (2020) The price of being polite: politeness, social status, and their joint impacts on community Q&A efficiency. J Comput Soc Sci :1–22

Whitson JR (2018) Voodoo software and boundary objects in game development: How developers collaborate and conflict with game engines and art tools. New Media Soc 20(7):2315–2332CrossRef

Wilcoxon F (1992) Individual comparisons by ranking methods. In: Breakthroughs in statistics, pp. 196–202. Springer

Wu Y, Wang S, Bezemer CP, Inoue K (2019) How do developers utilize source code from Stack Overflow? Empir Softw Eng 24(2):637–673CrossRef

Yang XL, Lo D, Xia X, Wan ZY, Sun JL (2016) What security questions do developers ask? a large-scale study of stack overflow posts. J Comput Sci Technol 31(5):910–924CrossRef

Zahedi M, Rajapakse RN, Babar MA (2020) Mining questions asked about continuous software engineering: A case study of stack overflow. In: Proceedings of the evaluation and assessment in software engineering, pp 41–50

Zhang H, Wang S, Chen TH, Hassan AE (2019) Reading answers on stack overflow: Not enough!. IEEE Trans Softw Eng

Zhang H, Wang S, Chen THP, Zou Y, Hassan AE (2019) An empirical study of obsolete answers on Stack Overflow. IEEE Trans Softw Eng

Zhang X, Liu S, Chen X, et al. (2017) Social capital, motivations, and knowledge sharing intention in health Q&A communities. Manag Decis

Zhang Y, Lu T, Phang CW, Zhang C (2019) Scientific knowledge communication in online Q&A communities: Linguistic devices as a tool to increase the popularity and perceived professionalism of knowledge contribution. J Assoc Inf Syst 20(8):3

Zhou J, Wang S, Bezemer CP, Hassan AE (2020) Bounties on technical Q&A sites: a case study of Stack Overflow bounties. Empir Softw Eng 25 (1):139–177CrossRef

Title: An empirical study of Q&A websites for game developers
Authors: Arthur Kamienski
Cor-Paul Bezemer
Publication date: 01-11-2021
Publisher: Springer US
Published in: Empirical Software Engineering / Issue 6/2021
Print ISSN: 1382-3256
Electronic ISSN: 1573-7616
DOI: https://doi.org/10.1007/s10664-021-10014-4

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 6/2021

Applying test case prioritization to software microbenchmarks

An empirical study of IoT topics in IoT developer discussions on Stack Overflow

Characterizing refactoring graphs in Java and JavaScript projects

A study of how Docker Compose is used to compose multi-component systems

FACER: An API usage-based code-example recommender for opportunistic reuse

Why and what happened? Aiding bug comprehension with automated category and causal link identification

Premium Partner