Top

Empirical Software Engineering

Published in:

01-03-2022

A study of gender in user reviews on the Google Play Store

Authors: Ehsan Noei, Kelly Lyons

Published in: Empirical Software Engineering | Issue 2/2022

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

User reviews that are posted on the Google Play Store provide app developers with important information such as bug reports, feature requests, and user experience. Developers should maintain their apps while taking user feedback into account to succeed in the competitive market of mobile apps. the Google Play Store provides a star-rating mechanism for users to rate apps on a scale of one to five. Apps that are ranked higher and have higher star ratings are more likely to be downloaded. In this paper, we investigate and compare men’s and women’s participation in user reviews that are posted on the Google Play Store. We analyze 438,707 user reviews of the top 156 Android apps over six months. We find that women give higher star ratings and use more positive sentiment in their reviews than men. Furthermore, women’s reviews receive more likes and are ranked higher in the top 10 by the Google Play Store. For the reviews from which user gender can be inferred, we find that men submit more reviews than women, making reviews by men more likely to be visible to app developers and other users. Past research has shown that app developers respond more to negative reviews with fewer stars. We found that developers respond to a greater percentage of men’s reviews than women’s. The small number of and more positive reviews by women are less likely to be addressed by app developers; thus, the resulting changes to apps will align more with the needs of men users, perhaps causing even less participation by women in the Google Play Store reviews. Our findings suggest that developers should take gender into consideration when responding to reviews to help mitigate a feedback loop of bias. Our observations also suggest a need for future research in this area to understand the motivations of men and women in reviewing apps and how developers respond to reviews.

previous article Two N-of-1 self-trials on readability differences between anonymous inner classes (AICs) and lambda expressions (LEs) on Java code snippets

next article Gamification in software engineering: the mediating role of developer engagement and job satisfaction

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Argamon S, Koppel M, Fine J, Shimoni A R (2003) Gender, genre, and writing style in formal written texts. Text-The Hague Then Amsterdam Then Berlin 23(3):321–346

Asiri O, Chang CK (2018) Investigating users’ experiences and attitudes towards mobile apps’ reviews. In: International Conference on Human-Computer Interaction. Springer, pp 481–499

Baek Y (2013) Analysis of user’s attitude toward apps, intention to use and continual consuming intention-focused on mobile commerce. Int J Content 9(4):35–44CrossRef

Bakar N S A A, Mahmud I (2013) Empirical analysis of android apps permissions. In: Proceedings of the 2013 International Conference on Advanced Computer Science Applications and Technologies (ACSAT). IEEE, pp 406–411

Basili V R (1992) Software modeling and measurement: the goal/question/metric paradigm. Technical report, Institute for advanced computer studies

Bavota G, Linares-Vasquez M, Bernal-Cardenas C E, Penta M D, Oliveto R, Poshyvanyk D (2015) The impact of api change-and fault-proneness on the user ratings of android apps. IEEE Trans Softw Eng 41(4):384–407CrossRef

Benjamini Y, Yekutieli D (2001) The control of the false discovery rate in multiple testing under dependency. Ann stat:1165–1188

Blei D M, Ng A Y, Jordan M I (2003) Latent dirichlet allocation. J Mach Learn Res 3:993–1022MATH

Bolboacȧ S D, Jäntschi L, Sestraṡ A F, Sestraṡ R E, Pamfil D C (2011) Pearson-fisher chi-square statistic revisited. Information 2(3):528–545CrossRef

Bonnington C (2013) Are men and women using mobile apps differently? do men and women have appreciably different tastes in apps? [Online]. Available: https://www.wired.com/2013/04/men-women-app-usage/

Burger JD, Henderson J, Kim G, Zarrella G (2011) Discriminating gender on twitter. In: Proceedings of the conference on empirical methods in natural language processing, Association for Computational Linguistics, pp 1301–1309

Collier B, Bear J (2012) Conflict, criticism, or confidence: an empirical examination of the gender gap in wikipedia contributions. In: Proceedings of the ACM 2012 conference on computer supported cooperative work, pp 383–392

De Marneffe M C, MacCartney B, Manning C D, et al. (2006) Generating typed dependency parses from phrase structure parses. In: 5Th international conference on language resources and evaluation, vol 6, pp 449–454

Deveaud R, SanJuan E, Bellot P (2014) Accurate and effective latent concept modeling for ad hoc information retrieval. Doc Numér 17(1):61–84CrossRef

Fink C, Kopecky J, Morawski M (2012) Inferring gender from the content of tweets: a region specific example. In: Sixth International AAAI Conference on Weblogs and Social Media

Foltz PW, Laham D, Landauer TK (1999) Automated essay scoring: Applications to educational technology. In: EdMedia+ innovate learning, Association for the Advancement of Computing in Education (AACE), pp 939–944

Ford D, Harkins A, Parnin C (2017) Someone Like me: How does peer parity influence participation of women on stack overflow? In: 2017 IEEE Symposium on visual languages and human-centric computing (VL/HCC). IEEE, pp 239–243

Fu B, Lin J, Li L, Faloutsos C, Hong J, Sadeh N (2013) Why people hate your app: Making sense of user feedback in a mobile app store. In: 19th International Conference on Knowledge Discovery and Data Mining. ACM, pp 1276–1284

Galvis Carreño LV, Winbladh K (2013) Analysis of user comments: an approach for software requirements evolution. In: 35th International Conference on Software Engineering. IEEE, pp 582–591

Genderize (2019) Genderize. [Online]. Available: http://www.genderize.io/

Google (2020) Google play store categories. [Online]. Available: https://support.google.com/googleplay/android-developer/answer/113475

Guzman E, Maalej W (2014) How do users like this feature? a fine grained sentiment analysis of app reviews. In: 22nd International Conference on Requirements Engineering. IEEE, pp 153–162

Hannák A, Wagner C, Garcia D, Mislove A, Strohmaier M, Wilson C (2017) Bias in online freelance marketplaces: Evidence from taskrabbit and fiverr. In: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, pp 1914–1933

Harman M, Jia Y, Zhang Y (2012) App store mining and analysis: Msr for app stores. In: 9th International Conference on Mining Software Repositories, MSR ’12. IEEE, Piscataway, pp 108–111

Harrell F E (2001) Regression modeling strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. Springer

Hemphill L, Otterbacher J (2012) Learning the lingo? gender, prestige and linguistic adaptation in review communities. In: Proceedings of the ACM 2012 conference on computer supported cooperative work, pp 305–314

Hmisc (2020) Harrell miscellaneous. [Online]. Available: http://cran.r-project.org/web/packages/Hmisc/index.html

Hong C, Chen ZF, Li C (2017) ``liking” and being ``liked”: How are personality traits and demographics associated with giving and receiving ``likes” on facebook? Comput Human Behav 68:292–299

Iacob C, Harrison R (2013) Retrieving and analyzing mobile apps feature requests from online reviews. In: 10th Working Conference on Mining Software Repositories, MSR ’13. IEEE, pp 41–44

Ibrahim H, Abdel-Razig S, Stadler DJ, Cofrancesco J, Archuleta S (2019) Assessment of gender equity among invited speakers and award recipients at us annual medical education conferences. JAMA Netw Open 2(11):e1916222–e1916222

Islam MR, Zibran MF (2017) Leveraging automated sentiment analysis in software engineering. In: 14th International Conference on Mining Software Repositories. IEEE Press, pp 203–214

Jazzy (2017) Jazzy spell checker. [Online]. Available: http://jazzy.sourceforge.net/

Karimi F, Wagner C, Lemmerich F, Jadidi M, Strohmaier M (2016) Inferring gender from names on the web: A comparative evaluation of gender detection methods. In: Proceedings of the 25th International Conference Companion on World Wide Web, pp 53–54

Khalid H, Nagappan M, Shihab E, Hassan AE (2014a) Prioritizing the devices to test your app on: A case study of android game apps. In: 22nd International Symposium on the Foundations of Software Engineering, pp 370–379

Khalid H, Shihab E, Nagappan M, Hassan A E (2014b) What do mobile app users complain about? IEEE Softw 32(3):70–77CrossRef

Kim HW, Lee H, Son J (2011a) An exploratory study on the determinants of smartphone app purchase. In: 11th International DSI and the 16th APDSI Joint Meeting

Kim HW, Lee H, Son J (2011b) An exploratory study on the determinants of smartphone app purchase. In: Proceedings of the 11th International Decision Science Institute and the 16th Asia Pacific Decision Sciences Institute Joint Meeting

Kim SM, Pantel P, Chklovski T, Pennacchiotti M (2006) Automatically assessing review helpfulness. In: 2006 Conference on empirical methods in natural language processing, Association for Computational Linguistics, pp 423–430

Kübler R, Pauwels K, Yildirim G, Fandrich T (2018) App popularity: Where in the world are consumers most sensitive to price and user ratings? J Mark 82(5):20–44CrossRef

Lakoff G (1975) Hedges: A study in meaning criteria and the logic of fuzzy concepts. In: Contemporary research in philosophical logic and linguistic semantics. Springer, pp 221–271

Lakoff R (1973) Language and woman’s place. Lang Soc 2(1):45–79MathSciNetCrossRef

Linares-Vásquez M, Bavota G, Bernal-Cárdenas C, Di Penta M, Oliveto R, Poshyvanyk D (2013) Api change and fault proneness: A threat to the success of android apps. In: 9th Joint Meeting on Foundations of Software Engineering. ACM, pp 477–487

Lovins J B (1968) Development of a stemming algorithm. MIT Information Processing Group, Electronic Systems Laboratory

Manning CD, Surdeanu M, Bauer J, Finkel J, Bethard SJ, McClosky D (2014) The stanford corenlp natural language processing toolkit. In: 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp 55–60

Martin W, Harman M, Jia Y, Sarro F, Zhang Y (2015) The app sampling problem for app store mining. In: 12th Working Conference on Mining Software Repositories. IEEE, pp 123–133

Martin W, Sarro F, Jia Y, Zhang Y, Harman M (2016) A survey of app store analysis for software engineering. IEEE Trans Softw Eng PP(99)

May A, Wachs J, Hannák A (2019) Gender differences in participation and reward on stack overflow. Empir Softw Eng 24(4):1997–2019CrossRef

McClave J T, Sincich T (2006) Statistics: Technology manual [and CD-ROM]. Pearson, Prentice Hall

McMillan C, Poshyvanyk D, Grechanik M, Xie Q, Fu C (2013) Portfolio: Searching for relevant functions and their usages in millions of lines of code. ACM Trans Softw Eng Methodol (TOSEM) 22(4):1–30CrossRef

Merchant A, Shah D, Bhatia GS, Ghosh A, Kumaraguru P (2019) Signals matter: understanding popularity and impact of users on stack overflow. In: The World Wide Web Conference, pp 3086–3092

Mukherjee S, Bala P K (2017) Gender classification of microblog text based on authorial style. IseB 15(1):117–138CrossRef

Nguyen TH, Adams B, Hassan AE (2010) Studying the impact of dependency network measures on software quality. In: Proceedings of the 26th International Conference on Software Maintenance. IEEE, pp 1–10

Niu H, Keivanloo I, Zou Y (2017) Learning to rank code examples for code search engines. Empir Softw Eng 22(1):259–291CrossRef

Noei E (2018) Succeeding in mobile application markets (from development point of view). PhD thesis, Queen’s University, Canada

Noei E, Lyons K (2019) A survey of utilizing user-reviews posted on Google Play Store. In: Proceedings of the 29th Annual International Conference on Computer Science and Software Engineering, pp 54–63

Noei E, Da Costa D A, Zou Y (2018) Winning the app production rally. In: 26Th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, ESEC/FSE. ACM, New York, pp 283–294

Noei E, Zhang F, Wang S, Zou Y (2019a) Towards prioritizing user-related issue reports of mobile applications. Empir Softw Eng 24(4):1964–1996CrossRef

Noei E, Zhang F, Zou Y (2019b) Too many user-reviews, what should app developers look at first? IEEE Transactions on Software Engineering

Nord C (2005) Text analysis in translation: Theory, methodology, and didactic application of a model for translation-oriented text analysis, pp 94. Rodopi

Otterbacher J (2010) Inferring gender of movie reviewers: exploiting writing style, content and metadata. In: Proceedings of the 19th ACM international conference on Information and knowledge management, pp 369–378

Palomba F, Linares-Vásquez M, Bavota G, Oliveto R, Di Penta M, Poshyvanyk D, De Lucia A (2015) User reviews matter! tracking crowdsourced reviews to support evolution of successful apps. In: 31st International Conference on Software Maintenance and Evolution. IEEE, pp 291–300

Panichella A, Dit B, Oliveto R, Di Penta M, Poshynanyk D, De Lucia A (2013) How to effectively use topic models for software engineering tasks? an approach based on genetic algorithms. In: 2013 35th International Conference on Software Engineering (ICSE). IEEE, pp 522–531

Panichella S, Di Sorbo A, Guzman E, Visaggio CA, Canfora G, Gall HC (2016) Ardoc: app reviews development oriented classifier. In: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, pp 1023–1027

Pinheiro J, Bates D, DebRoy S, Sarkar D, et al. (2007) Linear and nonlinear mixed effects models. R package version 3:57

Raftery A E, Lewis S, et al. (1992) How many iterations in the gibbs sampler. Bayesian Stat 4(2):763–773

Rajaraman A, Ullman JD (2012) Mining of massive datasets, vol 77. Cambridge University Press, Cambridge

Ruiz I J M, Nagappan M, Adams B, Berger T, Dienst S, Hassan A E (2016) Examining the rating system used in mobile-app stores. IEEE Softw 33(6):86–92CrossRef

Scheuerman M K, Spiel K, Haimson O L, Hamidi F, Branham SM (2019) Hci guidelines for gender equity and inclusivity

Shull F, Singer J, Sjøberg DI (2007) Guide to Advanced Empirical Software Engineering. Springer, New York

Smith BN, Singh M, Torvik VI (2013) A search engine approach to estimating temporal changes in gender orientation of first names. In: Proceedings of the 13th ACM/IEEE-CS joint conference on Digital libraries, pp 199–208

Statista (2020) Number of apps available in leading app stores as of march 2017. [Online]. Available: https://www.statista.com/statistics/276623/number-of-apps-available-in-leading-app-stores/

Stephens M (2013) Gender and the geoweb: divisions in the production of user-generated cartographic information. GeoJournal 78(6):981–996CrossRef

Terrell J, Kofink A, Middleton J, Rainear C, Murphy-Hill E, Parnin C, Stallings J (2017) Gender differences and bias in open source: Pull request acceptance of women versus men. PeerJ Comput Sci 3:e111CrossRef

Tian Y, Nagappan M, Lo D, Hassan AE (2015) What are the characteristics of high-rated apps? a case study on free android applications. In: Proceedings of the 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, pp 301–310

Topaz C M, Sen S (2016) Gender representation on journal editorial boards in the mathematical sciences. PLoS One 11(8):e0161357

Ugoni A, Walker B F (1995) The chi square test: an introduction. COMSIG Rev 4(3):61

Van Solingen R, Basili V, Caldiera G, Rombach HD (2002) Goal question metric (gqm) approach. Encyclopedia of software engineering

Vasa R, Hoon L, Mouzakis K, Noguchi A (2012) A preliminary analysis of mobile app user reviews. In: Proceedings of the 24th Australian Computer-Human Interaction Conference, pp 241–244

Vasilescu B, Capiluppi A, Serebrenik A (2014) Gender, representation and online participation: a quantitative study. Interact Comput 26(5):488–511CrossRef

Villarroel L, Bavota G, Russo B, Oliveto R, Di Penta M (2016) Release planning of mobile apps based on user reviews. In: 38th International Conference on Software Engineering. ACM, pp 14–24

Wachs J, Hannak A, Vörös A, Daróczy B (2017) Why do men get more attention? exploring factors behind success in an online design community. In: Eleventh International AAAI Conference on Web and Social Media

Wagner C, Graells-Garrido E, Garcia D, Menczer F (2016) Women through the glass ceiling: gender asymmetries in wikipedia. EPJ Data Sci 5(1):5CrossRef

Wais K (2016) Gender Prediction Methods Based on First Names with genderizeR. R J 8(1):17–37CrossRef

Weisberg S (2005) Applied linear regression, vol 528. Wiley

Title: A study of gender in user reviews on the Google Play Store
Authors: Ehsan Noei
Kelly Lyons
Publication date: 01-03-2022
Publisher: Springer US
Published in: Empirical Software Engineering / Issue 2/2022
Print ISSN: 1382-3256
Electronic ISSN: 1573-7616
DOI: https://doi.org/10.1007/s10664-021-10080-8

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 2/2022

Modeling Performance of Microservices Systems with Growth Theory

Analysis of a many-objective optimization approach for identifying microservices from legacy systems

Software testing and Android applications: a large-scale empirical study

Path context augmented statement and network for learning programs

A teamwork effectiveness model for agile software development

An extended study on applicability and performance of homogeneous cross-project defect prediction approaches under homogeneous cross-company effort estimation situation

Premium Partner