Cuiyun Gao
ABSTRACT
Detecting emerging issues (e.g., new bugs) timely and precisely is crucial for developers to update their apps. App reviews provide an opportunity to proactively collect user complaints and promptly improve apps' user experience, in terms of bug fixing and feature refinement. However, the tremendous quantities of reviews and noise words (e.g., misspelled words) increase the difficulties in accurately identifying newly-appearing app issues. In this paper, we propose a novel and automated framework IDEA, which aims to IDentify Emerging App issues effectively based on online review analysis. We evaluate IDEA on six popular apps from Google Play and Apple's App Store, employing the official app changelogs as our ground truth. Experiment results demonstrate the effectiveness of IDEA in identifying emerging app issues. Feedback from engineers and product managers shows that 88.9% of them think that the identified issues can facilitate app development in practice. Moreover, we have successfully applied IDEA to several products of Tencent, which serve hundreds of millions of users.
- Loulwah AlSumait, Daniel Barbará, and Carlotta Domeniconi. "On-line LDA: Adaptive Topic Models for Mining Text Streams with Applications to Topic Detection and Tracking". In: Proceedings of the 8th IEEE International Conference on Data Mining, ICDM 2008, December 15--19, 2008, Pisa, Italy. 2008, pp. 3--12. Google Scholar
Digital Library
- App Annie. https://www.appannie.com/en/.Google Scholar
- R. Arun et al. "On Finding the Natural Number of Topics with Latent Dirichlet Allocation: Some Observations". In: Advances in Knowledge Discovery and Data Mining, 14th Pacific-Asia Conference, PAKDD 2010, Hyderabad, India, June 21--24, 2010. Proceedings. Part I. 2010, pp. 391--402. Google ScholarDigital Library
- David M. Blei, Andrew Y. Ng, and Michael I. Jordan. "Latent Dirichlet Allocation". In: Journal of Machine Learning Research 3 (2003), pp. 993--1022. Google ScholarDigital Library
- Mario Cataldi, Luigi Di Caro, and Claudio Schifanella. "Emerging topic detection on twitter based on temporal and social terms evaluation". In: Proceedings of the Tenth International Workshop on Multimedia Data Mining (MDMKDD). ACM. 2010, p. 4. Google ScholarDigital Library
- Ning Chen et al. "AR-miner: mining informative reviews for developers from mobile app marketplace". In: 36th International Conference on Software Engineering, ICSE 2014, Hyderabad, India - May 31 - June 07, 2014. 2014, pp. 767--778. Google ScholarDigital Library
- Yan Chen et al. "Emerging topic detection for organizations from microblogs". In: The 36th International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2013, Dublin, Ireland - July 28 - August 01, 2013. 2013, pp. 43--52. Google ScholarDigital Library
- Facebook Messenger is getting slammed by tons of negative reviews. http://www.businessinsider.com/facebook-messenger-app-store-reviews-are-humiliating-2014-8.Google Scholar
- Bin Fu et al. "Why people hate your app: making sense of user feedback in a mobile app store". In: The 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2013, Chicago, IL, USA, August 11--14, 2013. 2013, pp. 1276--1284. Google ScholarDigital Library
- Cuiyun Gao et al. "AR-Tracker: Track the Dynamics of Mobile Apps via User Review Mining". In: 2015 IEEE Symposium on Service-Oriented System Engineering, SOSE 2015, San Francisco Bay, CA, USA, March 30 - April 3, 2015. 2015, pp. 284--290. Google ScholarDigital Library
- Cuiyun Gao et al. "PAID: Prioritizing app issues for developers by tracking user reviews over versions". In: 26th IEEE International Symposium on Software Reliability Engineering, ISSRE 2015, Gaithersbury, MD, USA, November 2--5, 2015. 2015, pp. 35--45. Google ScholarDigital Library
- Judith Gebauer, Ya Tang, and Chaiwat Baimai. "User requirements of mobile technology: results from a content analysis of user reviews". In: Inf. Syst. E-Business Management 6.4 (2008), pp. 361--384.Google ScholarCross Ref
- Xiaodong Gu and Sunghun Kim. ""What Parts of Your Apps are Loved by Users?" (T)". In: 30th IEEE/ACM International Conference on Automated Software Engineering, ASE 2015, Lincoln, NE, USA, November 9--13, 2015. 2015, pp. 760--770.Google ScholarDigital Library
- Emitza Guzman and Walid Maalej. "How Do Users Like This Feature? A Fine Grained Sentiment Analysis of App Reviews". In: IEEE 22nd International Requirements Engineering Conference, RE 2014, Karlskrona, Sweden, August 25--29, 2014. 2014, pp. 153--162.Google ScholarCross Ref
- Leonard Hoon et al. "An analysis of the mobile app review landscape: trends and implications". In: Faculty of Information and Communication Technologies, Swinburne University of Technology, Tech. Rep (2013).Google Scholar
- Jiajia Huang et al. "A probabilistic method for emerging topic tracking in Microblog stream". In: World Wide Web 20.2 (2017), pp. 325--350. Google ScholarDigital Library
- Claudia Iacob and Rachel Harrison. "Retrieving and analyzing mobile apps feature requests from online reviews". In: Proceedings of the 10th Working Conference on Mining Software Repositories, MSR 2013, San Francisco, CA, USA, May 18--19, 2013. 2013, pp. 41--44. Google ScholarDigital Library
- Aminul Islam and Diana Zaiu Inkpen. "Semantic text similarity using corpus-based word similarity and string similarity". In: TKDD 2.2 (2008), 10:1--10:25. Google ScholarDigital Library
- Jensen Shannon divergence. https://en.wikipedia.org/wiki/Jensen-Shannon_divergence.Google Scholar
- H Khalid et al. "What do mobile app users complain about? A study on free iOS apps. 2014". In: IEEE Software 10 (2015).Google Scholar
- Jieun Kim et al. "Trends and relationships of smartphone application services: Analysis of apple app store using text mining-based network analysis". In: Proceedings of the 4th ISPIM Innovation Symposium. 2012.Google Scholar
- Jey Han Lau, Nigel Collier, and Timothy Baldwin. "On-line Trend Analysis with Topic Models: \#twitter Trends Detection Topic Model Online". In: COLING 2012, 24th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, 8--15 December 2012, Mumbai, India. 2012, pp. 1519--1534.Google Scholar
- LDA on small datasets. https://stats.stackexchange.com/questions/78926/at-what-point-does-lda-latent-dirichlet-allocation-not-make-sense-to-use.Google Scholar
- Qingwei Lin et al. "iDice: problem identification for emerging issues". In: Proceedings of the 38th International Conference on Software Engineering, ICSE 2016, Austin, TX, USA, May 14--22, 2016. 2016, pp. 214--224. Google ScholarDigital Library
- Walid Maalej and Hadeer Nabil. "Bug report, feature request, or simply praise? On automatically classifying app reviews". In: 23rd IEEE International Requirements Engineering Conference, RE 2015, Ottawa, ON, Canada, August 24--28, 2015. 2015, pp. 116--125.Google ScholarCross Ref
- Yichuan Man et al. "Experience Report: Understanding Cross-Platform App Issues from User Reviews". In: 27th IEEE International Symposium on Software Reliability Engineering, ISSRE 2016, Ottawa, ON, Canada, October 23--27, 2016. 2016, pp. 138--149.Google Scholar
- William Martin, Federica Sarro, and Mark Harman. "Causal impact analysis for app releases in google play". In: Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016, Seattle, WA, USA, November 13--18, 2016. 2016, pp. 435--446. Google ScholarDigital Library
- William Martin et al. "A Survey of App Store Analysis for Software Engineering". In: IEEE Trans. Software Eng. 43.9 (2017), pp. 817--847.Google ScholarDigital Library
- Stuart McIlroy et al. "Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews". In: Empirical Software Engineering 21.3 (2016), pp. 1067--1106. Google ScholarDigital Library
- Qiaozhu Mei, Xuehua Shen, and ChengXiang Zhai. "Automatic labeling of multinomial topic models". In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Jose, California, USA, August 12--15, 2007. 2007, pp. 490--499. Google ScholarDigital Library
- Tomas Mikolov et al. "Distributed Representations of Words and Phrases and their Compositionality". In: Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5--8, 2013, Lake Tahoe, Nevada, United States. 2013, pp. 3111--3119. Google ScholarDigital Library
- Multi-tasking in iOS. https://developer.apple.com/ios/human-interface-guidelines/features/multitasking/.Google Scholar
- Thanh-Son Nguyen, Hady Wirawan Lauw, and Panayiotis Tsaparas. "Review Synthesis for Micro-Review Summarization". In: Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, WSDM 2015, Shanghai, China, February 2--6, 2015. 2015, pp. 169--178. Google ScholarDigital Library
- NLTK. http://www.nltk.org.Google Scholar
- PMI. https://en.wikipedia.org/wiki/Pointwise_mutual_information.Google Scholar
- Punkt tokenizer. http://www.nltk.org/modules/nltk/tokenize/punkt.html.Google Scholar
- Peter J. Rousseeuw and Mia Hubert. "Robust statistics for outlier detection". In: Wiley Interdisc. Rew.: Data Mining and Knowledge Discovery 1.1 (2011), pp. 73--79.Google ScholarCross Ref
- Federica Sarro et al. "Feature lifecycles as they spread, migrate, remain, and die in App Stores". In: 23rd IEEE International Requirements Engineering Conference, RE 2015, Ottawa, ON, Canada, August 24--28, 2015. 2015, pp. 76--85.Google Scholar
- Softmax function. https://en.wikipedia.org/wiki/Softmax_function.Google Scholar
- Andrea Di Sorbo et al. "What would users change in my app? summarizing app reviews for recommending software changes". In: Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016, Seattle, WA, USA, November 13--18, 2016. 2016, pp. 499--510. Google ScholarDigital Library
- User forum of Youtube iOS. https://productforums.google.com/forum/#!forum/youtube.Google Scholar
- Phong Minh Vu et al. "Mining User Opinions in Mobile App Reviews: A Keyword-Based Approach (T)". In: 30th IEEE/ACM International Conference on Automated Software Engineering, ASE 2015, Lincoln, NE, USA, November 9--13, 2015. 2015, pp. 749--759.Google Scholar
- Xiaojun Wan and Tianming Wang. "Automatic Labeling of Topic Models Using Text Summaries". In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7--12, 2016, Berlin, Germany, Volume 1: Long Papers. 2016.Google Scholar
- WizNote. https://www.wiz.cn/.Google Scholar
- Weizhong Zhao et al. "A heuristic approach to determine an appropriate number of topics in topic modeling". In: BMC Bioinformatics 16.13 (2015), S8.Google ScholarCross Ref
Index Terms
- Online app review analysis for identifying emerging issues
Recommendations
Crowdsourced App Review Manipulation
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information RetrievalWith the rapid adoption of smartphones worldwide and the reliance on app marketplaces to discover new apps, these marketplaces are critical for connecting users with apps. And yet, the user reviews and ratings on these marketplaces may be strategically ...
Can app changelogs improve requirements classification from app reviews?: an exploratory study
ESEM '18: Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement[Background] Recent research on mining app reviews for software evolution indicated that the elicitation and analysis of user requirements can benefit from supplementing user reviews by data from other sources. However, only a few studies reported ...
Emerging app issue identification from user feedback: experience on WeChat
ICSE-SEIP '19: Proceedings of the 41st International Conference on Software Engineering: Software Engineering in PracticeIt is vital for popular mobile apps with large numbers of users to release updates with rich features while keeping stable user experience. Timely and accurately locating emerging app issues can greatly help developers to maintain and update apps. User ...
Comments