Andrea Di Sorbo
Sebastiano Panichella
Gerardo Canfora
Harald C. Gall
ABSTRACT
Mobile app developers constantly monitor feedback in user reviews with the goal of improving their mobile apps and better meeting user expectations. Thus, automated approaches have been proposed in literature with the aim of reducing the effort required for analyzing feedback contained in user reviews via automatic classification/prioritization according to specific topics. In this paper, we introduce SURF (Summarizer of User Reviews Feedback), a novel approach to condense the enormous amount of information that developers of popular apps have to manage due to user feedback received on a daily basis. SURF relies on a conceptual model for capturing user needs useful for developers performing maintenance and evolution tasks. Then it uses sophisticated summarisation techniques for summarizing thousands of reviews and generating an interactive, structured and condensed agenda of recommended software changes. We performed an end-to-end evaluation of SURF on user reviews of 17 mobile apps (5 of them developed by Sony Mobile), involving 23 developers and researchers in total. Results demonstrate high accuracy of SURF in summarizing reviews and the usefulness of the recommended changes. In evaluating our approach we found that SURF helps developers in better understanding user needs, substantially reducing the time required by developers compared to manually analyzing user (change) requests and planning future software changes.
- U. Abelein, H. Sharp, and B. Paech. Does involving users in software development really influence system success? IEEE Software, 30(6):17–23, 2013. Google Scholar
Digital Library
- R. A. Baeza-Yates and B. Ribeiro-Neto. Modern Information Retrieval. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1999. Google ScholarDigital Library
- G. Bavota, M. Linares-Vasquez, C. Bernal-Cardenas, M. Di Penta, R. Oliveto, and D. Poshyvanyk. The impact of api change- and fault-proneness on the user ratings of android apps. Software Engineering, IEEE Transactions on, 41(4):384–407, April 2015.Google ScholarDigital Library
- R. Chandy and H. Gu. Identifying spam in the ios app store. In Proceedings of the 2Nd Joint WICOW/AIRWeb Workshop on Web Quality, WebQuality ’12, pages 56–59, New York, NY, USA, 2012. ACM. Google ScholarDigital Library
- N. Chen, J. Lin, S. C. H. Hoi, X. Xiao, and B. Zhang. Ar-miner: Mining informative reviews for developers from mobile app marketplace. In Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, pages 767–778, New York, NY, USA, 2014. ACM. Google ScholarDigital Library
- A. De Lucia, M. Di Penta, R. Oliveto, A. Panichella, and S. Panichella. Labeling source code with information retrieval methods: an empirical study. Empirical Software Engineering, 19(5):1383–1420, 2014. Google ScholarDigital Library
- A. Di Sorbo, S. Panichella, C. A. Visaggio, M. Di Penta, G. Canfora, and H. C. Gall. Development emails content analyzer: Intention mining in developer discussions (T). In 30th IEEE/ACM International Conference on Automated Software Engineering, ASE 2015, Lincoln, NE, USA, November 9-13, 2015, pages 12–23, 2015.Google ScholarDigital Library
- A. Di Sorbo, S. Panichella, C. A. Visaggio, M. Di Penta, G. Canfora, and H. C. Gall. DECA: development emails content analyzer. In Proceedings of the 38th International Conference on Software Engineering, ICSE 2016, Austin, TX, USA, May 14-22, 2016 - Companion Volume, pages 641–644, 2016. Google ScholarDigital Library
- B. Fu, J. Lin, L. Li, C. Faloutsos, J. Hong, and N. Sadeh. Why people hate your app: Making sense of user feedback in a mobile app store. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’13, pages 1276–1284, New York, NY, USA, 2013. ACM. Google ScholarDigital Library
- L. V. Galvis Carre˜ no and K. Winbladh. Analysis of user comments: An approach for software requirements evolution. In Proceedings of the 2013 International Conference on Software Engineering, ICSE ’13, pages 582–591, Piscataway, NJ, USA, 2013. IEEE Press. Google ScholarDigital Library
- X. Gu and S. Kim. What parts of your apps are loved by users? (T). In 30th IEEE/ACM International Conference on Automated Software Engineering, ASE 2015, Lincoln, NE, USA, November 9-13, 2015, pages 760–770, 2015.Google ScholarDigital Library
- E. Guzman and W. Maalej. How do users like this feature? a fine grained sentiment analysis of app reviews. In Requirements Engineering Conference (RE), 2014 IEEE 22nd International, pages 153–162, Aug 2014.Google ScholarCross Ref
- E. Ha and D. Wagner. Do android users write about electric sheep? examining consumer reviews in google play. In Consumer Communications and Networking Conference (CCNC), 2013 IEEE, pages 149–157, Jan 2013.Google ScholarCross Ref
- S. Haiduc, J. Aponte, L. Moreno, and A. Marcus. On the use of automated text summarization techniques for summarizing source code. In Proceedings of the International Working Conference on Reverse Engineering (WCRE), pages 35–44. IEEE, 2010. Google ScholarDigital Library
- M. Harman, Y. Jia, and Y. Zhang. App store mining and analysis: Msr for app stores. In Mining Software Repositories (MSR), 2012 9th IEEE Working Conference on, pages 108–111, June 2012. Google ScholarDigital Library
- K. Herzig, S. Just, and A. Zeller. It’s not a bug, it’s a feature: How misclassification impacts bug prediction. In Proceedings of the 2013 International Conference on Software Engineering, ICSE ’13, pages 392–401, Piscataway, NJ, USA, 2013. IEEE Press. Google ScholarDigital Library
- C. Iacob and R. Harrison. Retrieving and analyzing mobile apps feature requests from online reviews. In Proceedings of the 10th Working Conference on Mining Software Repositories, MSR ’13, pages 41–44, Piscataway, NJ, USA, 2013. IEEE Press. Google ScholarDigital Library
- C. Iacob, R. Harrison, and S. Faily. Online reviews as first class artifacts in mobile app development. In G. Memmi and U. Blanke, editors, Mobile Computing, Applications, and Services, volume 130 of Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, pages 47–53. Springer International Publishing, 2014.Google Scholar
- V. N. Inukollu, D. D. Keshamoni, T. Kang, and M. Inukollu. Factors Influencing Quality of Mobile Apps:Role of Mobile App Development Life Cycle. ArXiv e-prints, Oct. 2014.Google Scholar
- H. Khalid, E. Shihab, M. Nagappan, and A. E. Hassan. What do mobile app users complain about? IEEE Software, 32(3):70–77, 2015.Google ScholarDigital Library
- S. Krusche and B. Bruegge. User feedback in mobile development. In Proceedings of the 2Nd International Workshop on Mobile Development Lifecycle, MobileDeLi ’14, pages 25–26, New York, NY, USA, 2014. ACM. Google ScholarDigital Library
- S. A. Licorish, A. Tehir, M. F. Bosu, and S. G. MacDonell. On satisfying the android os community: User feedback still central to developers’ portfolios. In 2015 24th Australasian Software Engineering Conference (ASWEC, pages 78–87, 2015. Google ScholarDigital Library
- W. Maalej and H. Nabil. Bug report, feature request, or simply praise? on automatically classifying app reviews. In Requirements Engineering Conference (RE), 2015 IEEE 23rd International, pages 116–125, Aug 2015.Google ScholarCross Ref
- P. W. McBurney and C. McMillan. Automatic documentation generation via source code summarization of method context. In Proceedings of the International Conference on Program Comprehension (ICPC), pages 279–290. ACM, 2014. Google ScholarDigital Library
- G. A. Miller. Wordnet: A lexical database for english. Commun. ACM, 38(11):39–41, Nov. 1995. Google ScholarDigital Library
- L. Moreno, J. Aponte, G. Sridhara, A. Marcus, L. Pollock, and K. Vijay-Shanker. Automatic generation of natural language summaries for java classes. In Proceedings of the International Conference on Program Comprehension (ICPC), pages 23–32. IEEE, May 2013.Google Scholar
- G. C. Murphy. Lightweight Structural Summarization As an Aid to Software Evolution. PhD thesis, 1996. Google ScholarDigital Library
- AAI9704521.Google Scholar
- J. Oh, D. Kim, U. Lee, J.-G. Lee, and J. Song. Facilitating developer-user interactions with mobile app review digests. In CHI ’13 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’13, pages 1809–1814, New York, NY, USA, 2013. ACM. Google ScholarDigital Library
- D. Pagano and W. Maalej. User feedback in the appstore: An empirical study. In In Proceedings of the 21st IEEE International Requirements Engineering Conference (RE 2013), pages 125–134. IEEE Computer Society, 2013.Google ScholarCross Ref
- F. Palomba, M. Linares-Vasquez, G. Bavota, R. Oliveto, M. Di Penta, D. Poshyvanyk, and A. De Lucia. User reviews matter! tracking crowdsourced reviews to support evolution of successful apps. In Software Maintenance and Evolution (ICSME), 2015 IEEE International Conference on, pages 291–300, Sept 2015. Google ScholarDigital Library
- B. Pang, L. Lee, and S. Vaithyanathan. Thumbs up?: Sentiment classification using machine learning techniques. In Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing - Volume 10, EMNLP ’02, pages 79–86, Stroudsburg, PA, USA, 2002. Association for Computational Linguistics. Google ScholarDigital Library
- S. Panichella, J. Aponte, M. D. Penta, A. Marcus, and G. Canfora. Mining source code descriptions from developer communications. In Program Comprehension (ICPC), 2012 IEEE 20th International Conference on, pages 63–72, 2012.Google ScholarCross Ref
- S. Panichella, A. Di Sorbo, E. Guzman, C. Visaggio, G. Canfora, and H. Gall. How can I improve my app? classifying user reviews for software maintenance and evolution. In Software Maintenance and Evolution (ICSME), 2015 IEEE International Conference on, pages 281–290, Sept 2015. Google ScholarDigital Library
- S. Panichella, A. Panichella, M. Beller, A. Zaidman, and H. Gall. The impact of test case summaries on bug fixing performance: An empirical investigation. In Proceedings of the International Conference on Software Engineering (ICSE). IEEE, 2016. Google ScholarDigital Library
- M. F. Porter. An algorithm for suffix stripping. Program, 14(3):130–137, 1980.Google ScholarCross Ref
- O. Rambow, L. Shrestha, J. Chen, and C. Lauridsen. Summarizing email threads. In In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) Short Paper Section, 2004. Google ScholarDigital Library
- S. Rastkar, G. C. Murphy, and G. Murray. Summarizing software artifacts: A case study of bug reports. In Proceedings of the 32Nd ACM/IEEE International Conference on Software Engineering - Volume 1, ICSE ’10, pages 505–514, New York, NY, USA, 2010. ACM. Google ScholarDigital Library
- S. Rastkar, G. C. Murphy, and G. Murray. Automatic summarization of bug reports. IEEE Trans. Softw. Eng., 40(4):366–380, Apr. 2014. Google ScholarDigital Library
- F. Sarro, A. Al-Subaihin, M. Harman, Y. Jia, W. Martin, and Y. Zhang. Feature lifecycles as they spread, migrate, remain, and die in app stores. In Requirements Engineering Conference (RE), 2015 IEEE 23rd International, pages 76–85, 2015.Google ScholarCross Ref
- N. Seyff, G. Ollmann, and M. Bortenschlager. Appecho: A user-driven, in situ feedback approach for mobile platforms and applications. In Proceedings of the 1st International Conference on Mobile Software Engineering and Systems, MOBILESoft 2014, pages 99–108. ACM, 2014. Google ScholarDigital Library
- A. Sharma, Y. Tian, and D. Lo. Nirmal: Automatic identification of software relevant tweets leveraging language model. In Software Analysis, Evolution and Reengineering (SANER), 2015 IEEE 22nd International Conference on, pages 449–458, March 2015.Google Scholar
- G. Sridhara. Automatic Generation of Descriptive Summary Comments for Methods in Object-oriented Programs. PhD thesis, Newark, DE, USA, 2012. Google ScholarDigital Library
- AAI3499878.Google Scholar
- G. Sridhara, E. Hill, D. Muppaneni, L. Pollock, and K. Vijay-Shanker. Towards automatically generating summary comments for java methods. In Proceedings of the International Conference on Automated Software Engineering (ASE), pages 43–52. ACM, 2010. Google ScholarDigital Library
- C. Vassallo, S. Panichella, M. Di Penta, and G. Canfora. Codes: Mining source code descriptions from developers discussions. In Proceedings of the 22Nd International Conference on Program Comprehension, ICPC 2014, pages 106–109, New York, NY, USA, 2014. ACM. Google ScholarDigital Library
- T. Vithani. Modeling the mobile application development lifecycle. In Proceedings of the International MultiConference of Engineers and Computer Scientists 2014, Vol. I, IMECS 2014, pages 596–600, 2014.Google Scholar
- P. M. Vu, T. T. Nguyen, H. V. Pham, and T. T. Nguyen. Mining user opinions in mobile app reviews: A keyword-based approach. CoRR, abs/1505.04657, 2015.Google Scholar
- Z. Wu and M. Palmer. Verbs semantics and lexical selection. In Proceedings of the 32Nd Annual Meeting on Association for Computational Linguistics, ACL ’94, pages 133–138, Stroudsburg, PA, USA, 1994. Association for Computational Linguistics. Google ScholarDigital Library
Index Terms
- What would users change in my app? summarizing app reviews for recommending software changes
Recommendations
AR-miner: mining informative reviews for developers from mobile app marketplace
ICSE 2014: Proceedings of the 36th International Conference on Software EngineeringWith the popularity of smartphones and mobile devices, mobile application (a.k.a. “app”) markets have been growing exponentially in terms of number of users and downloads. App developers spend considerable effort on collecting and exploiting user ...
Mining User Reviews for Mobile App Comparisons
As the number of mobile apps keeps increasing, users often need to compare many apps, in order to choose one that best fits their needs. Fortunately, as there are so many users sharing an app market, it is likely that some other users with the same ...
Do Android app users care about accessibility?: an analysis of user reviews on the Google play store
IHC '19: Proceedings of the 18th Brazilian Symposium on Human Factors in Computing SystemsMobile applications (or apps) have become a popular platform to provide users with information, digital content and services. Unfortunately, many studies have shown that even popular mobile applications still have many accessibility issues. A particular ...
Comments