Top

Empirical Software Engineering

Published in:

04-05-2020

Characterizing the evolution of statically-detectable performance issues of Android apps

Authors: Teerath Das, Massimiliano Di Penta, Ivano Malavolta

Published in: Empirical Software Engineering | Issue 4/2020

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Mobile apps are playing a major role in our everyday life, and they are tending to become more and more complex and resource demanding. Because of that, performance issues may occur, disrupting the user experience or, even worse, preventing an effective use of the app. Ultimately, such problems can cause bad reviews and influence the app success. Developers deal with performance issues thorough dynamic analysis, i.e., performance testing and profiler tools, albeit static analysis tools can be a valid, relatively inexpensive complement for the early detection of some such issues. This paper empirically investigates how potential performance issues identified by a popular static analysis tool — Android Lint — are actually resolved in 316 open source Android apps among 724 apps we analyzed. More specifically, the study traces the issues detected by Android Lint since their introduction until they resolved, with the aim of studying (i) the overall evolution of performance issues in apps, (ii) the proportion of issues being resolved, as well as (iii) the distribution of their survival time, and (iv) the extent to which issue resolution are documented by developers in commit messages. Results indicate how some issues, especially related to the lack of resource recycle, tend to be more frequent than others. Also, while some issues, primarily of algorithmic nature, tend to be resolved quickly through well-known patterns, others tend to stay in the app longer, or not to be resolved at all. Finally, we found how only 10% of the issue resolution is documented in commit messages.

previous article What do Programmers Discuss about Deep Learning Frameworks

next article What am I testing and where? Comparing testing procedures based on lightweight requirements annotations

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

ACT. Act state of the app enconomy 2018, 2018.

Findbugs™- find bugs in java programs. http://findbugs.sourceforge.net/.

PMD - an extensible cross-language static code analyzer. https://pmd.github.io/.

Android studio project site - Android Lint. http://tools.android.com/tips/lint.

Findbugs™- find bugs in java programs. http://findbugs.sourceforge.net/.

Soot - A framework for analyzing and transforming Java and Android applications. https://sable.github.io/soot/.

At the time of writing only FindBugs also provides a dedicated category for performance-related issues, but it is not specific to Android apps.

https://github.com/S2-group/AndroidPerformanceIssues

https://github.com/features

https://developer.github.com/v3/

F-Droid - Free and Open Source Android App Repository. https://f-droid.org/en/.

At the time of writing this paper (June 2018) the search functionality on FDroid appears to be broken or not working. Furthermore, the https://f-droid.org/forums/search/ endpoint that was used in the mining script does not exist anymore.

Wikipedia page on open-source Android apps, 2017.

https://developer.android.com/guide/topics/manifest/manifest-intro.html

This check is sound since the web page of an app in the Google Play store follows a fixed pattern, i.e., https://play.google.com/store/apps/details?id=[app package name].

Step 5 and Step 7 are redundant, we deliberately decided to keep both of them because during the execution of the dataset creation process we had to experiment with different heuristics in Step 7 and having it as a stand-alone step within the pipeline helped us in easily run it in isolation.

http://developer.android.com/ndk

We take advantage of the fact that issues in Android Lint reports are tagged with a fixed set of categories (http://tools.android.com/tips/lint-checks) like performance, correctness, accessibility, usability, etc.

https://muhammad-asaduzzaman.com/research

http://tools.android.com/tips/lint-checks

Android floatmath documentation. https://developer.android.com/reference/android/util/FloatMath.

In the remainder of the study we will refer to performance issues as P and to lines of code as LOC.

It is important to note that we performed the statistical analysis for RQ₃ both with and without outliers and the conclusions did not change

https://git-scm.com/docs/git-log

Android studio project site. http://tools.android.com/tips/lint.

Sparsearray vs hashmap. http://www.sable.mcgill.ca/soot/.

Android lint checks.

Ahmed TM, Bezemer C-P, Chen T-H, Hassan AE, Shang W (2016) Studying the effectiveness of application performance management (apm) tools for detecting performance regressions for web applications: an experience report. In: Proceedings of the 13th international conference on mining software repositories. ACM, pp 1–12

Asaduzzaman M, Roy CK, Schneider KA, Di Penta M (2013a) Lhdiff: a language-independent hybrid approach for tracking source code lines. In: 2013 29th IEEE International conference on software maintenance (ICSM). IEEE, pp 230–239

Asaduzzaman M, Roy CK, Schneider KA, Di Penta M (2013b) Lhdiff: tracking source code lines to support software maintenance activities. In: 2013 29th IEEE International conference on software maintenance (ICSM). IEEE, pp 484–487

Ayewah N, Pugh W, Hovemeyer D, Morgenthaler JD, Penix J (2008) Using static analysis to find bugs. IEEE Softw 25(5):22–29CrossRef

Basili VR, Caldiera G, Rombach HD (1994) The goal question metric approach. In: Encyclopedia of software engineering. Wiley

Bessey A, Block K, Chelf B, Chou A, Fulton B, Hallem S, Henri-Gros C, Kamsky A, McPeak S, Engler D (2010) A few billion lines of code later: using static analysis to find bugs in the real world. Commun ACM 53(2):66–75CrossRef

Chatzigeorgiou A, Manakos A (2014) Investigating the evolution of code smells in object-oriented systems. ISSE 10(1):3–18

Cohen J (1968) Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychol Bull 70(4):213CrossRef

Couto C, Montandon JE, Silva C, Valente MT (2011) Static correspondence and correlation between field defects and warnings reported by a bug finding tool. Softw Qual J 21:241–257CrossRef

Cruz L, Abreu R (2017) Performance-based guidelines for energy efficient mobile applications. In: 4th IEEE/ACM International conference on mobile software engineering and systems, MOBILESoft@ICSE 2017, Buenos Aires, Argentina, May 22-23, 2017, pp 46–57

Cruz L, Abreu R (2018) Using automatic refactoring to improve energy efficiency of Android apps, arXiv:https://arxiv.org/abs/1803.05889

Das T, Di Penta M, Malavolta I (2016) A quantitative and qualitative investigation of performance-related commits in Android apps. In: ICSME ’16 Proceedings of the 32nd international conference on software maintenance and evolution. IEEE, pp 443–448

Di Penta M, Cerulo L, Aversano L (2009) The life and death of statically detected vulnerabilities: an empirical study. Inform Softw Technol 51(10):1469–1484CrossRef

Dunn O (1964) Multiple comparisons using rank sums. Technometrics 6:241–252

Foo KC, Jiang ZMJ, Adams B, Hassan AE, Zou Y, Flora P (2015) An industrial case study on the automated detection of performance regressions in heterogeneous environments. In: Proceedings of the 37th international conference on software engineering, vol 2. IEEE Press, pp 159–168

Gómez M, Rouvoy R, Adams B, Seinturier L (2016) Mining test repositories for automatic detection of UI performance regressions in Android apps. In: Proceedings of the 13th international conference on mining software repositories, MSR 2016, Austin, TX, USA, May 14-22, 2016, pp 13–24

Habchi S, Blanc X, Rouvoy R (2018) On adopting linters to deal with performance concerns in android apps. In: ASE18-proceedings of the 33rd IEEE/ACM international conference on automated software engineering, vol 11. ACM Press

Hassan F, Mostafa S, Lam ES, Wang X (2017) Automatic building of java projects in software repositories: a study on feasibility and challenges. In: 2017 ACM/IEEE International symposium on empirical software engineering and measurement (ESEM). IEEE, pp 38–47

Hecht G, Benomar O, Rouvoy R, Moha N, Duchien L (2015a) Tracking the software quality of android applications along their evolution (t). In: 2015 30th IEEE/ACM International conference on automated software engineering (ASE). IEEE, pp 236–247

Hecht G, Rouvoy R, Moha N, Duchien L (2015b) Detecting antipatterns in Android apps. In: 2nd ACM International conference on mobile software engineering and systems, MOBILESoft 2015, Florence, Italy, May 16-17, 2015, pp 148–149

Holm S (1979) A simple sequentially rejective Bonferroni test procedure. Scand J Stat 6:65–70MATH

Huchard M, Kästner C, Fraser G (eds) (2018) Proceedings of the 33rd ACM/IEEE international conference on automated software engineering, ASE 2018, Montpellier, France, September 3-7, 2018. ACM, New York

Joorabchi ME, Mesbah A, Kruchten P (2013) Real challenges in mobile app development. In: 2013 ACM / IEEE International symposium on empirical software engineering and measurement, pp 15–24

Kalliamvakou E, Gousios G, Blincoe K, Singer L, German DM, Damian D (2016) An in-depth study of the promises and perils of mining github. Empir Softw Eng 21(5):2035–2071CrossRef

Kim S, Ernst MD (2007) Which warnings should I fix first?. In: Proceedings of the joint meeting of the European software engineering conference and the ACM SIGSOFT international symposium on foundations of software engineering (ESEC/FSE), pp 45–54

Kruskal WH, Wallis A (1952) Use of ranks in one-criterion variance analysis. J Am Stat Assoc 47:583–621CrossRef

Leffingwell D (2010) Agile software requirements: lean requirements practices for teams, programs, and the enterprise. Addison-Wesley Professional

Li L, Bissyandé TF, Papadakis M, Rasthofer S, Bartel A, Octeau D, Klein J, Le Traon Y (2017) Static analysis of android apps: a systematic literature review. Information and Software Technology

Lidwell W, Holden K, Butler J (2010) Universal principles of design revised and updated: 125 ways to enhance usability, influence perception, increase appeal, make better design decisions, and teach through design, 2nd edn. Rockport Publishers

Liu Y, Xu C, Cheung S (2014) Characterizing and detecting performance bugs for smartphone applications. In: 36th International conference on software engineering, ICSE ’14, Hyderabad, India - May 31 - June 07, 2014, pp 1013–1024

Malik H, Hemmati H, Hassan AE (2013) Automatic detection of performance deviations in the load testing of large scale systems. In: Proceedings of the 2013 international conference on software engineering. IEEE Press, pp 1012–1021

Munson JC, churn S. G. Elbaum. (1998) Code A measure for estimating the impact of code change. In: International Conference on software maintenance, 1998. Proceedings. IEEE, pp 24–31

Nielson F, Nielson HR, Hankin C (2015) Principles of program analysis. Springer

Nistor A, Ravindranath L (2014) Suncat: helping developers understand and predict performance problems in smartphone applications. In: International symposium on software testing and analysis, ISSTA ’14, San Jose, CA, USA - July 21 - 26, 2014, pp 282–292

Palomba F, Di Nucci D, Panichella A, Zaidman A, De Lucia A (2017) Lightweight detection of Android-specific code smells: the adoctor project. In: IEEE 24th International conference on software analysis, evolution and reengineering, SANER 2017, Klagenfurt, Austria, February 20-24, 2017, pp 487–491

Palomba F, Vásquez ML, Bavota G, Oliveto R, Di Penta M, Poshyvanyk D, De Lucia A (2018) Crowdsourcing user reviews to support the evolution of mobile apps. J Syst Softw 137:143–162CrossRef

Pascarella L, Geiger F-X, Palomba F, Di Nucci D, Malavolta I, Bacchelli A (2018) Self-reported activities of android developers. In: 5th IEEE/ACM International conference on mobile software engineering and systems, page to appear. ACM, New York

Qian F, Wang Z, Gerber A, Mao ZM, Sen S, Spatscheck O (2011) Profiling resource usage for mobile applications: a cross-layer approach. In: Proceedings of the 9th international conference on mobile systems, applications, and services (MobiSys 2011), Bethesda, MD, USA, June 28 - July 01, 2011, pp 321–334

Ravindranath L, Padhye J, Agarwal S, Mahajan R, Obermiller I, Shayandeh S (2012) Appinsight: mobile app performance monitoring in the wild. In: 10th USENIX Symposium on operating systems design and implementation, OSDI 2012, Hollywood, CA, USA, October 8-10, 2012, pp 107–120

Reimann J, Brylski M, Aßmann U (2014) A tool-supported quality smell catalogue for Android developers. Softwaretechnik-Trends, 34(2)

Rosenberg J (2008) Statistical methods and measurement. In: Guide to advanced empirical software engineering. Springer, pp 155–184

Sadowski C, Aftandilian E, Eagle A, Miller-Cushon L, Jaspan C (2018) Lessons from building static analysis tools at google. Commun ACM 61(4):58–66CrossRef

Shull F, Singer J, Sjøberg DI (2007) Guide to advanced empirical software engineering. Springer

Spacco J, Hovemeyer D, Pugh W (2006) Tracking defect warnings across versions. In: Proceedings of the 2006 international workshop on mining software repositories. ACM, pp 133–136

Spencer D (2009) Card sorting: designing usable categories. Rosenfeld Media

Sulír M, Porubän J (2016) A quantitative study of java software buildability. In: Proceedings of the 7th international workshop on evaluation and usability of programming languages and tools. ACM, pp 17–25

Tufano M, Palomba F, Bavota G, Oliveto R, Di Penta M, De Lucia A, Poshyvanyk D (2017) When and why your code starts to smell bad (and whether the smells go away). IEEE Trans Softw Eng 43(11):1063–1088CrossRef

Tufano M, Watson C, Bavota G, Di Penta M, White M, Poshyvanyk D (2019) An empirical study on learning bug-fixing patches in the wild via neural machine translation. ACM Trans Softw Eng Methodol 28(4):19,1–19,29. https://doi.org/10.1145/3340544 CrossRef

Vásquez ML, Vendome C, Luo Q, Poshyvanyk D (2015) How developers detect and fix performance bottlenecks in Android apps. In: 2015 IEEE International conference on software maintenance and evolution, ICSME 2015, Bremen, Germany, September 29 - October 1, 2015, pp 352–361

Wedyan F, Alrmuny D, Bieman JM (2009) The effectiveness of automated static analysis tools for fault detection and refactoring prediction. In: 2009 International conference on software testing verification and validation. IEEE, pp 141–150

Wohlin C, Runeson P, Höst M, Ohlsson M, Regnell B, Wesslén A (2012) Experimentation in software engineering. Computer Science. Springer

Zaman S, Adams B, Hassan AE (2012) A qualitative study on performance bugs. In: Proceedings of the 9th IEEE working conference on mining software repositories. IEEE Press, pp 199–208

Title: Characterizing the evolution of statically-detectable performance issues of Android apps
Authors: Teerath Das
Massimiliano Di Penta
Ivano Malavolta
Publication date: 04-05-2020
Publisher: Springer US
Published in: Empirical Software Engineering / Issue 4/2020
Print ISSN: 1382-3256
Electronic ISSN: 1573-7616
DOI: https://doi.org/10.1007/s10664-019-09798-3

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 4/2020

On (Mis)perceptions of testing effectiveness: an empirical study

An empirical analysis of error propagation in critical software systems

What do Programmers Discuss about Deep Learning Frameworks

How software engineering research aligns with design science: a review

Explicit programming strategies

Retraction Note: Retraction note to: The smell of fear: on the relation between test smells and flaky tests

Premium Partner