Top

Published in:

2019 | OriginalPaper | Chapter

An Empirical Evaluation of Search Algorithms for App Testing

Authors : Leon Sell, Michael Auer, Christoph Frädrich, Michael Gruber, Philemon Werli, Gordon Fraser

Published in: Testing Software and Systems

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Automated testing techniques can effectively explore mobile applications in order to find faults that manifest as program crashes. A number of different techniques for automatically testing apps have been proposed and empirically compared, but previous studies focused on comparing different tools, rather than techniques. Although these studies have shown search-based approaches to be effective, it remains unclear whether superior performance of one tool compared to another is due to fundamental advantages of the underlying search technique, or due to certain engineering choices made during the implementation of the tools. In order to provide a better understanding of app testing as a search problem, we empirically study different search algorithms within the same app testing framework. Experiments on a selection of 10 non-trivial apps reveal that the costs of fitness evaluations are inhibitive, and prevent the choice of algorithm from having a major effect.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Multiple Mutation Testing for Timed Finite State Machine with Timed Guards and Timeouts

next chapter Performance Comparison of Two Search-Based Testing Strategies for ADAS System Validation

https://f-droid.org/en/.

https://www.eclemma.org/jacoco/.

Amalfitano, D., Amatucci, N., Fasolino, A.R., Tramontana, P.: Agrippin: a novel search based testing technique for Android applications. In: Proceedings of the 3rd International Workshop on Software Development Lifecycle for Mobile, pp. 5–12. ACM (2015)

Amalfitano, D., Fasolino, A.R., Tramontana, P., De Carmine, S., Memon, A.M.: Using GUI ripping for automated testing of Android applications. In: Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering, pp. 258–261 (2012)

Arcuri, A.: Test suite generation with the many independent objective (MIO) algorithm. Inf. Softw. Technol. 104, 195–206 (2018)CrossRef

Campos, J., Ge, Y., Fraser, G., Eler, M., Arcuri, A.: An empirical evaluation of evolutionary algorithms for test suite generation. In: Menzies, T., Petke, J. (eds.) SSBSE 2017. LNCS, vol. 10452, pp. 33–48. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66299-2_3CrossRef

Deb, K., Agrawal, S., Pratap, A., Meyarivan, T.: A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. In: Schoenauer, M., et al. (eds.) PPSN 2000. LNCS, vol. 1917, pp. 849–858. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-45356-3_83CrossRef

Android Developers: Application fundamentals, October 2018. https://developer.android.com/guide/components/fundamentals

Android Developers: Ui/application exerciser monkey, September 2018. https://developer.android.com/studio/test/monkey

Eler, M.M., Rojas, J.M., Ge, Y., Fraser, G.: Automated accessibility testing of mobile apps. In: 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST), pp. 116–126 (2018)

Fraser, G., Arcuri, A.: Whole test suite generation. IEEE Trans. Softw. Eng. 39(2), 276–291 (2013)CrossRef

10.

Machiry, A., Tahiliani, R., Naik, M.: Dynodroid: an input generation system for android apps. In: Proceedings of the 9th Joint Meeting on Foundations of Software Engineering, ESEC/FSE, pp. 224–234. ACM (2013)

11.

Mahmood, R., Mirzaei, N., Malek, S.: Evodroid: segmented evolutionary testing of Android apps. In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 599–609 (2014)

12.

Mao, K., Harman, M., Jia, Y.: Sapienz: multi-objective automated testing for android applications. In: Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA, pp. 94–105. ACM (2016)

13.

Panichella, A., Kifetew, F.M., Tonella, P.: Reformulating branch coverage as a many-objective optimization problem. In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), pp. 1–10 (2015)

14.

Rohella, A., Takada, S.: Testing Android applications using multi-objective evolutionary algorithms with a stopping criteria. In: 30th International Conference on Software Engineering and Knowledge Engineering, SEKE 2018, pp. 308–313. Knowledge Systems Institute Graduate School (2018)

15.

Roy Choudhary, S., Gorla, A., Orso, A.: Automated test input generation for Android: Are we there yet? (e). In: Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 429–440 (2015)

16.

Sasnauskas, R., Regehr, J.: Intent fuzzer: crafting intents of death. In: Proceedings of Joint International Workshop on Dynamic Analysis (WODA) and Software and System Performance Testing, Debugging, and Analytics (PERTEA), pp. 1–5 (2014)

17.

Su, T., et al.: Guided, stochastic model-based GUI testing of android apps. In: Proceedings of the 11th Joint Meeting on Foundations of Software Engineering, pp. 245–256 (2017)

18.

Vogel, T., Tran, C., Grunske, L.: Does diversity improve the test suite generation for mobile applications? In: Proceedings of the 11th Symposium on Search-Based Software Engineering (SSBSE 2019). Springer, Heidelberg (2019, to appear)CrossRef

19.

Wang, W., et al.: An empirical study of Android test generation tools in industrial cases. In: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, pp. 738–748 (2018)

20.

Zeng, X., et al.: Automated test input generation for Android: are we really there yet in an industrial case? In: SIGSOFT FSE (2016)

Title: An Empirical Evaluation of Search Algorithms for App Testing
Authors: Leon Sell
Michael Auer
Christoph Frädrich
Michael Gruber
Philemon Werli
Gordon Fraser
Publisher: Springer International Publishing
Book: Testing Software and Systems
Print ISBN: 978-3-030-31279-4

Electronic ISBN: 978-3-030-31280-0

Copyright Year: 2019
DOI: https://doi.org/10.1007/978-3-030-31280-0_8

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner