Skip to main content
Top

2018 | OriginalPaper | Chapter

Single-objective Versus Multi-objectivized Optimization for Evolutionary Crash Reproduction

Authors : Mozhan Soltani, Pouria Derakhshanfar, Annibale Panichella, Xavier Devroey, Andy Zaidman, Arie van Deursen

Published in: Search-Based Software Engineering

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

EvoCrash is a recent search-based approach to generate a test case that reproduces reported crashes. The search is guided by a fitness function that uses a weighted sum scalarization to combine three different heuristics: (i) code coverage, (ii) crash coverage and (iii) stack trace similarity. In this study, we propose and investigate two alternatives to the weighted sum scalarization: (i) the simple sum scalarization and (ii) the multi-objectivization, which decomposes the fitness function into several optimization objectives as an attempt to increase test case diversity. We implemented the three alternative optimizations as an extension of EvoSuite, a popular search-based unit test generator, and applied them on 33 real-world crashes. Our results indicate that for complex crashes the weighted sum reduces the test case generation time, compared to the simple sum, while for simpler crashes the effect is the opposite. Similarly, for complex crashes, multi-objectivization reduces test generation time compared to optimizing with the weighted sum; we also observe one crash that can be replicated only by multi-objectivization. Through our manual analysis, we found out that when optimizing the original weighted function gets trapped in local optima, optimization for decomposed objectives improves the search for crash reproduction. Generally, while multi-objectivization is under-explored, our results are promising and encourage further investigations of the approach.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Footnotes
1
Note that there might exist multiple tests that can replicate the target crash; however, these tests are coincident points as they will all have a zero-value for all objectives.
 
Literature
2.
go back to reference Arcuri, A., Fraser, G., Just, R.: Private API access and functional mocking in automated unit test generation. In: Proceedings of International Conference on Software Testing, Verification and Validation (ICST), pp. 126–137. IEEE (2017) Arcuri, A., Fraser, G., Just, R.: Private API access and functional mocking in automated unit test generation. In: Proceedings of International Conference on Software Testing, Verification and Validation (ICST), pp. 126–137. IEEE (2017)
3.
go back to reference Bianchi, F.A., Pezzè, M., Terragni, V.: Reproducing concurrency failures from crash stacks. In: Proceedings of the Joint Meeting on Foundations of Software Engineering (FSE), pp. 705–716. ACM (2017) Bianchi, F.A., Pezzè, M., Terragni, V.: Reproducing concurrency failures from crash stacks. In: Proceedings of the Joint Meeting on Foundations of Software Engineering (FSE), pp. 705–716. ACM (2017)
4.
go back to reference Chen, N., Kim, S.: STAR: stack trace based automatic crash reproduction via symbolic execution. IEEE Trans. Softw. Eng. 41(2), 198–220 (2015)CrossRef Chen, N., Kim, S.: STAR: stack trace based automatic crash reproduction via symbolic execution. IEEE Trans. Softw. Eng. 41(2), 198–220 (2015)CrossRef
5.
go back to reference Coello Coello, C.A.: Constraint-handling techniques used with evolutionary algorithms. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO Companion), pp. 563–587. ACM (2016) Coello Coello, C.A.: Constraint-handling techniques used with evolutionary algorithms. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO Companion), pp. 563–587. ACM (2016)
6.
go back to reference Črepinšek, M., Liu, S.H., Mernik, M.: Exploration and exploitation in evolutionary algorithms: a survey. ACM Comput. Surv. (CSUR) 45(3), 35 (2013)CrossRef Črepinšek, M., Liu, S.H., Mernik, M.: Exploration and exploitation in evolutionary algorithms: a survey. ACM Comput. Surv. (CSUR) 45(3), 35 (2013)CrossRef
7.
go back to reference Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. TEVC 6(2), 182–197 (2002) Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. TEVC 6(2), 182–197 (2002)
8.
go back to reference Feldt, R., Torkar, R., Gorschek, T., Afzal, W.: Searching for cognitively diverse tests: towards universal test diversity metrics. In: Proceedings of International Conference on Software Testing Verification and Validation Workshops (ICSTW), pp. 178–186. IEEE (2008) Feldt, R., Torkar, R., Gorschek, T., Afzal, W.: Searching for cognitively diverse tests: towards universal test diversity metrics. In: Proceedings of International Conference on Software Testing Verification and Validation Workshops (ICSTW), pp. 178–186. IEEE (2008)
9.
go back to reference Fraser, G., Arcuri, A.: Whole test suite generation. IEEE Trans. Softw. Eng. 39(2), 276–291 (2013)CrossRef Fraser, G., Arcuri, A.: Whole test suite generation. IEEE Trans. Softw. Eng. 39(2), 276–291 (2013)CrossRef
10.
go back to reference Fraser, G., Arcuri, A.: A large-scale evaluation of automated unit test generation using evosuite. TOSEM 24(2), 8 (2014)CrossRef Fraser, G., Arcuri, A.: A large-scale evaluation of automated unit test generation using evosuite. TOSEM 24(2), 8 (2014)CrossRef
11.
go back to reference Goldberg, D.E., Richardson, J.: Genetic algorithms with sharing for multimodal function optimization. In: Proceedings of International Conference on Genetic Algorithms and Their Application, pp. 41–49. Lawrence Erlbaum Associates Inc. (1987) Goldberg, D.E., Richardson, J.: Genetic algorithms with sharing for multimodal function optimization. In: Proceedings of International Conference on Genetic Algorithms and Their Application, pp. 41–49. Lawrence Erlbaum Associates Inc. (1987)
12.
go back to reference Harman, M., McMinn, P.: A theoretical & empirical analysis of evolutionary testing and hill climbing for structural test data generation. In: Proceedings of the 2007 International Symposium on Software Testing and Analysis, pp. 73–83. ACM (2007) Harman, M., McMinn, P.: A theoretical & empirical analysis of evolutionary testing and hill climbing for structural test data generation. In: Proceedings of the 2007 International Symposium on Software Testing and Analysis, pp. 73–83. ACM (2007)
13.
go back to reference Jähne, M., Li, X., Branke, J.: Evolutionary algorithms and multi-objectivization for the travelling salesman problem. In: Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, pp. 595–602. ACM (2009) Jähne, M., Li, X., Branke, J.: Evolutionary algorithms and multi-objectivization for the travelling salesman problem. In: Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, pp. 595–602. ACM (2009)
14.
go back to reference Jin, W., Orso, A.: Bugredux: reproducing field failures for in-house debugging. In: Proceedings of International Conference on Software Engineering (ICSE), pp. 474–484. IEEE Press (2012) Jin, W., Orso, A.: Bugredux: reproducing field failures for in-house debugging. In: Proceedings of International Conference on Software Engineering (ICSE), pp. 474–484. IEEE Press (2012)
15.
go back to reference Just, R., Jalali, D., Ernst, M.D.: Defects4J: a database of existing faults to enable controlled testing studies for Java programs. In: Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), pp. 437–440. ACM (2014) Just, R., Jalali, D., Ernst, M.D.: Defects4J: a database of existing faults to enable controlled testing studies for Java programs. In: Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), pp. 437–440. ACM (2014)
16.
go back to reference Kifetew, F.M., Panichella, A., De Lucia, A., Oliveto, R., Tonella, P.: Orthogonal exploration of the search space in evolutionary test case generation. In: Proceedings of International Symposium on Software Testing and Analysis (ISSTA), pp. 257–267. ACM (2013) Kifetew, F.M., Panichella, A., De Lucia, A., Oliveto, R., Tonella, P.: Orthogonal exploration of the search space in evolutionary test case generation. In: Proceedings of International Symposium on Software Testing and Analysis (ISSTA), pp. 257–267. ACM (2013)
17.
18.
go back to reference McMinn, P.: Search-based software test data generation: a survey. Softw. Test. Verification Reliab. 14(2), 105–156 (2004)CrossRef McMinn, P.: Search-based software test data generation: a survey. Softw. Test. Verification Reliab. 14(2), 105–156 (2004)CrossRef
19.
go back to reference Nayrolles, M., Hamou-Lhadj, A., Tahar, S., Larsson, A.: A bug reproduction approach based on directed model checking and crash traces. J. Softw.: Evol. Process 29(3), e1789 (2017) Nayrolles, M., Hamou-Lhadj, A., Tahar, S., Larsson, A.: A bug reproduction approach based on directed model checking and crash traces. J. Softw.: Evol. Process 29(3), e1789 (2017)
20.
go back to reference Rößler, J., Zeller, A., Fraser, G., Zamfir, C., Candea, G.: Reconstructing core dumps. In: Proceedings of International Conference on Software Testing, Verification and Validation (ICST), pp. 114–123. IEEE (2013) Rößler, J., Zeller, A., Fraser, G., Zamfir, C., Candea, G.: Reconstructing core dumps. In: Proceedings of International Conference on Software Testing, Verification and Validation (ICST), pp. 114–123. IEEE (2013)
21.
go back to reference Soltani, M., Panichella, A., van Deursen, A.: Evolutionary testing for crash reproduction. In: Proceedings of the 9th International Workshop on Search-Based Software Testing - SBST 2016, pp. 1–4 (2016) Soltani, M., Panichella, A., van Deursen, A.: Evolutionary testing for crash reproduction. In: Proceedings of the 9th International Workshop on Search-Based Software Testing - SBST 2016, pp. 1–4 (2016)
22.
go back to reference Soltani, M., Panichella, A., van Deursen, A.: A guided genetic algorithm for automated crash reproduction. In: International Conference on Software Engineering (ICSE), pp. 209–220. IEEE, May 2017 Soltani, M., Panichella, A., van Deursen, A.: A guided genetic algorithm for automated crash reproduction. In: International Conference on Software Engineering (ICSE), pp. 209–220. IEEE, May 2017
23.
go back to reference Vargha, A., Delaney, H.D.: A critique and improvement of the CL common language effect size statistics of McGraw and Wong. J. Educ. Behav. Stat. 25(2), 101–132 (2000) Vargha, A., Delaney, H.D.: A critique and improvement of the CL common language effect size statistics of McGraw and Wong. J. Educ. Behav. Stat. 25(2), 101–132 (2000)
24.
go back to reference Xuan, J., Xie, X., Monperrus, M.: Crash reproduction via test case mutation: let existing test cases help. In: Proceedings of the Joint Meeting on Foundations of Software Engineering (ESEC/FSE), pp. 910–913. ACM (2015) Xuan, J., Xie, X., Monperrus, M.: Crash reproduction via test case mutation: let existing test cases help. In: Proceedings of the Joint Meeting on Foundations of Software Engineering (ESEC/FSE), pp. 910–913. ACM (2015)
Metadata
Title
Single-objective Versus Multi-objectivized Optimization for Evolutionary Crash Reproduction
Authors
Mozhan Soltani
Pouria Derakhshanfar
Annibale Panichella
Xavier Devroey
Andy Zaidman
Arie van Deursen
Copyright Year
2018
DOI
https://doi.org/10.1007/978-3-319-99241-9_18

Premium Partner