Skip to main content
Erschienen in: Soft Computing 6/2019

10.11.2017 | Methodologies and Application

An extensive evaluation of search-based software testing: a review

verfasst von: Manju Khari, Prabhat Kumar

Erschienen in: Soft Computing | Ausgabe 6/2019

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In recent years, search-based software testing (SBST) is the active research topic in software testing. SBST is the process of generating test cases that use metaheuristics for optimization of a task in the framework of software testing to solve difficult NP-hard problems. The best fitness results must be found with the heuristic search among many possibilities for a more cost-effective testing process and automate the process of generating test cases. Although search-based test data generation is a field of interest, some challenges remain unknown. The main objective of this survey is to find the main topics and trends in this emerging field of search-based software testing by examining the methods and the literature of software testing. A review of earlier studies of search-based software testing from the year 1996 to 2016 is discussed with the application of metaheuristics for the optimization of software testing.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Afzal W, Torkar R, Feldt R (2009) A systematic review of search-based testing for non-functional system properties. Inf Softw Technol 51:957–976CrossRef Afzal W, Torkar R, Feldt R (2009) A systematic review of search-based testing for non-functional system properties. Inf Softw Technol 51:957–976CrossRef
Zurück zum Zitat Afzal W, Alone S, Glocksien K, Torkar R (2016) Software test process improvement approaches. J Syst Softw 111:1–33CrossRef Afzal W, Alone S, Glocksien K, Torkar R (2016) Software test process improvement approaches. J Syst Softw 111:1–33CrossRef
Zurück zum Zitat Alander J, Mantere T, Turunen P (1998) Genetic algorithm based software testing. Springer, Vienna, pp 325–328 Alander J, Mantere T, Turunen P (1998) Genetic algorithm based software testing. Springer, Vienna, pp 325–328
Zurück zum Zitat Alba E, Chicano FJ (2006) Software testing with evolutionary strategies. Springer, Vienna, pp 50–65 Alba E, Chicano FJ (2006) Software testing with evolutionary strategies. Springer, Vienna, pp 50–65
Zurück zum Zitat Alégroth E, Feldt R, Kolström P (2016) Maintenance of automated test suites in industry: an empirical study on Visual GUI Testing. Inf Softw Technol 73:66–80CrossRef Alégroth E, Feldt R, Kolström P (2016) Maintenance of automated test suites in industry: an empirical study on Visual GUI Testing. Inf Softw Technol 73:66–80CrossRef
Zurück zum Zitat Ali S, Briand LC, Hemmati H, Rajwinder K, Panesar-Walawege RK (2010) A systematic review of the application and empirical investigation of search-based test case generation. IEEE Trans Softw Eng 36(6):742–762CrossRef Ali S, Briand LC, Hemmati H, Rajwinder K, Panesar-Walawege RK (2010) A systematic review of the application and empirical investigation of search-based test case generation. IEEE Trans Softw Eng 36(6):742–762CrossRef
Zurück zum Zitat Arcuri A (2010) It does matter how you normalise the branch distance in search based software testing. IEEE, pp 205–214 Arcuri A (2010) It does matter how you normalise the branch distance in search based software testing. IEEE, pp 205–214
Zurück zum Zitat Bauersfeld S, Wappler S, Wegener J (2011) A metaheuristic approach to test sequence generation for applications with a GUI. Springer, vol 69(56), pp 173–187 Bauersfeld S, Wappler S, Wegener J (2011) A metaheuristic approach to test sequence generation for applications with a GUI. Springer, vol 69(56), pp 173–187
Zurück zum Zitat Bertolino A (2007) Software testing research. In: Achievements, challenges, dreams. IEEE, pp 85–103 Bertolino A (2007) Software testing research. In: Achievements, challenges, dreams. IEEE, pp 85–103
Zurück zum Zitat Daka E, Campos J, Fraser G, Dorn J, Weimer W (2015) Modeling readability to improve unit tests. ACM, pp 107–118 Daka E, Campos J, Fraser G, Dorn J, Weimer W (2015) Modeling readability to improve unit tests. ACM, pp 107–118
Zurück zum Zitat Díaz E, Tuya J, Blanco R (2003) Automated software testing using a metaheuristic technique based on Tabu search. IEEE, pp 310–313 Díaz E, Tuya J, Blanco R (2003) Automated software testing using a metaheuristic technique based on Tabu search. IEEE, pp 310–313
Zurück zum Zitat Dobuneh N, Raeisi M, Jawawi DN, Ghazali M, Malakooti MV (2014) Development test case prioritization technique in regression testing based on hybrid criteria. IEEE, pp 301–305 Dobuneh N, Raeisi M, Jawawi DN, Ghazali M, Malakooti MV (2014) Development test case prioritization technique in regression testing based on hybrid criteria. IEEE, pp 301–305
Zurück zum Zitat Eickelmann NS, Richardson DJ (1996) An evaluation of software test environment architectures. IEEE, pp 353–364 Eickelmann NS, Richardson DJ (1996) An evaluation of software test environment architectures. IEEE, pp 353–364
Zurück zum Zitat El-Serafy A, El-Sayed G, Salama C, Wahba(2015) An enhanced genetic algorithm for MC/DC test data generation. IEEE, pp 1–8 El-Serafy A, El-Sayed G, Salama C, Wahba(2015) An enhanced genetic algorithm for MC/DC test data generation. IEEE, pp 1–8
Zurück zum Zitat Ferguson R, Korel B (1996) The chaining approach for software test data generation. ACM Trans Softw Eng Methodol 5(1):63–86CrossRef Ferguson R, Korel B (1996) The chaining approach for software test data generation. ACM Trans Softw Eng Methodol 5(1):63–86CrossRef
Zurück zum Zitat Fin A, Fummi F, Pravadelli G, Amleto (2001) A multi-language environment for functional test generation. IEEE, pp 821–829 Fin A, Fummi F, Pravadelli G, Amleto (2001) A multi-language environment for functional test generation. IEEE, pp 821–829
Zurück zum Zitat Fraser G, Zeller A (2012) Mutation-driven generation of unit tests and oracles. IEEE Trans Softw Eng 38(2):278–292CrossRef Fraser G, Zeller A (2012) Mutation-driven generation of unit tests and oracles. IEEE Trans Softw Eng 38(2):278–292CrossRef
Zurück zum Zitat Fraser G, Arcuri A (2012) Sound empirical evidence in software testing. IEEE, pp 178–188 Fraser G, Arcuri A (2012) Sound empirical evidence in software testing. IEEE, pp 178–188
Zurück zum Zitat Gallagher MJ, Narasimhan VL (1997) Adtest: a test data generation suite for ada software systems. IEEE Trans Softw Eng 23(8):473–484CrossRef Gallagher MJ, Narasimhan VL (1997) Adtest: a test data generation suite for ada software systems. IEEE Trans Softw Eng 23(8):473–484CrossRef
Zurück zum Zitat Gotlieb A, Botella B, Rueher M (1998) Automatic test data generation using constraint solving techniques. ACM SIGSOFT 23(2) Gotlieb A, Botella B, Rueher M (1998) Automatic test data generation using constraint solving techniques. ACM SIGSOFT 23(2)
Zurück zum Zitat Gupta N, Mathur AP, Soffa ML (1998) Automated test data generation using an iterative relaxation method. ACM SIGSOFT Softw Eng Notes 23(6):231–244CrossRef Gupta N, Mathur AP, Soffa ML (1998) Automated test data generation using an iterative relaxation method. ACM SIGSOFT Softw Eng Notes 23(6):231–244CrossRef
Zurück zum Zitat Harman M (2007) Search based software engineering for program comprehension. Program Comprehension. IEEE Harman M (2007) Search based software engineering for program comprehension. Program Comprehension. IEEE
Zurück zum Zitat Harman M (2008) Open problems in testability transformation. IEEE, pp 196–209 Harman M (2008) Open problems in testability transformation. IEEE, pp 196–209
Zurück zum Zitat Harman M, Hassoun Y, Lakhotia K, McMinn P, Wegener J (2007) The impact of input domain reduction on search-based test data generation. ACM SIGSOFT, pp 155–164 Harman M, Hassoun Y, Lakhotia K, McMinn P, Wegener J (2007) The impact of input domain reduction on search-based test data generation. ACM SIGSOFT, pp 155–164
Zurück zum Zitat Harman M, Jia Y, Zhang Y (2015) Achievements, open problems and challenges for search based software testing. In: Software Testing, Verification and Validation (ICST), 2015 IEEE 8th International Conference. IEEE, pp 1–12 Harman M, Jia Y, Zhang Y (2015) Achievements, open problems and challenges for search based software testing. In: Software Testing, Verification and Validation (ICST), 2015 IEEE 8th International Conference. IEEE, pp 1–12
Zurück zum Zitat Harman M, McMinn P (2010) A theoretical and empirical study of search-based testing: Local, global, and hybrid search. IEEE, vol 36(2), pp 226–247 Harman M, McMinn P (2010) A theoretical and empirical study of search-based testing: Local, global, and hybrid search. IEEE, vol 36(2), pp 226–247
Zurück zum Zitat Harman M, McMinn P, De Souza JT, Yoo S (2012) Search based software engineering: techniques, taxonomy, tutorial. Springer, vol 7007, pp 1–59 Harman M, McMinn P, De Souza JT, Yoo S (2012) Search based software engineering: techniques, taxonomy, tutorial. Springer, vol 7007, pp 1–59
Zurück zum Zitat Kapfhammer GM, McMinn P, Chris J (2013) Wright. Search-based testing of relational schema integrity constraints across multiple database management systems. IEEE. pp 31–40 Kapfhammer GM, McMinn P, Chris J (2013) Wright. Search-based testing of relational schema integrity constraints across multiple database management systems. IEEE. pp 31–40
Zurück zum Zitat Korel B (1996) Automated test data generation for programs with procedures. ACM SIGSOFT, vol 21(3), pp 209–215 Korel B (1996) Automated test data generation for programs with procedures. ACM SIGSOFT, vol 21(3), pp 209–215
Zurück zum Zitat Korel B, Harman M, Chung S, Apirukvorapinit P, Gupta R, Zhang Q (2005) Data dependence based testability transformation in automated test generation. IEEE, pp 245–254 Korel B, Harman M, Chung S, Apirukvorapinit P, Gupta R, Zhang Q (2005) Data dependence based testability transformation in automated test generation. IEEE, pp 245–254
Zurück zum Zitat Kos T, Mernik M, Kosar T (2016) Test automation of a measurement system using a domain-specific modelling language. J Syst Softw 111:74–88CrossRef Kos T, Mernik M, Kosar T (2016) Test automation of a measurement system using a domain-specific modelling language. J Syst Softw 111:74–88CrossRef
Zurück zum Zitat Kotelyanskii A, Kapfhammer GM (2014) Parameter tuning for search-based test-data generation revisited: Support for previous results. IEEE, pp 79–84 Kotelyanskii A, Kapfhammer GM (2014) Parameter tuning for search-based test-data generation revisited: Support for previous results. IEEE, pp 79–84
Zurück zum Zitat Kuhn DR, Wallace DR, Gallo AM Jr (2004) Software fault interactions and implications for software testing. IEEE Trans Softw Eng 30(6):418–421CrossRef Kuhn DR, Wallace DR, Gallo AM Jr (2004) Software fault interactions and implications for software testing. IEEE Trans Softw Eng 30(6):418–421CrossRef
Zurück zum Zitat Lakhotia K McMinn P, Harman M (2009) Automated test data generation for coverage: Haven’t we solved this problem yet? IEEE, pp 95–104 Lakhotia K McMinn P, Harman M (2009) Automated test data generation for coverage: Haven’t we solved this problem yet? IEEE, pp 95–104
Zurück zum Zitat Lefticaru R, Ipate F (2008) Functional search-based testing from state machines. IEEE, pp 525–528 Lefticaru R, Ipate F (2008) Functional search-based testing from state machines. IEEE, pp 525–528
Zurück zum Zitat Mahali P, Acharya AA, Mohapatra DP (2016) Test case prioritization using association rule mining and business criticality test value. Springer, vol 2, pp 335–345 Mahali P, Acharya AA, Mohapatra DP (2016) Test case prioritization using association rule mining and business criticality test value. Springer, vol 2, pp 335–345
Zurück zum Zitat Malhotra R, Khari M (2013) Heuristic search-based approach for automated test data generation: a survey. Int J Bio-Inspired Comput 5(1):1–18CrossRef Malhotra R, Khari M (2013) Heuristic search-based approach for automated test data generation: a survey. Int J Bio-Inspired Comput 5(1):1–18CrossRef
Zurück zum Zitat Marín B, Gallardo C, Quiroga D, Giachetti G, Serral E (2016) Testing of model-driven development applications. Springer, New York, pp 1–29 Marín B, Gallardo C, Quiroga D, Giachetti G, Serral E (2016) Testing of model-driven development applications. Springer, New York, pp 1–29
Zurück zum Zitat Martins E, Sabião SB, Ambrosio AM (1999) ConData: a tool for automating specification-based test case generation for communication systems. Softw Qual J 8(4):303–320CrossRef Martins E, Sabião SB, Ambrosio AM (1999) ConData: a tool for automating specification-based test case generation for communication systems. Softw Qual J 8(4):303–320CrossRef
Zurück zum Zitat McMinn P, Binkley D, Harman M (2009) Empirical evaluation of a nesting testability transformation for evolutionary testing. ACM Trans Softw Eng Methodol 18(3):1–27CrossRef McMinn P, Binkley D, Harman M (2009) Empirical evaluation of a nesting testability transformation for evolutionary testing. ACM Trans Softw Eng Methodol 18(3):1–27CrossRef
Zurück zum Zitat McMinn P, Harman M, Lakhotia K, Hassoun Y, Wegener J (2012) Input domain reduction through irrelevant variable removal and its effect on local, global, and hybrid search-based structural test data generation. IEEE Trans Softw Eng 38(2):453–477CrossRef McMinn P, Harman M, Lakhotia K, Hassoun Y, Wegener J (2012) Input domain reduction through irrelevant variable removal and its effect on local, global, and hybrid search-based structural test data generation. IEEE Trans Softw Eng 38(2):453–477CrossRef
Zurück zum Zitat McMinn P, Harman M, Binkley D, Tonella P (2006) The species per path approach to search based test data generation. ACM, pp 13–24 McMinn P, Harman M, Binkley D, Tonella P (2006) The species per path approach to search based test data generation. ACM, pp 13–24
Zurück zum Zitat McMinn P, Shahbaz M, Stevenson M (2012) Search-based test input generation for string data types using the results of web queries. IEEE, pp 141–150 McMinn P, Shahbaz M, Stevenson M (2012) Search-based test input generation for string data types using the results of web queries. IEEE, pp 141–150
Zurück zum Zitat Michael CC, McGraw G (1998) Automated software test data generation for complex programs. IEEE, pp 136–146 Michael CC, McGraw G (1998) Automated software test data generation for complex programs. IEEE, pp 136–146
Zurück zum Zitat Michael CC, McGraw GE, Schatz MA, Walton CC (1997) Genetic algorithms for dynamic test data generation. IEEE, pp 307–308 Michael CC, McGraw GE, Schatz MA, Walton CC (1997) Genetic algorithms for dynamic test data generation. IEEE, pp 307–308
Zurück zum Zitat Michael CC, McGraw G, Schatz MA (2001) Generating software test data by evolution. Ser Softw Eng Knowl Eng 27(12):1085–1110 Michael CC, McGraw G, Schatz MA (2001) Generating software test data by evolution. Ser Softw Eng Knowl Eng 27(12):1085–1110
Zurück zum Zitat Miller W, Spooner D (1976) Automatic generation of floating point test data. IEEE Trans Softw Eng 2(3):223–226MathSciNetCrossRef Miller W, Spooner D (1976) Automatic generation of floating point test data. IEEE Trans Softw Eng 2(3):223–226MathSciNetCrossRef
Zurück zum Zitat Moadab S, Rashidi H (2016) Automatic path-oriented test data generation by boundary hyper cuboids. J King Saud Univ Comput Inf Sci 28(1):82–97CrossRef Moadab S, Rashidi H (2016) Automatic path-oriented test data generation by boundary hyper cuboids. J King Saud Univ Comput Inf Sci 28(1):82–97CrossRef
Zurück zum Zitat Nguyen Tony, Nassif N (2016) Optimization of HVAC systems using genetic algorithm. Springer, New York, pp 203–209 Nguyen Tony, Nassif N (2016) Optimization of HVAC systems using genetic algorithm. Springer, New York, pp 203–209
Zurück zum Zitat Nguyen TB, Delaunay M, Robach C (2005) Testability analysis of data-flow software. Electron Notes Theor Comput Sci 116:213–225CrossRef Nguyen TB, Delaunay M, Robach C (2005) Testability analysis of data-flow software. Electron Notes Theor Comput Sci 116:213–225CrossRef
Zurück zum Zitat Patrick M (2016) Metaheuristic optimisation and mutation-driven test data generation. Springer, pp 89–115 Patrick M (2016) Metaheuristic optimisation and mutation-driven test data generation. Springer, pp 89–115
Zurück zum Zitat Prabu M, Narasimhan D, Raghuram S (2016) An effective tool for optimizing the number of test paths in data flow testing for anomaly detection. Springer, pp 505–518 Prabu M, Narasimhan D, Raghuram S (2016) An effective tool for optimizing the number of test paths in data flow testing for anomaly detection. Springer, pp 505–518
Zurück zum Zitat Priyanka, Inderveer, Rana A (2016) Cloud-based automatic test data generation framework. Elsevier, pp 1–33 Priyanka, Inderveer, Rana A (2016) Cloud-based automatic test data generation framework. Elsevier, pp 1–33
Zurück zum Zitat Rauf A, Anwar S, Jaffer MA (2010) Automated GUI test coverage analysis using GA. IEEE, pp 1057–1062 Rauf A, Anwar S, Jaffer MA (2010) Automated GUI test coverage analysis using GA. IEEE, pp 1057–1062
Zurück zum Zitat Rogstad E, Briand L (2016) Cost-effective strategies for the regression testing of database applications: case study and lessons learned. J Syst Softw 113:257–274CrossRef Rogstad E, Briand L (2016) Cost-effective strategies for the regression testing of database applications: case study and lessons learned. J Syst Softw 113:257–274CrossRef
Zurück zum Zitat Salman YD, Hashim NL (2016) Automatic test case generation from UML state chart diagram: a survey. Springer, vol 362, pp 123–134 Salman YD, Hashim NL (2016) Automatic test case generation from UML state chart diagram: a survey. Springer, vol 362, pp 123–134
Zurück zum Zitat Shahbaz M, McMinn P, Stevenson M (2015) Automatic generation of valid and invalid test data for string validation routines using web searches and regular expressions. Sci Comput Program 97:405–425CrossRef Shahbaz M, McMinn P, Stevenson M (2015) Automatic generation of valid and invalid test data for string validation routines using web searches and regular expressions. Sci Comput Program 97:405–425CrossRef
Zurück zum Zitat Shah S, Sudarshan S, Kajbaje S, Patidar S, Gupta BP, Vira D (2011) Generating test data for killing SQL mutants: a constraint-based approach. IEEE, pp 1175–1186 Shah S, Sudarshan S, Kajbaje S, Patidar S, Gupta BP, Vira D (2011) Generating test data for killing SQL mutants: a constraint-based approach. IEEE, pp 1175–1186
Zurück zum Zitat Shen X, Wang Q, Wang P, Zhou B (2009) Automatic generation of test case based on GATS algorithm. IEEE, pp 496–500 Shen X, Wang Q, Wang P, Zhou B (2009) Automatic generation of test case based on GATS algorithm. IEEE, pp 496–500
Zurück zum Zitat Sofokleous AA, Andreas AS (2007) Batch-optimistic test-cases generation using genetic algorithms. IEEE, vol 1, pp 157–164 Sofokleous AA, Andreas AS (2007) Batch-optimistic test-cases generation using genetic algorithms. IEEE, vol 1, pp 157–164
Zurück zum Zitat Sofokleous AA, Andreou AS (2008) Automatic, evolutionary test data generation for dynamic software testing. J Syst Softw 81(11):1883–1898CrossRef Sofokleous AA, Andreou AS (2008) Automatic, evolutionary test data generation for dynamic software testing. J Syst Softw 81(11):1883–1898CrossRef
Zurück zum Zitat Utting M, Legeard B, Bouquet F, Fourneret E, Peureux F, Vernotte A (2016) Recent advances in model-based testing. Adv Comput 101:53–120CrossRef Utting M, Legeard B, Bouquet F, Fourneret E, Peureux F, Vernotte A (2016) Recent advances in model-based testing. Adv Comput 101:53–120CrossRef
Metadaten
Titel
An extensive evaluation of search-based software testing: a review
verfasst von
Manju Khari
Prabhat Kumar
Publikationsdatum
10.11.2017
Verlag
Springer Berlin Heidelberg
Erschienen in
Soft Computing / Ausgabe 6/2019
Print ISSN: 1432-7643
Elektronische ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-017-2906-y

Weitere Artikel der Ausgabe 6/2019

Soft Computing 6/2019 Zur Ausgabe