Skip to main content
Top

2018 | OriginalPaper | Chapter

Tuning Multi-Objective Optimization Algorithms for the Integration and Testing Order Problem

Authors : Miha Ravber, Matej Črepinšek, Marjan Mernik, Tomaž Kosar

Published in: Bioinspired Optimization Methods and Their Applications

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Multi-Objective Evolutionary Algorithms (MOEAs) are one of the most used search techniques in Search-Based Software Engineering (SBSE). However, MOEAs have many control parameters which must be configured for the problem at hand. This can be a very challenging task by itself. To make matters worse, in Multi-Objective Optimization (MOO) different aspects of quality of the obtained Pareto front need to be taken in to account. A novel method called MOCRS-Tuning is proposed to address this problem. MOCRS-Tuning is a meta-evolutionary algorithm which uses a chess rating system with quality indicator ensemble. The chess rating system enables us to determine the performance of an MOEA on different problems easily. The ensemble of quality indicators ensures that different aspects of quality are considered. The tuning was carried out on five different MOEAs on the Integration and Test Order Problem (ITO). The experimental results show significant improvement after tuning of all five MOEAs used in the experiment.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Harman, M., Mansouri, S.A., Zhang, Y.: Search-based software engineering: trends, techniques and applications. ACM Comput. Surv. (CSUR) 45(1), 11 (2012)CrossRef Harman, M., Mansouri, S.A., Zhang, Y.: Search-based software engineering: trends, techniques and applications. ACM Comput. Surv. (CSUR) 45(1), 11 (2012)CrossRef
2.
go back to reference Afzal, W., Torkar, R., Feldt, R.: A systematic review of search-based testing for non-functional system properties. Inf. Softw. Technol. 51(6), 957–976 (2009)CrossRef Afzal, W., Torkar, R., Feldt, R.: A systematic review of search-based testing for non-functional system properties. Inf. Softw. Technol. 51(6), 957–976 (2009)CrossRef
3.
go back to reference McMinn, P.: Search-based software test data generation: a survey. Softw. Test. Verification Reliab. 14(2), 105–156 (2004)CrossRef McMinn, P.: Search-based software test data generation: a survey. Softw. Test. Verification Reliab. 14(2), 105–156 (2004)CrossRef
4.
go back to reference Guizzo, G., Vergilio, S.R., Pozo, A.T., Fritsche, G.M.: A multi-objective and evolutionary hyper-heuristic applied to the integration and test order problem. Appl. Soft Comput. 56, 331–344 (2017)CrossRef Guizzo, G., Vergilio, S.R., Pozo, A.T., Fritsche, G.M.: A multi-objective and evolutionary hyper-heuristic applied to the integration and test order problem. Appl. Soft Comput. 56, 331–344 (2017)CrossRef
5.
go back to reference Vos, T.E., Baars, A.I., Lindlar, F.F., Kruse, P.M., Windisch, A., Wegener, J.: Industrial scaled automated structural testing with the evolutionary testing tool. In: 2010 Third International Conference on Software Testing, Verification and Validation (ICST), pp. 175–184. IEEE (2010) Vos, T.E., Baars, A.I., Lindlar, F.F., Kruse, P.M., Windisch, A., Wegener, J.: Industrial scaled automated structural testing with the evolutionary testing tool. In: 2010 Third International Conference on Software Testing, Verification and Validation (ICST), pp. 175–184. IEEE (2010)
6.
go back to reference Eiben, A.E., Smit, S.K.: Parameter tuning for configuring and analyzing evolutionary algorithms. Swarm Evol. Comput. 1(1), 19–31 (2011)CrossRef Eiben, A.E., Smit, S.K.: Parameter tuning for configuring and analyzing evolutionary algorithms. Swarm Evol. Comput. 1(1), 19–31 (2011)CrossRef
7.
go back to reference Karafotias, G., Hoogendoorn, M., Eiben, Á.E.: Parameter control in evolutionary algorithms: trends and challenges. IEEE Trans. Evol. Comput. 19(2), 167–187 (2015)CrossRef Karafotias, G., Hoogendoorn, M., Eiben, Á.E.: Parameter control in evolutionary algorithms: trends and challenges. IEEE Trans. Evol. Comput. 19(2), 167–187 (2015)CrossRef
8.
go back to reference Veček, N., Mernik, M., Filipič, B., Črepinšek, M.: Parameter tuning with chess rating system (CRS-tuning) for meta-heuristic algorithms. Inf. Sci. 372, 446–469 (2016)CrossRef Veček, N., Mernik, M., Filipič, B., Črepinšek, M.: Parameter tuning with chess rating system (CRS-tuning) for meta-heuristic algorithms. Inf. Sci. 372, 446–469 (2016)CrossRef
9.
go back to reference Brest, J., Zumer, V., Maucec, M.: Self-adaptive differential evolution algorithm in constrained real-parameter optimization. In: IEEE Congress on Evolutionary Computation, CEC 2006, pp. 215–222. IEEE (2006) Brest, J., Zumer, V., Maucec, M.: Self-adaptive differential evolution algorithm in constrained real-parameter optimization. In: IEEE Congress on Evolutionary Computation, CEC 2006, pp. 215–222. IEEE (2006)
10.
go back to reference Ravber, M., Mernik, M., Črepinšek, M.: Ranking multi-objective evolutionary algorithms using a chess rating system with quality indicator ensemble. In: 2017 IEEE Congress on Evolutionary Computation (CEC), pp. 1503–1510. IEEE (2017) Ravber, M., Mernik, M., Črepinšek, M.: Ranking multi-objective evolutionary algorithms using a chess rating system with quality indicator ensemble. In: 2017 IEEE Congress on Evolutionary Computation (CEC), pp. 1503–1510. IEEE (2017)
11.
go back to reference Ravber, M., Mernik, M., Črepinšek, M.: The impact of quality indicators on the rating of multi-objective evolutionary algorithms. In: 7th International Conference on Bioinspired Optimization Methods and their Applications (BIOMA 2016), pp. 119–130 (2016) Ravber, M., Mernik, M., Črepinšek, M.: The impact of quality indicators on the rating of multi-objective evolutionary algorithms. In: 7th International Conference on Bioinspired Optimization Methods and their Applications (BIOMA 2016), pp. 119–130 (2016)
12.
go back to reference Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)CrossRef Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)CrossRef
13.
go back to reference Assunção, W.K.G., Colanzi, T.E., Vergilio, S.R., Pozo, A.: A multi-objective optimization approach for the integration and test order problem. Inf. Sci. 267, 119–139 (2014)MathSciNetCrossRef Assunção, W.K.G., Colanzi, T.E., Vergilio, S.R., Pozo, A.: A multi-objective optimization approach for the integration and test order problem. Inf. Sci. 267, 119–139 (2014)MathSciNetCrossRef
14.
go back to reference Assunção, W.K.G., Colanzi, T.E., Pozo, A.T.R., Vergilio, S.R.: Establishing integration test orders of classes with several coupling measures. In: Proceedings of the 13th annual conference on Genetic and evolutionary computation, pp. 1867–1874. ACM (2011) Assunção, W.K.G., Colanzi, T.E., Pozo, A.T.R., Vergilio, S.R.: Establishing integration test orders of classes with several coupling measures. In: Proceedings of the 13th annual conference on Genetic and evolutionary computation, pp. 1867–1874. ACM (2011)
15.
go back to reference Beizer, B.: Software Testing Techniques. Dreamtech Press, New Delhi (2003)MATH Beizer, B.: Software Testing Techniques. Dreamtech Press, New Delhi (2003)MATH
17.
go back to reference Hashim, N.L., Schmidt, H.W., Ramakrishnan, S.: Test order for class-based integration testing of Java applications. In: Fifth International Conference on Quality Software (QSIC 2005), pp. 11–18. IEEE (2005) Hashim, N.L., Schmidt, H.W., Ramakrishnan, S.: Test order for class-based integration testing of Java applications. In: Fifth International Conference on Quality Software (QSIC 2005), pp. 11–18. IEEE (2005)
18.
go back to reference Glickman, M.E.: Example of the Glicko-2 System. Boston University, Boston (2012) Glickman, M.E.: Example of the Glicko-2 System. Boston University, Boston (2012)
19.
go back to reference Zitzler, E., Künzli, S.: Indicator-based selection in multiobjective search. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., Merelo-Guervós, J.J., Bullinaria, J.A., Rowe, J.E., Tiňo, P., Kabán, A., Schwefel, H.-P. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 832–842. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30217-9_84CrossRef Zitzler, E., Künzli, S.: Indicator-based selection in multiobjective search. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., Merelo-Guervós, J.J., Bullinaria, J.A., Rowe, J.E., Tiňo, P., Kabán, A., Schwefel, H.-P. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 832–842. Springer, Heidelberg (2004). https://​doi.​org/​10.​1007/​978-3-540-30217-9_​84CrossRef
20.
go back to reference Zhang, Q., Li, H.: MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans. Evol. Comput. 11(6), 712–731 (2007)CrossRef Zhang, Q., Li, H.: MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans. Evol. Comput. 11(6), 712–731 (2007)CrossRef
21.
go back to reference Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)CrossRef Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)CrossRef
22.
go back to reference Corne, D.W., Jerram, N.R., Knowles, J.D., Oates, M.J., et al.: PESA-II: region-based selection in evolutionary multiobjective optimization. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2001), pp. 124–130 (2001) Corne, D.W., Jerram, N.R., Knowles, J.D., Oates, M.J., et al.: PESA-II: region-based selection in evolutionary multiobjective optimization. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2001), pp. 124–130 (2001)
23.
go back to reference Zitzler, E., Laumanns, M., Thiele, L., Zitzler, E., Zitzler, E., Thiele, L., Thiele, L.: SPEA2: improving the strength Pareto evolutionary algorithm (2001) Zitzler, E., Laumanns, M., Thiele, L., Zitzler, E., Zitzler, E., Thiele, L., Thiele, L.: SPEA2: improving the strength Pareto evolutionary algorithm (2001)
24.
go back to reference Ishibuchi, H., Masuda, H., Tanigaki, Y., Nojima, Y.: Difficulties in specifying reference points to calculate the inverted generational distance for many-objective optimization problems. In: IEEE Symposium on Computational Intelligence in Multi-criteria (MCDMDecision-Making ), pp. 170–177. IEEE (2014) Ishibuchi, H., Masuda, H., Tanigaki, Y., Nojima, Y.: Difficulties in specifying reference points to calculate the inverted generational distance for many-objective optimization problems. In: IEEE Symposium on Computational Intelligence in Multi-criteria (MCDMDecision-Making ), pp. 170–177. IEEE (2014)
25.
go back to reference Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach. IEEE Trans. Evol. Comput. 3(4), 257–271 (1999)CrossRef Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach. IEEE Trans. Evol. Comput. 3(4), 257–271 (1999)CrossRef
26.
go back to reference Hansen, M.P., Jaszkiewicz, A.: Evaluating The Quality of Approximations to the Non-dominated set. IMM, Technical University of Denmark, Department of Mathematical Modelling (1998) Hansen, M.P., Jaszkiewicz, A.: Evaluating The Quality of Approximations to the Non-dominated set. IMM, Technical University of Denmark, Department of Mathematical Modelling (1998)
27.
go back to reference Yen, G.G., He, Z.: Performance metric ensemble for multiobjective evolutionary algorithms. IEEE Trans. Evol. Comput. 18(1), 131–144 (2014)CrossRef Yen, G.G., He, Z.: Performance metric ensemble for multiobjective evolutionary algorithms. IEEE Trans. Evol. Comput. 18(1), 131–144 (2014)CrossRef
28.
go back to reference Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C.M., Da Fonseca, V.G.: Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans. Evol. Comput. 7(2), 117–132 (2003)CrossRef Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C.M., Da Fonseca, V.G.: Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans. Evol. Comput. 7(2), 117–132 (2003)CrossRef
29.
go back to reference Durillo, J.J., Nebro, A.J.: jMetal: a Java framework for multi-objective optimization. Adv. Eng. Soft. 42(10), 760–771 (2011)CrossRef Durillo, J.J., Nebro, A.J.: jMetal: a Java framework for multi-objective optimization. Adv. Eng. Soft. 42(10), 760–771 (2011)CrossRef
30.
go back to reference Veček, N., Mernik, M., Črepinšek, M.: A chess rating system for evolutionary algorithms: a new method for the comparison and ranking of evolutionary algorithms. Inf. Sci. 277, 656–679 (2014)MathSciNetCrossRef Veček, N., Mernik, M., Črepinšek, M.: A chess rating system for evolutionary algorithms: a new method for the comparison and ranking of evolutionary algorithms. Inf. Sci. 277, 656–679 (2014)MathSciNetCrossRef
Metadata
Title
Tuning Multi-Objective Optimization Algorithms for the Integration and Testing Order Problem
Authors
Miha Ravber
Matej Črepinšek
Marjan Mernik
Tomaž Kosar
Copyright Year
2018
DOI
https://doi.org/10.1007/978-3-319-91641-5_20

Premium Partner