Skip to main content
SpringerLink
Log in

Determining the significance and relative importance of parameters of a simulated quenching algorithm using statistical tools

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

When search methods are being designed it is very important to know which parameters have the greatest influence on the behaviour and performance of the algorithm. To this end, algorithm parameters are commonly calibrated by means of either theoretic analysis or intensive experimentation. When undertaking a detailed statistical analysis of the influence of each parameter, the designer should pay attention mostly to the parameters that are statistically significant. In this paper the ANOVA (ANalysis Of the VAriance) method is used to carry out an exhaustive analysis of a simulated annealing based method and the different parameters it requires. Following this idea, the significance and relative importance of the parameters regarding the obtained results, as well as suitable values for each of these, were obtained using ANOVA and post-hoc Tukey HSD test, on four well known function optimization problems and the likelihood function that is used to estimate the parameters involved in the lognormal diffusion process. Through this statistical study we have verified the adequacy of parameter values available in the bibliography using parametric hypothesis tests.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Eiben AE, Smith JE (2003) Introduction to evolutionary computing. Springer, Berlin. ISBN 3-540-40184-9

    MATH  Google Scholar 

  2. Michalewicz Z, Fogel DB (2004) How to solve it: modern heuristics, 2nd edn. Springer, Berlin

    MATH  Google Scholar 

  3. Yang XS (2010) Nature-inspired metaheuristic algorithms, 2nd edn. Luniver Press, Cambridge

    Google Scholar 

  4. Kirkpatrick S, Gerlatt CD, Vecchi MP (1983) Optimization by simulated annealing. Science 220:671–680

    Article  MathSciNet  MATH  Google Scholar 

  5. Ansari N, Sarasa R, Wang G (1993) An efficient annealing algorithm for global optimization in Boltzmann machines. Appl Intell 3:177–192

    Article  Google Scholar 

  6. Das A, Chakrabarti BK (2005) Quantum annealing and related optimization methods. Lecture note in physics, vol 679. Springer, Heidelberg

    Book  MATH  Google Scholar 

  7. De Vicente J, Lanchares J, Hermida R (2003) Placement by thermodynamic simulated annealing. Phys Lett A 317(5–6):415–423

    Article  MATH  Google Scholar 

  8. Van-Hentenryck P, Michel L (2005) Constraint-based local search. The MIT Press, Cambridge. ISBN: 978-0-262-22077-4

    Google Scholar 

  9. Hoos HH, Stutzle T (2004) Stochastic local search. Foundations and applications. Morgan Kaufmann/Elsevier, San Mateo/Amsterdam

    Google Scholar 

  10. Bambha NK, Bhattacharyya SS, Teich J, Zitzler E (2004) Systematic integration of parameterized local search into evolutionary algorithms. IEEE Trans Evol Comput 8(2):137–155. ISSN: 1089-778X

    Article  Google Scholar 

  11. Eiben AE, Hinterding R, Michalewicz Z (1999) Parameter control in evolutionary algorithms. IEEE Trans Evol Comput 3(2):124–141. ISSN: 1089-778X. doi:10.1109/4235.771166

    Article  Google Scholar 

  12. Karr CL, Wilson E (2003) A self-tuning evolutionary algorithm applied to an inverse partial differential equation. Appl Intell 19(3):147–155

    Article  MATH  Google Scholar 

  13. Smit SK, Eiben AE (2010) Beating the world champion evolutionary algorithm via REVAC tuning. In: Proc of the WCCI2010 IEEE world congress on computational intelligence. IEEE Press, Barcelona, pp 1295–1302. ISSN: 978-1-4244-8126-2

    Google Scholar 

  14. Fisher RA (1925) Theory of statistical estimation. Proc Camb Philos Soc 22:700–725

    Article  MATH  Google Scholar 

  15. Dickinson P, Chow B (1971) Some properties of the Tukey test to Duckworth’s specification/by Peter Dickinson, Bryant Chow. Office of Institutional Research, University of Southwestern Louisiana, Lafayette, Louisiana, USA

  16. Ingber L (1993) Simulated annealing: practice versus theory. Math Comput Model 18(11):29–57

    Article  MathSciNet  MATH  Google Scholar 

  17. Davis L (1991) Handbook of genetic algorithms. Van Nostrand Reinhold, New York

    Google Scholar 

  18. Jagielska I, Matthews C, Whitfort T (1999) An investigation into the application of neural networks, fuzzy logic, genetic algorithms, and rough sets to automated knowledge acquisition problems. Neurocomputing 24(1–3):37–54

    Article  MATH  Google Scholar 

  19. Das S, Chowdhury A, Abraham A (2009) A bacterial evolutionary algorithm for automatic data clustering. In: Proceedings of the eleventh conference on congress on evolutionary computation, CEC’09, Piscataway, NJ, USA. IEEE Press, New York, pp 2403–2410

    Google Scholar 

  20. De Jong KA (1975) An analysis of the behavior of a class of genetic adaptive systems. PhD thesis, University of Michigan, Ann Arbor

  21. Eiben AE, Michalewicz Z, Schoenauer M, Smith JE (2007) Parameter control in evolutionary algorithms. In: Parameter setting in evolutionary algorithms, studies in computational intelligence, vol 54/2007. Springer, Berlin/Heidelberg, pp 19–46. ISBN 978-3-540-69431-1, ISSN 1860-949X

    Chapter  Google Scholar 

  22. Smit SK, Eiben AE (2009) Using entropy for parameter analysis of evolutionary algorithms. In: Beielstein B et al (eds) Empirical methods for the analysis of optimization algorithms. Natural computing series. Springer, Berlin

    Google Scholar 

  23. Eiben AE (2009) Principled approaches to tuning EA parameters. Tutorials—IEEE Congress on Evolutionary Computation (CEC2009). Available at http://www.few.vu.nl/~gusz/papers/eiben-cec-2009-tutorial-corrected.pdf

  24. Smit SK, Eiben AE (2009) Comparing parameter tuning methods for evolutionary algorithms. In: Haddow P et al (eds) CEC’09: proc. of the eleventh conference on congress on evolutionary computation, Piscataway, NJ, USA. IEEE Press, New York, pp 399–406

    Google Scholar 

  25. Gong Y, Fukunaga A (2011) Distributed island-model genetic algorithms using heterogeneous parameter settings. In: Proceedings of the 2011 IEEE congress on evolutionary computation (CEC2011), New Orleans, LA, USA, pp 820–827. ISBN 978-1-4244-7834-7

    Chapter  Google Scholar 

  26. Grefenstette JJ (1986) Optimization of control parameters for genetic algorithms. IEEE Trans Syst Man Cybern 16(1):122–128

    Article  Google Scholar 

  27. Kim D, Kim C (1997) Forecasting time series with genetic fuzzy predictor ensemble. IEEE Trans Fuzzy Syst 5(4):523–535

    Article  Google Scholar 

  28. Liang K, Yao X, Newton CS (2001) Adapting self-adaptive parameters in evolutionary algorithms. Appl Intell 15:171–180

    Article  MATH  Google Scholar 

  29. Gacto MJ, Alcala R, Herrera F (2010) A multi-objective evolutionary algorithm for an effective tuning of fuzzy logic controllers in heating, ventilating and air conditioning systems. Appl Intell 33:1–18

    Article  Google Scholar 

  30. Diosan L, Rogozan A, Pecuchet JP (2010) Improving classification performance of support vector machine by genetically optimising kernel shape and hyper-parameters. Appl Intell 33:1–15

    Article  Google Scholar 

  31. Aksac A, Uzun E, Ozyer T (2011) Areal time traffic simulator utilizing an adaptive fuzzy inference mechanism by tuning fuzzy parameters. Appl Intell 34:1–23

    Article  Google Scholar 

  32. Bäck T, Fogel D, Michalewicz Z (1997) Handbook of evolutionary computation. Institute of Physics Publishing Ltd, Bristol and Oxford University Press, London

    Book  MATH  Google Scholar 

  33. Hinterding R, Michalewicz Z, Eiben AE (1997) Adaptation in evolutionary computation: a survey. In: Proceedings of the 4th IEEE conference on evolutionary computation. IEEE Press, New York, pp 65–69

    Google Scholar 

  34. Harik GR, Lobo FG (1999) A parameter-less genetic algorithm. In: Banzhaf W, Daida J, Eiben AE, Garzon MH, Honavar V, Jakiela M, Smith RE (eds) Proceedings of the genetic and evolutionary computation conference, Orlando, Florida, USA, vol 1, Morgan Kaufmann, San Mateo, pp 258–265. 13–17

    Google Scholar 

  35. Bäck T (1993) Optimal mutation rates in genetic search. In: Forrest S (ed) Proceedings of the 5th international conference on genetic algorithms. Morgan Kaufmann, San Mateo, pp 2–8

    Google Scholar 

  36. Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley, Boston

    MATH  Google Scholar 

  37. Goldberg DE, Deb K, Theirens D (1991) Toward a better understanding of mixing in genetic algorithms. In: Belew RK, Booker LB (eds) Proc of the 4th int conf on genetic algorithms. Morgan Kaufmann, San Mateo, pp 190–195

    Google Scholar 

  38. Harik G, Cantú-Paz E, Goldberg DE, Miller BL (1997) The gambler’s ruin problem, genetic algorithms, and the sizing of populations. In: Proceedings of the 4th IEEE conference on evolutionary computation. IEEE Press, New York, pp 7–12. ISBN: 0-7803-3949-5

    Chapter  Google Scholar 

  39. Schaffer JD, Morishima A (1987) An adaptive crossover distribution mechanism for genetic algorithms. In: Grefenstette JJ (ed) Proceedings of the 2nd international conference on genetic algorithms and their applications. Lawrence Erlbaum Associates, Hillsdale, pp 36–40

    Google Scholar 

  40. Thierens D (1996) Dimensional analysis of allele-wise mixing revisited. In: Voigt HM, Ebeling W, Rechenberg I, Schwefel HP (eds) Proceedings of the 4th conference on parallel problem solving from nature. Lecture notes in computer science, vol 1141. Springer, Berlin, pp 255–265

    Chapter  Google Scholar 

  41. Jansen T, Weyland D (2007) Analysis of evolutionary algorithms for the longest common subsequence problem. In: GECCO’07: proceedings of the 9th annual conference on genetic and evolutionary computation. ACM, New York, pp 939–946

    Chapter  Google Scholar 

  42. Jansen T, Weyland D (2010) Analysis of evolutionary algorithms for the longest common subsequence problem. Algorithmica 57(1):170–186

    Article  MathSciNet  MATH  Google Scholar 

  43. Weyland D (2008) Simulated annealing, its parameter settings and the longest common subsequence problem. In: GECCO’08: proceedings of the 10th annual conference on genetic and evolutionary computation. ACM, New York, pp 803–810

    Chapter  Google Scholar 

  44. Lobo FG, Lima CF, Michalewicz Z (2007) Parameter setting in evolutionary algorithms. Springer, Berlin

    Book  MATH  Google Scholar 

  45. Nannen V, Eiben AE (2006) A method for parameter calibration and relevance estimation in evolutionary algorithms. In: Keijzer M (ed) Proc of the genetic and evolutionary computation conference (GECCO2006). Morgan Kaufmann, San Francisco, pp 183–190

    Chapter  Google Scholar 

  46. Nannen V, Eiben AE (2007) Relevance estimation and value calibration of evolutionary algorithm parameters. In: Proc of the international joint conference on artificial intelligence (IJCAI2007). AAAI Press, Menlo Park, pp 975–980

    Google Scholar 

  47. Smit SK, Eiben AE (2009) Comparing parameter tuning methods for evolutionary algorithms. In: Tyrrell A (ed) Proc of the IEEE congress on evolutionary computation, Thondheim. IEEE Press, New York, pp 399–406

    Chapter  Google Scholar 

  48. Bartz T, Parsopoulos KE, Vrahatis MN (2004) Analysis of particle swarm optimization using computational statistics. In: Chalkis (ed) Proc of the international conference of numerical analysis and applied mathematics (ICAAM2004), pp 34–37

    Google Scholar 

  49. Bartz T, Markon S (2004) Tuning search algorithms for real-world applications: a regression tree based approach. Technical Report of the Collaborative Research Centre 531 Computational Intelligence CI-172/04, University of Dortmund, March 2004

  50. Bartz T, Lasarczyk CWG, Preuss M (2005) Sequential parameter optimization. In: Proc of the 2005 IEEE congress on evolutionary computation, Edinburgh, UK, vol 1, pp 773–780

    Chapter  Google Scholar 

  51. Birattari M, Stutzle T, Paquete L, Varrentrapp K (2002) A racing algorithm for configuring metaheuristics. In: Proc of the genetic and evolutionary computation conference. Morgan Kaufmann, San Mateo, pp 11–18

    Google Scholar 

  52. Hutter F, Hoos HH, Stutzle T (2007) Automatic algorithm configuration based on local search. In: Proc of the twenty-second conference on artificial intelligence (AAAI 2007). AAAI Press, Menlo Park, pp 1152–1157

    Google Scholar 

  53. Montero E, Riff MC, Neveu B (2010) New requirements for off-line parameter calibration algorithms. In: Proc of the WCCI2010 IEEE world congress on computational intelligence. IEEE Press, Barcelona, pp 3466–3473. ISSN 978-1-4244-8126-2

    Google Scholar 

  54. Leung SW, Yuen SY, Chow CK (2010) Parameter control by the entire search history: case study of history-driven evolutionary algorithm. In: Proc of the WCCI2010 IEEE world congress on computational intelligence. IEEE Press, Barcelona, pp 3688–3695. ISSN 978-1-4244-8126-2

    Google Scholar 

  55. Smit SK, Eiben AE (2010) Parameter tuning of evolutionary algorithms: generalist vs specialist. In: Di Chio Cecilia et al (eds) Applications of evolutionary computation. LNCS, vol 6024. Springer, Berlin, pp 542–551

    Chapter  Google Scholar 

  56. He J, Tan AH, Tan CL (2003) On machine learning methods for Chinese document categorization. Appl Intell 18:311–322

    Article  MATH  Google Scholar 

  57. Maudes J, Rodriguez JJ, Garcia-Osorio C, Pardo C (2011) Random projections for linear SVM ensembles. Appl Intell 34:347–359

    Article  Google Scholar 

  58. Shilane D, Martikainen J, Dudoit S, Ovaska SJ (2008) A general framework for statistical performance comparison of evolutionary computation algorithms. Inf Sci 178(14):2870–2879

    Article  Google Scholar 

  59. Garcia S, Molina D, Lozano M, Herrera F (2009) A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 special session on real parameter optimization. J Heuristics 15:617–644

    Article  MATH  Google Scholar 

  60. Rojas I, González J, Pomares H, Merelo JJ, Castillo PA, Romero G (2002) Statistical analisys of the main parameters involved in the desing of a genetic algorithm. IEEE Trans Syst Man Cibern, Part C 32(10):31–37. ISSN:1094-6977

    Article  Google Scholar 

  61. Castillo PA, Merelo JJ, Romero G, Prieto A, Rojas I (2002) Statistical analysis of the parameters of a neuro-genetic algorithm. IEEE Trans Neural Netw 13(6):1374–1394. ISSN:1045-9227

    Article  Google Scholar 

  62. Talbi EG (2009) Metaheuristics. From design to implementation. University of Lille CNRS Inria. Wiley, New York. ISBN 978-0-470-27858-1

    Book  Google Scholar 

  63. Lockett AJ, Miikkulainen R (2011) Measure-theoretic evolutionary annealing. In: Proceedings of the 2011 IEEE congress on evolutionary computation (CEC2011), New Orleans, LA, USA, pp 2139–2146. ISBN 978-1-4244-7834-7

    Chapter  Google Scholar 

  64. Debuse JCW, Rayward-Smith VJ (1999) Discretisation of continuous commercial database features for a simulated annealing data mining algorithm. Appl Intell 11:285–295

    Article  Google Scholar 

  65. Sun S, Zhuge F, Rosenberg J, Steiner R, Rubin G, Napel S (2008) Learning-enhanced simulated annealing: method, evaluation, and application to lung nodule registration. Appl Intell 28:83–99

    Article  Google Scholar 

  66. Bonev B, Cazorla M, Martín F, Matellan V (2010) Portable autonomous walk calibration for 4-legged robots. Applied Intelligence 1–12

  67. Michalewicz Z (1996) Genetic algorithms + data structures = evolution programs, 3rd edn. Springer, Berlin

    MATH  Google Scholar 

  68. Wang ZG, Rahman M, Wong YS, Neo KS (2006) Development of heterogeneous parallel genetic simulated annealing using multi-niche crowding. Intl J Comput Intel 3(1):55–62

    Google Scholar 

  69. van Laarhoven PJ, Aarts EH (1987) Simulated annealing: theory and applications (mathematics and its applications), 1st edn. Springer, Berlin

    MATH  Google Scholar 

  70. Kirkpatrick S, Gelatt CD Jr, Vecchi MP (1983) Optimization by simulated annealing. Science 220:671–680

    Article  MathSciNet  MATH  Google Scholar 

  71. Weicker K (2002) Evolutionare algorithmen. Teubner, Leipzig

    Google Scholar 

  72. Smith GB (1987) Preface to S Geman, D Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images”, pp 562–563

  73. Griewank AO (1981) Generalized descent for global optimization. J Optim Theory Appl 34(1):11–39

    Article  MathSciNet  MATH  Google Scholar 

  74. Törn A, Zilinskas A (1989) Global optimization. Springer lecture notes in computer science, vol 350. Springer, Berlin, pp 1–24

    MATH  Google Scholar 

  75. Ackley DH (1987) A connectionist machine for genetic hillclimbing. Kluwer, Boston

    Book  Google Scholar 

  76. Bäck T (1996) Evolutionary algorithms in theory and practice. Oxford University Press, London

    MATH  Google Scholar 

  77. Whitley D, Garrett D, Watson JP (2003) Quad search and hybrid genetic algorithms. In: Genetic and evolutionary computation—GECCO 2003. Springer, Berlin, pp 1469–1480

    Chapter  Google Scholar 

  78. Gutiérrez R, Roman P, Torres F (2001) Inference on some parametric functions in the univariate lognormal diffusion process with exogenous factors. Test 10(2):357–373

    Article  MathSciNet  MATH  Google Scholar 

  79. Cho H, Olivera F, Guikema SD (2008) A derivation of the number of minima of the Griewank function. Appl Math Comput 204(2):694–701

    Article  MathSciNet  MATH  Google Scholar 

  80. Capocelli RM, Ricciardi LM (1974) A diffusion model for population growth in random environment. Theor Popul Biol 5:28–41

    Article  Google Scholar 

  81. Basel MAE, Ahmad SA, Wafaa MS (2004) Modelling the CPI using a lognormal diffusion process and implications on forecasting inflation. IMA J Manag Math 15:39–51

    Article  MathSciNet  MATH  Google Scholar 

  82. Herrera F, Lozano M, Sánchez AM (2003) A taxonomy for the crossover operator for real-coded genetic algorithms: an experimental study. Int J Intell Syst 18:309–338

    Article  MATH  Google Scholar 

  83. Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1):67–82

    Article  Google Scholar 

  84. Benjamini Yoav (1988) Opening the box of a boxplot. Am Stat 42(4):257–262

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Depto. Arquitectura y Tecnología de Computadores, ETSIIT, CITIC, Universidad de Granada, Granada, Spain

    P. A. Castillo, M. G. Arenas, N. Rico, A. M. Mora, P. García-Sánchez, J. L. J. Laredo & J. J. Merelo

Authors
  1. P. A. Castillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. G. Arenas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. N. Rico
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. A. M. Mora
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. P. García-Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. J. L. J. Laredo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. J. J. Merelo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to P. A. Castillo.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Castillo, P.A., Arenas, M.G., Rico, N. et al. Determining the significance and relative importance of parameters of a simulated quenching algorithm using statistical tools. Appl Intell 37, 239–254 (2012). https://doi.org/10.1007/s10489-011-0324-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-011-0324-x

Keywords

Search

Navigation