Skip to main content

2016 | OriginalPaper | Buchkapitel

Exploring Sparse Covariance Estimation Techniques in Evolution Strategies

verfasst von : Silja Meyer-Nieberg, Erik Kropat

Erschienen in: Recent Advances in Computational Optimization

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

When considering continuous search spaces, evolution strategies are among the well-performing metaheuristics. In contrast to other evolutionary algorithms, their main search operator is mutation which necessitates its adaptation during the run. Here, the covariance matrix plays an important role. Modern Evolution Strategies apply forms of covariance matrix adaptation. However, the quality of the common estimate of the covariance is known to be questionable for high search space dimensions. This paper presents a new approach by considering sparse covariance matrix techniques together with a space transformation.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
Literatur
1.
Zurück zum Zitat Bäck, T., Foussette, C., Krause, Peter: Contemporary Evolution Strategies. Natural Computing. Springer, Berlin (2013)CrossRefMATH Bäck, T., Foussette, C., Krause, Peter: Contemporary Evolution Strategies. Natural Computing. Springer, Berlin (2013)CrossRefMATH
2.
Zurück zum Zitat Beyer, H.-G., Meyer-Nieberg, S.: Self-adaptation of evolution strategies under noisy fitness evaluations. Genet. Program. Evolv. Mach. 7(4), 295–328 (2006)CrossRef Beyer, H.-G., Meyer-Nieberg, S.: Self-adaptation of evolution strategies under noisy fitness evaluations. Genet. Program. Evolv. Mach. 7(4), 295–328 (2006)CrossRef
4.
Zurück zum Zitat Beyer, H.-G., Sendhoff, B.: Lecture Notes in Computer Science. In: Rudolph, G., et al. (eds.) PPSN. Covariance matrix adaptation revisited - the CMSA evolution strategy, vol. 5199, pp. 123–132. Springer, Berlin (2008) Beyer, H.-G., Sendhoff, B.: Lecture Notes in Computer Science. In: Rudolph, G., et al. (eds.) PPSN. Covariance matrix adaptation revisited - the CMSA evolution strategy, vol. 5199, pp. 123–132. Springer, Berlin (2008)
5.
Zurück zum Zitat Cai, Tony, Liu, Weidong: Adaptive thresholding for sparse covariance matrix estimation. J. Am. Stat. Assoc. 106(494), 672–684 (2011)MathSciNetCrossRefMATH Cai, Tony, Liu, Weidong: Adaptive thresholding for sparse covariance matrix estimation. J. Am. Stat. Assoc. 106(494), 672–684 (2011)MathSciNetCrossRefMATH
6.
Zurück zum Zitat Chen, X., Wang, Z.J., McKeown, M.J.: Shrinkage-to-tapering estimation of large covariance matrices. IEEE Trans. Signal Process. 60(11), 5640–5656 (2012)MathSciNetCrossRef Chen, X., Wang, Z.J., McKeown, M.J.: Shrinkage-to-tapering estimation of large covariance matrices. IEEE Trans. Signal Process. 60(11), 5640–5656 (2012)MathSciNetCrossRef
7.
Zurück zum Zitat Dong, W., Yao, X.: Covariance matrix repairing in gaussian based EDAs. In: 2007 IEEE Congress on Evolutionary Computation, 2007. CEC, pp. 415–422 (2007) Dong, W., Yao, X.: Covariance matrix repairing in gaussian based EDAs. In: 2007 IEEE Congress on Evolutionary Computation, 2007. CEC, pp. 415–422 (2007)
8.
Zurück zum Zitat Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing. Natural Computing Series. Springer, Berlin (2003)CrossRefMATH Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing. Natural Computing Series. Springer, Berlin (2003)CrossRefMATH
10.
Zurück zum Zitat Fan, J., Liao, Y., Mincheva, Martina: Large covariance estimation by thresholding principal orthogonal complements. J. R. Stat. Soc.: Ser. B (Stat. Methodol.) 75(4), 603–680 (2013)MathSciNetCrossRef Fan, J., Liao, Y., Mincheva, Martina: Large covariance estimation by thresholding principal orthogonal complements. J. R. Stat. Soc.: Ser. B (Stat. Methodol.) 75(4), 603–680 (2013)MathSciNetCrossRef
11.
Zurück zum Zitat Finck, S., Hansen, N., Ros, R., Auger, A.: Real-parameter black-box optimization benchmarking 2010: presentation of the noiseless functions. Technical report, Institute National de Recherche en Informatique et Automatique (2010) 2009/22 Finck, S., Hansen, N., Ros, R., Auger, A.: Real-parameter black-box optimization benchmarking 2010: presentation of the noiseless functions. Technical report, Institute National de Recherche en Informatique et Automatique (2010) 2009/22
12.
Zurück zum Zitat Fisher, T.J., Sun, Xiaoqian: Improved Stein-type shrinkage estimators for the high-dimensional multivariate normal covariance matrix. Comput. Stat. Data Anal. 55(5), 1909–1918 (2011)MathSciNetCrossRefMATH Fisher, T.J., Sun, Xiaoqian: Improved Stein-type shrinkage estimators for the high-dimensional multivariate normal covariance matrix. Comput. Stat. Data Anal. 55(5), 1909–1918 (2011)MathSciNetCrossRefMATH
13.
Zurück zum Zitat Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3), 432–441 (2008)CrossRefMATH Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3), 432–441 (2008)CrossRefMATH
14.
Zurück zum Zitat Guillot, D., Rajaratnam, B.: Functions preserving positive definiteness for sparse matrices. Trans. Am. Math. Soc. 367(1), 627–649 (2015)MathSciNetCrossRefMATH Guillot, D., Rajaratnam, B.: Functions preserving positive definiteness for sparse matrices. Trans. Am. Math. Soc. 367(1), 627–649 (2015)MathSciNetCrossRefMATH
15.
Zurück zum Zitat Hansen, N.: The CMA evolution strategy: a comparing review. In: Lozano, J.A. et al., (ed.) Towards a new evolutionary computation. Advances in estimation of distribution algorithms, pp. 75–102. Springer (2006) Hansen, N.: The CMA evolution strategy: a comparing review. In: Lozano, J.A. et al., (ed.) Towards a new evolutionary computation. Advances in estimation of distribution algorithms, pp. 75–102. Springer (2006)
16.
Zurück zum Zitat Hansen, N., Auger, A., Finck, S., Ros, R.: Real-parameter black-box optimization benchmarking 2012: Experimental setup. Technical report, INRIA (2012) Hansen, N., Auger, A., Finck, S., Ros, R.: Real-parameter black-box optimization benchmarking 2012: Experimental setup. Technical report, INRIA (2012)
17.
Zurück zum Zitat Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolut. Comput. 9(2), 159–195 (2001)CrossRef Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolut. Comput. 9(2), 159–195 (2001)CrossRef
18.
Zurück zum Zitat Hansen, Nikolaus: Adaptive encoding: How to render search coordinate system invariant. In: Rudolph, G., Jansen, T., Beume, N., Lucas, Simon, Poloni, Carlo (eds.) Parallel Problem Solving from Nature PPSN X. Lecture Notes in Computer Science, vol. 5199, pp. 205–214. Springer, Berlin (2008)CrossRef Hansen, Nikolaus: Adaptive encoding: How to render search coordinate system invariant. In: Rudolph, G., Jansen, T., Beume, N., Lucas, Simon, Poloni, Carlo (eds.) Parallel Problem Solving from Nature PPSN X. Lecture Notes in Computer Science, vol. 5199, pp. 205–214. Springer, Berlin (2008)CrossRef
19.
Zurück zum Zitat Ledoit, O., Wolf, Michael: A well-conditioned estimator for large dimensional covariance matrices. J. Multivar. Anal. Arch. 88(2), 265–411 (2004)MathSciNetMATH Ledoit, O., Wolf, Michael: A well-conditioned estimator for large dimensional covariance matrices. J. Multivar. Anal. Arch. 88(2), 265–411 (2004)MathSciNetMATH
20.
Zurück zum Zitat Levina, E., Rothman, A., Zhu, J.: Sparse estimation of large covariance matrices via a nested lasso penalty. Ann. Appl. Stat. 2(1), 245–263 (2008)MathSciNetCrossRefMATH Levina, E., Rothman, A., Zhu, J.: Sparse estimation of large covariance matrices via a nested lasso penalty. Ann. Appl. Stat. 2(1), 245–263 (2008)MathSciNetCrossRefMATH
21.
Zurück zum Zitat Marčenko, V.A., Pastur, L.A.: Distribution of eigenvalues for some sets of random matrices. Sbornik: Math. 1(4), 457–483 (1967)CrossRef Marčenko, V.A., Pastur, L.A.: Distribution of eigenvalues for some sets of random matrices. Sbornik: Math. 1(4), 457–483 (1967)CrossRef
22.
Zurück zum Zitat Meyer-Nieberg, S., Kropat, E: Adapting the covariance in evolution strategies. In: Proceedings of ICORES 2014, pp. 89–99. SCITEPRESS (2014) Meyer-Nieberg, S., Kropat, E: Adapting the covariance in evolution strategies. In: Proceedings of ICORES 2014, pp. 89–99. SCITEPRESS (2014)
23.
Zurück zum Zitat Meyer-Nieberg, S., Kropat, E.: Communications in Computer and Information Science. In: Pinson, E., Valente, F., Vitoriano, B. (eds.) Operations Research and Enterprise System. A new look at the covariance matrix estimation in evolution strategies, vol. 509, pp. 157–172. Springer International Publishing, Berlin (2015) Meyer-Nieberg, S., Kropat, E.: Communications in Computer and Information Science. In: Pinson, E., Valente, F., Vitoriano, B. (eds.) Operations Research and Enterprise System. A new look at the covariance matrix estimation in evolution strategies, vol. 509, pp. 157–172. Springer International Publishing, Berlin (2015)
24.
Zurück zum Zitat Pourahmadi, M.: High-Dimensional Covariance Estimation: With High-Dimensional Data. Wiley, New York (2013)CrossRefMATH Pourahmadi, M.: High-Dimensional Covariance Estimation: With High-Dimensional Data. Wiley, New York (2013)CrossRefMATH
25.
Zurück zum Zitat Rechenberg, I.: Evolutionsstrategie: Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. Frommann-Holzboog, Stuttgart (1973) Rechenberg, I.: Evolutionsstrategie: Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. Frommann-Holzboog, Stuttgart (1973)
26.
Zurück zum Zitat Ros, R., Hansen, N.: Parallel Problem Solving from Nature – PPSN X: 10th International Conference, Dortmund, Germany, Sept 13-17, 2008. Proceedings, chapter A Simple Modification in CMA-ES Achieving Linear Time and Space Complexity, pp. 296–305. Springer, Heidelberg (2008) Ros, R., Hansen, N.: Parallel Problem Solving from Nature – PPSN X: 10th International Conference, Dortmund, Germany, Sept 13-17, 2008. Proceedings, chapter A Simple Modification in CMA-ES Achieving Linear Time and Space Complexity, pp. 296–305. Springer, Heidelberg (2008)
27.
Zurück zum Zitat Schäffer, J., Strimmer, K.: A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Stat. Appl. Genet. Mol. Biol. 4(1), 32 (2005)MathSciNet Schäffer, J., Strimmer, K.: A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Stat. Appl. Genet. Mol. Biol. 4(1), 32 (2005)MathSciNet
28.
Zurück zum Zitat Schwefel, H.-P.: Numerical Optimization of Computer Models. Wiley, Chichester (1981)MATH Schwefel, H.-P.: Numerical Optimization of Computer Models. Wiley, Chichester (1981)MATH
29.
Zurück zum Zitat Stein, C.: Inadmissibility of the usual estimator for the mean of a multivariate distribution. In: Proceedings of 3rd Berkeley Symposium on Mathematical Statistics Probability, vol.1, pp. 197–206. Berkeley, CA (1956) Stein, C.: Inadmissibility of the usual estimator for the mean of a multivariate distribution. In: Proceedings of 3rd Berkeley Symposium on Mathematical Statistics Probability, vol.1, pp. 197–206. Berkeley, CA (1956)
30.
Zurück zum Zitat Stein, C.: Estimation of a covariance matrix. In: Rietz Lecture, 39th Annual Meeting. IMS, Atlanta, GA (1975) Stein, C.: Estimation of a covariance matrix. In: Rietz Lecture, 39th Annual Meeting. IMS, Atlanta, GA (1975)
Metadaten
Titel
Exploring Sparse Covariance Estimation Techniques in Evolution Strategies
verfasst von
Silja Meyer-Nieberg
Erik Kropat
Copyright-Jahr
2016
DOI
https://doi.org/10.1007/978-3-319-40132-4_15