Top

Published in:

2020 | OriginalPaper | Chapter

Bayesian Optimization Approaches for Massively Multi-modal Problems

Authors : Ibai Roman, Alexander Mendiburu, Roberto Santana, Jose A. Lozano

Published in: Learning and Intelligent Optimization

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

The optimization of massively multi-modal functions is a challenging task, particularly for problems where the search space can lead the optimization process to local optima. While evolutionary algorithms have been extensively investigated for these optimization problems, Bayesian Optimization algorithms have not been explored to the same extent. In this paper, we study the behavior of Bayesian Optimization as part of a hybrid approach for solving several massively multi-modal functions. We use well-known benchmarks and metrics to evaluate how different variants of Bayesian Optimization deal with multi-modality.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Optimization of Generalized Halton Sequences by Differential Evolution

Brochu, E., Cora, V.M., de Freitas, N.: A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning, December 2010. arXiv:1012.2599

Fieldsend, J.E.: Running up those hills: multi-modal search with the niching migratory multi-swarm optimiser. In: 2014 IEEE Congress on Evolutionary Computation (CEC), pp. 2593–2600. IEEE (2014)

Ginsbourger, D., Riche, R.L., Carraro, L.: A multi-points criterion for deterministic parallel global optimization based on Gaussian processes. Technical report, CCSD, March 2008. https://hal.archives-ouvertes.fr/hal-00260579/document

Goldberg, D.E., Richardson, J.: Genetic algorithms with sharing for multi-modal function optimisation. In: Proceedings of the of Second International Conference on Genetic Algorithms and Their Applications, pp. 41–49 (1987)

González, J., Dai, Z., Hennig, P., Lawrence, N.D.: Batch Bayesian optimization via local penalization (2015). arXiv:1505.08052

Guhaniyogi, R., Li, C., Savitsky, T.D., Srivastava, S.: A divide-and-conquer Bayesian approach to large-scale kriging, December 2017. arXiv:1712.09767

Jones, D.R., Perttunen, C.D., Stuckman, B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79(1), 157–181 (1993). https://doi.org/10.1007/BF00941892MathSciNetCrossRefMATH

Jones, D.R.: A taxonomy of global optimization methods based on response surfaces. J. Global Optim. 21(4), 345–383 (2001). https://doi.org/10.1023/A:1012771025575MathSciNetCrossRefMATH

Kvasov, D.E., Sergeyev, Y.D.: Deterministic approaches for solving practical black-box global optimization problems. Adv. Eng. Softw. 80, 58–66 (2015). https://doi.org/10.1016/j.advengsoft.2014.09.014. http://www.sciencedirect.com/science/article/pii/S096599781400163XCrossRef

10.

Li, X., Engelbrecht, A., Epitropakis, M.G.: Benchmark functions for CEC 2013 special session and competition on niching methods for multimodal function optimization. Technical report, Royal Melbourne Institute of Technology, March 2013. http://goanna.cs.rmit.edu.au/xiaodong/cec13-niching/

11.

Mockus, J.: The Bayesian approach to local optimization. In: Mockus, J. (ed.) Bayesian Approach to Global Optimization. Mathematics and Its Applications, vol. 37, pp. 125–156. Springer, Dordrecht (1989). https://doi.org/10.1007/978-94-009-0909-0_7CrossRefMATH

12.

Petrowski, A.: A clearing procedure as a niching method for genetic algorithms. In: Proceedings of IEEE International Conference on Evolutionary Computation, pp. 798–803 (1996). https://doi.org/10.1109/ICEC.1996.542703

13.

Powell, M.J.D.: An efficient method for finding the minimum of a function of several variables without calculating derivatives. Comput. J. 7(2), 155–162 (1964). https://doi.org/10.1093/comjnl/7.2.155. http://comjnl.oxfordjournals.org/content/7/2/155MathSciNetCrossRefMATH

14.

Preuss, M.: Niching the CMA-ES via nearest-better clustering. In: Proceedings of the 12th Annual Conference Companion on Genetic and Evolutionary Computation, GECCO 2010, pp. 1711–1718. ACM, New York (2010). https://doi.org/10.1145/1830761.1830793

15.

Preuss, M.: Multimodal Optimization by Means of Evolutionary Algorithms. Natural Computing Series. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-07407-8CrossRefMATH

16.

Quiñonero-Candela, J., Rasmussen, C.E.: A unifying view of sparse approximate Gaussian process regression. J. Mach. Learn. Res. 6(Dec), 1939–1959 (2005). http://www.jmlr.org/papers/v6/quinonero-candela05aMathSciNetMATH

17.

Rasmussen, C.E., Williams, C.K.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)MATH

18.

Roman, I., Santana, R., Mendiburu, A., Lozano, J.A.: Dynamic kernel selection criteria for Bayesian optimization. In: BayesOpt 2014: NIPS Workshop on Bayesian Optimization, Montreal (2014). http://bayesopt.github.io/papers/paper13.pdf

19.

Sareni, B., Krahenbuhl, L.: Fitness sharing and niching methods revisited. IEEE Trans. Evol. Comput. 2(3), 97–106 (1998)CrossRef

20.

Silverman, B.W.: Some aspects of the spline smoothing approach to non-parametric regression curve fitting. J. Roy. Stat. Soc. Series B (Method.) 47(1), 1–52 (1985)MathSciNetMATH

21.

Singh, G., Deb, K.: Comparison of multi-modal optimization algorithms based on evolutionary algorithms. In: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, GECCO 2006, pp. 1305–1312. ACM, New York (2006). https://doi.org/10.1145/1143997.1144200

22.

Srinivas, N., Krause, A., Kakade, S., Seeger, M.: Gaussian process optimization in the bandit setting: no regret and experimental design. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), Haifa, 21–24 June 2010, pp. 1015–1022 (2010). http://www.icml2010.org/papers/422.pdf

23.

Wang, H., van Stein, B., Emmerich, M., Bäck, T.: Time complexity reduction in efficient global optimization using cluster kriging. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2017, pp. 889–896. ACM, New York (2017). https://doi.org/10.1145/3071178.3071321

24.

Wong, K.C., Leung, K.S., Wong, M.H.: Protein structure prediction on a lattice model via multimodal optimization techniques. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, GECCO 2010, pp. 155–162. ACM, New York (2010). https://doi.org/10.1145/1830483.1830513

25.

Yin, X., Germay, N.: Investigations on solving the load flow problem by genetic algorithms. Electr. Power Syst. Res. 22(3), 151–163 (1991). https://doi.org/10.1016/0378-7796(91)90001-4. http://www.sciencedirect.com/science/article/pii/0378779691900014CrossRef

26.

Zhigljavsky, A., Žilinskas, A.: Methods based on statistical models of multimodal functions. In: Zhigljavsky, A., Žilinskas, A. (eds.) Stochastic Global Optimization. Springer Optimization and Its Applications, vol. 9, pp. 149–244. Springer, Boston (2008). https://doi.org/10.1007/978-0-387-74740-8_4CrossRefMATH

Title: Bayesian Optimization Approaches for Massively Multi-modal Problems
Authors: Ibai Roman
Alexander Mendiburu
Roberto Santana
Jose A. Lozano
Publisher: Springer International Publishing
Book: Learning and Intelligent Optimization
Print ISBN: 978-3-030-38628-3

Electronic ISBN: 978-3-030-38629-0

Copyright Year: 2020
DOI: https://doi.org/10.1007/978-3-030-38629-0_31

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner