Abstract
In many global optimization problems motivated by engineering applications, the number of function evaluations is severely limited by time or cost. To ensure that each of these evaluations usefully contributes to the localization of good candidates for the role of global minimizer, a stochastic model of the function can be built to conduct a sequential choice of evaluation points. Based on Gaussian processes and Kriging, the authors have recently introduced the informational approach to global optimization (IAGO) which provides a one-step optimal choice of evaluation points in terms of reduction of uncertainty on the location of the minimizers. To do so, the probability density of the minimizers is approximated using conditional simulations of the Gaussian process model behind Kriging. In this paper, an empirical comparison between the underlying sampling criterion called conditional minimizer entropy (CME) and the standard expected improvement sampling criterion (EI) is presented. Classical test functions are used as well as sample paths of the Gaussian model and an industrial application. They show the interest of the CME sampling criterion in terms of evaluation savings.
Similar content being viewed by others
References
Ackley D. (1987). A Connectionist Machine for Genetic Hill-climbing. Kluwer Academic Publishers, Norwell
Barton R. (1984). Minimization algorithms for functions with random noise. Am. J. Math. Manage. Sci. 4: 109–138
Branin F. (1972). Widely convergent methods for finding multiple solutions of simultaneous nonlinear equations. IBM J. Res. Dev. 16: 504–522
Chilès J. and Delfiner P. (1999). Geostatistics, Modeling Spatial Uncertainty. Wiley, New York
Cover T.M. and Thomas A.J. (1991). Elements of Information Theory. Wiley, New York
Geman, D., Jedynak, B.: An active testing model for tracking roads in satellite images. Tech. Rep. 2757, Institut National de Recherche en Informatique et en Automatique (INRIA) (1995)
Hartman J. (1973). Some experiments in global optimization. Nav. Res. Logist. Q. 20: 569–576
Huang D., Allen T., Notz W. and Zeng N. (2006). Global optimization of stochastic black-box systems via sequential kriging meta-models. J. Glob. Optim. 34: 441–466
Jones D. (2001). A taxonomy of global optimization methods based on response surfaces. J. Glob. Optim. 21: 345–383
Jones D., Schonlau M. and William J. (1998). Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13: 455–492
Knowles J. (2003). Parego: a hybrid algorithm with on-line landscape approxiamtion for expensive multiobjective optimization problems. IEEE Trans. Evol. Comput. 7(2): 100–116
Knowles, J., Thiele, L., Zitzler, E.: A tutorial on the performance assessment of stochastive multiobjective optimizers. Tech. Rep. 214, Computer Engineering and Networks Laboratory, ETH Zurich (2006)
Lumley J. (1999). Engines: An Introduction. Cambridge University Press, Cambridge
Mockus J. (1989). Bayesian Approach to Global Optimization: Theory and Applications. Kluwer Academic Publishers, Dordrecht
Myers R. and Montgomery D. (2002). Response Surface Methodology: Process and Product Optimization Using Designed Experiments. Wiley-Interscience, New York
Perttunen, C.: A computational geometric approach to feasible region division in constrained global optimization. In: Proceedings of the 1991 IEEE Conference on Systems, Man, and Cybernetics, vol. 1, pp. 585–590 (1991)
Sasena M., Papalambros P. and Goovaerts P. (2002). Exploration of metamodeling sampling criteria for constrained global optimization. Eng. Opt. 34: 263–278
Stein M. (1999). Interpolation of Spatial Data: Some Theory for Kriging. Springer, New-York
Vazquez, E., Walter, É.: Estimating derivatives and integrals with Kriging. In: Proceedings of the Joint 44th IEEE Conference on Decision and European Control Conference, pp. 8156–8161 (2005)
Villemonteix, J., Vazquez, E., Walter, É.: An informational approach to the global optimization of expensive-to-evaluate functions. J. Glob. Optim. (2006) (Submitted)
Wahba G. (1998). Support vector machines, reproducing kernel Hilbert spaces and randomized GACV. In: Schölkopf, B., Burges, C., and Smola, A. (eds) Advances in Kernel Methods—Support Vector Learning, vol. 6, pp 69–87. MIT Press, Boston
Williams B., Santner T. and Notz W. (2000). Sequential design of computer experiments to minimize integrated response functions. Stat. Sin. 10: 1133–1152
Yaglom A. (1986). Correlation Theory of Stationary and Related Random Functions I: Basic Results. Springer Series in Statistics. Springer-Verlag, New-York
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Villemonteix, J., Vazquez, E., Sidorkiewicz, M. et al. Global optimization of expensive-to-evaluate functions: an empirical comparison of two sampling criteria. J Glob Optim 43, 373–389 (2009). https://doi.org/10.1007/s10898-008-9313-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-008-9313-y