Skip to main content
Top
Published in: Structural and Multidisciplinary Optimization 4/2021

18-11-2020 | Research Paper

A multi-fidelity Bayesian optimization approach based on the expected further improvement

Authors: Leshi Shu, Ping Jiang, Yan Wang

Published in: Structural and Multidisciplinary Optimization | Issue 4/2021

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Sampling efficiency is important for simulation-based design optimization. While Bayesian optimization (BO) has been successfully applied in engineering problems, the cost associated with large-scale simulations has not been fully addressed. Extending the standard BO approaches to multi-fidelity optimization can utilize the information of low-fidelity models to further reduce the optimization cost. In this work, a multi-fidelity Bayesian optimization approach is proposed, in which hierarchical Kriging is used for constructing the multi-fidelity metamodel. The proposed approach quantifies the effect of HF and LF samples in multi-fidelity optimization based on a new concept of expected further improvement. A novel acquisition function is proposed to determine both the location and fidelity level of the next sample simultaneously, with the consideration of balance between the value of information provided by the new sample and the associated sampling cost. The proposed approach is compared with some state-of-the-art methods for multi-fidelity global optimization with numerical examples and an engineering case. The results show that the proposed approach can obtain global optimal solutions with reduced computational costs.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
go back to reference Bakr MH, Bandler JW, Madsen K, Søndergaard J (2001) An introduction to the space mapping technique. Optim Eng 2:369–384MathSciNetCrossRef Bakr MH, Bandler JW, Madsen K, Søndergaard J (2001) An introduction to the space mapping technique. Optim Eng 2:369–384MathSciNetCrossRef
go back to reference Bandler JW, Biernacki RM, Chen SH, Grobelny PA, Hemmers RH (1994) Space mapping technique for electromagnetic optimization. IEEE Trans Microw Theory Tech 42:2536–2544CrossRef Bandler JW, Biernacki RM, Chen SH, Grobelny PA, Hemmers RH (1994) Space mapping technique for electromagnetic optimization. IEEE Trans Microw Theory Tech 42:2536–2544CrossRef
go back to reference Cai X, Qiu H, Gao L, Yang P, Shao X (2016) An enhanced RBF-HDMR integrated with an adaptive sampling method for approximating high dimensional problems in engineering design. Struct Multidiscip Optim 53:1209–1229CrossRef Cai X, Qiu H, Gao L, Yang P, Shao X (2016) An enhanced RBF-HDMR integrated with an adaptive sampling method for approximating high dimensional problems in engineering design. Struct Multidiscip Optim 53:1209–1229CrossRef
go back to reference Chang KJ, Haftka RT, Giles GL, Kao I-J (1993) Sensitivity-based scaling for approximating structural response. J Aircr 30:283–288CrossRef Chang KJ, Haftka RT, Giles GL, Kao I-J (1993) Sensitivity-based scaling for approximating structural response. J Aircr 30:283–288CrossRef
go back to reference Coello CAC (2000) Use of a self-adaptive penalty approach for engineering optimization problems. Comput Ind 41:113–127CrossRef Coello CAC (2000) Use of a self-adaptive penalty approach for engineering optimization problems. Comput Ind 41:113–127CrossRef
go back to reference Courrier N, Boucard P-A, Soulier B (2016) Variable-fidelity modeling of structural analysis of assemblies. J Glob Optim 64:577–613MathSciNetCrossRef Courrier N, Boucard P-A, Soulier B (2016) Variable-fidelity modeling of structural analysis of assemblies. J Glob Optim 64:577–613MathSciNetCrossRef
go back to reference Gano SE, Renaud JE, Sanders B (2005) Hybrid variable fidelity optimization by using a kriging-based scaling function. AIAA J 43:2422–2433CrossRef Gano SE, Renaud JE, Sanders B (2005) Hybrid variable fidelity optimization by using a kriging-based scaling function. AIAA J 43:2422–2433CrossRef
go back to reference Gardner JR, Kusner MJ, Xu ZE, Weinberger KQ, Cunningham JP (2014) Bayesian optimization with inequality constraints. In: ICML. pp. 937–945 Gardner JR, Kusner MJ, Xu ZE, Weinberger KQ, Cunningham JP (2014) Bayesian optimization with inequality constraints. In: ICML. pp. 937–945
go back to reference Gelbart MA, Snoek J, Adams RP (2014) Bayesian optimization with unknown constraints. arXiv preprint arXiv:1403.5607 Gelbart MA, Snoek J, Adams RP (2014) Bayesian optimization with unknown constraints. arXiv preprint arXiv:1403.5607
go back to reference Ghoreishi SF, Allaire D (2019) Multi-information source constrained Bayesian optimization. Struct Multidiscip Optim 59:977–991MathSciNetCrossRef Ghoreishi SF, Allaire D (2019) Multi-information source constrained Bayesian optimization. Struct Multidiscip Optim 59:977–991MathSciNetCrossRef
go back to reference Ghoreishi SF, Molkeri A, Srivastava A, Arroyave R, Allaire D (2018) Multi-information source fusion and optimization to realize ICME: application to dual-phase materials. J Mech Des 140:111409CrossRef Ghoreishi SF, Molkeri A, Srivastava A, Arroyave R, Allaire D (2018) Multi-information source fusion and optimization to realize ICME: application to dual-phase materials. J Mech Des 140:111409CrossRef
go back to reference Ghoreishi SF, Molkeri A, Arróyave R, Allaire D, Srivastava A (2019) Efficient use of multiple information sources in material design. Acta Mater 180:260–271CrossRef Ghoreishi SF, Molkeri A, Arróyave R, Allaire D, Srivastava A (2019) Efficient use of multiple information sources in material design. Acta Mater 180:260–271CrossRef
go back to reference Han Z-H, Görtz S (2012) Hierarchical kriging model for variable-fidelity surrogate modeling. AIAA J 50:1885–1896CrossRef Han Z-H, Görtz S (2012) Hierarchical kriging model for variable-fidelity surrogate modeling. AIAA J 50:1885–1896CrossRef
go back to reference Huang D, Allen TT, Notz WI, Miller RA (2006) Sequential kriging optimization using multiple-fidelity evaluations. Struct Multidiscip Optim 32:369–382CrossRef Huang D, Allen TT, Notz WI, Miller RA (2006) Sequential kriging optimization using multiple-fidelity evaluations. Struct Multidiscip Optim 32:369–382CrossRef
go back to reference Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492MathSciNetCrossRef Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492MathSciNetCrossRef
go back to reference Kennedy MC, O'Hagan A (2000) Predicting the output from a complex computer code when fast approximations are available. Biometrika 87:1–13MathSciNetCrossRef Kennedy MC, O'Hagan A (2000) Predicting the output from a complex computer code when fast approximations are available. Biometrika 87:1–13MathSciNetCrossRef
go back to reference Kim Y, Lee S, Yee K, Rhee D-H (2017) High-to-low initial sample ratio of hierarchical kriging for film hole array optimization. J Propuls Power 34:108–115CrossRef Kim Y, Lee S, Yee K, Rhee D-H (2017) High-to-low initial sample ratio of hierarchical kriging for film hole array optimization. J Propuls Power 34:108–115CrossRef
go back to reference Koziel S, Bandler JW, Madsen K (2006) A space-mapping framework for engineering optimization—theory and implementation IEEE transactions on microwave. Theory Tech 54:3721–3730CrossRef Koziel S, Bandler JW, Madsen K (2006) A space-mapping framework for engineering optimization—theory and implementation IEEE transactions on microwave. Theory Tech 54:3721–3730CrossRef
go back to reference Liu Y, Chen S, Wang F, Xiong F (2018) Sequential optimization using multi-level co-Kriging and extended expected improvement criterion. Struct Multidiscip Optim 58:1155–1173MathSciNetCrossRef Liu Y, Chen S, Wang F, Xiong F (2018) Sequential optimization using multi-level co-Kriging and extended expected improvement criterion. Struct Multidiscip Optim 58:1155–1173MathSciNetCrossRef
go back to reference McIntire M, Ratner D, Ermon S. (2016). Sparse Gaussian processes for Bayesian optimization. Proceedings of the Thirty-Second Conference on Uncertainty in Artificial Intelligence, pp. 517-526 McIntire M, Ratner D, Ermon S. (2016). Sparse Gaussian processes for Bayesian optimization. Proceedings of the Thirty-Second Conference on Uncertainty in Artificial Intelligence, pp. 517-526
go back to reference Palar PS, Shimoyama K (2017) Multi-fidelity uncertainty analysis in CFD using hierarchical kriging. In: 35th AIAA Applied Aerodynamics Conference, p. 3261 Palar PS, Shimoyama K (2017) Multi-fidelity uncertainty analysis in CFD using hierarchical kriging. In: 35th AIAA Applied Aerodynamics Conference, p. 3261
go back to reference Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker PK (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 41:1–28CrossRef Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker PK (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 41:1–28CrossRef
go back to reference Schonlau M, Welch WJ, Jones DR (1998) Global versus local search in constrained optimization of computer models. Lecture Notes-Monograph Series 11-25 Schonlau M, Welch WJ, Jones DR (1998) Global versus local search in constrained optimization of computer models. Lecture Notes-Monograph Series 11-25
go back to reference Shu L, Jiang P, Wan L, Zhou Q, Shao X, Zhang Y (2017) Metamodel-based design optimization employing a novel sequential sampling strategy. Eng Comput 34:2547–2564CrossRef Shu L, Jiang P, Wan L, Zhou Q, Shao X, Zhang Y (2017) Metamodel-based design optimization employing a novel sequential sampling strategy. Eng Comput 34:2547–2564CrossRef
go back to reference Shu L, Jiang P, Song X, Zhou Q (2019a) Novel approach for selecting low-fidelity scale factor in multifidelity metamodeling. AIAA J 57:5320–5330CrossRef Shu L, Jiang P, Song X, Zhou Q (2019a) Novel approach for selecting low-fidelity scale factor in multifidelity metamodeling. AIAA J 57:5320–5330CrossRef
go back to reference Shu L, Jiang P, Zhou Q, Xie T (2019b) An online variable-fidelity optimization approach for multi-objective design optimization. Struct Multidiscip Optim 60:1059–1077MathSciNetCrossRef Shu L, Jiang P, Zhou Q, Xie T (2019b) An online variable-fidelity optimization approach for multi-objective design optimization. Struct Multidiscip Optim 60:1059–1077MathSciNetCrossRef
go back to reference Song C, Song W, Yang X (2017) Gradient-enhanced hierarchical kriging model for aerodynamic design optimization. J Aerosp Eng 30:04017072CrossRef Song C, Song W, Yang X (2017) Gradient-enhanced hierarchical kriging model for aerodynamic design optimization. J Aerosp Eng 30:04017072CrossRef
go back to reference Tran A, Sun J, Furlan JM, Pagalthivarthi KV, Visintainer RJ, Wang Y (2019a) pBO-2GP-3B: a batch parallel known/unknown constrained Bayesian optimization with feasibility classification and its applications in computational fluid dynamics. Comput Methods Appl Mech Eng 347:827–852MathSciNetCrossRef Tran A, Sun J, Furlan JM, Pagalthivarthi KV, Visintainer RJ, Wang Y (2019a) pBO-2GP-3B: a batch parallel known/unknown constrained Bayesian optimization with feasibility classification and its applications in computational fluid dynamics. Comput Methods Appl Mech Eng 347:827–852MathSciNetCrossRef
go back to reference Tran A, Tran M, Wang Y (2019b) Constrained mixed-integer Gaussian mixture Bayesian optimization and its applications in designing fractal and auxetic metamaterials. Struct Multidiscip Optim 59:2131–2154MathSciNetCrossRef Tran A, Tran M, Wang Y (2019b) Constrained mixed-integer Gaussian mixture Bayesian optimization and its applications in designing fractal and auxetic metamaterials. Struct Multidiscip Optim 59:2131–2154MathSciNetCrossRef
go back to reference Tran A, Wildey T, McCann S (2020) sMF-BO-2CoGP: a sequential multi-fidelity constrained Bayesian optimization framework for design applications. J Comput Inf Sci Eng 20:031007CrossRef Tran A, Wildey T, McCann S (2020) sMF-BO-2CoGP: a sequential multi-fidelity constrained Bayesian optimization framework for design applications. J Comput Inf Sci Eng 20:031007CrossRef
go back to reference Ulaganathan S, Couckuyt I, Ferranti F, Laermans E, Dhaene T (2015) Performance study of multi-fidelity gradient enhanced kriging. Struct Multidiscip Optim 51:1017–1033CrossRef Ulaganathan S, Couckuyt I, Ferranti F, Laermans E, Dhaene T (2015) Performance study of multi-fidelity gradient enhanced kriging. Struct Multidiscip Optim 51:1017–1033CrossRef
go back to reference Wang GG (2003) Adaptive response surface method using inherited Latin hypercube design points. J Mech Des 125:210–220CrossRef Wang GG (2003) Adaptive response surface method using inherited Latin hypercube design points. J Mech Des 125:210–220CrossRef
go back to reference Wang GG, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. J Mech Des 129:370–380CrossRef Wang GG, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. J Mech Des 129:370–380CrossRef
go back to reference Xiong Y, Chen W, Tsui K-L (2008) A new variable-fidelity optimization framework based on model fusion and objective-oriented sequential sampling. J Mech Des 130:111401CrossRef Xiong Y, Chen W, Tsui K-L (2008) A new variable-fidelity optimization framework based on model fusion and objective-oriented sequential sampling. J Mech Des 130:111401CrossRef
go back to reference Zhang Y, Han Z-H, Liu J, Song W-P (2015) Efficient variable-fidelity optimization applied to benchmark transonic airfoil design. In: 7th Asia-Pac int Symp Aerosp Technol, Cairns, Australia, pp. 25–27 Zhang Y, Han Z-H, Liu J, Song W-P (2015) Efficient variable-fidelity optimization applied to benchmark transonic airfoil design. In: 7th Asia-Pac int Symp Aerosp Technol, Cairns, Australia, pp. 25–27
go back to reference Zhang Y, Han Z-H, Zhang K-S (2018) Variable-fidelity expected improvement method for efficient global optimization of expensive functions. Struct Multidiscip Optim 58:1431–1451MathSciNetCrossRef Zhang Y, Han Z-H, Zhang K-S (2018) Variable-fidelity expected improvement method for efficient global optimization of expensive functions. Struct Multidiscip Optim 58:1431–1451MathSciNetCrossRef
go back to reference Zhang J, Yao X, Liu M, Wang Y. (2019) A Bayesian discrete optimization algorithm for permutation problems. Proceedings of 2019 IEEE Symposium Series on Computational Intelligence (SSCI 2019), pp.871-881 Zhang J, Yao X, Liu M, Wang Y. (2019) A Bayesian discrete optimization algorithm for permutation problems. Proceedings of 2019 IEEE Symposium Series on Computational Intelligence (SSCI 2019), pp.871-881
go back to reference Zhou Q, Shao X, Jiang P, Zhou H, Shu L (2015) An adaptive global variable fidelity metamodeling strategy using a support vector regression based scaling function. Simul Model Pract Theory 59:18–35CrossRef Zhou Q, Shao X, Jiang P, Zhou H, Shu L (2015) An adaptive global variable fidelity metamodeling strategy using a support vector regression based scaling function. Simul Model Pract Theory 59:18–35CrossRef
go back to reference Zhou Q, Shao X, Jiang P, Gao Z, Wang C, Shu L (2016) An active learning metamodeling approach by sequentially exploiting difference information from variable-fidelity models. Adv Eng Inform 30:283–297CrossRef Zhou Q, Shao X, Jiang P, Gao Z, Wang C, Shu L (2016) An active learning metamodeling approach by sequentially exploiting difference information from variable-fidelity models. Adv Eng Inform 30:283–297CrossRef
go back to reference Zhou Q, Wang Y, Choi S-K, Jiang P, Shao X, Hu J (2017) A sequential multi-fidelity metamodeling approach for data regression. Knowl-Based Syst 134:199–212CrossRef Zhou Q, Wang Y, Choi S-K, Jiang P, Shao X, Hu J (2017) A sequential multi-fidelity metamodeling approach for data regression. Knowl-Based Syst 134:199–212CrossRef
Metadata
Title
A multi-fidelity Bayesian optimization approach based on the expected further improvement
Authors
Leshi Shu
Ping Jiang
Yan Wang
Publication date
18-11-2020
Publisher
Springer Berlin Heidelberg
Published in
Structural and Multidisciplinary Optimization / Issue 4/2021
Print ISSN: 1615-147X
Electronic ISSN: 1615-1488
DOI
https://doi.org/10.1007/s00158-020-02772-4

Other articles of this Issue 4/2021

Structural and Multidisciplinary Optimization 4/2021 Go to the issue

Premium Partners