Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Learning Enabled Constrained Black-Box Optimization Read chapter Read first chapter

2021 | OriginalPaper | Chapter

Black-Box Optimization: Methods and Applications

Authors : Ishan Bajaj, Akhil Arora, M. M. Faruque Hasan

Published in: Black Box Optimization, Machine Learning, and No-Free Lunch Theorems

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Black-box optimization (BBO) is a rapidly growing field of optimization and a topic of critical importance in many areas including complex systems engineering, energy and the environment, materials design, drug discovery, chemical process synthesis, and computational biology. In this chapter, we present an overview of theoretical advancements, algorithmic developments, implementations, and several applications of BBO. Lastly, we point to open problems and provide future research directions.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Learning Enabled Constrained Black-Box Optimization
next chapter Tuning Algorithms for Stochastic Black-Box Optimization: State of the Art and Future Perspectives
Literature
1.
Abdollahi, A., Shams, M.: Optimization of heat transfer enhancement of nanofluid in a channel with winglet vortex generator. Appl. Thermal Eng. 91, 1116–1126 (2015)CrossRef
2.
Abramson, M.A., Audet, C., Dennis, J.E.: Generalized pattern searches with derivative information. Math. Program. 100(1), 3–25 (2004)MathSciNetMATHCrossRef
3.
Abramson, M.A., Audet, C., Dennis Jr, J.: Filter pattern search algorithms for mixed variable constrained optimization problems. Tech. rep., Air Force Inst of Tech Wright-Patterson AFB OH (2004)
4.
Abramson, M.A., Audet, C., Dennis Jr, J.E., Digabel, S.L.: OrthoMADS: a deterministic MADS instance with orthogonal directions. SIAM J. Optim. 20(2), 948–966 (2009)MathSciNetMATHCrossRef
5.
Arora, A., Bajaj, I., Iyer, S.S., Hasan, M.F.: Optimal synthesis of periodic sorption enhanced reaction processes with application to hydrogen production. Comput. Chem. Eng. 115, 89–111 (2018)CrossRef
6.
Arora, A., Iyer, S.S., Bajaj, I., Hasan, M.F.: Optimal methanol production via sorption-enhanced reaction process. Ind. Eng. Chem. Res. 57(42), 14143–14161 (2018)CrossRef
7.
Artus, V., Durlofsky, L.J., Onwunalu, J., Aziz, K.: Optimization of nonconventional wells under uncertainty using statistical proxies. Comput. Geosci. 10(4), 389–404 (2006)MATHCrossRef
8.
Asadollahi, M., Nævdal, G., Dadashpour, M., Kleppe, J.: Production optimization using derivative free methods applied to Brugge field case. J. Pet. Sci. Eng. 114, 22–37 (2014)CrossRef
9.
10.
11.
Audet, C., Dennis Jr, J.E.: A pattern search filter method for nonlinear programming without derivatives. SIAM J. Optim. 14(4), 980–1010 (2004)MathSciNetMATHCrossRef
12.
Audet, C., Dennis Jr, J.E.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)MathSciNetMATHCrossRef
13.
Audet, C., Dennis Jr, J.E.: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20(1), 445–472 (2009)MathSciNetMATHCrossRef
14.
15.
Audet, C., Ianni, A., Le Digabel, S., Tribes, C.: Reducing the number of function evaluations in mesh adaptive direct search algorithms. SIAM J. Optim. 24(2), 621–642 (2014)MathSciNetMATHCrossRef
16.
Audet, C., Ihaddadene, A., Le Digabel, S., Tribes, C.: Robust optimization of noisy blackbox problems using the mesh adaptive direct search algorithm. Optim. Lett. 12(4), 675–689 (2018)MathSciNetMATHCrossRef
17.
Audet, C., Orban, D.: Finding optimal algorithmic parameters using derivative-free optimization. SIAM J. Optim. 17(3), 642–664 (2006)MathSciNetMATHCrossRef
18.
Aytug, H., Khouja, M., Vergara, F.: Use of genetic algorithms to solve production and operations management problems: a review. Int. J. Prod. Res. 41(17), 3955–4009 (2003)CrossRef
19.
Bajaj, I., Hasan, M.F.: Deterministic global derivative-free optimization of black-box problems with bounded hessian. Optim. Lett. 14, 1011–1026 (2020)MathSciNetMATHCrossRef
20.
Bajaj, I., Hasan, M.F.: UNIPOPT: Univariate projection-based optimization without derivatives. Comput. Chem. Eng. 127, 71–87 (2019)CrossRef
21.
Bajaj, I., Iyer, S.S., Hasan, M.F.: A trust region-based two phase algorithm for constrained black-box and grey-box optimization with infeasible initial point. Comput. Chem. Eng. 116, 306–321 (2018)CrossRef
22.
Bangerth, W., Klie, H., Wheeler, M.F., Stoffa, P.L., Sen, M.K.: On optimization algorithms for the reservoir oil well placement problem. Comput. Geosci. 10(3), 303–319 (2006)MATHCrossRef
23.
Bélisle, C.J., Romeijn, H.E., Smith, R.L.: Hit-and-run algorithms for generating multivariate distributions. Math. Oper. Res. 18(2), 255–266 (1993)MathSciNetMATHCrossRef
24.
Berbee, H., Boender, C., Ran, A.R., Scheffer, C., Smith, R.L., Telgen, J.: Hit-and-run algorithms for the identification of nonredundant linear inequalities. Math. Program. 37(2), 184–207 (1987)MathSciNetMATHCrossRef
25.
Bischl, B., Richter, J., Bossek, J., Horn, D., Thomas, J., Lang, M.: mlrMBO: a modular framework for model-based optimization of expensive black-box functions (2017). arXiv preprint arXiv:1703.03373
26.
Bittencourt, A.C., Horne, R.N., et al.: Reservoir development and design optimization. In: SPE Annual Technical Conference and Exhibition. Society of Petroleum Engineers (1997)
27.
Björkman, M., Holmström, K.: Global optimization using the direct algorithm in Matlab (1999)
28.
Boneh, A., Golan, A.: Constraints redundancy and feasible region boundedness by random feasible point generator (RFPG). In: Third European Congress on Operations Research (EURO III), Amsterdam (1979)
29.
Booker, A.J., Dennis, J.E., Frank, P.D., Serafini, D.B., Torczon, V., Trosset, M.W.: A rigorous framework for optimization of expensive functions by surrogates. Struct. Optim. 17(1), 1–13 (1999)CrossRef
30.
Boukouvala, F., Hasan, M.F., Floudas, C.A.: Global optimization of general constrained grey-box models: new method and its application to constrained PDEs for pressure swing adsorption. J. Global Optim. 67(1–2), 3–42 (2017)MathSciNetMATHCrossRef
31.
Boukouvala, F., Ierapetritou, M.G.: Surrogate-based optimization of expensive flowsheet modeling for continuous pharmaceutical manufacturing. J. Pharm. Innov. 8(2), 131–145 (2013)CrossRef
32.
Boukouvala, F., Ierapetritou, M.G.: Derivative-free optimization for expensive constrained problems using a novel expected improvement objective function. AIChE J. 60(7), 2462–2474 (2014)CrossRef
33.
Boukouvala, F., Misener, R., Floudas, C.A.: Global optimization advances in mixed-integer nonlinear programming, MINLP, and constrained derivative-free optimization, CDFO. Eur. J. Oper. Res. 252(3), 701–727 (2016)MathSciNetMATHCrossRef
34.
Bűrmen, Á., Puhan, J., Tuma, T.: Grid restrained Nelder-Mead algorithm. Comput. Optim. Appl. 34(3), 359–375 (2006)
35.
Caballero, J.A., Grossmann, I.E.: An algorithm for the use of surrogate models in modular flowsheet optimization. AIChE J. 54(10), 2633–2650 (2008)CrossRef
36.
Caballero, J.A., Grossmann, I.E.: Rigorous flowsheet optimization using process simulators and surrogate models. In: Computer Aided Chemical Engineering, vol. 25, pp. 551–556. Elsevier, Amsterdam (2008)
37.
Carter, R., Gablonsky, J., Patrick, A., Kelley, C.T., Eslinger, O.: Algorithms for noisy problems in gas transmission pipeline optimization. Optim. Eng. 2(2), 139–157 (2001)MathSciNetMATHCrossRef
38.
Cavazzuti, M., Corticelli, M.A.: Optimization of heat exchanger enhanced surfaces through multiobjective genetic algorithms. Numer. Heat Transf. A Appl. 54(6), 603–624 (2008)CrossRef
39.
Chambers, M., Mount-Campbell, C.: Process optimization via neural network metamodeling. Int. J. Prod. Econ. 79(2), 93–100 (2002)CrossRef
40.
Chan, F.T., Kumar, V., Mishra, N.: A CMPSO algorithm based approach to solve the multi-plant supply chain problem. Swarm Intell. Focus Ant Particle Swarm Optim. 532, 54879954 (2007)
41.
Chen, M., Wang, J., Zhao, S., Xu, C., Feng, L.: Optimization of dual-impeller configurations in a gas–liquid stirred tank based on computational fluid dynamics and multiobjective evolutionary algorithm. Ind. Eng. Chem. Res. 55(33), 9054–9063 (2016)CrossRef
42.
Chen, Q.: Ventilation performance prediction for buildings: a method overview and recent applications. Build. Environ. 44(4), 848–858 (2009)CrossRef
43.
Ciaurri, D.E., Mukerji, T., Durlofsky, L.J.: Derivative-free optimization for oil field operations. In: Computational Optimization and Applications in Engineering and Industry, pp. 19–55. Springer, Berlin (2011)
44.
Conn, A., Scheinberg, K., Toint, P.: Manual for Fortran Software Package DFO v1. 2 (2000)
45.
Conn, A.R., Gould, N.I., Toint, P.: A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds. SIAM J. Numer. Anal. 28(2), 545–572 (1991)MathSciNetMATHCrossRef
46.
Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust Region Methods, vol. 1. SIAM (2000)
47.
Conn, A.R., Le Digabel, S.: Use of quadratic models with mesh-adaptive direct search for constrained black box optimization. Optim. Methods Softw. 28(1), 139–158 (2013)MathSciNetMATHCrossRef
48.
Conn, A.R., Scheinberg, K., Vicente, L.N.: Geometry of interpolation sets in derivative free optimization. Math. Program. 111(1–2), 141–172 (2008)MathSciNetMATH
49.
Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to derivative-free optimization, vol. 8. SIAM (2009)
50.
Custódio, A.L., Vicente, L.N.: Using sampling and simplex derivatives in pattern search methods. SIAM J. Optim. 18(2), 537–555 (2007)MathSciNetMATHCrossRef
51.
Dhingra, S., Bhushan, G., Dubey, K.K.: Development of a combined approach for improvement and optimization of Karanja biodiesel using response surface methodology and genetic algorithm. Front. Energy 7(4), 495–505 (2013)CrossRef
52.
Ding, S., Jiang, H., Li, J., Tang, G.: Optimization of well placement by combination of a modified particle swarm optimization algorithm and quality map method. Comput. Geosci. 18(5), 747–762 (2014)CrossRef
53.
Dipama, J., Teyssedou, A., Sorin, M.: Synthesis of heat exchanger networks using genetic algorithms. Appl. Thermal Eng. 28(14–15), 1763–1773 (2008)CrossRef
54.
55.
Eason, J.P., Biegler, L.T.: A trust region filter method for glass box/black box optimization. AIChE J. 62(9), 3124–3136 (2016)CrossRef
56.
Eberhard, P., Sedlaczek, K.: Using augmented Lagrangian particle swarm optimization for constrained problems in engineering. In: Advanced Design of Mechanical Systems: From Analysis to Optimization, pp. 253–271. Springer, Berlin (2009)
57.
Egea, J.A., Martí, R., Banga, J.R.: An evolutionary method for complex-process optimization. Comput. Oper. Res. 37(2), 315–324 (2010)MATHCrossRef
58.
59.
Eiben, A.E., Raue, P.E., Ruttkay, Z.: Genetic algorithms with multi-parent recombination. In: International Conference on Parallel Problem Solving from Nature, pp. 78–87. Springer, Berlin (1994)
60.
Eldred, M.S., Giunta, A.A., van Bloemen Waanders, B.G., Wojtkiewicz, S.F., Hart, W.E., Alleva, M.P.: DAKOTA, a multilevel parallel object-oriented framework for design optimization, parameter estimation, uncertainty quantification, and sensitivity analysis. Tech. rep., CiteSeer (2006)
61.
Esfe, M.H., Hajmohammad, H., Moradi, R., Arani, A.A.A.: Multi-objective optimization of cost and thermal performance of double walled carbon nanotubes/water nanofluids by NSGA-II using response surface method. Appl. Thermal Eng. 112, 1648–1657 (2017)CrossRef
62.
Esmin, A.A., Coelho, R.A., Matwin, S.: A review on particle swarm optimization algorithm and its variants to clustering high-dimensional data. Artif. Intell. Rev. 44(1), 23–45 (2015)CrossRef
63.
Esmin, A.A., Lambert-Torres, G., De Souza, A.Z.: A hybrid particle swarm optimization applied to loss power minimization. IEEE Trans. Power Syst. 20(2), 859–866 (2005)CrossRef
64.
Fahmi, I., Cremaschi, S.: Process synthesis of biodiesel production plant using artificial neural networks as the surrogate models. Comput. Chem. Eng. 46, 105–123 (2012)CrossRef
65.
Farshi, M.M.: Improving genetic algorithms for optimum well placement. Ph.D. thesis, Stanford University Stanford, CA (2008)
66.
Fasano, G., Liuzzi, G., Lucidi, S., Rinaldi, F.: A linesearch-based derivative-free approach for nonsmooth constrained optimization. SIAM J. Optim. 24(3), 959–992 (2014)MathSciNetMATHCrossRef
67.
Fernandes, F.A.: Optimization of Fischer–Tropsch synthesis using neural networks. Chem. Eng. Technol. Ind. Chem. Plant Equip. Process Eng. Biotechnol. 29(4), 449–453 (2006)
68.
First, E.L., Hasan, M.F., Floudas, C.A.: Discovery of novel zeolites for natural gas purification through combined material screening and process optimization. AIChE J. 60(5), 1767–1785 (2014)CrossRef
69.
Fletcher, R., Gould, N.I., Leyffer, S., Toint, P.L., Wächter, A.: Global convergence of a trust-region SQP-filter algorithm for general nonlinear programming. SIAM J. Optim. 13(3), 635–659 (2002)MathSciNetMATHCrossRef
70.
Fletcher, R., Leyffer, S., Toint, P.L., et al.: A brief history of filter methods. Preprint ANL/MCS-P1372-0906, Argonne National Laboratory, Mathematics and Computer Science Division, vol. 36 (2006)
71.
Foli, K., Okabe, T., Olhofer, M., Jin, Y., Sendhoff, B.: Optimization of micro heat exchanger: CFD, analytical approach and multi-objective evolutionary algorithms. Int. J. Heat Mass Transf. 49(5–6), 1090–1099 (2006)MATHCrossRef
72.
Foroud, T., Baradaran, A., Seifi, A.: A comparative evaluation of global search algorithms in black box optimization of oil production: a case study on Brugge field. J. Pet. Sci. Eng. 167, 131–151 (2018)CrossRef
73.
Forrester, A.I., Bressloff, N.W., Keane, A.J.: Optimization using surrogate models and partially converged computational fluid dynamics simulations. Proc. R. Soc. A Math. Phys. Eng. Sci. 462(2071), 2177–2204 (2006)MATH
74.
Gassner, M., Maréchal, F.: Methodology for the optimal thermo-economic, multi-objective design of thermochemical fuel production from biomass. Comput. Chem. Eng. 33(3), 769–781 (2009)CrossRef
75.
Goldberg, D.E., Holland, J.H.: Genetic algorithms and machine learning (1988)
76.
Golovin, D., Solnik, B., Moitra, S., Kochanski, G., Karro, J., Sculley, D.: Google Vizier: a service for black-box optimization. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1487–1495 (2017)
77.
Gosselin, L., Tye-Gingras, M., Mathieu-Potvin, F.: Review of utilization of genetic algorithms in heat transfer problems. Int. J. Heat Mass Transf. 52(9–10), 2169–2188 (2009)MATHCrossRef
78.
Gramacy, R.B., Gray, G.A., Le Digabel, S., Lee, H.K., Ranjan, P., Wells, G., Wild, S.M.: Modeling an augmented Lagrangian for blackbox constrained optimization. Technometrics 58(1), 1–11 (2016)MathSciNetCrossRef
79.
80.
Gratton, S., Vicente, L.N.: A surrogate management framework using rigorous trust-region steps. Optim. Methods Softw. 29(1), 10–23 (2014)MathSciNetMATHCrossRef
81.
Griffin, J.D., Fowler, K.R., Gray, G.A., Hemker, T., Parno, M.D.: Derivative-free optimization via evolutionary algorithms guiding local search. Sandia National Laboratories, Albuquerque, NM, Tech. Rep. SAND2010-3023J (2010)
82.
Griffin, J.D., Kolda, T.G.: Nonlinearly-constrained optimization using asynchronous parallel generating set search. Tech. rep., Sandia National Laboratories (2007)
83.
Griffin, J.D., Kolda, T.G.: Asynchronous parallel hybrid optimization combining DIRECT and GSS. Optim. Methods Softw. 25(5), 797–817 (2010)MathSciNetMATHCrossRef
84.
Gross, B., Roosen, P.: Total process optimization in chemical engineering with evolutionary algorithms. Comput. Chem. Eng. 22, S229–S236 (1998)CrossRef
85.
Güyagüler, B., Horne, R.N., Rogers, L., Rosenzweig, J.J., et al.: Optimization of well placement in a gulf of Mexico waterflooding project. SPE Reserv. Eval. Eng. 5(03), 229–236 (2002)CrossRef
86.
Guyaguler, B., Horne, R.N., et al.: Uncertainty assessment of well placement optimization. In: SPE Annual Technical Conference and Exhibition. Society of Petroleum Engineers (2001)
87.
Häglund, S.: A surrogate-based parameter tuning heuristic for Carmen crew optimizers. Master’s thesis, Chalmers University of Technology (2010)
88.
Hansen, N.: The CMA evolution strategy: a comparing review. In: Towards a New Evolutionary Computation, pp. 75–102. Springer, Berlin (2006)
89.
Hart, W.E.: Evolutionary pattern search algorithms for unconstrained and linearly constrained optimization. IEEE Trans. Evol. Comput. 5(4), 388–397 (2001)CrossRef
90.
Hart, W.E., Hunter, K.O.: A performance analysis of evolutionary pattern search with generalized mutation steps. In: Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406), vol. 1, pp. 672–679. IEEE, Piscataway (1999)
91.
Hasan, M.F.: An edge-concave underestimator for the global optimization of twice-differentiable nonconvex problems. J. Global Optim. 71(4), 735–752 (2018)MathSciNetMATHCrossRef
92.
Hasan, M.F., Baliban, R.C., Elia, J.A., Floudas, C.A.: Modeling, simulation, and optimization of postcombustion https://static-content.springer.com/image/chp%3A10.1007%2F978-3-030-66515-9_2/478260_1_En_2_IEq8_HTML.gif capture for variable feed concentration and flow rate. 2. pressure swing adsorption and vacuum swing adsorption processes. Ind. Eng. Chem. Res. 51(48), 15665–15682 (2012)
93.
Hasan, M.F., Boukouvala, F., First, E.L., Floudas, C.A.: Nationwide, regional, and statewide https://static-content.springer.com/image/chp%3A10.1007%2F978-3-030-66515-9_2/478260_1_En_2_IEq9_HTML.gif capture, utilization, and sequestration supply chain network optimization. Ind. Eng. Chem. Res. 53(18), 7489–7506 (2014)
94.
Hasan, M.F., First, E.L., Floudas, C.A.: Cost-effective CO2 capture based on in silico screening of zeolites and process optimization. Phys. Chem. Chem. Phys. 15(40), 17601–17618 (2013)CrossRef
95.
Hemker, T., Werner, C.: Direct using local search on surrogates. Pac. J. Optim. 7(3), 443–466 (2011)MathSciNetMATH
96.
Henao, C.A., Maravelias, C.T.: Surrogate-based superstructure optimization framework. AIChE J. 57(5), 1216–1232 (2011)CrossRef
97.
Holland, J.H.: Adaptation in Natural and Artificial Systems. Society for Industrial and Applied Mathematics, Philadelphia (1976)
98.
Holmström, K.: The TOMLAB optimization environment in Matlab (1999)
99.
Holmstrom, K., Goran, A., Edvall, M.: User’s guide for Tomlab 4.0. 6. Tomlab Optimization, Sweden (2003)
100.
Humphries, T.D., Haynes, R.D.: Joint optimization of well placement and control for nonconventional well types. J. Pet. Sci. Eng. 126, 242–253 (2015)CrossRef
101.
Hutter, F., Hoos, H.H., Leyton-Brown, K.: Automated configuration of mixed integer programming solvers. In: International Conference on Integration of Artificial Intelligence (AI) and Operations Research (OR) Techniques in Constraint Programming, pp. 186–202. Springer, Berlin (2010)
102.
103.
Huyer, W., Neumaier, A.: SNOBFIT—stable noisy optimization by branch and fit. ACM Trans. Math. Softw. 35(2), 1–25 (2008)MathSciNetCrossRef
104.
Ingber, L.: Adaptive simulated annealing (ASA): lessons learned (2000). arXiv preprint cs/0001018
105.
Ingber, L., et al.: Adaptive Simulated Annealing (ASA). Global Optimization C-code. Caltech Alumni Association, Pasadena (1993)
106.
Isebor, O.J., Durlofsky, L.J., Ciaurri, D.E.: A derivative-free methodology with local and global search for the constrained joint optimization of well locations and controls. Comput. Geosci. 18(3–4), 463–482 (2014)CrossRef
107.
Isebor, O.J., Echeverría Ciaurri, D., Durlofsky, L.J., et al.: Generalized field-development optimization with derivative-free procedures. SPE J. 19(5), 891–908 (2014)CrossRef
108.
Iyer, S.S., Bajaj, I., Balasubramanian, P., Hasan, M.F.: Integrated carbon capture and conversion to produce syngas: novel process design, intensification, and optimization. Ind. Eng. Chem. Res. 56(30), 8622–8648 (2017)CrossRef
109.
Jie, J., Zeng, J., Han, C.: Self-organization particle swarm optimization based on information feedback. In: International Conference on Natural Computation, pp. 913–922. Springer, Berlin (2006)
110.
Jones, D.R.: Direct global optimization algorithm. Encyclopedia Optim. 1(1), 431–440 (2009)
111.
Jones, D.R., Perttunen, C.D., Stuckman, B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optimization Theory Appl. 79(1), 157–181 (1993)MathSciNetMATHCrossRef
112.
Jung, I., Kshetrimayum, K.S., Park, S., Na, J., Lee, Y., An, J., Park, S., Lee, C.J., Han, C.: Computational fluid dynamics based optimal design of guiding channel geometry in u-type coolant layer manifold of large-scale microchannel Fischer–Tropsch reactor. Ind. Eng. Chem. Res. 55(2), 505–515 (2016)CrossRef
113.
Kandasamy, K., Dasarathy, G., Oliva, J., Schneider, J., Poczos, B.: Multi-fidelity Gaussian process bandit optimisation. J. Artif. Intell. Res. 66, 151–196 (2019)MathSciNetMATHCrossRef
114.
Kandasamy, K., Dasarathy, G., Oliva, J.B., Schneider, J., Póczos, B.: Gaussian process bandit optimisation with multi-fidelity evaluations. In: Advances in Neural Information Processing Systems, pp. 992–1000 (2016)
115.
Kandasamy, K., Dasarathy, G., Schneider, J., Póczos, B.: Multi-fidelity Bayesian optimisation with continuous approximations. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 1799–1808 (2017). JMLR.​org
116.
Kato, S., Lee, J.H.: Optimization of hybrid air-conditioning system with natural ventilation by GA and CFD. In: 25th AIVC Conference, Ventilation and Retrofitting (2004)
117.
Kaufman, D.E., Smith, R.L.: Optimal direction choice for hit-and-run sampling. Tech. rep. (1991)
118.
119.
Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)
120.
121.
Kolda, T.G., Lewis, R.M., Torczon, V., et al.: A generating set direct search augmented Lagrangian algorithm for optimization with a combination of general and linear constraints, vol. 6. Sandia National Laboratories (2006)
122.
Kramer, O.: A review of constraint-handling techniques for evolution strategies. Appl. Comput. Intell. Soft Comput. 2010, 185063 (2010)
123.
Krink, T., VesterstrOm, J.S., Riget, J.: Particle swarm optimisation with spatial particle extension. In: Proceedings of the 2002 Congress on Evolutionary Computation. CEC’02 (Cat. No. 02TH8600), vol. 2, pp. 1474–1479. IEEE, Piscataway (2002)
124.
Lagarias, J.C., Reeds, J.A., Wright, M.H., Wright, P.E.: Convergence properties of the Nelder–Mead simplex method in low dimensions. SIAM J. Optim. 9(1), 112–147 (1998)MathSciNetMATHCrossRef
125.
126.
127.
Leardi, R.: Genetic algorithms in chemometrics and chemistry: a review. J. Chemometrics 15(7), 559–569 (2001)CrossRef
128.
Lehnhäuser, T., Schäfer, M.: A numerical approach for shape optimization of fluid flow domains. Comput. Methods Appl. Mech. Eng. 194(50–52), 5221–5241 (2005)MATHCrossRef
129.
Lewin, D.R., Wang, H., Shalev, O.: A generalized method for HEN synthesis using stochastic optimization—I. General framework and MER optimal synthesis. Comput. Chem. Eng. 22(10), 1503–1513 (1998)
130.
Lewis, R.M., Torczon, V.: A globally convergent augmented Lagrangian pattern search algorithm for optimization with general constraints and simple bounds. SIAM J. Optim. 12(4), 1075–1089 (2002)MathSciNetMATHCrossRef
131.
Lewis, R.M., Torczon, V.: A direct search approach to nonlinear programming problems using an augmented Lagrangian method with explicit treatment of linear constraints. Technical Report of the College of William and Mary pp. 1–25 (2010)
132.
Li, H.Q., Li, L.: A novel hybrid particle swarm optimization algorithm combined with harmony search for high dimensional optimization problems. In: The 2007 International Conference on Intelligent Pervasive Computing (IPC 2007), pp. 94–97. IEEE, Piscataway (2007)
133.
Litvak, M.L., Gane, B.R., Williams, G., Mansfield, M., Angert, P.F., Macdonald, C.J., McMurray, L.S., Skinner, R.C., Walker, G.J., et al.: Field development optimization technology. In: SPE Reservoir Simulation Symposium. Society of Petroleum Engineers, Richardson (2007)
134.
Liu, J., Ploskas, N., Sahinidis, N.V.: Tuning BARON using derivative-free optimization algorithms. J. Global Optim. 74(4), 611–637 (2019)MathSciNetMATHCrossRef
135.
Liu, T., First, E.L., Hasan, M.F., Floudas, C.A.: A multi-scale approach for the discovery of zeolites for hydrogen sulfide removal. Comput. Chem. Eng. 91, 206–218 (2016)CrossRef
136.
Liuzzi, G., Lucidi, S., Piccialli, V.: Exploiting derivative-free local searches in direct-type algorithms for global optimization. Comput. Optim. Appl. 65(2), 449–475 (2016)MathSciNetMATHCrossRef
137.
Liuzzi, G., Lucidi, S., Sciandrone, M.: Sequential penalty derivative-free methods for nonlinear constrained optimization. SIAM J. Optim. 20(5), 2614–2635 (2010)MathSciNetMATHCrossRef
138.
Macek, K., Rojicek, J., Kontes, G., Rovas, D.: Black-box optimisation for buildings and its enhancement by advanced communication infrastructure. Adv. Distrib. Comput. Artif. Intell. J. 2013, 53–64 (2013)
139.
Malkawi, A.M., Srinivasan, R.S., Yi, Y.K., Choudhary, R.: Performance-based design evolution: the use of genetic algorithms and CFD. In: Eighth International IBPSA. Eindhoven, Netherlands pp. 793–798 (2003)
140.
Malkawi, A.M., Srinivasan, R.S., Yun, K.Y., Choudhary, R.: Decision support and design evolution: integrating genetic algorithms, CFD and visualization. Autom. Constr. 14(1), 33–44 (2005)CrossRef
141.
Marinakis, Y., Marinaki, M., Matsatsinis, N.: A hybrid bumble bees mating optimization-grasp algorithm for clustering. In: International Conference on Hybrid Artificial Intelligence Systems, pp. 549–556. Springer, Berlin (2009)
142.
Marsden, A.L., Feinstein, J.A., Taylor, C.A.: A computational framework for derivative-free optimization of cardiovascular geometries. Comput. Methods Appl. Mech. Eng. 197(21–24), 1890–1905 (2008)MathSciNetMATHCrossRef
143.
Marsden, A.L., Wang, M., Dennis, J., Moin, P.: Trailing-edge noise reduction using derivative-free optimization and large-eddy simulation. J. Fluid Mech. 572, 13–36 (2007)MathSciNetMATHCrossRef
144.
Marsden, A.L., Wang, M., Dennis, J.E., Moin, P.: Optimal aeroacoustic shape design using the surrogate management framework. Optim. Eng. 5(2), 235–262 (2004)MathSciNetMATHCrossRef
145.
Marsden, A.L., Wang, M., Dennis Jr, J.E., Moin, P.: Suppression of vortex-shedding noise via derivative-free shape optimization. Phys. fluids 16(10), L83–L86 (2004)MATHCrossRef
146.
Martelli, E., Amaldi, E.: PGS-COM: a hybrid method for constrained non-smooth black-box optimization problems: brief review, novel algorithm and comparative evaluation. Comput. Chem. Eng. 63, 108–139 (2014)CrossRef
147.
Martelli, E., Amaldi, E., Consonni, S.: Numerical optimization of heat recovery steam cycles: mathematical model, two-stage algorithm and applications. Comput. Chem. Eng. 35(12), 2799–2823 (2011)CrossRef
148.
149.
Meissner, M., Schmuker, M., Schneider, G.: Optimized particle swarm optimization (OPSO) and its application to artificial neural network training. BMC Bioinform. 7(1), 125 (2006)CrossRef
150.
Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953)MATHCrossRef
151.
Mezura-Montes, E., Coello, C.A.C.: Constraint-handling in nature-inspired numerical optimization: past, present and future. Swarm Evol. Comput. 1(4), 173–194 (2011)CrossRef
152.
Mezura-Montes, E., Flores-Mendoza, J.I.: Improved particle swarm optimization in constrained numerical search spaces. In: Nature-Inspired Algorithms for Optimisation, pp. 299–332. Springer, Berlin (2009)
153.
Michalewicz, Z., Schoenauer, M.: Evolutionary algorithms for constrained parameter optimization problems. Evol. Comput. 4(1), 1–32 (1996)CrossRef
154.
Moles, C.G., Mendes, P., Banga, J.R.: Parameter estimation in biochemical pathways: a comparison of global optimization methods. Genome Res. 13(11), 2467–2474 (2003)CrossRef
155.
Morin, A., Wahl, P.E., Mølnvik, M.: Using evolutionary search to optimise the energy consumption for natural gas liquefaction. Chem. Eng. Res. Design 89(11), 2428–2441 (2011)CrossRef
156.
Na, J., Kshetrimayum, K.S., Lee, U., Han, C.: Multi-objective optimization of microchannel reactor for Fischer–Tropsch synthesis using computational fluid dynamics and genetic algorithm. Chem. Eng. J. 313, 1521–1534 (2017)CrossRef
157.
Nascimento, C.A.O., Giudici, R., Guardani, R.: Neural network based approach for optimization of industrial chemical processes. Comput. Chem. Eng. 24(9–10), 2303–2314 (2000)CrossRef
158.
159.
Onwunalu, J.E., Durlofsky, L.J.: Application of a particle swarm optimization algorithm for determining optimum well location and type. Comput. Geosci. 14(1), 183–198 (2010)MATHCrossRef
160.
Pariyani, A., Gupta, A., Ghosh, P.: Design of heat exchanger networks using randomized algorithm. Comput. Chem. Eng. 30(6–7), 1046–1053 (2006)CrossRef
161.
Park, S., Na, J., Kim, M., Lee, J.M.: Multi-objective Bayesian optimization of chemical reactor design using computational fluid dynamics. Comput. Chem. Eng. 119, 25–37 (2018)CrossRef
162.
Payne, J.L., Eppstein, M.J.: A hybrid genetic algorithm with pattern search for finding heavy atoms in protein crystals. In: Proceedings of the 7th Annual Conference on Genetic and Evolutionary Computation, pp. 377–384 (2005)
163.
Plantenga, T.D.: Hopspack 2.0 user manual. Sandia National Laboratories Technical Report Sandia National Laboratories Technical Report SAND2009-6265 (2009)
164.
Ploskas, N., Laughman, C., Raghunathan, A.U., Sahinidis, N.V.: Optimization of circuitry arrangements for heat exchangers using derivative-free optimization. Chem. Eng. Res. Design 131, 16–28 (2018)CrossRef
165.
Pourfattah, F., Sabzpooshani, M., Bayer, Ö., Toghraie, D., Asadi, A.: On the optimization of a vertical twisted tape arrangement in a channel subjected to MWCNT–water nanofluid by coupling numerical simulation and genetic algorithm. J. Thermal Anal. Calorim. (2020). https://​doi.​org/​10.​1007/​s10973-020-09490-5
166.
Powell, M.: On the Lagrange functions of quadratic models that are defined by interpolation. Optim. Methods Softw. 16(1–4), 289–309 (2001)MathSciNetMATHCrossRef
167.
Powell, M.J.: A direct search optimization method that models the objective and constraint functions by linear interpolation. In: Advances in Optimization and Numerical Analysis, pp. 51–67. Springer, Berlin (1994)
168.
169.
Powell, M.J.: The NEWUOA software for unconstrained optimization without derivatives. In: Large-Scale Nonlinear Optimization, pp. 255–297. Springer, Berlin (2006)
170.
171.
Powell, M.J.: The BOBYQA algorithm for bound constrained optimization without derivatives. Cambridge NA Report NA2009/06, University of Cambridge, Cambridge pp. 26–46 (2009)
172.
Powell, M.J.: On fast trust region methods for quadratic models with linear constraints. Math. Program. Comput. 7(3), 237–267 (2015)MathSciNetMATHCrossRef
173.
174.
Raidl, G.R.: A unified view on hybrid metaheuristics. In: International Workshop on Hybrid Metaheuristics, pp. 1–12. Springer, Berlin (2006)
175.
Ravagnani, M., Silva, A., Arroyo, P., Constantino, A.: Heat exchanger network synthesis and optimisation using genetic algorithm. Appl. Thermal Eng. 25(7), 1003–1017 (2005)CrossRef
176.
Regis, R.G.: Stochastic radial basis function algorithms for large-scale optimization involving expensive black-box objective and constraint functions. Comput. Oper. Res. 38(5), 837–853 (2011)MathSciNetMATHCrossRef
177.
Regis, R.G.: Evolutionary programming for high-dimensional constrained expensive black-box optimization using radial basis functions. IEEE Trans. Evol. Comput. 18(3), 326–347 (2013)CrossRef
178.
Regis, R.G.: Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points. Eng. Optim. 46(2), 218–243 (2014)MathSciNetCrossRef
179.
Rodriguez-Fernandez, M., Egea, J.A., Banga, J.R.: Novel metaheuristic for parameter estimation in nonlinear dynamic biological systems. BMC Bioinform. 7(1), 483 (2006)CrossRef
180.
Rößger, P., Richter, A.: Performance of different optimization concepts for reactive flow systems based on combined CFD and response surface methods. Comput. Chem. Eng. 108, 232–239 (2018)CrossRef
181.
Safikhani, H., Abbassi, A., Khalkhali, A., Kalteh, M.: Multi-objective optimization of nanofluid flow in flat tubes using CFD, artificial neural networks and genetic algorithms. Adv. Powder Technol. 25(5), 1608–1617 (2014)CrossRef
182.
Sen, R., Kandasamy, K., Shakkottai, S.: Multi-fidelity black-box optimization with hierarchical partitions. In: International Conference on Machine Learning, pp. 4538–4547 (2018)
183.
Siavashi, M., Doranehgard, M.H.: Particle swarm optimization of thermal enhanced oil recovery from oilfields with temperature control. Appl. Thermal Eng. 123, 658–669 (2017)CrossRef
184.
Silva, A., Neves, A., Costa, E.: Chasing the swarm: a predator prey approach to function optimisation. InL Proceedings of Mendal, pp. 5–7 (2002)
185.
Sivanandam, S., Deepa, S.: Genetic algorithms. In: Introduction to Genetic Algorithms, pp. 15–37. Springer, Berlin (2008)
186.
Smith, R.L.: Monte Carlo procedures for generating random feasible solutions to mathematical programs. In: A Bulletin of the ORSA/TIMS Joint National Meeting, Washington, DC, vol. 101 (1980)
187.
Smith, R.L.: Efficient Monte Carlo procedures for generating points uniformly distributed over bounded regions. Oper. Res. 32(6), 1296–1308 (1984)MathSciNetMATHCrossRef
188.
Srivastava, R., Rawlings, J.B.: Parameter estimation in stochastic chemical kinetic models using derivative free optimization and bootstrapping. Comput. Chem. Eng. 63, 152–158 (2014)CrossRef
189.
Talbi, E.G.: A taxonomy of hybrid metaheuristics. J. Heuristics 8(5), 541–564 (2002)CrossRef
190.
191.
Tran, A., Sun, J., Furlan, J.M., Pagalthivarthi, K.V., Visintainer, R.J., Wang, Y.: pBO-2GP-3B: a batch parallel known/unknown constrained Bayesian optimization with feasibility classification and its applications in computational fluid dynamics. Comput. Methods Appl. Mech. Eng. 347, 827–852 (2019)MathSciNetMATHCrossRef
192.
193.
Uebel, K., Rößger, P., Prüfert, U., Richter, A., Meyer, B.: CFD-based multi-objective optimization of a quench reactor design. Fuel Proces. Technol. 149, 290–304 (2016)CrossRef
194.
Vaz, A.I.F., Vicente, L.N.: A particle swarm pattern search method for bound constrained global optimization. J. Global Optim. 39(2), 197–219 (2007)MathSciNetMATHCrossRef
195.
Vaz, A.I.F., Vicente, L.N.: PSwarm: a hybrid solver for linearly constrained global derivative-free optimization. Optim. Methods Softw. 24(4–5), 669–685 (2009)MathSciNetMATHCrossRef
196.
197.
Wachowiak, M.P., Peters, T.M.: Parallel optimization approaches for medical image registration. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 781–788. Springer, Berlin (2004)
198.
Wachowiak, M.P., Peters, T.M.: Combining global and local parallel optimization for medical image registration. In: Medical Imaging 2005: Image Processing, vol. 5747, pp. 1189–1200. International Society for Optics and Photonics (2005)
199.
Wan, X., Pekny, J.F., Reklaitis, G.V.: Simulation-based optimization with surrogate models—application to supply chain management. Comput. Chem. Eng. 29(6), 1317–1328 (2005)CrossRef
200.
Weile, D.S., Michielssen, E.: Genetic algorithm optimization applied to electromagnetics: a review. IEEE Trans. Antennas Propag. 45(3), 343–353 (1997)CrossRef
201.
Woods, D.J.: An interactive approach for solving multi-objective optimization problems. Tech. rep. (1985)
202.
Wright, J., Zhang, Y., Angelov, P., Hanby, V., Buswell, R.: Evolutionary synthesis of HVAC system configurations: algorithm development (RP-1049). HVAC&R Res. 14(1), 33–55 (2008)CrossRef
203.
Wright, J.A., Loosemore, H.A., Farmani, R.: Optimization of building thermal design and control by multi-criterion genetic algorithm. Energy Build. 34(9), 959–972 (2002)CrossRef
204.
Yehui, P., Zhenhai, L.: A derivative-free algorithm for unconstrained optimization. Appl. Math. A J. Chin. Univ. 20(4), 491–498 (2005)MathSciNetMATHCrossRef
205.
Yeten, B., Durlofsky, L., Aziz, K.: Optimization of nonconventional well type. Location Trajectory, SPE 77565, 14 (2002)
206.
Zhang, Y., Wright, J.A., Hanby, V.I.: Energy aspects of HVAC system configurations—problem definition and test cases. HVAC&R Res. 12(S3), 871–888 (2006)CrossRef
207.
Zhao, B., Guo, C., Cao, Y.: A multiagent-based particle swarm optimization approach for optimal reactive power dispatch. IEEE Trans. Power Syst. 20(2), 1070–1078 (2005)CrossRef
208.
Zhou, Z., Ong, Y.S., Nair, P.B., Keane, A.J., Lum, K.Y.: Combining global and local surrogate models to accelerate evolutionary optimization. IEEE Trans. Syst. Man Cybern. C (Appl. Rev.) 37(1), 66–76 (2006)
Metadata
Title
Black-Box Optimization: Methods and Applications
Authors
Ishan Bajaj
Akhil Arora
M. M. Faruque Hasan
Copyright Year
2021
Book
Black Box Optimization, Machine Learning, and No-Free Lunch Theorems

Print ISBN: 978-3-030-66514-2

Electronic ISBN: 978-3-030-66515-9

Copyright Year: 2021

DOI
https://doi.org/10.1007/978-3-030-66515-9_2

Premium Partner

    Image Credits