Top

Structural and Multidisciplinary Optimization

Published in:

01-02-2011 | Research Paper

Surrogate modeling approximation using a mixture of experts based on EM joint estimation

Authors: Dimitri Bettebghor, Nathalie Bartoli, Stéphane Grihon, Joseph Morlier, Manuel Samuelides

Published in: Structural and Multidisciplinary Optimization | Issue 2/2011

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

An automatic method to combine several local surrogate models is presented. This method is intended to build accurate and smooth approximation of discontinuous functions that are to be used in structural optimization problems. It strongly relies on the Expectation−Maximization (EM) algorithm for Gaussian mixture models (GMM). To the end of regression, the inputs are clustered together with their output values by means of parameter estimation of the joint distribution. A local expert is then built (linear, quadratic, artificial neural network, moving least squares) on each cluster. Lastly, the local experts are combined using the Gaussian mixture model parameters found by the EM algorithm to obtain a global model. This method is tested over both mathematical test cases and an engineering optimization problem from aeronautics and is found to improve the accuracy of the approximation.

previous article Geometrical aspects of optimum truss like structures

next article Structural sensitivity analysis in nonlinear and transient problems using the local response function technique

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

The Mahalanobis distance of a random variable X ∈ \(\mathbb{R}^{d}\) is the distance defined by the inverse the variance-covariance matrix \(\Gamma = {\mbox{Var}(X)}\): for ω ₁, \(\omega_2 \in {\mathbb{R}^{d}}\) the Mahalanobis distance is \(D_{M}(\omega_1,\omega_2) = ||\omega_1-\omega_2||_{\Gamma^{-1}} = \sqrt{(\omega_1-\omega_2)^T\Gamma^{-1}(\omega_1-\omega_2)}\). It does define a distance since the inverse of Γ (sometimes called the precision matrix) is symmetric positive definite.

All these matrices are symmetric positive definite but they can become nearly-singular especially in case of redundant data (linearity), QR factorization performs better than Gaussian reduction and even Choleski factorization.

This boundary is often known in Probability as the Bayes classifier.

Such regressions are mainly used to speed up pre-sizing of the aircraft and are known as design curves.

In this article we focused on Gaussian mixture models that were fully free, i.e. all the parameters of the Gaussian mixture models are not constrained and EM algorithm estimates all the parameters. There are more simple Gaussian mixture models that assume that all the means are the same or all of the variance-covariance matrices are of the form \(\sigma^2 I_n\) (this hypothesis is known in statistics as homoscedascity).

Acar E, Rais-Rohani M (2009) Ensemble of metamodels with optimized weight factors. Struct Multidisc Optim 37(3):279–294CrossRef

Berkhin P (2002) Survey of clustering data mining techniques

Bradley PS, Fayyad U, Reina C (1998) Scaling EM (expectation-maximization) clustering to large databases. Microsoft Research Report, MSR-TR-98-35

Buhmann MD (2001) Radial basis functions. Acta Numer 9:1–38MathSciNet

Burnham KP, Anderson DR (2004) Multimodel inference: understanding AIC and BIC in model selection. Sociol Methods Res 33(2):261CrossRefMathSciNet

Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc B (Methodological) 39:1–38MATHMathSciNet

Dreyfus G (2005) Neural networks: methodology and applications. Springer, BerlinMATH

Fasshauer GE (2005) Dual bases and discrete reproducing kernels: a unified framework for RBF and MLS approximation. Eng Anal Bound Elem 29(4):313–325MATHCrossRef

Forrester AIJ, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1–3):50–79CrossRef

Friedman J, Hastie T, Tibshirani R (2001) The elements of statistical learning

Haftka RT, Gurdal Z (1992) Elements of structural optimization. Kluwer

Haykin S (2008) Neural networks: a comprehensive foundation. Prentice Hall

Jordan MI, Jacobs RA (1994) Hierarchical mixtures of experts and the EM algorithm. Neural Comput 6(2):181–214CrossRef

Kleijnen JPC, Sanchez SM, Lucas TW, Cioppa TM (2005) A user’s guide to the brave new world of designing simulation experiments. INFORMS J Comput 17(3):263–289CrossRef

Kohavi R (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: International joint conference on artificial intelligence, vol 14. Citeseer, pp 1137–1145

Levin D (1998) The approximation power of moving least-squares. Math Comput 67(224):1517–1532MATHCrossRef

Liu B, Haftka RT, Watson LT (2004) Global-local structural optimization using response surfaces of local optimization margins. Struct Multidisc Optim 27(5):352–359CrossRef

Meir R, Ratsch G (2003) An introduction to boosting and leveraging. Lect Notes Comput Sci 2600:118–183CrossRef

Merval A (2008) Application des modèles réduits à l’optimisation multi-niveau de structures aéronautiques. Thèse SupAéro

Merval A, Samuelides M, Grihon S (2006) Application of response surface methodology to stiffened panel optimization. In: 47th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics, and materials conference, 1–4 May 2006, Newport, RI

Myers RH, Montgomery DC, Anderson-Cook CM (2009) Response surface methodology, process and product optimization using designed experiments. Wiley

Nealen A, Darmstadt TU (2004) An as-short-as-possible introduction to the least squares, weighted least squares and moving least squares methods for scattered data approximation and interpolation. URL: http://www.nealen.com/projects

Picard RR, Cook RD (1984) Cross-validation of regression models. J Am Stat Assoc 79(387):575–583MATHCrossRefMathSciNet

Sanchez E, Pintos S, Queipo NV (2008) Toward an optimal ensemble of kernel-based approximations with engineering applications. Struct Multidisc Optim 36(3):247–261CrossRef

Simpson TW, Toropov V, Balabanov V, Viana FAC (2008) Design and analysis of computer experiments in multidisciplinary design optimization: a review of how far we have come or not. In: Proceedings of the 12th AIAA/ISSMO multidisciplinary analysis and optimization conference, 2008 MAO. Victoria, Canada

Smola AJ, Schölkopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199–222CrossRefMathSciNet

van Beers WCM, Kleijnen JPC (2004) Kriging interpolation in simulation: a survey. In: Proceedings of the 36th conference on winter simulation. Winter simulation conference, pp 113–121

Viana FAC, Haftka RT, Steffen V (2009) Multiple surrogates: how cross-validation errors can help us to obtain the best predictor. Struct Multidisc Optim 39(4):439–457CrossRef

Wang GG, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. J Mech Des 129:370CrossRef

Wu CFJ (1983) On the convergence properties of the EM algorithm. Ann Stat 11(1):95–103MATHCrossRef

Yang Y (2003) Regression with multiple candidate models: selecting or mixing? Stat Sin 13(3):783–810MATH

Zerpa LE, Queipo NV, Pintos S, Salager JL (2005) An optimization methodology of alkaline—surfactant—polymer flooding processes using field scale numerical simulation and multiple surrogates. J Pet Sci Eng 47(3–4):197–208CrossRef

Title: Surrogate modeling approximation using a mixture of experts based on EM joint estimation
Authors: Dimitri Bettebghor
Nathalie Bartoli
Stéphane Grihon
Joseph Morlier
Manuel Samuelides
Publication date: 01-02-2011
Publisher: Springer-Verlag
Published in: Structural and Multidisciplinary Optimization / Issue 2/2011
Print ISSN: 1615-147X
Electronic ISSN: 1615-1488
DOI: https://doi.org/10.1007/s00158-010-0554-2

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Other articles of this Issue 2/2011

Reliability-based topology optimization using a standard response surface method for three-dimensional structures

Application of layout and topology optimization using pattern gradation for the conceptual design of buildings

Multi-objective topology and sizing optimization of truss structures based on adaptive multi-island search strategy

A design optimization strategy for creating devices that traverse the Pareto frontier over time

Structural sensitivity analysis in nonlinear and transient problems using the local response function technique

Geometry and topology optimization of geodesic domes using charged system search

Premium Partners