Skip to main content
Erschienen in: Engineering with Computers 4/2011

01.10.2011 | Original Article

Validation of metamodels in simulation: a new metric

verfasst von: Husam Hamad

Erschienen in: Engineering with Computers | Ausgabe 4/2011

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Metamodels are used to provide more efficient predictions than the underlying simulation models do, but at the price of reduced prediction accuracy. Statistics used to quantify this prediction accuracy include the root-mean square error (RMSE), the coefficient of determination R-square, and the average absolute error (AAE). Such statistics depend on the average prediction accuracy over the validation sample; i.e., these metrics are sensitive to the size of the validation sample. This article, therefore, introduces a new metric, called the Model acceptability score (MAS). Preliminary results indicate that MAS is less sensitive to the validation sample size. The article focuses on deterministic simulation, which is used in various engineering disciplines, e.g., electronic engineering.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Wang G, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. ASME J Mech Des 129:370–380CrossRef Wang G, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. ASME J Mech Des 129:370–380CrossRef
2.
Zurück zum Zitat Tang B (1993) Orthogonal array-based Latin hypercubes. J Am Stat Assoc 88:1392–1397MATHCrossRef Tang B (1993) Orthogonal array-based Latin hypercubes. J Am Stat Assoc 88:1392–1397MATHCrossRef
3.
Zurück zum Zitat Park J (1994) Optimal Latin-hypercube designs for computer experiments. J Stat Plan Inference 39:95–111MATHCrossRef Park J (1994) Optimal Latin-hypercube designs for computer experiments. J Stat Plan Inference 39:95–111MATHCrossRef
4.
Zurück zum Zitat Ye K, Li W, Sudjianto A (2000) Algorithmic construction of optimal symmetric Latin hypercube designs. J Stat Plan Inferences 90:145–159MathSciNetMATHCrossRef Ye K, Li W, Sudjianto A (2000) Algorithmic construction of optimal symmetric Latin hypercube designs. J Stat Plan Inferences 90:145–159MathSciNetMATHCrossRef
5.
6.
Zurück zum Zitat Angün E, Kleijnin J, Hertog D, Gürkan G (2002) Response surface methodology revisited. Proceedings IEEE 2002 Winter Simulation Conference, pp 377–383 Angün E, Kleijnin J, Hertog D, Gürkan G (2002) Response surface methodology revisited. Proceedings IEEE 2002 Winter Simulation Conference, pp 377–383
7.
Zurück zum Zitat Martin JD, Simpson TW (2005) Use of kriging models to approximate deterministic computer models. AIAA J 43:853–863CrossRef Martin JD, Simpson TW (2005) Use of kriging models to approximate deterministic computer models. AIAA J 43:853–863CrossRef
8.
9.
Zurück zum Zitat Shin M, Sargent R, Goel A (2002) Gaussian radial basis functions for simulation metamodeling. Proceedings IEEE 2002 Winter Simulation Conference, pp 483–488 Shin M, Sargent R, Goel A (2002) Gaussian radial basis functions for simulation metamodeling. Proceedings IEEE 2002 Winter Simulation Conference, pp 483–488
10.
Zurück zum Zitat Clarke S, Griebsch J, Simpson T (2005) Analysis of support vector regression for approximation of complex engineering analyses. J Mech Des 127:1077–1087CrossRef Clarke S, Griebsch J, Simpson T (2005) Analysis of support vector regression for approximation of complex engineering analyses. J Mech Des 127:1077–1087CrossRef
11.
Zurück zum Zitat Simpson T, Peplinski J, Koch P, Allen J (2001) Metamodels for computer-based engineering design: survey and recommendations. Eng Comput 17:129–150MATHCrossRef Simpson T, Peplinski J, Koch P, Allen J (2001) Metamodels for computer-based engineering design: survey and recommendations. Eng Comput 17:129–150MATHCrossRef
13.
Zurück zum Zitat Barton (1999) Graphical Methods for the Design of Experiments, Springer, New York Barton (1999) Graphical Methods for the Design of Experiments, Springer, New York
14.
Zurück zum Zitat Sargent R (2004) Validation and verification of simulation models. Proceedings 2004 Winter Simulation Conference, pp 13–24 Sargent R (2004) Validation and verification of simulation models. Proceedings 2004 Winter Simulation Conference, pp 13–24
15.
Zurück zum Zitat Kleijnen J, Deflandre D (2006) Validation of regression metamodels in simulation: bootstrap approach. Eur J Oper Res 170:120–131MathSciNetMATHCrossRef Kleijnen J, Deflandre D (2006) Validation of regression metamodels in simulation: bootstrap approach. Eur J Oper Res 170:120–131MathSciNetMATHCrossRef
16.
Zurück zum Zitat Hamad H, Al-Hamdan S (2007) Discovering metamodels’ quality-of-fit via graphical techniques. Eur J Oper Res 178:543–559MATHCrossRef Hamad H, Al-Hamdan S (2007) Discovering metamodels’ quality-of-fit via graphical techniques. Eur J Oper Res 178:543–559MATHCrossRef
17.
Zurück zum Zitat Hamad H, Al-Smadi A (2007) Space partitioning in engineering design via metamodel acceptance score distribution. Eng Comput 23:175–185CrossRef Hamad H, Al-Smadi A (2007) Space partitioning in engineering design via metamodel acceptance score distribution. Eng Comput 23:175–185CrossRef
18.
Zurück zum Zitat Hamad H (2006) A new metric for measuring metamodels quality-of-fit for deterministic simulations. Proceedings 2006 Winter Simulation Conference, pp 882–888 Hamad H (2006) A new metric for measuring metamodels quality-of-fit for deterministic simulations. Proceedings 2006 Winter Simulation Conference, pp 882–888
19.
Zurück zum Zitat Guinta A, Wojtkiewicz S, Eldred M (2003) Overview of modern design of experiment methods for computational sciences Proceedings. 41st Aerospace Sciences Meeting and Exhibit, AIAA, Washington, pp 649 Guinta A, Wojtkiewicz S, Eldred M (2003) Overview of modern design of experiment methods for computational sciences Proceedings. 41st Aerospace Sciences Meeting and Exhibit, AIAA, Washington, pp 649
20.
Zurück zum Zitat Palmer K, Tsui K-L (2001) A minimum bias Latin hypercube design. IIE Trans 33:793–808 Palmer K, Tsui K-L (2001) A minimum bias Latin hypercube design. IIE Trans 33:793–808
Metadaten
Titel
Validation of metamodels in simulation: a new metric
verfasst von
Husam Hamad
Publikationsdatum
01.10.2011
Verlag
Springer-Verlag
Erschienen in
Engineering with Computers / Ausgabe 4/2011
Print ISSN: 0177-0667
Elektronische ISSN: 1435-5663
DOI
https://doi.org/10.1007/s00366-010-0200-z

Weitere Artikel der Ausgabe 4/2011

Engineering with Computers 4/2011 Zur Ausgabe

Neuer Inhalt