scroll identifier for mobile
main-content

This book describes methods for designing and analyzing experiments that are conducted using a computer code, a computer experiment, and, when possible, a physical experiment. Computer experiments continue to increase in popularity as surrogates for and adjuncts to physical experiments. Since the publication of the first edition, there have been many methodological advances and software developments to implement these new methodologies. The computer experiments literature has emphasized the construction of algorithms for various data analysis tasks (design construction, prediction, sensitivity analysis, calibration among others), and the development of web-based repositories of designs for immediate application. While it is written at a level that is accessible to readers with Masters-level training in Statistics, the book is written in sufficient detail to be useful for practitioners and researchers.

New to this revised and expanded edition:

• An expanded presentation of basic material on computer experiments and Gaussian processes with additional simulations and examples

• A new comparison of plug-in prediction methodologies for real-valued simulator output

• An enlarged discussion of space-filling designs including Latin Hypercube designs (LHDs), near-orthogonal designs, and nonrectangular regions

• A chapter length description of process-based designs for optimization, to improve good overall fit, quantile estimation, and Pareto optimization

• A new chapter describing graphical and numerical sensitivity analysis tools

• Substantial new material on calibration-based prediction and inference for calibration parameters

• Lists of software that can be used to fit models discussed in the book to aid practitioners

### Chapter 1. Physical Experiments and Computer Experiments

Experiments have long been used to study the relationship between a set of inputs to a physical system and the resulting output. Termed physical experiments in this text, there is a growing trend to replace or supplement the physical system used in such an experiment with a deterministic simulator.
Thomas J. Santner, Brian J. Williams, William I. Notz

### Chapter 2. Stochastic Process Models for Describing Computer Simulator Output

Recall from Chap. 1 that $$\boldsymbol{x}$$ denotes a generic input to our computer simulator and $$y(\boldsymbol{x})$$ denotes the associated output. This chapter will introduce several classes of random function models for $$y(\boldsymbol{x})$$ that will serve as the core building blocks for the interpolators, experimental designs, calibration, and tuning methodologies that will be introduced in later chapters. The reason that the random function approach is so useful is that accurate prediction based on black box computer simulator output requires a rich class of $$y(\boldsymbol{x})$$ options when only a minimal amount might be known about the output function. Indeed, regression mean modeling of simulator output is usually based on a rather arbitrarily selected parametric form.
Thomas J. Santner, Brian J. Williams, William I. Notz

### Chapter 3. Empirical Best Linear Unbiased Prediction of Computer Simulator Output

This chapter and Chap. 4 discuss techniques for predicting output for a computer simulator based on “training” runs from the model. Knowing how to predict computer output is a prerequisite for answering most practical research questions that involve computer simulators including those listed in Sect. 1.​3. As an example where the prediction methods described below will be central, Chap. 6 will present a sequential design for a computer experiment to find input conditions $$\boldsymbol{x}$$ that maximize a computer output which requires prediction of $$y(\boldsymbol{x})$$ at all untried sites.
Thomas J. Santner, Brian J. Williams, William I. Notz

### Chapter 4. Bayesian Inference for Simulator Output

In Chap. 3 the correlation and precision parameters are completely unknown for the process model assumed to generate simulator output. In contrast this chapter assumes that the researcher has prior knowledge about the unknown parameters that is quantifiable in the form of a prior distribution.
Thomas J. Santner, Brian J. Williams, William I. Notz

### Chapter 5. Space-Filling Designs for Computer Experiments

This chapter and the next discuss how to select inputs at which to compute the output of a computer experiment to achieve specific goals. The inputs one selects constitute the “experimental design.” As in previous chapters, the inputs are referred to as “runs.” The region corresponding to the values of the inputs that is to be studied is called the experimental region. A point in this region corresponds to a specific set of values of the inputs. Thus, an experimental design is a specification of points (runs) in the experimental region at which the response is to be computed.
Thomas J. Santner, Brian J. Williams, William I. Notz

### Chapter 6. Some Criterion-Based Experimental Designs

Chapter 5 considered designs that attempt to spread observations “evenly” throughout the experimental region. Such designs were called space-filling designs. Recall that one rationale for using a space-filling design is the following. If it is believed that interesting features of the true model are just as likely to be in one part of the input region as another, observations should be taken in all portions of the input region. There are many heuristic criteria for producing designs that might be considered space-filling; several of these were discussed in Chap. 5. However none of the methods was tied to a statistical justification, and no single criterion was singled out as best.
Thomas J. Santner, Brian J. Williams, William I. Notz

### Chapter 7. Sensitivity Analysis and Variable Screening

This chapter discusses sensitivity analysis and the related topic of variable screening. The setup is as follows. A vector of inputs $$\boldsymbol{x} = (x_{1},\ldots,x_{d})$$ is given which potentially affects a “response” function $$y(\boldsymbol{x}) = y(x_{1},\ldots,x_{d})$$. Sensitivity analysis seeks to quantify how variation in $$y(\boldsymbol{x})$$ can be apportioned to the inputs x 1, , x d and to the interactions among these inputs.
Thomas J. Santner, Brian J. Williams, William I. Notz

### Chapter 8. Calibration

Ideally, every computer simulator should be calibrated using observations from the physical system that is modeled by the simulator. Roughly, calibration uses data from dual simulator and physical system platforms to estimate, with uncertainty, the unknown values of the calibration inputs that govern the physical system (and which can be set in the simulator).
Thomas J. Santner, Brian J. Williams, William I. Notz