Abstract
The problem of optimal experimental design for response optimization is considered. The optimal point (control)x * of a response surface is to be determined by estimating the response parametersθ from measurements performed at design pointsx i,i=1,...,N. Classical sequential approaches for choosing thex i's are recalled. A loss function related to the issue of response optimization is used to define control-oriented design criteria. The design policies differ depending on whether least-squares or minimum risk estimation is used to estimateθ. Connections between various criteria suggested in the literature are exhibited. Special attention is given to quadratic model responses. Most approaches presented assume that the response is correctly described by a given parametric function over the region of interest. Possible deterministic departures from this function raise the problem of model robustness, and the literature on the subject is briefly surveyed.
Similar content being viewed by others
References
Nelder, J. A. and Mead, R.: A simplex method for function minimization,Computer J. 7 (1965), 308–313.
Kiefer, J. and Wolfowitz, J.: Stochastic estimation of the maximum of a regression function,Ann. Math. Statist. 23 (1952), 462–466.
Polyak, B. T. and Tsypkin Ya. Z.: Pseudogradient adaptation and training algorithms,Autom. Remote Control 34(3), (1973), 377–397. (Translated fromAutom. Telemekh. 3 (1973), 45–68.
Dvoretzky, A.: On stochastic approximation,Proc. 3rd Berkeley Symp. Math. Statist. Probab., vol. 1 (1956), 39–55.
Box, G. E. P. and Wilson, K. B.: On the experimental attainment of optimum conditions (with discussion),J. Royal Statist. Soc. B13(1), (1951), 1–45.
Hill, W. J. and Hunter, W. G.: A review of response surface methodology: a literature survey,Technometrics 8 (1966), 571–590.
Myers, R. H.:Response Surface Methodology, Allyn and Bacon, Boston, 1976.
Montgomery, D. C.:Design and Analysis of Experiments, Wiley, New York, 1976.
Steinberg, D. M.: Model robust response surface designs: scaling two-level factorials,Biometrika 72(3), (1985) 513–526.
Brooks, S. H. and Mickey, R. M.: Optimum estimation of gradient direction in steepest ascent experiments,Biometrics 17 (1961), 48–56.
Scheffé, H.: Experiments with mixtures,J. Royal Statist. Soc. B21 (1958), 344–360.
Galil, Z. and Kiefer, J.: Comparison of simplex designs for quadratic mixture models,Technometrics 19(3), (1977) 445–453.
Box, G. E. P. and Hunter, J. S.: Multi-factor experimental designs for exploring response surfaces,Ann. Math. Statist. 28 (1957), 195–241.
Lindley, D. V.: The choice of variables in multiple regression,J. Royal Statist. Soc. B32 (1968), 31–53.
Brooks, R. J.: Optimal regression design for control in linear regression,Biometrika 64(2), (1977), 319–325.
Verdinelli, I. and Wynn, H. P.: Target attainment and experimental design, a Bayesian approach, in Y. Dodge, V. V. Fedorov and H. P. Wynn (eds),Optimal Design and Analysis of Experiments, North-Holland, Amsterdam, 1988, pp. 319–324.
Goodwin, G. C. and Payne, R. L.:Dynamic System Identification: Experiment Design and Data Analysis, Academic Press, New York, 1977.
Chatterjee, S. K. and Mandal, N. K.: Response surface designs for estimating the optimal point,Calcutta Statist. Assoc. Bull. 30 (1981), 145–169.
Chaloner, K.: Optimal Bayesian design for nonlinear estimation, Technical Report 468, University of Minnesota, 1986.
Buonaccorsi, J. P. and Iyer, H. K.: Optimal designs for ratio of linear combinations in the general linear model,J. Statist. Plann. Inference 13 (1986), 345–356.
Silvey, S. D.:Optimal Design, Chapman and Hall, London, 1980.
Chaloner, K. and Larntz, K.: Optimal Bayesian design applied to logistic regression experiments, Technical Report 483, University of Minnesota, 1986.
Chaloner, K.: An approach to experimental design for generalized linear models, in V. V. Fedorov and H. LÄuter (eds),Model Oriented Data Analysis, Springer, Berlin, 1988, pp. 3–12.
Fedorov, V. V.:Theory of Optimal Experiments, Academic Press, New York, 1972.
Pazman, A.:Foundations of Optimum Experimental Design, D. Reidel, Dordrecht, and Veda, Bratislava, 1986.
Elfving, G.: Optimum allocation in linear regression theory,Ann. Math. Statist. 23 (1952), 255–262.
Studden, W. J.: Elfving's theorem and optimal designs for quadratic loss.Ann. Math. Statist. 42 (1971), 1613–1621.
Silvey, S. D.: Optimal design measures with singular information matrices,Biometrika 65(3), (1978) 553–559.
Atwood, C. L.: Convergent design sequences, for sufficiently regular optimality criteria, II: singular case,Ann. Statist. 8(4), (1980), 894–912.
Fedorov, V. V. and Atkinson, A. C.: The optimum design of experiments in the presence of uncontrolled variability and prior information, in Y. Dodge, V. V. Fedorov and H. P. Wynn (eds),Optimal Design and Analysis of Experiments, North-Holland, Amsterdam, 1988, pp. 327–344.
Walter, E. and Pronzato, L.: Qualitative and quantitative experiment design for phenomenological models survey,Automatica 26(2), (1990), 195–213.
Lindley, D. V.: On a measure of information provided by an experiment,Ann. Math. Statist. 27 (1956), 986–1005.
Mandal, N. K.:D-optimal designs for estimating the optimum point in a quadratic response surface rectangular region,J. Statist. Plann. Inference 23 (1989), 243–252.
Pronzato, L. and Walter, E.: Robust experiment design via stochastic approximation,Math. Biosci. 75 (1985), 103–120.
Walter, E. and Pronzato, L.: How to design experiments that are robust to parameter uncertainty,Preprints 7th IFAC/IFORS Symp. Identification and System Parameter Estimation, York, vol. 1 (1985), 921–926.
Walter, E. and Pronzato, L.: Robust experiment design: between qualitative and quantitative identifiabilities, in E. Walter (ed.),Identifiability of Parametric Models, Pergamon, Oxford, 1987, pp. 104–113.
Pronzato, L. and Walter, E.: Robust experiment design for nonlinear regression models, in V. V. Fedorov and H. LÄuter (eds.),Model Oriented Data Analysis, Springer, Berlin, 1988, pp. 77–86.
Pronzato, L. and Walter, E.: Robust experiment design via maximin optimization,Math. Biosci. 89 (1988), 161–176.
Pronzato, L. and Walter, E.: Nonsequential Bayesian experimental design for response optimization, in V. Fedorov, W. G. Müller and I. N. Vuchkov (eds),Proc. 2nd Internat. Workshop on ModelOriented Data Analysis, St. Kyrik, Bulgaria, Physica-Verlag, Heidelberg, 1992, pp. 89–102.
Pilz, J.:Bayesian Estimation and Experimental Design in Linear Regression Models, Teubner Texte zur Mathematik, Leipzig, 1983.
Bandemer, H., NÄttier, W. and Pilz, J.: Once more: Optimal experimental design for regression models (with discussion),Statistics 18(2), (1987), 171–217.
Giovagnoli, A. and Verdinelli, I.: BayesD-optimal andE-optimal block designs,Biometrika 70(3) (1983), 695–706.
Chaloner, K.: Optimal Bayesian experimental design for linear models,Ann. Statist. 12(1) (1984), 283–300.
Ylvisaker, D.: Prediction and design,Ann. Statist. 15(1) (1987), 1–19.
Steinberg, D. M. and Hunter, W. G.: Experimental design: review and comment (with discussion),Technometrics 26(2) (1984), 71–130.
Box, G. E. P. and Draper, N. R.: A basis for the selection of a response surface design,J. Amer. Statist. Assoc. 54 (1959), 622–654.
Karson, M. J., Manson, A. R. and Hader, R. J.: Minimum bias estimation and experimental design for response surfaces,Technometrics 11 (1969), 461–475.
Kiefer, J.: Optimum designs for fitting biased multiresponse surfaces, in P. R. Krishnaiah (ed.),Maltivariate Analysis 3, Academic Press, New York, 1973, pp. 287–297.
Stigler, S. M.: Optimal experimental design for polynomial regression,J. Amer. Statist. Assoc. 66 (1971), 311–318.
Studden, W. J.: Some robust-typeD-optimal designs in polynomial regression,J. Amer. Statist. Assoc. 77 (1982), 916–921.
Huber, P. J.: Robustness and designs, in J. N. Srivastava (ed.),A Survey of Statistical Design and Linear Models, North-Holland, Amsterdam, 1975, pp. 287–301.
Marcus, M. B. and Sacks, J.: Robust designs for regression problems, inStatistical Decision Theory and Related Topics II, Academic Press, New York, 1977, pp. 245–268.
Sacks, J. and Ylvisaker, D.: Linear estimation for approximately linear models,Ann. Statist. 6(5) (1978), 1122–1137.
Sacks, J. and Ylvisaker, D.: Some model robust designs in regression,Ann. Statist. 12(4) (1984), 1324–1348.
Pesotchinsky, L.: Optimal robust designs: linear regression in 67-01,Ann. Statist. 10(2) (1982), 511–525.
Welch, W. J.: A mean squared error criterion for the design of experiments,Biometrika 70(1) (1983), 205–213.
Kiefer, J.: General equivalence theory for optimum designs (approximate theory).Ann. Statist. 2(5) (1974), 849–879.
O'Hagan, A.: Curve fitting and optimal design for prediction (with discussion),J. Royal Statist. Soc. B40(1) (1978), 1–42.
Goldstein, M.: The linear Bayes regression estimator under weak prior assumptions,Biometrika 67(3) (1980), 621–628.
Blight, B. J. N. and Ott, L.: A Bayesian approach to model inadequacy for polynomial regression,Biometrika 62(1) (1975), 79–88.
Sacks, J. and Ylvisaker, D.: Model robust design in regression: Bayes theory, in L. M. Le Cam and R. A. Olshen (eds),Proc. Berkeley Conference in Honour of J. Neyman and J. Kiefer, vol.2, 1985, pp. 667–679.
Currin, C., Mitchell, T., Morris, M. and Ylvisaker D.: A Bayesian approach to the design and analysis of computer experiments, ORNL Technical Report 6498, available from the National Technical Information Service, 5285 Port Royal Road, Springfield, VA 22161, 1988.
Sacks, J., Welch, W.J., Mitchell, T. and Wynn, H. P.: Design and analysis of computer experiments (with discussion),Statist. Sci. 4(4) (1989), 409–435.
Shewry, M. C. and Wynn, H. P.: Maximum entropy sampling,J. Appl. Statist. 14 (1987), 165–170.
Sacks, J., Schiller, S. B. and Welch, W. J.: Designs for computer experiments,Technometrics 31 (1989), 41–47.
Atkinson, A. C. and Donev, A. N.: Algorithms, exact design and blocking in response surface and mixture designs, in Y. Dodge, V. V. Fedorov and H. P. Wynn (eds),Optimal Design and Analysis of Experiments, North-Holland, Amsterdam, 1988, pp. 61–69.
Yonchev, H.: New computer procedures for constructingD-optimal designs, in Y. Dodge, V.V. Fedorov and H. P. Wynn (eds),Optimal Design and Analysis of Experiments, North-Holland, Amsterdam, 1988, pp. 71–80.
Atkinson, A. C. and Cox, D. R.: Planning experiments for discriminating between models (with discussion),J. Royal Statist. Soc. B36 (1974), 321–348.
Hill, P. D. H.: A review of experimental design procedures for regression model discrimination,Technometrics 20(1) (1978), 15–21.
Ford, I. and Silvey, S. D.: A sequentially constructed design for estimating a nonlinear parametric function,Biometrika 67(2) (1980), 381–388.
Wu, C. F. J.: Asymptotic inference from sequential design in a nonlinear situation,Biometrika 72(3) (1985), 553–558.
Ford, I., Titterington, D. M. and Wu, C. F. J.: Inference and sequential design,Biometrika 72(3) (1985), 545–551.
Zacks, S.: Problems and approaches in design of experiments for estimation and testing in nonlinear models, in P. R. Krishnaiah (ed.),Multivariate Analysis IV, North-Holland, Amsterdam, 1977, pp. 209–223.
Bayard, D. S. and Schumitzky, A.: A stochastic control approach to optimal sampling design, Univ. of Southern California, School of Medicine, Lab. of Applied Pharmacokinetics, Technical Report,90-1 (1990).
Chernoff, H.: Approaches in sequential design of experiments, in J. N. Srivastava (ed.),A Survey of Statistical Design and Linear Models, North-Holland, Amsterdam, 1975, pp. 67–90.
Fel'dbaum, A. A.:Optimal Control Systems, Academic Press, New York, 1965.
Bar-Shalom, Y. and Tse, E.: Dual effect, certainty equivalence, and separation in stochastic control,IEEE Trans. Automat. Control AC-19(5) (1974), 494–500.
Padilla, C. S. and Cruz, JR. J. B.: A linear dynamic feedback controller for stochastic systems with unknown parameters,IEEE Trans. Automat. Control AC-22(1) (1977), 50–55.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Pronzato, L., Walter, E. Experimental design for estimating the optimum point in a response surface. Acta Appl Math 33, 45–68 (1993). https://doi.org/10.1007/BF00995494
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF00995494