Optimal design for smooth supersaturated models
Introduction
The basic idea of smooth supersaturated models (SSM) on which this paper is founded appears in Bates et al. (in press), and follows a few years of development (an arXiv version has been available since 2009), particularly in the context of computer experiments. In the present paper a theory of optimal experimental designs for SSM is developed. Insofar as a high order SSM can be considered as an approximation to a multidimensional spline, a solution to the optimal design problem for SSM gives an approximate solution to optimal design for splines which, in high dimensions is not very much studied, but see Studden and VanArmann (1969) and Dette et al. (2001) for some work in the area.
As with splines there is the problem of the choice of knots. We shall explain how optimal design and optimal choice of knots arise as two different problems and suggest solutions to each.
Let be a general point in Rk. A monomial is defined by a non-negative integer vector : Following the experimental design avenue of algebraic statistics (Pistone et al., 2001), it is clear that for observations over any design Dn with n points in Rk there is at least one exact polynomial interpolator. Specifically, let a design be defined as a set of distinct points in Rk. A general polynomial model can be written asfor some set M of distinct index vectors, α. The algebraic theory shows that there is always a set of indices M for which we have an exact interpolation of a set of observations at design points respectively and for which the size of M is n: . Moreover there is a method of finding M based on Gröbner bases and M has a hierarchical structure: if then , for any , where is the usual entrywise ordering. We speak informally of “the model M”. The algebraic method is the starting point or at least a theoretical underpinning for SSM.
A supersaturated polynomial model is one in which the number of parameters, p, is larger than that suggested by the size of the design, the number of observations n. In the present day terminology we may say that this is a “p bigger than n problem”. However, the SSM approach is a little different. Initially we increase the size of the model, so that . In statistical terminology this leaves “free” degrees of freedom which we use to increase the smoothness of the model, in a well defined sense, while still interpolating the original data set y.
Section snippets
The SSM construction
We start with a data set y over a design Dn, with n points (D for short). The data y is given as a column vector of size n. Write the vector of model terms as so thatwhere θ is the vector of coefficients for monomials in f(x) in a suitable order according to elements of M. Denote the number of model terms as and assume that . Let the region of interest be , which we call the “integration region”. Our measure of smoothness Bates et al. (2003) is
Large bases and splines
The optimality criteria ϕ is precisely that for which splines, particularly cubic beta splines and thin plate splines are known to be optimal in the class of functions with bounded, continuous derivatives of given order. It ought to be true that as the size of the basis given by gets large in an appropriate way, we tend to splines. We state a somewhat more general definition than is available in the literature and refer to Dupont and Scott (1980). Definition 5 Let Ω be a bounded Lebesgue integrable
Orthogonal polynomials and computation
The computations required for smooth supersaturated models can be easily implemented when the number of factors is small, say less than ten, and also the number of observations and terms is not huge, say in the region of a few hundred observations and a few hundred extra terms. However, as the number of factors increase, computations may slow considerably. Computing the matrix K involves summation over k2 pairs of factors, and in each case a matrix of size N×N is to be computed. Some efficiency
Optimal knots
If one considers the interpolation discussed here as a method which can be applied to any data set y, at the selected knot design D, then there is a sense in which D should be independent of the actual (true) underlying process yielding the data.
For a model given by M and given that , we may consider simple measures on the matrix Q. Consider for motivation the case when the vector of observations y has a multivariate distribution with mean vector μ and covariance matrix . Then the
Optimal design
As discussed briefly above we shall study the pure optimal design problem for the kernel basis given by the SSM basis: . For this we take the classical approach. We assume that we have a design d with m points: . The model for observation Yi, taken at point , is Here β is the vector of parameters and errors εi are independent and normally distributed with zero mean and variance σ2. To study design we need the design matrix
Discussion
An issue not covered in this paper is the fact that the integration region can be quite general (provided the integral exists) and one could also change the measure integration of Eq. (4) all without changing the basic theory. This is implied also by the general definition of a Duchon spline given in Section 3, see Duchon (1976). We could also have given a version for general measures. Thus one can see the paper as an approach to solving spline-like problems of a quite general nature,
Acknowledgments
The third and fourth authors acknowledge support under EPSRC UK Grants EP/D048893/1 and EP/K036106/1 and EPSRC PhD funding for the first author. The fourth author acknowledges a Leverhulme emeritus fellowship.
References (12)
- et al.
A global selection procedure for polynomial interpolators
Technometrics
(2003) - Bates, R., Maruri-Aguilar, H., Wynn, H.P. Smooth supersaturated models. J. Stat. Comput. Simul.,...
- et al.
A lasso for hierarchical interactions
Ann. Stat.
(2013) - Dette, H., Melas, V.B., Pepelyshev, A., 2011. Optimal design for smoothing splines. Ann. Inst. Stat. Math. 63,...
- Duchon, J., 1976. Splines minimizing rotation invariant semi-norms in Sobolev spaces. In: Lectures Notes in...
- et al.
Polynomial approximation of functions in Sobolev spaces
Math. Comput.
(1980)