Response surface methodology with prediction uncertainty: A multi-objective optimisation approach

Abstract

In the field of response surface methodology (RSM), the prediction uncertainty of the empirical model needs to be considered for effective process optimisation. Current methods combine the prediction mean and uncertainty through certain weighting strategies, either explicitly or implicitly, to form a single objective function for optimisation. This paper proposes to address this problem under the multi-objective optimisation framework. Overall, the method iterates through initial experimental design, empirical modelling and model-based optimisation to allocate promising experiments for the next iteration. Specifically, the Gaussian process regression is adopted as the empirical model due to its demonstrated prediction accuracy and reliable quantification of prediction uncertainty in the literature. The non-dominated sorting genetic algorithm II (NSGA-II) is used to search for Pareto points that are further clustered to give experimental points to be conducted in the next iteration. The application study, on the optimisation of a catalytic epoxidation process, demonstrates that the proposed method is a powerful tool to aid the development of chemical and potentially other processes.

Introduction

Response surface methodology (RSM) is a widely used technology for rational experimental design and process optimisation in the absence of mechanistic information (Box and Draper, 1987, Myers and Montgomery, 1995). RSM initiates from design of experiments (DoE) to determine the factors’ values for conducting experiments and collecting data. The data are then used to develop an empirical model that relates the process response to the factors. Subsequently, the model facilitates to search for better process response, which is validated through experiment(s). The above procedure iterates until an optimal process is identified or the limit on experimental resources is reached. RSM has seen diverse applications in almost every area of scientific research and engineering practice, including the development of chemical and biochemical processes (Agatonovic-Kustrin et al., 1998, Baumes et al., 2004, Dutta et al., 2004, Hadjmohammadi and Kamel, 2008, Shao et al., 2007, Tang et al., 2010, Yan et al., 2011a, Yan et al., 2011b).

In traditional RSM, the first- or second-order polynomial function is adopted for empirical modelling. However, the restrictive functional form of polynomials has long been recognised as ineffective in modelling complex processes. Progress in adopting more flexible models in RSM includes artificial neural networks (ANN) (Agatonovic-Kustrin et al., 1998, Baumes et al., 2004, Dutta et al., 2004, Shao et al., 2007), support vector regression (SVR) (Hadjmohammadi and Kamel, 2008, Serna et al., 2008), and more recently Gaussian process regression (GPR) (Tang et al., 2010, Yan et al., 2011a, Yan et al., 2011b, Yuan et al., 2008). GPR, also known as kriging model with slightly different formulation, has been accepted as a powerful modelling tool in various fields, in particular in process systems engineering (Ge and Song, 2010, Grancharova et al., 2008, Likar and Kocijan, 2007). GPR is attractive partly because of the sound theoretical foundation: it can be derived from the perspective of either ANN in the limit of an infinite network, or Bayesian regression (Rasmussen and Williams, 2006). In practice, GPR has been shown to be superior to or comparable with ANN and SVR in terms of prediction accuracy (Hernández et al., 2008, Rasmussen, 1996, Yuan et al., 2008). Therefore, GPR is utilised in this study for empirical modelling.

When an empirical model has been developed, the usual approach to process optimisation is to find factors’ value x^* that gives the maximal predicted response, and then conduct a new experiment at x^*. (Throughout this paper, we assume that the objective is to maximise the response variable.) However, this is not an ideal method, since it ignores the predictive uncertainty that quantifies the mismatch between model prediction and the actual process. In fact, predictive uncertainty, usually expressed in terms of variance, is available in all empirical models through either classical statistical inference (e.g. for polynomial regression, ANN and SVR) or Bayesian approach (e.g. for Bayesian ANN and GPR). A large predictive variance usually suggests that the experimental data around this point are not sufficient to give a reliable prediction. Hence, a design point that is predicted to give inferior response with high variance may actually result in improved process. Therefore, both predictive mean and variance must be jointly considered in the optimisation algorithm. In particular, new experiment(s) should be allocated so that either the mean prediction is large, or the prediction uncertainty is large.

In the literature, several methods have been proposed to handle prediction uncertainty when using empirical models for optimisation, including maximisation of prediction bounds (Apley et al., 2006, Tang et al., 2010, Yuan et al., 2008), minimisation of information free energy (Lin and Jang, 1998, Chen et al., 1998), maximisation of relative information gain (Coleman and Block, 2007), and maximisation of expected improvement (Jones et al., 1998, Jones, 2001). The basic rationale of these methods is to combine the prediction mean and uncertainty in the optimisation algorithm by using a user-determined weight, either explicitly or implicitly (this will be discussed subsequently). Clearly, the appropriateness of the selected weight needs to be carefully examined to ensure effective optimisation.

This paper proposes an alternative approach to RSM in the presence of model uncertainty. The idea is to cast this problem into a multi-objective optimisation framework (Deb, 2001) that seeks to maximise both prediction mean and variance simultaneously. Through this formulation, Pareto solutions can be identified using a standard multi-objective genetic algorithm; the nondominated sorting genetic algorithm II (NSGA-II) (Deb et al., 2002) is adopted in this study. It has been well recognised that seeking the entire Pareto set gives a more complete picture of multi-objective problems than using a fixed weighting strategy (Deb, 2001). In addition, in face of limited experimental resource, the identified Pareto points will be clustered into a few groups, and only the points that are closest to the cluster centres will be selected for experimentation in the next iteration. We further suggest to visualise the clustered Pareto points and their predictive mean/uncertainty in order to aid the decision-making by the experimenters. Compared with fully automatic algorithm, this “interactive” approach may receive wider acceptance when used for investigating real processes, because it involves active and subjective decision of the experimenter and this human intervention brings in domain knowledge that is often difficult to be properly incorporated in the modelling framework.

The proposed algorithm will be validated through maximising the conversion rate of a catalytic reaction process for the epoxidation of cis-cyclooctene. Cyclooctene oxide is an important intermediate used in the synthesis of various fine chemicals and pharmaceuticals. Recently, cobalt ion-exchanged faujasite zeolite (Co²⁺–NaX) has been reported as an efficient heterogeneous catalyst for several epoxidation processes (Sebastian et al., 2006, Tang et al., 2010, Yan et al., 2011b), and it is being tested for cis-cyclooctene epoxidation in our laboratory. Hence, the current work serves a dual purpose: to propose a novel solution to RSM in the presence of model uncertainty, and to demonstrate its application to an important catalytic reaction.