On-line monitoring of batch processes using generalized additive kernel principal component analysis
Introduction
Batch process monitoring is very important to ensure operational safety and high quality in the biochemical, polymer, pharmaceuticals, semiconductor and food industries. To detect faults accurately and quickly in order to recover the normal operation as soon as possible, on-line batch process monitoring has gotten much attention. Compared with traditional continuous processes, batch processes consider each batch of finite time duration as a unit. The collected dataset of a batch process is often organized as a three-way array , which contains the values of J monitoring variables at K time intervals in I batches. For the three-way array, many researchers have proposed different processing approaches from different viewpoints to make it adapt to traditional multivariate statistical process control (MSPC) methods such as principal component analysis (PCA), partial least squares (PLS) and independent component analysis (ICA) [1], [2], [3], [4], [5], [6], [7], [8], [9]. The commonly used modeling approaches summarized by Camacho et al. [8] mainly include: (i) single model approach based on batch-wise unfolding (BWU), variable-wise unfolding (VWU) or batch dynamic unfolding (BDU); (ii) K-models approach based on local, evolving, uniformly weighted moving window or exponential weighted evolving window models; (iii) hierarchical or multi-block approach; and (iv) multi-phase approach.
In [8], Camacho et al. compared these approaches in theory by analyzing their covariance structures, and they further compared their modeling performance of the corresponding PLS models on the actual industrial datasets in [10]. Generally speaking, VWU considers each J-dimensional sampling vector as a sample, so it just considers the correlation between different variables and cannot capture the dynamics of a process. In contrast, BWU considers each batch as a KJ-dimensional vector and fairly treats different monitoring variables and time intervals in a batch run, so its model considers both the correlation between different variables and the dynamic information simultaneously and synthetically. As a general unfolding mechanism, BDU uses the number of lagged measurement vectors (LMVs) as the additional degree of freedom in the model to balance the capability of capturing dynamic structure and the parsimony of the model. Based on the results in [10], often the performance of BWU and BDU is better than VWU, while there is no statistically significant difference between BDU and BWU. Furthermore, with the increasing of the number of models, e.g. using the K-models approach, the computational cost will increase. Therefore, the simple and direct unfolding approach of BWU which also gets stable and satisfying monitoring performance has been widely applied in the batch process monitoring problems [1], [2], [6], [11], [12], [13].
Among various monitoring methods, multiway principal component analysis (MPCA) [1], [11] based on BWU is one of the most important on-line monitoring methods. On one hand, like BWU discussed above, MPCA takes into account of the correlation between different variables and the dynamics of the process. On the other hand, it fully utilizes the structural feature of the three-way array and loading vectors to design the on-line monitoring strategies that are easy to realize [11]. For example, based on the fact that each batch sample can be decomposed through the sampling time, the total squared prediction error (SPE) statistic of a whole batch can be decomposed as the sum of the local SPE values at K time intervals. For on-line estimating the score vectors, besides the approach of filling the unknown observations with zeros or the current sampling values (assuming that all the trajectories have been centered and scaled in advance), MPCA can directly calculate the scores by projecting the already known observations into a reduced space based on the least squares approach. These characteristics help MPCA get the concise on-line monitoring strategies and obtain satisfying actual monitoring performance as well.
Nevertheless, MPCA assumes that the relationships between different variables and time intervals are only linear, which does not hold in some situations. In some complicated industrial processes, nonlinear relationships may exist between some variables and/or time intervals, so the nonlinear monitoring methods are needed. Generally there are two categories of nonlinear monitoring methods so far [14]. One is introducing the kernel trick to the above linear monitoring methods [15], [16], [17]. The other is utilizing the idea of one-class classification in machine learning, such as Gaussian mixture models (GMM) [18], [19], k-nearest neighbor (k-NN) [20], [21], one-class support vector machine (OCSVM) [22], and support vector data description (SVDD) [23], [24] methods. The former is more direct and has strong extendibility. The linear monitoring methods mentioned above can all be extended to their kernel counterparts. The nonlinear version of MPCA called multiway kernel principal component analysis (MKPCA) is first studied by Lee et al. [15]. It maps each batch (using BWU) from the original space into the feature space via a nonlinear transformation. Based on the kernel trick, MKPCA performs the eigen-decomposition on the kernel matrix (using polynomial, Gaussian, sigmoid, or other kernels) to compute the principal components in the feature space. Compared with other nonlinear methods, KPCA is easy to realize, which just needs linear algebra and needs no nonlinear optimization [15]. Like MPCA, MKPCA also constructs two statistics to monitor the systematic part (i.e. the principal component subspace) and noisy part (i.e. the residual subspace) respectively. Via filling the unknown observations with zeros or the current sampling values, MKPCA can be used for on-line monitoring of batch processes. However, using the above kernel functions contains the interaction terms between different time intervals, and thus mixes the data information at different time intervals in the feature space. It makes MKPCA face some problems in the on-line monitoring applications. For example, the total SPE statistic in MKPCA often could not be decomposed into K components associated with K time intervals independently, and the score vectors could not be estimated by projecting the already known observations into a reduced feature space. More details could be found in Section 2. Therefore, MKPCA has not inherited all the good properties of MPCA for on-line batch process monitoring.
For solving these problems, this paper explores a novel nonlinear PCA method for on-line monitoring of batch processes via utilizing the structural characteristics of kernel matrices and the concept of generalized additive kernels. The corresponding method is called generalized additive kernel PCA (GAKPCA) in brief. The proposed method can handle the possible nonlinear relationships between different variables and/or time intervals, and remains the good properties of MPCA used for on-line monitoring at the same time, such as the division of the total SPE statistic and directly estimating the score vectors. Based on the generalized additive kernels, the corresponding kernel matrices of BWU, VWU and BDU show particular connections, which complement the results in [8] which analyzes their covariance matrices. Specially, when the Gaussian kernel is chosen as the kernel function at each time interval, the entire kernel function between two batches can be regarded as a generalized similarity measure between two J-dimensional random variables, which corresponds to the concept of correntropy defined in information theoretic learning [25]. The property of the correntropy induced metric (CIM) [25] brings robustness to our method if the correntropy is utilized during the whole modeling and monitoring process.
In summary, the rest of this paper is organized as follows. By analyzing the differences between MKPCA and MPCA for on-line batch process monitoring in Section 2, the rationale of the proposed GAKPCA method is elaborated in Section 3. Section 3.4 discusses the relationship between our method and the generalized additive kernels, and compares different unfolding approaches by analyzing their kernel matrices. After that, Section 4 further compares the special cases of GAKPCA and MKPCA by using the Gaussian kernel, and considers the connection of the corresponding GAKPCA method and the correntropy which leads to the underlying robustness. In Section 5, a fed-batch fermentation process is utilized to test the actual on-line monitoring performance of the proposed method. Finally, some conclusions are drawn.
Section snippets
Preliminaries
In general, after trajectory synchronization and alignment [26], [27], [28], [29], the batch dataset often can be organized as a three-way array denoted by , where I is the number of batches, J is the number of variables, and K is the number of time intervals in a batch. The kth observation of variable j in batch i is denoted by xi,j,k (i = 1, …, I, j = 1, …, J, and k = 1, …, K). The sampling vector at time k in batch i is denoted by xi,k = [xi,1,k, …, xi,J,k]T, the whole sampling vector of
Nonlinear PCA with special structure
Unlike the traditional MKPCA method, here we map each sampling vector xi,k into a feature space. The sampling vector in the feature space is denoted by ϕ(xi,k), which is assumed to be an M-dimensional column vector (M can be infinite). Then batch i in the feature space can be denoted by , which is composed of K sampling components and hence has the dimension of KM. Here, we use two different symbols ϕ(·) and Φ(·) to denote the nonlinear transformation functions of
GAKPCA using the Gaussian kernel
Among various kernel functions, this section takes the Gaussian kernel as a special example to analyze the difference between GAKPCA and MKPCA on handling the nonlinear correlation. Based on its connection with the correntropy, GAKPCA presents some other interesting properties.
Case study
A fed-batch penicillin fermentation process is considered as the case study to evaluate the on-line monitoring performance of the proposed method, and a standard modular simulator (Pensim V.2.0) developed by Birol et al. [42] is used to generate the batch dataset. The penicillin fermentation process consists of a main fermenter and some accessory devices for such as substrate feed, pH and temperature control. It has been widely used in many papers [15], [18], [43] for its friendly interface,
Conclusions
In this paper, a special nonlinear PCA method called generalized additive kernel PCA (GAKPCA) has been proposed for on-line batch process monitoring based on the generalized additive kernels (sequential additive). From both the theoretic analysis and experimental results, the proposed method presents the satisfying and robust on-line monitoring performance. Fig. 4 summarizes the relationships between the GAKPCA method and some other methods such as MPCA, MKPCA and correntropy. The main
Acknowledgements
The authors would like to thank the anonymous referees for their good comments that help to improve this paper.
References (49)
- et al.
Multi-way partial least squares in monitoring batch processes
Chemom. Intell. Lab. Syst.
(1995) - et al.
Modelling and diagnostics of batch processes and analogous kinetic experiments
Chemom. Intell. Lab. Syst.
(1998) - et al.
Critical evaluation of approaches for on-line batch process monitoring
Chem. Eng. Sci.
(2002) - et al.
On-line monitoring of batch processes using multiway independent component analysis
Chemom. Intell. Lab. Syst.
(2004) - et al.
Fault detection properties of global, local and time evolving models for batch process monitoring
J. Process Contr.
(2005) - et al.
The best approaches in the on-line monitoring of batch processes based on PCA: does the modelling structure matter?
Anal. Chim. Acta
(2009) - et al.
Multivariate statistical process control based on multiway locality preserving projections
J. Process Contr.
(2008) - et al.
Fault detection of batch processes using multiway kernel principal component analysis
Comput. Chem. Eng.
(2004) - et al.
On-line batch process monitoring using hierarchical kernel partial least squares
Chem. Eng. Res. Des.
(2011) - et al.
On-line multivariate statistical monitoring of batch processes using Gaussian mixture model
Comput. Chem. Eng.
(2010)
Fault detection and diagnosis in process data using one-class support vector machines
J. Process Contr.
Batch process monitoring based on support vector data description method
J. Process Contr.
Bagging support vector data description model for batch process monitoring
J. Process Contr.
Real-time synchronization of batch trajectories for on-line multivariate statistical process control using dynamic time warping
Chemom. Intell. Lab. Syst.
Experiences in batch trajectory alignment for pharmaceutical process improvement through multivariate latent variable modelling
J. Process Contr.
Adaptive batch monitoring using hierarchical PCA
Chemom. Intell. Lab. Syst.
Principal component analysis based on non-parametric maximum entropy
Neurocomputing
Removal of the effects of outliers in batch process data through maximum correntropy estimator
Chemom. Intell. Lab. Syst.
Correntropy estimator for data reconciliation
Chem. Eng. Sci.
A modular simulation package for fed-batch fermentation: penicillin production
Comput. Chem. Eng.
Batch process monitoring in the original measurement's space
J. Process Contr.
Determining the number of principal components for best reconstruction
J. Process Contr.
Kernel PCA for novelty detection
Pattern Recogn.
Batch process monitoring based on functional data analysis and support vector data description
J. Process Contr.
Cited by (43)
Evolution stage identification of haze pollution episodes in beijing using constrained dynamic time warping and multiway principal component analysis
2023, Environmental Modelling and SoftwareCriteria for optimizing kernel methods in fault monitoring process: A survey
2022, ISA TransactionsCitation Excerpt :Deng et al. [49] propose a novel similarity factor using KPCA to reveal the statistics information hidden in original measured variables. Alternatively, Yao et al. [51] explore the concept of generalized additive KPCA for the batch monitoring processes. On the other hand, Wang et al. [50] utilize the functional KPCA to handle nonlinear correlations between monitoring variables and/or sampling times.
Linear and nonlinear hierarchical multivariate time delay analytics for dynamic modeling and process monitoring
2021, Journal of Process ControlCitation Excerpt :Due to the complexity of industrial equipment, there are significantly nonlinear relationships between process variables. Kernel method [9] is a typical technique to deal with nonlinearity, which embeds the original data into an appropriate high-dimensional feature space through some nonlinear mappings. However, it may be tough to find suitable kernel functions for data with different characteristics.
A multi-feature extraction technique based on principal component analysis for nonlinear dynamic process monitoring
2020, Journal of Process Control