Unsupervised Process Monitoring and Fault Diagnosis with Machine Learning Methods

In this introductory chapter, the drivers of process monitoring technology are reviewed to ensure safe, profitable and environmentally responsible process operation. The resultant trends are considered in terms of developments in instrumentation, computational and telecommunications hardware and process analytical developments and data-driven control strategies. Moreover, a generalized framework for data-driven fault diagnosis is discussed, as well as the role of machine learning in this framework. This framework consists of a data matrix representative of the process, a diagnostic feature matrix, a reconstructed data matrix and a residual matrix. The feature, reconstructed data and residual matrices are all derived from the data matrix. This can be accomplished by different methods, and some of those based on machine learning are summarized in broad terms, focusing on supervised and unsupervised learning, semi-supervised learning, reinforcement learning and self-taught or transfer learning.

This overview of process fault diagnosis concentrates on steady-state processes, continuous dynamic processes and batch processes. In steady-state processes, the classic linear model for process fault diagnosis based on the use of principal component analysis is discussed in some detail, followed by extensions of this model to nonlinear steady-state (non)Gaussian processes. These extensions include higher-order statistical models, such as based on the use of independent components, the use of principal curves and surfaces as well as neural networks as nonlinear extensions of principal component analysis. Likewise, innovations and applications of kernel methods are among other considered, including kernel principal component analysis, kernel partial least squares, kernel independent component analysis as well as multiple kernel learning variants of some of these approaches. Continuous dynamic processes are considered in terms of manifold models, adaptive methods and phase space methods, where the application of process diagnostics, such as correlation dimension and recurrence quantitative analyses, has been proposed. The multitude of recent developments in batch processing are similarly reviewed in terms of the multiway principal component model, extended to multiphase and multiblock models. These developments are considered in the broad framework outlined in Chap.​ 1.

The history, development and basic methodology of the most important artificial neural networks are reviewed, focusing on unsupervised learning, in keeping with the generalized framework for process fault diagnosis proposed previously. This starts with a brief summary of the construction of multilayer perceptrons, with some examples of their strengths and limitations, as well as their application in the analysis of models. Various autoassociative neural networks are discussed as an extension of multilayer perceptrons to nonlinear principal component analysis, including simple, hierarchical, inverse and circular nonlinear principal component analysis. This is followed by an introduction to radial basis function neural networks and self-organizing feature maps with basic examples of their applications. In the last parts of the chapter, restricted Boltzmann machines and deep neural network architectures are considered, as well as the basic operation of extreme learning machines. These are relatively new additions to the family of neural networks that have not become established in process fault diagnosis as yet but have the potential to make a significant contribution in the near future.

The basics of kernel methods and their position in the generalized data-driven fault diagnostic framework are reviewed. The review starts out with statistical learning theory, covering concepts such as loss functions, overfitting and structural and empirical risk minimization. This is followed by linear margin classifiers, kernels and support vector machines. Transductive support vector machines are discussed and illustrated by way of an example related to multivariate image analysis of coal particles on conveyor belts. Finally, unsupervised kernel methods, such as kernel principal component analysis, are considered in detail, analogous to the application of linear principal component analysis in multivariate statistical process control. Fault diagnosis in a simulated nonlinear system by the use of kernel principal component analysis is included as an example to illustrate the concepts.

In this chapter, tree-based methods are discussed as another of the three major machine learning paradigms considered in the book. This includes the basic information theoretical approach used to construct classification and regression trees and a few simple examples to illustrate the characteristics of decision tree models. Following this is a short introduction to ensemble theory and ensembles of decision trees, leading to random forest models, which are discussed in detail. Unsupervised learning of random forests in particular is reviewed, as these characteristics are potentially important in unsupervised fault diagnostic systems. The interpretation of random forest models includes a discussion on the assessment of the importance of variables in the model, as well as partial dependence analysis to examine the relationship between predictor variables and the response variable. A brief review of boosted trees follows that of random forests, including discussion of concepts, such as gradient boosting and the AdaBoost algorithm. The use of tree-based ensemble models is illustrated by an example on rotogravure printing and the identification of defects in hot rolled steel plate.

The different machine learning paradigms reviewed in the previous chapters are considered in depth in the context of fault diagnosis in steady-state process systems. Data-driven process fault diagnosis consists of two stages, an offline training stage and an online application stage, and both of these are revisited before considering a number of case studies. In the offline training stage, consideration is given to practical issues, such as the selection of the number of features used to represent normal operating conditions, the derivation of control limits in the feature space, where the distribution of the data is generally unknown, as well as various performance metrics, such as alarm rates, alarm run lengths, detection delays and receiver operating characteristic curves. These issues are subsequently illustrated by simulations, including that of a simple nonlinear system, the benchmark Tennessee Eastman system widely investigated in the process engineering literature and a sugar refinery. In these case studies, the performance of fault diagnostic models representative of the major classes of machine learning models, as well as principal component analysis, is considered. Moreover, a variety of performance measures, each with their different strengths and weaknesses, are discussed in depth.

Unlike steady-state process variables that do not change meaningfully over time, continuous dynamic systems exhibit stable dynamic behaviour or (quasi)periodic behaviour within a bounded region in the variable space. As a consequence, steady-state process monitoring methods may not be able to account for such behaviour and may therefore be unsuitable for monitoring such systems. In this chapter, a methodology for monitoring dynamic process systems is outlined, based on embedding of the variables in a phase space prior to the extraction of diagnostic features from the data. Three different techniques are considered, namely, singular spectrum analysis, feature extraction with random forests and feature extraction from the data with inverse nonlinear principal component analysis by the use of an autoassociative neural network. In addition, the application of recurrence quantification analysis is also investigated. Recurrence quantification analysis characterizes an attractor in the phase space by identifying and quantifying the repeated occurrences (recurrences) of points in the same neighbourhood. This can then be used as a sensitive diagnostic of changes in the underlying dynamics of the process system. The use of these methods is illustrated by the use of three nonlinear systems, that is, a Lotka–Volterra predator–prey model, the Belousov–Zhabotinsky reaction as well as a simulated autocatalytic process. Although the different strategies considered are all in principle capable of detecting changes in complex data structures, in the case studies associated with the autocatalytic process, detection of change was difficult, since the change in the geometry of the attractor was very subtle. The trajectory of the new data remained mostly within the decision envelopes of the monitoring schemes, although the density distribution of the new data within this envelope changed. However, in this instance, diagnostic variables derived from the recurrence plots of the data were able to detect these changes, and the use of such methods to complement machine learning methods could lead to more capable diagnostic approaches to deal with complex data.

Principal component analysis is widely used in disturbance detection, isolation and diagnosis in industrial and chemical processes, and several extensions of the basic principal component methodology have been considered in previous chapters to handle features such as autocorrelation in data, time–frequency localization and nonlinearity. In this chapter, a statistical process control approach based on singular spectrum analysis is proposed. The method involves expressing a time series as the sum of identifiable components whose basis functions are obtained from measurements. Using decomposition by means of singular spectrum analysis, a multimodal representation is obtained that can be used together with existing statistical process control methods to construct a novel process monitoring scheme. It is observed that singular spectrum analysis can perform significantly better than other methods, particularly in detecting mean shift changes. However, the performance of the approach can degrade in the presence of parameter changes, as well as excessive autocorrelation of the variables.