Data-Driven Prediction for Industrial Processes and Their Applications

This chapter gives an overall introduction to this book. First, we discuss the importance of the prediction for industrial process. Then, we divide the data-driven prediction methodology discussed in this book into a number of categories. Specifically, there are three categories, i.e., data feature-based methods, time scale-based ones, and prediction reliability-based ones. Besides, considering the characteristics of prediction modeling and industrial demands, this book introduces some commonly used prediction techniques, including the time series-based methods, the factor-based methods, the prediction intervals (PIs) construction methods, and the granular-based long-term prediction methods.

Time series prediction is a significant way for forecasting the variables involved in industrial process, which usually identifies the latent rules hidden behind the time series data of the variables by means of auto-regression. In this chapter we introduce the phase space reconstruction technique, which aims to construct the training dataset for modeling, and then a series of data-driven machine learning methods are provided for time series prediction, where some well-known artificial neural networks (ANNs) models are introduced, and a dual estimation-based echo state network (ESN) model is particularly proposed to simultaneously estimate the uncertainties of the output weights and the internal states by using a nonlinear Kalman-filter and a linear one for noisy industrial time series. In addition, the kernel based methods, including Gaussian processes (GP) model and support vector machine (SVM) model, are also presented in this chapter. Specifically, an improved GP-based ESN model is proposed for time series prediction, in which the output weights in ESN modeled by using GP avoids the ill-conditioned phenomenon associated with the generic ESN version. A number of case studies related to industrial energy system are provided to validate the performance of these methods.

This chapter gives the factors-based prediction methods for industrial processes. Different from the mentioned time series-based prediction, this kind of approaches construct a forecasting model by treating the process variables (not the output or target variables) called “factors” as the model inputs, rather than the auto-regression mode used in time series version. To select the factors from lots of candidates, this chapter firstly introduces some commonly used feature selection approaches such as gray correlation method and convolution-based methods. As for the single-output model, this chapter introduces NNs-based model, Takagi-Sugeno (T-S) fuzzy model, and SVM. In particular, a multi-kernel setting of a LSSVM model can perform better explanatary ability for learning a nonlinear model and fit the regression problem more effectively. Besides, this chapter also introduces a multi-output LSSVM model, which considers the single fitting error of each output and the combined error as well, and aims at the issues of multiple interactional outputs in industrial system. This chapter also provides some case studies on industrial energy system for performance verification.

Prediction intervals (PIs) construction is a comprehensive prediction technique that provides not only the point estimates of the industrial variables, but also the reliability of the prediction results indicated by an interval. Reviewing the conventional PIs construction methods (e.g., delta method, mean and variance-based estimation method, Bayesian method, and bootstrap technique), we provide some recently developed approaches in this chapter. Here, a bootstrapping-based ESN ensemble (BESNE) model is specially proposed to produce reliable PIs for industrial time series, in which a simultaneous training method based on Bayesian linear regression is developed. Besides, to cope with the error accumulation caused by the traditional iterative mode of time series prediction, a non-iterative granular ESN is also reported for PIs construction, where the network connections are represented by the interval-valued information granules. In addition, we present a mixed Gaussian kernel-based regression model to construct PIs, in which a gradient descent algorithm is derived to optimize the hyper-parameters of the mixed Gaussian kernel. In order to tackle the incomplete testing input problem, a kernel-based high order dynamic Bayesian network (DBN) model for industrial time series is then proposed, which directly deals with the missing points involved in the inputs. Finally, we provide some case studies to verify the effectiveness of these approaches.

In industrial practice, long-term prediction for process variables is fairly significant for the process industry, which is capable of providing the guidance for equipment control, operational scheduling, and decision-making. This chapter firstly introduces the basic principles of granularity partition, and a long-term prediction model for time series and factor-based prediction are developed in this chapter. In terms of time series prediction, the unequal-length temporal granules are constructed by exploiting dynamic time warping (DTW) technique, and a granular-computing (GrC)-based hybrid collaborative fuzzy clustering (HCFC) algorithm is proposed for the mentioned factor-based prediction problem. Besides, in this chapter, the long-term prediction approach is also combined with the PIs construction in order to provide the prediction reliability in the context of long-term time series task. Similarly, the PIs construction on multi-dimension problem is also introduced by employing the structure of the HCFC algorithm. To verify the effectiveness of these approaches, this chapter provides some experimental analysis on industrial data coming from an energy data center of a steel plant.

The selection of parameters or hyper-parameters gives great impact on the performance of a data-driven model. This chapter introduces some commonly used parameter optimization and estimation methods, such as the gradient-based methods (e.g., gradient descend, Newton method, and conjugate gradient method) and the intelligent optimization ones (e.g., genetic algorithm, differential evolution algorithm, and particle swarm optimization). In particular, in this chapter, the conjugate gradient method is employed to optimize the hyper-parameters in a LSSVM model based on noise estimation, which enable to alleviate the impact of noise on the performance of the LSSVM. As for dynamic models, this chapter introduces nonlinear Kalman-filter methods for parameter estimation. The well-known ones include the extended Kalman-filter, the unscented Kalman-filter, and the cubature Kalman-filter. Here, a dual estimation model based on two Kalman-filters is illustrated, which simultaneously estimates the uncertainties of internal state and the output. Besides, the probabilistic methods for parameter estimation are also introduced, where a Bayesian model, especially a variational inference framework, is elaborated in details. In such a framework, a particular variational relevance vector machine (RVM) model based on automatic relevance determination kernel is introduced, which provides the approximated posterior distributions over the kernel parameters. Finally, we give some case studies by employing a number of industrial data.

This chapter discusses the computational cost of machine learning model. To reduce its training time is a requisite of its industrial applications since a production process usually requires real-time responses. The commonly used method to accelerate the training process is to develop a parallel computing framework. In literature, two kinds of popular methods speeding up the training involves the one with a computer equipped with graphics processor unit (GPU) and the one with computer cluster including a number of computers. This chapter firstly introduces the basic ideas of GPU acceleration (e.g., the compute unified device architecture (CUDA) created by NVIDIA™) and the computer cluster framework (e.g., the MapReduce framework), then gives some specified examples of them. When training an EKF-based Elman network, the inversion operation of a Jacobian matrix is the most time-consuming procedure; a parallel computing strategy for such an operation is therefore proposed by using the CUDA-based GPU acceleration. Besides, with regard to the LSSVM modeling, a CUDA-based parallel PSO is then introduced for its hyper-parameters optimization. As for the computer cluster version, we design a parallelized EKF based on ESN by using MapReduce framework for acceleration. At the end, we also present a series of experimental analysis by using the practical energy data in steel industry to validate the performance of the accelerating approaches.

Based on the results of a number of different forecasting modes introduced in the previous chapters, this chapter provides a practical case study related to the optimal scheduling for energy system in steel industry based on the prediction outcomes. As for the by-product gas scheduling problem, a two-stage scheduling method is introduced here. On the prediction stage, the states of the optimized objectives, the consumption of the outsourcing natural gas and oil, the power generation, and the gas holder levels are forecasted by using the previous data-driven learning methods. On the optimal scheduling stage, a rolling optimization procedure is performed by employing the predicted results. More typically, with respect to the scheduling task for the oxygen/nitrogen system in steel industry, a similar two-stage method is also developed, in which a granular-computing (GrC)-based prediction model is firstly established on the stage of a long-term prediction, and the scheduling solution is also optimized later. Furthermore, the results of the scheduling system applications also indicate the effectiveness of the real-time prediction and scheduling optimization.