Fault diagnosis based on imbalance modified kernel Fisher discriminant analysis

Abstract

Process data with imbalance class distribution has brought a significant drawback to most existing pattern recognition based fault diagnosis algorithms, which have assumed that the process data have an equal misclassification cost and relatively balanced class distribution. The frequent occurrence of the imbalance problem in real industrial process indicates the need for extra research efforts. In this paper, three novel imbalance modified kernel Fisher discriminant analysis (IM-KFDA) approaches are proposed to handle this problem. Two sample-level approaches, over-sampling KFDA and under-sampling KFDA, are presented along with proper stochastic sampling strategies. One algorithm-level approach, inductive bias KFDA, is also proposed with incorporating a novel regular weighted matrix (RWM) into the minimum Euclid distance based pattern classification rule. To improve the fault diagnosis performance, model updating modes for the sample-level and algorithm-level approaches are described, respectively. A simulation case study of Tennessee Eastman (TE) process is conducted to evaluate the proposed fault diagnosis approaches.

Introduction

Multivariable statistical process control (MSPC) which uses the statistic projection technique to extract and select key process information from massive process data has gained great attentions in last decades (MacGregor and Kourti, 1995, Akbaryan and Bishnoi, 2001, Qin, 2003, Wang et al., 2005, Zhang and Dudzic, 2006, Chew et al., 2007, AlGhazzawi and Lennox, 2008). MSPC methods, such as principal component analysis (PCA), independent component analysis, partial least square and their developed algorithms have been widely applied to fault diagnosis in chemical industry (Chen and Liu, 2002, Lee et al., 2004a, Lee et al., 2004b, Lee et al., 2004c, Chen and Chen, 2006, Lieftucht et al., 2006, Zhang and Qin, 2007)_. However, the MSPC methods mainly focus on fault detection. MSPC may not function well for fault isolation and diagnosis since the causalities among different process variables are lost after the MSPC transformation (Ge and Song, 2007).

Since the fault diagnosis problem can be considered as a multi-class classification problem, pattern recognition methods with good generalization and accurate performances have been proposed in recent years. Choi et al. (2004) proposed a fault detection and isolation methodology based on principal component analysis–Gaussian mixture model and discriminant analysis–Gaussian mixture model. Fisher discriminant analysis (FDA) has been proved to outperform PCA in discriminating different classes, in the aspect that PCA aims at reconstruction instead of classification, while FDA seeks directions that are optimal for discrimination (Chiang et al., 2001). However, FDA is a linear method. In order to handle the nonlinear problem of process data, kernel FDA (KFDA) is proposed by Mika et al. (1999). KFDA performs a nonlinear discriminant through kernel feature space mapping before FDA method is used. Yang et al. (2004) made an in-depth analysis on the KFDA algorithm, and reformulated it as a two-step procedure: kernel principal component analysis (KPCA) plus FDA. Recently, KFDA has been proved superior to PCA and FDA in fault diagnosis, which makes it a promising way for process monitoring (Cho, 2007a, Cho, 2007b).

However, general KFDA does not consider the imbalance data problem. Imbalance data problem is characterized by having more instances of some patterns than others. Classifiers generally perform poorly on imbalance datasets because it pays more attention to majority patterns than to the minority ones. The key problem of imbalance is that discriminant for pending samples will be apt to the majority class. In many real-world domains classifying the minority pattern is more important and is of primary interest. Examples abound and include diagnoses of rare medical conditions such as thyroid diseases (Murphy and Aha, 1994), detection of fraudulent telephone calls (Fawcett and Provost, 1997), oil spills in satellite images of the sea surface (Kubat et al., 1998), text categorization (Mladenic and Grobelnik, 1999) and so on. In real industrial processes, a large number of normal data can be obtained easily since industrial processes are usually under control. The faulty samples are quite rare compared with the normal case, since the operation units will be shut down if any abnormality is detected. As the collection of representative faulty samples often requires a lot of effort and expensive resources which may not always be available, the amount of faulty data will be much smaller.

In process monitoring domain, normal process is the majority pattern, while faulty patterns are the minority ones. Moreover, there will be different number of samples for each fault, which makes fault diagnosis a multi-class identification task with imbalance problem. Since the learning concern is the identification of the minority ones, a favorable fault diagnosis model is the one which provides higher identification rates of the faulty patterns, while the rate of the normal pattern recognized as abnormity is comparably stable. Meanwhile, as industrial processes run, more and more real-time operation data are collected, so the fault diagnosis model should be updated properly. Therefore, it is necessary and important to propose imbalance modified KFDA (IM-KFDA) approaches with updating modes to improve the performance of fault diagnosis. In this paper, imbalance problem is introduced into the area of process monitoring, and it is extended from two-class to multi-class. Three IM-KFDA approaches, two sample-level approaches named over-sampling KFDA and under-sampling KFDA and an algorithm-level approach named inductive bias KFDA are proposed. A further contribution is that updating modes of proposed approaches are suggested.

The rest of the paper is organized as follows. KFDA is outlined as the preliminary in Section 2. In Section 3, three novel IM-KFDA approaches are presented first, and then remarks of these approaches and the update modes are described. Section 4 gives the simulations of Tennessee Eastman (TE) process. Finally, Section 5 concludes with an assessment of the proposed approaches and points to further research issues.