Multiple Fuzzy Classification Systems

Classifying objects described by a set of their numerical features is one of the basic tasks of pattern recognition and data mining. It is applied in many domains such as medicine, economics, fraud or fault detection, etc. Designing better classifiers is a subject of sustained research and through the years many classification methods were developed [2, 3, 4, 7, 11, 19, 13, 29, 30, 31]. The most popular ones are those based on i.a. neural networks, nearest-neighbour, decision trees and support vector machines. There does not exist one best classification method. We can choose only one, best classifier for a given task using laborious trial and error method but in this way we miss collective knowledge of discarded classifiers. To address this problem many methods for automated aggregation of classifiers trained for the same task have been developed [12, 14, 24]. These methods demonstrate that the classification accuracy nearly always improves after combining different classification methods or classifiers trained with different datasets. Nowadays combining single classifiers into larger ensembles is an established method for improving the accuracy. Of course, the obvious improvement is bound up with increased storage space (memory) and computational burden. However this trade-off is easy to accept with modern computers.

Fuzzy logic since its conception in 1965 [11] has been used in various areas of science, economics, manufacturing, medicine etc [1, 2, 3, 4, 7, 8, 9]. It constitutes, along with other methods such as neural networks or evolutionary algorithms, the idea of soft computing [5]. Fuzzy sets used in fuzzy rules can be a tool to model linguistic values like “small” or “high” [12]. This chapter presents basic definitions of fuzzy logic and fuzzy systems based on [10].

Combining single classifiers into larger ensembles is an established method for improving the accuracy. Of course, the obvious improvement is bound up with increased storage space (memory) and computational burden. However this trade-off is easy to accept with modern computers. Classifiers can be combined at the level of features or data subsets and by the use of different classifiers or different combiners, see Figure 3.1. Popular methods are bagging and boosting which are meta algorithms for learning different classifiers.

This chapter presents the fuzzy relational model. In such model we define all possible connections between input and output linguistic terms [6, 12]. An advantage of this approach is great flexibility of the system. Input and output terms are fully interconnected. Moreover, the connections can be modeled by changing the elements of the relation matrix. The relation matrix can be regarded as a set of elements similar to rule weights in classic fuzzy systems [8, 11]. Relational fuzzy systems are used successfully to e.g. control [4] and classification tasks [1, 21, 23]. In this chapter, relational neuro-fuzzy systems [17, 20] will be used. Such neural network like structures allow to use more scenarios than in the case of ordinary relational structures. For example, we can set fuzzy linguistic values in advance and then fine tune the model mapping by changing relation elements using gradient learning. Gradient learning is an important advantage of relational neuro-fuzzy systems comparing to original fuzzy relational systems. Furthermore, this chapter presents the AdaBoost ensembles of relational neuro-fuzzy classifiers. A serious drawback of fuzzy system boosting ensembles is that such ensembles contain separate rule bases which cannot be directly merged. As systems are separate, we cannot treat fuzzy rules coming from different systems as rules from the same (single) system. The problem is addressed by a novel design of fuzzy systems constituting the ensemble, resulting in normalization of individual rule bases during learning. There were some attempts to combine fuzzy models, e.g. [13] or rough-fuzzy models [9] but none of them solved the problem of multiple rule bases in the ensemble.

The previous chapter described the Mamdani neuro-fuzzy systems which are the most common neuro-fuzzy systems. This chapter presents systems with a fuzzy implication connecting the antecedents and the consequents of fuzzy rules. Such systems are proved to perform better in classification tasks [5].

The Takagi-Sugeno systems (for short, to be denoted TS) are one of the most common fuzzy models. In such systems consequents are functions of inputs. This chapter shows a modification of such models as members of an classifier ensemble. The problem of incapability of merging several rule bases is addressed by a novel design of fuzzy systems constituting the ensemble, resulting in normalization of individual rule bases during learning.

Neuro-fuzzy systems presented so far in the book are not able to cope with missing data. Generally, there are two ways to solve the problem of missing data: