Computational Intelligence Paradigms

This chapter presents an introduction to computational intelligence (CI) paradigms. A number of CI definitions are first presented to provide a general concept of this new, innovative computing field. The main constituents of CI, which include artificial neural networks, fuzzy systems, and evolutionary algorithms, are explained. In addition, different hybrid CI models arisen from synergy of neural, fuzzy, and evolutionary computational paradigms are discussed.

The agenda of fuzzy neurocomputing focuses on the development of artifacts that are both adaptable (so any learning pursuits could be carried out in an efficient manner) and interpretable (so that the results are easily understood by the user or designer). The logic is the language of interpretable constructs. Neural architectures offer a flexible and convenient setting for learning. The study conveys a message that a suitable combination of logic incorporated into the structure of a specialized neuron leads to interpretable and elastic processing units one can refer to as fuzzy neurons. We investigate the main categories of such neurons and elaborate on the ensuing topologies of the networks emphasizing a remarkably rich landscape of logic architectures associated with the use of the logic neurons.

In this chapter, we describe a new data-driven transformation that facilitates many data mining, interpretation, and analysis tasks. This approach, called MembershipMap, strives to granulate and extract the underlying sub-concepts of each raw attribute. The orthogonal union of these sub-concepts are then used to define a new membership space. The sub-concept soft labels of each point in the original space determine the position of that point in the new space. Since sub-concept labels are prone to uncertainty inherent in the original data and in the initial extraction process, a combination of labeling schemes that are based on different measures of uncertainty will be presented. In particular, we introduce the CrispMap, the FuzzyMap, and the PossibilisticMap. We outline the advantages and disadvantages of each mapping scheme, and we show that the three transformed spaces are complementary. We also show that in addition to improving the performance of clustering by taking advantage of the richer information content, the MembershipMap can be used as a flexible pre-processing tool to support such tasks as: sampling, data cleaning, and outlier detection.

Since its inception in 1992, the fuzzy ARTMAP (FAM) neural network (NN) has attracted researchers’ attention as a fast, accurate, off and online pattern classifier. Since then, many studies have explored different issues concerning FAM optimization, training and evaluation, e.g., model sensitivity to parameters, ordering strategy for the presentation of the training patterns, training method and method of predicting the classification accuracy. Other studies have suggested variants to FAM to improve its generalization capability or overcome the prime limitation of the model, which is category proliferation (i.e., model complexity that increases with data complexity). Category proliferation is pronounced in problems that are noisy or contain a large degree of class statistical overlapping. In many investigations, FAM was improved by incorporating elements of optimization theory, Bayes’ decision theory, evolutionary learning, and cluster analysis. Due to its appealing characteristics, FAM and its variants have been applied extensively and successfully to real-world classification problems. Numerous applications were reported in, for example, the processing of signals from different sources, images, speech, and text; recognition of speakers, image objects, handwritten, and genetic abnormalities; and medical and fault diagnoses. When compared to other state-of-the-art machine learning classifiers, FAM and its variants showed superior speed and ease of training, and in most cases they delivered comparable classification accuracy.

Many real-life data problems require effective classification algorithms able to model structural dependencies between multiple labels and to perform classification in a multivariate setting, i.e. such that complex, non-scalar predictions must be produced in correspondence to input vectors. Examples of these tasks range from natural language parsing to speech recognition, machine translation, image segmentation, handwritten character recognition or gene prediction.

Recently many algorithms have been developed in this direction in the machine learning community. They are commonly referred as structured output learning approaches. The main idea behind them is to produce an effective and flexible representation of the data exploiting general dependencies between labels. It has been shown that in many applications structured prediction methods outperform models that do not directly represent correlation between inputs and output labels.

Among the variety of the approaches developed in last few years, in particular large margin methods deserve attention since they have proved to be successful in several tasks. These techniques are based on the smart integration between Support Vector Machines (SVMs) and probabilistic graphical models (PGMs), so they combine the ability to learn in high dimensional feature spaces typical of kernel methods with the algorithmic efficiency and the flexibility in representing data inherited by PGMs.

In this paper we review some of the most recent large margin methods summarizing the main theoretical results, addressing some important computational issues, and presenting the most successful applications. Specifically, we show results in the context of biological sequence alignment and for sequence labeling and parsing in the natural language processing field. We finally discuss some of the main challenges in this new and promising research field.

Multi-layer perceptrons (MLP) make powerful classifiers that may provide superior performance compared with other classifiers, but are often criticized for the number of free parameters. Most commonly, parameters are set with the help of either a validation set or cross-validation techniques, but there is no guarantee that a pseudo-test set is representative. Further difficulties with MLPs include long training times and local minima. In this chapter, an ensemble of MLP classifiers is proposed to solve these problems. Parameter selection for optimal performance is performed using measures that correlate well with generalisation error.

In this chapter, we deal with functional principal points and functional cluster analysis. The k principal points [4] are defined as the set of k points which minimizes the sum of expected squared distances from every points in the distribution to the nearest points of the set, and are mathematically equivalent to centers of gravity by k-means clustering [3]. The concept of principal points can be extended for functional data analysis [16]. We call the extended principal points functional principal points.

We consider the problem of partitioning a data set of n data objects into c homogeneous subsets or clusters (that is, data objects in the same subset should be similar to each other) with constraints on the number of data per cluster. The proposed techniques can be used for various purposes. If a set of items, jobs or customers has to be distributed among a limited number of resources and the workload for each resource shall be balanced, clusters of approximately the same size would be needed. If the resources have different capacities, then clusters of the corresponding sizes need to be found. We also extend our approach to avoid extremely small or large clusters in standard cluster analysis. Another extension offers a measure for comparing different prototype-based clustring results.

Regression analysis is a well known and a widely used technique in multivariate data analysis. The efficiency of it is extensively recognized. Recently, several proposed regression models have exploited the spatial classification structure of data. The purpose of this inclusion of the spatial classification structure is to set a heterogeneous data structure to homogeneous structure in order to adjust the heterogeneous data structure to a single regression model. One such method is a blocking regression model. However, the ordinal blocking regression model can not reflect the complex classification structure satisfactorily. Therefore, the fuzzy blocking regression models are offered to represent the classification structure by using fuzzy clustering methods. This chapter’s focus is on the methods of the fuzzy clustering based blocking regression models. They are extensions of the conventional blocking regression model.

Environment perception is one of the most challenging and underlying task which allows a mobile robot to perceive obstacles, landmarks and extract useful information to navigate safely. In this sense, classification techniques applied to sensor data may enhance the way mobile robots sense their surroundings. Amongst several techniques to classify data and to extract relevant information from the environment, Support Vector Machines (SVM) have demonstrated promising results, being used in several practical approaches. This chapter presents the core theory of SVM, and applications in two different scopes: using Lidar (Light Detection and Ranging) to label specific places, and vision-based human detection aided by Lidar.

We review algorithms which have been proposed that identify linkage by applying perturbations, by building a probabilistic model of promising strings, and a recombination of the both of the above.

We also introduce a context-dependent crossover that can utilize overlapping linkage information in a sophisticated manner. By employing linkage identification techniques with context dependent crossover, we can solve practical real-world application problems that usually have complex problem structures without knowing them before optimization.