Pattern Recognition and Computational Intelligence Techniques Using Matlab

Dimensionality reduction technique involves finding out the transformation matrix that maps from the random vector in the higher dimensional space to the lower dimensional space. This is obtained by identifying the orthonormal basis using PCA, LDA, KLDA, and ICA. In PCA, the basis vectors are identified in the direction of the maximum variance of the data. PCA doesn’t care about the class index associated with the training vectors. This is circumvented using LDA. In this case, the distance between the vectors within the class is minimized and the distance between the centroids of various classes is made apart. After subjected to PCA, the data are made uncorrelated. These are further made independent by projecting the data using ICA basis. In almost all the techniques in pattern recognition techniques, the individual classes are assumed as Gaussian distributed. But in real-time applications, data aren’t Gaussian distributed. Thus the Gaussianity of the data can be minimizing the absolute of the kurtosis measured using the given data or by maximizing the neg entropy of the given data.

Linear classifier techniques involve identifying the hyperplane that separates between the classes. The vector under test is treated as the one belonging to the particular class k if it is near to centroid of the k ^th class (Nearest Mean). The r vectors nearest to the vector under test is considered. Among r vectors, if the vector under consideration is nearest to maximum number of vectors belonging to the particular (say k ^th class, then the vector under test is declared as the one belonging to the k ^th class (Nearest Neighbour). The Support Vector Machine is used to identify the equation of the hyperplane that partitions two classes. This is achieved by using the vectors mapped to the higher dimensional space without explicit mapping to the higher dimensional space using “kernel trick.” With modification in the objective functions and the constraints, Support Vector Machine is used for regression techniques. The coefficients describing the equation of the hyperplane are assumed as Gaussian distributed with nonidentical diagonal co-variance matrix and are estimated using Relevance vector machine (RSVM). This helps to get sparse coefficients. RSVM is also used for the classification technique.

The set of input vectors x ₁ ⋯ x _N (outcome of the random vector) influence the system to obtain the corresponding set of output vectors t ₁ ⋯ t _N. The requirement is to estimate the outcome of the random vector t corresponding to the arbitrary input vector x. The MMSE solution is E(t∕x). Usually this problem is solved using parametric approach with linear model as t _n = y(x _n, w) + 𝜖, where y(x _n) = w ^T x _n and 𝜖 is noise vector with zero mean and co-variance matrix \(\frac {1}{\beta }I\). The w is estimated by solving the least square problem, maximizing the likelihood function and by using the Bayes approach. It is observed that maximum likelihood solution and the least square problem solution end up with identical solution if the noise is assumed as Gaussian distributed. The regularization technique is used to avoid over-fitting problem in estimating the parameter w. In the case of Bayes technique, the prior density function of the random vector w is assumed as Gaussian distributed with mean vector zero and co-variance matrix \(\frac {1}{\alpha }I\). It is also observed that Bayes technique and the regularization techniques are identical when regularization factor \(\lambda =\frac {\alpha }{\beta }\). The α and β are estimated iteratively by maximizing the posterior distribution for α and β given the training set data (evidence function).

Let p(c _k) be the prior probability of the class k. Also p(c _k∕x) be the posterior probability of the class c _k given x. The class conditional density function is given as p(x∕c _k). We assign the test vector x belonging to the class r as \(arg_{r=1}^{r=K} max p(c_{r}/\mathbf {x})\). Probabilistic approach for the classification technique is broadly classified into (a) probabilistic generative model approach and (b) probabilistic discriminative model approach. In the case of probabilistic generative model, the parametric model of p(x∕c _k) is considered. In the case of probabilistic discriminative model, the parametric model of p(c _k∕x) is considered.

The algorithms are formulated based on the inspiration of the natural behavior of the environmental components. The Particle Swarm Optimization (PSO) is inspired from the birds’ behavior on how they are able to reach the destination. ANT colony optimization is structured from the observation of the ant’s movement in the particular path (based on the concentration of the pheromone secretion along the path). The SEOA (Social Emotional Optimization Algorithm) and SELA (Social Evolutionary learning algorithm) are based on the human behavior model in the society. The other algorithms include genetic algorithm, bacterial foraging, and simulated annealing. Similarly the mathematical model of the Artificial Neural Network is formulated based on the study on the functions of the neurons in the brain. The ANN models like fully connected feed-forward network, convolutional network, Generative Adversarial Network, multi-class model network, auto encoder network, and Long–Short Term Memory (LSTM) network are discussed in this chapter.

Given the few experimental sample observations, we are interested in testing whether they are the outcomes of the random variable X that follows the typical distribution function. For instance, if we observe the percentage of success obtained by the typical constructed classifier by performing repeated experiments. We would like to know whether they are the outcomes of the random variable X (Gaussian distributed) with mean (average mark) greater than the typical value μ (say). This is needed to conclude the statement “The average POS obtained using the constructed classifier exceeds μ with the identified confident level.” The confident level is obtained using statistical mean test. Similarly, the statistical mean tests are needed to show the statistical evidence to claim the better performance of the proposed classifier. Statistical variance tests are also used to test the consistence of the performance of the typical classifier. The performance of the k classifiers is tested with multiple data sets. We need to check whether the performance of the classifier depends upon the data set chosen. Suppose we collect the features from the data and are subjected to classification. We would like to know which feature is more responsible for classification. This is done using ANOVA test (analysis of variance test). These are known as feature selection technique. In place of mean test, median test is performed using signed rank values. This chapter gives the introduction about how to perform the basic statistical test like mean test, variance test, Proportion test, ANOVA, Wilcoxon/Mann–Whitney test, and Kruskal–Wallis test.