Advances in Neuro-Information Processing

This paper presents a distance invariance method to construct the low dimension manifold that preserves the neighborhood topological relations among data patterns. This manifold can display close relationships among patterns.

This paper presents a non-linear classifier based on a region oriented approach for interval data. Each example of the learning set is described by a interval feature vector. Concerning the learning step, each class is described by a region (or a set of regions) in

$\Re^{p}$

defined by hypercube of the objects belonging to this class. In the allocation step, the assignment of a new object to a class is based on a suitable

L

r

Minkowski distance between intervals. Experiments with two synthetic interval data sets have been performed in order to show the usefulness of this classifier. The prediction accuracy (error rate) of the proposed classifier is calculated through a Monte Carlo simulation method with 100 replications.

The aim of Symbolic Data Analysis (SDA) is to provide a set of techniques to summarize large data sets into smaller ones called symbolic data tables. This paper considers a kind of symbolic data called Interval-Valued Data (IVD) which stores data intrinsic variability and/or uncertainty from the original data set. Recent works have been proposed to fit the classic linear regression model to symbolic data. However, those works do not consider the presence of symbolic data outliers. Generally, most specialists treat outliers as errors and discard them. Nevertheless, a single interval-data outlier holds significant information which should not be discarded or ignored. This work introduces a prediction method for IVD based on the symmetrical linear regression (SLR) analysis whose response model is less susceptible to the IVD outliers. The model considers a symmetrical distribution for error which allows to the model possibility of applying regular statistical hypothesis tests.

The paper proposes new neuron model with an aggregation function based on Generalized harmonic mean of the inputs. Information-maximization approach has been used for training the new neuron model. The paper focuss on illustrating the efficiency of the proposed neuron model for blind source separation. It has been shown on various generated mixtures (for blind source separation) that the new neuron model performs far superior compared to the conventional neuron model.

The paper aims at training multilayer perceptron with different new error measures. Traditionally in MLP, Least Mean Square error (LMSE) based on Euclidean distance measure is used. However Euclidean distance measure is optimal distance metric for Gaussian distribution. Often in real life situations, data does not follow the Gaussian distribution. In such a case, one has to resort to error measures other than LMSE which are based on different distance metrics [7,8]. It has been illustrated in this paper on wide variety of well known time series prediction problems that generalized geometric and harmonic error measures perform better than LMSE for wide class of problems.

Feature selection has improved the performance of text clustering. Global feature selection tries to identify a single subset of features which are relevant to all clusters. However, the clustering process might be improved by considering different subsets of features for locally describing each cluster. In this work, we introduce the method ZOOM-IN to perform local feature selection for partitional hierarchical clustering of text collections. The proposed method explores the diversity of clusters generated by the hierarchical algorithm, selecting a variable number of features according to the size of the clusters. Experiments were conducted on Reuters collection, by evaluating the bisecting K-means algorithm with both global and local approaches to feature selection. The results of the experiments showed an improvement in clustering performance with the use of the proposed local method.

In text classification, one key problem is its inherent dichotomy of polysemy and synonym; the other problem is the insufficient usage of abundant useful, but unlabeled text documents. Targeting on solving these problems, we incorporate a sprinkling Latent Semantic Indexing (LSI) with background knowledge for text classification. The motivation comes from: 1) LSI is a popular technique for information retrieval and it also succeeds in text classification solving the problem of polysemy and synonym; 2) By fusing the sprinkling terms and unlabeled terms, our method not only considers the class relationship, but also explores the unlabeled information. Finally, experimental results on text documents demonstrate our proposed method benefits for improving the classification performance.

In this paper we present a comparison of multiple cluster algorithms and their suitability for clustering text data. The clustering is based on similarities only, employing the Kolmogorov complexity as a similiarity measure. This motivates the set of considered clustering algorithms which take into account the similarity between objects exclusively. Compared cluster algorithms are Median kMeans, Median Neural Gas, Relational Neural Gas, Spectral Clustering and Affinity Propagation.

Internet literature queries return a long lists of citations, ordered according to their relevance or date. Query results may also be represented using Visual Language that takes as input a small set of semantically related concepts present in the citations. First experiments with such visualization have been done using PubMed neuronal plasticity citations with manually created semantic graphs. Here neurocognitive inspirations are used to create similar semantic graphs in an automated fashion. This way a long list of citations is changed to small semantic graphs that allow semi-automated query refinement and literature based discovery.

This study concentrates on adapting the user profile model (UPM) based on individual’s continuous interaction with preferred search engines. UPM re-ranks the retrieved information from World Wide Web (WWW) to provide effective personalization for a given search query. The temporal adaptation of UPM is considered as a one-to-one socio-interaction between the dynamics of WWW and cognitive information seeking behavior of the user. The dynamics of WWW and consensus relevant ranking of information is a collaborative effect of inter-connected users, which makes it difficult to analyze in-parts. The proposed system is named as Search-in-Synchrony and a preliminary study is done on user group with background in computational neuroscience. Human-agent interaction (HAI) can implicitly model these dynamics. Hence, a primary attempt to converge the two fields is highlighted - HAI and statistically learned UPM to incorporate cognitive abilities to search agents.

One of the simplest creative act is the invention of a new word that captures some characteristics of objects or processes, for example industrial or software products, activity of companies, or the main topic of web pages. Neurocognitive mechanisms responsible for this process are partially understood and in a simplified form may be implemented in a computational model. An algorithm based on such inspirations is described and tested, generating many interesting novel names associated with a set of keywords.

In this paper we study personal credit scoring using several machine learning algorithms: Multilayer Perceptron, Logistic Regression, Support Vector Machines, AddaboostM1 and Hidden Layer Learning Vector Quantization. The scoring models were tested on a large dataset from a Portuguese bank. Results are benchmarked against traditional methods under consideration for commercial applications. A measure of the usefulness of a scoring model is presented and we show that HLVQ-C is the most accurate model.

This paper proposes the use of behavior-based control architecture and investigates on some techniques inspired by Nature- a combination of reinforcement and supervised learning algorithms to accomplish the sub-goals of a mission of building adaptive controller. The approach iteratively improves its control strategies by exploiting only relevant parts of action and is able to learn completely in on-line mode. To illustrate this, it has been applied to non-linear, non-stationary control task: Cart-Pole balancing. The results demonstrate that our hybrid approach is adaptable and can significantly improve the performance of TD methods while speed up learning process.

Efficient retrieval of information with regards to its meaning and content is an important problem in data mining systems for the creation, management and querying of very large information databases existing in the World Wide Web. In this paper we deal with the main aspect of the problem of content based retrieval, namely, with the problem of document classification, outlining a novel improved and systematic approach to it’s solution. We present a document classification system for non-domain specific content based on the learning and generalization capabilities mainly of SVM neural networks. The main contribution of this paper lies on the feature extraction methodology which, first, involves word semantic categories and not raw words as other rival approaches. As a consequence of coping with the problem of dimensionality reduction, the proposed approach introduces a novel higher order approach for document categorization feature extraction by considering word semantic categories higher order correlation analysis, both two and three dimensional, based on cooccurrence analysis. The suggested methodology compares favourably to widely accepted, raw word frequency based techniques in a collection of documents concerning the Dewey Decimal Classification (DDC) system. In these comparisons different Multilayer Perceptrons (MLP) algorithms as well as the Support Vector Machine (SVM), the LVQ and the conventional k-NN technique are involved. SVM models seem to outperform all other rival methods in this study.

T. Kohonen and P. Somervuo have shown that self organizing maps (

SOMs

) are not restricted to numerical data. This paper proposes a symbolic measure that is used to implement a string self organizing map based on

SOM

algorithm. Such measure between two strings is a new string. Computation over strings is performed using a priority relationship among symbols, in this case, symbolic measure is able to generate new symbols. A complementary operation is defined in order to apply such measure to

DNA

strands. Finally, an algorithm is proposed in order to be able to implement a string self organizing map. This paper discusses the possibility of defining neural networks to rely on similarity instead of distance and shows examples of such networks for symbol strings.

The problem of increasing the accuracy of geonavigation data being used for the control of the drilling oil-gas well trajectory is considered. The approach to solving the problem based on the distortion and measurement noise filtration with the use of the smoothing neural network is proposed. The generalized algorithm of the smoothing neural network design on the basis of the multilevel regularization is discussed. The peculiarities of the algorithm realization with the use of the offered vector regularization criterion of network parameters ranking is considered. The example of smoothing the geonavigation data on the basis of designed RBF network is considered.

The technology of artificial neural networks has been proven to be well-suited for the mining of useful information from vast quantities of data. Most work focuses on the pursuit of accurate results but neglects the reasoning process. This “black-box” feature is the main drawback of artificial neural network mining models. However, the practicability of many mining tasks relies not only on accuracy, reliability and tolerance but also on the explanatory ability. Rule extraction is a technique for extracting symbolic rules from artificial neural networks and can therefore transfer the features of artificial neural networks from “black-box” into “white-box”. This paper proposes a novel approach in which knowledge is extracted, in the forms of symbolic rules, from one-dimensional self-organizing maps. Three data sets are used in this paper. The experimental results demonstrate that this proposed approach not only equips the self-organizing map with an explanatory ability based on symbolic rules, but also provides a robust generalized ability for unseen data sets.

In general, wavelet neural networks have a problem on the curse of dimensionality, i.e. the number of hidden units to be required are exponentially rose with increasing an input dimension. To solve the above problem, a wavelet neural network incorporating a local linear model has already been proposed. On their network design, however, the number of hidden units is empirically determined and fixed during learning. In the present paper, a design method based on Bayesian method is proposed for the local linear wavelet neural network. The performance of the proposed method is evaluated through computer simulation.

Expectation Propagation (EP) extends belief propagation by approximating messages with expectations of statistics, in which users can choose the statistics. In this paper, we discuss how a choice of statistics affects accuracy of EP’s estimates. We approximate estimation error of EP via perturbation analysis based on information geometry. By comparing the approximated estimation error, we show that adding statistics does not necessarily improve the accuracy of EP. A numerical example confirms validity of our analytical results.

In this paper, we propose a new method to extract explicit features for competitive learning as well as self-organizing maps. The method aims to enhance final internal representations by conventional methods. We first train networks by conventional methods and compute enhanced information by focusing upon some specific input units or variables. Because we focus upon some specific inputs and activate competitive units, this enhancement is called

partial enhancement

. Then, networks are retrained to imitate the states obtained by partial enhancement. Final representations obtained by this retraining generate representations influenced by these specific variables. We applied the method to the famous Iris problem and the air pollution problem. In both problems, partial enhancement methods could produce clearer feature maps, superior to those obtained by self-organizing maps.

In this paper, we propose structural enhanced information for detecting main features in input patterns. In structural enhanced information, three types of enhanced information can be differentiated, that is, the first-, the second- and the third-order enhanced information. The first-order information is related to the enhancement of competitive units themselves through some elements in a network, and the second-order information is dependent upon the enhancement of competitive units with input patterns. Then, the third-order information is obtained by subtracting the effect of the first-order information from the second-order information. Thus, the third-order information more explicitly represents information on input patterns. With this structural enhanced information, we can estimate more detailed features in input patterns. We applied the method to the well-known Iris problem. In both problems, we succeeded in extracting detailed and important features especially by using the third-order information.

A slightly simplified version of the Spike Response Model SRM

0

of a spiking neuron is tailored to gradient learning. In particular, the evolution of spike trains along the weight and delay parameter trajectories is made perfectly smooth. For this model a back-propagation-like learning rule is derived which propagates the error also along the time axis. This approach overcomes the difficulties with the discontinuous-in-time nature of spiking neurons, which encounter previous gradient learning algorithms (e.g.

SpikeProp

). The new algorithm can naturally cope with multiple spikes and preliminary experiments confirm the smoothness of spike creation/deletion process.

In this article, we proposed training methods for improving the generalization capability of a learning machine that is defined by a weighted sum of many fixed basis functions and is used as a nonparametric regression method. In the basis of the proposed methods, vectors of basis function outputs are orthogonalized and coefficients of the orthogonal vectors are estimated instead of weights. The coefficients are set to zero if those are less than predetermined threshold levels which are theoretically reasonable under the assumption of Gaussian noise. We then obtain a resulting weight vector by transforming the thresholded coefficients. When we apply an eigen-decomposition based orthogonalization procedure, it yields shrinkage estimators of weights. If we employ the Gram-Schmidt orthogonalization scheme, it produces a sparse representation of a target function in terms of basis functions. A simple numerical experiment showed the validity of the proposed methods by comparing with other alternative methods including the leave-one-out cross validation.

We analyzed the equilibrium states of an Ising spin neural network model in which both spins and interactions evolve simultaneously over time. The interactions are Mexican-hat-type, which are used for lateral inhibition models. The model shows a bump activity, which is the locally activated network state. The time-dependent interactions are driven by Langevin noise and Hebbian learning. The analysis results reveal that Hebbian learning expands the bistable regions of the ferromagnetic and local excitation phases.

In this paper, we propose a Divided Chaotic Associative Memory for Successive Learning (DCAMSL). The proposed model is based on the Improved Chaotic Associative Memory for Successive Learning (ICAMSL) and the Divided Chaotic Associative Memory for Successive Learning using Internal Patterns (DCAMSL-IP) which were proposed in order to improve the storage capacity. In most of the conventional neural network models, the learning process and the recall process are divided, and therefore they need all information to learning in advance. However, in the real world, it is very difficult to get all information to learn in advance. So we need the model whose learning and recall processes are not divided. As such model, although some models have been proposed, their storage capacity is small. In the proposed DCAMSL, the learning process and the recall process are not divided and its storage capacity is larger than that of the conventional ICAMSL.

In this paper, we propose a reinforcement learning method using Kohonen Feature Map Associative Memory with Refractoriness based on Area Representation. The proposed method is based on the actor-critic method, and the actor is realized by the Kohonen Feature Map Associative Memory with Refractoriness based on Area Representation. The Kohonen Feature Map Associative Memory with Refractoriness based on Area Representation is based on the self-organizing feature map, and it can realize successive learning and one-to-many associations. Moreover, it has robustness for noisy input and damaged neurons because it is based on the area representation. The proposed method makes use of this property in order to realize the learning during the practice of task. We carried out a series of computer experiments, and confirmed the effectiveness of the proposed method in path-finding problem.

In this paper, we propose a learning method called

Corrected Error Backpropagation

which maximizes the corrected log-likelihood which works like Akaike Information Criterion. For the purpose of maximizing the corrected log-likelihood, we introduce temperature parameter for the corrected log-likelihood. This paper also shows an optimal scheduling of the temperature parameter. Applying to our method to a linear regression model on the Boston house price estimation problem and multi layered perceptrons on the DELVE datasets, the method gives good results.

Hybrid systems consisting of model-based and model-free systems will be engaged in the behavior/dialog control systems of future robots/agents to satisfy several user’s requirements and simultaneously cope with diverse and unexpected situations. We have constructed a modular neural network model based on reinforcement learning for model-free learning. For an effective hybrid system, the model-free learning system should be aware of the current targets. This can be achieved by automatically acquiring a list of important sequential events. We propose a basic mechanism that can automatically acquire the list of sequential events with confidence measures reflecting current situations.

Constructive cascade algorithms are powerful methods for training feedforward neural networks with automation of the task of specifying the size and topology of network to use. A series of empirical studies were performed to examine the effect of imposing constraints on constructive cascade neural network architectures. Building

a priori

knowledge of the task into the network gives better generalisation performance. We introduce our Local Feature Constructive Cascade (LoCC) and Symmetry Local Feature Constructive Cascade (SymLoCC) algorithms, and show them to have good generalisation and network construction properties on face recognition tasks.

Developing neural controllers for autonomous robotics is a tedious task as the desired state trajectory of the robot is very often not known in advance. This led to the large success of evolutionary algorithm in this field. In this paper we introduce SOMA (Synchronized Oriented Mutations Algorithm), which presents an alternative for rapidly minimizing the parameters characterizing a given individual. SOMA is characterized by its easy implementation and its flexibility: it can use any continuous fitness function and be applied to optimize neural network of diverse topologies using any kind of activation functions. Contrary to evolutionary approach, it is applied on a single individual rather than on a population. Because the procedure is very fast, it allows for rapid screening and selection of good candidates. In this paper, the efficiency of SOMA at training ordered connection feed forward networks on function modeling problem, classification problem and robotic controllers is investigated.

The values selected for the free parameters of Artificial Neural Networks usually have a high impact on their performance. As a result, several works investigate the use of optimization techniques, mainly metaheuristics, for the selection of values related to the network architecture, like number of hidden neurons, number of hidden layers, activation function, and to the learning algorithm, like learning rate, momentum coefficient, etc. A large number of these works use Genetic Algorithms for parameter optimization. Lately, other bioinspired optimization techniques, like Ant Colony optimization, Particle Swarm Optimization, among others, have been successfully used. Although bioinspired optimization techniques have been successfully adopted to tune neural networks parameter values, little is known about the relation between the quality of the estimates of the fitness of a solution used during the search process and the quality of the solution obtained by the optimization method. In this paper, we describe an empirical study on this issue. To focus our analysis, we restricted the datasets to the domain of gene expression analysis. Our results indicate that, although the computational power saved by using simpler estimation methods can be used to increase the number of solutions tested in the search process, the use of accurate estimates to guide that search is the most important factor to obtain good solutions.

Feature selection (FS) plays a crucial role in machine learning to build a robust model for either learning or classification from a large amount of data. Among feature selection techniques, the Relief algorithm is one of the most common due to its simplicity and effectiveness. The performance of the Relief algorithm, however, could be dramatically affected by the consistency of the data patterns. For instance, Relief-F could become less accurate in the presence of noise. The accuracy would decrease further if an outlier sample was included in the dataset. Therefore, it is very important to select the samples to be included in the dataset carefully. This paper presents an effort to improve the effectiveness of Relief algorithm by filtering samples before selecting features. This method is termed Sample Filtering Relief Algorithm (SFRA). The main idea of this method is to discriminate outlier samples out of the main pattern using self organizing map (SOM) and then proceed with feature selection using the Relief algorithm. We have tested SFRA with a gene expression dataset of

interferon

-

α

(IFN-

α

) response of Hepatitis B patients that contains outlier data. SFRA could successfully remove outlier samples that have been verified by visual inspection by experts. Also, it has better accuracy in separating the relevant and irrelevant features than other feature selection methods considered.

In this paper, we propose a simple but powerful method to visualize connection weights by SOM. The conventional SOM has been well established and extensively used to visualize complex data. There have been a number of methods to visualize final connection weights. However, even sophisticated visualization techniques may be ineffective in dealing with ambiguous connection weights due to the complexity of the data set. To cope with this problem, we retrain a network with connection weights obtained by SOM. At this time, we do not optimize networks in terms of errors but we train networks to enhance the characteristics of connection weights at the price of optimality. This enhancement can be realized by smaller Gaussian width. Though these smaller Gaussian widths are not optimal ones in terms of errors, it may give some insights into the characteristics of connection weights. We applied the method to the famous Iris problem and a classification problem for OECD countries. In both problems, we can obtain U-matrices with more explicit boundaries for easy interpretation.

Missing data is a common issue in almost every real-world dataset. In this work, we investigate the relative merits of applying two imputation schemes for coping with this problem while designing radial basis function network classifiers, which show sensitiveness to the existence of missing values. Whereas the first scheme centers upon the

k

-nearest neighbor algorithm and has been deployed with success in other supervised/unsupervised learning contexts, the second is based on a simple genetic algorithm model and has not been fully explored so far.

Echo State neural networks are a special case of recurrent neural networks. The most important part of Echo State neural networks is so called ”dynamic reservoir”. Echo State neural networks use dynamics of this massive and randomly initialized dynamic reservoir to extract interesting properties of incoming sequences. A standard training of these neural networks uses pseudo inverse matrix for one-step learning of weights from hidden to output neurons. In this approach, we have merged this dynamic reservoir with standard feedforward neural network, with a goal to achieve greater prediction ability. This approach was tested in laser fluctuations and Mackey-Glass time series prediction. The prediction error achieved by this approach was substantially smaller in comparison with prediction error achieved by standard algorithm or time delay neural network with backpropagation algorithm.

Human learners are known to exploit statistical dependencies of language elements such as syllables or words during acquisition and processing. Recent research suggests that underlying computations relate not only to adjacent but also to nonadjacent elements such as subject/verb agreement or tense marking in English. The latter type of computations is more difficult and appears to work under certain conditions, as formulated by the variability hypothesis. We model this finding using a simple recurrent network and show that higher variability of the intervening syllables facilitates the generalization in the continuous stream of 3-syllable words. We also test the network performance in case of more realistic, two intervening syllables and show that only a more complex training algorithm can lead to satisfactory learning of nonadjacent dependencies.

Pulsed Neuron (PN) model was proposed as one of the simplest models working by pulse trains. PN model has a membrane potential to deal with the temporal information, and the calculation process is inexpensive. However, as the output function of PN model is an Unit Step function, PN model cannot directly use the back-propagation (BP) method. It would be possible to solve general pattern recognition problems if the PN model could be trained by the BP method. In this paper, we propose a BP method for multilayer pulsed neural networks. The proposed method uses the duality of PN model, in which the desired output of hidden layer neuron is calculated from output layer neurons’ weights and output. Experimental results show that the multilayer pulsed neural networks can learn and recognize non-linear problems using the proposed method.

One of the main reasons for the slow convergence and the suboptimal generalization results of MLP (Multilayer Perceptrons) based on gradient descent training is the lack of a proper initialization of the weights to be adjusted. Even sophisticated learning procedures are not able to compensate for bad initial values of weights, while good initial guess leads to fast convergence and or better generalization capability even with simple gradient-based error minimization techniques. Although initial weight space in MLPs seems so critical there is no study so far of its properties with regards to which regions lead to solutions or failures concerning generalization and convergence in real world problems. There exist only some preliminary studies for toy problems, like XOR. A data mining approach, based on Self Organizing Feature Maps (SOM), is involved in this paper to demonstrate that a complete analysis of the MLP weight space is possible. This is the main novelty of this paper. The conclusions drawn from this novel application of SOM algorithm in MLP analysis extend significantly previous preliminary results in the literature. MLP initialization procedures are overviewed along with all conclusions so far drawn in the literature and an extensive experimental study on more representative tasks, using our data mining approach, reveals important initial weight space properties of MLPs, extending previous knowledge and literature results.

In the past two decades, the use of the weight decay regularizer for improving the generalization ability of neural networks has been extensively investigated. However, most existing results focus on the fault-free neural networks only. This papers extends the analysis on the generalization ability for networks with multiplicative weight noise. Our analysis result allows us not only to estimate the generalization ability of a faulty network, but also to select a good model from various settings. Simulated experiments are performed to verify theoretical result.

While injecting fault during training has long been demonstrated as an effective method to improve fault tolerance of a neural network, not much theoretical work has been done to explain these results. In this paper, two different node-fault-injection-based on-line learning algorithms, including (1) injecting multinode fault during training and (2) weight decay with injecting multinode fault, are studied. Their almost sure convergence will be proved and thus their corresponding objective functions are deduced.

This paper presents a new approach to

rule extraction

from Support Vector Machines (SVMs). SVMs have been applied successfully in many areas with excellent generalization results; rule extraction can offer

explanation capability

to SVMs. We propose to approximate the SVM classification boundary by solving an optimization problem through sampling and querying followed by boundary searching, rule extraction and post-processing. A theorem and experimental results then indicate that the rules can be used to validate the SVM with high accuracy and very high fidelity.

In this paper, an asbestos detection method from microscope images is proposed. The asbestos particles have different colors in two specific angles of the polarizing plate. Therefore, human examiners use the color information to detect asbestos. To detect the asbestos by computer, we develop the detector based on Support Vector Machine (SVM) of local color features. However, when it is applied to each pixel independently, there are many false positives and negatives because it does not use the relation with neighboring pixels. To take into consideration of the relation with neighboring pixels, Conditional Random Field (CRF) with SVM outputs is used. We confirm that the accuracy of asbestos detection is improved by using the relation with neighboring pixels.

In this paper Gaussian process is applied to linear and nonlinear acoustic echo cancellation. Gaussian process is a kernel method in which predictions to new inputs are made based on the linear combination of kernel functions evaluated at each training data. First order acoustic echo-path is modeled by a linear equation of input data and second order acoustic echo-path is modeled by the second order polynomials. The performance of the cancellation is evaluated by white signal, stationary colored signal, non-stationary colored signal and real speech data. It is shown that more than 70 dB echo cancellation can be acieved within 400 ms.

Asbestos-related illnesses become a nationwide problem in Japan. Now human inspectors check whether asbestos is contained in building material or not. To judge whether the specimen contains asbestos or not, 3,000 particles must be counted from microscope images. This is a major labor-intensive bottleneck. In this paper, we propose an automatic particle counting method for automatic judgement system whether the specimen is hazardous or not. However, the size, shape and color of particles are not constant. Therefore, it is difficult to model the particle class. On the other hand, the non-particle class is not varied much. In addition, the area of non-particles is wider than that of particles. Thus, we use One-Class Support Vector Machine (OCSVM). OCSVM identifies “outlier” from input samples. Namely, we model the non-particle class to detect the particle class as outlier. In experiments, the proposed method gives higher accuracy and smaller number of false positives than a preliminary method of our project.

Protein folding classification is a meaningful step to improve analysis of the whole structures. We have designed committee Support Vector Machines (committee SVMs) and their array (committee SVM array) for the prediction of the folding classes. Learning and test data are amino acid sequences drawn from SCOP (Structure Classification Of Protein database). The classification category is compatible with the SCOP. SVMs and committee SVMs are designed in an one-versus-others style both for chemical data and sliding window patterns (spectrum kernels). This generates the committee SVM array. Classification performances are measured in view of the Receiver Operating Characteristic curves (ROC). Superiority of the committee SVM array to existing prediction methods is obtained through extensive experiments to compute the ROCs.

This paper presents a MLP kernel. It maps all patterns in a class into a single point in the output layer space and maps different classes into different points. These widely separated class points can be used for further classifications. It is a layered feed-forward network. Each layer is trained using the class differences and trained independently layer after layer using a bottom-up construction. The class labels are not used in the training process. It can be used in separating multiple classes.

Although Support Vector Machines (SVMs) have been successfully applied to many problems, they are considered “black box models”. Some methods have been developed to reduce this limitation, among them the FREx_SVM, which extracts fuzzy rules from trained SVMs for multi-class problems. This work deals with an extension to the FREx_SVM method, including a wrapper feature subset selection algorithm for SVMs. The method was evaluated in four benchmark databases. Results show that the proposed extension improves the original FREX_SVM, providing better rule coverage and a lower number of rules, which is a considerable gain in terms of interpretability.

In this paper we present an evaluation of different approaches to cross-language adaptation for Indian languages. We also propose a method for cross-language adaptation of the SVM (support vector machine) based system. The proposed method gives approximately the same performance as pooling, with a reduction in the training time. The adaptation methods such as Bootstrap, MAP (Maximum A Posterior) and MLLR (Maximum Likelihood Linear Regression) have been used for cross-language adaptation in the HMM (hidden Markov model) based systems. We present a comparison of these adaptation techniques for three Indian languages, Tamil, Telugu and Hindi. The results show that the SVM based methods perform better than the HMM based methods when the 2-best and 5-best performance is considered.

The early detection of subjects with probable Alzheimer Type Dementia (ATD) is crucial for effective appliance of treatment strategies. Functional brain imaging including SPECT (Single-Photon Emission Computed Tomography) and PET (Positron Emission Tomography) are commonly used to guide the clinician’s diagnosis. Nowadays, no automatic tool has been developed to aid the experts to diagnose the ATD. Instead, conventional evaluation of these scans often relies on subjective, time consuming and prone to error steps. This paper shows a fully automatic computer-aided diagnosis (CAD) system for improving the accuracy in the early diagnosis of the ATD. The proposed approach is based on the majority voting cast by an ensemble of Support Vector Machine (SVM) classifiers, trained on a component-based feature extraction technique which searches the most discriminant regions over the brain volume.

Alzheimer disease (AD) is a progressive neurodegenerative disorder first affecting memory functions and then gradually affecting all cognitive functions with behavioral impairments. As the number of patients with AD has increased, early diagnosis has received more attention for both social and medical reasons. However, currently, accuracy in the early diagnosis of certain neurodegenerative diseases such as the Alzheimer type dementia is below 70% and, frequently, these do not receive the suitable treatment. Functional brain imaging including single-photon emission computed tomography (SPECT) is commonly used to guide the clinician’s diagnosis. However, conventional evaluation of SPECT scans often relies on manual reorientation, visual reading and semiquantitative analysis of certain regions of the brain. These steps are time consuming, subjective and prone to error. This paper shows a fully automatic computer-aided diagnosis (CAD) system for improving the accuracy in the early diagnosis of the AD. The proposed approach is based on feature selection and support vector machine (SVM) classification. The proposed system yields clear improvements over existing techniques such as the voxel as features (VAF) approach attaining a 90% AD diagnosis accuracy.

This paper presents a computer-aided diagnosis technique for improving the accuracy of the early diagnosis of the Alzheimer type dementia. The proposed methodology is based on the combination of support vector machine learning with linear kernels and classification trees. The classification tree technique allows to choose wisely the most discriminant set of voxels in the images. Thus, the classification tree produces a considerably improvement upon considering the support vector machine classifier only.

Electroencephalogram (EEG) is widely regarded as chaotic signal. Modeling and prediction of EEG signals is important for many applications. The method using support vectors machine (SVM) based on the structure risk minimization provides us an effective way of learning machine. The performance of SVM is much better than the traditional learning machine. Now the SVM is used in classification and regression. But solving the quadratic programming problem for training SVM becomes a bottle-neck of using SVM because of the long time of SVM training. In this paper, a local-SVM method is proposed for predicting the signals. The local method is presented for improving the speed of the prediction of EEG signals. The simulation results show that the training of the local-SVM obtains a good behavior. In addition, the local SVM method significantly improves the prediction precision.

Nowadays, it is possible to find out different viable architectures that implements P Systems in a distributed cluster of processors. These proposed architectures have reached a certain compromise between the massively parallelism character of the system and the evolution step times. They are based in the distribution of several membranes in each processor, the use of proxies to control the communication between membranes and mainly, the suitable distribution of the architecture in a balanced tree of processors. For a given P-system and

K

processors, there exists a great volume of possible distributions of membranes over these. The main disadvantage related with these architectures is focused in the selection of the distribution of membranes that minimizes the external communications between them and maximizes the parallelism grade. In this paper, we suggest the use of Self-Organizing Neural Networks (SONN) with growing capability to help in this selection process for a given P-system.

This paper presents the artificial neural network (ANN) that used to perform the short-term load forecasting (STLF). The input data of ANN is comprises of multiple lags of hourly peak load. Hence, imperative information regarding to the movement patterns of a time series can be obtained based on the multiple time lags of chronological hourly peak load. This may assist towards the improvement of ANN in forecasting the hourly peak loads. The Levenberg-Marquardt optimization technique is used as a back propagation algorithm for the ANN. The Malaysian hourly peak loads are used as a case study in the estimation of STLF using ANN. The results have shown that the proposed technique is robust in forecasting the future hourly peak loads with less error.

We present a system for regression using MLP neural networks with hyperbolic tangent functions in the input, hidden and output layer. The activation functions in the input and output layer are adjusted during the network training to fit better the distribution of the underlying data, while the network weights are trained to fit desired input-output mapping. A non-gradient variable step size training algorithm is used since it proved effective for that kind of problems. Finally we present a practical implementation, the system found in the optimization of metallurgical processes.

This paper evaluates the performance of the new hybrid neuro-fuzzy model, Reinforcement Learning Hierarchical Neuro-Fuzzy System (RL-HNFP), in a trade decision application. The proposed model was tested with the Euro/Yen negotiated in Foreign Exchange Market. The main objective of the trading system is to optimize the resource allocation, in order to determine the best investment strategy. The performance of the RL-HNFP was compared with different benchmark models. The results showed that the system was able to detect long term strategies, obtaining more profitability with smaller number of trades.

Combining multiple neural networks appears to be a very promising approach in improving neural network generalization since it is very difficult, if not impossible, to develop a solution that is close to global optimum using single neural network. In this paper, individual networks are developed from bootstrap re-sample of the original training and testing data sets. Instead of combining all the developed networks, this paper proposed backward elimination. In backward elimination, all the individual networks are initially aggregated and some of the individual networks are then gradually eliminated until the aggregated network error on the original training and testing data sets cannot be further reduced. The proposed techniques are applied to nonlinear process modeling and application results demonstrate that the proposed techniques can significantly improve model performance better than aggregating all the individual networks.

The computational and power resource limitations applicable to intelligent sensor systems in mobile implementations have gained much attention for industrial and medical applications. Probabilistic Neural Networks (PNN) are one of a successful classifier used to solve many classification problems. Currently, in PNN all patterns of training set are used to estimate the probability density function (pdf) of a given class as the sum of isotropic Gaussian kernels. However, the computational effort and the storage requirement of PNN method will prohibitively increase as the number of patterns used in the training set increases. In this paper, we propose as a remedy an Adaptive Resource-Aware Probabilistic Neural Networks (ARAPNN) based on two optimization goals tackle by Particle Swarm Optimization (PSO), which are finding the proper position and number of prototypes (Michigan approach) as well as the best smoothing factor

σ

(Pittsburgh approach). Our proposed algorithm was be tested with five benchmark data sets. The results show that the ARAPNN is able to find solutions with significantly reduced number of prototypes that classify data with competitive or better accuracy than the original PNN and Nearest Neighbor classifiers.

This paper presents a method of imputing missing data that combines principal component analysis and neuro-fuzzy (PCA-NF) modeling in conjunction with genetic algorithms (GA). The ability of the model to impute missing data is tested using the South African HIV sero-prevalence dataset. The results indicate an average increase in accuracy from 60 % when using the neuro-fuzzy model independently to 99 % when the proposed model is used.

The multi-population agent genetic algorithm (MPAGAFS) for feature selection is proposed. The double chain-like agent structure is introduced to enhance the diversity of population. The structure can help to construct multi-population agent GA, thereby realizing parallel searching for an optimal feature subset. The experimental results show that the MPAGAFS can not only be used for serial feature selection but also parallel feature selection with satisfying precision.

Fuzzy clustering helps to find natural vague boundaries in data. The fuzzy c-means (FCM) is one of the most popular clustering methods based on minimization of a criterion function because it works fast in most situations. However, it is sensitive to initialization and is easily trapped in local optima. Particle swarm optimization (PSO) and differential evolution (DE) are two promising algorithms for numerical optimization. Two hybrid data clustering algorithms based the two evolution algorithms and the FCM algorithm, called HPSOFCM and HDEFCM respectively, are proposed in this research. The hybrid clustering algorithms make full use of the merits of the evolutionary algorithms and the FCM algorithm. The performances of the HPSOFCM algorithm and the HDEFCM algorithm are compared with those of the FCM algorithm on six data sets. Experimental results indicate the HPSOFCM algorithm and the HDEFCM algorithm can help the FCM algorithm escape from local optima.

This paper proposed a simulation-optimization energy saving strategy for heating, ventilating and air conditioning (HVAC) systems’ condenser water loop through intelligent control of single speed cooling towers’ components. An analysis of system components has showed the interactions of control variables inside the cooling towers and between the cooling tower and chillers. Based on the analysis, a model based optimization approach was developed with evolutionary computation. A simulation application demonstrated the effectiveness of the proposed strategy. This strategy can also be easily modified and applied to single speed tools in the refrigerant loops.

In this paper an investigation of the distribution of the weights and the biases of the Multilayered Perceptron is conducted, in particular the variance of the weight vector (weights and biases) with the aim of indicating the existence of the structural diversity within the ensemble. This will indicate how well the weight vector samples are distributed from the mean and this will be used to serve as an indicator of the structural diversity of the classifiers within the ensemble. This is inspired by the fact that many measures of ensemble diversity are focused on the outcomes and not the classifier’s structure and hence may lose out in diversity measures that correlate well with ensemble performance. Three ensembles were compared, one non-diverse and the other two ensembles made diverse. The generalization across all the ensembles was approximately the same (74 % accuracy). This could be attributed to the data used. Certainty measures were also conducted and indicated that the non-diverse ensemble was biased, even though the performance across the ensembles was the same.

This paper proposes a way to approximate ground truth for real-world stereo sequences, and applies this for evaluating the performance of different variants of dynamic programming stereo analysis. This illustrates a way of performance evaluation, also allowing to derive sequence analysis diagrams. Obtained results differ from those obtained for the discussed algorithms on smaller, or engineered test data. This also shows the value of real-world testing.

Currently human-computer interaction, especially emotional interaction, still lacks intuition. In health care, it is very important for the medical robot, who assumes the responsibility of taking care of patients, to understand the patient’s feeling, such as happiness and sadness. We propose an approach to facial expression recognition for estimating patients’ emotion. Two expressions (happiness and sadness) are classified in this paper. Our method uses a novel geometric feature parameter, which we call the Emotion Geometry Feature (EGF). The active shape model (ASM), which can be categorized mainly for non-rigid shapes, is used to locate Emotion Geometry Feature (EGF) points. Meanwhile, the Support Vector Machine (SVM) is used to do classification. Our method was tested on a Japanese Female Facial Expression (JAFFE) database. Experimental results, with the average recognition rate of 97.3%, show the efficiency of our method.

In this paper, at first a new line symmetry based distance is proposed which calculates the amount of symmetry of a point with respect to the first principal axis of a data set. The proposed distance uses a recently developed point symmetry based distance in its computation. Kd-tree based nearest neighbor search is used to reduce the complexity of computing the closest symmetric point. Thereafter an evolutionary clustering technique is described that uses this new principal axis based line symmetry distance for assignment of points to different clusters. The proposed GA with line symmetry distance based (GALS) clustering technique is able to detect any type of clusters, irrespective of their geometrical shape and overlapping nature, as long as they possess the characteristics of line symmetry. GALS is compared with the existing well-known GAK-means clustering algorithm. Three artificially generated and three real-life data sets are used to demonstrate its superiority.

Feature extraction and feature selection are very important steps for face recognition. In this paper, we propose to use a class-dependent feature selection method to select different feature subsets for different classes after using principal component analysis to extract important information from face images. We then use the support vector machine (SVM) for classification. The experimental result shows that class-dependent feature selection can produce better classification accuracy with fewer features, compared with using the class-independent feature selection method.

Magnetic resonance imaging (MRI) is a imaging and diagnostic tool widely used, with excellent spatial resolution, and efficient in distinguishing between soft tissues. Here, we present a method for semi-automatic identification of brain tissues in MRI, based on a combination of machine learning approaches. Our approach uses self-organising maps (SOMs) for voxel labelling, which are used to seed the discriminative clustering (DC) classification algorithm. This method reduces the intensive need for a specialist, and allows for a rather systematic follow-up of the evolution of brain lesions, or their treatment.

This paper’s intention is to adapt prediction algorithms well known in the field of time series analysis to problems being faced in the field of mobile robotics and Human-Robot-Interaction (HRI). The idea is to predict movement data by understanding it as time series. The prediction takes place with a black box model, which means that no further knowledge on motion dynamics is used then the past of the trajectory itself. This means, the suggested approaches are able to adapt to different situations. Several state-of-the-art algorithms such as Local Modeling, Cluster Weighted Modeling, Echo State Networks and Autoregressive Models are evaluated and compared. For experiments, real movement trajectories of a human are used. Since mobile robots highly depend on real-time application, computing time is also considered. Experiments show that Echo State Networks and Local Model show impressive results for long term motion prediction.

This paper discloses a method for simple and efficient optical coupling of a robotic arm with a tool with unknown location without exerting forces to the tool. Current solutions involve moving the robot in force-control mode and coupling by means of a manual gripper. This poses the problem with the transfer of unwanted forces to the tool while attempting to secure the design. With the intrinsic solution presented here, the camera is placed on the coupling axis and thence measures the distance and orientation to the target, the user will have the ability to safely guide the robotic arm towards the tool and smoothly couple the tool with the robot’s end effector. The mechanical prototype is not here described; this paper emphasizes the image processing, consequent data interpretation and general approach. After the explanation of the technique, its theoretical performance limit was examined and confirmed against the practically achieved performance.

This paper describes an automatic plaque boundary extraction in the intravascular ultrasound image by a fuzzy inference. In the proposed method, the membership functions in the antecedent parts of the fuzzy rules are adaptively allocated by using the information of the seed points given by a medical doctor. The present method not only improved the accuracy of plaque boundary extraction but also reduced the workload of medical doctors.

This research focuses on utilizing the biometrics recognition to trigger the speech expresser. Our selected biometric is facial expression. Though CPC have no verbal language ability, they have facial expression ability that can be interpreted to relate to their voice speech needs. However facial expression of a CPC may not be exactly identical at all times. Furthermore CPC are unique and require special speech profiles. After a thorough research in face recognition and artificial intelligence domain, neural network coupled with Gabor feature extraction is found to outperform others. A Neural Network with Gabor filters is built to train the facial expression classifiers. This research has proven successful to help CPC to express their voice speech through software with 98% successful facial recognition rate.

There has been a significant drop in the cost as well as an increase in the quality of imaging sensors due to stiff competition as well as production improvements. Consequently, real-time surveillance of private or public spaces which relies on such equipment is gaining wider acceptance. While the human brain is very good at image analysis, fatigue and boredom may contribute to a less-than-optimum level of monitoring performance. Clearly, it would be good if highly accurate vision systems could complement the role of humans in round-the-clock video surveillance. This paper addresses an image analysis problem for video surveillance based on the particle swarm computing paradigm. In this study three separate datasets were used. The overall finding of the paper suggests that clustering using Particle Swarm Optimization leads to better and more consistent results, in terms of both cluster characteristics and subsequent recognition, as compared to traditional techniques such as K-Means.

Coloured network motifs are small subgraphs that enable to discover and interpret the patterns of interaction within the complex networks. The analysis of three-nodes motifs where the colour of the node reflects its high – white node or low – black node centrality in the social network is presented in the paper. The importance of the vertices is assessed by utilizing two measures: degree prestige and degree centrality. The distribution of motifs in these two cases is compared to mine the interconnection patterns between nodes. The analysis is performed on the social network derived from email communication.

It is well known that the clusters produced by a clustering algorithm depend on the chosen initial centers. In this paper we present a measure for the degree to which a given clustering algorithm depends on the choice of initial centers, for a given data set. This measure is calculated for four well-known offline clustering algorithms (k-means Forgy, k-means Hartigan, k-means Lloyd and fuzzy c-means), for five benchmark data sets. The measure is also calculated for ECM, an online algorithm that does not require the number of initial centers as input, but for which the resulting clusters can depend on the order that the input arrives. Our main finding is that this initialization dependence measure can also be used to determine the optimal number of clusters.

In this paper, we propose a method to detect object boundaries from spectral information. Previous image boundary detection techniques draw their attention on spatial image features such as brightness, color and texture. Different from traditional feature descriptor methods, we started from the analysis of natural image statistics in spectral domain and proposed a method to detect image boundaries by analyzing log spectrum residual of images. We find that the spatial transform of log spectrum residual of images are qualified as boundary maps. In the experiment section we show that our results are similar to human segmentations compared to common methods like the Canny detector.

Practical Recurrent Learning (PRL) has been proposed as a simple learning algorithm for recurrent neural networks[1][2]. This algorithm enables learning with practical order

O

(

n

2

) of memory capacity and computational cost, which cannot be realized by conventional Back Propagation Through Time (BPTT) or Real Time Recurrent Learning (RTRL). It was shown in the previous work[1] that 3-bit parity problem could be learned successfully by PRL, but the learning performance was quite inferior to BPTT. In this paper, a simple calculation is introduced to prevent monotonous oscillations from being biased to the saturation range of the sigmoid function during learning. It is shown that the learning performance of the PRL method can be improved in the 3-bit parity problem. Finally, this improved PRL is applied to a scanned digit pattern classification task for which the results are inferior but comparable to the conventional BPTT.

The paper presents a novel sentence-based language classifier that accepts a sentence as input and produces a confidence value for each target language. The proposed classifier incorporates Unicode based features and a neural network. The three features Unicode, exclusive Unicode and word matching score are extracted and fed to a neural network for obtaining a final confidence value. The word matching score is calculated by matching words in an input sentence against a common word list for each target language. In a common word list, the most frequently used words for each language are statistically collected and a database is created. The preliminary experiments were performed using test samples from web documents for languages such as English, German, Polish, French, Spanish, Chinese, Japanese and Korean. The classification accuracy of 98.88% has been achieved on a small database.

Most of the existing gender classification approaches are based on face appearance only. In this paper, we present a gender classification system that integrates face and hair features. Instead of using the whole face we extract features from eyes, nose and mouth regions with Maximum Margin Criterion (MMC), and the hair feature is represented by a fragment-based encoding. We use Support Vector Machines with probabilistic output (SVM-PO) as individual classifiers. Fuzzy integration based classifier combination mechanism is used to fusing the four different classifiers on eyes, nose, mouth and hair respectively. The experimental results show that the MMC outperforms Principal Component Analysis and Fisher Discriminant Analysis and incorporating hair feature improves gender classification performance.

Face perception and text reading are two of the most developed visual perceptual skills in humans. Understanding which features in the respective visual patterns make them differ from each other is very important for us to investigate the correlation between human’s visual behavior and cognitive processes. We introduce our fuzzy signatures with a Levenberg-Marquardt optimization method based hybrid approach for recognizing the different eye gaze patterns when a human is viewing faces or text documents. Our experimental results show the effectiveness of using this method for the real world case. A further comparison with Support Vector Machines (SVM) also demonstrates that by defining the classification process in a similar way to SVM, our hybrid approach is able to provide a comparable performance but with a more interpretable form of the learned structure.

Kyushu Electric Power Co.,Inc. collects different sensor data and weather information (hereafter, operation data) to maintain the safety of hydroelectric power plants while the plants are running. It is very rare to occur trouble condition in the plants. And it is hard to construct an experimental power generation plant for collecting the trouble condition data. Because its cost is too high. In this situation, we have to find trouble condition sign. In this paper, we consider that the rise inclination of

special unusual condition data

gives trouble condition sign. And we propose a trouble condition sign discovery method for hydroelectric power plants by using a one class support vector machine and a normal support vector machine. This paper shows the proposed method is useful method as a method of risk management for hydroelectric power plants.

In this paper, an asbestos counting method from microscope images of building materials is proposed. Since asbestos particles have unique color and shape, we use color and shape features for detecting and counting asbestos by computer. To classify asbestos and other particles, the Support Vector Machine (SVM) is used. When one kernel is applied to a feature vector which consists of color and shape, the similarity of each feature is not used effectively. Thus, kernels are applied to color and shape independently, and the summation kernel of color and shape is used. We confirm that the accuracy of asbestos detection is improved by using the summation kernel.

The prediction accuracy of QRS wave that show electric excitement by the ventricle of the heart is low in linear predictions of electrocardiogram (ECG) used a conventional linear autoregressive model, and it is a problem that the prediction accuracy is not improved even if the prediction order is set second and third or more. The causes are that QRS wave generated by the nonlinear generation mechanism and the nonlinear components which the linear models cannot predict is included in ECG. Then, Non-recursion type 1

st

-order Volterra neuron network (N1VNN) and Non-recursion type 2

nd

-order Volterra neuron network (N2VNN) were evaluated about nonlinear prediction accuracies for ECG. The results of comparing nonlinear predictions of both networks showed that N2VNN is 17.6 % smaller about the minimum root mean square error indicating prediction accuracy than N1VNN.

A novel neural network design–the adaptive resonance theory least mean square (ART-LMS) neural network–is proposed for the restoration of images corrupted by impulse noise. The network design is based on the concept of counterpropagation network (CPN). There is a vigilance parameter the ART network uses to automatically generate the cluster layer node for the Kohonen learning algorithm in CPN. In addition, the LMS learning algorithm is used to adjust the weight vectors between the cluster layer and the output layer for the Grossberg learning algorithm in CPN. The advantages of the ART-LMS network include an effective solution to the initial weight problem and a good ability to handle the cluster layer nodes for the CPN learning process. Experimental results have demonstrated that the proposed filter based on ART-LMS outperforms many well-accepted conventional as well as new filters in terms of noise suppression and detail preservation.

For particle filtering tracking method, particle choosing is random to some degree according to the dynamics equation, which may cause inaccurate tracking results. To compensate, an improved particle filtering tracking method is presented. A moving vehicle is detected by redundant discrete wavelet transforms method (RDWT), and then the key points are obtained by scale invariant feature transform. The matching key points in the follow-up frames obtained by SIFT method are used as the initial particles to improve the tracking performance. Experimental results show that more particles centralize in the region of motion area by the presented method than traditional particle filtering, and tracking results of moving vehicles are more accurate. The method has been adopted by Tianjin traffic bureau of China, and has a certain actual application prospect.

This paper presents a two stage diagnosis system that consists of Self-Organizing Map (SOM) and Learning Vector Quantization (LVQ) subsystems for diagnosis of fundus images. The first stage performs clustering and pseudo-classification of the input feature data by a SOM. The use of the pseudo-classes is able to improve the performance of the second stage consisting of a LVQ codebook. The proposed system has been tested on real medical treatment image data. In the experiments we have achieved a maximum accuracy rate of 71.2%, which is comparable to other results in literature.

In this paper two facial expression recognition (FER) techniques are proposed. Lower-frequency 2-D DCT coefficients of binarized edge images are utilized in both methods as features for recognition. The first approach uses a constructive one-hidden-layer (OHL) feedforward neural network (OHL-NN) and the second approach is based on the K-means algorithm as classifiers. The 2-D DCT is used to compress the binarized edge images to capture the important features for recognition. Facial expression “neutral” is regarded as a subject of recognition in addition to two other expressions, “smile” and “surprise”. The two proposed recognition techniques are applied to two databases which contain 2-D front face images of 60 men (database (a)) and 60 women (database (b)), respectively. Experimental results reveal that the proposed two techniques yield performances that are comparable to or better than that of two other recognition methods using vector matching and fixed-size BP-based NNs, respectively. The first proposed method yields testing recognition rates as high as 100% and 95%, and the second one achieves as high as 100% and 98.33%, for databases (a) and (b), respectively.

Living cells possess properties that enable them to withstand the physiological environment as well as mechanical stimuli occurring within and outside the body. Any deviation from these properties will undermine the physical integrity of the cells as well as their biological functions. Thus, a quantitative study in single cell mechanics needs to be conducted. In this paper we will examine fluid flow and Neo-Hookean deformation. Particularly, a mechanical model to describe the cellular adhesion with detachment is proposed. Restricting the interest on the contact surface and elaborating again the computational results, it is possible to develop our idea about to reproduce the phases coexistence in the adhesion strip. Subsequently, a number of simulations have been carried out, involving a number of human cells with different mechanical properties. All the collected data have been used in order to train and test a suitable Artificial Neural Network (ANN) in order to classify the kind of cell. Obtained results assure good performances of the implemented classifier, with very interesting applications.

This paper employs pattern classification methods for assisting contact centers in determining caller interaction at a ’Say account’ field within an Interactive Voice Response application. Binary and real coded genetic algorithms (GAs) that employed normalized geometric ranking as well as tournament selection functions were utilized to optimize the Multi-Layer Perceptron neural network architecture. The binary coded genetic algorithm (GA) that used tournament selection function yielded the most optimal solution. However, this algorithm was not the most computationally efficient. This algorithm demonstrated acceptable repeatability abilities. The binary coded GA that used normalized geometric selection function yielded poor repeatability capabilities. GAs that employed normalized geometric ranking selection function were computationally efficient, but yielded solutions that were approximately equal. The real coded tournament selection function GA produced classifiers that were approximately 3% less accurate than the binary coded tournament selection function GA.

A novel background model based on Marr wavelet kernel and a background subtraction technique based on binary discrete wavelet transforms are introduced. The background model keeps a sample of intensity values for each pixel in the image and uses this sample to estimate the probability density function of the pixel intensity. The density function is estimated using a new Marr wavelet kernel density estimation technique. Since this approach is quite general, the model can approximate any distribution for the pixel intensity without any assumptions about the underlying distribution shape. The background and current frame are transformed in the binary discrete wavelet domain, and background subtraction is performed in each sub-band. Experiments show that the simple method produces good results with much lower computational complexity and can effectively extract the moving objects, even though the objects are similar to the background, thus good moving objects segmentation can be obtained.

The two groups of popularly used texture analysis techniques for classification problems are the statistical and signal processing methods. In this paper, we propose to use a signal processing method, the Gabor filters to produce the feature images, and a statistical method, the covariance matrix to produce a set of features which show the statistical information of frequency domain. The experiments are conducted on 32 textures from the Brodatz texture dataset. The result that is obtained for the use of 24 Gabor filters to generate a 24 × 24 covariance matrix is 91.86%. The experiment results show that the use of Gabor filters as the feature image is better than the use of edge information and co-occurrence matrices.

This paper presents a method of determining whether demographic properties such as education, race, age, physical location, gravidity and parity influence the ability to classify the HIV status of a patient. The degree to which these variables influence the HIV classification is investigated by using an ensemble of autoassociative neural networks that are trained using the Bayesian framework. The HIV classification is treated as a missing data problem and the ensemble of autoassociative neural networks coupled with an optimization technique are used to determine a set of possible estimates. The set of possible estimates are aggregated together to give a predictive certainty measure. This measure is the percentage of the most likely estimate from all possible estimates. Changes to the state of each of the demographic properties are made and changes in the predictive certainty are recorded. It was found that the education level and the race of the patients are influential on the predictability of the HIV status. Significant knowledge discovery about the demographic influences on predicting a patients HIV status is obtained by the methods presented in this paper.

This paper presents how hardware-based machine learning models can be designed for the task of object recognition. The process is composed of automatic representation of objects as covariance matrices follow by a machine learning detector based on random forest (RF) that operate in on-line mode. We describe the architecture of our random forest (RF) classifier employing Logarithmic Number Systems (LNS), which is optimized towards a System-on-Chip (Soc) platform implementation. Results demonstrate that the proposed model yields in object recognition performance comparable to the benchmark standard RF, AdaBoost, and SVM classifiers, while allow fair comparisons between the precision requirements in LNS and of using traditional floating-point.

A novel neural oscillation model is proposed to perform the contour detection and separation for color images. The model improves the prototype of relaxation oscillation and combines four kinds of image features as the external stimulation, in order to control the oscillation. Experimental results show that the contours of different objects can be detected by oscillation and separated by desynchronization. This model may help to promote the investigation on the mechanism of the neural system.

A Pulse Coupled Neural Network (PCNN) is a kind of numerical model of cat visual cortex and it can explain synchronous dynamics of neurons’ activity in the visual cortex. On the other hand, as an engineering application, it is shown that the PCNN can applied to the image processing,

e.g.

segmentation, edge enhancement, and so on. The PCNN model consists of neurons and two kind of inputs, namely feeding inputs and linking inputs with leaky integrators. These inputs lead to discrete time evolution of its internal state and neurons generate spike output according to the internal state. The linking and feeding inputs are received from the neurons’ receptive field which is defined by excitatory synaptic weights. In this study, we propose a PCNN with inhibitory connections and describe an application to a color image segmentation. In proposed model, inhibitory connections are defined by negative synaptic weights among specific neurons which detect RGB component of particular pixel of the image. Simulation results show successful results for the color image segmentation.

A computational theory concept generating saliency maps from feature maps generated in the bottom-up using various filters such as Fourier transformation was discussed. We proposed saliency map related to motion based on self-organized feature extracting not using general filter such as Fourier transform. We introduce the ICA base function to realize the self-organized Saliency Map. We extend the ICA base function estimation to apply for the non-uniform positioned photoreceptor cells which receives the current image and the previous image to get the motion information. We show the effectiveness of our model by applying this model for real images.

Content-based Image Retrieval (CBIR) systems have been rapidly developing over the years, both in labs and in real world applications. Face Image Retrieval (FIR) is a specialised CBIR system where a user submits a query (image of a face) to the FIR system which searches and retrieves the most visually similar face images from a database. In this paper, we use a neural-network based similarity measure and compare the retrieval performance to Lp-norm similarity measures. Further we examined the effect of user relevance-feedback on retrieval performance. It was found that the neural-similarity measure provided significant performance gains over Lp-norm similarity measures for both the training and test data sets.

Recently, it was shown how some metaphors, adopted from the infant vision system, were useful for face recognition. In this paper we adopt those biological hypotheses and apply them to the 3D object recognition problem. As the infant vision responds to low frequencies of the signal, a low-filter is used to remove high frequency components from the image. Then we detect subtle features in the image by means of a random feature selection detector. At last, a dynamic associative memory (DAM) is fed with this information for training and recognition. To test the accuracy of the proposal we use the Columbia Object Image Library (COIL 100).

This paper presents a method for defining one or more virtual restricted zones within a surveillance area which is observed with stereo cameras. When an object enters a restricted zone, the system highlights the object shown in the monitoring screen or triggers other devices to produce a visual or auditory alarm. The proposed method works by extracting the foreground objects for both the left and the right images from their respective stereo cameras. Then it estimates the object’s position in terms of depth plane using image shifting and number of overlapping pixels. Finally, it determines whether there is a collision between objects and restricted zones in order to trigger an alarm where necessary. The algorithm has been tested with a series of stereo videos, in which samples of it are presented in this paper.

The internal load in humans caused by an external exposure is different in each person and mainly depends on metabolism. Using the recently proposed method of mnSOM we are able to describe the human metabolism using a functional module (linear or nonlinear) for each individual.mnSOM enables us to subdivide individuals into classes based on the functional description of each individuals metabolism. Furthermore the shown approach is able to show dependencies between external exposure and internal load in humans. In environmental epidemiology this will be used to establish links between external exposure and internal load patterns to gather clinical relevant information for practitioners.

The ongoing advance in micro technologies gives rise to increasingly versatile and capable sensors as well as unprecedented computational power and communication options in diminishing scale. The notion of smart dust summarizes ubiquitous computing and sensing application systems, which can serve for local as well as global information acquisition and decision making. For off-the-shelf-nodes, and even more for dedicated physical designs, the system design process becomes increasingly challenging and potentially intractable. Automated design methods emerging for intelligent systems are introduced as a remedy. These considerations will be extended to variations, that multiple system instances have to face in real-world applications and potential compensation by incorporation of self-x properties. These concepts are elucidated for the case of reconfigurable and evolvable sensor electronics. Finally

,

an application perspective of the presented approach for integrated distributed sensing in home automation, assisted living, and in particular, smart kitchen applications, denoted as culinary assistance systems will be presented.

This paper discusses the hardware design and implementation of a hand sign recognition system with a simplified discrete Fourier transforms (DFTs) that calculate the magnitude spectrum. Two alternative hardware design solutions that implement the system are proposed. One uses parallel classifier network, the other uses serial one. With the parallel network, the circuit size of the recognition system is over 280,000-gate while the system with the serial classifier network requires about 90,000-gate of hardware resources. Regarding the operating speed, it has been revealed that the operation speed of the both system is quick enough to process NTSC video frame in real time.

We investigated the performance of a blind source separation (BSS) system based on stochastic computing in the case of an aperiodic source signal by both simulation and a field programmable gate array (FPGA) experiment. We confirmed that our BSS system can successfully infer source signals from mixed signals. We show that the system succeeds in separating source signals from mixed signals after about 3.7 seconds at a clock frequency of 32 MHz on an FPGA.

We previously proposed a neural segmentation model suitable for implementation with complementary metal-oxide-semiconductor (CMOS) circuits. The model consists of neural oscillators mutually coupled through synaptic connections. The learning is governed by a symmetric spike-timing-dependent plasticity (STDP). Here we demonstrate and evaluate the circuit operation of the proposed model with a network consisting of six oscillators. Moreover, we explore the effects of mismatch in the threshold voltage of transistors, and demonstrate that the network was tolerant to mismatch (noise).

Several applications would emerge from the development of efficient and robust sound classification systems able to identify the nature of non-speech sound sources. This paper proposes a novel approach that combines a simple feature generation procedure, a supervised learning process and fewer parameters in order to obtain an efficient sound classification system solution in hardware. The system is based on the signal processing modules of a previously proposed sound processing system, which convert the input signal in spike trains. The feature generation method creates simple binary features vectors, used as the training data of a standard LVQ neural network. An output temporal layer uses the time information of the sound signals in order to eliminate the misclassifications of the classifier. The result is a robust, hardware friendly model for sound classification, presenting high accuracy for the eight sound source signals used on the experiments, while requiring small FPGA logic and memory resources.

This paper proposes a complex valued generalized product neuron (GPN) which tries to incorporate polynomial structure in the aggregation of inputs. The advantage of using this model is to bring in the non-linearity in aggregation function by taking a product of linear terms in each dimension of the input space. This aggregation function has the ability to capture higher-order correlations in the input data. Such neurons are capable of learning any problem irrespective of whether the multi dimensional data is linearly separable or not which resembles higher order neurons. But these neurons do not have combinatorial increase of the number of weights in the dimensions of inputs as higher order neurons. The learning and generalization capabilities of proposed neuron are demonstrated through variety of problems. It has been shown that some benchmark problems can be solved with single GPN only without hidden layer.

In recent years, synaptic plasticity, which is dependent on the order and time interval of pre- and post-synaptic spikes, has been observed by physiological experiments. There are two types of STDP which are characterized by an asymmetric time window and a symmetric time window (mexican hat type window). A symmetric time window especially depends on the influence of an inhibitory neuron. In this paper, we investigate the synaptic circuit and the synaptic weight control circuit using STDP by inhibitory interneuron input or no input. As a result, we show that the synaptic circuit using STDP with the time windows of these two types could be constructed with a simple circuit configuration considering an inhibitory interneuron by using the circuit simulator PSpice. Furthermore, we show the characteristic of reinforcement and suppression.

A continuous time neural network of Hopfield type is considered. It is a W(inner) T(akes) A(ll) selector. Its inputs are capacitively coupled to model the parasitics or faults of overcrowded chip layers. A certain parameter setting allows the correct selection of the maximum element from an input list. As processing time is a performance criterion, we infer upper bounds of it, explicitly depending on circuit and list parameters. Our method consists of converting the system of nonlinear differential equations describing the circuit to a system of decoupled linear inequalities. The results are checked by numerical simulation.

P systems or Membrane Computing are a type of a distributed, massively parallel and non deterministic system based on biological membranes. These systems perform a computation through transition between two consecutive configurations. As it is well known in membrane computing, a configuration consists in a

m

-tuple of multisets present at any moment in the existing

m

regions of the system at that moment time. Transitions between two configurations are performed by using evolution rules which are in each region of the system in a non-deterministic maximally parallel manner. This article shows the development of a hardware circuit of selection of active rules in a membrane of a transition P-system. This development has been researched by using the Quartus II tool of Altera Semiconductors. In the first place, the initial specifications are defined in orfer to outline the synthesis of the circuit of active rules selection. Later on the design and synthesis of the circuit will be shown, as well as, the operation tests required to present the obtained results.

AdaBoost is being used widely in information systems, artificial intelligence and bioinformatics, because it provides an efficient function approximation method. However, AdaBoost does not employ either the maximum likelihood method or Bayes estimation, and hence its generalized performance is not yet known. Therefore an optimization method for the minimum generalization error has not yet been established. In this paper, we propose a new method to select an optimal model using formal information criteria, AIC and BIC. Although neither AIC nor BIC theoretically corresponds to the generalization error in AdaBoost, we show experimentally that an optimal model can be chosen by formal AIC and BIC.

The concept of Ensemble Learning has been shown to increase predictive power over single base learners. Given the bias-variance-covariance decomposition, diversity is characteristic factor, since ensemble error decreases as diversity increases. In this study, we apply Bagging and Random Subspace Method (RSM) to ensembles of Local Linear Map (LLM)-type, which achieve non-linearity through local linear approximation, supplied with different vector quantization algorithms. The results are compared for several benchmark data sets to those of RandomForest and neural networks. We can show which parameters are of major influence on diversity in ensembles and that using our proposed method of LLM combining RSM we are able to achieve results obtained by other reference ensemble architectures.

While injecting weight noise during training has been proposed for more than a decade to improve the convergence, generalization and fault tolerance of a neural network, not much theoretical work has been done to its convergence proof and the objective function that it is minimizing. By applying the Gladyshev Theorem, it is shown that the convergence of injecting weight noise during training an RBF network is almost sure. Besides, the corresponding objective function is essentially the mean square errors (MSE). This objective function indicates that injecting weight noise during training an radial basis function (RBF) network is not able to improve fault tolerance. Despite this technique has been effectively applied to multilayer perceptron, further analysis on the expected update equation of training MLP with weight noise injection is presented. The performance difference between these two models by applying weight injection is discussed.

This paper proposed a simulation-optimization energy saving strategy for heating, ventilating and air conditioning (HVAC) systems’ condenser water loop through intelligent control of single speed cooling towers’ components. An analysis of system components has showed the interactions of control variables inside the cooling towers and between the cooling tower and chillers. Based on the analysis, a model based optimization approach was developed with evolutionary computation. A simulation application demonstrated the effectiveness of the proposed strategy. This strategy can also be easily modified and applied to single speed tools in the refrigerant loops.

We discuss Bregman divergences and the very close relationship between a class of these divergences and the regular family of exponential distributions before applying them to various topology preserving dimension reducing algorithms. We apply these to multidimensional scaling (MDS) and show the effect of different Bregman divergences. In particular we derive a mapping similar to the Sammon mapping. We apply these methods to face identification.

In this paper, we propose a new method called

collective activations

to realize self-organizing maps. We suppose that all neurons collectively respond to input stimuli, and this collectiveness is represented by the sum of all neurons’ activations. Learning consists of imitating these collective activations as much as possible. We applied the method to artificial data and a broadband survey problem. In all these problems, we could obtain self-organizing maps similar or, in some cases, superior to those obtained by conventional SOM. Thus, the present study is considered to be the first step toward more realistic self-organizing maps.

Several researchers have recently proposed alternative estimation methods of Boltzmann machines (BMs) beyond the standard maximum likelihood framework. Examples are the contrastive divergence or the ratio matching, and also a rather classic pseudolikelihood method. With a loss of statistical efficiency, alternative methods can often speed-up the computation and/or simplify the implementation. In this article, as an extreme of this direction, we show the parameter estimation of BMs can be done even with a closed-form estimator, by recasting the problem into linear regression. We confirm our estimator can actually approach the true parameter as the sample size increases, while the convergence can be slow, by a simple simulation experiment.

The non-negative matrix factorization (NMF) is capable of factorizing strictly positive data into strictly positive activations and base vectors. In its standard form, the input data must be presented as a batch of data. This means the NMF is only able to represent the input space contained in this batch of data whereas it is not able to adapt to changes afterwards. In this paper we propose a method to overcome this limitation and to enable the NMF to incrementally and continously adapt to new data. The proposed algorithm is able to cover the (possibly growing) input space

without

putting further constraints on the algorithm. We show that using our method the NMF is able to approximate the dimensionality of a dataset and therefore is capable to determine the required number of base vectors automatically.

For the progress in developing human-like intelligence in robots, autonomous and purposive learning of adaptive memory function is significant. The combination of reinforcement learning (RL) and recurrent neural network (RNN) seems promising for it. However, it has not been applied to a continuous state-action space task, nor has its internal representations been analyzed in depth. In this paper, in a continuous state-action space task, it is shown that a robot learned to memorize necessary information and to behave appropriately according to it even though no special technique other than RL and RNN was utilized. Three types of hidden neurons that seemed to contribute to remembering the necessary information were observed. Furthermore, by manipulate them, the robot changed its behavior as if the memorized information was forgotten or swapped. That makes us feel a potential towards the emergence of higher functions in this very simple learning system.

Does Independent Component Analysis (ICA) denature EEG signals? We applied ICA to two groups of subjects (mild Alzheimer patients and control subjects). The aim of this study was to examine whether or not the ICA method can reduce both group differences and within-subject variability. We found that ICA diminished Leave-One-Out root mean square error (RMSE) of validation (from 0.32 to 0.28), indicative of the reduction of group difference. More interestingly, ICA reduced the inter-subject variability within each group (

σ

= 2.54 in the

δ

range before ICA,

σ

= 1.56 after, Bartlett p = 0.046 after Bonferroni correction). Additionally, we present a method to limit the impact of human error (≃ 13.8%, with 75.6% inter-cleaner agreement) during ICA cleaning, and reduce human bias. These findings suggests the novel usefulness of ICA in clinical EEG in Alzheimer’s disease for reduction of subject variability.

The Nonnegative Matrix Factorization (NMF) is a widely used method in approximating high dimensional data. All the NMF type methods find only the local minimizers. In this paper, we use filled function method to find the global minimizer of the Projective Nonnegative Matrix Factorization optimal problem.

For extracting sparse structures in images adaptively, the prior probabilities over the coefficients are modeled with a flexible parametric Cauchy density, which can describe a class of super-Gaussian distributions by varying the steepness and the scale parameters in the density function. The derivatives of the sparseness cost function are continuous at each point of its domain, which is convenient for gradient techniques based learning algorithms, and may provide a better approximation of the volume contribution from the prior. The performance of the flexible prior is demonstrated on a set of natural images.

In this paper, a one-layer recurrent neural network is proposed for solving non-smooth convex optimization problems with linear equality constraints. Comparing with the existing neural networks, the proposed neural network has simpler architecture and the number of neurons is the same as that of decision variables in the optimization problems. The global convergence of the neural network can be guaranteed if the non-smooth objective function is convex. Simulation results are provided to show that the state trajectories of the neural network can converge to the optimal solutions of the non-smooth convex optimization problems and show the performance of the proposed neural network.

Recently, a lot of research on a Brain Computer Interface (BCI) which enables patients like those with Amyotrophic Lateral Sclerosis to control some equipment or to communicate with other people has been reported. One of the problems in BCI research is a trade-off between the speed and the accuracy. In the field of data transmission, on the other hand, Reliability-Based Hybrid ARQ (RB-HARQ) has been developed to achieve both of the performances. In this paper, therefore, BCIs are considered as communications between users and computers, and Reliability-Based ARQ, similar to RB-HARQ, is applied to BCIs. Through simulations and experiments, it is shown that the proposed method is superior to other methods.

Saccade-related electroencephalogram (EEG) signals have been the subject of application oriented research by our group toward developing a brain computer interface (BCI). Our goal is to develop novel BCI based on eye movements system employing EEG signals on-line. Most of the analysis of the saccade-related EEG data has been performed using ensemble averaging approaches. In signal processing method for BCI, raw EEG signals are analyzed. In ensemble averaging method which is major EEG analysis is not suitable for processing raw EEG signals. In order to process raw EEG data, we use independent component analysis. This paper presents extraction rate of saccade-related EEG signals by four ICA algorithms and six window size. In terms of extracting rates across ICA algorithms, The JADE and Fast ICA have good results. As you know, calculation time in Fast ICA is faster than calculation time in JADE. Therefore, in this case, Fast ICA is the best in order to extract saccade-related ICs. Next, we focus on extracting rates in each window. The windows not including EEG signals after saccade and the windows which has small window size has better extracting rates.

The potential market of robots that can helpfully work at home is increasing, and such robots are required to possess force and tactile sensors achieving dynamic and cooperative interactions with their users. Virtual realization of force/tactile sensors in robots, using user’s biological signals such as EMG and postural information, is a versatile solution allowing high spatial resolution and degrees of freedom. In this paper, however, we first show the virtual force sensing approach does not work for a three-dimensional cooperative task in which the user is requested to move a load by an upper-limb of the user cooperatively with the robot, and discuss about inevitable problems. We then propose to apply policy gradient learning to overcome the problems, and demonstrate preliminary but promising learning results.

Online artifact rejection, feature extraction, and pattern recognition are essential to advance the Brain Computer Interface (BCI) technology so as to be practical for real-world applications. The goals of BCI system should be a small size, rugged, lightweight, and have low power consumption to meet the requirements of wearability, portability, and durability. This study proposes and implements a moving-windowed Independent Component Analysis (ICA) on a battery-powered, miniature, embedded BCI. This study also tests the embedded BCI on simulated and real EEG signals. Experimental results indicated that the efficacy of the online ICA decomposition is comparable with that of the offline version of the same algorithm, suggesting the feasibility of ICA for online analysis of EEG in a BCI. To demonstrate the feasibility of the wearable embedded BCI, this study also implements an online spectral analysis to the resultant component activations to continuously estimate subject’s task performance in near real time.

This paper describes the design automation issues and techniques used to design a massively parallel processing platform – SpiNNaker – from a hardware and systems design perspective. The emphasis of this paper is addressing the key problem of resource mapping, where multiple threaded programs are to be targeted onto a hardware platform that consists of multiple ARM cores and other resources such as memory and networks. In addition, the design environment is considered to ensure that a designer can program applications onto this environment in a practical manner.

Real-time modelling of large neural systems places critical demands on the processing system’s dynamic model. With spiking neural networks it is convenient to abstract each spike to a point event. In addition to the representational simplification, the event model confers the ability to defer state updates, if the model does not propagate the effects of the current event instantaneously. Using the SpiNNaker dedicated neural chip multiprocessor as an example system, we develop models for neural dynamics and synaptic learning that delay actual updates until the next input event while performing processing in background between events, using the difference between “electronic time” and “neural time” to achieve real-time performance. The model relaxes both local memory and update scheduling requirements to levels realistic for the hardware. The delayed-event model represents a useful way to recast the real-time updating problem into a question of time to the

next

event.

We introduce a processing performance of Particle Swarm Optimization with SIDM-oriented Fast Mersenne Twister on the Cell Broadband Engine. Extreme-high processing performance is demanded for solving very complex optimization problem in a small amount of time. In this research, we verified the effectiveness of employing SIMD-oriented Fast Mersenne Twister on the Cell Broadband Engine for the processing of Particle Swarm Optimization by numerical simulations.

DNA computing-inspired pattern classification based on the hypernetwork model is a novel approach to pattern classification problems. The hypernetwork model has been shown to be a powerful tool for analysis of gene expression data. However, the ordinary hypernetwork model has limitations, such as using only binary data and operating sequentially. In this paper, we propose an improved method to process four-level data and to implement a hardware circuit for DNA computing-inspired pattern classifier. We show simulation results of multi-class cancer classification from the DNA microarray data for performance evaluation. Experiments show competitive diagnosis results over other conventional machine learning algorithms. Our four-level data approach also results stable and improved performance over the ordinary hypernetwork model.