Computational Intelligence and Bioinspired Systems

The generalization ability of different sizes architectures with one and two hidden layers trained with backpropagation combined with early stopping have been analyzed. The dependence of the generalization process on the complexity of the function being implemented is studied using a recently introduced measure for the complexity of Boolean functions. For a whole set of Boolean symmetric functions it is found that large neural networks have a better generalization ability on a large complexity range of the functions in comparison to smaller ones and also that the introduction of a small second hidden layer of neurons further improves the generalization ability for very complex functions. Quasi-random generated Boolean functions were also analyzed and we found that in this case the generalization ability shows small variability across different network sizes both with one and two hidden layer network architectures.

In this paper, the behavior of the Sanger hebbian artificial neural networks [6] is analyzed. Hebbian neural networks are employed in communications and signal processing applications, among others, due to their capability to implement Principal Component Analysis (PCA). Different improvements over the original model due to Oja have been developed in the last two decades. Among them, Sanger model was designed to directly provide the eigenvectors of the correlation matrix[8]. The behavior of these models has been traditionally considered on a continuous-time formulation whose validity is justified via some analytical procedures that presume, among other hypotheses, an specific asymptotic behavior of the learning gain. In practical applications, these assumptions cannot be guaranteed. This paper addresses the study of a deterministic discrete-time (DDT) formulation that characterizes the average evolution of the net, preserving the discrete-time form of the original network and gathering a more realistic behavior of the learning gain[13]. The dynamics behavior Sanger model is analyzed in this more realistic context. The results thoroughly characterize the relationship between the learning gain and the eigenvalue structure of the correlation matrix.

This paper proposes a neural network structure as well as an adaptation of the backpropagation algorithm for its training that provides a way to consider multidimensional information directly in its original space. Traditionally, when inputting multidimensional information to artificial neural networks, its components are fed individually through different inputs and basically processed separately throughout the network. In the present structure, the multidimensional information, in the form of vectors is processed as such in the network, thus preserving in a simple way all the multidimensional neighbourhood relationships. The projection into the dimensionality of the output space is also carried out within the network. This procedure allows for a simpler processing of multidimensional signals such as multi or hyperspectral cubes as used in remote sensing or colour signals in images, which is the example we present as a test for the algorithm.

Research efforts in metaheuristics have shown that an intelligent incorporation of more classical optimization techniques in metaheuristics can be very beneficial. In this paper, we combine the metaheuristic ant colony optimization with dynamic programming for the application to the NP – hard

k

-cardinality tree problem. Given an undirected graph

G

with node and/or edge weights, the problem consists of finding a tree in

G

with exactly

k

edges such that the sum of the weights is minimal. In a standard ant colony optimization algorithm, ants construct trees with exactly

k

edges. In our algorithm, ants may construct trees that have more than

k

edges, in which case we use a recent dynamic programming algorithm to find—in polynomial time—the best

k

-cardinality tree embedded in the bigger tree constructed by the ants. We show that our hybrid algorithm improves over the standard ant colony optimization algorithm and, for node-weighted grid graph instances, is a current state-of-the-art method.

There are many papers in the literature that deal with the problem of the design of a fuzzy system from a set of given training examples. Those who get the best approximation accuracy are based on TSK fuzzy rules, which have the problem of not being as interpretable as Mamdany-type Fuzzy Systems. A question now is posed: How can the interpretability of the generated fuzzy rule-table base be increased? A possible response is to try to reduce the rule-base size by generalizing fuzzy-rules consequents which are symbolic functions instead of fixed scalar values or polynomials, and apply symbolic interpolations techniques in fuzzy system generation. A first approximation to this idea is presented in this paper for 1-D functions.

This paper presents a study based on the empirical results of the average first hitting time of Estimation of Distribution Algorithms. The algorithms are applied to one example of linear, pseudo-modular, and unimax functions. By means of this study, the paper also addresses recent issues in Estimation of Distribution Algorithms: (i) the relationship between the complexity of the probabilistic model used by the algorithm and its efficiency, and (ii) the matching between this model and the relationship among the variables of the objective function. After analyzing the results, we conclude that the order of convergence is not related to the complexity of the probabilistic model, and that an algorithm whose probabilistic model mimics the structure of the objective function does not guarantee a low order of convergence.

The Shortest Common Supersequence problem is a hard combinatorial optimization problem with numerous practical applications. Several evolutionary approaches are proposed for this problem, considering the utilization of penalty functions, GRASP-based decoders, or repairing mechanisms. An empirical comparison is conducted, using an extensive benchmark comprising problem instances of different size and structure. The empirical results indicate that there is no single best approach, and that the size of the alphabet, and the structure of strings are crucial factors for determining performance. Nevertheless, the repair-based EA seems to provide the best performance tradeoff.

Artificial Neural Networks (ANNs)

Neural networks (NNs) and genetic algorithms (GAs) are the two most popular bio-inspired techniques. Criticism of these approaches includes the tendency of recurrent neural networks to produce infeasible solutions, the lack of generalize of the self-organizing approaches, and the requirement of tuning many internal parameters and operators of genetic algorithms. This paper proposes a new technique which enables feasible solutions, removes the tuning phase, and improves solutions quality of typical combinatorial optimization problems as the p-median problem. Moreover, several biology inspired approaches are analyzed for solving traditional benchmarks.

The problem of optimal strategies of resource allocation for companies competing in the shopping malls in a metropolis is investigated in the context of two-dimensional three state Potts model in statistical physics. The aim of each company is to find the best strategy of initial distribution of resource to achieve market dominance in the shortest time. Evolutionary Algorithm is used to encode the ensemble of initial patterns of three states Potts model and the fitness of the configuration is measured by the market share of a chosen company after a fixed number of Monte Carlo steps of evolution. Numerical simulation indicates that initial patterns with certain topological properties do evolve faster to market dominance. The description of these topological properties is measured by the degree distribution of each company. Insight on the initial patterns that entail fast dominance is discussed.

In this paper we present a new method for hybrid evolutionary algorithms where only a few best individuals are subject to local optimization. Moreover, the optimization algorithm is only applied at specific stages of the evolutionary process. The key aspect of our work is the use of a clustering algorithm to select the individuals to be optimized. The underlying idea is that we can achieve a very good performance if, instead of optimizing many very similar individuals, we optimize just a few different individuals. This approach is less computationally expensive. Our results show a very interesting performance when this model is compared to other standard algorithms. The proposed model is evaluated in the optimization of the structure and weights of product-unit based neural networks.

Based on Lamarckism and Immune Clonal Selection Theory, Lamarckian Clonal Selection Algorithm (LCSA) is proposed in this paper. In the novel algorithm, the idea that Lamarckian evolution described how organism can evolve through learning, namely the point of “Gain and Convey” is applied, then this kind of learning mechanism is introduced into Standard Clonal Selection Algorithm (SCSA). Through the experimental results of optimizing complex multimodal functions, compared with SCSA and the relevant evolutionary algorithm, LCSA is more robust and has better convergence.

Once the behaviour of particular brain circuits has been analyzed, we have added up some of these patterns to Artificial Neural Networks; thus a new hybrid learning method has emerged

.

In order to find the best solution to a given problem, this method combines the use of Genetic Algorithms with particular changes to connection weights based in the behaviour observed in the brain circuits analyzed. The design and implementation of this combination is shown in feed-forward multilayer artificial neural networks, specifically created to solve a simple problem. We also illustrate the benefits obtained with these new nets from a comparison with previous results achieved by the optimal Artificial Neural Networks used so far for solving the same problem.

Neurons communicate through spikes; their arrangement in different sequences generates the neural code. Spikes are transmitted between neurons via synapses; the mechanism underlying synaptic transmission involves numerous processes including neurotransmitter release and diffusion, postsynaptic receptor activation, and intrinsic electroresponsiveness. Based on available experimental data and theoretical considerations, we have developed a realistic model predicting the dynamics of neurotransmission at the mossy fiber – granule cell synapse of the cerebellum. The model permits systematic investigation of the multiple mechanisms regulating synaptic transmission and provides predictions on the role of the numerous factors driving synaptic plasticity. The model is also employed to quantify information transfer at the mossy fiber – granule cell synaptic relay. This work was funded in part by the EU SpikeForce project (IST-2001-35271 www.spikeforce.org).

The Fast Spiking (FS) interneurons are coupled by both electrical and inhibitory synapses and experimental findings suggest that they operate as a clockwork affecting the processing of neural information. At present, it is not known which is the functional role of electrical synapses in a network of inhibitory interneurons. In our contribution, by using a single compartment biophysical model of an FS cell, we determine the parameter values leading to the emergence of synchronous regimes in a network of FS interneurons coupled by chemical and electrical synapses. We also compare our results with those recently obtained for a pair of coupled Integrate & Fire neural models [1].

Many cells in cat and monkey visual cortex (area V1 and area 17) respond to gratings and bar patterns of different orientation between center and surround[18]. It has been shown that these cells respond on average 3.3 times stronger to a crossing pattern than to a single bar[16]. In this paper a computational model for a group of neurons that respond solely to crossing patterns is proposed, and has been implemented in visual programming environment TiViPE [10]. Simulations show that the operator responds very accurately to crossing patterns that have an angular difference between 2 bars of 40 degrees or more, the operator responds appropriately to bar widths that are bound by 50 to 200 percent of the preferred bar width and is insensitive to non-uniform illumination conditions, which appear to be consistent with the experimental results.

Models of neurons based on iterative maps allows the simulation of big networks of coupled neurons without loss of biophysical properties such as spiking, bursting or tonic bursting and with an affordable computational effort. A piecewise linear two dimensional map with one fast and one slow variable is used to model spiking–bursting neural behavior. This map shows oscillations similar to other phenomenological models based on maps that require a much higher computational effort. The dynamics of coupled neurons is studied for different coupling strengths.

A spiking neural network modeling the cerebellum is presented. The model, consisting of more than 2000 conductance-based neurons and more than 50 000 synapses, runs in real-time on a dual-processor computer. The model is implemented on an event-driven spiking neural network simulator with table-based conductance and voltage computations. The cerebellar model interacts every millisecond with a time-driven simulation of a simple environment in which adaptation experiments are setup. Learning is achieved in real-time using spike time dependent plasticity rules, which drive synaptic weight changes depending on the neurons activity and the timing in the spiking representation of an error signal. The cerebellar model is tested on learning to continuously predict a target position moving along periodical trajectories. This setup reproduces experiments with primates learning the smooth pursuit of visual targets on a screen. The model learns effectively and concurrently different target trajectories. This is true even though the spiking rate of the error representation is very low, reproducing physiological conditions. Hence, we present a complete physiologically relevant spiking cerebellar model that runs and learns in real-time in realistic conditions reproducing psychophysical experiments. This work was funded in part by the EC SpikeFORCE project (IST-2001-35271, www.spikeforce.org).

Computing in nature as is the case with the human brain is an emerging research area in theoretical computer science. The present paper’s aim is to explore biological neural cell processes of interest and to model them with foundational concepts of computer science. We have started by discovering and studying certain primitive symbolic neural operations of neuron functions, and we have formalized them with Lindenmayer (L) systems.

Prior studies showed that stimulus equivalence did not emerge in nonhuman and it may be what distinguish humans from non-humans. We think that stimulus equivalence is the origin of human fs illogical reasoning.

For applying neural networks to stimulu equivalence, a problem of missing input features and self-supervised learning must be solved. In this paper, we propose a neural network model based on the iterative inversion method which has a potential possibility to explain the stimulus equivalence and demonstrated the validity of the proposed model by computer simulations. Furthermore, it was discussed that the proposed model was an appropriate model of symmetry for human reasoning.

We studied auto–associative networks in which synapses are

noisy

on a time scale much shorter that the one for the neuron dynamics. In our model a presynaptic noise causes postsynaptic depression as recently observed in neurobiological systems. This results in a nonequilibrium condition in which the network sensitivity to an external stimulus is enhanced. In particular, the fixed points are qualitatively modified, and the system may easily scape from the attractors. As a result, in addition to pattern recognition, the model is useful for class identification and categorization.

A novel method for efficiently simulating large scale realistic neural networks is described. Most information transmission in these networks is accomplished by the so called action potentials, events which are considerably sparse and well-localized in time. This facilitates a dramatic reduction of the computational load through the application of the event-driven simulation schemes. However, some complex neuronal models require the simulator to calculate large expressions, in order to update the neuronal state variables between these events. This requirement slows down these neural state updates, impeding the simulation of very active large neural populations in real-time. Moreover, neurons of some of these complex models produce firings (action potentials) some time after the arrival of the presynaptic potentials. The calculation of this delay involves the computation of expressions that sometimes are difficult to solve analytically. To deal with these problems, our method makes use of precalculated lookup tables for both, fast update of the neural variables and the prediction of the firing delays, allowing efficient simulation of large populations with detailed neural models.

We develop a methodology for solving high dimensional dependency estimation problems between pairs of data types, which is viable in the case where the output of interest has very high dimension, e.g., thousands of dimensions. This is achieved by mapping the objects into continuous or discrete spaces, using joint kernels. Known correlations between input and output can be defined by such kernels, some of which can maintain linearity in the outputs to provide simple (closed form) pre-images. We provide examples of such kernels and empirical results.

The goal of combining the outputs of multiple models is to form an improved meta-model with higher generalization capability than the best single model used in isolation. Most popular ensemble methods do specify neither the number of component models nor their complexity. However, these parameters strongly influence the generalization capability of the meta-model. In this paper we propose an ensemble method which generates a meta-model with optimal values for these parameters. The proposed method suggests using resampling techniques to generate multiple estimations of the generalization error and multiple comparison procedures to select the models that will be combined to form the meta-model. Experimental results show the performance of the model on regression and classification tasks using artificial and real databases.

The aim of the paper is to develop hypothesis testing procedures both for variable selection and model adequacy to facilitate a model selection strategy for neural networks. The approach, based on statical inference tools, uses the subsampling to overcome the analytical and probabilistic difficulties related to the estimation of the sampling distribution of the test statistics involved. Some illustrative examples are also discussed.

Boosting constructs a weighted classifier out of possibly weak learners by successively concentrating on those patterns harder to classify. While giving excellent results in many problems, its performance can deteriorate in the presence of patterns with incorrect labels. In this work we shall use parallel perceptrons (PP), a novel approach to the classical committee machines, to detect whether a pattern’s label may not be correct and also whether it is redundant in the sense of being well represented in the training sample by many other similar patterns. Among other things, PP allow to naturally define margins for hidden unit activations, that we shall use to define the above pattern types. This pattern type classification allows a more nuanced approach to boosting. In particular, the procedure we shall propose, balanced boosting, uses it to modify boosting distribution updates. As we shall illustrate numerically, balanced boosting gives very good results on relatively hard classification problems, particularly in some that present a marked imbalance between class sizes.

When agent chooses some action and does state transition in present state in reinforcement learning, it is important subject to decide how will reward for conduct that agent chooses. In this paper, we suggest multi colony interaction ant reinforcement learning model using TD-error to original Ant-Q learning. This method is a hybrid of multi colony interaction by elite strategy and reinforcement learning applying TD-error to Ant-Q. We could know through an experiment that proposed reinforcement learning method converges faster to optimal solution than original ACS and Ant-Q.

The selection of the frequencies of the new hidden units for sequential Feed-forward Neural Networks (FNNs) usually involves a non-linear optimization problem that cannot be solved analytically. Most models found in the literature choose the new frequency so that it matches the previous residue as best as possible. Several exceptions to the idea of matching the residue perform an (implicit or explicit) orthogonalization of the output vectors of the hidden units. An experimental study of the aforementioned approaches to select the frequencies in sequential FNNs is presented. Our experimental results indicate that the orthogonalization of the hidden vectors outperforms the strategy of matching the residue, both for approximation and generalization purposes.

Many problems in pattern recognition are focused to learn one main task,

SingleTask

Learning

(STL). However, most of them can be formulated from learning several tasks related to the main task at the same time while using a shared representation,

MultitaskLearning

(MTL). In this paper, a new MLT architecture is proposed and its performance is compared with those obtained from other previous schemes used in MTL. This new MTL scheme makes use of private subnetworks to induce a bias in the learning process. The results provided from artificial and real data sets show how the use of this private subnetworks in MTL produces a better generalization capabilities and a faster learning.

A large class of problems requires real-time processing of complex temporal inputs in real-time. These are difficult tasks for state-of-the-art techniques, since they require capturing complex structures and relationships in massive quantities of low precision, ambiguous noisy data. A recently-introduced Liquid-State-Machine (LSM) paradigm provides a computational framework for applying a model of cortical neural microcircuit as a core computational unit in classification and recognition tasks of real-time temporal data. We extend the computational power of this framework by closing the loop. This is accomplished by applying, in parallel to the supervised learning of the readouts, a biologically-realistic learning within the framework of the microcircuit. This approach is inspired by neurobiological findings from ex-vivo multi-cellular electrical recordings and injection of dopamine to the neural culture. We show that by closing the loop we obtain a much more effective performance with the new Co-Evolutionary Liquid Architecture. We illustrate the added value of the closed-loop approach to liquid architectures by executing a speech recognition task.

In sparse nonnegative component analysis (sparse NMF) a given dataset is decomposed into a mixing matrix and a feature data set, which are both nonnegative and fulfill certain sparsity constraints. In this paper, we extend the sparse NMF algorithm to allow for varying sparsity in each feature and discuss the uniqueness of an involved projection step. Furthermore, the eligibility of the extended sparse NMF algorithm for blind source separation is investigated.

Radial Basis Neural Networks (RBNN) can approximate any regular function and have a faster training phase than other similar neural networks. However, the activation of each neuron depends on the euclidean distance between a pattern and the neuron center. Therefore, the activation function is symmetrical and all attributes are considered equally relevant. This could be solved by altering the metric used in the activation function (i.e. using non-symmetrical metrics). The Mahalanobis distance is such a metric, that takes into account the variability of the attributes and their correlations. However, this distance is computed directly from the variance-covariance matrix and does not consider the accuracy of the learning algorithm. In this paper, we propose to use a generalized euclidean metric, following the Mahalanobis structure, but evolved by a Genetic Algorithm (GA). This GA searches for the distance matrix that minimizes the error produced by a fixed RBNN. Our approach has been tested on two domains and positive results have been observed in both cases.

Neural networks are intended to be used in future nanoelectronics since these architectures seem to be robust against malfunctioning elements and noise. In this paper we analyze the robustness of radial basis function networks and determine upper bounds on the mean square error under noise contaminated weights and inputs.

Clustering algorithms have been applied in several disciplines successfully. One of those applications is the initialization of Radial Basis Functions (RBF) centers composing a Neural Network, designed to solve functional approximation problems. The Clustering for Function Approximation (CFA) algorithm was presented as a new clustering technique that provides better results than other clustering algorithms that were traditionally used to initialize RBF centers. Even though CFA improves performance against other clustering algorithms, it has some flaws that can be improved. Within those flaws, it can be mentioned the way the partition of the input data is done, the complex migration process, the algorithm’s speed, the existence of some parameters that have to be set in order to obtain good solutions, and the convergence is not guaranteed. In this paper, it is proposed an improved version of this algorithm that solves the problems that its predecessor has using fuzzy logic successfully. In the experiments section, it will be shown how the new algorithm performs better than its predecessor and how important is to make a correct initialization of the RBF centers to obtain small approximation errors.

In this paper we propose a new technique focused on the search of new architectures for modelling complex systems in function approximation problems, in order to avoid the exponential increase in the complexity of the system that is usual when dealing with many input variables. The new hierarchical network proposed, is composed of complete Radial Basis Function Networks (RBFNs) that are in charge of a reduced set of input variables. For the optimization of the whole net, we propose a new method to select the more important input variables, thus reducing the dimension of the input variable space for each RBFN. We also provide an algorithm which automatically finds the most suitable topology of the proposed hierarchical structure and selects the more important input variables for it. Therefore, our goal is to find the most suitable of the proposed families of hierarchical architectures in order to approximate a system from which a set of input/output (I/O) data has been extracted.

In this paper, we deal with the problem of function approximation from a given set of input/output data. This problem consists of analyzing these training examples so that we can predict the output of the model given new inputs. We present a new method for function approximation of the I/O data using radial basis functions (RBFs). This approach is based on a new efficient method of clustering of the centres of the RBF Network (RBFN); it uses the objective output of the RBFN to move the clusters instead of just the input values of the I/O data. This method of clustering, especially designed for function approximation problems, improves the performance of the approximator system obtained, compared with other models derived from traditional algorithms.

In this paper the behaviour of a multiobjective cooperative-coevolutive hybrid algorithm for the optimization of the parameters defining a Radial Basis Function Network developed by our group, is analyzed. In order to demonstrate the robustness of the behaviour of the presented methodology when the parameters of the algorithm are modified, a statistical analysis has been carried out. In the present contribution, the relevance and relative importance of the parameters involved in the design of the multiobjective cooperative-coevolutive hybrid algorithm presented are investigated by using a powerful statistical tool, the ANalysis Of the VAriance (ANOVA). To demonstrate the robustness of our algorithm, a functional approximation problem is investigated.

Carbon nanotubes are often seen as the only alternative technology to silicon transistors. While they are the most likely short-term alternative, other longer-term alternatives should be studied as well, even if their properties are less familiar to chip designers. While contemplating biological neurons as an alternative component may seem preposterous at first sight, significant recent progress in CMOS-neuron interface suggests this direction may not be unrealistic; moreover, biological neurons are known to self-assemble into very large networks capable of complex information processing tasks, something that has yet to be achieved with other emerging technologies.

The first step to designing computing systems on top of biological neurons is to build an abstract model of self-assembled biological neural networks, much like computer architects manipulate abstract models of transistors and circuits. In this article, we propose a first model of the

structure

of biological neural networks. We provide empirical evidence that this model matches the biological neural networks found in living organisms, and exhibits the

small-world

graph structure properties commonly found in many large and self-organized systems, including biological neural networks. More importantly, we extract the simple local rules and characteristics governing the growth of such networks, enabling the development of potentially large but realistic biological neural networks, as would be needed for complex information processing/computing tasks. Based on this model, future work will be targeted to understanding the evolution and learning properties of such networks, and how they can be used to build computing systems.

Solving a NP-Complete problem precisely is spiny: the combinative explosion is the ransom of this accurateness. It is the reason for which we have often resort to approached methods assuring the obtaining of a good solution in a reasonable time. In this paper we aim to introduce a new intelligent approach or meta-heuristic named “Bees Swarm Optimization”, BSO for short, which is inspired from the behaviour of real bees. An adaptation to the features of the MAX-W-SAT problem is done to contribute to its resolution. We provide an overview of the results of empirical tests performed on the hard Johnson benchmark. A comparative study with well known procedures for MAX-W-SAT is done and shows that BSO outperforms the other evolutionary algorithms especially AC-SAT, an ant colony algorithm for SAT.

Soft topology-preserving map and its batch version are proven to be reduced to cortical map and elastic net, respectively. This verifies numerous results of numerical simulations described in the literature demonstrating similarities of neural patterns produced by lateral and elastic synaptic interactions.

This work deals with the problem of automatically obtaining ANNs that cooperate in modelling of complex functions. We propose an algorithm where the combination of networks takes place at the phenotypic operational level. Thus, we evolve a population of networks that are automatically classified into different species depending on the performance of their phenotype, and individuals of each species cooperate forming a group to obtain a complex output. The components that make up the groups are basic ANNs (primitives) and could be reused in other search processes as seeds or could be combined to generate new solutions. The magnitude that reflects the difference between ANNs is their affinity vector, which must be automatically created and modified. The main objective of this approach is to model complex functions such as environment models in robotics or multidimensional signals.

The Growing Hierarchical Self Organizing Map (GHSOM) was introduced as a dynamical neural network model that adapts its architecture during its unsupervised training process to represents the hierarchical relation of the data.

However, the dynamical algorithm of the GHSOM is sensitive to the presence of noise and outliers, and the model will no longer preserve the topology of the data space as we will show in this paper. The outliers introduce an influence to the GHSOM model during the training process by locating prototypes far from the majority of data and generating maps for few samples data. Therefore, the network will not effectively represent the topological structure of the data under study.

In this paper, we propose a variant to the GHSOM algorithm that is robust under the presence of outliers in the data by being resistant to these deviations. We call this algorithm Robust GHSOM (RGHSOM). We will illustrate our technique on synthetic and real data sets.

The web contains rich and dynamic collections of hyperlink information, web page access, and usage information providing rich sources for data mining. From this, we need a system to recommend a visitor good information. This recommendation system can be constructed by web usage mining process. The web usage mining mines web log records to discover user access patterns of web pages. Also it is the application of data mining techniques to large web log data in order to extract usage patterns from user’s click streams. In general, the size of web log records is so large that we have difficulty to analyze web log data. To make matter worse, the web log records are very sparse. So it is very hard to estimate the dependency between the web pages. In this paper, we solved this difficulty of web usage mining using support vector machine. In the experiments, we verified our proposed method by given data from UCI machine learning repository and KDD cup 2000.

This paper presents a method to iteratively grow a compact Support Vector Regressor so that the balance between size of the machine and its performance can be user-controlled. The algorithm is able to combine Gaussian kernels with different spread parameter, skipping the ‘a priori’ parameter estimation by allowing a progressive incorporation of nodes with decreasing values of the spread parameter, until a cross-validation stopping criterion is met. Experimental results show the significant reduction achieved in the size of the machines trained with this new algorithm and their good generalization capabilities.

Cellular neural networks (CNNs) introduced by Chua and Yang in 1988 are recurrent artificial neural networks. Due to their cyclic connections and to the neurons’ nonlinear activation functions, recurrent neural networks are nonlinear dynamic systems, which display stable and unstable fixed points, limit cycles and chaotic behavior. Since the field of neural networks is still a recent one, improving the stability conditions for such systems is an obvious and quasi-permanent task. This paper focuses on CNNs affected by time delays. We are interested to obtain sufficient conditions for the asymptotic stability of a cellular neural network with time delay feedback and zero control templates. Due to their sector restricted nonlinearities, stability of the neural networks is strongly connected to robust stability. With respect to this we shall use a quadratic Liapunov functional constructed

via

the technique due to V. L. Kharitonov for uncertain linear time delay systems, combined with an approach suggested by Malkin for systems with sector restricted nonlinearities.

Global exponential stability of cellular neural networks with time-varying coefficients and delays is considered in this paper. By utilizing a delay differential inequality, a new sufficient condition ensuring global exponential stability for cellular neural networks with time-varying coefficients and delays is presented. Since the condition does not require that the delay function be differentiable or the coefficients be bounded, the results here improve and extend those given in the earlier literature.

Being based on the theory of evolution and natural selection, the Genetic Algorithms (GA) represent a technique that has been proved as good enough for the resolution of those problems that require a search through a complex space of possible solutions. The maintenance of a population of possible solutions that are in constant evolution may lead to its diversity being lost, consequently it would be more difficult, not only the achievement of a final solution but also the supply of more than one solution The method that is described here tries to overcome those difficulties by means of a modification in traditional GA’s. Such modification involves the inclusion of an additional population that might avoid the mentioned loss of diversity of classical GA’s. This new population would also provide the piece of exhaustive search that allows to provide more than one solution.

We propose the identification of fuzzy systems with the aid of genetic fuzzy granulation to carry out the model identification of complex and nonlinear systems. The proposed fuzzy model implements system structure and parameter identification with the aid of genetic algorithms and information granulation. To identify the structure of fuzzy rules we use genetic algorithms. Granulation of information realized with Hard C-Means clustering help determine the initial parameters of fuzzy model such as the initial apexes of the membership functions and the initial values of polynomial functions being used in the premise and consequence part of the fuzzy rules. And the initial parameters are tuned effectively with the aid of the genetic algorithms and the least square method. An example is given to evaluate the validity of the proposed model.

TSK models are a very powerful tool for function approximation problems given a dataset of input/output data. Given a global error function to approximate, there are several methodologies for training (adjust the parameters and find the optimal structure) the TSK model. Nevertheless, this strategy implies that the interpretability of the rules comprising the neuro-fuzzy TSK system as linearizations of the nonlinear subjacent system can be lost. Several recent works have addressed this problem with partial success, sometimes performing a tradeoff between global accuracy and local models interpretability. In this paper we propose an accurate modified TSK neuro-fuzzy model for function approximation that solves the cited problem, and that furthermore allows us to interprete the output of the rules as the Taylor Series Expansion of the system output around the rule centres.

We introduce an advanced architecture of genetically optimized Hybrid Fuzzy Neural Networks (gHFNN) and develop a comprehensive design methodology supporting their construction. The gHFNN architecture results from a synergistic usage of the hybrid system generated by combining Fuzzy Neural Networks (FNN) with Polynomial Neural Networks (PNN). We distinguish between two types of the linear fuzzy inference rule-based FNN structures showing how this taxonomy depends upon the type of a fuzzy partition of input variables. As to the consequence part of the gHFNN, the development of the PNN dwells on two general optimization mechanisms: the structural optimization is realized via GAs whereas in case of the parametric optimization we proceed with a standard least square method-based learning.

In this paper, we introduce a neo scheme of fuzzy-neural networks – Fuzzy Polynomial Neural Networks (FPNN) with a new fuzzy set-based polynomial neurons (FSPNs) whose fuzzy rules include the information granules (about the real system) obtained through Information Granulation(IG). We investigate the proposed networks from two different aspects to improve the performance of the fuzzy-neural networks. First, We have developed a design methodology (genetic optimization using Genetic Algorithms) to find the optimal structure for fuzzy-neural networks that expanded from Group Method of Data Handling (GMDH). It is the number of input variables, the order of the polynomial, the number of membership functions, and a collection of the specific subset of input variables that are the parameters of FPNN fixed by aid of genetic optimization that has search capability to find the optimal solution on the solution space. Second, we have been interested in the architecture of fuzzy rules that mimic the real world, namely sub-model (node) composing the fuzzy-neural networks. We adopt fuzzy set-based fuzzy rules as substitute for fuzzy relation-based fuzzy rules and apply the concept of Information Granulation to the proposed fuzzy set-based rules. The performance of genetically optimized FPNN (gFPNN) with fuzzy set-based polynomia neurons (FSPNs) composed of fuzzy set-based rules is quantified through experimentation where we use a number of modeling benchmarks data which are already experimented with in fuzzy or neurofuzzy modeling.

This work introduces two new neuro-fuzzy systems for intelligent agents called Reinforcement Learning – Hierarchical Neuro-Fuzzy Systems BSP (RL-HNFB) and Reinforcement Learning – Hierarchical Neuro-Fuzzy Systems Politree (RL-HNFP). By using hierarchical partitioning methods, together with the Reinforcement Learning (RL) methodology, a new class of Neuro-Fuzzy Systems (SNF) was obtained, which executes, in addition to automatically learning its structure, the autonomous learning of the actions to be taken by an agent. These characteristics have been developed in order to bypass the traditional drawbacks of neuro-fuzzy systems. The paper details the two novel RL_HNF systems and evaluates their performance in a benchmark application – the cart-centering problem. The results obtained demonstrate the capacity of the proposed models in extracting knowledge from the agent’s direct interaction with large and/or continuous environments.

In order to extract learning algorithms from living neural aggregates it would be advantageous to achieve one-to-one neuron-electrode interfacing with in vitro networks. Towards this goal, we have developed a hybrid glass-elastomer technology, which allows topology specification in small networks (of the order of 10 neurons) and recording of extracellular potentials from individual neurites grown through microfluidic channels. Here we report on progress towards adhesion-free placement of cells within microwells, promotion of neurite growth and recording of intra-channel extracellular spikes.

Based on explicit numerical constructions for Kolmogorov’s superpositions (KS) linear

size

circuits are possible. Because classical Boolean as well as threshold logic implementations require exponential

size

in the worst case, it follows that

size

-optimal solutions for arbitrary Boolean functions (BFs) should rely (at least partly) on KS. In this paper, we will present previous theoretical results while examining the particular case of 3-input BFs in detail. This shows that there is still room for improvement on the synthesis of BFs. Such

size

reductions (which can be achieved systematically) could help alleviate the challenging power consumption problem, and advocate for the design of Kolmogorov-inspired gates, as well as for the development of the theory, the algorithms, and the CAD tools that would allow taking advantage of such optimal combinations of different logic styles.

This presentation has two goals: (i) to review the recently suggested concept of bio-inspired CrossNet architectures for future hybrid CMOL VLSI circuits and (ii) to present new results concerning the prospects and problems of using these neuromorphic networks as classifiers of very large patterns, in particular of high-resolution optical images. We show that the unparalleled density and speed of CMOL circuits may enable to perform such important and challenging tasks as, for example, online recognition of a face in a high-resolution image of a large crowd.

An analog VLSI implementation of adaptive synapses being part of an associative memory realised with pulsed neurons is presented. VLSI implementations of dynamic synapses and pulsed neurons are expected to provide robustness and low energy consumption like observed in the human brain. We have developed a VLSI implementation of synaptic connections for an associative memory which is used in a biological inspired image processing system using pulse coded neural networks. The system consists of different layers for feature extraction to decompose the image in several features. The pulsed associative memory is used for completing or binding features. In this paper, we focus on the dynamics and the analog implementation of adaptive synapses. The discussed circuits were designed in a 130 nm CMOS process.

This work shows the design and application of a mixed-mode analog-digital neural network circuit for sensor conditioning applications. The proposed architecture provides a high extension of the linear range for non-linear output sensors as negative temperature coefficient resistors (NTC) or giant magnetoresistive (GMR) angular position sensors, by using analog current-mode circuits with digital 8-bit weight storage. We present an analog current-based neuron model with digital weights, showing its architecture and features. By modifying the algorithm used in off-chip weight fitting, main differences of the electronic architecture, compared to the ideal model, are compensated. A small neural network based on the proposed architecture is applied to improve the output of NTC thermistors and GMR sensors, showing good results. Circuit complexity and performance make these systems suitable to be implemented as on-chip compensation modules.

A computing platform is described for simulating arbitrary networks of spiking neurons in real time. A hybrid computing scheme is adopted that uses both software and hardware components. We focus on conductance-based models for neurons that emulate the temporal dynamics of the synaptic integration process. We have designed an efficient computing architecture using reconfigurable hardware in which the different stages of the neuron model are processed in parallel (using a customized pipeline structure). Further improvements occur by computing multiple neurons in parallel using multiple processing units. The computing platform is described and its scalability and performance evaluated. The goal is to investigate biologically realistic models for the control of robots operating within closed perception-action loops.

Address-Event-Representation (AER) is a communication protocol for transferring images between chips, originally developed for bio-inspired image processing systems. Such systems may consist of a complicated hierarchical structure with many chips that transmit images among them in real time, while performing some processing (for example, convolutions). In developing AER based systems it is very convenient to have available some kind of means of generating AER streams from on-computer stored images. In this paper we present a hardware method for generating AER streams in real time from a sequence of images stored in a computer’s memory. The Kolmogorov-Smirnov test has been applied to quantify that this method follows a Poisson distribution of the spikes. A USB-AER board and a PCI-AER board, developed by our RTCAR group, have been used.

In this paper we analyse a serial (ripple carry) and a parallel (Kogge-Stone) adder when operating in subthreshold at 100nm and 70nm. These are targeted for ultra low power consumption applications. The elementary gates used are threshold logic gates (perceptrons). Simulations have been performed both with and without considering the delay on the wires. These simulations confirm that wires play a significant role, reducing the speed advantage of the parallel adder (over the serial one) from 4.5x to 2.2–2.4x. A promising result is that the speed of both adders improves more than 10x when migrating from 100nm to 70nm. The full adder based on threshold logic gates (used in the ripple carry adder) improves on previously known full adders, achieving 1.6fJ when operated at 200mV in 120nm CMOS. Finally, the speed of the parallel adder can be matched by the serial adder when operating at only 10–20% higher

V

dd

, while still requiring less power and energy.

In this paper, a fully functional prototype of an asynchronous 4-to-4 Address Event Representation (AER) mapper is presented. AER is an event driven communication protocol originally used in VLSI implementations of neural networks to transfer action potentials between neurons. Often, this protocol is used for direct inter-chip communication between neuromorphic chips containing assemblies of neurons. Without an active device between two such chips, the network connections between them are hard-wired in the chip design. More flexibility can be achieved by communicating through an AER mapper: The network can freely be configured and, furthermore, several AER busses can be merged and split to form a complex network structure. We present here an asynchronous AER mapper which offers an easy and versatile solution. The AER mapper receives input from four different AER busses and redirects the input AE to four output AER busses. The control circuitry is implemented on an FPGA and is fully asynchronous, and pipelining is used to maximize throughput. The mapping is performed according to preprogrammed lookup tables, which is stored on external RAM. The mapper can emulate a network of up to 2

19

direct connections and test results show that the mapper can handle as much as 30× 10

6

events/second.

This paper reports the recent steps to the attainment of a compact high-speed optoelectronic neuroprocessor based on an optical broadcast architecture that is used as the processing core of a vision system. The optical broadcast architecture is composed of a set of electronic processing elements that work in parallel and whose input is introduced by means of an optical sequential broadcast interconnection. Because of the special characteristics of the architecture, that exploits electronics for computing and optics for communicating, it is readily scalable in number of neurons and speed, thus improving the performance of the vision system. This paper focuses on the improvement of the optoelectronic system and electronic neuron design to increase operation speed with respect to previous designs.

Recent advances in arrays of microelectrodes open the door to both better understanding of the way the brain works and to the restoration of damaged perceptive and motor functions. In the case of sensorial inputs, direct multi-channel interfacing with the brain for neuro-stimulation requires a computational layer capable of handling the translation from external stimuli into appropriate trains of spikes. The work here presented aims to provide automated and reconfigurable transformation of visual inputs into addresses of microelectrodes in a cortical implant for the blind. The development of neuroprostheses such as this one will contribute to reveal the neural language of the brain for the representation of perceptions, and offers a hope to persons with deep visual impairments. Our system serves as a highly flexible test-bench for almost any kind of retina model, and allows the validation of these models against multi-electrode recordings from experiments with biological retinas. The current version is a PC-based platform, and a compact stand-alone device is under development for the autonomy and portability required in chronic implants. This tool is useful for psychologists, neurophysiologists, and neural engineers as it offers a way to deal with the complexity of multi-channel electrical interfaces for the brain.

Address-Event-Representation (AER) is a communication protocol for transferring spikes between bio-inspired chips. Such systems may consist of a hierarchical structure with several chips that transmit spikes among them in real time, while performing some processing. To develop and test AER based systems it is convenient to have a set of instruments that would allow to: generate AER streams, monitor the output produced by neural chips and modify the spike stream produced by an emitting chip to adapt it to the requirements of the receiving elements. In this paper we present a set of tools that implement these functions developed in the CAVIAR EU project.

This paper describes a tool devised for automatic design of bioinspired visual processing models using reconfigurable digital hardware. The whole system is indicated for the analysis of vision models, especially those with real–time requirements. We achieve a synthesizable FPGA/ASIC design starting from a high level description of a retina, which is made and simulated through an ad-hoc program. Our tool allows a thorough simulation of the visual model at different abstraction levels, from functional simulation of the visual specifications up to hardware-oriented simulation of the developed FPGA model. The main objective of this work is to build a portable and flexible system for a visual neuro-prosthesis and to stimulate efficiently an array of intra–cortical implanted microelectrodes. A set of parameters can be adjusted in every step of the design flow in order to maximize the design flexibility of the model. Furthermore these parameters allow the different scientists who have to deal with the development to modify a well known characteristic.

Address-Event-Representation (AER) is a communications protocol for transferring images between chips, originally developed for bio-inspired image processing systems. In [6], [5] various software methods for synthetic AER generation were presented. But in neuro-inspired research field, hardware methods are needed to generate AER from laptop computers. In this paper two real time implementations of the exhaustive method, proposed in [6], [5], are presented. These implementations can transmit, through AER bus, images stored in a computer using USB-AER board developed by our RTCAR group for the CAVIAR EU project.

In this paper, we propose a Parallel Genetic Algorithm (PGA) based on a modified survival method and discuss its efficient implementation. For parallel computation, we use a hybrid distributed architecture based on the coarse-grain and fine-grain. Moreover, we propose a modified survival-based GA using tournament selection method. To show the validity of a proposed PGA, we evaluate its performance with optimization problems such as DeJong’s functions, mathematical function, and set covering problem. In addition, we implement a PGA processor with ALTERA EP2A40672F FPGA device. The experimental results will be shown that proposed PGA remarkably improves the speed of finding optimal solution than single GAP.

This paper presents a strategy for the implementation of large scale spiking neural network topologies on FPGA devices based on the I&F conductance model. Analysis of the logic requirements demonstrate that large scale implementations are not viable if a fully parallel implementation strategy is utilised. Thus the paper presents an alternative approach where a trade off in terms of speed/area is made and time multiplexing of the neuron model implemented on the FPGA is used to generate large network topologies. FPGA implementation results demonstrate a performance increase over a PC based simulation.

Field Programmable Gate Arrays (FPGAs) are being used as platforms for the digital implementation of intelligent systems. Binary digital systems provide an accurate, robust, stable performance that is free from the drift and manufacturing tolerances associated with analogue systems. However binary systems have a much lower functional density than their analogue counterparts resulting in inefficient use of silicon surface area. A design for a novel Configurable Logic Block (CLB) is presented which retains the robust qualities of digital processing whilst providing increased functional density. The circuit design uses Si/SiGe Inter-band Tunneling Diodes (ITDs) and NMOS/CMOS transistors to create quaternary memory cells in a topology and architecture suited to the implementation of neural networks. The performance of the CLB is simulated in HSPICE and the results are presented.

In this contribution, we describe a hardware platform for evolving a fuzzy system by using Fuzzy CoCo — a cooperative coevolutionary methodology for fuzzy system design — in order to speed up both evolution and execution. Reconfigurable hardware arises between hardware and software solutions providing a trade-off between flexibility and performance. We present an architecture that exploits the dynamic partial reconfiguration capabilities of recent FPGAs so as to provide adaptation at two different levels: major structural changes and fuzzy parameter tuning.

This contribution presents the hardware implementation of a neural system, which is a variant of a Hopfield network, modified to perform parametric identification of dynamical systems, so that the resulting network possess time-varying weights. The implementation, which is accomplished on FPGA circuits, is carefully designed so that it is able to deal with these dynamic weights, as well as preserve the natural parallelism of neural networks, at a limited cost in terms of occupied area and processing time. The design achieves modularity and flexibility, due to the usage of parametric VHDL to describe the network. The functional simulation and the synthesis show the viability of the design, whose refinement will lead to the development of an embedded adaptive controller for autonomous systems.

The architecture of a general purpose fuzzy logic coprocessor and its implementation on an FPGA based System on Chip is described. Thanks to its ability to support a fast dynamic reconfiguration of all its parameters, it is suitable for implementing adaptive fuzzy logic algorithms, or for the execution of different fuzzy algorithms in a time sharing fashion. The high throughput obtained using a pipelined structure and the efficient data organization allows significant increase of the computational capabilities strongly desired in applications with hard real-time constraints.

Neural network ensembles are widely use for classification and regression problems as an alternative to the use of isolated networks. In many applications, ensembles has proven a performance above the performance of just one network.

In this paper we present a new approach to neural network ensembles that we call “cascade ensembles”. The approach is based on two ideas: (i) the ensemble is created constructively, and (ii) the output of each network is fed to the inputs of the subsequent networks. In this way we make a cascade of networks.

This method is compared with standard ensembles in several problems of classification with excellent performance.

As shown in the bibliography, training an ensemble of networks is an interesting way to improve the performance with respect to a single network. However there are several methods to construct the ensemble. In this paper we present some new results in a comparison of twenty different methods. We have trained ensembles of 3, 9, 20 and 40 networks to show results in a wide spectrum of values. The results show that the improvement in performance above 9 networks in the ensemble depends on the method but it is usually low. Also, the best method for a ensemble of 3 networks is called “Decorrelated” and uses a penalty term in the usual Backpropagation function to decorrelate the network outputs in the ensemble. For the case of 9 and 20 networks the best method is conservative boosting. And finally for 40 networks the best method is Cels.

Among visual processings in the visual networks, movement detections are carried out in the visual cortex. The visual cortex for the movement detection, consist of two layered networks, called the primary visual cortex (V1), followed by the middle temporal area (MT). In the biological visual neural networks, a characteristic feature is nonlinear functions, which will play important roles in the visual systems. In this paper, V1 and MT model networks, are decomposed into sub-asymmetrical networks. By the optimization of the asymmetric networks, movement detection equations are derived. Then, it was clarified that asymmetric networks with the even-odd nonlinearity combined , are fundamental in the movement detection. These facts are applied to two layered V1 and MT networks, in which it was clarified that the second layer MT has an efficient ability to detect the movement.

We present a novel technique to reduce the computational burden associated to the operational phase of neural networks. To get this, we develop a very simple procedure for fast classification that can be applied to any network whose output is calculated as a weighted sum of terms, which comprises a wide variety of neural schemes, such as multi-net networks and Radial Basis Function (RBF) networks, among many others. Basically, the idea consists on sequentially evaluating the sum terms, using a series of thresholds which are associated to the confidence that a partial output will coincide with the overall network classification criterion. The possibilities of this strategy are well-illustrated by some experiments on a benchmark of binary classification problems, using RealAdaboost and RBF networks as the underlying technologies.

In this article it is made a study of the characterization capacity and synthesis of objects of the self-organizing neural models. These networks, by means of their competitive learning, try to preserve the topology of an input space. This capacity is being used for the representation of objects and their movement with topology preserving networks. We characterized the object to represent by means of the obtained maps and kept information solely on the coordinates and the colour from the neurons. From this information it is made the synthesis of the original images, applying mathematical morphology and simple filters on the information which it is had.

In this paper we analyze the performance of several evolutionary algorithms in the feature and instance selection problem. It is also introduced the ARGEN + AREPO search algorithm which has been tested in the same problem. There is no need to adapt parameters in this genetic algorithm, except the population size. The reported preliminary results show that using this technique in a wrapper model to search data subsets, we can obtain similar accuracy like with the hill-climbers and genetic algorithms models also here presented, but keeping a less amount of data.

Accounts of perceptual decision making, such as evidence accrual models, represent mental states as noisy numerical vectors describing the stimuli. As such, these are not biological models. An alternative scheme is presented in which mental states are represented as functions. This generalises an analogue coding scheme for numbers, and might be biologically implemented as functions of cortical activity. Some properties of this representation are illustrated in modelling accuracy and response time patterns observed in a classic experiment into perceptual processes.

An associative neural network with chaotic neuron model and synaptic depression (CSDNN) is constructed. Memory switching phenomenon in the network is demonstrated. Simulation results show that with various parameter value settings and with various initial conditions, the memory retrieval frequency of CSDNN distributes uniformly among the stored patterns, and the rate of memory retrieval of CSDNN is much higher than that of a chaotic neural network. The possible utilization of memory retrieval in CSDNN is also discussed.

In stereoscopic vision, there are two artificial eyes implemented so that it can obtain two separate views of the scene and simulate the binocular depth perception of human beings. Traditionally, camera calibration and 3D reconstruction of such a vision sensor are performed by geometrical solutions. However, the traditional camera model is very complicated since nonlinear factors in it and needs to approximate the light projection scheme by a number of parameters. It is even very difficult to model some highly distorted vision sensors, such as fish-eye lens. In order to simplify both the camera calibration and 3D reconstruction procedures, this work presents a method based on neural networks which is brought forward according to the characteristics of neural network and stereoscopic vision. The relation between spatial points and image points is established by training the network without the parameters of the cameras, such as focus, distortions besides the geometry of the system. The training set for our neural network consists of a variety of stereo-pair images and corresponding 3D world coordinates. Then the 3D reconstruction of a new s cene is simply using the trained network. Such a method is more similar to how human’s eyes work. Simulations and real data are used to demonstrate and evaluate the procedure. We observe that the errors obtained our experimentation are accurate enough for most machine-vision applications.

This paper shows how Evolutionary Algorithm (EA) robustness help to solve a difficult problem with a minimal expert knowledge about it. The problem consist in the design of a Brain-Computer Interface (BCI), which allows a person to communicate without using nerves and muscles. Input electroencephalographic (EEG) activity recorded from the scalp must be translated into outputs that control external devices. Our BCI is based in a Multilayer Perceptron (MLP) trained by an EA. This kind of training avoids the main problem of MLPs training algorithms: overfitting. Experimental results produceMLPs with a classification ability better than those in the literature.

This paper proposes a new vision-based system that can extract walking parameters from human demonstration. The system uses only a non-calibrated USB webcam connected to a standard PC, and the human is only required to put three color patches on one of his legs and walk roughly in a perpendicular plane with respect to camera orientation. The walking parameters are then extracted in real time, using a local tracking system to follow the markers and a fast decision layer to detect the main features of the leg movement. As only one leg can be tracked properly using only one camera, we assume symmetric movement for left and right legs. Once extracted, the parameters have been successfully tested by generating walking sequences for both simulated and real Robo-Erectus humanoid robots.

Hand-design of control systems for autonomous robots that act in dynamic or noisy environments is a complex task.

In this paper, a new technique for controller design, termed decisionvector, is presented. An evolutionary approach is proposed: the control systems (candidate solutions) are made up of the set of robot states with respect to the obstacles it can detect, and the corresponding actions to take on each one of those situations.

This initial work carries out the evolution of controllers in two environments, so that it is clear that, in spite of the simplicity of the proposed model, it is powerful enough to guide the robot to reach a target avoiding obstacles, and even, tracking a spread mark on the ground.

Two-link robot arm model is extensively used in literatures for that it is simple enough to simulate conveniently, yet contains all the nonlinear terms arising in general n-link manipulators. And neural networks are reported to be computationally efficient compared with traditional PID control and adaptive control. However, when a neural network is applied, one of the key step is to choose the optimal number of neurons. In this paper, a relative large number of neurons are initially used, which is pruned during the training. The conic sector theory is introduced in the design of this robust neural control system, which aims at providing guaranteed boundedness for both the input-output(I/O) signals and the weights of the neural network.

We propose a technique to speed up the learning of the inverse kinematics of a robot manipulator by decomposing it into two or more virtual robot arms. Unlike previous decomposition approaches, this one does not place any requirement on the robot architecture and, thus, it is completely general. Parametrized Self-Organizing Maps (PSOM) are particularly adequate for this type of learning, and permit comparing results obtained directly and through the decomposition. Experimentation shows that time reductions of up to two orders of magnitude are easily attained.

The implementation in a robot of the coordination between different sensors and actuators in order to achieve a task requires a high formulation and modelisation effort, specially when the number of sensors/actuators and degrees of freedom available in the robot is huge. This paper introduces a highly distributed architecture that is independent from the robot platform, capable of the generation of such a coordination in an automatic way by using evolutionary methods. The architecture is completely neural network based and it allows the control of the whole robot for, in principle, any type of task based on sensory-motor coordination. The article shows how the proposed architecture is capable of controlling an Aibo robot for the performance of three different difficult tasks (standing, standing up and walking) using exactly the same neural distribution. It is also expected that it will be directly scalable for higher levels of control and general design in evolutionary robotics.

Combining previous experience and knowledge to contemplate tasks of increasing complexity is one of the most interesting problems in autonomous robotics. Here we present an ANN based modular architecture that uses the concept of modulation to increase the possibilities of reusing previously obtained modules. A first approximation to the modulation of the actuators was tested in a previous paper where we showed how it was useful to obtain more complex behaviours that those obtained using only activation / inhibition. In this paper we extend the concept to sensor modulation, which enables the architecture to easily modify the required behaviour for a module, we show how both types of modulation can be used at the same time and how the activation / inhibition can be seen as a particular case of modulation. Some examples in a real robot illustrate the capabilities of the whole architecture.

In the field of explorative data analysis self-organizing maps have been used successfully for a lot of applications. In our case, we apply the self-organizing map for the analysis of semiconductor fabrication data by training recorded high dimensional data sets. Usually, the training result is displayed by using appropriate visualization techniques and the results are evaluated manually. Especially for large data sets an automated post-processing of the training result is essential. In this paper an automatic training result analysis based on specific image processing is introduced. Dependencies between components maps are calculated by structure overlapping analysis based on the segmentation of component maps. This novel method has been integrated into the data analysis software DanI, that simulates self-organizing maps for data analysis with several pre-processing and post-processing capabilities.

Profile Hidden Markov Models (Profile HMM) are well suited to modelling multiple alignment and are widely used in molecular biology. Usually, heuristic algorithms such as Baum-Welch are used to estimate the model parameters. However, Baum-Welch has a tendency to stagnate on local optima. A more involved approach is to use some form of stochastic search algorithm that ‘bumps’ Baum-Welch off from local maxima. In this paper, a hybrid genetic algorithm is presented for training profile HMM (hybrid GA-HMM training) and producing multiple sequence alignment from groups of unaligned protein sequences. The quality of the alignments produced by hybrid GA-HMM training is compared to that by the other Profile HMM training methods. The experimental results prove very competitive with and even better than the other tested profile HMM training methods. Analysis of the behavior of the algorithm sheds light on possible improvement.

In this work, we suggest a new feature selection technique that lets us use the wrapper approach for finding a well suited feature set for distinguishing experiment classes in high dimensional data sets. Our method is based on the relevance and redundancy idea, in the sense that a ranked-feature is chosen if additional information is gained by adding it. This heuristic leads to considerably better accuracy results, in comparison to the full set, and other representative feature selection algorithms in twelve well–known data sets, coupled with notable dimensionality reduction.

The PCASOM is a novel self-organizing neural model that performs Principal Components Analysis (PCA). It is also related to the ASSOM network, but its training equations are simpler. The PCASOM has the ability to learn self-organizing maps of the means and correlations of complex input distributions. Here we propose a method to extend this capability to build intrinsic dimensionality maps. These maps model the underlaying structure of the input. Experimental results are reported, which show the self-organizing map formation performed by the proposed network.

Modern data analysis tools have to work on high-dimensional data, whose components are not independently distributed. High-dimensional spaces show surprising, counter-intuitive geometrical properties that have a large influence on the performances of data analysis tools. Among these properties, the concentration of the norm phenomenon results in the fact that Euclidean norms and Gaussian kernels, both commonly used in models, become inappropriate in high-dimensional spaces. This papers presents alternative distance measures and kernels, together with geometrical methods to decrease the dimension of the space. The methodology is applied to a typical time series prediction example.

In real-world database, generally, attribute values of objects are numerical, from real-world practice point of view, numeral is too detail to obtaining good information or decision. Hence, a linguistic data summary of a set of data, which is expressed by a sentence or a small number of sentences in a natural language, would be very desirable and human consistent. In this paper, from the structure and valuation of fuzzy statement point of view, extracting linguistic data summarize is discussed. To extract complex linguistic data summaries, a new aggregation operator for aggregating linguistic terms is proposed, a numerical example of Personnel Database is also provided.

In Machine Learning, ensembles are combination of classifiers. Their objective is to improve the accuracy. In previous works, we have presented a method for the generation of ensembles, named rotation-based. It transforms the training data set; it groups, randomly, the attributes in different subgroups, and applies, for each group, an axis rotation. If the used method for the induction of the classifiers is not invariant to rotations in the data set, the generated classifiers can be very different. In this way, different classifiers can be obtained (and combined) using the same induction method.

The bias-variance decomposition of the error is used to get some insight into the behaviour of a classifier. It has been used to explain the success of ensemble learning techniques. In this work the bias and variance for the presented and other ensemble methods are calculated and used for comparison purposes.

The incompleteness of data is a most common source of uncertainty in real-world Data Mining applications. The management of this uncertainty is, therefore, a task of paramount importance for the data analyst. Many methods have been developed for missing data imputation, but few of them approach the problem of imputation as part of a general data density estimation scheme. Amongst the latter, a method for imputing and visualizing multivariate missing data using Generative Topographic Mapping was recently presented. This model and some of its extensions are tested under different experimental conditions. Its performance is compared to that of other missing data imputation techniques, thus assessing its relative capabilities and limitations.

In this paper we present CIDIM (Control of Induction by sample DIvision Method), an algorithm that has been developed to induce small and accurate decision trees using a set of examples. It uses an internal control of induction to stop the induction and to avoid the overfitting. Other ideas like a dichotomic division or groups of consecutive values are used to improve the performance of the algorithm. CIDIM has been successfully compared with ID3 and C4.5. It induces trees that are significantly better than those induced by ID3 or C4.5 in almost every experiment.

The supervised learning methods applying evolutionary algorithms to generate knowledge model are extremely costly in time and space. Fundamentally, this high computational cost is fundamentally due to the evaluation process that needs to go through the whole datasets to assess their goodness of the genetic individuals. Often, this process carries out some redundant operations which can be avoided. In this paper, we present an example reduction method to reduce the computational cost of the evolutionary learning algorithms by means of extraction, storage and processing only the useful information in the evaluation process.

Differential Evolution (DE) algorithm is a new heuristic approach which has been proposed particulary for numeric optimization problems. It is a population based algorithm like genetic algorithms using the similar operators; crossover, mutation and selection. In this work, DE algorithm has been applied to the design of fixed point digital Finite Impuls Response (FIR) filters and its performance has been compared to that of Genetic Algorithm (GA) and Least Squares Algorithm (LSQ).

In many practical applications, the data is organized along a manifold of lower dimension than the dimension of the embedding space. This additional information can be used when learning the model parameters of Gaussian mixtures. Based on a mismatch measure between the Euclidian and the geodesic distance, manifold constrained responsibilities are introduced. Experiments in density estimation show that manifold Gaussian mixtures outperform ordinary Gaussian mixtures.

This paper is concerned with the comparison of three types of Gaussian based Artificial Neural Networks in the very high dimensionality classification problems found in hyperspectral signal processing. In particular, they have been compared for the spectral unmixing problem given the fact that the requirements for this type of classification are very different from other realms in two aspects: there are usually very few training samples leading to networks that are very easily overtrained, and these samples are not usually representative in terms of sampling the whole input-output space. The networks selected for comparison go from the classical Radial Basis Function (RBF) network to the more complex Gaussian Synapse Based Network (GSBN) considering an intermediate type, the Radial Basis Function with Multiple Deviation (RBFMD). The comparisons were carried out when processing a benchmark set of synthetic hyperspectral images containing mixtures of spectra from materials found in the US Geological Service database.

A robust and effective voice activity detection (VAD) algorithm is proposed for improving speech recognition performance in noisy environments. The approach is based on filtering the input channel to avoid high energy noisy components and then the determination of the speech/non-speech bispectra by means of third order auto-cumulants. This algorithm differs from many others in the way the decision rule is formulated (detection tests) and the domain used in this approach. Clear improvements in speech/non-speech discrimination accuracy demonstrate the effectiveness of the proposed VAD. It is shown that application of statistical detection test leads to a better separation of the speech and noise distributions, thus allowing a more effective discrimination and a tradeoff between complexity and performance. The algorithm also incorporates a previous noise reduction block improving the accuracy in detecting speech and non-speech.

Due to the demand for cellular wireless services, recent interests are in techniques, which can improve the capacity of CDMA systems. On such technique is multi-user detection. Multi-user Detection (MUD) is the intelligent estimation/demodulation of transmitted bits in the presence of Multiple Access Interference (MAI). In this paper, we will show the role of matched filter used as pre-processing tool for MUD in DS-CDMA system.

In this paper we propose an endpoint detection system based on the use of several features extracted from each speech frame, followed by a robust classifier (i.e Adaboost and Bagging of decision trees, and a multilayer perceptron) and a finite state automata (FSA). We present results for four different classifiers. The FSA module consisted of a 4-state decision logic that filtered false alarms and false positives. We compare the use of four different classifiers in this task. The look ahead of the method that we propose was of 7 frames, which are the number of frames that maximized the accuracy of the system. The system was tested with real signals recorded inside a car, with signal to noise ratio that ranged from 6 dB to 30dB. Finally we present experimental results demonstrating that the system yields robust endpoint detection.

In this paper we study direction of arrival (DOA) estimation of multiple audio sources by canonical correlation analysis (CCA), which is based on a sparse linear arrays. This array is composed of two separated subarrays. From the receiving data set, we can obtain the separate components by CCA. After a simple correlation, time difference can be obtained, and then we can compute the azimuth of different audio sources. The important contribution of this new estimation method is that it can reduce the effect of inter-sensor spacing to DOA estimation and the computation burden is light. Simulation result confirms the validity and practicality of the proposed approach. Results of DOA estimation are more accurate and stable based on this new method.

The work presented in this paper suggests a Traffic Sign Recognition (TSR) system whose core is based on a Multilayer Perceptron (MLP). A pre-processing of the traffic sign image (blob) is applied before the core. This operation is made to reduce the redundancy contained in the blob, to reduce the computational cost of the core and to improve its performance. For comparison purposes, the performance of the a statistical method like the k-Nearest Neighbour (k-NN) is included. The number of hidden neurons of the MLP is studied to obtain the value that minimizes the total classification error rate. Once obtained the best network size, the results of the experiments with this parameter show that the MLP achieves a total error probability of 3.85%, which is almost the half of the best obtained with the k-NN.

In this paper, a new algorithm for traffic sign recognition is presented. It is based on a shape detection algorithm that classifies the shape of the content of a sign using the capabilities of a Support Vector Machine (SVM). Basically, the algorithm extracts the shape inside a traffic sign, computes the projection of this shape and classifies it into one of the shapes previously trained with the SVM. The most important advances of the algorithm is its robustness against image rotation and scaling due to camera projections, and its good performance over images with different levels of illumination. This work is part of a traffic sign detection and recognition system, and in this paper we will focus solely on the recognition step.

A computer vision system to recognize license plates of vehicles in real-time environments is presented in this study. The images of moving vehicles are taken with a digital camera and analyzed in real-time. An artificial neural network (ANN) system is used to locate the area and position of the license plate. The system has the following stages: (i) Image acquisition and determination of the location of the vehicle license plate (VLP), (ii) segmentation of the VLP into separate characters using image processing techniques, and (iii) recognition of each symbol in VLP using a feedforward artificial neural network (ANN) and assembly of the characters. Performance results are presented at the end.

Electronic Ink is a stored data in the form of the handwritten text or the script without converting it into ASCII by handwritten recognition on the pen-based computers and Personal Digital Assistants(PDA) for supporting natural and convenient data input. One of the most important issue is to search the electronic ink in order to use it. We proposed and implemented a script matching algorithm for the electronic ink. Proposed matching algorithm separated the input stroke into a set of primitive stroke using the curvature of the stroke curve. After determining the type of separated strokes, it produced a stroke feature vector. And then it calculated the distance between the stroke feature vector of input strokes and one of strokes in the database using the dynamic programming technique.

Linear Discriminant Analysis (LDA) technique is an important and well-developed area of image recognition and to date many linear discrimination methods have been put forward. Despite these efforts, there persist in the traditional LDA some weaknesses. In this paper, we propose a new LDA-based method called Block LDA (BLDA) that can outperform the traditional Linear Dicriminant Analysis (LDA) methods. As opposed to conventional LDA, BLDA is based on 2D matrices rather than 1D vectors. That is, we firstly divides the original image into blocks. Then, we transform the image into a vector of blocks. By using row vector to represent each block, we can get the new matrix which is the representation of the image. Finally LDA can be applied directly on these matrices. In contrast to the between-class and within-class covariance matrices of LDA, the size of the these covariance matrices using BLDA is much smaller. As a result, BLDA has three important advantages over LDA. First, it is easier to evaluate the between-class and within-class covariance matrices accurately. Second, less time is required to determine the corresponding eigenvectors. And finally, block size could be changed to get the best results. Experiment results show our method achieves better performance in comparison with the other methods.

A cellular neural network is proposed as the main processing core in a novel FPGA-based augmented reality system. The described application is focused on visually impaired people aid. The aim is to enhance the user’s knowledge of the environment with useful information extracted by image processing. A CNN architecture oriented to hardware implementation on FPGA is presented, and used as the image processor in a fully FPGA-based system. So, CNNs and FPGAs are combined in a system which makes the most of their characteristics to achieve high performance and versatility.

Magnetic Resonance Images(MRI) are piecewise constant functions that can be corrupted by an inhomogeneous illumination field. We propose a gradient descent parametric illumination correction algorithm for MRI. The illumination bias is modelled as a linear combination of 2D products of Legendre polynomials. The error function is related to the classification error in the bias corrected image. In this work the intensity classes are given beforehand, so the adaptive algorithm is used only to estimate the bias field. We test our algorithm against Maximum A Posteriori algorithms over some images from the ISBR public domain database.

By analyzing electroencephalograms taken from healthy subjects and epilepsy patients, we investigated whether the complexity of the electroencephalogram (EEG) could be characterized by a multifractal. Our results showed that the EEGs from the two sets exhibit higher complexity than monofractal 1/

f

scaling. A significant finding was the observation that the dynamics of the epileptic EEGs exhibited anticorrelated, correlated, and uncorrelated behaviors. In conclusion, multifractal formalism based on the wavelet transform modulus maxima (WTMM) may be a good tool to characterize the various dynamics of the two sets.

When dealing with multi-class classification tasks, a popular and applicable way is to decompose the original problem into a set of binary subproblems. The most well-known decomposition strategy is

one-against-one

and the corresponding widely-used method to recombine the outputs of all binary classifiers is

pairwise coupling

(PWC). However PWC has an intrinsic shortcoming; many meaningless partial classification results contribute to the global prediction result. In this paper, this problem is tackled by the use of

correcting classifier

s. A novel algorithm is proposed which works in two steps: First the original

pairwise probabilities

are converted into a new set of

pairwise probabilities

, then

pairwise coupling

is employed to construct the global posterior probabilities. This algorithm is applied to face recognition on the ORL face database, experimental results show that it is effective and efficient.

Face recognition is one of the most important image processing research topics which is widely used in personal identification, verification and security applications. In this paper, a face recognition system, based on the principal component analysis (PCA) and the feedforward neural network is developed. The system consists of two phases which are the PCA preprocessing phase, and the neural network classification phase. PCA is applied to calculate the feature projection vector of a given face which is then used for face identification by the feedforward neural network. The proposed PCA and neural network based identification system provides improvement on the recognition rates, when compared with a face classifier based on the PCA and Euclidean Distance.

An edge detection is one of the most important tasks in image processing. Image segmentation, registration and identification are based on edge detection. In the literature, there is some techniques developed to achive this task such as Sobel, Prewitt, Laplacian and Laplacian of Gaussian. In this paper, a novel knowledge-based approach which have been used to realize control techniques for past years is proposed for edge detection. Some of the classical techniques are used with certain parameters such as threshold and

σ

to implement edge detection process. The another restricts about classial approach, results generally have fixed edge thickness. The rule-based approach offers most advantages such as giving permission to adapt some parameters easily. The edges thickness can be changed easily by adding new rules or changing output parameters. That is to say rule-based approach has flexible structure which can be adapted any time or any where easily and new fuzzy approach produces nice result as well as classical techniques at least.

This paper investigates the fault detection and prediction of rhythmically soniferous products, such as clocks, watches and timers. Such products with fault cannot work steadily or probably cause malfunction. The authors extend the concept of computer audition and establish an architectural model of product fault prediction system based on probabilistic neural networks. The system listens to the product sound by the multimedia technology and the sound features are extracted to detect and predict faults by the neural network. The paper analyzes the reasons of timer faults and the corresponding sound features. Experiments are made in the laboratory to demonstrate the proposed method. The technology is expected to apply in factories in coming years for realizing automatic product test and improving efficiency of product inspection.

In this paper we report the results obtained using a kernel-based approach to predict the temporal development of four response signals in the process control of a glass melting tank with 16 input parameters. The data set is a revised version from the modelling challenge in EUNITE-2003. The central difficulties are: large time-delays between changes in the inputs and the outputs, large number of data, and a general lack of knowledge about the relevant variables that intervene in the process. The methodology proposed here comprises Support Vector Machines (SVM) and Regularization Networks (RN). We use the idea of sparse approximation both as a means of regularization and as a means of reducing the computational complexity. Furthermore, we will use an incremental approach to add new training examples to the kernel-based method and efficiently update the current solution. This allows us to use a sophisticated learning scheme, where we iterate between prediction and training, with good computational efficiency and satisfactory results.

This paper presents and application of Genetic Programming (GP) for time series forecast. Although this kind of application has been carried out with a wide range of techniques and with very good results, this paper presents a different approach. In most of the experiments done in time series forecasting the objective is, from a consecutive set of samples or time interval, to obtain the value of the sample in the next time step. The aim of this paper is to study the forecasting not only on the next sample, but in general several samples forward. This will allow the building of more complete prediction systems. With this objective, one of the most widely used series for this kind of application has been used, the Mackey-Glass series.

Recently, multiple works proposed multi-model based approaches to model nonlinear systems. Such approaches could also be seen as some “specific” approach, inspired from ANN operation mode, where each neuron, represented by one of the local models, realizes some higher level transfer function. We are involved in nonlinear dynamic systems identification and nonlinear dynamic behavior prediction, which are key steps in several areas of industrial applications. In this paper, two identifiers architectures issued from the multi-model concept are presented, in the frame of nonlinear system’s behavior prediction context. The first one, based on “equation error” identifier, performs a prediction based on system’s inputs and outputs. However, if the system’s inputs are often accessible, its outputs are not always available in prediction phase. The second one, called “output error” based identifier/predictor needs only the system’s inputs to achieve the prediction task. Experimental results validating presented multi-model based structures have been reported and discussed.

This paper presents

k

-NN as an approximator for time series prediction problems. The main advantage of this approximator is its simplicity. Despite the simplicity,

k

-NN can be used to perform input selection for nonlinear models and it also provides accurate approximations. Three model structure selection methods are presented: Leave-one-out, Bootstrap and Bootstrap 632. We will show that both Bootstraps provide a good estimate of the number of neighbors,

k

, where Leave-one-out fails. Results of the methods are presented with the Electric load from Poland data set.

In nonlinear robust identification context, a process model is represented by a nominal model and possible deviations. With parametric models this process model can be expressed as the so-called Feasible Parameter Set (

FPS

), which derives from the minimization of identification error specific norms. In this work, several norms are used simultaneously to obtain the

FPS

. This fact improves the model quality but, as counterpart, it increases the optimization problem complexity resulting in a multimodal problem with an infinite number of minima with the same value which constitutes

FPS

contour. A special Evolutionary Algorithm (

ε

– GA) has been developed to find this contour. Finally, an application to a thermal process identification is presented.

Prediction of time series is an important problem in many areas of science and engineering. Extending the horizon of predictions further to the future is the challenging and difficult task of long-term prediction. In this paper, we investigate the problem of selecting non-contiguous input variables for an autoregressive prediction model in order to improve the prediction ability. We present an algorithm in the spirit of backward selection which removes variables sequentially from the prediction models based on the significance of the individual regressors. We successfully test the algorithm with a non-linear system by selecting inputs with a linear model and finally train a non-linear predictor with the selected variables on Santa Fe laser data set.

This paper presents a comparison between direct and recursive prediction strategies. In order to perform the input selection, an approach based on mutual information is used. The mutual information is computed between all the possible input sets and the outputs. Least Squares Support Vector Machines are used as non-linear models to avoid local minima problems. Results are illustrated on the Poland electricity load benchmark and they show the superiority of the direct prediction strategy.

Based on the Nyström approximation and the primal-dual formulation of Least Squares Support Vector Machines (LS-SVM), it becomes possible to apply a nonlinear model to a large scale regression problem. This is done by using a sparse approximation of the nonlinear mapping induced by the kernel matrix, with an active selection of support vectors based on quadratic Renyi entropy criteria. The methodology is applied to the case of load forecasting as an example of a real-life large scale problem in industry, for the case of 24-hours ahead predictions. The results are reported for different number of initial support vectors, which cover between 1% and 4% of the entire sample, with satisfactory results.

There exists a wide range of paradigms and a high number of different methodologies applied to the problem of Time Series Prediction. Most of them are presented as a modified function approximation problem using I/O data, in which the input data is expanded using outputs at previous steps. Thus the model obtained normally predicts the value of the series at a time (

t

+

h

) using previous time steps (

t

–

τ

1

), (

t

–

τ

2

),...,(

t

–

τ

n

). Nevertheless, learning a model for long term time series prediction might be seen as a completely different task, since it will generally use its own outputs as inputs for further training, as in recurrent networks. In this paper we present the utility of the TaSe model using the well-known Mackey Glass time series and an approach that upgrades the performance of the TaSe one-step-ahead prediction model for long term prediction.

Recently, the application of Independent Component Analysis (ICA) to natural images has raised a great interest. Some outstanding features have been observed, like the sparse distribution of the independent components and the special appearance of the ICA bases (most of them look like edges). This paper provides a new insight on this behaviour, being supported by experimental results. In particular, a mathematical proof of the relation between ICA and sparse coding is given.

The task of recovering a set of unknown sources from another set of mixtures directly observable and little more information about the way they were mixed is called the blind source separation problem. If the assumption in order to obtain the original sources is their statistical independence, then ICA (Independent Component Analysis) may be the technique to recover the signals. In this contribution, we propose and analyze three evaluation functions (contrast functions in Independent Component Analysis terminology) for the use in a genetic algorithm (PNL-GABSS, Post-NonLinear Genetic Algorithm for Blind Source Separation) which solves source separation in nonlinear mixtures, assuming the post-nonlinear mixture model. A thorough analysis of the performance of the chosen contrast functions is made by means of ANOVA (Analysis of Variance), showing the validity of the three approaches.

Microarray Data Processing is becoming a field of important activity for Signal Processing and Pattern Recognition areas, as the extraction and mining of meaningful data from large groupings of microarray patterns is of vital importance in Medicine, Genomics, Proteomics, Pharmacology, etc. In this paper emphasis is placed on studying and cataloging the nature of possible sources of corruption of microarray data and in establishing a pre-processing methodology for discriminating sources of corruption from microarray data (de-noising). We also discuss ways of precisely reconstructing original contributions (theoretically hybridized data) using ICA methods. Some classical examples are shown, and a discussion follows the presentation of results.

Traditional approaches to gas sensing are usually related with gas identification and classification, i.e., recognition of aromas. In this work we propose an innovative approach to determine the concentration of the single species in a gas mixture by using nonlinear source separation techniques. Additionally, responses of tin oxide sensor arrays were analyzed using nonlinear regression techniques to determine the concentrations of ammonia and acetone in gas mixtures. The use of the source separation approach allows the compensation of some of the most important sensor disadvantages: the parameter spreading and time drift.

In this paper we propose a new algorithm for the clustering of signals using incomplete independent component analysis (ICA). In the first step we apply the ICA to the dataset without dimension reduction, in the second step we reduce the dimension of the data to find clusters of independent components that are similar in their entries in the mixture matrix found by the ICA. We demonstrate that our algorithm out-performs

k-means

in the case of toy data and works well with a real world fMRI example, thus allowing a closer look the way how different parts of the brain work together.

Independent component analysis (ICA) as well as blind source separation (BSS) often faces the problem of assigning the independent or uncorrelated components estimated with ICA or BSS techniques to underlying source signals, artifacts or noise contributions. In this work an automatic assignment tool is presented which uses

a priori

knowledge about the form of some of the signals to be extracted. The algorithm is applied to the problem of removing water artifacts from 2D NOESY NMR spectra. The algorithm uses local PCA to approximate the water artifact and defines a suitable cost function which is optimized using simulated annealing. The blind source separation of the water artifact from the remaining protein spectrum is done with the recently developed algorithm dAMUSE.

Cerebral processing mainly relies on functional connectivity among involved regions. Neuro-imaging techniques able to assess these links with suitable time resolution are electro- and magneto-encephalography (EEG and MEG), even if it is difficult to localize recorded extra-cranial information, particularly within restricted areas, due to complexity of the ‘inverse problem’. By means of Independent Component Analysis (ICA) a procedure ‘blind’ to position and biophysical properties of the generators, our aim in this work was to identify cerebral functionally different sources in a restricted area. MEG data of 5 subjects were collected performing a relax-movement motor task in 5 different days. ICA reliably extracted neural networks differently modulated during the task in the frequency range of interest. In conclusion, a procedure solely based on statistical properties of the signals, disregarding their spatial positions, was demonstrated able to discriminate functionally different neuronal pools activities in a very restricted cortical area.

When using ICA for image separation, a well-known problem is that most often a large correlation exists between the sources. Because of this dependence, there is no more guarantee that the global maximum of the ICA contrast matches the outputs to the sources. In order to overcome this problem, some preprocessing can be used, like e.g. band-pass filtering. However, those processings involve parameters, for which the optimal values could be tedious to adjust. In this paper, it is shown that a simple ICA algorithm can recover the sources, without any other preprocessing than whitening, when they are correlated in a specific way. First, a single source is extracted, and next, a parameter-free postprocessing is applied for optimizing the extraction of the remaining sources.

The authors propose a new solution to the blind robust watermarking of digital images. In this approach we embed the watermark into the independent components of the image. Since independent components are related to the edges of the image, this method has a little perceptual impact on the watermarked image. Besides, we exploit the orthogonality of independent components and spread-spectrum generated watermarks in the blind extraction of the watermark. As extraction algorithm we use a simple matched filter. We also improve this novel method with standard techniques such as perceptual masking and holographic properties. Some experiments are included to illustrate the good performance of the algorithm against compression, cropping, filtering or quantization based attacks.

This paper proposes a new controller based on neural network and fuzzy logic technologies for load frequency control to allow for the incorporation of both heuristics and deep knowledge to exploit the best characteristics of each. A “Dynamical Fuzzy Network (DFN)” that contains dynamical elements such as delayers or integrators in their processing units is used in the adaptive controller design for load frequency control. A DFN is connected between the two area power systems. The input signals of the DFN are the ACEs and their changes. The outputs of the DFN are the control signals for the two area load frequency control. Adaptation is based on adjusting parameters of DFN for load frequency control. This is done by minimizing the cost functional of load frequency errors. The cost gradients with respect to the network parameters are calculated by adjoint sensitivity. In this paper, it is illustrated that this control approach is more successful than conventional integral controller for load frequency control in two area systems.

In this paper the knowledge extraction from neural and fuzzy models, and the quality and the explanation capacities of this knowledge are tracked. Nowadays the application of algorithms and methodologies based on artificial neural networks and fuzzy logic are very usual in most of the scientific and technical areas in order to generate models driven by data. But sometimes to obtain a good model by these techniques is not enough when some explanations about the model behaviour are mandatory, and these models are very near, most of the cases, to ”black boxes” or its explanatory capacity is very poor. In literature, several methods are been published in order to extraction, simplification and interpretability of the knowledge stored in these types of models.

In this paper a real problem is involved: to model an AC motor on several functioning modes (faults) by several neural/fuzzy approaches, making a comparison on the knowledge extracted from each one: Feedforward network + Backpropagation, Substractive Clustering + ANFIS, FasArt and FasBack.

This paper presents an adaptive Power System Stabilizer (PSS) using an Adaptive Network Based Fuzzy Inference System (ANFIS) and Genetic Algorithms (GAs). Firstly, genetic algorithms are used to tune a conventional PSS on a wide range of operating conditions and then, the relationship between these operating points and the PSS parameters is learned by the ANFIS. The ANFIS optimally selectes the classical PSS parameters based on machine loading conditions. The proposed stabilizer has been tested by performing nonlinear simulations using a synchronous machine-infinite bus model. The results show the robustness and the capability of the stabilizer to enhance system damping over a wide range of operating conditions and system parameter variations.

Six artificial neural networks (ANN) were developed to compute the membership values of the consequents of a fuzzy rule evaluation table, which is part of a Mamdani type fuzzy controller. This controller is aimed to regulate the neutron power of a research nuclear reactor. The neural networks obtained were validated over a wide range of input data. These ANN offer the possibility of a parallel processing of the fuzzy inputs, thus reducing the response time of the controller.

Schizophrenia is a frequent and devastating disorder beginning in early adulthood. Until now, the heterogeneity of this disease has been a major pitfall for identifying the aetiological, genetic or environmental factors. Age at onset or several other quantitative variables could allow for categorizing more homogeneous subgroups of patients, although there is little information on which are the boundaries for such categories. The Bayesian networks classifier approach is one of the most popular formalisms for reasoning under uncertainty. We used this approach to determine the best cut-off point for three continuous variables (i.e. age at onset of schizophrenia and neurological soft signs) with a minimal loss of information, using a data set including genotypes of selected candidate genes for schizophrenia.

In this work, the well-known Graph Partitioning (GP) problem for undirected weighted graphs has been studied from two points of view: maximizing (MaxCut) or minimizing (MinCut) the cost of the cut induced in the graph by the partition. An unified model, based on a neural technique for optimization problems, has been applied to these two concrete problems. A detailed description of the model is presented, and the technique to minimize an energy function, that measures the goodness of solutions, is fully described. Some techniques to escape from local optima are presented as well. It has proved to be a very competitive and efficient algorithm, in terms of quality of solutions and computational time, when compared to the state-of-the-art methods. Some simulation results are presented in this paper, to show the comparative efficiency of the methods.

Dynamical Random Neural Network (DRNN) has been suggested as tools for the solution of optimization problems [1, 2]. Here DRNN method is applied to solve the problem of optimal resource allocation with both minimum and maximum activation levels and fixed cost. The problem is NP-hard. The conclusion shows that the DRNN method provides results of the optimal resource allocation problem better than those given by [3].

A new approach for personal identification using hand geometry based upon geometrical and shape features is presented. We propose a new pegless hand geometry verification system where the users are free to put their hand in arbitrary fashion. A Linear Discirminant Analysis if applied to the raw data in order to perform a best clustering of the feature space. The combination of three different neural network classifiers (unsupervised SOM, supervised SOM and LVQ) gives 0.35% FAR and 0.15% FRR. The method has been tested on a large size database of 1400 images for training and 1400 for test from 280 individuals suitable for medium and low security applications.

This Paper describes a hand geometry biometric identification system. We have acquired a database of 22 people using a conventional document scanner. The experimental section consists of a study about the discrimination capability of different extracted features, and the identification rate using different classifiers based on neural networks.

This Paper studies different committees of neural networks for biometric pattern recognition. We use the neural nets as classifiers for identification and verification purposes. We show that a committee of nets can improve the recognition rates when compared with a multi-start initialization algorithm that just picks up the neural net which offers the best performance. On the other hand, we found that there is no strong correlation between identification and verification applications using the same classifier.

The aerodynamic design of wind turbine blades is carried out by means of evolutionary techniques within an automatic design environment based on evolution. A simple, fast, and robust aerodynamic simulator is embedded in the design environment to predict the performance of any turbine produced as intermediate individual of the evolutionary process. The aerodynamic simulator is based on blade element theory in which a panel method is combined with an integral boundary layer code to calculate blade airfoils’ characteristics. In order to reduce computations some simplifications were contemplated and the results corrected by means of the application of neural network based approximations. Results of the simulations obtained using this technique, of the application of the automatic design procedure and of the operation of the wind turbines thus obtained are presented.

Sign language, which is a highly visual-spatial, linguistically complete and natural language, is the main mode of communication among deaf people. In this paper, an American Sign Language (ASL) word recognition system is being developed using artificial neural networks (ANN) to translate the ASL words into English. The system uses a sensory glove Cyberglove

TM

and a Flock of Birds

®

3- D motion tracker to extract the gesture features. The finger joint angle data obtained from strain gauges in the sensory glove define the hand-shape while the data from the tracker describe the trajectory of hand movement. The trajectory of hand is normalized for increase of the signer position flexibility. The data from these devices are processed by two neural networks, a velocity network and a word recognition network. The velocity network uses hand speed to determine the duration of words. To convey the meaning of a sign, signs are defined by feature vectors such as hand shape, hand location, orientation, movement, bounding box, and distance. The second network is used as a classifier to convert ASL signs into words based on features. We trained and tested our ANN model for 60 ASL words for different number of samples. Our test results show that the accuracy of recognition is 92% .

The paper presents the application of the wavelet transform of the frequency histogram of connected elements to the detection of very thin cracks in used pallets. First, the paper presents this novel concept and introduces the parameters that define a connected element, showing that the conventional grayscale intensity histogram of a digital image is a particular case of the histogram of connected elements. Then, the discriminant capability of the wavelet transform of this generalized histogram is analyzed. In particular, the information conveyed by the histogram of connected elements is exploited to detect very thin cracks in used pallets. An artificial neural network classifier to discriminate sound wood from defective wood with very thin cracks has been designed. The exhaustive experimental test carried out with numerous boards of used pallets has validated the proposed method, in particular its remarkably low ratio of false alarms.

A proper gait assessment in patients with knee or hip injuries strongly determines the diagnosis and consequently the evolution of the pathology, the quality of life of implanted patients, and the overall costs involved. Among the different strategies to clinically assess gait, 3D optical tracking provides a reliable and objective evaluation. This method involves state-of-the-art image analysis that performs anatomical measurements upon bony landmarks identified by markers attached to the patient. We show how this technology can be used to perform patients diagnosis and follow-up by grouping the results of gait measurement with a competitive neural network where the number of clusters is automatically determined.

A major drawback of artificial neural networks is their black-box character. In this paper, we use the equivalence between artificial neural networks and a specific fuzzy rule base to extract the knowledge embedded in the network. We demonstrate this using a benchmark problem: the recognition of digits produced by a LED device. The method provides a symbolic and comprehensible description of the knowledge learned by the network during its training.

This paper addresses the development of a new control strategy to regulate cutting force in a high-speed machining process. Fuzzy basis functions (FBF), on the basis of L.X.Wang’s approach, serve as basement for designing and implementing adaptive fuzzy control system in an open computerized numerical control (CNC). The controller uses cutting force measured from a dynamometric platform, and mathematically processed by means of an integrated application, to perform real-time modification of feed rate. The integration process, design steps and results of applying the adaptive fuzzy-control system in actual high-speed machining operations corroborate the suitability of the proposed control strategy for real-time applications. Moreover, the results show a good transient response in the cutting force pattern despite the complexity of the mechanized part.

One of the most important problems to be solved in a DS-SS system is the acquisition of the PN sequence. In time-varying environments this fact becomes even more important because the data decoding depends on the performance of the acquisition and tracking. In this work a new sequential acquisition system based on a fuzzy logic controller is proposed. The fuzzy logic controller extracts rellevant information about the transmission conditions improving the the stability and robustness of the receiver.

A fuzzy evaluation system to decide critical students’ final marks is presented. The marks of the assessment tests made by students along the academic year are transformed into linguistic terms and used for assigning values to linguistic variables. Subjective criteria as i.e.

student’s interest

or

student’s progression

are used in order to decide whether every analyzed student passes or fails the considered subject. This paper presents the features of the subject where the fuzzy system has been applied in, the proposed fuzzy decision system, and results when it is applied using real students’ marks.