Artificial Neural Networks – ICANN 2007

We present a model for the ontogenesis of information routing architectures in the brain based on chemical markers guiding axon growth. The model produces all-to-all connectivity between given populations of input and output nodes using a minimum of cortical resources (links and intermediate nodes). The resulting structures are similar to architectures proposed in the literature, but with interesting qualitative differences making them biologically more plausible.

It is well documented that superior colliculus (SC) neurons integrate stimuli of different modalities (e.g., visual and auditory). In this work, a mathematical model of the integrative response of SC neurons is presented, to gain a deeper insight into the possible mechanisms implicated. The model includes two unimodal areas (auditory and visual, respectively) sending information to a third area (in the SC) responsible for multisensory integration. Each neuron is represented via a sigmoidal relationship and a first-order dynamic. Neurons in the same area interact via lateral synapses. Simulations show that the model can mimic various responses to different combinations of stimuli: i) an increase in the neuron response in presence of multisensory stimulation, ii) the inverse effectiveness principle; iii) the existence of within- and cross-modality suppression between spatially disparate stimuli. The model suggests that non linearities in neural responses and synaptic connections can explain several aspects of multisensory integration.

Neurotransmitter fields differ from neural fields in the underlying principle that the state variables are not the neuron action potentials, but the chemical concentration of neurotransmitters in the extracellular space. The dendritic arbor of a new electro-chemical neuron model performs a computation on the surrounding field of neurotransmitters. These fields may represent quantities such as position, force, momentum, or energy. Any computation performed by a neural network has a direct analog to a neurotransmitter field computation. While models that use action potentials as state variables may form associations using matrix operations on a large vector of neuron outputs, the neurotransmitter state model makes it possible for a small number of neurons, even a single neuron, to establish an association between an arbitrary pattern in the input field and an arbitrary output pattern. A single layer of neurons, in effect, performs the computation of a two-layer neural network.

Motivated by a better understanding of cerebral information processing, a lot of work has been done recently in bringing together connectionist numerical models and symbolic cognitive frameworks, allowing for a better modelling of some cerebral mechanisms. However, a gap still exists between models that describe functionally small neural populations and cognitive architectures that are used to predict cerebral activity. The model presented here tries to fill partly this gap. It uses existing knowledge of the brain structure to describe neuroimaging data in terms of interacting functional units. Its merits rely on an explicit handling of neural populations proximity in the brain, relating it to similarity between the pieces of information processed.

We investigate human motor learning in an unknown environment using a force measurement as the input to a computer controlled plant. We propose to use the Feedback Error Learning (FEL) framework to model the overt behavior of motor response to unexpected changes in plant parameters. This framework assumes a specific feedforward and feedback structure. The feedforward component predicts the required motor commands given the reference trajectory, and the feedback component stabilizes the system in case of imprecise estimates and initial conditions. To estimate the feedback gain, we employ a novel technique in which we probe the stability properties of the system by artificially inducing a time delay in the sensory feedback pathway. By altering the pole location of the plant during a sinusoidal tracking task, a feedforward learning bandwidth was computed for each subject which measures the ability to adaptively track time-varying changes in the plant dynamics. Lastly, we use the learning bandwidth to compute a learning rate with respect to the FEL model. This learning rate reflects the ability of the subjects’ internal model to adapt to changes in an unknown environment.

Many episodes are acquired in the hippocampus. An episode is expressed by a sequence of elements that are perceived in an event. Episodes are associated each other by events that contain information shared among the episodes. Sequences must be recalled individually, even if the sequences are overlapped at some representations. Therefore, sequence disambiguation is an essential function to dissociate overlapped sequences. In this study, we especially focus on the location-dependencies of the STDP effects on synaptic summation and the expression of AMPA receptor. We firstly show that the hippocampal CA3 is divided into two regions in which one region has spatial selectivity and the other has temporal selectivity. Moreover, we confirm that the divided CA3 could generate a code for sequence disambiguation in computer simulations. Consequently, we suggest that the CA3 can be divided into two regions characterized by their selectivity, and the divided CA3 contributes to sequence disambiguation.

Lateral and elastic interactions are known to build a topology in different systems. We demonstrate how the models with weak lateral interactions can be reduced to the models with corresponding weak elastic interactions. Namely, the batch version of soft topology-preserving map can be rigorously reduced to the elastic net. Owing to the latter, both models produce similar behaviour when applied to the TSP. Unlike, the incremental (online) version of soft topology-preserving map is reduced to the cortical map only in the limit of low temperature, which makes their behaviours different when applied to the ocular dominance formation.

The objective of this paper is to introduce briefly the various soft computing methodologies and to present various applications in medicine. The scope is to demonstrate the possibilities of applying soft computing to medicine related problems. The recent published knowledge about use of soft computing in medicine is observed from the literature surveyed and reviewed. This study detects which methodology or methodologies of soft computing are used frequently together to solve the special problems of medicine. According to database searches, the rates of preference of soft computing methodologies in medicine are found as 70% of fuzzy logic-neural networks, 27% of neural networks-genetic algorithms and 3% of fuzzy logic-genetic algorithms in our study results. So far, fuzzy logic-neural networks methodology was significantly used in clinical science of medicine. On the other hand neural networks-genetic algorithms and fuzzy logic-genetic algorithms methodologies were mostly preferred by basic science of medicine. The study showed that there is undeniable interest in studying soft computing methodologies in genetics, physiology, radiology, cardiology, and neurology disciplines.

Exploratory matrix factorization methods like ICA and LNMF are applied to identify marker genes and classify gene expression data sets into different categories for diagnostic purposes or group genes into functional categories for further investigation of related regulatory pathways. Gene expression levels of either human breast cancer (HBC) cell lines [5] mediating bone metastasis or cell lines from Niemann Pick C patients monitoring monocyte - macrophage differentiation are considered.

In todays bioinformatics, Mass spectrometry (MS) is the key technique for the identification of proteins. A prediction of spectrum peak intensities from pre computed molecular features would pave the way to better understanding of spectrometry data and improved spectrum evaluation. We propose a neural network architecture of Local Linear Map (LLM)-type for peptide prototyping and learning locally tuned regression functions for peak intensity prediction in MALDI-TOF mass spectra. We obtain results comparable to those obtained by

ν

-Support Vector Regression and show how the LLM learning architecture provides a basis for peptide feature profiling and visualisation.

The identification of

cis

-regulatory binding sites in DNA is a difficult problem in computational biology. To obtain a full understanding of the complex machinery embodied in genetic regulatory networks it is necessary to know both the identity of the regulatory transcription factors together with the location of their binding sites in the genome. We show that using an SVM together with data sampling, to integrate the results of individual algorithms specialised for the prediction of binding site locations, can produce significant improvements upon the original algorithms. These results make more tractable the expensive experimental procedure of actually verifying the predictions.

DNA Microarray technology allows us to monitor the expression level of thousands of genes simultaneously. This technique has become a relevant tool to identify different types of cancer.

Several machine learning techniques such as the Support Vector Machines (SVM) have been proposed to this aim. However, common SVM algorithms are based on Euclidean distances which do not reflect accurately the proximities among the sample profiles. The SVM has been extended to work with non-Euclidean dissimilarities. However, no dissimilarity can be considered superior to the others because each one reflects different features of the data.

In this paper, we propose to combine several Support Vector Machines that are based on different dissimilarities to improve the performance of classifiers based on a single measure. The experimental results suggest that our method reduces the misclassification errors of classifiers based on a single dissimilarity and a widely used combination strategy such as Bagging.

This paper presents a locally recurrent globally feedforward fuzzy neural network, with internal feedback, that performs the task of separation of lung sounds, obtained from patients with pulmonary pathology. The filter is a novel generalized Takagi-Sugeno-Kang fuzzy model, where the consequent parts of the fuzzy rules are Block-Diagonal Recurrent Neural Networks. Extensive experimental results, regarding the lung sound category of squawks, are given, and a performance comparison with a series of other fuzzy and neural filters is conducted, underlining the separation capabilities of the proposed filter.

In order to understand the general principles along which sensory processing is organized, several recent studies optimized particular coding objectives on natural inputs for different modalities. The homogeneity of neocortex indicates that a sensitive objective should be able to explain response properties of different sensory modalities. The temporal stability objective was successfully applied to somatosensory and visual processing. We investigate if this objective can also be applied to auditory processing and serves as a general optimization objective for sensory processing. In case of audition, this translates to a set of non-linear complex filters optimized for temporal stability on natural sounds. We show that following this approach we can develop filters that are localized in frequency and time and extract the frequency content of the sound wave. A subset of these filters respond invariant to the phase of the sound. A comparison of the tuning of these filters to the tuning of cat auditory nerves shows a close match. This suggests that temporal stability can be seen as a general objective describing somatosensory, visual and auditory processing.

In this paper we compare thirteen different methods to obtain multi-class probability estimates in view of two medical case studies. The basic classification method used to implement all methods are least squares support vector machine (LS-SVM) classifiers. Results indicate that multi-class kernel logistic regression performs very well, together with a method based on ensembles of nested dichotomies. Also, a Bayesian LS-SVM method imposing sparseness performed very well for methods that combine binary probabilities into multi-class probabilities.

In this paper we consider the question of whether it is possible to classify n-back EEG data into different memory loads across subjects. To capture relevant information from the EEG signal we use three types of features: power spectrum, conditional entropy, and conditional mutual information. In order to reduce irrelevant and misleading features we use a feature selection method that maximizes mutual information between features and classes and minimizes redundancy among features. Using a selected group of features we show that all classifiers can successfully generalize to the new subject for bands 1-40Hz and 1-60Hz. The classification rates are statistically significant and the best classification rates, close to 90%, are obtained using conditional entropy features.

As a causality criterion we propose the conditional relative entropy. The relationship with information theoretic functionals mutual information and entropy is established. The conditional relative entropy criterion is compared with 3 well-established techniques for causality detection: ‘Sims‘, ‘Geweke- Meese-Dent‘ and ‘Granger‘. It is shown that the conditional relative entropy, as opposed to these 3 criteria, is sensitive to0. non-linear causal relationships. All results are illustrated on real-world time series of human gait.

Finding a template image in another larger image is a problem that has applications in many vision research areas such as models for object detection and tracking. The main problem here is that under real-world conditions the searched image usually is a deformed version of the template, so that these deformations have to be taken into account by the matching procedure. A common way to do this is by minimizing the difference between the template and patches of the search image assuming that the template can undergo 2D affine transformations. A popular differential algorithm for achieving this has been proposed by Lucas and Kanade [1], with the disadvantage that it works only for small transformations. Here we investigate the transformation properties of a differential template matching approach by using resolution pyramids in combination with transformation pyramids, and show how we can do template matching under large-scale transformations, with simulation results indicating that the scale and rotation ranges can be doubled using a 3 stage pyramid.

In this paper, we discuss fuzzy classifiers based on Kernel Discriminant Analysis (KDA) for two-class problems. In our method, first we employ KDA to the given training data and calculate the component that maximally separates two classes in the feature space. Then, in the one-dimensional space obtained by KDA, we generate fuzzy rules with one-dimensional membership functions and tune the slopes and bias terms based on the same training algorithm as that of linear SVMs. Through the computer experiments for two-class problems, we show that the performance of the proposed classifier is comparable to that of SVMs, and we can easily and visually analyze its behavior using the degrees of membership functions.

Within the taxonomy of feature extraction methods, recently the Wrapper approaches lost some popularity due to the associated computational burden, compared to Embedded or Filter methods. The dominating factor in terms of computational costs is the number of adaption cycles used to train the black box classifier or function approximator, e.g. a Multi Layer Perceptron. To keep a wrapper approach feasible, the number of adaption cycles has to be minimized, without increasing the risk of missing important feature subset combinations.

We propose a search strategy, that exploits the interesting properties of Chow-Liu trees to reduce the number of considered subsets significantly. Our approach restricts the candidate set of possible new features in a forward selection step to children from certain tree nodes. We compare our algorithm with some basic and well known approaches for feature subset selection. The results obtained demonstrate the efficiency and effectiveness of our method.

A novel discriminant analysis method is presented for the face recognition problem. It has been recently shown that the predictive objectives based on Parzen estimation are advantageous for learning discriminative projections if the class distributions are complicated in the projected space. However, the existing algorithms based on Parzen estimators require expensive computation to obtain the gradient for optimization. We propose here an accelerating technique by reformulating the gradient and implement its computation by matrix products. Furthermore, we point out that regularization is necessary for high-dimensional face recognition problems. The discriminative objective is therefore extended by a smoothness constraint of facial images. Our Parzen Discriminant Analysis method can be trained much faster and achieve higher recognition accuracies than the compared algorithms in experiments on two popularly used face databases.

Parts-based recognition has been suggested for generalizing from few training views in categorization scenarios. In this paper we present the results of a comparative investigation of different feature types with regard to their suitability for category discrimination. So patches of gray-scale images were compared with SIFT descriptors and patches from the high-level output of a feedforward hierarchy related to the ventral visual pathway. We discuss the conceptual differences, resulting performance and consequences for hierarchical models of visual recognition.

The goal of keyword spotting is to detect the presence of specific spoken words in unconstrained speech. The majority of keyword spotting systems are based on generative hidden Markov models and lack discriminative capabilities. However, discriminative keyword spotting systems are currently based on frame-level posterior probabilities of sub-word units. This paper presents a discriminative keyword spotting system based on recurrent neural networks only, that uses information from long time spans to estimate word-level posterior probabilities. In a keyword spotting task on a large database of unconstrained speech the system achieved a keyword spotting accuracy of 84.5%.

The spatiostructural features proposed for recognition of online handwritten characters refer to offline-like features that convey information about both the positional and structural (shape) characteristics of the handwriting unit. This paper demonstrates the effectiveness of representing an online handwritten stroke using spatiostructural features, as indicated by its effect on the stroke classification accuracy by a Support Vector Machine (SVM) based classifier. The study has been done on two major Indian writing systems, Devanagari and Tamil. The importance of localization information of the structural features and handling of translational variance is studied using appropriate approaches to zoning the handwritten character.

This paper describes an improved version of a previously developed ANOVA and Functional Networks Feature Selection method. This wrapper feature selection method is based on a functional decomposition that grows exponentially as the number of features increases. Since exponential complexity limits the scope of application of the method, a new version is proposed that subdivides this functional decomposition and increases its complexity gradually. The improved version can be applied to a broader set of data. The performance of the improved version was tested against several real datasets. The results obtained are comparable, or better, to those obtained by other standard and innovative feature selection methods.

The simultaneous determination of odor classes and concentrations is solved by a kind of parallel-series perceptron models. Two groups of parallel single-output perceptrons are in series, and the former is responsible for classification, and the latter for location. The number of parallel perceptrons is equal to the number of odor classes. A multi-class learning problem is first decomposed into multiple two-class problems, and then solved by multiple parallel perceptrons, one by one. Each training subset is composed of the most necessary samples. And furthermore, some virtual samples are added to the weak side of any two-class learning subsets in order to arrive at a virtual balance. The experimental results for 4 kinds of fragrant materials show that the proposed parallel-series perceptrons with the electronic nose are effective.

Present work introduces a probabilistic recognition scheme for hand gestures. Self organizing feature maps are used to model spatiotemporal information extracted through image processing. Two models are built for each gesture category and, along with appropriate distance metrics, produce a validated classification mechanism that performs consistently during experi-ments on acted gestures video sequences.

Classification of structured data (i.e., data that are represented as graphs) is a topic of interest in the machine learning community. This paper presents a different, simple approach to the problem of structured pattern recognition, relying on the description of graphs in terms of algebraic binary relations. Maximum-a-posteriori decision rules over relations require the estimation of class-conditional probability density functions (pdf) defined on graphs. A nonparametric technique for the estimation of the pdfs is introduced, on the basis of a factorization of joint probabilities into individual densities that are modeled, in an unsupervised fashion, via Support Vector Machine (SVM). The SVM training is accomplished applying support vector regression on an unbiased variant of the Parzen Window. The behavior of the estimation algorithm is first demonstrated on a synthetic distribution. Finally, experiments of graph-structured image recognition from the Caltech Benchmark dataset are reported, showing a dramatic improvement over the results (available in the literature) yielded by state-of-the-art connectionist models for graph processing, namely recursive neural nets and graph neural nets.

This paper describes a new model for extracting large-field optical flow patterns to generate distributed representations of neural activation to control complex visual tasks such as 3D egomotion. The neural mechanisms draw upon experimental findings about the response properties and specificities of cells in areas V1, MT and MSTd along the dorsal pathway. Model V1 cells detect local motion estimates. Model MT cells in different pools are suggested to be selective to motion patterns integrating from V1 as well as to velocity gradients. Model MSTd cells considered here integrate MT gradient cells over a much larger spatial neighborhood to generate the observed pattern selectivity for expansion/contraction, rotation and spiral motion, providing the necessary input for spatial navigation mechanisms. Our model also incorporates feedback processing between areas V1-MT and MT-MSTd. We demonstrate that such a re-entry of context-related information helps to disambiguate and stabilize more localized processing along the primary motion pathway.

In this article we present an incremental method for building a mixture model. Given the desired number of clusters

K

 ≥ 2, we start with a two-component mixture and we optimize the likelihood by repeatedly applying a

Split-Merge

operation. When an optimum is obtained, we add a new component to the model by splitting in two, a properly chosen cluster. This goes on until the number of components reaches a preset limiting value. We have performed numerical experiments on several data–sets and report a performance comparison with other rival methods.

Aside from the Expectation-Maximization (EM) algorithm, Least-Mean-Square (LMS) is devised to further train the model parameters as a complementary training algorithm for Cluster-Weighted Modeling (CWM). Due to different objective functions of EM and LMS, the training result of LMS can be used to reinitialize CWM’s model parameters which provides an approach to mitigate local minimum problems.

In this paper, we examine analysis of clusters of labeled samples to identify their underlying hierarchical structure. The key in this identification is to select a suitable measure of dissimilarity among clusters characterized by subpopulations of the samples. Accordingly, we introduce a dissimilarity measure suitable for measuring a hierarchical structure of subpopulations that fit the mixture model. Glass identification is used as a practical problem for hierarchical cluster analysis, in the experiments in this paper. In the experimental results, we exhibit the effectiveness of the introduced measure, compared to several others.

Many clustering algorithms require some parameters that often are neither a priori known nor easy to estimate, like the number of classes. Measures of clustering quality can consequently be used to a posteriori estimate these values. This paper proposes such an index of clustering evaluation that deals with kernel methods like kernel-k-means. More precisely, it presents an extension of the well-known Davies & Bouldin’s index. Kernel clustering methods are particularly relevant because of their ability to deal with initially non-linearly separable clusters. The interest of the following clustering evaluation is then to get around the issue of the not explicitly known data transformation of such kernel methods. Kernel Davies & Bouldin’s index is finally used to a posteriori estimate the parameters of the kernel-k-means method applied on some toys datasets and Fisher’s Iris dataset.

In the field of computer vision feature matching in high dimensional feature spaces is a commonly used technique for object recognition. One major problem is to find an adequate similarity measure for the particular feature space, as there is usually only little knowledge about the structure of that space. As a possible solution to this problem this paper presents a method to obtain a similarity measure suitable for the task of feature matching without the need for structural information of the particular feature space. As the described similarity measure is based on the topology of the feature space and the topology is generated by a growing neural gas, no knowledge about the particular structure of the feature space is needed. In addition, the used neural gas quantizes the feature vectors and thus reduces the amount of data which has to be stored and retrieved for the purpose of object recognition.

Fuzzy Labeled Self-Organizing Map is a semisupervised learning that allows the prototype vectors to be updated taking into account information related to the clusters of the data set. In this paper, this algorithm is extended to update individually the kernel radii according to Van Hulle’s approach. A significant reduction of the mean quantization error of the numerical prototype vectors is expected.

Early studies on visual pathway circuitry demonstrated that synapses arrange to self-organize cortical orientation selectivity maps. It is still a debate how these maps are set up, so that diverse studies point to different directions to conclude about the main role played by feed-forward or intracortical recurrent connectivity. It is also a subject of discussion the way neurons communicate each other to transmit the information necessary to configure the circuits supporting the features of the central nervous system. Some studies claim for the necessity of a precise spike timing to provide effective neural codes. In this article we simulate networks consisting of three layers of integrate-and-fire neurons with feed-forward excitatory modifiable synapses that arrange to conform orientation selectivity maps. Features of receptive fields in these maps change when the precision of correlated firing decreases as an effect of increasing synaptic transmission jitters.

The

Self-Organising Map

is a popular unsupervised neural network model which has been used successfully in various contexts for clustering data. Even though labelled data is not required for the training process, in many applications class labelling of some sort is available. A visualisation uncovering the distribution and arrangement of the classes over the map can help the user to gain a better understanding and analysis of the mapping created by the SOM, e.g. through comparing the results of the manual labelling and automatic arrangement. In this paper, we present such a visualisation technique, which smoothly colours a SOM according to the distribution and location of the given class labels. It allows the user to easier assess the quality of the manual labelling by highlighting outliers and border data close to different classes.

Self-organizing map (SOM) has been studied as a model of map formation in the brain cortex. Neurons in the cortex present a refractory period in which they are not able to be activated, restriction that should be included in the SOM if a better description is to be achieved. Altough several works have been presented in order to include this biological restriction to the SOM, they do not reflect biological plausibility. Here, we present a modification in the SOM that allows neurons to enter a refractory period (SOM-RP) if they are the best matching unit (BMU) or if they belong to its neighborhood. This refractory period is the same for all affected neurons, which contrasts with previous models. By including this biological restriction, SOM dynamics resembles in more detail behavior shown by the cortex, such as non-radial activity patterns and long distance influence, besides the refractory period. As a side effect, two error measures are lower in maps formed by SOM-RP than in those formed by SOM.

A technique called component planes is commonly used to visualize variables behavior with Self-Organizing Maps (SOMs). Nevertheless, when the component planes are too many the visualization becomes difficult. A methodology has been developed to enhance the component planes analysis process. This methodology improves the correlation hunting in the component planes with a tree-structured cluster representation based on the SOM distance matrix. The methodology presented here was used in the classification of similar agro-ecological variables and productivity in the sugar cane culture. Analyzing the obtained groups it was possible to extract new knowledge about the variables more related with the highest productivities.

We present a dynamical model of processing and learning in the visual cortex, which reflects the anatomy of V1 cortical columns and properties of their neuronal receptive fields (RFs). The model is described by a set of coupled differential equations and learns by self-organizing the RFs of its computational units – sub-populations of excitatory neurons. If natural image patches are presented as input, self-organization results in Gabor-like RFs. In quantitative comparison with

in vivo

measurements, we find that these RFs capture statistical properties of V1 simple-cells that learning algorithms such as ICA and sparse coding fail to reproduce.

In this paper, we propose an evolutionary approach to deal with shortcomings on conventional focused crawling systems in semantic web environment. Thereby, meta-evolution strategy for collaboration among multiple crawlers has to be efficiently carried out. It is based on incremental aggregation of partial semantic structures extracted from web resources, which are in advance annotated with local ontologies. To do this, we employ similarity-based matching algorithm, so that fitness function is formulated by summing all possible semantic similarities. As a result, the best mapping condition (i.e., the fitness is maximized) is obtained for efficiently

i

) reconciling semantic conflicts between multiple crawlers, and

ii

) evolving semantic structures of web spaces over time.

This paper introduces the use of 15 different readability indices as a fingerprint that enables the classification of documents into different categories. While a classification based on such fingerprints alone is not necessarily superior to document categorization based on dedicated dictionaries per se, the document fingerprints can enhance the overall classification rate by applying proper data fusion techniques. For other applications text mining related applications such as language classification, the detection of plagiarism, or author identification, the accuracy of text categorization methods based on readability fingerprints can even exceed a dictionary-based approach. A novel addition to the readability indices is the addition of histograms based on the word length of all the dictionary words used in the text and a dictionary of the most common easy words in the English language.

The immense amount of unstructured information available on the Web poses increasing difficulties to fulfill users’ needs. New tools are needed to automatically collect and filter information that meets users’ demands. This paper presents the architecture of a personal information agent that mines web sources and retrieves documents according to users’ interests. The agent operates in two modes: "generation of space of concepts" and "document filtering". A space of concepts for a domain is represented by a matrix of asymmetrical coefficients of similarity for each pair of relevant terms in the domain. This matrix is seen as a Hopfield neural network, used for document filtering, where terms represent neurons and the coefficients of similarity the weights of the links that connect the neurons. Experiments conducted to evaluate the approach show that it exhibits satisfactory effectiveness.

Recurrent Neural Network (RNN) models have been shown to perform well on artificial grammars for sequential classification tasks over long-term time-dependencies. However, there is a distinct lack of the application of RNNs to real-world text classification tasks. This paper presents results on the capabilities of extended two-context layer SRN models (xRNN) applied to the classification of the Reuters-21578 corpus. The results show that the introduction of high levels of noise to sequences of words in titles, where noise is defined as the unimportant stopwords found in natural language text, is very robustly handled by the classifiers which maintain consistent levels of performance. Comparisons are made with SRN and MLP models, as well as other existing classifiers for the text classification task.

Multidimensional Scaling algorithms (MDS) are useful tools that help to discover high dimensional object relationships. They have been applied to a wide range of practical problems and particularly to the visualization of the semantic relations among documents or terms in textual databases.

The MDS algorithms proposed in the literature often suffer from a low discriminant power due to its unsupervised nature and to the ‘curse of dimensionality’. Fortunately, textual databases provide frequently a manually created classification for a subset of documents that may help to overcome this problem.

In this paper we propose a semi-supervised version of the Torgerson MDS algorithm that takes advantage of this document classification to improve the discriminant power of the word maps generated. The algorithm has been applied to the visualization of term relationships. The experimental results show that the model proposed outperforms well known unsupervised alternatives.

Forecasting Internet traffic is receiving an increasing attention from the computer networks domain. Indeed, by improving this task efficient traffic engineering and anomaly detection tools can be developed, leading to economic gains due to better resource management. This paper presents a Neural Network (NN) approach to predict TCP/IP traffic for all links of a backbone network, using both univariate and multivariate strategies. The former uses only past values of the forecasted link, while the latter is based on the neighbor links of the backbone topology. Several experiments were held by considering real-world data from the UK education and research network. Also, different time scales (e.g. every ten minutes and hourly) were analyzed. Overall, the proposed NN approach outperformed other forecasting methods (e.g. Holt-Winters).

An ANN-based classifier for voltage wave disturbance was developed. Voltage signals captured on the power transmission system of CHESF, Federal Power Utility, were processed in two steps: by wavelet transform and principal component analysis. The classification was carried out using a combination of six MLPs with different architectures: five representing the first to fifth-level details, and one representing the fifth-level approximation. Network combination was formed using the boosting algorithm which weights a model’s contribution by its performance rather than giving equal weight to all models. Experimental results with real data indicate that boosting is clearly an effective way to improve disturbance classification accuracy when compared with the simple average and the individual models.

This paper presents an approach (SOINN-DTW) for recognition of temporal data that is based on Self-Organizing Incremental Neural Network (SOINN) and Dynamic Time Warping. Using SOINN’s function that eliminates noise in the input data and represents topological structure of input data, SOINN-DTW method approximates output distribution of each state and is able to construct robust model for temporal data. SOINN-DTW method is the novel method that enhanced Stochastic Dynamic Time Warping Method (Nakagawa,1986). To confirm the effectiveness of SOINN-DTW method, we present an extensive set of experiments that show how our method outperforms HMM and Stochastic Dynamic Time Warping Method in classifying phone data and gesture data.

Quantitative models are very successful forr extrapolating the basic trend-cycle component of time series. On the contrary time series models failed to handle adequately shocks or irregular events, that is non-periodic events such as oil crises, promotions, strikes, announcements, legislation etc. Forecasters usually prefer to use their own judgment in such problems. However their efficiency in such tasks is in doubt too and as a result the need of decision support tools in this procedure seem to be quite important. Forecasting with neural networks has been very popular across the Academia in the last decade. Estimating the impact of irregular events has been one of the most successful application areas. This study examines the relative performance of Artificial Neural Networks versus Multiple Linear Regression for estimating the impact of expected irregular future events.

Biomedical signals are generally contaminated with artifacts and noise. In case artifacts dominate, the useful signal can easily be extracted with projective subspace techniques. Then, biomedical signals which often represent one dimensional time series, need to be transformed to multi-dimensional signal vectors for the latter techniques to be applicable. In this work we propose the application of a greedy kernel Principal Component Analysis(KPCA) which allows to decompose the multidimensional vectors into components, and we will show that the one related with the largest eigenvalues correspond to an high-amplitude artifact that can be subtracted from the original.

Recent neurocognitive researches demonstrate how the natural processing of auditory sentences occurs. Nowadays, there is not an appropriate human-computer speech interaction, and this constitutes a computational challenge to be overtaked. In this direction, we propose a speech comprehension software architecture to represent the flow of this neurocognitive model. In this architecture, the first step is the speech signal processing to written words and prosody coding. Afterwards, this coding is used as input in syntactic and prosodic-semantic analyses. Both analyses are done concomitantly and their outputs are matched to verify the best result. The computational implementation applies wavelets transforms to speech signal codification and data prosodic extraction and connectionist models to syntactic parsing and prosodic-semantic mapping.

In time series prediction problems in which the current series presents a certain seasonality, the long term and short term prediction capabilities of a learned model can be improved by considering that seasonality as additional information within it. Kernel methods and specifically LS-SVM are receiving increasing attention in the last years thanks to many interesting properties; among them, these type of models can include any additional information by simply adding new variables to the problem, without increasing the computational cost. This work evaluates how including the seasonal information of a series in a designed recursive model might not only upgrade the performance of the predictor, but also allows to diminish the number of input variables needed to perform the modelling, thus being able to increase both the generalization and interpretability capabilities of the model.

Testing the validity of the Efficient Market Hypothesis (EMH) has been an unsolved argument for the investment community. The EMH states that the current market price incorporates all the information available, which leads to a conclusion that given the information available, no prediction of the future price changes can be made. On the other hand, technical analysis, which is essentially the search for recurrent and predictable patterns in asset prices, attempts to forecast future price changes. To the extend that the total return of a technical trading strategy can be regarded as a measure of predictability, technical analysis can be seen as a test of the EMH and in particular of the independent increments version of random walk. This paper is an initial attempt on creating an automated process, based on a combination of a rule-based system and a neural network, of recognizing one of the most common and reliable patterns in technical analysis, the head and shoulders pattern. The systematic application of this automated process on the identification of the head and shoulders pattern and the subsequent analysis of price behavior, in various markets can in principle work as a test of the EMH.

In this paper we discuss sparse least squares support vector regressors (sparse LS SVRs) defined in the reduced empirical feature space, which is a subspace of mapped training data. Namely, we define an LS SVR in the primal form in the empirical feature space, which results in solving a set of linear equations. The independent components in the empirical feature space are obtained by deleting dependent components during the Cholesky factorization of the kernel matrix. The independent components are associated with support vectors and controlling the threshold of the Cholesky factorization we obtain a sparse LS SVM. For linear kernels the number of support vectors is the number of input variables at most and if we use the input axes as support vectors, the primal and dual forms are equivalent. By computer experiments we show that we can reduce the number of support vectors without deteriorating the generalization ability.

Content-based image retrieval (CBIR) is an important and widely studied topic since it can have significant impact on multimedia information retrieval. Recently, support vector machine (SVM) has been applied to the problem of CBIR. The SVM-based method has been compared with other methods such as neural network (NN) and logistic regression, and has shown good results. Genetic algorithm (GA) has been increasingly applied in conjunction with other AI techniques. However, few studies have dealt with the combining GA and SVM, though there is a great potential for useful applications in this area. This paper focuses on simultaneously optimizing the parameters and feature subset selection for SVM without degrading the SVM classification accuracy by combining GA for CBIR. In this study, we show that the proposed approach outperforms the image classification problem for CBIR. Compared with NN and pure SVM, the proposed approach significantly improves the classification accuracy and has fewer input features for SVM.

In this paper, we describe a feature extraction method: Global and Local Preserving Projection (GLPP). GLPP is based on PCA and the recently proposed Locality Preserving Projection (LPP) method. LPP can preserve local information, while GLPP can preserve both global and local information. In this paper we investigate the potential of using GLPP for image categorization. More specifically, we experiment on palmprint images. Palmprint image has been attracting more and more attentions in the image categorization/recognition area in recent years. Experiment is based on benchmark dataset PolyU, using Error Rate as performance measure. Comparison with LPP and traditional algorithms show that GLPP is promising.

This paper presents iris recognition for personal identification using neural networks. Iris recognition system consists of localization of the iris region and generation of data set of iris images and then iris pattern recognition. One of the problems in iris recognition is fast and accurate localization of the iris image. In this paper, fast algorithm is used for the localization of the inner and outer boundaries of the iris region. Located iris is extracted from an eye image, and, after normalization and enhancement it is represented by a data set. Using this data set a neural network is applied for the classification of iris patterns. Results of simulations illustrate the effectiveness of the neural system in personal identification.

Imaging algorithms often require reliable methods to evaluate the quality effects of the visual artifacts that digital processing brings about. This paper adopts a no-reference objective method for predicting the perceived quality of images in a deterministic fashion. Principal Component Analysis is first used to assemble a set of objective features that best characterize the information in image data. Then a neural network, based on the Circular Back-Propagation (CBP) model, associates the selected features with the corresponding predictions of quality ratings and reproduces the scores process of human assessors. The neural model allows one to decouple the process of feature selection from the task of mapping features into a quality score. Results on a public database for an image-quality experiment involving JPEG compressed-images and comparisons with existing objective methods confirm the approach effectiveness.

This paper presents a bio-inspired connectionist approach for motion description through sequences of images. First, this approach is based on the architecture of oriented columns and the strong local and distributed interactions of the neurons in the primary visual cortex (V1). Secondly, in the integration and combination of their responses in the middle temporal area (MT). I propose an architecture in two layers : a causal spatio-temporal filtering (CSTF) of Gabor-like type which captures the oriented contrast and a mechanism of antagonist inhibitions (MAI) which estimates the motion. The first layer estimates the local orientation and speed, the second layer classifies the motion (global response) and both describe the motion and the pursuit trajectory. This architecture has been evaluated on sequences of natural and synthetic images.

This work investigates the role of color in object recognition. We approach the problem from a computational perspective by measuring the performance of biologically inspired object recognition methods. As benchmarks, we use image datasets proceeding from a real-world object detection scenario and compare classification performance using color and gray-scale versions of the same datasets. In order to make our results as general as possible, we consider object classes with and without intrinsic color, partitioned into 4 datasets of increasing difficulty and complexity. For the same reason, we use two independent bio-inspired models of object classification which make use of color in different ways. We measure the qualitative dependency of classification performance on classifier type and dataset difficulty (and used color space) and compare to results on gray-scale images. Thus, we are able to draw conclusions about the role and the optimal use of color in classification and find that our results are in good agreement with recent psychophysical results.

Poses and gestures are an important part of the nonverbal inter-human communication. In the last years many different methods for estimating poses and gestures in the field of Human-Machine-Interfaces were developed. In this paper for the first time we present an experimental comparison of several re-implemented Neural Network based approaches for a demanding visual instruction task on a mobile system. For the comparison we used several Neural Networks (Neural Gas, SOM, LLM, PSOM and MLP) and a k-Nearest-Neighbourhood classificator on a common data set of images, which we recorded on our mobile robot

Horos

under real world conditions. For feature extraction we use Gaborjets and the features of a special histogram on the image. We also compare the results of the different approaches with the results of human subjects who estimated the target point of a pointing pose. The results obtained demonstrate that a cascade of MLPs is best suited to cope with the task and achieves results equal to human subjects.

In this paper, a SVM regression based method is proposed for background estimation and foreground detection. Incoming frames are treated as time series and a fixed-scale working-set selecting strategy is specifically designed for real-time background estimation. Experiments on two representat-ive videos demonstrate the application potential of the proposed algorithm and also reveal some problems underlying it. Both the positive and negative reports from our study offer some useful information for researchers both in the field of image processing and that of machine learning.

This study deals with image superresolution problems simultaneously with accompanying image registration problems. The goal of superresolution is to generate a high resolution image by integrating low-resolution degraded observed images. We propose a Bayesian approach whose prior is modeled as a compound Gaussian Markov random field (MRF). This approach is advantageous in preserving discontinuity in the original image, in comparison to the existing single-layer Gaussian MRF models. Maximum-marginalized-likelihood estimation of the registration parameters is carried out by a variational EM algorithm where hidden variables are marginalized out and the posterior distribution is approximated by a factorized trial distribution. High resolution image estimates are obtained as by-products of the EM algorithm. Experiments show that our Bayesian approach with two-layer compound models exhibits better performance in terms of mean square error and visual quality than the single-layer model.

This paper presents an adaptive neurofuzzy controller for tip position tracking control of a single link flexible manipulator. The controller has a self- organizing fuzzy neural structure in which fuzzy rules are generated during the control process using an online learning algorithm. In order to demonstrate the superior performance of the proposed controller, the results are compared with those obtained by using the proportional-derivative (PD) and neural network controllers. Moreover, since the proposed controller requires no a priori knowledge about the system, it can efficiently cope with the uncertainties such as payload mass variations.

This paper details a computationally efficient (suboptimal) nonlinear Model Predictive Control (MPC) algorithm with structured neural models and discusses its application to a polymerisation reactor. Thanks to the nature of the model it is not used recursively, the prediction error is not propagated. The model is used on-line to determine a local linearisation and a nonlinear free trajectory. The algorithm needs solving on-line only a quadratic programming problem. It gives closed-loop control performance similar to that obtained in the fully-fledged nonlinear MPC, which hinges on non-convex optimisation.

A primary goal for an autonomous mobile robot is to explore and perfrom simultaneous localization and mapping (SLAM). During SLAM, the robot must balance the opposing desires of pose certainty maintenance and information gain. Much of previous research has ignored the need of pose maintenance. This paper provides the first step in developing a neural dynamics based algorithm which considers both information gain and pose maintenance when determining the robot’s next pose. Simulation results show that the algorithm is able to provide the robot with an exploration plan to fully explore the tested environments. The next step is to apply the algorithm in a full SLAM environment.

This paper discusses the problem of cooperation of economic optimisation with Model Predictive Control (MPC) algorithms when the dynamics of disturbances is comparable with the dynamics of the process. A dynamic neural model is used in the suboptimal nonlinear MPC algorithm with Nonlinear Prediction and Linearisation (MPC-NPL), a steady-state neural model is used in approximate economic optimisation which is executed as frequently as the MPC algorithm. The MPC-NPL algorithm requires solving on-line only a quadratic programming problem whereas approximate economic optimisation needs solving a linear programming problem. As a result, the necessity of repeating two nonlinear optimisation problems at each sampling instant is avoided.

Reservoir Computing (RC) uses a randomly created recurrent neural network where only a linear readout layer is trained. In this work, RC is used for detecting complex events in autonomous robot navigation. This can be extended to robot localization based solely on sensory information. The robot thus builds an implicit map of the environment without the use of odometry data. These techniques are demonstrated in simulation on several complex and even dynamic environments.

This paper demonstrates the use of CMAC neural networks in real world applications for the system identification and control of nonlinear systems. As a testbed application, the problem of regulating fluid height in a column is considered. A dynamic nonlinear model of the process is obtained using fundamental physical laws and by training a CMAC neural network using experimental input-output data. The CMAC model is used to implement a model reference control system. Successful experimental results are obtained in the presence of disturbances.

This work presents a methodology for the land-cover classification of satellite images based on clustering of the Kohonen’s self-organizing map (SOM). The classification task is carried out using a three-stage approach. At the first stage, the SOM is used to quantize and to represent the original patterns of the image in a space of smaller dimension. At the second stage of the method, a filtering process is applied on the SOM prototypes, wherein prototypes associated to input patterns that incorporate more than one land cover class and prototypes that have null activity are excluded in the next stage or simply eliminated of the analysis. At the third and last stage, the SOM prototypes are segmented through a hierarchical clustering method which uses the neighborhood relation of the neurons and incorporates spatial information in its merging criterion. The experimental results show an application example of the proposed methodology on an IKONOS image.

In this paper, a Multilayer Perceptron (MLP) is proposed as a radar detector of known targets in Weibull-distributed clutter. The MLP is trained in a supervised way using the Levenberg-Marquardt backpropagation algorithm to minimize the Mean Square Error, which is able to approximate the Neyman-Pearson detector. Due to the impossibility to find analytical expressions of the optimum detector for this kind of clutter, a suboptimum detector is taken as reference, the Target Sequence Known A Priori (TSKAP) detector. Several sizes of MLP are considered, where even MLPs with very low sizes are able to outperform the TSKAP detector. On the other hand, a sensitivity study with respect to target parameters, as its doppler frequency, is made for different clutter conditions. This study reveals that both detectors work better for high values of target doppler frequency and one-lag correlation coefficient of the clutter. But the most important conclusion is that, for all the cases of the study, the MLP-based detector outperforms the TSKAP one. Moreover, the performance improvement achieved by the MLP-based detector is higher for lower probabilities of false alarm than for higher ones.

This work describes a complete indoor location system, from its creation, development and deployment. This location system is a capable way of retrieving the position of wireless devices using a simple software solution, no additional hardware is necessary. The positioning engine uses artificial neural networks (ANN) to describe the behaviour of a specific indoor propagation channel. The training of the ANN is assured using a slight variation of the radio frequency fingerprinting technique. Results show that the location system has high accuracy with an average error below two meters.

The accurate calculation of the resonance frequency and input resistance of microstrip antennas is a key factor to guarantee their correct behavior. In this paper we presented an adaptive neuro-fuzzy inference system (ANFIS) that calculates resonant frequency and input impedance of the microstrip dipole antenna’s (MSDAs). Although the MSDAs’ resonant frequency greatly depends on the dipole’s length, it also depends on the dipole’s width, the antenna substrate’s permittivity value, and its size (which affects resonant frequency). Input impedance, like resonant frequency, changes with these parameters. According to test results accuracy of ANFIS is calculated 98.91% for resonant frequency while 95.81% for input resistance calculation.

A procedure to estimate the parameters of GARCH processes with non-parametric innovations is proposed. We also design an improved technique to estimate the density of heavy-tailed distributions with real support from empirical data. The performance of GARCH processes with non-parametric innovations is evaluated in a series of experiments on the daily log-returns of IBM stocks. These experiments demonstrate the capacity of the improved estimator to yield a precise quantification of market risk.

This paper describes a research where the main goal was to predict the future values of a time series of the hourly demand of Portugal global electricity consumption in the following day. In a preliminary phase several regression techniques were experimented: K Nearest Neighbors, Multiple Linear Regression, Projection Pursuit Regression, Regression Trees, Multivariate Adaptive Regression Splines and Artificial Neural Networks (ANN). Having the best results been achieved with ANN, this technique was selected as the primary tool for the load forecasting process. The prediction for holidays and days following holidays is analyzed and dealt with. Temperature significance on consumption level is also studied. Results attained support the adopted approach.

This work uses artificial neural networks, whose architecture were developed using genetic algorithm to realize the hourly load forecasting based on the monthly total load consumption registered by the Energy Company of Pernambuco (CELPE). The proposed Hybrid Intelligent System – HIS was able to find the trade-off between forecast errors and network complexity. The load forecasting produces the essence to increase and strengthen in the basic grid, moreover study into program and planning of the system operation. The load forecasting quality contributes substantially to indicating more accurate consuming market, and making electrical system planning and operating more efficient. The forecast models developed comprise the period of 45 and 49 days ahead. Comparisons between the four models were achieved by using historical data from 2005.

Many applications would emerge from the development of artificial systems able to accurately localize and identify sound sources. However, one of the main difficulties of such kind of system is the natural presence of multiple sound sources in real environments. This paper proposes a pulsed neural network based system for separation and recognition of multiple sound sources based on the difference on time lag of the different sources. The system uses two microphones, extracting the time difference between the two channels with a chain of coincidence detection pulsed neurons. An unsupervised neural network processes the firing information corresponding to each time lag in order to recognize the type of the sound source. Experimental results show that three simultaneous musical instruments’ sounds could be successfully separated and recognized.

This paper presents a novel system that performs text-independent speaker authentication using new spiking neural network (SNN) architectures. Each speaker is represented by a set of prototype vectors that is trained with standard Hebbian rule and

winner-takes-all

approach. For every speaker there is a separated spiking network that computes normalized similarity scores of MFCC (Mel Frequency Cepstrum Coefficients) features considering speaker and background models. Experiments with the VidTimit dataset show similar performance of the system when compared with a benchmark method based on vector quantization. As the main property, the system enables optimization in terms of performance, speed and energy efficiency. A procedure to create/merge neurons is also presented, which enables adaptive and on-line training in an evolvable way.

We present a biologically-inspired neural model addressing the problem of transformations across frames of reference in a posture imitation task. Our modeling is based on the hypothesis that imitation is mediated by two concurrent transformations selectively sensitive to spatial and anatomical cues. In contrast to classical approaches, we also assume that separate instances of this pair of transformations are responsible for the control of each side of the body. We also devised an experimental paradigm which allowed us to model the interference patterns caused by the interaction between the anatomical on one hand, and the spatial imitative strategy on the other hand. The results from our simulation studies thus provide predictions of real behavioral responses.

The main topic in this work was the development of a hybrid intelligent system for the hourly load forecasting in a time period of 7 days ahead, using a combination of Artificial Neural Network and Adaptive Neuro-Fuzzy Inference System. The hourly load forecasting was accomplished in two steps: in the first one, two ANNs are used to forecast the total load of the day, where one of the networks forecasts the working days (Monday through Friday), and the other forecasts the Saturdays, Sundays and public holidays; in the second step, the ANFIS was used to give the hourly consumption rate of the load. The proposed system presented a better performance as against the system currently used by Energy Company of Pernambuco, named PREVER. The simulation results showed an hourly mean absolute percentage error of 2.81% for the year 2005.

A universal approximator, such as multilayer perceptron, is a tool that allows mapping of any multidimensional continuous function. The aim of this paper is to discuss a method of perceptron training that would result in its ability to map the functions constituting the solutions of partial differential equations of first and second order. The developed algorithm has been validated by means of equations whose analytical solutions are known.

Capturing regularities in high-dimensional data is an important problem in machine learning and signal processing. Here we present a statistical model that learns a nonlinear representation from the data that reflects abstract, invariant properties of the signal without making requirements about the kind of signal that can be processed. The model has a hierarchy of two layers, with the first layer broadly corresponding to Independent Component Analysis (ICA) and a second layer to represent higher order structure. We estimate the model using the mathematical framework of Score Matching (SM), a novel method for the estimation of non-normalized statistical models. The model incorporates a squaring nonlinearity, which we propose to be suitable for forming a higher-order code of invariances. Additionally the squaring can be viewed as modelling subspaces to capture residual dependencies, which linear models cannot capture.

We propose a new nonparametric test for component independence which is based on application of data compressors to ranked data. For two-component data sample the idea is to break the sample in two parts and permute one of the components in the second part, while leaving the first part intact. The resulting two samples are then jointly ranked and a data compressor is applied to the resulting (binary) data string. The components are deemed independent if the string cannot be compressed. This procedure gives a provably valid test against all possible alternatives (that is, the test is distribution-free) provided the data compressor was ideal.

In this paper, the

K

-pages graph layout problem is solved by a new neural model. This model consists of two neural networks performing jointly in order to minimize the same energy function. The neural technique applied to this problem allows to reduce the energy function by changing outputs from both networks –outputs of first network representing location of nodes in the nodes line, while the outputs of the second one meaning the page where the edges are drawn.

A detailed description of the model is presented, and the technique to minimize an energy function is fully described. It has proved to be a very competitive and efficient algorithm, in terms of quality of solutions and computational time, when compared to the state-of-the-art heuristic methods specifically designed for this problem. Some simulation results are presented in this paper, to show the comparative efficiency of the methods.

Treatment of general structured information by neural networks is an emerging research topic. Here we show how representations for graphs preserving all the information can be devised by Recursive Principal Components Analysis learning. These representations are derived from eigenanalysis of extended vectorial representations of the input graphs. Experimental results performed on a set of chemical compounds represented as undirected graphs show the feasibility and effectiveness of the proposed approach.

We propose a method to estimate the graph structure from data for a Markov random field (MRF) model. The method is valuable in many practical situations where the true topology is uncertain. First the similarities of the MRF variables are estimated by applying methods from information theory. Then, employing multidimensional scaling on the dissimilarity matrix obtained leads to a 2D topology estimate of the system. Finally, applying uniform thresholding on the node distances in the topology estimate gives the neighbourhood relations of the variables, hence defining the MRF graph estimate. Conditional independence properties of a MRF model are defined by the graph topology estimate thus enabling the estimation of the MRF model parameters e.g. through the pseudolikelihood estimation scheme. The proposed method is demonstrated by identifying MRF model for a telecommunications network, which can be used e.g. in analysing the effects of stochastic disturbances to the network state.

In an attempt to bridge the gap between appraisal theory and the neuroscience of emotions, we have created a computational neural model in which a discrepancy between the internal value of global self-esteem and a more temporary, stimulus-inspired current self-esteem initiates an ongiong emotional response. We assign possible neural correlates to the nodes in this model, amongst which the orbitofrontal cortex and cingulate gyrus. We propose disruptions of the model analogous to states of depression.

In this paper, we will address the endeavors of three disciplines, Psychology, Neuroscience, and Artificial Neural Network (ANN) modeling, in explaining how the mind perceives and attends information. More precisely, we will shed some light on the efforts to understand the allocation of attentional resources to the processing of emotional stimuli. This review aims at informing the three disciplines about converging points of their research and to provide a starting point for discussion.

Over the last few years, functional Magnetic Resonance Imaging (fMRI) has emerged as a new and powerful method to map the cognitive states of a human subject to specific functional areas of the subject brain. Although fMRI has been widely used to determine average activation in different brain regions, the problem of automatically decoding the cognitive state from instantaneous brain activations has received little attention. In this paper, we study this prediction problem on a complex time-series dataset that relates fMRI data (brain images) with the corresponding cognitive states of the subjects while watching three 20 minute movies. This work describes the process we used to reduce the extremely high-dimensional feature space and a comparison of the models used for prediction. To solve the prediction task we explored a standard linear model frequently used by neuroscientists, as well as a

k-nearest neighbor

model, that now are the state-of-art in this area. Finally, we provide experimental evidence that non-linear models such as

multi-layer perceptron

and especially

recurrent neural networks

are significantly better.

Attention and emotion are closely interlinked and recent results have shown some of the neuro-physiological details of the effects of attention on emotion through the distractor devaluation (DD) effect. We develop a possible neural attention control architecture to explain the DD effect, and show by specific simulation how the N2pc (an early component of attention movement) can be encoded to produce encoding of devaluation of distractors.

Taking neuromodulation as a mechanism underlying emotions, this paper investigates how such a mechanism can bias an artificial neural network towards exploration of new courses of action, as seems to be the case in positive emotions, or exploitation of known possibilities, as in negative emotions such as predatory fear. We use neural networks of spiking leaky integrate-and-fire neurons acting as minimal disturbance systems, and test them with continuous actions. The networks have to balance the activations of all their output neurons concurrently. We have found that having the middle layer modulate the output layer helps balance the activations of the output neurons. A second discovery is that when the network is modulated in this way, it performs better at tasks requiring the exploitation of actions that are found to be rewarding. This is complementary to previous findings where having the input layer modulate the middle layer biases the network towards exploration of alternative actions. We conclude that a network can be biased towards either exploration of exploitation depending on which layers are being modulated.

We discuss evidence for the existence of mirror systems in the brain, including recent experimental results that demonstrate the use of shared pathways for the observation and execution of reaching and grasping actions. We then describe a brain based model of observational learning that explains the similarities and differences in levels of activation of brain regions during observation and execution of actions. We simulate a very simple paradigm whereby an actor performs an action which is observed and then repeated by the simulated animal. We discuss the implications and possible extensions of our model.

It is well known that our prior knowledge and experiences affect how we learn new concepts. Although several formal modeling attempts have been made to quantitatively describe the mechanisms about how prior knowledge influences concept learning behaviors, the underlying cognitive mechanisms that give rise to the prior knowledge effects remains unclear. In this paper, we introduce a computational cognitive modeling framework that is intended to describe how prior knowledge and experiences influence learning behaviors. In particular, we assume that it is not simply the prior knowledge stored in our memory trace influencing our behaviors, but it is also the learning strategies acquired through previous learning experiences that affect our learning behaviors. Two simulation studies were conducted and the results showed promising outcomes.

The paper discusses a novel model for concept learning and representation. Two levels of representation are used: exemplars and (generalized concepts) prototypes. The internal structure of the model is based on a semantic network using spreading activation. Categorisation and addition operations are supported in parallel. Forgetting of learned concepts is used in order to track dynamic and novel environments. The model is inspired by the corresponding psychological theories of exemplars and prototypes. Simulation results support the formulation of the model.

We present a model of a recurrent neural network, embodied in a minimalist articulated agent with a single link and joint. The configuration of the agent defined by one angle (degree of freedom), is determined by the activation state of the neural network. This is done by contracting a muscle with many muscular fibers, whose contraction state needs to be coordinated to generate high amplitude link displacements. In networks without homeostasic (self-regulatory) mechanism the neural state dynamics and the configuration state dynamics converges to a fixed point. Introduction of random noise, shows that fixed points are meta-stable. When neural units are endowed with homeostasic mechanisms in the form of threshold adjustment, the dynamics of the configuration angle and neural state becomes aperiodic. Learning mechanisms foster functional and structural cluster formation, and modifies the distribution of the kinetic energy of the network. We also present a meta-model of embodied neural agents, that identifies self-perturbation as a mechanism for neural development without a teacher.

Language skills are dominantly implemented in one hemisphere (usually the left), with the pre-frontal areas playing a critical part (the inferior frontal area of Broca and the superior temporal area of Wernicke), but a network of additional regions in the brain, including some from the non-dominant hemisphere, are necessary for complete language functionality. This paper presents a neural architecture built on spiking neurons which implements a mechanism of associating representations of concepts in different modalities; as well as integrating sequential language input into a coherent representation/interpretation of an instruction. It follows the paradigm of temporal binding, namely synchronisation and phase locking of distributed representations in nested gamma-theta oscillations. The functionality of the architecture is presented in a set of experiments of language instructions given to a real robot.

Neurocognitive processes responsible for representation of meaning and understanding of words are investigated. First a review of current knowledge about word representation, recent experiments linking it to associative memory and to right hemisphere synchronous activity is presented. Various conjectures on how meaning arises and how reasoning and problem solving is done are presented. These inspirations are used to make systematic approximation to spreading activation in semantic memory networks. Using hierarchical ontologies representations of short texts are enhanced and it is shown that high-dimensional vector models may be treated as a snapshot approximation of the neural activity. Clustering short medical texts into different categories is greatly enhanced by this process, thus facilitating understanding of the text.

The purpose of this study is to construct a computational model of the metaphor understanding process. This study assumes that metaphor understanding consists of two processes. The first is a categorization process; a target is assigned to an ad hoc category of which the vehicle is a prototypical member. The second is a dynamic interaction process; the target assigned to the ad hoc category is influenced by dynamic interaction among features. Feature emergence is extracted through this dynamic interaction. In this study, a model of metaphor understanding is constructed based on this assumption by applying a statistical analysis of large-scale corpus. Further a psychological experiment is conducted in order to verify the psychological validity of the constructed model of metaphor understanding. Reflecting the fact that the constructed model represents more appropriate features of a metaphor than a model incorporating only the categorization process, the experimental results support its validity.

We develop a model of a set of novelty and familiarity detectors in the hippocampus which possess unique properties, and have been recently reported in [1]. The model uses both inhibition and disinhibition, together with a suitable output function of prefrontal object representations, to create the separate novelty and familiarity detectors with the observed properties. We conclude the paper with a discussion of the relation of this novelty system with that presented by numerous other techniques.