Adaptive and Natural Computing Algorithms

Many times we come across the need to simplify or reduce an input data space in order to achieve a better model or better performance of an artificial intelligence solution. The well known PCA, ICA and rough sets can simplify and reduce input data space but they cannot transform real input data vectors into binary ones. Binary training vectors can simplify a training process of neural networks and let them to construct more compact topologies. This paper introduces a new algorithm that reduces input data space and simultaneously automatically lossy transforms real input training data vectors into binary vectors so that they do not lose their discrimination properties. The problem is how to effectively transform real input training data vectors into binary vectors so that an input data space could be simplified and the transformed binary vectors would be enough representative to be able to discriminate all training samples of all classes correctly? The described lossy conversion makes possible to achieve better generalization results for various soft-computing algorithms, can be widely used and avoids the curse of dimensionality problem. This paper introduces a new Automatic Discriminative Lossy Binary Conversion Algorithm (ADLBCA) that is able to solve all these tasks. Generally, no other method can simultaneously and so fast do all these tasks.

The tractability of neural-network approximation is investigated. The dependence of worst-case errors on the number of variables is studied. Estimates for Gaussian radial-basis-function and perceptron networks are derived.

In this paper new approach to treat incomplete data has been proposed. It has been based on the evolution of imputation strategies built using both non-parametric and parametric imputation methods. Genetic algorithms and multilayer perceptrons have been applied to develop a framework for constructing the imputation strategies addressing multiple incomplete attributes. Furthermore we evaluate imputation methods in the context of not only the data they are applied to, but also the model using the data. The accuracy of classification on data sets completed using obtained imputation strategies has been described. The results outperform the corresponding results calculated for the same data sets completed using standard strategies.

The generalization capability of a multilayer perceptron can be adjusted by adding a penalty (weight decay) term to the cost function used in the training process. In this paper we present a possible heuristic method for finding a good coefficient for this regularization term while, at the same time, looking for a well-regularized MLP model. The simple heuristic is based on validation error, but not strictly in the sense of early stopping; instead, we compare different coefficients using a subdivision of the training data for quality evaluation, and in this way we try to find a coefficient that yields good generalization even after a training run that ends up in full convergence to a cost minimum, given a certain accuracy goal. At the time of writing, we are still working on benchmarking and improving the heuristic, published here for the first time.

The problem we address in this paper is that of finding effective and parsimonious patterns of connectivity in sparse associative memories. This problem must be addressed in real neuronal systems, so results in artificial systems could throw light on real systems. We show that there are efficient patterns of connectivity and that these patterns are effective in models with either spiking or non-spiking neurons. This suggests that there may be some underlying general principles governing good connectivity in such networks.

Some enhancements and comments to approximation of 2D functions in orthogonal basis are presented. This is a direct extension of the results obtained in [2]. First of all we prove that a constant bias extracted from the function contributes to the error decrease. We demonstrate how to choose that bias prooving an appropriate theorem. Secondly we discuss how to select a 2D basis among orthonormal functions to achieve minimum error for a fixed dimension of an approximation space. Thirdly we prove that loss of orthonormality due to truncation of the arguments range of the basis functions does not effect the overall error of approximation and the formula for calculating of the expansion coefficients remains the same. As an illustrative example, we show how these enhanencements can be used.

In this paper, we investigate the use of artificial neural networks in order to solve the Shortest Common Superstring Problem. Concretely, the neural network used in this work is based on a multivalued model, MREM, very suitable for solving combinatorial optimization problems. We describe the foundations of this neural model, and how it can be implemented in the context of this problem, by taking advantage of a better representation than in other models, which, in turn, contributes to ease the computational dynamics of the model. Experimental results prove that our model outperforms other heuristic approaches known from the specialized literature.

Feedforward neural networks have been established as versatile tools for nonlinear black-box modeling, but in many data-mining tasks the choice of relevant inputs and network complexity is still a major challenge. Statistical tests for detecting relations between inputs and outputs are largely based on linear theory, and laborious retraining combined with the risk of getting stuck in local minima make the application of exhaustive search through all possible network configurations impossible but for toy problems. This paper proposes a systematic method to tackle the problem where an output shall be estimated on the basis of a (large) set of potential inputs. Feedforward neural networks of multi-layer perceptron type are used in the three-stage modeling approach: First, starting from sufficiently large networks an efficient pruning method is applied to detect a pool of potential model candidates. Next, the Akaike weights are used as to select the actual Kullback-Leibler best models in the pool. Third, the hidden nodes of these networks are available for the final network, where mixed-integer linear programming is applied to find the optimal combination of

M

hidden nodes, and the corresponding upper-layer weights. The procedure outlined is demonstrated to yield parsimonious models for a nonlinear benchmark problem, and to detect the relevant inputs.

This paper discusses a Model Predictive Control (MPC) structure for economic optimisation of nonlinear technological processes. It contains two parts: an MPC economic optimiser/constraint governor and an unconstrained MPC algorithm. Two neural models are used: a dynamic one for control and a steady-state one for economic optimisation. Both models are linearised on-line. As a result, an easy to solve on-line one quadratic programming problem is formulated. Unlike the classical multilayer control system structure, the necessity of repeating two nonlinear optimisation problems at each sampling instant is avoided.

This paper is concerned with RBF neural multi-models and a computationally efficient nonlinear Model Predictive Control (MPC) algorithm based on such models. The multi-model has an ability to calculate predictions over the whole prediction horizon without using previous predictions. Unlike the classical Nonlinear Auto Regressive with eXternal input (NARX) model, the multi-model is not used recursively in MPC, the prediction error is not propagated. The presented MPC algorithm needs solving on-line only a quadratic programming problem but in practice it gives closed-loop control performance similar to that obtained in nonlinear MPC, which hinges on on-line non-convex optimisation.

Neural networks have proved to be effective in solving a wide range of problems. As problems become more and more demanding, they require larger neural networks, and the time used for learning is consequently greater. Parallel implementations of learning algorithms are therefore vital for a useful application. Implementation, however, strongly depends on the features of the learning algorithm and the underlying hardware architecture. For this experimental work a dynamic problem was chosen which implicates the use of recurrent neural networks and a learning algorithm based on the paradigm of learning automata. Two parallel implementations of the algorithm were applied - one on a computing cluster using MPI and OpenMP libraries and one on a graphics processing unit using the CUDA library. The performance of both parallel implementations justifies the development of parallel algorithms.

The aim of this paper is to present the use of Growing Competitive Neural Networks as a precise method to track moving objects for video-surveillance. The number of neurons in this neural model can be automatically increased or decreased in order to get a one-to-one association between objects currently in the scene and neurons. This association is kept in each frame, what constitutes the foundations of this tracking system. Experiments show that our method is capable to accurately track objects in real-world video sequences.

In this study, a self-organizing map (SOM) -based process analysis and parameter approximation method was used to the emission analysis of a circulating fluidized bed process. The aim was to obtain the optimal process parameters in respect to the flue gas nitrogen oxide (NO

x

) content in different predefined states of process. The data processing procedure in the research went as follows. First, the process data were processed by using a self-organizing map and k-means clustering to generate subsets representing the separate process states in the boiler. These process states represent the higher level process conditions in the combustion, and can include for example start-ups, shutdowns, and idle times in addition to the normal process flow. Next, optimal areas were discovered from the map within each process state, and the reference vectors of the optimal neurons were used to approximate the values of desired process parameters. In addition, a subtraction analysis of reference vectors was performed to analyze the optimal situations. In conclusion, the method showed potential considering its wider use in the field of energy production.

This paper presents a hierarchical self-organizing neural network for intrusion detection. The proposed neural model consists of a hierarchical architecture composed of independent growing self-organizing maps (SOMs). The SOMs have shown to be successful for the analysis of high-dimensional input data as in data mining applications such as network security. An intrusion detection system (IDS) monitors the IP packets flowing over the network to capture intrusions or anomalies. One of the techniques used for anomaly detection is building statistical models using metrics derived from observation of the user’s actions. The proposed growing hierarchical SOM (GHSOM) address the limitations of the SOM related to their static architecture. Experimental results are provided by applying the well-known KDD Cup 1999 benchmark data set, which contains a great variety of simulated networks attacks. Randomly selected subsets that contain both attacks and normal records from this benchmark are used for training the GHSOM. Before training, a transformation for qualitative features present in the benchmark data set is proposed in order to compute distance among qualitative values. Comparative results with other related works are also provided.

This research deals with the use of self-organising maps for the classification of text documents. The aim was to classify documents to separate classes according to their topics. We therefore constructed self-organising maps that were effective for this task and tested them with German newspaper documents. We compared the results gained to those of

k

nearest neighbour searching and

k

-means clustering. For five and ten classes, the self-organising maps were better yielding as high average classification accuracies as 88-89%, whereas nearest neighbour searching gave 74-83% and

k

-means clustering 72-79% as their highest accuracies.

A genetic algorithm for constructing cocyclic Hadamard matrices over a given group is described. The novelty of this algorithm is the guided heuristic procedure for reproduction, instead of the classical crossover and mutation operators. We include some runs of the algorithm for dihedral groups, which are known to give rise to a large amount of cocyclic Hadamard matrices.

Evolutionary computing is widely used to tune intelligent systems which incorporate expert knowledge with data. The linguistic equation (LE) approach is an efficient technique for developing truly adaptive, yet understandable, systems for highly complex applications. Process insight is maintained, while data-driven tuning relates the measurements to the operating areas. Genetic algorithms are well suited for LE models based on nonlinear scaling and linear interactions. New parameter definitions have been developed for the scaling functions to handle efficiently the parameter constraints of the monotonously increasing second order polynomials. While identification approaches are used to define the model structures of the dynamic models. Cascade models, effective delays and working point models are also represented with LE models, i.e. the whole system is configured with a set of parameters. Results show that the efficiency of the systems improves considerably after the implementation of simultaneous tuning of all parameters.

A new evolutionary algorithm, Elitistic Evolution (termed EEv), is proposed in this paper. EEv is an evolutionary method for numerical optimization with adaptive behavior. EEv uses small populations (smaller than 10 individuals). It have an adaptive parameter to adjust the balance between global exploration and local exploitation. Elitism have great influence in EEv’ proccess and that influence is also controlled by the adaptive parameter. EEv’ crossover operator allows a recently generated offspring individual to be parent of other offspring individuals of its generation. It requires the configuration of two user parameters (many state-of-the-art approaches uses at least three). EEv is tested solving a set of 16 benchmark functions and then compared with Differential Evolution and also with some well-known Memetic Algorithms to show its efficiency. Finally, EEv is tested solving a set of 10 benchmark functions with very high dimensionality (50, 100 and 200 dimensions) to show its robustness.

This paper deals with a Multiple Sequence Alignment problem, for which an implementation of the Prototype Optimization with Evolved Improvement Steps (POEMS) algorithm has been proposed. The key feature of the POEMS is that it takes some initial solution, which is then iteratively improved by means of what we call evolved hypermutations. In this work, the POEMS is seeded with a solution provided by the Clustal X algorithm. Major result of the presented experiments was that the proposed POEMS implementation performs significantly better than the other two compared algorithms, which rely on random hypermutations only. Based on the carried out analyses we proposed two modifications of the POEMS algorithm that might further improve its performance.

Scatter search (SS) is an evolutionary algorithm (EA) becoming more important in current researches as the increasing number of publications shows. It is a very promising method for solving combinatorial and nonlinear optimisation problems. This algorithm is being widely implemented for solving problems not taking long, but in case of processes requiring of high execution times likely to be executed using grid computing there is not an implementation for it. Some problems arise when we try to execute this algorithm using the grid, but once they are solved, the obtained results are really promising for many complex and scientific applications like, for example, applications for optimising nuclear fusion devices. Using concurrent programming and distributed techniques associated to the grid, the algorithm works as it could do it in a single computer.

The paper proposes combining a multi-agent system paradigm with the gene expression programming (GEP) to obtain solutions to the resource constrained project scheduling problem with time lags. The idea is to increase efficiency of the GEP algorithm through parallelization and distribution of the computational effort. The paper includes the problem formulation, the description of the proposed GEP algorithm and details of its implementation using the JABAT platform. To validate the approach computational experiment has been carried out. Its results confirm that the agent based gene expression programming can be considered as a promising tool for solving difficult combinatorial optimization problems.

Barkhausen noise is used in non-destructive testing of ferromagnetic materials. It has been shown to be sensitive to material properties but the reported results are more or less qualitative. The quantitative prediction of the material properties from the Barkhausen noise signal is challenging. In order to develop reliable models, the feature selection is critical. The feature selection method applied in this study utilizes genetic algorithms with cross-validation based objective function. Cross-validation is used because the amount of data is limited. The results show that genetic algorithms can be successfully applied to feature selection. The obtained results are reliable and rather consistent with the results obtained earlier.

We propose a statistical methodology for comparing the performance of evolutionary algorithms that iteratively generate candidate optima over the course of many generations. Performance data are analyzed using multiple hypothesis testing to compare competing algorithms. Such comparisons may be drawn for general performance metrics of any iterative evolutionary algorithm with any data distribution. We also propose a data reduction technique to reduce computational costs.

The paper presents a novel identification method, which makes use of genetic programming for concomitant flexible selection of models structure and parameters. The case of nonlinear models, linear in parameters is addressed. To increase the convergence speed, the proposed algorithm considers customized genetic operators and a local optimization procedure, based on QR decomposition, able to efficiently exploit the linearity of the model subject to its parameters. Both the model accuracy and parsimony are improved via a multiobjective optimization, considering different priority levels for the involved objectives. An enhanced Pareto loop is implemented, by means of a special fitness assignment technique and a migration mechanism, in order to evolve accurate and compact representations of dynamic nonlinear systems. The experimental results reveal the benefits of the proposed methodology within the framework of an industrial system identification.

In this paper we present application of genetic programming (GP) [1] to evolution of indirect encoding of neural network weights. We compare usage of original HyperNEAT algorithm with our implementation, in which we replaced the underlying NEAT with genetic programming. The algorithm was named HyperGP. The evolved neural networks were used as controllers of autonomous mobile agents (robots) in simulation. The agents were trained to drive with maximum average speed. This forces them to learn how to drive on roads and avoid collisions. The genetic programming lacking the NEAT complexification property shows better exploration ability and tends to generate more complex solutions in fewer generations. On the other hand, the basic genetic programming generates quite complex functions for weights generation. Both approaches generate neural controllers with similar abilities.

Results of simulation studies on the application of genetic algorithms (GA) for solving an inverse problem, tomographic reconstruction, using time-of-flight (TOF) data from ultrasound transmission tomography are presented. The TOF data is simulated without taking into consideration the diffraction effects of ultrasound which is reasonably valid when the impedance mismatch in the specimen under consideration is small. The proposed GA based reconstruction algorithm is described and the results for a number of cases are discussed. The sensitivity of the proposed algorithm is studied for various GA parameters viz. the population size, maximum number of generations, crossover probability, and mutation probability. A time complexity analysis of the proposed algorithm shows that the reconstruction times and number of unknowns bears a near quadratic relation enabling the prediction of reconstruction times when dealing with higher resolutions. The performance of proposed algorithm to the reconstruction when TOF data is contaminated with noise is also analyzed and presented. The results obtained are found to be consistent for a wide range of resolutions, type, size, and shape of inclusions.

We propose a method for estimating sensor network topology using only time-series sensor data without prior knowledge of the locations of sensors. Along with the advances in computer equipment and sensor devices, various sensor network applications have been proposed. Topology information is often mandatory for predicting and assisting human activities in these systems. However, it is not easy to configure and maintain this information for applications in which many sensors are used. The proposed method estimates the topology accurately and efficiently using ant colony optimization (ACO). Our basic premise is to integrate ACO with the reliability of acquired sensor data for the adjacency to construct the accurate topology. We evaluated our method using actual sensor data and showed that it is superior to previous methods.

The impact of problem extents and network sizes on learning in recurrent neural networks is analysed in terms of structural parameters of related graphs. In previous work the influence of learning on the changes of the typical parameters such as characteristic path length, clustering coefficient, degree distribution and entropy, was investigated. In the present work the focus is enlarged to the scaling problem of the learning paradigm. The results prove the scalability of learning procedures due to the retained dynamics of the parameters during learning with different problem extents and network sizes.

A novel architecture for a hierarchical classifier (

HC

) is defined. The objective is to combine several weak classifiers to form a strong one, but a different approach from those known,

e.g.

AdaBoost, is taken: the training set is split on the basis of previous classifier misclassification between output classes. The problem is split into overlapping subproblems, each classifying into a different set of output classes. This allows for a task size reduction as each sub-problem is smaller in the sense of lower number of output classes, and for higher accuracy. The

HC

proposes a different approach to the boosting approach.

The groups of output classes overlap, thus examples from a single class may end up in several subproblems. It is shown, that this approach ensures that such hierarchical classifier achieves better accuracy. A notion of generalized accuracy is introduced.

The sub-problems generation is simple as it is performed with a clustering algorithm operating on classifier outputs. We propose to use the Growing Neural Gas [1] algorithm, because of its good adaptiveness.

A statistical model is presented as an alternative to negative selection in anomaly detection of discrete data. We extend the use of probabilistic generative models from fixed-length binary strings into variable-length strings from a finite symbol alphabet using a mixture model of multinomial distributions for the frequency of adjacent symbols in a sliding window over a string. Robust and localized change analysis of text corpora is viewed as an application area.

Random swap-based clustering is very simple to implement and guaranteed to find the correct clustering if iterated long enough. However, its quadratic dependency on the number of clusters can be too slow in case of some data sets. Deterministic selection of the swapped prototype can speed-up the algorithm but only if the swap can be performed fast enough. In this work, we introduce an efficient implementation of the swap-based heuristic and compare its time-distortion efficiency against random and deterministic variants of the swap-based clustering, and repeated k-means.

Different clustering algorithms achieve different results with certain data sets because most clustering algorithms are sensitive to the input parameters and the structure of data sets. The way of evaluating the result of the clustering algorithms, cluster validity, is one of the problems in cluster analysis. In this paper, we build a framework for cluster validity process, while proposing a sum-of-squares based index for purpose of cluster validity. We use the resampling method in the framework to analyze the stability of the clustering algorithm, and the certainty of the cluster validity index. For homogeneous data based on independent variables, the proposed clustering validity index is effective in comparison to some other commonly used indexes.

We propose a generalized model with configurable discretizer actuators as a solution to the problem of the discretization of massive numerical datasets. Our solution is based on a concurrent distribution of the actuators and uses dynamic memory management schemes to provide a complete scalable basis for the optimization strategy. This prevents the limited memory from halting while minimizing the discretization time and adapting new observations without re-scanning the entire old data. Using different discretization algorithms on publicly available massive datasets, we conducted a number of experiments which showed that using our discretizer actuators with the Hellinger’s algorithm results in better performance compared to using conventional discretization algorithms implemented in the Hugin and Weka in terms of memory and computational resources. By showing that massive numerical datasets can be discretized within limited memory and time, these results suggest the integration of our configurable actuators into the learning process to reduce the computational complexity of modeling Bayesian networks to a minimum acceptable level.

The Levenstein or edit distance was developed as a metric for calculating distances between character strings. We are looking at weighting the different edit operations (insertion, deletion, substitution) to obtain different types of classifications of sets of strings. As a more general and less constrained approach we introduce topological notions and in particular uniformities.

Dynamic Bayesian networks (DBNs) are temporal probabilistic models for reasoning over time which are rapidly gaining popularity in modern Artificial Intelligence (AI) for planning. A number of Hidden Markov Model (HMM) representations of dynamic Bayesian networks with different characteristics have been developed. However, the varieties of DBNs have obviously opened up challenging problems of how to choose the most suitable model for specific real life applications especially by non-expert practitioners. Problem of convergence over wider time steps is also challenging. Finding solutions to these challenges is difficult. In this paper, we propose a new probabilistic modeling called Emergent Future Situation Awareness (EFSA) which predicts trends over future time steps to mitigate the worries of choosing a DBN model type and avoid convergence problems when predicting over wider time steps. Its prediction strategy is based on the automatic emergence of temporal models over two dimensional (2D) time steps from historical Multivariate Time Series (MTS). Using real life publicly available MTS data on a number of comparative evaluations, our experimental results show that EFSA outperforms popular HMM and logistic regression models. This excellent performance suggests its wider application in research and industries.

Hold-out and cross-validation are among the most useful methods for model selection and performance assessment of machine learning algorithms. In this paper, we present a computationally efficient algorithm for calculating the hold-out performance for sparse regularized least-squares (RLS) in case the method is already trained with the whole training set. The computational complexity of performing the hold-out is

, where

is the size of the hold-out set and

n

is the number of basis vectors. The algorithm can thus be used to calculate various types of cross-validation estimates effectively. For example, when

m

is the number of training examples, the complexities of

N

-fold and leave-one-out cross-validations are

O

(

m

3

/

N

2

 + (

m

2

n

)/

N

) and

O

(

mn

), respectively. Further, since sparse RLS can be trained in

O

(

mn

2

) time for several regularization parameter values in parallel, the fast hold-out algorithm enables efficient selection of the optimal parameter value.

The key problem in reinforcement learning is the exploration-exploitation tradeoff. An optimistic initialisation of the value function is a popular RL strategy. The problem of this approach is that the algorithm may have relatively low performance after many episodes of learning. In this paper, two extensions to standard optimistic exploration are proposed. The first one is based on different initialisation of the value function of goal states. The second one which builds on the previous idea explicitly separates propagation of low and high values in the state space. Proposed extensions show improvement in empirical comparisons with basic optimistic initialisation. Additionally, they improve anytime performance and help on domains where learning takes place on the sub-space of the large state space, that is, where the standard optimistic approach faces more difficulties.

We propose a learning framework to address multiclass challenges, namely visualization, scalability and performance. We focus on supervised problems by presenting an approach that uses prior information about training labels, manifold learning and support vector machines (SVMs).

We employ manifold learning as a feature reduction step, nonlinearly embedding data in a low dimensional space using Isomap (Isometric Mapping), enhancing geometric characteristics and preserving the geodesic distance within the manifold. Structured SVMs are used in a multiclass setting with benefits for final multiclass classification in this reduced space. Results on a text classification toy example and on ISOLET, an isolated letter speech recognition problem, demonstrate the remarkable visualization capabilities of the method for multiclass problems in the severely reduced space, whilst improving SVMs baseline performance.

Actor-Critics constitute an important class of reinforcement learning algorithms that can deal with continuous actions and states in an easy and natural way. In their original, sequential form, these algorithms are usually to slow to be applicable to real-life problems. However, they can be augmented by the technique of experience replay to obtain a satisfactory of learning without degrading their convergence properties. In this paper experimental results are presented that show that the combination of experience replay and Actor-Critics yields very fast learning algorithms that achieve successful policies for nontrivial control tasks in considerably short time. Namely, a policy for a model of 6-degree-of-freedom walking robot is obtained after 4 hours of the robot’s time.

In this study, the previously proposed Support Vector Machines Based Generalized Predictive Control (SVM-Based GPC) method [1] has been applied in controlling the experimental three-tank system. The SVM regression algorithms have been successfully employed in modeling nonlinear systems due to their advantageous peculiarities such as assurance of the global minima and higher generalization capability. Thus, the fact that better modeling accuracy yields better control performance has motivated us to use an SVM model in the GPC loop [1]. In the method, the SVM model of the unknown plant is used to predict future behavior of the plant and also to extract the gradient information which is used in the Cost Function Minimization (CFM) block. The experimental results have revealed that SVM-Based GPC provides very high performance in controlling the system, i.e., the liquid level of the system can track the different types of reference inputs with very small transient- and steady-state errors even in a noisy environment when it is controlled by SVM-Based GPC.

Time series clustering has been shown effective in providing useful information in various applications. This paper presents an efficient computational method for time series clustering and its application focusing creation of more accurate electricity use load curves for small customers. Presented approach was based on extraction of statistical features and their use in feature-based clustering of customer specific hourly measured electricity use data. The feature-based clustering was able to cluster time series using just a set of derive statistical features. The main advantages of this method were; ability to reduce the dimensionality of original time series, it is less sensitive to missing values and it can handle different lengths of time series. The performance of the approach was evaluated using real hourly measured data for 1035 customers during 84 days testing time period. After all, clustering resulted into more accurate load curves for this set of customers than present load curves used earlier. This kind of approach helps energy companies to take advantage of new hourly information for example in electricity distribution network planning, load management, customer service and billing.

Many cerebellar learning theories assume that long-term depression (LTD) of synapses between parallel fibres (PFs) and Purkinje cells (PCs) provides the basis for pattern recognition in the cerebellum. Previous work has suggested that PCs can use a novel neural code based on the duration of silent periods. These simulations have used a simplified learning rule, where the synaptic conductance was halved each time a pattern was learned. However, experimental studies in cerebellar slices show that the synaptic conductance saturates and is rarely reduced to less than 50% of its baseline value. Moreover, the previous simulations did not include plasticity of the synapses between inhibitory interneurons and PCs. Here we study the effect of LTD saturation and inhibitory synaptic plasticity on pattern recognition in a complex PC model. We find that the PC model is very sensitive to the value at which LTD saturates, but is unaffected by inhibitory synaptic plasticity.

The identification of two-phase flow regimes that occur in heated pipes is of paramount importance for monitoring nuclear installations such as boiling water reactors. A Sugeno-type fuzzy inference system is put forward for non-invasive, on-line flow regime identification. The proposed system is particularly efficient in that it employs a single directly computable input, four outputs calculated via subtractive clustering - each corresponding to one flow regime –, and four fuzzy inference rules. Despite its simplicity, the system accomplishes accurate identification of the flow regime of sequences of images from neutron radiography videos.

Analytical fuzzy predictive controllers are composed of a few local controllers grouped in the fuzzy Takagi–Sugeno model. Usually, they are designed using the PDC method. Stability of the resulting system (controller + control plant) may be checked using variants of Lyapunov stability criterion. One of the first such variants was the criterion developed by Tanaka and Sugeno. It is rather conservative criterion because, in its basic form, it does not take into consideration the shape of the membership functions. However, this drawback can be exploited in the proposed approach. After finding the Lyapunov matrix for the system with the analytical fuzzy predictive controller, using e.g. LMIs, it is possible to change the membership functions of the controller without sacrificing stability. It is done using the heuristic method. Thus, practically any shape of membership functions may be assumed.

A study is made of whether there is a significant statistical difference in performance between crisp and fuzzy rule-based classification. To do that, 12 datasets were chosen from the UCI repository that are widely used in the literature, and use was made of four different algorithms for rule induction —two crisp and two fuzzy— to classify them. Then a non-parametric statistical test was used for measuring the significance of the results, which indicated that both paradigms —crisp and fuzzy classification— are not different in the statistical meaning.

Model predictive control (MPC) algorithms are widely used in practical applications. They are usually formulated as optimization problems. If the model used for prediction is linear (or linearized on–line) then the optimization problem is standard, quadratic one. Otherwise, it is a nonlinear, in general, non–convex optimization problem. In the latter case the numerical problems may occur and time needed to calculate the control signals cannot be determined. Therefore approaches based on linear or linearized models are preferred in practical applications. In the paper a new algorithm is proposed, with prediction which employs heuristic fuzzy modeling. The algorithm is formulated as quadratic optimization problem but offers performance very close to that of MPC algorithm with nonlinear optimization. The efficiency of the proposed algorithm is demonstrated in the control system of the nonlinear control plant with inverse response – a chemical CSTR reactor.

Classifier combining is a popular method for improving quality of classification – instead of using one classifier, several classifiers are organized into a classifier system and their results are aggregated into a final prediction. However, most of the commonly used aggregation methods are static, i.e., they do not adapt to the currently classified pattern. In this paper, we provide a general framework for dynamic classifier systems, which use dynamic confidence measures to adapt to a particular pattern. Our experiments with random forests on 5 artificial and 11 real-world benchmark datasets show that dynamic classifier systems can significantly outperform both confidence-free and static classifier systems.

Generalization of string languages describing shapes in order to apply them to analyze a contour of bones in hand radiographs is proposed in this paper. An algorithm to construct a fuzzy shape descriptor is introduced. Next, basing on the fuzzy descriptor, a univocal description by fuzzy interference is realized. In prospects this method will be used to erosion detection of hand bones visible in hand radiographs.

This paper presents a method of music tunes recognition based on the Hough transform as well as its fuzzy version. One can also find here experimental results showing the effectiveness of the presented solutions. Perspectives of further work and quality improvements are also stated as a base for subsequent research.

This paper investigates the feasibility of automated benthic macro-invertebrate taxon identification based on support vector machines and a novel gradient based feature. Biomonitoring can efficiently pinpoint subtle environmental changes and is therefore globally widely used in ecological status assessment. However, all biomonitoring is cost-intensive due to the expert work needed to identify organisms. To relieve this problem an automated image recognition system for benthic macro-invertebrate taxonomical analysis is proposed in this work. Using a novel approach, we present high accuracy classification results, suggesting that automated taxa recognition for benthic macro-invertebrates is viable. Our study indicates that automated image recognition techniques can match human taxonomic identification accuracy and greatly reduce the costs of future taxonomic analysis.

High dimensionality, missing values, noise, and outliers are standard problems in gene expression data and are usually dealt with separately. In this paper, we propose an ideal point model that performs feature extraction, imputes missing values, and is robust to noise and outliers in a unified and unsupervised framework. We use the simplifying assumption that genes are either expressed or not expressed in order to obtain a parsimonious model. We present a fast Bayesian method for estimating the large number of parameters in the ideal point model. We apply the ideal point model to a leukemia data set, where it outperforms independent component analysis (ICA), a state of the art unsupervised feature extraction method.

The paper develops the methods of selection of the most important gene sequence on the basis of the gene expression microarray, corresponding to different types of cancer. Special two stage strategy of selection has been proposed. In the first stage we apply few different methods of assessment of the importance of genes. Each method stresses different aspects of the problem. In the second stage the selected genes are compared and the genes chosen most frequently by all methods of selection are treated as the most important and representative for the particular type of problem. The results of selection are analyzed using PCA and the selected genes form the input to the SVM classifier recognizing the classes of cancer. The numerical experiments confirm the efficiency of the proposed approach.

The geography which underpins the collection of Australian economic and social data is based on administrative areas, rather than having behavioural significance. Within most EU countries Coombes’ rules-based grouping algorithm [1] which uses commuting flow data has been employed to construct Travel to Work Areas, but other approaches including Intramax (a hierarchical technique) have also been utilised. Recent developments in fuzzy set theory have enabled the comparison of the local accuracy of the solutions associated with different grouping methods. This paper will utilise both the Intramax technique and the modified version of Coombes’ updated algorithm [2] to compare the properties of the solutions associated with grouping the Australian Statistical Local Areas using Journey to Work data from the 2006 Census.

In this paper we present our original methodology, in which Matching Pursuit is used for networks anomaly and intrusion detection. The architecture of anomaly-based IDS based on signal processing is presented. We propose to use mean projection of the reconstructed network signal to determine if the examined trace is normal or attacked. Experimental results confirm the efficiency of our method in worm detection scenario. The practical usability of the proposed approach in the intrusion detection tolerance system (

IDTS

) in the INTERSECTION project is presented.

In the recent financial crisis the incidence of important cases of bankruptcy led to a growing interest in corporate bankruptcy prediction models. In addition to building appropriate financial distress prediction models, it is also of extreme importance to devise dimensionality reduction methods able to extract the most discriminative features. Here we show that Non-Negative Matrix Factorization (NMF) is a powerful technique for successful extraction of features in this financial setting. NMF is a technique that decomposes financial multivariate data into a few basis functions and encodings using non-negative constraints. We propose an approach that first performs proper initialization of NMF taking into account original data using K-means clustering. Second, builds a bankruptcy prediction model using the discriminative financial ratios extracted by NMF decomposition. Model predictive accuracies evaluated in real database of French companies with statuses belonging to two classes (healthy and distressed) are illustrated showing the effectiveness of our approach.

Many complex, real world phenomena are difficult to study directly using controlled experiments. Instead, the use of computer simulations has become commonplace as a feasible alternative. However, due to the computational cost of these high fidelity simulations, the use of neural networks, kernel methods, and other surrogate modeling techniques has become indispensable. Surrogate models are compact and cheap to evaluate, and have proven very useful for tasks such as optimization, design space exploration, visualization, prototyping, and sensitivity analysis. Consequently, there is great interest in techniques that facilitate the construction of such regression models, while minimizing the computational cost and maximizing model accuracy. The model calibration problem in rainfall runoff modeling is an important problem from hydrology that can benefit from advances in surrogate modeling and machine learning in general. This paper presents a novel, fully automated approach to tackling this problem. Drawing upon advances in machine learning, hyperparameter optimization, model type selection, and sample selection (active learning) are all handled automatically.Increasing the utility of such methods for the domain expert.

Hybrid Gene Trajectory Clustering (GTC) algorithm [1,2] proves to be a good candidate to cluster multi-dimensional noisy time series. In this paper we apply the hybrid GTC to learn the structure of the stock market and to infer interesting relationships out of closing prices data. We conclude that hybrid GTC can successfully identify homogeneous and stable stock clusters and these clusters can further help the investors.

Prediction of financial distress of companies is analyzed with several machine learning approaches. We used Diane, a large database containing financial records from small and medium size French companies, from the year of 2002 up to 2007. It is shown that inclusion of historical data, up to 3 years priori to the analysis, increases the prediction accuracy and that Support Vector Machines are the most accurate predictor.

The paper presents some models and algorithms for the nonlinear optimization problem of multi-objects movement scheduling to synchronize their movement as well as properties of presented algorithms. Similarities and differences between defined problems and the classical tasks scheduling problem on parallel processors are shown. Two algorithms for synchronous movement scheduling are proposed and their properties are considered. One of the algorithm is based on dynamic programming approach and the second one uses some approximation techniques. Theoretical and experimental analysis of complexity and effectiveness of the algorithms as well as their practical usefulness are discussed.

Automatic segmentation of X-ray hand images is an important process. In studies such as skeletal bone age assessment, bone densitometry and analyzing of bone fractures, it is a necessary extremely difficult and complicated task. In this study, hand X-ray images were segmented by using C-means classifier. Extraction of bone tissue was realized in three steps: i) preprocessing, ii) feature extraction and iii) automatic segmentation. In preprocessing scheme, inhomogeneous intensity distribution is eliminated and some structural pre-information about hand was obtained in order to use in feature extraction block. In feature extraction process, edges between soft and bone tissues were extracted by proposed enhancement process. In automatic segmentation process, the image was segmented using C-mean classifier by taking care of local information. In the study, hand images of ten different people were segmented with high performances above 95%.

Image metamorphosis, commonly known as morphing, is a powerful tool for visual effects that consists of the fluid transformation of one digital image into another. There are many techniques for image metamorphosis, but in all of them is a person who supplies the correspondence between the features in the source image and target image. In this paper we use a method to find the faces in the image and the Active Shape Models to find the features in the face images and perform the metamorphosis of face images in frontal view automatically.

Distributed data mining (DDM) is an important research area. The task of distributed data mining is to extract and integrate knowledge from different sources. Solving such tasks requires a special approach and tools, different from those applied to learning from data located in a single database. One of the approaches suitable for the DDM is to select relevant local patterns from the distributed databases. Such patterns often called prototypes, are subsequently merged to create a compact representation of the distributed data repositories. Next, the global classifier, called combiner, can be learned from such a compact representation. The paper proposes and reviews several strategies for constructing combiner classifiers to be used in solving the DDM tasks. Suggested strategies are evaluated experimentally. The evaluation process is based on several well-known benchmark data sets.

Modern process and condition monitoring systems produce a huge amount of data which is hard to analyze manually. Previous analyzing techniques disregard time information and concentrate only for the indentification of normal and abnormal operational states. We present a new method for visualizing operational states and overall order of the transitions between them. This method is implemented to a visualization tool which helps the user to see the overall development of operational states allowing to find causes for abnormal behaviour. In the end visualization tool is tested in practice with real time series data collected from gear unit.