Artificial Neural Nets and Genetic Algorithms

Radial basis functions network (RBFN) is considered as a knowledge model. The model is established from a data set by learning. For the performance assessment a novel model validation method is introduced. The method consists of sensitivity analysis integrated into a mathematical-based technique known as analytical hierarchy process (AHP). It ranks the relative importance of factors being compared where the factors are the sensitivities in this case. The relative importance of the sensitivities is computed from the model and based on this information, the consistency of this information is tested by AHP. The degree of consistency is a measure of confidence for the validity of the model.

The recurrent neural networks of generalized architecture (GARNN) are general continuous dynamic systems. It was shown elsewhere that they can successfully manage the problem of on-line inference of finite automata. In addition, they can successfully solve problems of a continuous nature because they are continuous systems. A frequently used problem domain to check this are the well-known trajectory tracking problems. Some new problems of this problem domain are defined in this paper. The experiments are carried out with the generalized recurrent neural networks and solutions are found for each trajectory of the problem domain.

In this paper, an architecture of a fuzzy probabilistic neural network is considered. A learning algorithm for the activation function parameters is proposed. The advantages of this network lie in the possibility of classification of the data with substantially overlapping clusters, and tuning of the activation function parameters improves the accuracy of classification. Simulation results confirm the efficiency of the proposed approach in the data classification problems.

We adapt a boosting algorithm to the problem of predicting future values of time series, using recurrent neural networks as base learners. The experiments we performed show that boosting actually provides improved results and that the weighted median is better for combining the learners than the weighted mean.

We derive upper bounds on rates of convergence of neural network approximation in Lp-spaces. Our bounds are based on a version of Maurey-Jones-Barron Theorem for Lp-spaces. They are established in terms of L₁-norm of a weight function in a neural network with continuum of hidden units representing the function to be approximated.

For tapped delay neural architecture impact of discretisation process i.e. sampling and amplitude measurement of the input continuous signals is analysed. By using relative matrix rank and indistinguishable signals concepts we derrive upper and lower bounds of sampling time and input signal frequencies.

An attempt to facilitate learning of recurrent neural networks is made. The proposed extended recurrent neural network adaptively latches unimportant input vectors and thus effectively reduces the size of the training set. A register of latches is added as the input layer of the network. The latch is implemented with a multiplexer 2/1 whose output is differentiable with respect to all of its inputs, thus enabling the derivatives to be propagated through the network. The relevance of input vectors is learned together with the weights of the network using a gradient-based algorithm.

A denoising unit based on wavelet multiresolution analysis is added ahead of the multilayered perceptron with global recurrent connections. The learning algorithm is developed which uses the same cost function for setting all free parameters, those of the denoising unit and those of the neural network. It is illustrated that the proposed model outmatches the models without denoising unit and/or without recurrent connections in noisy time series prediction.

A reinforced search algorithm for the stochastic feedforward neural networks is described. A stochastic neuron is used in a network as a searching unit. Reinforcement signal from environment is used for weights and variance adaptation. This is experimentally compared with more traditional techniques like gradient-based learning algorithm and evolutionary algorithm.

In modeling with neural networks, the choice of network architecture and size is of utmost importance. The use of a too small network may result in poor performance because of lack of expressional capacity, while a too large network fits noise or apparent relations in the data sets studied. The work required to find a parsimonious network is often considerable with respect to both time and computational effort. This paper presents a method for training feedforward neural networks based on a genetic algorithm (GA), which simultaneously optimizes both weights and network connectivity structure. The proposed method has been found to yield dense and descriptive networks even from training sets of few observations.

Learning from data is studied in the framework of approximate minimization of functionals. A set of admissible solutions over which such functionals are minimized is approximated by a nested family of sets of functions computable by neural networks with n hidden units. There are derived upper bounds on the speed of convergence of infima achievable over such approximations of admissible sets to a global infimum. The bounds are expressed in terms of a certain norm tailored to the type of network units and modulus of continuity of the functional to be minimized. The results are applied to empirical error functionals regularized using stabilizers that are defined as squares of norms in reproducing kernel Hilbert spaces.

Data compression of high dimentional complex patterns is one of the most challaging task of information technology today. Proposed approach is based on feature extraction procedure which maps original patterns into features (factors) space of reduced, possibily very small, dimension. In this paper, we outline that Hebbian unsupervised learning of Hopfield-like neural network is a natural procedure for factor extraction. Due to this learning, factors become the attractors of network dynamics, hence they can be revealed by the random search. The neurodynamics is modeled by Single-Step approximation which is known [5] to be rather accurate for sparsely encoded Hopfield-network. This paper is limited to the case of sparsely encoded factors, but it is not realy constraint for data compression.

Support Vector Machine is a statistical learning machine developed by Vapnik in his statistical learning theory. This machine present very interesting proprieties in both classification and regression problems in the high dimensional space. In this paper, we propose β_SVM, a new kernel function for SVM, with special proprieties and high discrimination ability. We have applied this kernel in the pattern recognition, and we have compare the different performances of many other kernels, results show that the new kernel is very performant.

The construction of a good predicting model by learning algorithms does not necessarily imply a correct answer during the generalisation step. That is why one gives confidence intervals on the predicted value, often needing some hypothesis on the data’s density distribution. These hypothesis can hardly be verified when a little number of samples is given, which is the most frequent case in practice. We follow a local approach on the basis of an optimal neighbourhood choice. We use this neighbourhood to predict as well as to give some simple model quality indicators for any sample.

Neural networks are potential tools that can be used to improve process quality control. In fact, various neural algorithms have been applied successfully for detecting groups of well-defined unnatural patterns in the output measurements of manufacturing processes. This paper discusses the use of a neural network as a means for recognising changes in the state of the monitored process, rather than for identifying a restricted set of unnatural patterns on the output data. In particular, a control algorithm, which is based on the Fuzzy ART neural network, is first presented, and then studied in a specific reference case by means of Monte Carlo simulation. Comparisons between the performances of the proposed neural approach, and those of the CUSUM control chart, are also presented in the paper. The results indicate that the proposed neural network is a practical alternative to the existing control schemes.

Urban air pollutants have emerged as a severe problem which causes health effects and even premature deaths among sensitive groups. Therefore a warning system for air pollution episodes is widely needed to minimize negative health effects. However the forecasting of air pollution episodes has been observed to be problematic partly due their rareness and short-term nature. The research presented here aims to evaluate different neural network based models for forecasting urban air pollution (N02) hourly time series and particularly the episode peaks. The performances of three multi-layer perceptron (MLP) models namely basic MLP and two hybrid models were compared by calculating several statistical indicators. In the hybrid models evaluated here, training data set was clustered to several air quality episodes using the k-means (KM) and fuzzy c-means (FCM) algorithms and then several MLP models were applied to the clustered data, each one representing one cluster. The results showed that the hybrid models have some advantages over a basic MLP model in the forecasting air quality episodes, but the performance achieved also show that architectural issues cannot solely solve the model performance problems.

This contribution describes the influence of the Czech language parameters selection on the coarticulation of the phonemes for the modelling of prosody features by the artificial neural network (ANN) in a text-to-speech (TTS) synthesis. The GUHA method and neural network pruning can be used for this reason. In our work we analyzed the errors between the target and calculated values of F₀ and D from the point of view of the different context of speech units. The context of three phonemes combinations CCC, VVC, VCV, CVV, VCC, CCV, and CVC (C = consonant, V = vowel) were analyzed for the determination of a next improvement of prosody. The qualitative criteria have been found in this contribution.

In this paper we look at the role that vertical fields can play in enhancing the performance of a feedforward neural network. Vertical fields help us to determine zones in the input space that are mapped onto the same output, they act in a similar way to kernels of linear mappings but in a nonlinear setting. In the paper we illustrate our ideas using data from a real application, namely forecasting atmospheric pollution for the town of Saint-Etienne in France.

In developing data-driven models of complex real-world systems, a common problem is how to select relevant inputs from a large set of measurements. If the observations of the outputs to be predicted by the model are scarce, which may be the case if the outputs are indices determined in toilsome laboratory tests, strict constraints may be imposed on the number of model parameters. In neural network modeling, such a limitation in practice also restricts the number of input variables, since the dimension of the weight vector strongly depends on it. This paper presents a systematic method for the selection of input variables for feedforward layered neural networks. The method is illustrated on a problem from ironmaking industry, where sinter quality indices are predicted on the basis of raw material properties. Furthermore, an inversion technique of the resulting network models is proposed, where an optimization problem is solved to maximize the performance of the sintering operation by manipulating the inputs.

Stimulated by the growing demand for improving system performance and reliability, fault-tolerant system design has been receiving significant attention. This paper proposes a new framework for fault-tolerant and quality control design based on the learning capabilities of neural networks. In highly nonlinear systems, with slow process variation, and a high degree of reactivity, quick changes (in the environment, for example) cause fast responses. This class of systems is broad enough so that it is not only of theoretical interest but also of practical applicability. Moreover, the fault-tolerance ability of the adaptive controller will be further improved by exploiting information estimated from a fault-detection and diagnosis unit designed by a pattern-matching strategy in multiple faults models interfaced with the overall system. Results of the approach are presented in a practical case of a plastic injection moulding process in industry.

In this work several approaches to prediction of natural gas consumption with neural and fuzzy neural systems are analyzed and tested. The data covers daily natural gas load in a certain region of Poland. Prediction strategies tested in the paper include: single neural net module approach, combination of three neural modules, temperature clusterization based method, and application of fuzzy neural networks. The results indicate the superiority of temperature clusterization based method over modular and fuzzy neural approaches. One of the interesting issues observed in the paper is relatively good performance of tested methods in the case of a long-term (four week) prediction compared to mid-term (one week) prediction. Generally, the .results are significantly better than those obtained by statistical methods currently used for this task in the gas company under consideration.

This paper presents a relatively new identification method based on Artificial Neural Networks, which can be used for multi-input multi-output systems. In particular, a Group Method of Data Handling neural network with dynamic neurons is considered. The final part of this work contains an illustrative example regarding the application of the proposed approach to a fault detection system.

The paper describes aspects of an on-going research programme in collaboration with AvestaPolarit (UK) Ltd, Sheffield, Corus Group plc, Rotherham and Eurotherm, Worthing. The work presented here involves the design and development of a neural network control strategy for continuously operated high temperature gas-fired industrial furnaces. At present, a bilinear control strategy is already applied routinely to the furnace. The results in this paper present the development of different neural network controllers compared to the bilinear controller.

Automatic text categorisation of documents has received a resounding interest in last years due to the increased availability of documents in digital form and the commanding need to organize them. In this paper, our main focus is the development of tools that will enable very fast and accurate text classifiers in large scale databases. To pursue this objective, we start by introducing the main issues of text categorisation and present possible ways of handling them. Kernel based methods, such as, Support Vector Machines (SVMs), are learning methods with strong potential for solving the tasks involved in automatic text categorisation. The first results achieved with Reuters-21578 collection are reported and some points of possible improvements are identified.

The lack of meteorological measurements at a location of interest (target location) constitutes a problem that is crucial for the purposes of both weather forecasting and energy system design/validation. This paper constitutes a pilot study for the accurate estimation of meteorological values at a target location employing the meteorological measurements collected at a nearby location (reference location). Artificial neural networks (ANNs) are investigated and compared with traditional estimation methods. The significance of the improvement obtained via the ANN approach both over the traditional estimation methods as well as over simply considering the measurements at the reference location is demonstrated in a number of energy applications.

A Jaguar Car engine is used to provide data for modelling the throttle body, engine pumping and manifold body. Based on the gas law of the intake dynamics, input/output variables are identified and used to train a neural network. Various structures are compared and assessed. The best structure is then used for fault detection. A neural network observer is developed and error stability is assessed. Two fault scenarios are considered.

This paper presents RBFG neural networks trained by ROM and applied in prediction. Two techniques of Gaussian centers disposition are presented: regular trellis in a one dimension and an original method combining the regular trellis in a two dimensions and the K-mean clustering method. The second technique presents the best prediction.

A data mining application is described, which is focused on the analysis of high-dimensional meteorological data collected long-term (over 43 years) at 130 weather stations in Greece. A hybrid clustering method (combining artificial neural networks and statistical-based techniques) has been employed for grouping the data. The proposed method has been found effective in clustering the stations and partitioning Greece into areas with distinct meteorological profiles: stations in the same area have similar meteorological profiles, while those classified in different areas have distinct meteorological profiles. Additionally, the most salient parameters per area have been determined, whereby the parameters characterizing the different meteorological profiles of Greece have been uncovered.

Recently several authors have turned their attention to the design problems of very large arrays of intelligent sensors, such as would be suitable for inclusion as an integral part of a’ smart structure’. Such systems have architectural and data throughput advantages, but to be used effectively need to be designed so as to be ‘controllerless’ and ‘decentralised’. This paper explores some of these design issues, and illustrates how artificial intelligence (in particular Neural Networks techniques) could be effectively incorporated within the design of large intelligent microsystems with the scope of enhancing their functionality. Micromachined acceleration sensors are considered as a case study.The paper opens with a discussion on the ever-increasing appropriateness of local sensor health monitoring, fault diagnosis and measurement confidence indices. The use of Artificial Intelligence techniques is suggested for implementing on-chip sensor diagnosis. Design issues for large, intelligent arrays of microsensors are brought forward.

In this paper we have developed a prototype of lecture support system using PDA’s, This lecture support system concern the fourth students. The extraction of parameters is based on the visual encoding gives 72% as a extraction rate. The originality of the on-line recognition method is in the vector of input of the neuronal network system. We developed database contains 50 000 Arabic words that 70% are used for learning the neural network system and 30% are used for testing the recognition system. The recognition rate obtained is 90%.

The standard Genetic Algorithm has several limitations when dealing with dynamic environments. The most harmful limitation as to do with the tendency for the large majority of the members of a population to convergence prematurely to a particular region of the search space, making thus difficult for the GA to find other solutions when changes in the environment occur. Several approaches have been tested to overcome this limitation by introducing diversity in the population or through the incorporation of memory in order to help the algorithm when situations of the past can be observed in future situations. In this paper, we propose a GA inspired in the immune system ideas in order to deal with dynamic environments. This algorithm combines the two aspects mentioned above: diversity and memory and we will show that our algorithm is also more adaptable and accurate than the other algorithms proposed in the literature.

Transformation is a biologically inspired genetic operator that, when incorporated in the standard Genetic Algorithm can promote diversity in the population. Previous work using this genetic operator in the domain of function optimization and combinatorial optimization showed that the premature convergence of the population is avoided. Furthermore, the solutions obtained were, in general, superior to the solutions achieved by the GA with standard 1-point, 2-point and uniform crossover. In this paper we present an extensive empirical study carried to determine the best parameter setting to use with transformation in order to enhance the GA’s performance. These parameters include the gene segment length, the replacement rate (percentage of individuals of the previous population used to update the gene segment pool), and the mutation and transformation rates.

This paper describes a the application of ant colony optimization algorithms, which draw inspiration from the way ants organize themselves in searching for food, to the well-known bioinformatics problem of aligning several protein sequences.

We present here the Radial Basis Function Gene Model as a new approach of evolutionary computation. This model enables us to relax the “locus constraint” that limits classical genetic algorithms. Both the principles and first results are presented, showing the great interest of this model.

In this paper we propose to exploit the spiking neuron model to control an animat. Our basic assumption is that those neurons are well designed to exhibit both learning abilities and the modular structure of reactive control. Evolving the net of neurons is therefore a mean to study the complex interactions of spikes to produce a simple behaviour of predation and homing in simulation.

Biological Crossover occurs during the early stages of meiosis. During this process the chromosomes undergoing crossover are synapsed together at a number of homogenous sequence sections, it is within such synapsed sections that crossover occurs. The SVLC (Synapsing Variable Length Crossover) Algorithm recurrently synapses homogenous genetic sequences together in order of length. The genomes are considered to be flexible with crossover only being permitted within the synapsed sections. Consequently, common sequences are automatically preserved with only the genetic differences being exchanged, independent of the length of such differences. In addition to providing a rationale for variable length crossover it also provides a genotypic similarity metric for variable length genomes enabling standard niche formation techniques to be utilised. In a simple variable length test problem the SVLC algorithm outperforms current variable length crossover techniques.

One of the approaches used in Evolutionary Algorithms (EAs) for problems in which the environment changes from time to time is to use techniques that preserve the diversity in population. We have tested and compared several algorithms that try to keep the population as diverse as possible. One of those approaches applies a new biologically inspired genetic operator called transformation, previously used with success in static optimization problems. We tested two EAs using transformation and two other classical approaches: random immigrants and hypermutation. The comparative study was made using the dynamic 0/1 Knapsack optimization problem. Depending on the characteristics of the dynamic changes, the best results were obtained with transformation or with hypermutation.

A general concept of representation of connected groups of agents (schemes) within a multi-agent system is introduced and utilized for automatic building of schemes to solve a given computational task. We propose a combination of an evolutionary algorithm and a formal logic resolution system which is able to propose and verify new schemes. The approach is illustrated on simple examples.

We propose an evolutionary method for the design of Beta basis function neural networks (BBFNN). Classical training algorithms start with a predetermined network structure for neural networks. Generally speaking the neural network is either insufficient or over-complicated. This paper describes a hierarchical genetic learning model of the BBFNN. In order to examine the performance of the proposed algorithm, it is used for functional approximation problem. The results obtained have been encouraging.

The paper proposes applying the population-learning algorithm to solving a single mode resource constrained project scheduling problem with makespan minimization as an objective function. The paper contains problem formulation and a description of the proposed implementation of the population learning algorithm (PLA). To validate the approach a computational experiment has been carried. It has involved 1440 instances from the available benchmark data set. Experiment results show that the proposed PLA implementation is an effective tool for solving single mode resource constrained project scheduling problems. In a single run the algorithm has produced solutions with mean relative error value well below 1% as compared with available upper bounds for benchmark problems.

Contrary to Genetics Snakes, the current methods of mouth modeling are very sensitive to initialization (position of a snake or a deformable contour before convergence) and fall easily into local minima. We propose in this article to make converge two snakes in parallel via a genetic algorithm. The coding of the chromosome takes into account at the same time gradients and region type information contained in the image. In addition we introduce the use of STM (Sparse Template Matching) into the field of leapreading. Thanks to a temporal filter, word signatures (stored in Sparse Templates) make it possible to recognize various words pronounced several times at one week interval.

The container transhipment problem involves scheduling a fleet of lorries to pick up or drop off containers of various sizes, while minimising the total distance travelled by the lorries. The problem originates in the need for logistics companies to solve this problem on a regular basis as part of their daily operations. In this paper we investigate the use of a genetic algorithm for the solution of the problem. Results are discussed and a general analysis and comparison of the validity and robustness of two approaches are given.

Routing of wires and piping in airframe design is an arduous process. Current processes attempt to wire bundles of wires individually, resulting in a lengthy iterative process as attempts are made to satisfy the many constraints for each type of wire or bundle. This paper describes a new approach to this problem, in which pre defined routeways are established at an early stage of the design, so as to satisfy the constraints of the wiring and piping to be placed within. The work described here provides for automatic optimization of these routeways using genetic algorithms. Solutions produced this way are compared with those produced by a maze solver, and found to offer better use of the available space. Variations of the detail design of the genetic algorithm used have been prototyped and evaluated.

We introduce a universal relational scheme of parameterized sets of objects of arbitrary types, taking also variables and topological structures into account, to represent precise and imprecise (“vague”) knowledge. By operations applied to the scheme further knowledge can be deduced and by functional “queries” facts can be interrogated. As illustrative examples applications to databases, pattern recognition and logical deduction are considered.

In this paper we address problems arising from the use of categorical valued data in rule induction. By naively using categorical values in rule induction, we risk reducing the chances of finding a good rule in terms both of confidence (accuracy) and of support or coverage. In this paper we introduce a technique called arcsin transformation where categorical valued data is replaced with numeric values. Our results show that on relatively large databases, containing many unordered categorical attributes, larger databases incorporating both unordered and numeric data, and especially those databases that are small containing rare cases, this technique is highly effective when dealing with categorical valued data.

A simple model for social group interactions is introduced. The model is investigated for the minimal group size for the model N=3. Examples of model of model behaviour in terms of dynamic and static equilibrium are presented. Directions for further study of the model are considered.

Throughout the world, there is a trend towards the growth of vehicle traffic and an increase in road transport. The number of cars is growing twice to three times faster than the rate of growth of the population and car use is increasing even more rapidly.Growing congestion has led to ever greater environmental nuisance and pollution is causing lethal consequences for humans.Reducing emissions from motor vehicles must involve a global strategy, bringing into play both technical innovation and strict regulations.In this context Hybrid electric vehicles power train can offer a sensible improvement of the overall vehicle environmental impact, achieving at the same time a rational energy employment. The environmental challenges of this century require a new generation vehicles. Making the most of the modern control systems and design techniques, they will be more electrified and designed by total systems approaches, involving new materials, alternative fuels, and, above all, new powertrains.Advanced energy flow management unit to split the instantaneous vehicle power demand between the internal combustion engine and the electric motor, will ensure that the power sources will be operated at high efficiency operating points and the related vehicle emissions will be minimized.This led study to develop innovative methodologies for the real-time identification of the components operating points.Starting from these considerations in the paper a knowledge-based control system for splitting the vehicle’s power demand between the engine and motor is presented. The proposed approach by using fuzzy clustering and genetic algorithm permits to achieve better performance, mainly in terms of a reduced computational effort and an improved efficiency of the control system over various driving cycles.To validate this approach, simulation tests and comparisons with other energy management strategies are discussed.