Artificial Neural Nets and Genetic Algorithms

This paper presents a method for obstacle recognition to be used by a mobile robot. Data are made of range measurements issued from a phased array ultrasonic sensor, characterized by a narrow beam width and an electronically controlled scan. Different methods are proposed: a simulation study using a neural network, and a signal analysis using an image representation. Finally, a solution combining both approaches has been validated.

The paper presents a way of extending complementary reinforcement backpropagation learning (CRBP) to modular architectures using a new version of the gating network approach in the context of reactive navigation tasks for a simulated mobile robot. The gating network has partially recurrent connections to enable the co-ordination of reinforcement learning across both modules successive time steps. The experiments reported explore the possibility that architectures based on this approach can support concurrent acquisition of different reactive navigation related competences while the robot pursues light-seeking goals.

Hybrids of genetic algorithms and artificial neural networks can be used successfully in many robotics applications. The approach to this is known as evolutionaryrobotics. Evolutionary robotics is advantageous because it gives a semi-automatic procedure to the development of a task-fulfilling control system for real robots. It is disadvantageous to some extent because of its great time consumption. Here, I will show how the time consumption can be reduced dramatically by using a simulator before transferring the evolved neural network control systems to the real robot. Secondly, the time consumption is reduced by realizing what are the sufficient neural network controllers for specific tasks. It is shown in an evolutionary robotics experiment with the Khepera robot, that a simple 2 layer feedforward neural network is sufficient to solve a robotics task that seemingly would demand encoding of time, for example in the form of recurrent connections or time input. The evolved neural network controllers are sufficient for exploration and homing behaviour with a very exact timing, even though the robot (controller) has no knowledge about time itself.

In practice, general-purpose learning algorithms are not sufficient by themselves to allow robots to acquire complex skills — domain knowledge from a human designer is needed to bias the learning in order to achieve success. In this paper we argue that there are good ways and bad ways of supplying this bias and we present a novel evolutionary architecture that supports our particular approach. Results from preliminary experiments are presented in which we attempt to evolve a simple tracking behaviour in simulation.

We examine here the feasibility of using evolutionary techniques to produce controllers for a standard robot arm. The main advantage of our technique of solving path planning problems is that the neural network (once trained) can be used for the same robot, with a variety of start and target positions. The genetic algorithm learns, and encodes implicitly, the calibration parameters of both the robot and the overhead camera, as well as the inverse kinematics of the robot. The results show that the evolved neural network controllers are reusable and allow multiple start and target positions.

A method based on genetic algorithms for obtaining coordinated motion plans of manipulator robots is presented. A decoupled planning approach has been used; that is, the problem has been decomposed into two subproblems: path planning and trajectory planning. This paper focuses on the second problem. The generated plans minimize the total motion time of the robots along their paths. The optimization problem is solved by evolutionary algorithms using a variable-length individuals codification and specific genetic operators.

Ensembles of neural networks is a technique that uses several different networks working together to find a relationship that exactly matches a many-to-one training set for digital classification problems. Each network learns a different region of the training space and all these regions fit together, like pieces of a jigsaw puzzle, to cover the entire training space. The individual networks are ‘grown’ as they are needed to form either cascades or branches of networks. The networks can be of any type such as backpropagation, cascade etc. However, virtually any other technique can be used in place of the networks: GA, EA, DRS, tabu search, nearest neighbour, and so on. Methods are discussed to improve the generalisation.

This paper proposes a two layer modular neural system. The basic building blocks of the architecture are multilayer perceptrons trained with the backpropagation algorithm. Due to the proposed modular architecture the number of weight connections is less than in a fully connected multilayer perceptron. The modular network is designed to combine two different approaches of generalization known from connectionist and logical neural networks; this enhances the generalization abilities of the network. The architecture introduced here is especially useful in solving problems with a large number of input attributes.

One of the major concerns when implementing a supervised artificial neural network solution to a classification or prediction problem, is the network’s performance on unseen data. The phenomenon of the network overfitting the training data, is understood and reported in the literature. Most researchers recommend a ‘trial and error’ approach to selecting the optimal number of weights for the network, which is time consuming, or start with a large network and prune to an optimal size. Current pruning techniques based on approximations of the Hessian matrix of the error surface are computationally intensive and prone to severe approximation errors if a suitable minimal training error has not been achieved. We propose a novel and simple design heuristic for a three layer multi-layer perceptron (MLP) based on an eigenvalue decomposition of the covariance matrix of the middle layer output. This technique identifies the neurons which are contributing to the redundancy of data through the network and as such are additional effective network parameters which have a deleterious effect on the classifier surface smoothness. This technique identifies redundancy in the network data and so is not dependant on the network training having reached a minimal error value making the Levenberg-Marquardt approximation valid. We report on simulations using the double-convex benchmark which show the utility of the proposed method.

Artificial neural networks have been applied to solve hard problems in different engineering domains, thanks to their capability of universal function approximators [4]. However, when these networks are used in their standard forms, ‘black-box models’, their performances are inferior to dedicated statistical solutions. Performances can be largely improved if we introduce prior knowledge in network architectures. If the real problem has an obvious decomposition, then it may be possible to design a network architecture by hand. Unfortunately, this is not always possible.

This paper presents an evolution algorithm for optimizing a neural network architecture. The procedure establishes the structure and the training algorithm, as well as searching the minimal topology of the network, eliminating neurons and interconnection weights. The model network, this is, feedforward or feedback, can be selected by the user. This methodology is applied to the real problem of the forecasting in power system load in the city of Málaga (Spain) between the years 1992 and 1993. The results produced by the evolution algorithm are tested with a statistical regression analysis and with other training algorithms of paradigms of neural networks.

The design of pressurised water reactor (PWR) reload cores is not only a formidable optimization problem but also in many instances a multiobjective problem. This paper describes a genetic algorithm (GA) designed to perform true multiobjective optimization on such problems.

This paper describes an investigation into using artificial neural networks (ANNs) to model the non-linearities in a complex system, a nuclear reactor. A simple one compartment finite difference model of the plant is developed and an exact ANN equivalent formed directly without training. Conventional training using standard transfer functions available in ANN packages is compared with this. A novel method is used to produce the ANN training and test sets. A twenty five compartment model is built from directly formed ANNs and the results compared to a simulator model.

The use of interactive activation and competition (IAC) and backpropagation (BP) artificial neural networks (ANNs) for transit time estimation has been investigated in this piece of research. Owing to its competitive nature, the IAC ANN has been found able to correctly estimate the current transit time from short records of signals as well as to quickly follow changes in transit time and to detect when the transit time falls outside a predefined expected range. On the other hand, the interactive nature of the IAC ANN allows it to be robust to significant levels of noise and of the global component. A BP ANN has been appended to the IAC ANN, further allowing for the accurate estimation of decimated transit times.

We have previously shown that non-uniform cellular automata (CA) can be evolved to perform computational tasks, using the cellular programming algorithm. In this paper we focus on two novel issues, namely, the evolution of asynchronous CAs, and the scalability of evolved synchronous systems. We find that asynchrony presents a more difficult case for evolution though good CAs can still be attained. We describe an empirically derived scaling procedure by which successful CAs of any size may be obtained from a particular evolved system. Our motivation for this study stems in part from our desire to attain realistic systems that axe more amenable to implementation as “evolving ware,” evolware.

This paper aims at stimulating discussion and research into a new concept called ‘one-chip evolvable hardware (1C-EHW)’ by proposing a generic architecture and methodology to generate the ultimate in evolvable hardware speed, where all the evolvable hardware components are placed on a single chip.

A new approach for the application of genetic programming to vision problems is presented. Sets of atomic subprograms are genetically combined to solve more advanced problems within low-level vision or image preprocessing. We present the main ideas and give a brief sketch of their implementation in the distributed simulation environment GENCODER. This system forms the basis for some introductory experiments obtained. Finally, some aspects of the gained results together with interesting possibilities for future research are portrayed.

The non-linear receptive field (NLRF) neural network consists of a homogeneous, uniformly distributed series of locally connected non-linear receptive fields. Each receptive field exploits a set of local connections, with weights which axe symmetrical around the center of the receptive field. The nonlinear behaviour is the result of three properties of the network. First, the activation is accumulated in the output layer units. Second, the recurrent feedback of activation from the output layer back onto itself. Third, the overlap of receptive fields. The nonlinear nature of the network allows it to perform relatively complex tasks, in spite of its simple architecture. The non-linear receptive field learning algorithm (NLRFLA) provides a way of finding the optimal set of connection weights for a given problem. The NLRFLA learning algorithm is essentially a recurrent backpropagation learning algorithm, with some special conditions.

This paper details work on the development of an adaptive active vision system for an automated guided vehicle. An initial solution to the task of providing intelligent control of the saccades with which the AGV examines its environment is presented. A simple fuzzy logic technique suitable for implementation on a microcontroller is developed by using stochastic transition matrices. The results presented show the success of the technique in maintaining interest in objects previously located within the environment, locating new objects in an environment and making a compromise between the two.

In this paper we propose a hardware implementation of a binary neural network architecture obtained from a new efficient constructive algorithm. This algorithm is particularly interesting because it can treat boolean as well as real valued classification problems with an arbitrary number of outputs. The networks obtained consist of binary neurons organized in two hidden layers. The first layer is implemented on a systolic architecture which represents a good tradeoff between speed and area. Due to the particular computation performed by the second hidden layer, its implementation is straightforward and well-suited to the systolic architecture. A limited number of logical gates is needed for its implementation. The output neurons are also easy to implement but require a small size memory.

The Meddis computational model of auditory nerve fibre activity [9, 10] is widely used as a research tool in the study of auditory processing. The model is governed by a set of control parameters that allows it to simulate responses derived from physiological observations. In this paper, we describe a novel method for automatically determining the parameters required to simulate a range of rate-intensity responses from auditory nerve fibres using this model. A genetic algorithm [4] is employed to explore possible parameter combinations and to determine a ‘best fit’ solution. Two sets of experiments used to demonstrate the flexibility of the technique are described. Some possible wider applications of the technique are discussed.

We use a simple network which uses negative feedback of activation and simple Hebbian learning to self-organise in such a way as to produce a feature map which has the property of identifying the relative proportions of the components of the input data. Thus it evaluates the angular properties of the input data space and ignores the magnitude of the input data. When used on unprocessed voice data, the network is shown to extract both the phase and the frequency information from the raw data. We show how to use this network for classification of vowels.

This paper is devoted to a neurobiologically plausible approach for the design of speech processing systems. The temporal organization map (TOM) neural net model is a connectionist model for time representation. The definition of a generic neural unit, inspired by the neurobiological model of the cortical column, allows the model to be used for problems including the temporal dimension. In the framework of automatic speech recognition, TOM has been previously tested with conventional techniques of signal processing. An auditory model as front-end processor is now used with TOM, in order to test the efficiency and the accuracy of a physiologically based speech recognition system. Preliminary results axe presented for speaker-dependent and speaker-independent speech recognition experiments. The interest of auditory model is the possibility to develop more valuable processing and communication strategies between TOM and the front-end processor, including afferent and efferent information flow.

It has been shown experimentally that Oja’s nonlinear principal component analysis (PCA) algorithm is capable of performing an independent component analysis (ICA) on a specific data set [7]. However, the dynamic stability requirements of the nonlinear PCA algorithm restrict its use to data which has sub-gaussian probability densities [6]. The restriction is particularly severe as this precludes the application of the algorithm from performing ICA on naturally occurring data such as speech, music and certain visual images. We have shown that the nonlinear PCA algorithm can be considered as minimising an information theoretic contrast function and develop a more direct link between ICA and the algorithm function [6]. To remove the sub-gaussian restriction and enable a generalised ICA which will span the full range of possible data kurtosis, we propose the use of Fahlman type activation functions [2] in the nonlinear PCA algorithm. We show that variants of these functions satisfy all the dynamic and asymptotic stability requirements of the algorithm and successfully remove the sub-gaussian restriction. We also report on simulations which demonstrate the blind separating ability of the nonlinear PCA algorithm with the Fahlman type functions on mixtures of super-Gaussian data (natural speech).

In this paper, a two-layer neural network is presented that organizes itself to perform blind source separation, i.e. it extracts the unknown independent source signals out of their linear mixtures. The convergence behaviour of the network is analyzed, and experimental results of separating historical speeches of four different speakers are presented.

This paper deals with neural network approaches for higher order spectral estimation. The emphasis is put on how to use analog neural networks to perform in realtime major computations required in the ARMA model based bispectral estimation and the fourth order cumulant based Pisarenko’s harmonic method. The proposed approaches are useful for the real-time signal processing with higher order spectral estimation.

The 1/f family of fractal signals constitutes an important class in signal processing. In many applications a precise estimation of the fractal parameter is needed, which is not easy in many environments. This paper is a study on estimating the fractal parameter, i.e. Hurst index, under the wavelet domain and using genetic algorithms. The results show that this method is robust and converges quickly. Due to the intrinsically parallel nature of genetic algorithms, this method is especially useful in real time applications, e.g. estimation of parameter in self-similar traffic in ATM networks.

This work in progress shows a method for classifying dendritic spines by their shape. Focal points are the extraction of features from three-dimensional spine data and the following classification of the spines. Hence there will be only little reflection of biological aspects of this problem. Feature extraction based on moments and spherical coordinates will be discussed. Furthermore, this paper shows and describes a modified kind of self organizing maps (SOM)), which is used for the classification of the dendritic spines.

This paper describes a straightforward method of analysing the colour spectra of natural images using a hue histogramming technique. Examples of post-processed first moment data from these histograms axe then analysed using simple feedforward neural networks. These networks are shown to provide a good level of generalisation and can therefore be used for classification.

Synthetic aperture radar (SAR) images are intrinsically noisy, and processing them attracts a high computational overhead. This paper relates to developments, involving the use of an ANN to reduce the overhead, in respect of earlier work by the authors on the identification of small objects. It describes how the ANN is utilised, and how it was trained using an artificially created training set.

This article studies the implementation of a handwritten character recognition task using neural networks. Two logic neural network models axe employed to classify the Essex dataset, which comprises real-world hand-written characters. To reduce the underlying dataset variation, several pre-processing approaches are investigated. This allows the comparison of the network models on the basis of their classification accuracy for datasets with different characteristics.

We consider an epidemiological dataset, concerned with predicting pulmonary response to air pollution. To gain more knowledge of nonlinear effects and interactions in the data, nonlinear neural network techniques were applied to model the data. Initially, we modelled the data with standard feedforward network models. Based on the epidemiologic effect of heterogeneity in response, we propose a novel combined neural network modelling strategy to improve prediction quality. Also, we propose the use of a neural network strategy for reducing correlation between covariants to improve modelling quality. The results presented are promising when compared to standard feedforward network modelling.

This paper deals with a new methodology for the development of an expert system (ES) using a hybrid architecture. This architecture simplifies the knowledge acquisition phase, by providing some sort of learning corresponding to the training phase of the neural network. So, it is possible to start with the nucleus of a knowledge base and the system will improve during the learning phase using examples.

This work presents a study of learning (case-based) in an interactive activation and competition (IAC) connectionist model. In this type of neural network, the basic learning mode may be classified as rote learning, and no iterative algorithm is used. The knowledge elicitation corresponds directly to the connection weights and its values are obtained by a type of engineering called connectionengineering. In a way it is similar to the knowledge engineering in that it obtains functioning rules for an expert system. In this sense, an example of differential diagnosis in rheumatology is used to study the learning performance of a neural network with the introduction of real clinical cases, presented by a expert doctor. These clinical cases are used as a source of additional knowledge that represent relations between diseases and symptoms.

Most of the genetic algorithms (GAs) used in practice work on linear chromosomes (e.g. binary strings or sequences of some other types of symbols). However some results have been published revealing that for certain problems multidimensional encoding and crossover may give better results than the one dimensional (linear) ones [1, 2, 3]. While some theoretical results have been obtained, no clear criteria are known for deciding the suitable dimensionality of the encoding to be used for a given problem.In this paper we consider a class of problems for which we define a multidimensional encoding and a corresponding genetic operator. We show that for a genetic algorithm (GA) using this encoding and operator we can obtain theoretical results similar to (under certain conditions even better than) those known for linear encoding. We demonstrate these theoretical results using a set of test examples.

A non traditional highly disruptive crossover operator, called Möbius crossover, is introduced in the context of Excursion Set Mediated Genetic Algorithm (ESMGA). The new operator and the algorithm are applied to two GA deceptive problems and the results are reported.

In nature, sexually reproducing organisms do not mate indiscriminately — the choice of mate has an impact upon their offspring’s fitness. The investigation described here shows that, for a wide range of problems in the literature, using sexual selection proved to be a robust method for enhancing genetic algorithm performance. In addition, this investigation provides evidence for which parameters are important for a successful implementation.

The selection of operators and parameters for genetic algorithms (GA) depends upon the situation, and the choice is usually left to the users. Identifying the optimum selection is very time consuming and, therefore, it is important to develop a system which can assist the users in their selections. In our fuzzy Taguchi controller, we present a hybrid system, which combines the Taguchi method with fuzzy logic, to select near optimum settings for the design parameters. The Taguchi method selects an optimal orthogonal array from experimental design theory, to reduce the number of experiments required to study the parameter space. Our controller uses this array to determine the selection for fuzzy membership in the dynamic selection process. It then applies fuzzy logic to evaluate the beneficial genes which affect the GA performance. We use the hybrid procedure to produce evidence from simulations and this information is then used to refine the GA behaviour. The system utilises a fuzzy matrix to rearrange the sequence of gene groups within the chromosome and applies a fuzzy knowledge base to tune the GA parameter selection. This provides a simple and easy method to assist users to direct their search and optimisation in an efficient way.

Theorems axe given establishing the epistatic bounds for problems that can be stated as mathematical expressions. Examples of the application of the theorems and techniques for controlling epistasis are presented.

In this paper a multiple subpopulations technique for evolutionary algorithms is proposed. Each subpopulation is distinguished by the mutation radius and mutation probability assigned to it, with mutation radius being a function of mutation probability. Mutation probabilities across the subpopulations range from 0.005 to 0.75. There is no crossover between subpopulations in the normal course of breeding, and the mechanisms of elite migration from a higher mutation subpopulation to the adjacent subpopulation of lower mutation is used to introduce new genetic material. The evolution of artificial neural networks for solving a variety of problems is demonstrated, with convergence times typically half as long as a standard evolutionary algorithm.

This paper deals with an important class of optimization problems, the multiobjective problems. A new genetic algorithm, called the dual genetic algorithm, is presented. Through two theoretical problems, we show that this approach appears to be efficient for multiobjective optimization.

Recently an abstraction of genetic algorithms has been developed in which a population of GAs in any epoch is represented by a single vector whose elements are the the probabilities of the corresponding bit positions being equivalent to 1. The process of evolution is represented by learning the elements of the probability vector. We have previously extended this to model homeotic genes which are environmentally driven and turn other genes on and off. In this paper we incrementally develop the algorithm on a set of standard problems used to compare methods for the simultaneous optimisation of conflicting criteria within a single population.

A multi-population genetic algorithm is proposed which is hierarchical in nature. This allows the algorithm to solve problems which consist of smaller tasks contributing to the solution of an overall problem. The algorithm feeds the entire pool of individuals between local populations (solving the smaller problems) and a global population (solving the overall task). Results on categorisation tasks are presented.

In this paper we describe a redundant representation for use by genetic algorithms on continuous, parameter, optimisation problems and subject it to crossover. We examine the schemata induced by the representation and test its performance against a gray coded, binary representation and a real-coded representation acted on by the recombination operator BLX-0.5.

One large problem when employing a similarity function to measure the similarities between new and prior cases is to determine the weights of the features. This paper proposes a new method of learning weights using a genetic algorithm based on the similarity information of given examples. This method is suitable for both linear and nonlinear similarity functions. Our experimental results show the computational efficiency of the proposed approach.

A genetic algorithm for inferring stochastic context-free grammars from finite language samples is described. Solutions to the inference problem are found by optimizing the parameters of a covering grammar for a given language sample. We describe a number of experiments in learning grammars for a range of formal languages. The results of these experiments are encouraging and compare very favourably with other approaches to stochastic grammatical inference.

Successful product development depends on many factors. Among the most important factors are identification and satisfaction of customers’ perceived needs, the accessibility, size and growth rate of the target market, and, of course, production costs. Since the early 1970s marketing researchers achieved remarkable results in developing methods to measure consumer preferences of multiattributed products. Additionally, market segmentation methods have been an important issue in strategic marketing research. This study, however, concentrates on a new method of product design optimization. It is shown how genetic algorithms are used to simultaneously discover optimal multi-attributed products for different customer preferences. For that purpose we chose an interactive version of the genetic algorithm where genetic operators like selection, mutation and crossover are applied as usual. The use of the interactive genetic algorithm is most suitable, where measures of utility are difficult or impossible to specify mathematically. Imprecise optimization in terms of a priori unknown individual consumer decision rules and preferences is an important issue for marketing researchers. The interactive genetic algorithm tries to solve design problems in that the consumer plays the role of the objective function during data collection.

The aim of the paper is to propose a means by which neural network fitness evaluation can be applied to a genetic algorithm (GA), and an application of this system to musical rhythm composition. An adaptive resonance theory (ART) neural network is trained using binary information representing classification patterns. By comparing new genetically derived individuals to clustered data, a measure of fitness of the new patterns is determined; the patterns of higher fitness values then being used in successive generations to further improve the overall population fitness. A proposed application for this system is described — a genetic composer that utilises clustered representations of rhythm styles to interactively generate rhythm patterns to the user’s general stylistic requirements.

H. Asoh and H. Mühlenbein investigated empirically the relation among the mean convergence time, the population size, and the chromosome length of genetic algorithms (GAs). In this paper, from the mathematical point of view, the relation they revealed is convincing. Our analyses of GAs make use of the Markov chain formalism based on the Wright-Fisher model, which is a typical and well-known model in population genetics. We also give the mean convergence time under genetic drift. Genetic drift can be described by the Wright-Fisher model. We determine the stationary states of the corresponding Markov chain model and the mean convergence time to reach one of these stationary states. Furthermore, we derive the most effective mutation rate for the standard GAs and also the most effective migration rate for the island model parallel GAs with some restrictions. These rates are coincide with known empirical results.

This paper describes the application of parallel genetic algorithms (coupled with CFD analysis) to problems of optimal aerodynamic or aerodynamic-structural design of wings and airfoils. The method has been implemented on a variety of parallel architectures, and results to illustrate its application are presented. A common problem with genetic algorithms (GAs) is how to maintain diversity of the gene pool and avoid premature convergence of the population. Subdivision of the population into semi-isolated subpopulations (commonly referred to as ‘demes’) not only helps significantly in this regard, but is ideally suited to implementation on parallel environments. Considerable further advantages may be obtained when some form of automated supervision is added to direct the operation of the parallel GA. A supervision strategy and its parallel implementation are also considered.

A clustering method based on a genetic algorithm for solving the multi-depot routing problem is proposed. An efficient post optimiser enhanced by reduction tests is embedded into the search to further improve the solutions. Preliminary results, based on a set of problems given in the literature, are encouraging.

An approach to combine a genetic algorithm with traditional linear programming based branch and bound for integer programming is described in this paper. Branch and bound provides a systematic search procedure for pure integer programming problems and a genetic approach offers the possibility of rapid movement towards a useful solution. Hence the two approaches look worthy of combination as a way to solve certain {0,1} integer programming problems. The approach has been tested out on satisfiability problems and computational results look promising in certain aspects of speed and solution quality.

Standard heuristic algorithms for the geometric travelling salesman problem (GTSP) frequently produce poor solutions in excess of 25% above the true optimum. In this paper we present some preliminary work that demonstrates the potential of genetic algorithms (GAs) to perturb city coordinates in such a way that the heuristic is ‘fooled’ into producing much better solutions to the GTSP. Initial results for our GA show that by using the nearest neighbour tour construction heuristic on perturbed coordinate sets it is possible to consistently obtain solutions to within a fraction of a percent of the optimum for problems of several hundred cities.

In this paper we present MAX-MIN Ant System (MMAS) that improves on the Ant system. MMAS is a general purpose heuristic algorithm based on a cooperative search paradigm that is applicable to the solution of combinatorial optimization problems. In the experiments we apply MMAS to symmetric and asymmetric travelling salesman problems. We describe in detail the improvements on Ant system, discuss the addition of local search to MMAS, and report on our computational results, showing that our system also improves over other variations of Ant system.

A hybrid genetic algorithm based in local search is described. Local optimisation is not explicitly performed but it is embedded in the exploration of a search metaspace. This algorithm is applied to a NP-hard problem. When it is compared with other GA-based approaches and an exact technique (a branch and bound algorithm), this algorithm exhibits a better overall performance in both cases. Then, a coarse-grain parallel version is tested, yielding notably improved results.

The purpose of the work was to test the feasibility of genetic algorithms (GAs) in the landing call allocation problem of an elevator group.In the first test case the results given by GAs were compared with two current control programs by using an elevator simulator program. In the second test case there were three different buildings which were tested by three different realisations of the same type of passenger traffic flows generated by an elevator simulation program. The results obtained are given as averages of these runs. In the second test case the GA controller was compared with the controller that proved to be better in the first test case.According to the results, it seems that GAs would be suitable for the elevator allocation problem or to solve similar demanding real-time optimization problems. In the simulation tests performed, the average waiting time decreased achieved by GA-based controller was (evaluated with 99% confidence interval) at most 15–33%, when traffic intensity was 140% and at least 1–13%, when traffic intensity was 40%. Accordingly, when traffic intensity was nominal (100%) average waiting time was decreased by GA-controller at most 15–24% and at least 4–12%.

The aim of generator maintenance scheduling (GMS) in an electric power system is to allocate a proper maintenance timetable for generators while maintaining a high system reliability, reducing total production cost, extending generator life time etc. In order to solve this complex problem a genetic algorithm technique is proposed here. The paper discusses the implementation of GAs to GMS problems with two approaches: generational and steady state. The results of applying these GAs to a test GMS problem based on a practical power system scenario are presented and analysed. The effect of different GA parameters is also studied.

We had a problem to be solved: the thermal generator maintenance scheduling problem [13]. We wanted to look at stochastic methods and this paper will present three methods and discuss the pros and cons of each. We will also present evidence that strongly suggests that for this problem, tabu search was the most effective and efficient technique.The problem is concerned with scheduling essential maintenance over a fixed length repeated planning horizon for a number of thermal generator units while minimising the maintenance costs and providing enough capacity to meet the anticipated demand.Traditional optimisation based techniques such as integer programming [2], dynamic programming [14, 15] and branch and bound [3] have been proposed to solve this problem. For small problems these methods give an exact optimal solution. However, as the size of the problem increases, the size of the solution space increases exponentially and hence also the running time of these algorithms.To overcome this difficulty, modern techniques such as simulated annealing [6, 12], stochastic evolution [8], genetic algorithms [4] and tabu search [7] have been proposed as an alternative where the problem size precludes traditional techniques.The method explored in this paper is tabu search and a comparison is made with simulated annealing (the application of simulated annealing to this problem is given in [9]), genetic algorithms and a hybrid algorithm composed with elements of tabu search and simulated annealing.

A topic of interest in genetic algorithm (GA) research is the design of GAs for highly constrained optimization problems. We believe that the most general approach which may be taken in dealing with such difficulty is incorporating domain-specific knowledge into GA architecture. To demonstrate this idea, we consider application of GA to a highly constrained combinatorial optimization problem, the generic crew scheduling problem (CSP). We design a steady-state genetic algorithm for solving the CSP. Computational results axe given for a number of standard test problems. Experimental results demonstrate the effectiveness of the proposed algorithm and support the applicability of our idea.

This paper investigates a number of approaches to encoding and crossover to support timetable design using genetic algorithms, thus extending the range of techniques available for solving such problems. Timetabling is used in this paper to refer to organising a weekly lecture timetable, as used in universities. In addition the algorithm is designed to produce a ‘good’ timetable as defined by a fitness function rather than merely a legal solution. The first approach to encoding timetabling dealt with in this paper uses a ‘greedy algorithm’ variant and a variety of standard crossover methods. The second encoding method searches a wider space of solutions but requires a new adaptation of existing order and position-based crossover algorithms. Results are compared with a traditional search technique and timetables provided by lecturers. These results demonstrate the effectiveness of genetic algorithms when used to optimise a timetable and introduce a combinatorial crossover operator which can deal with a more general class of problem than the normal order and position based operators. The greedy algorithm version of the genetic algorithm outperformed the other methods, despite the fact it cannot search the whole of the legal solution space.

Recent research has yielded a variety of mainly evolutionary algorithm (EA) based methods for dealing with exam and course timetabling problems continually faced by educational institutions A problem as yet unaddressed in the timetabling research literature, however, is the partition/timetabling problem (PTP). This problem most typically arises early during a university term and requires the need to partition groups of students into small tutorial or lab groups, and then timetable these sessions. Even in cases where an institution has effectively ‘solved’ the main course or exam timetabling problems it faces, the continual need to address combined partition/timetabling problems still costs much staff time and effort. This article describes recent work which addresses the PTP. Three different techniques are tried, and results using these techniques axe compared on some real world PTPs. Best results so far are achieved by a two-stage method in which specially developed PTP heuristics are first used to derive a good partition of students into the several tutorial/lab (etc) sessions, and then a relatively standard timetabling EA generates a timetable based on this partition, incorporating operators which may slightly alter the partition.

A novel approach to solving network routing and restoration problems using the genetic programming (GP) paradigm is presented, in which a single robust and fault-tolerant program is evolved which determines the near-shortest paths through a network subject to link failures. The approach is then applied to five different test networks. In addition, two multi-population GP techniques are tried and the results compared to simple GP.

Ring based structures axe often desirable in telecommunication networks since they offer a structure which is inherently fault-tolerant. The simplest such structure consists of a set of nodes connected by links to form a simple cycle. This simple configuration provides high survivability since every pair of nodes is connected by a physically diverse route, and hence no single link failure will disconnect the ring. In the event of failure, all traffic can be diverted as long as the ring has enough capacity. The ring sizing problem is then to determine the minimum capacity to handle all traffic while guaranteeing fault protection. In order to solve this problem, the traffic on the ring has to be routed in such a way as to minimise the maximum load on any link, this is termed the ring loading problem. In this paper we apply both a genetic algorithm and a simulated annealing algorithm to solve this problem.

Evolutionary computation techniques have received a great deal of attention regarding their potential as optimization techniques for complex functions. In this paper, we consider three of them: multiple restart hill-climbing, population-based incremental learning and genetic algorithms. Their binary version and a real-coded variant of each of these techniques are experimented on a real problem: traffic supervision in telephone networks. Indeed, this task need to determine streams responsible for call losses in a network by comparing their traffic values to nominal values. However, stream traffic values are not directly available from the on-line data acquisition system and, hence, have to be computed by inverting a computational model of stream propagation in circuit-switched networks only based on the Erlang’s formula plus qualitative knowledge about the network. Then, our stream propagation model inversion has been computed thanks to the previous techniques and using several fitness measures to show how their choice can impact on the final results.

This paper describes the use of a genetic-algorithm (GA)/heuristic hybrid approach in a tool for optical network modelling, optimisation and design (NOMaD) being developed by the author at the University of Essex [11] and, in particular, early results from its application to virtual-topology design.NOMaD is used as part of the author’s own research into the application of GA/heuristic hybrid optimisation techniques to network design, as well as in several research projects, including two Advanced Communications Technologies and Services (ACTS) projects [6]: WOTAN (wavelength-agile optical transport and access network) and OPEN (optical pan-European network).

The paper presents an approach to topology optimization of all-optical telecommunications network, minimizing pair(s) of unavailability — cost values, by means of a genetic algorithm. The solutions satisfy traffic requirements, technological limitations, and the defined routing rules.

This paper describes a genetic algorithm for solving the minimum span frequency assignment problem (MS-FAP). The MSFAP involves assigning frequencies to each transmitter in a region, subject to a number of constraints being satisfied, such that the span, i.e. range of frequencies used, is minimised. The technique involves finding an ordering of the transmitters for use in a sequential (greedy) assignment process. Results are given for several practical problem instances.

A practical algorithm using a combination of simulated annealing and two problem specific heuristics has been developed for frequency planning in cellular radio networks. It is designed to rapidly generate complete Frequency plans. Experimental work is described that characterises the performance and behaviour of our algorithm.

The frequency assignment problem belongs to the quite difficult to deal with class of NP (nondeterministic polynomial) - hard combinatorial optimization problems [3]. Its computational complexity directs researchers in the field at developing efficient techniques for finding solutions realizing minimum (or maximum) values of an objective function subject to a set of, often conflicting, constraints. To seek an optimal (or near optimal) solution, many methods have been proposed, such as dynamic programming methods, branch and bound methods, etc., and, lately, some heuristic algorithms relating to physical and biological phenomena. They include tabu search, genetic algorithms, simulated annealing and artificial neural networks [4]. We propose a Hop-field neural network model with chaotic neurodynamics to overcome the obstacle of local minima in the energy function and obtain optimal solutions in less iterations than the time-consuming convergent dynamics.

The frequency assignment problem is a computationally hard problem with many applications including the mobile telephone industry and tactical communications. The problem may be modelled mathematically as a T-colouring problem for an undirected weighted graph; it is required to assign to each vertex a value from a given set such that for each edge the difference in absolute value between the values at the corresponding vertices is greater than or equal to the weight of the edge. Tabu search, simulated annealing, simulated neural networks and other heuristic algorithms have been applied to this problem. In this paper we describe a divide and conquer technique incorporating heuristics and present results using test data from real problems which show that with respect to both quality of solutions and speed of execution it is superior to simulated annealing and steepest descent algorithms. It is particularly successful in minimising the span and order of the set of frequencies required to solve the problem.

In this work we axe studying possibilities to test software using genetic algorithm search. The idea is to produce test cases in order to find problematic situations like processing time extremes. The proposed test method comes under the heading of automated dynamic stress testing.

The problem of logistics and resource management in disease control projects in the developing world can hardly be understated. One example is the occurance of regional imbalances in supply. A prototype system, based upon evolutionary and ‘meta-heuristic’ optimisation techniques is described that recommends a plan for the redistribution of available resources to minimise shortages. Evaluation of the system on data from real world situations indicated that the generation of good, feasible redistribution plans is possible even on large datasets. Comparison of the optimisers showed that evolutionary techniques perform poorly on this problem compared to stochastic hill climbing.

We introduce a method, based on a genetic algorithm (GA) approach, to design combinational logic circuits. This problem is quite difficult for a traditional GA, but we have overcome these difficulties and have implemented a computer program that can automatically generate high-quality circuit designs. We describe the important issues to consider when solving this circuit design problem: the importance of the representation scheme, the encoding function, and the definition of the fitness function. We present several circuits derived by our system under various assumed constraints, such as the maximum number of allowable gates and the types of available gates. We compare the solutions produced by our system against those generated by a human designer. We also show that our representation approach, when compared to a standard binary encoding, produces better performance both in terms of quality of solution and in terms of speed of convergence.

The prediction of time series phenomena is a hard and complex task. The selection of a proper statistical model and the setup of its parameters (in terms of the number of coefficients and their values) is also a difficult task and it is usually solved by trial and error. This paper presents a hybrid system that integrates genetic algorithms and traditional statistical models to overcome the model selection and tuning problem. The system is applied to the domain of river Nile inflows forecasting. This domain is characterized by the availability of large amount of data and prediction models. Finally, the results of applying the proposed system are presented and discussed.

In the last years we developed ENZO, an evolutionary neural network optimizer which we compare in this study to standard techniques for topology optimization: optimal brain surgeon (OBS), magnitude based pruning (MbP), and unit-OBS, an improved algorithm deduced from OBS. The algorithms are evaluated on several benchmark problems. We conclude that using an evolutionary algorithm as meta-heuristic like ENZO does is currently the best available optimization technique with regard to network size and performance. We show that the time complexity of ENZO is similar to magnitude based pruning and unit-OBS, while achieving significantly smaller topologies. Standard OBS is outperformed in both size reduction and time complexity.

The effects of a neural network’s topology on its performance are well known, yet the question of finding optimal configurations automatically remains largely open. This paper proposes a solution to this problem for RBF networks. A self-optimising approach, driven by an evolutionary strategy, is taken. The algorithm uses output information and a computationally efficient approximation of RBF networks to optimise the K-means clustering process by co-evolving the two determinant parameters of the network’s layout: the number of centroids and the centroids’ positions. Empirical results demonstrate promise.

We extend our previous theoretical results concerning functional equivalence of Gaussian RBF networks and test the proposed canonical genetic learning algorithm on two problems. In our experiments, canonical learning achieved the same error threshold about two times faster in comparison to standard GA.

We apply genetic algorithms to the Hopfield model of associative memory. Previously, we reported that a genetic algorithm evolves a network with random synaptic weights to store eventually a set of random patterns. In this paper, we show how the Baldwin effect on the evolution enhances the storage capacity.

This paper considers the role of the embryological algorithm or embryology in defining artificial neural network architecture. Such an approach is based on the biology and growth of the embryonic nervous system and operates by ‘growing’ the neural network from a simple to a complex form. The operation of both the embryology and the genetic algorithm are considered, contrasted and compared. A practical algorithm is presented, together with results demonstrating the relevance, application and advantages of the algorithm.

This paper presents the empirical results achieved in the computer system ROBOTS. This program simulates a virtual world, where agents, called robots, interact with an environment. Each robot is controlled by a neural network. The evolution of the robot behaviour (which is determined by the variation of the weights in the neural network) is done using a genetic algorithm. We describe the conceptual model used in ROBOTS. We also show how the genetic parameters and the environment itself influence the robot’s adaptation to the environment.

The determination of the appropriate structure of artificial neural networks for a specific problem or problem domain remains an open question. One attempt to solve this optimization problem is the application of evolutionary algorithms and the choice of an appropriate coding, the genotype → phenotype mapping. We employ a coding procedure from the class of recursivecoding methods and apply the optimization process to the problem of prediction and modeling of chaotic time series. The network structure and the inital weight setting are determined by an evolutionary process and the ‘fine tuning’ of weights is achieved by a standard back-propagation algorithm. We focus on the properties of the coding procedure and the understanding of the network structures in this context.

Starting with an evolutionary algorithm to optimize the architecture of an artificial neural network (ANN), it will be shown that it is possible, with the help of a graph database and genetic engineering, to find modular structures for these networks. A new graph rewriting is used to construct families of architectures from these modular structures. Simulation results for two problems are given. This technique can be useful as an alternative to automatic defined functions for computing intensive structure optimization problems, where modularity is needed.

Recurrent networks have proved to be more powerful than feedforward neural networks in terms of classes of functions they can compute. But, because training of recurrent networks is a difficult task, it is not clear that these networks provide an advantage over feedforward networks for learning from examples. This communication proposes a general computation model that lays the foundations for characterizing the classes of functions computed by feedforward nets and convergent recurrent nets. Then a mathematical statement proves that convergent nets outperform feedforward nets on data fitting problems. It provides the basis to devise a new learning procedure that constraints the attractor set of a recurrent net and assures a convergent dynamic by using orthogonal inverse tools. The learning algorithm is based on an evolutionary selection mechanism. Using the previous procedure as evaluation function, it has been shown to be robust and well adapted to train convergent recurrent nets when feedforward nets cannot approximate a real parameter mapping.

In this paper we study the combination of two powerful approaches, evolutionary topology optimization (ENZO) and temporal difference learning (TD(λ)) which is up to our knowledge the first time. Temporal difference learning was proven to be a well suited technique for learning strategies for solving reinforcement problems based on neural network models, whereas evolutionary topology optimization is concurrently the most efficient network optimization technique. On two benchmarks, a labyrinth problem and the game Nine Men’s Morris, the power of the approach is demonstrated. We conclude that this combination of evolution and reinforcement learning algorithms is a suitable framework that uses the advantages of both methods leading to small and high performing networks for reinforcement problems.

Reinforcement learning algorithms have been used as reasonably efficient model-free techniques for solving small, perfectly observable Markov decision processes. When perfect state determination is impossible, performance is expected to degrade as a result of incorrect updates carried out in wrong regions of the state space. It is shown here that in this case a modified spreading version of Q-learning which takes into account its own uncertainty about the visited states is advantageous if the spreading mechanism fits a measure of similarity on the action-state space. In particular, an agent with an active perception capacity can use an expectation of similar past histories leading to similar results as a justification for this spreading mechanism.

During the past several years, numerous papers and applications designing control systems with genetic algorithms (GAs) have been written. Many of these studies end when the simulated behaviour of the system with the controller is satisfactory. They suppose that the final stage of application of the controller to the real system will be similar to the one from the traditional design.This paper explains the conclusions of the real application of one such publication: the control of an inverted pendulum using a well-known variant of GAs: genetic programming (GP). The aim of this paper is to study the existence of possible special problems in the application stage of genetic-designed controllers. As will be seen, the application stage is more difficult for GAs than for traditional methods, and more knowledge is needed about the system.

In this paper, the stock index S&P 500 is used to test the predicting performance of genetic programming (GP) and genetic programming neural networks (GPNN). While both GP and GPNN are considered universalapproximators, in this practical financial application, they perform differently. GPNN seemed to suffer the overlearning problem more seriously than GP; the latter outdid the former in all the simulations.

This paper studies genetic programming (GP) and its relation to the genetic algorithm (GA). Since the programs used as chromosomes by GP are non-homologous, GP uses a different crossover operator than GA. Thus, by modifying the GA, GP loses the theoretical foundations which have been developed for GA. This paper describes an algorithm (called EPI for evolutionary program induction) that stays within the canonical GA paradigm yet breeds programs in a similar manner to GP. EPI has been tested on three problems whose behavior under GP is known; EPI performed identically to GP over this test suite. The success of the implementation shows that the special crossover used in GP is not necessary to solve program induction using a GA.

The aim of this paper is to refute the claim that the success of genetic programming depends on problem-specific high-level primitives. We therefore apply genetic programming to the λ-calculus, a Turing complete formalism with only two (very low-level) primitives.Genetic programming is suited to find the predecessor function in the space of λ-definable functions without apriori knowledge. The predecessor function is historically important and documented to be ‘a challenge’ and ‘difficult’ to find.

In this paper, we present a new approach, improving the performances of a genetic algorithm (GA). Such algorithms are iterative search procedures based on natural genetics. We use an original genetic algorithm that manipulates pairs of twins in its population: DGA, dual-based genetic algorithm. We show that this approach is relevant for genetic programming (GP), which manipulates populations of trees. In particular, we show that duals can transform a deceptive problem into a convergent one. We also prove that using pairs of dual functions in the primitive function set, is more efficient in the problem of learning boolean functions. Here, in order to prove the theoretical interest of our approach (DGP: dual-based genetic programming), we perform a numerical simulation.

This paper proposes a study of the performance of inductive genetic programming with decision trees. The investigation concerns the influence of the fitness function, the genetic mutation operator and the categorical distribution of the examples in inductive tasks on the search process. The approach uses statistical correlations in order to clarify two aspects: the global and the local search characteristics of the structure of the fitness landscape. The work is motivated by the fact that the structure of the fitness landscape is the only information which helps to navigate in the search space of the inductive task. It was found that the analysis of the landscape structure allows tuning the landscape and increasing the exploratory power of the operator on this landscape.

Parallel Distributed Genetic Programming (PDGP) is a new form of genetic programming suitable for the development of parallel programs in which symbolic and neural processing elements can be combined in a free and natural way. This paper describes the representation for programs and the genetic operators on which PDGP is based. Experimental results on the XOR problem axe also reported.

A number of published studies have investigated the application of neural network technology to residential property appraisal. The majority of these studies have concentrated on homogeneous areas (that is areas where properties are subject to the same environmental and locational factors). This is generally done to restrict the data set to one local sub-market. However, the models created are specialised and not locationally portable. This paper presents a methodology, which builds on research in which a Kohonen map is used to uncover sub-markets within a large data set that are subsequently independently used to train a series of multi layer perceptron (MLP) networks. The study concludes that by modelling possible sub-markets an acceptable accuracy over a heterogeneous area can be achieved. The work presented in this paper is funded via a Realising Our Potential Award under the auspices of the ESRC.

An artificial neural network which is based on harmony theory has been applied to the graph planarisation problem. The presented artificial neural network is capable of solving both aspects of graph planarisation (creation of the optimally planarised as well as of the maximally planar forms of any graph). Optimal solutions are always produced independent of the complexity or planarity of the original graph; if more than one equally good solutions exist they are produced with the same probability.

This paper outlines a research project to develop artificial neural networks as a diagnostic tool for the automatic identification of rotating machine faults. This work was instigated by the DTI Neural Computing Learning Solutions Campaign’s AXON Neural Projects Club. Industrial sponsors are Entek/IRD, Diagnostic Instruments Ltd., and Arjo Wiggins Paper Mills. The biggest problem encountered by developers of SMART software systems is that examples of all conditions to be identified are required. In practice this is not possible due to the routine method of plant machinery data collection, and due to the individual behaviour of the machinery. A method is required which is capable of diagnosing a previously unseen fault upon any bearing. This paper proposes a hybrid neural network approach which first determines a novel condition, then a knowledge base categorises the condition. The system is currently working off-line in support of the maintenance technician at the sponsors paper mill plant.

This paper considers polling policies represented by recurrent neural networks and investigates the effect of feedback weights on the optimality. The polling system consists of a single server and multiple stations and whenever the server finishes serving one of the stations, it determines the next station to visit according to the output of the neural network for the current system state. By using the simulated annealing method, we improve the polling policy in such a way that the mean delay of customers is to be minimized in the steady state. The benefit of applying recurrent networks is in that they can represent a broader class of policies than feedforward networks. Numerical results show that recurrent networks can substantially reduce the (sub-)optimal mean delay in comparison with feedforward networks.

This paper examines the problems associated with equity valuation and exposes the weaknesses of current computer based modelling systems. The paper identifies the necessary requirements of a strategic software system for equity valuation and proposes a software system with an integrated architecture, which combines both artificial intelligence technologies with conventional software. The paper then describes a powerful strategic software system, EXPRESS, developed by the authors to significantly de-skill and improve stock valuation. The EXPRESS system is a state-of the-art windows based application capable of performing the three accepted methods of stock valuation namely, quantitative, technical and fundamental analysis. The paper outlines the integrated architecture of EXPRESS and describes the function of each of the three integrated software systems, the EXPRESS decision support generator system, the EXPRESS artificial neural network system and the EXPRESS extension model system.

This paper discusses the design of a virtual table-tennis environment, and the design of neural network based controllers to play in that environment. The motivation behind the work is to provide an interesting and entertaining forum in which to carry out research on adaptive control and planning problems that stretch the limits of current neural network paradigms.

This paper reports on recent experiments applying classifier systems to the problem of supporting both local and global navigation in a simulated animat. The basis of this research is a hybrid learning system that extends the classifier representation to enable environmental feedback to impinge directly upon the classifier population. The system applies a connectionist representation to the condition sets of classifiers which enables the direct encoding of classifier condition/fitness values onto network nodes. The goal of the system is to achieve domain objectives by calibrating classifier behaviour during the exploration of the domain and evolving new classifiers to exploit the domain by discovering goal states.

This paper outlines the form and structure of an activation passing context rich network ideally suited to tasks such as; recovering from noisy or damaged data, recognition or spelling correction. Such a network has been shown to be in many ways similar to N-Gram analysis or the transition matrix of Markov Models. However it is superior in that the scope of context (N) to be considered is dynamically and locally identified for each node. A pair of algorithms are presented which can be used to produce such networks. The first uses global working memory to deal with the time dependent nature of the input data, the second uses time delay nodes within the network. The latter, however, is considered superior as it reduces algorithmic complexity and increases computational efficiency.

We study the problem of modeling long, complex symbolic sequences with recurrent neural networks (RNNs) and stochastic machines (SMs). RNNs are trained to predict the next symbol and the training process is monitored with information theory based performance measures. SMs are constructed using Kohonen self-organizing map quantizing RNN state space. We compare generative models through entropy spectra computed from sequences, or directly from the machines.

We present a novel neural network method for extraction of the embedding function of a time series. We give results on two sets of computer-generated data which are known to show exponentially increasing divergence from nearby initial conditions. We use the network to predict the future evolution of these artificial mappings.

In this paper, two different approaches of time-series prediction with neural networks are presented. The first is called combinatorial because it deals with a finite set of classes, obtained from the differences between several consequent function values. It is implemented through a modular neural network. The second describes time-series with interval functions or sequences of successive function values and is therefore a sequential approach, employing Kaiman neural gas networks. In the first case the future value (prediction) of an input vector depends on the classes (from input vectors, possibly together with the next values) obtained from learning the history of a time-series. In the second, based on the sequence of last input vector(s), the closest covering neuron (interval function) is defined, and is responsible for a future value calculation. A linear autoregressive method (AR) and multilayer perceptron (MLP) are used as references, and with the help of three different time-series, the efficiency of the suggested methods is given.

This paper describes a new numerical method for defining the threshold and delay parameters of k-order self-exciting threshold autoregressive (SETAR) models. The idea of the method is to divide a time series into ascending and descending parts. This division can especially be exploited in building prediction models: it is a considerably easier task to predict the ascending and descending parts separately than to try to predict both of them at the same time.Another issue of this paper is to build the prediction model with only the most significant predictor variables. This is achieved with the help of evolutionary optimizing. In the first step we generate a number of models (networks) with different predictor variables, then we train each of the networks and the fittest ones are used for reproduction. In the beginning we also set the possible number of predictor variables to some constant. In the next step we reduce the number of predictor variables by leaving out the ones not chosen by the algorithm in the first step. Then we repeat the training and dropping procedure until the genetic algorithm does not drop any predictors anymore.I have shown in this paper that the division procedure works very well, that building separate prediction models increases the accuracy of the predictions noticeably, and that using evolutionary optimization is succesful in dropping out the irrelevant predictors.

The incorporation of the random vector functional link (RVFL) concept into mixture models for predicting conditional probability densities achieves a considerable speed-up of the training process. This allows the creation of a large ensemble of predictors, which results in an improvement in the generalization performance.

Quantum computation uses microscopic quantum level effects to perform computational tasks and has produced results that in some cases are exponentially faster than their classical counterparts. Choosing the best weights for a neural network is a time consuming problem that makes the harnessing of this ‘quantum parallelism’ appealing. This paper briefly covers necessary high-level quantum theory and introduces a model for a quantum neuron.

An architecture for neural computation is proposed. The costliest parts of neural computation: inter-unit-communication, weight representation, and weighted sum execution are all implemented by natures of wave propagation and interaction. As a result, unit interaction is realized without wires, weighted sum is executed without adders or multipliers, and the weight values are coded as unit positions. Some experimental results, which are based on software models of the wave natures, show that the approach provides a promising computation potential regardless of its limited ability to represent connection weights.

A two-node fully recurrent neural network producing oscillations is analysed. The network has no true inputs and the outputs from the network exhibit a circular phase portrait. The weight configuration of the network is investigated, resulting in analytical weight expressions. The theoretical predictions are compared with numerical weight estimates obtained by training the network on the desired trajectory. The analysis shows that the analytical expressions agree well with the findings of the numerical study.

Real-valued functions with multiple boolean variables are represented by one-hidden-layer Heaviside perceptron networks with an exponential number of hidden units. We derive upper bounds on approximation error using a given number n of hidden units. The bounds on error axe of the form $$\frac{c}{\sqrt{n}}$$ where c depends on certain norms of the function being approximated and n is the number of hidden units. We show examples of functions for which these norms grow polynomially and exponentially with increasing input dimension.

It is well known that large-scale neural networks suffer from serious problems such as the scale problem and the local minima problems. Modular architecture neural network is an approach to alleviate these problems. It is important that the construction of modular neural network is the selection or construction of a network that can converge and has the good generalization ability for a task, and the Akaike Information Criterion (AIC) is a criterion of evaluation of estimated model from observed parameters is a very useful tool for selection of network. This paper proposes a method for task allocation in a modular architecture neural network. The method allocates a best fit network that has a good generalization ability from multiple neural networks for a task with (AIC) and the state of convergence of a network, simply and certainly. The performance of proposed method is evaluated with the Fisher’ Iris data and the What and Where vision tasks.

This paper proposes an application independent method of automating learning rule parameter selection using a group of supervisor neural networks, known as meta neural networks, to alter the value of a learning rule parameter during training. Each meta neural network is trained using data generated by observing the training of a neural network and recording the effects of the selection of various parameter values. A group of meta neural networks is then polled to obtain a parameter value for a learning rule. Experiments are undertaken to see how this method performs by using it to adapt a global parameter of RPROP.

Using artificial evolution to successfully create neural networks requires appropriate developmental algorithms. The aim is to determine the least complex set of rules that allow a range of networks to evolve. This paper presents a set of generic growth rules that abstractly model the biological processes associated with the development of neuron-to-neuron connections. Substantially different 3D artificial neural structures can be grown by changing parameter values associated with the rules. A genetic algorithm has been successfully employed in determining parameter values that lead to specific neural structures.

Probabilistic neural networks (PNN) typically learn more quickly than many neural network models and have had success on a variety of applications. However, in their basic form, they tend to have a large number of hidden nodes. One common solution to this problem is to keep only a randomly selected subset of the original training data in building the network. This paper presents an algorithm called the reduced probabilistic neural network (RPNN) that seeks to choose a better than random subset of the available instances to use as center points of nodes in the network. The algorithm tends to retain non-noisy border points while removing nodes with instances in regions of the input space that are highly homogeneous. In experiments on 22 datasets, the RPNN had better average generalization accuracy than two other PNN models, while requiring an average of less than one-third the number of nodes.

This paper presents a theoretical and empirical analysis of the evolution of a feedforward neural network (FFNN) trained using backpropagation (BP). The results of two sets of experiments axe presented which illustrate the nature of BP’s search through weight space as the network learns to classify the training data. The search is shown to be driven by the initial values of the weights in the output layer of neurons.

In this paper we propose a method of classification based on standard feedforward neural networks. The novelty of the approach is that we calculate local approximations of Lie algebras which generate the leaves of a foliation, each leaf corresponds to a class. From these linear approximations we pass to the case where a point on a leaf is not known with precision but can be specified using fuzzy set theory. Integrating the approximating linear equations then provides us with ‘fuzzy leaves’ or fuzzy classes.

In this paper, an overview of a new constructive algorithm is proposed. Most common binary-unit based constructive algorithms are faced with 3 major drawbacks: binary inputs, only one output and a complex connectivity. The proposed algorithm aims to overcome these problems. An extension to multiple outputs of the sequential learning algorithm in combination with the Barycentric correction procedure is used. The performance of this new algorithm is evaluated in comparison with cascade correlation, on the Vowel classification benchmark.

The wall or cooling water temperatures in the iron-making blast furnace are important sources of information about the internal conditions of the process, which cannot be measured directly because of high temperatures, mechanic wear and hostile environment. Two alternative methods of classification of wall thermal load profiles are studied: a feedforward neural network trained on manually classified temperature profiles, and an unsupervised classification using a Kohonen network. Both approaches were found to yield interesting results. In the former approach, the results can be easily explained since the characteristics of the profile classes are known, while in the latter approach the generalization properties and the ease of retraining the networks were considered advantageous. The classifiers are being implemented in the automation system of a Finnish blast furnace.

In this paper, we introduce a method that allows to evaluate efficiently the ‘importance’ of each coordinate of the input vector of a neural network. This measurement can be used to obtain information about the studied data. It can also be used to suppress irrelevant inputs in order to speed up the classification process conducted by the network.

The main aim of this paper is three fold: a) to understand the working of a classifier system based on possibility distribution functions, b) to evaluate its performance against other superior methods such as fuzzy and non-fuzzy neural networks on real data, c) and finally to recommend changes for enhancing its performance. The paper explains how to construct a possibility based classifier system which is used with conventional error-estimation techniques such as cross-validation and bootstrapping. The results were obtained on a set of electronic nose data and this performance was compared with earlier published results on the same data using fuzzy and non-fuzzy neural networks. The results show that the possibility approach is superior to the non-fuzzy approach, however, further work needs to be done.

This paper introduces a computational network which combines computational intelligent programs into a general framework for intelligent data analysis. Integrating more than one program may potentially lead to more powerful and versatile results.

Induction of useful rules from databases has been studied by several researchers. There remains need for systematic comparison of alternative such methods, especially considering the available variety of rule representation strategies, genetic operators, evolutionary algorithm designs, and so forth. Here, the performance of five commonly employed evolutionary algorithms are examined on a collection of 100 separate rule induction tasks on five freely available datasets. All tasks require the generation of rules in disjunctive normal form with either a fixed or free consequent maximising an accuracy/applicability tradeoff measure; tasks differ in terms of the dataset used, the identity of a fixed consequent (or no fixed consequent), and the maximum number of disjuncts allowed in the antecedent. Results generally indicate that single-member based methods (hill climbing, simulated annealing, tabu search) fare at least as well as population based techniques when rules are restricted to fairly low complexity, but this situation is reversed as rules are allowed to be more complex. These results are of import to data mining application developers and researchers wishing to find the appropriate search strategy for rule induction with respect to their particular needs.

The stochastic competitive evolutionary neural tree (SCENT) is a new unsupervised neural net that dynamically evolves a representational structure in response to its training data. Uniquely SCENT requires no initial parameter setting as it autonomously creates appropriate parameterisation at runtime. Pruning and convergence are stochastically controlled using locally calculated heuristics. A thorough investigation into the performance of SCENT is presented. The network is compared to other dynamic tree based models and to a high quality flat clusterer over a variety of data sets and runs.

Non-stationary clustering deals with the clustering of a sequence of data samples obtained at a diverse time instant. A paradigm case of non-stationary clustering is the color quantization of image sequences. We propose an efficient evolution strategy to compute adaptively the color representatives for each image in the sequence.

By mediating excursion sets in the selection phase of the standard evolutionary strategies (ES), Plus and Comma, two possible new variants which we call, Semicolon, and Colon strategies have been devised and their optimization behavior on model landscapes reported. By an internal mechanism, these strategies systematically vary the excursion level, and achieve stronger self adaptive exploration of the fitness landscapes exhibiting emergent characteristics of both Comma and Plus to the advantage of them.

This paper investigates the application of the mutual information for the evaluation of neuron inputs and for the selection of the relevant ones. The rules extraction method is based on the notion of weights templates, parameterizing regions of weights space using the mutual information criteria. The simulation results obtained with this method are very satisfactory.

In this paper we examine a previously published algorithm which addresses the problem of network growth by implementing a clustering algorithm to operate on time dependant data. The computational constraints of the problem forced the development of an architecture, which in retrospect can be analysed in terms of a computational and symbolic module operating symbiotically. Here we attempt to identify the computational constraints that necessitate the use of this architecture, and any further merits it has. Further, we analyse the nature of the interaction between the two modules and highlight the manner in which the behaviour the symbiotic modules correlates with what is known of human problem solving behaviour.

Genetic algorithms are proposed as a new and novel technique to solve the problem of designing a robust PID controller for a plant with model uncertainties. The evolutionary scheme used, involves generating two separate populations, one representing the controller and the other the plant. The controller population is then co-evolved against a fixed population of plants covering the plant uncertainty search space, such that the controller can control all the plants effectively. A time domain cost function subjected to a frequency domain vector margin stability constraint, is then deployed in order to obtain a robust controller design. This evolutionary approach is illustrated by evolving a PID controller for a linear plant which has a set of prescribed model uncertainties.

This text describes the use of a coevolutionary genetic algorithm (CGA) for process control. A CGA combines two artificial life techniques - life-time fitness evaluation (LTFE) and coevolution - to improve the genetic search for a neural network (NN) controlling a given process.Here, the approach is illustrated and tested on a well-known bioreactor control problem which involves issues of delay, nonlinearity and instability.

This paper introduces an extension to Potter and De Jong’s [5] work on cooperative coevolutionary GAs (CCGA). We discuss the problem of inventory control and discuss the potential advantages of applying an evolutionary method to this problem. We describe our approach and experimental design for solving the inventory control problem using a CCGA. We also show how the CCGA can be extended and modified in order to handle larger inventory control optimisation problems.

This work shows the use of a novel neural model for identification of non-linear process. The neural model make use of internal dynamic with dynamical neurons. The parameters responsible for the dynamic of the neural net are adjustable, giving a high flexibility for the neural model in process identification.

This paper addresses stability issues of the learning process when the distal-in-space approach is used to learn inverse models of dynamical systems. Both direct and indirect versions of this approach are analysed for linear plants. It is shown that none of them is suitable when the plant is an unstable non-minimum phase system. When the plant is unstable, an additional problem must be solved: stability of the control system must be guaranteed at the beginning of the learning process. We do not deal with this additional problem but concentrate on the stability and the speed of the learning process. We propose solutions in the case of the direct version, applied to non-minimum phase stable plants. These solutions are compared on a linear plant control problem. Extensions to nonlinear systems are briefly discussed.

Genetic algorithms have been recently applied to model both linear and non-linear systems. Different methods of coding the problem solutions were proposed and were claimed to have good performance. This paper presents a comparative study of three of the methods with their strengths and weaknesses highlighted.

This paper presents an application of multilayered feedforward neural networks for controlling a PT326 process trainer laboratory equipment. Firstly, the process as well as its inverse have been identified using the Levenberg-Marquardt algorithm for neural network training. Secondly an internal model control (IMC) strategy has been used for neurocontrol. Different architectures and learning methods have been investigated for model approximation. Control of the process has been implemented in real-time using the Simulink/Matlab environment. Experimental results regarding the performance of the control scheme axe included in a comparative study.

A large number of design strategies exist for multivariable control situations. Many of the methods require a linear time-invariant process characterisation in the form of a state space model or transfer function matrix description. Quite often this is not available and could be expensive to realise. If this latter route is pursued there needs to be considerable benefits in the quality of the resulting closed-loop performance. One alternative is to use the ‘expert’ approach of fuzzy logic where the plant is not modelled but the expert operator is. Application of such a controller is not so straightforward as many parameters need to be ‘tuned’ in order to provide precise control of a non-linear system.

The aim of this project is to improve the quality and consistency of coiling in a steel hot strip mill at British Steel Strip Products, Integrated Works. The artificial intelligence paradigm of learning classifier systems (LCS) is proposed for the processing of plant data. Improvements to a basic LCS, that allow operation on industrial data, are detailed. Initial experimental results show that the technique of LCS has the potential to become a very useful tool for processing industrial data. The stochastic computational technique will produce off- line rules to aid operator and engineering decision making. Improvements in availability, coil presentation and ultimately customer satisfaction will result in cost benefit to British Steel Plc.

This paper describes experiments using multiple classifier system (CS) agents to play the iterated prisoner’s dilemma (IPD) under various conditions. Our main interest is in how, and under what circumstances, co-operation is most likely to emerge through competition between these agents. Experiments are conducted with agents playing fixed strategies and other agents individually and in tournaments, with differing CS parameters. Performance improves when reward is stored and averaged over longer periods, and when a genetic algorithm (GA) is used more frequently. Increasing the memory of the system improves performance to a point, but long memories proved difficult to reinforce fully and performed less well.

This study seeks to understand the effect of complexity cost and two types of transmission noise on equilibrium selection in populations of finite automata playing the repeated prisoner’s dilemma. Results indicate that noise and complexity cost have a harmful effect on the types of conditionally co-operative strategies essential for the emergence of co-operative behaviour in the population. In contrast, the unconditionally defecting strategy responsible for the dominance of mutual defection is relatively unharmed under these conditions.