Engineering Applications of Neural Networks

This paper presents the design and the development of an agent-based intelligent hybrid system. The system consists of a network of interacting intelligent agents aiming not only towards real-time air pollution monitoring but towards proposing proper corrective actions as well. In this manner, the concentration of air pollutants is managed in a real-time scale and as the system is informed continuously on the situation an iterative process is initiated. Four distinct types of intelligent agents are utilized: Sensor, Evaluation, Decision and Actuator. There are also several types of Decision agents depending on the air pollution factor examined. The whole project has a Hybrid nature, since it utilizes fuzzy logic – fuzzy algebra concepts and also crisp values and a rule based inference mechanism. The system has been tested by the application of actual air pollution data related to four years of measurements in the area of Athens.

The aim of the paper is to describe a methodology of damage detection which is based on artificial neural networks in combination with stochastic analysis. The damage is defined as a stiffness reduction (bending or torsion) in certain part of a structure. The key stone of the method is feed-forward multilayer network. It is impossible to obtain appropriate training set for real structure in usage, therefore stochastic analysis using numerical model is carried out to get training set virtually. Due to possible time demanding nonlinear calculations the effective simulation Latin Hypercube Sampling is used here. The important part of identification process is proper selection of input information. In case of dynamically loaded structures their modal properties seem to be proper input information as those are not dependent on actual loading (traffic, wind, temperature). The methodology verification was carried out using laboratory beam.

The treatment of incomplete data is an important step in the pre-processing of data. We propose a non-parametric multiple imputation algorithm (GMI) for the reconstruction of missing data, based on Generalized Regression Neural Networks (GRNN). We compare GMI with popular missing data imputation algorithms: EM (Expectation Maximization) MI (Multiple Imputation), MCMC (Markov Chain Monte Carlo) MI, and hot deck MI. A separate GRNN classifier is trained and tested on the dataset imputed with each imputation algorithm. The imputation algorithms are evaluated based on the accuracy of the GRNN classifier after the imputation process. We show the effectiveness of our proposed algorithm on twenty-six real datasets.

The paper presents the modelling possibilities of kernel-based approaches on a complex real-world problem, i.e. municipal creditworthiness classification. A model design includes data pre-processing, labelling of individual parameters’ vectors using expert knowledge, and the design of various support vector machines with supervised learning and kernel-based approaches with semi-supervised learning.

The batch Self-Organizing Map (SOM) is applied to clustering of pressure fluctuation in liquid-liquid flow inside a microchannel. When time-series data of the static pressure are computed by the SOM, several clusters of pressure fluctuation with different amplitudes are extracted in the visible way. Since the signal composition of the fluctuation is considered to change with flow rates of the water and the organic solvent, the ratio to the each cluster, which is estimated by the recalling, is classified by using the SOM. Consequently, the operating condition of flow rate is classified to three groups, which indicate characteristic behavior of interface between two flows in the microchannel. Furthermore, predictive performance for behavior of the interface is demonstrated to be good by the recalling.

This paper presents the design of a fuzzy algebra model and the implementation of its corresponding Intelligent System (IS). The System is capable of estimating the risk due to extreme disaster phenomena and especially due to natural hazards. Based on the considered risk parameters, an equal number of fuzzy sets are defined. For all of the defined fuzzy sets trapezoidal membership functions are used for the production of the partial risk indices. The fuzzy sets are aggregated to a single one that encapsulates the overall degree of risk. The aggregation operation is performed in several different ways by using various Fuzzy Relations. The degree of membership of each case to an aggregated fuzzy set is the final overall degree of risk. The IS has been applied in the problem of torrential risk estimation, with data from river Evros. The compatibility of the system to existing models has been tested and also the results obtained by two distinct fuzzy approaches have been compared.

Authentication is becoming ever more important in computer-based applications because the amount of sensitive data stored in such systems is growing. However, in embedded computer-system applications, authentication is difficult to implement because resources are scarce. Using fuzzy logic and artificial neural networks to process biometric data can yield improvements in authentication performance by limiting memory and processing-power requirements. A multibiometric platform that combines voiceprint and fingerprint authentication has been developed. It uses traditional pattern-matching algorithms to match hard-biometric features. An artificial neural network was trained to match soft-biometric features. A fuzzy logic inference engine performs smart decision fusion and authentication. Finally, a digital signal processor is used to embed the entire identification system. The embedded implementation demonstrates that improvement in performance is attainable, despite limited system resources.

There have been many studies which analyze complex alpha spectra based on numerically fitting the peaks to calculate the activity level of the sample. In the present work we propose a different approach – the application of neural network techniques to fit the peaks in alpha spectra. Instead of using a mathematical function to fit the peak, the fitting is done by a neural network trained with experimental data corresponding to peaks of different characteristics. We have designed a feed-forward (FF) multi-layer perceptron (MLP) artificial neural network (ANN), with supervised training based on a back-propagation (BP) algorithm, trained on the peaks of Polonium, extracted from many spectra of real samples analyzed in the laboratory. With this method, we have achieved a fitting procedure that does not introduce any error greater than the error of measurement, evaluated to be 10%.

In this paper, we propose a new information-theoretic method called ”information enhancement learning” to realize competitive learning and self-organizing maps. In addition, we propose a computational method to detect the importance of input variables and to find the optimal input variables. In our information enhancement learning, there are three types of information, namely, self-enhancement, collective enhancement and local enhancement. With self-enhancement and collective enhancement, we can realize self-organizing maps. In addition, we use local enhanced information to detect the importance of input units or input variables. Then, the variance of local information is used to determine the optimal values of the enhanced information. We applied the method to an artificial data. In the problem, information enhancement learning was able to produce self-organizing maps close to those produced by the conventional SOM. In addition, the importance of input variables detected by local enhanced information corresponded to the importance obtained by directly computing errors.

The feasibility of flash flood forecasting without making use of rainfall predictions is investigated. After a presentation of the “cevenol flash floods“, which caused 1.2 billion Euros of economical damages and 22 fatalities in 2002, the difficulties incurred in the forecasting of such events are analyzed, with emphasis on the nature of the database and the origins of measurement noise. The high level of noise in water level measurements raises a real challenge. For this reason, two regularization methods have been investigated and compared: early stopping and weight decay. It appears that regularization by early stopping provides networks with lower complexity and more accurate predicted hydrographs than regularization by weight decay. Satisfactory results can thus be obtained up to a forecasting horizon of three hours, thereby allowing an early warning of the populations.

Support Vector Machines (SVMs) have strong theoretical foundations and excellent empirical success in many pattern recognition and data mining applications. However, when induced by imbalanced training sets, where the examples of the target class (minority) are outnumbered by the examples of the non-target class (majority), the performance of SVM classifier is not so successful. In medical diagnosis and text classification, for instance, small and heavily imbalanced data sets are common. In this paper, we propose the Boundary Elimination and Domination algorithm (BED) to enhance SVM class-prediction accuracy on applications with imbalanced class distributions. BED is an informative resampling strategy in input space. In order to balance the class distributions, our algorithm considers density information in training sets to remove noisy examples of the majority class and generate new synthetic examples of the minority class. In our experiments, we compared BED with original SVM and Synthetic Minority Oversampling Technique (SMOTE), a popular resampling strategy in the literature. Our results demonstrate that this new approach improves SVM classifier performance on several real world imbalanced problems.

In this paper we propose a visual approach for the analysis of nonlinear multivariable systems whose dynamic behaviour can be defined in terms of locally linear MIMO (Multiple Input, Multiple Output) behaviours that change depending on a given set of variables (such as e.g. the working point). The proposed approach is carried out in two steps: 1) building a smooth 2-D map of such set of variables using Self-Organizing Maps (SOM) and 2) obtaining a local MIMO ARX (Auto-Regressive with eXogenous input) model for each SOM unit. The resulting structure allows to estimate the process data with an accuracy comparable to other state-of-the-art nonlinear estimation techniques but, in addition, it allows to visualize the MIMO dynamics information stored in the SOM using component planes as done in the SOM literature, bringing the power of visualization to acquire insight useful for process understanding and for control system design. The proposed approach is applied to an industrial-scale version of the well known 4-tank plant, showing a comparison in terms of estimation accuracy with a global linear estimator and with a NARX (Nonlinear Auto-Regressive with eXogenous input) estimator based on a Multi-Layer Perceptron (MLP), as well as, visualizations of MIMO dynamic features such as directionality, RGA (Relative Gain Array), and singular frequency gains for the aforementioned plant.

Visualising data for exploratory analysis is a major challenge in many applications. Visualisation allows scientists to gain insight into the structure and distribution of the data, for example finding common patterns and relationships between samples as well as variables. Typically, visualisation methods like principal component analysis and multi-dimensional scaling are employed. These methods are favoured because of their simplicity, but they cannot cope with missing data and it is difficult to incorporate prior knowledge about properties of the variable space into the analysis; this is particularly important in the high-dimensional, sparse datasets typical in geochemistry. In this paper we show how to utilise a block-structured correlation matrix using a modification of a well known non-linear probabilistic visualisation model, the Generative Topographic Mapping (GTM), which can cope with missing data. The block structure supports direct modelling of strongly correlated variables. We show that including prior structural information it is possible to improve both the data visualisation and the model fit. These benefits are demonstrated on artificial data as well as a real geochemical dataset used for oil exploration, where the proposed modifications improved the missing data imputation results by 3 to 13%.

We present a 2D triangle mesh simplification model, which is able to produce high quality approximations of any original planar mesh, regardless of the shape of the original mesh. This model is applied to reduce the urban concentration of a real geographical area, with the property to maintain the original shape of the urban area. We consider the representation of an urbanized area as a 2D triangle mesh, where each node represents a house. In this context, the neural network model can be applied to simplify the network, what represents a reduction of the urban concentration. A real example is detailed with the purpose to demonstrate the ability of the model to perform the task to simplify an urban network.

The paper describes a classification method of multidimensional signals, based upon a dissimilarity measure between signals. Each new signal is compared to some reference signals through a conjoint dynamic time warping algorithm of their time features series, of which proposed cost function gives out a normalized dissimilarity degree. The classification then consists in presenting these degrees to a classifier, like k-NN, MLP or SVM. This recognition scheme is applied to the automatic estimation of the Phytoplanktonic composition of a marine sample from cytometric curves. At present, biologists are used to a manual classification of signals, that consists in a visual comparison of Phytoplanktonic profiles. The proposed method consequently provides an automatic process, as well as a similar comparison of the signal shapes. We show the relevance of the proposed dissimilarity-based classifier in this environmental application, and compare it with classifiers based on the classical DTW cost-function and also with features-based classifiers.

Abdominal pain in childhood is a common cause of emergency admission in hospital. Its assessment and diagnosis, especially the decision about performing a surgical operation of the abdomen, continues to be a clinical challenge. This study investigates the possibilities of applying state of the art computational intelligence methods for the analysis of abdominal pain data. Specifically, the application of a Genetic Clustering Algorithm and of the Random Forests algorithm (RF) is explored. Clinical a

ppendicitis prediction

involves the estimation of at least 15 clinical and laboratory factors (features). The contribution of each factor to the prediction is not known. Thus, the goal of abdominal pain data analysis is not restricted to the classification of the data, but includes the exploration of the underlying data structure. In this study a genetic clustering algorithm is employed for the later task and its performance is compared to a classical K-means clustering approach. For classification purposes, tree methods are frequently used in medical applications since they often reveal simple relationships between variables that can be used to interpret the data. They are however very prone to overfitting problems. Random Forests, applied in this study, is a novel ensemble classifier which builds a number of decision trees to improve the single tree classifier generalization ability. The application of the above mentioned algorithms to real data resulted in very low error rates, (less than 5%), indicating the usefulness of the respective approach. The most informative diagnostic features as proposed by the algorithms are in accordance with known medical expert knowledge. The experimental results furthermore confirmed both, the greater ability of the genetic clustering algorithm to reveal the underlying data patterns as compared to the K-means approach and the effectiveness of RF-based diagnosis as compared to a single decision tree algorithm.

Most printed material is produced by printing halftone dot patterns. One of the key issues that determine the attainable print quality is the structure of the paper surface but the relation is non-deterministic in nature. We examine the halftone print quality and study the statistical dependence between the defects in printed dots and the topography measurement of the unprinted paper. The work concerns SC paper samples printed by an IGT gravure test printer. We have small-scale 2D measurements of the unprinted paper surface topography and the reflectance of the print result. The measurements before and after printing are aligned with subpixel resolution and individual printed dots are detected. First, the quality of the printed dots is studied using Self Organizing Map and clustering and the properties of the corresponding areas in the unprinted topography are examined. The printed dots are divided into high and low print quality. Features from the unprinted paper surface topography are then used to classify the corresponding paper areas using Support Vector Machine classification. The results show that the topography of the paper can explain some of the print defects. However, there are many other factors that affect the print quality and the topography alone is not adequate to predict the print quality.

Gene regulatory networks dynamically orchestrate the level of expression for each gene in the genome. With such unique characteristics, they can be modeled as reliable and robust control mechanisms for robots. In this work we devise a recurrent neural network-based GRN model to control robots. To simulate the regulatory effects and make our model inferable from time-series data, we develop an enhanced learning algorithm, coupled with some heuristic techniques of data processing for performance improvement. We also establish a method of programming by demonstration to collect behavior sequence data of the robot as the expression profiles, and then employ our framework to infer controllers automatically. To verify the proposed approach, experiments have been conducted and the results show that our regulatory model can be inferred for robot control successfully.

Recognizing expressions are a key part of human social interaction, and processing of facial expression information is largely automatic for humans, but it is a non-trivial task for a computational system. The purpose of this work is to develop computational models capable of differentiating between a range of human facial expressions. Raw face images are examples of high dimensional data, so here we use two dimensionality reduction techniques: Principal Component Analysis and Curvilinear Component Analysis. We also preprocess the images with a bank of Gabor filters, so that important features in the face images are identified. Subsequently the faces are classified using a Support Vector Machine. We show that it is possible to differentiate faces with a neutral expression from those with a happy expression and neutral expression from those of angry expressions and neutral expression with better accuracy. Moreover we can achieve this with data that has been massively reduced in size: in the best case the original images are reduced to just 5 components with happy faces and 5 components with angry faces.

In this paper, we use wavelet neural networks in order to model a mean-reverting Ornstein-Uhlenbeck temperature process, with seasonality in the level and volatility. We forecast up to two months ahead out of sample daily temperatures and we simulate the corresponding Cumulative Average Temperature and Heating Degree Day indices. The proposed model is validated in 8 European and 5 USA cities all traded in Chicago Mercantile Exchange. Our results suggest that the proposed method outperforms alternative pricing methods proposed in prior studies in most cases. Our findings suggest that wavelet networks can model the temperature process very well and consequently they constitute a very accurate and efficient tool for weather derivatives pricing. Finally, we provide the pricing equations for temperature futures on Heating Degree Day index.

The automated detection of defective modules within software systems could lead to reduced development costs and more reliable software. In this work the static code metrics for a collection of modules contained within eleven NASA data sets are used with a Support Vector Machine classifier. A rigorous sequence of pre-processing steps were applied to the data prior to classification, including the balancing of both classes (defective or otherwise) and the removal of a large number of repeating instances. The Support Vector Machine in this experiment yields an average accuracy of 70% on previously unseen data.

Improving bit error rates in optical communication systems is a difficult and important problem. Error detection and correction must take place at high speed, and be extremely accurate. Also, different communication channels have different characteristics, and those characteristics may change over time. We show the feasibility of using simple artificial neural networks to address these problems, and examine the effect of using different representations of signal waveforms on the accuracy of error correction. The results we have obtained lead us to the conclusion that a machine learning system based on these principles can improve on the performance of existing error correction hardware at the speed required, whilst being able to adapt to suit the characteristics of different communication channels.

The application of neural networks to solve an engineering problem is introduced in the paper. Artificial neural networks (ANN) are used for model parameters identification of aircraft motion. Unlike conventional identification methods, neural networks have memory, so results are verified and accumulated during repeated “training” cycles (when new samples of initial data are used). TheDCSL (Dynamic Cell Structure) neural network from “Adaptive Neural Network Library” is selected as the identification tool. The problem is solved using Matlab Simulink tool. The program includes math model of aircraft motion along runway. The data accumulated from flight tests in real conditions were used to form samples for training of neural networks.. The math modeling results have been tested for convergence with experimental data.

This paper presents a novel Boundary-based approach in one-class classification that is inspired by support vector data description (SVDD). The SVDD is a popular kernel method which tries to fit a hypersphere around the target objects and of course more precise boundary is relied on selecting proper parameters for the kernel functions. Even with a flexible Gaussian kernel function, the SVDD could sometimes generate a loose decision boundary. Here we modify the structure of the SVDD by using a hyperellipse to specify the boundary of the target objects with more precision, in the input space. Due to the usage of a hyperellipse instead of a hypersphere as the decision boundary, we named it "Ellipse Support Vector Data Description" (ESVDD). We show that the ESVDD can generate a tighter data description in the kernel space as well as the input space. Furthermore the proposed algorithm boundaries on the contrary of SVDD boundaries are less influenced by change of the user defined parameters.

In this work SymbPar, a parallel co-evolutionary algorithm for automatically design the Radial Basis Function Networks, is proposed. It tries to solve the problem of huge execution time of Symbiotic_CHC_RBF, in which method are based. Thus, the main goal of SymbPar is to automatically design RBF neural networks reducing the computation cost and keeping good results with respect to the percentage of classification and net size. This new algorithm parallelizes the evaluation of the individuals using independent agents for every individual who should be evaluated, allowing to approach in a future bigger size problems reducing significantly the necessary time to obtain the results. SymbPar yields good results regarding the percentage of correct classified patterns and the size of nets, reducing drastically the execution time.

The elastic net algorithm, formulated by Durbin-Willshaw as an heuristic method and initially applied to solve the travelling salesman problem, can be used as a tool for data clustering in n-dimensional space. With the help of statistical mechanics it can be formulated as an deterministic annealing method in which a chain of nodes interacts at different temperatures with the data cloud. From a given temperature on the nodes are found to be the optimal centroid’s of fuzzy clusters, if the number of nodes is much smaller then number of data points.

We show in this contribution that for this temperature the centroid’s of hard clusters, defined by the nearest neighbor clusters of every node, are in the same position as the optimal centroid’s of the fuzzy clusters. This result can be used as a stopping criterion for the annealing process. The stopping temperature and the number and size of the hard clusters depend on the number of nodes in the chain.

Test were made with homogeneous and inhomogeneous artificial clusters in two dimensions.

Call centre technology requires the assignment of a large volume of incoming calls to agents with the required skills to process them. In order to perform the right assignment of call types to agents in a production environment, an efficient prediction of call arrivals is needed. In this paper, we introduce a prediction approach to incoming phone calls forecasting in a multi-skill call centre by modelling and learning the problem with an Improved Backpropagation Neural Network which have been compared with other methods. This model has been trained and analysed by using a real-time data flow in a production system from our call centre, and the results obtained outperform other forecasting methods. The reader can learn which forecasting method to use in a real-world application and some guidelines to better adapt an improved backpropagation neural network to his needs. A comparison among techniques and some statistics are shown to corroborate our results.

In this paper we propose an adaptive, self-learning system, which utilizes relational reinforcement learning (RRL), and apply it to a computer vision problem. A common problem in computer vision consists in the discrimination between similar objects which differ in salient features visible from distinct views only. Usually existing object recognition systems have to scan an object from a large number of views for a reliable discrimination. Optimization is achieved at most with heuristics to reduce the amount of computing time or to save storage space. We apply RRL in an appearance-based approach to the problem of discriminating similar objects, which are presented from arbitray views. We are able to rapidly learn scan paths for the objects and to reliably distinguish them from only a few recorded views. The appearance-based approach and the possibility to define states and actions of the RRL system with logical descriptions allow for a large reduction of the dimensionality of the state space and thus save storage and computing time.

This research effort has two main orientations. The first is the sensitivity analysis performance of the parameters that are considered to influence the problem of forest fires. This is conducted by the Pearson’s correlation analysis for each factor separately. The second target is the development of an intelligent fuzzy (Rule Based) Inference System that performs ranking of the Greek forest departments in accordance to their degree of forest fire risk. The system uses fuzzy algebra in order to categorize each forest department as “risky” or “non-risky”. The Rule Based system was built under the MATLAB Fuzzy integrated environment and the sensitivity analysis was conducted by using SPSS.

In this paper, we present a formulation of the learning problem that allows deterministic nonmonotone learning behaviour to be generated, i.e. the values of the error function are allowed to increase temporarily although learning behaviour is progressively improved. This is achieved by introducing a nonmonotone strategy on the error function values. We present four training algorithms which are equipped with nonmonotone strategy and investigate their performance in symbolic sequence processing problems. Experimental results show that introducing nonmonotone mechanism can improve traditional learning strategies and make them more effective in the sequence problems tested.

In this paper, we propose an indirect adaptive control scheme using Hopfield-based dynamic neural network for SISO nonlinear systems with external disturbances. Hopfield-based dynamic neural networks are used to obtain uncertain function estimations in an indirect adaptive controller, and a compensation controller is used to suppress the effect of approximation error and disturbance. The weights of Hopfield-based dynamic neural network are on-line tuned by the adaptive laws derived in the sense of Lyapunov, so that the stability of the closed-loop system can be guaranteed. In addition, the tracking error can be attenuated to a desired level by selecting some parameters adequately. Simulation results illustrate the applicability of the proposed control scheme. The designed parsimonious structure of the Hopfield-based dynamic neural network makes the practical implementation of the work in this paper much easier.

One challenging application for Artificial Neural Networks (ANN) would be to try and actually mimic the behaviour of the system that has inspired their creation as computational algorithms. That is to use ANN in order to simulate important brain functions. In this report we attempt to do so, by proposing a Neural Network computational model for simulating visual selective attention, which is a specific aspect of human attention. The internal operation of the model is based on recent neurophysiologic evidence emphasizing the importance of neural synchronization between different areas of the brain. Synchronization of neuronal activity has been shown to be involved in several fundamental functions in the brain especially in attention. We investigate this theory by applying in the model a correlation control module comprised by basic integrate and fire model neurons combined with coincidence detector neurons. Thus providing the ability to the model to capture the correlation between spike trains originating from endogenous or internal goals and spike trains generated by the saliency of a stimulus such as in tasks that involve top – down attention [1]. The theoretical structure of this model is based on the temporal correlation of neural activity as initially proposed by Niebur and Koch [9]. More specifically; visual stimuli are represented by the rate and temporal coding of spiking neurons. The rate is mainly based on the saliency of each stimuli (i.e. brightness intensity etc.) while the temporal correlation of neural activity plays a critical role in a later stage of processing were neural activity passes through the correlation control system and based on the correlation, the corresponding neural activity is either enhanced or suppressed. In this way, attended stimulus will cause an increase in the synchronization as well as additional reinforcement of the corresponding neural activity and therefore it will “win” a place in working memory. We have successfully tested the model by simulating behavioural data from the “attentional blink” paradigm [11].

This paper presents a spiking neural network simulator suitable for biologically plausible large neural networks, named DAMNED for “Distributed And Multi-threaded Neural Event-Driven”. The simulator is designed to run efficiently on a variety of hardware. DAMNED makes use of multi-threaded programming and non-blocking communications in order to optimize communications and computations overlap. This paper details the even-driven architecture of the simulator. Some original contributions are presented, such as the handling of a distributed virtual clock and an efficient circular event queue taking into account spike propagation delays. DAMNED is evaluated on a cluster of computers for networks from 10

3

to 10

5

neurons. Simulation and network creation speedups are presented. Finally, scalability is discussed regarding number of processors, network size and activity of the simulated NN.

This paper presents the application of Neural Networks for the prediction of the critical frequency

foF2

of the ionospheric F2 layer over Cyprus. This ionospheric characteristic (

foF2

) constitutes the most important parameter in HF (High Frequency) communications since it is used to derive the optimum operating frequency in HF links. The model is based on ionosonde measurements obtained over a period of 10 years. The developed model successfully captures the variability of the

foF2

parameter.

We describe in this paper advanced protocols for the discrimination and classification of neuronal spike waveforms within multichannel electrophysiological recordings. The programs are capable of detecting and classifying the spikes from multiple, simultaneously active neurons, even in situations where there is a high degree of spike waveform superposition on the recording channels. Sparse Decomposition (SD) approach was used to define the linearly independent signals underlying sensory information in cortical spike firing patterns. We have investigated motor cortex responses recorded during movement in freely moving rats to provide evidence for the relationship between these patterns and special behavioral task. Ensembles of neurons were simultaneously recorded in this during long periods of spontaneous behaviour. Waveforms provided from the neural activity were then processed and classified. Typically, most information correlated across neurons in the ensemble were concentrated in a small number of signals. This showed that these encoding vectors functioned as a feature detector capable of selectively predicting significant sensory or behavioural events. Thus it encoded global magnitude of ensemble activity, caused either by combined sensory inputs or intrinsic network activity.

SD on an overcomplete dictionary has recently attracted a lot of attention in the literature, because of its potential application in many different areas including Compressive Sensing (CS). SD approach is compared to the generative approach derived from the likelihood-based framework, in which each class is modeled by a known or unknown density function. The classification of electroencephalographic (EEG) waveforms present 2 main statistical issues: high dimensional data and signal representation.

When dealing with computationally expensive simulation codes or process measurement data, global surrogate modeling methods are firmly established as facilitators for design space exploration, sensitivity analysis, visualization and optimization. Popular surrogate model types include neural networks, support vector machines, and splines. In addition, the cost of each simulation mandates the use of active learning strategies where data points (simulations) are selected intelligently and incrementally. When applying surrogate models to multi-output systems, the hyperparameter optimization problem is typically formulated in a single objective way. The different response outputs are modeled separately by independent models. Instead, a multi-objective approach would benefit the domain expert by giving information about output correlation, facilitate the generation of diverse ensembles, and enable automatic model type selection for each output on the fly. This paper outlines a multi-objective approach to surrogate model generation including its application to two problems.

In this study we aim to define a mapping function that relates the general index value among a set of shares to the prices of individual shares. In more general terms this is problem of defining the relationship between multivariate data distributions and a specific source of variation within these distributions where the source of variation in question represents a quantity of interest related to a particular problem domain. In this respect we aim to learn a complex mapping function that can be used for mapping different values of the quantity of interest to typical novel samples of the distribution. In our investigation we compare the performance of standard neural network based methods like Multilayer Perceptrons (MLPs) and Radial Basis Functions (RBFs) as well as Mixture Density Networks (MDNs) and a latent variable method, the General Topographic Mapping (GTM). According to the results, MLPs and RBFs outperform MDNs and the GTM for this one-to-many mapping problem.

Multiple algorithms have been developed to correct user’s typing mistakes. However, an optimum solution is hardly identified among them. Moreover, these solutions rarely produce a single answer or share common results, and the answers may change with time and context. These have led this research to combine some distinct word correction algorithms to produce an optimal prediction based on database updates and neural network learning. In this paper, three distinct typing correction algorithms are integrated as a pilot research. Key factors including Time Change, Context Change and User Feedback are considered. Experimental results show that 57.50% Ranking First Hitting Rate (HR) with the samples of category one and a best Ranking First Hitting Rate of 74.69% within category four are achieved.

This paper is to introduce an application of Artificial Intelligence (AI) to Moving Picture Expert Group-4 (MPEG-4) video compression over IEEE.802.15.1 wireless communication in order to improve quality of picture. 2.4GHz Industrial, Scientific and Medical (ISM) frequency band is used for the IEEE 802.15.1 standard. Due to other wireless frequency devices sharing the same carrier, IEEE 802.15.1 can be affected by noise and interference. The noise and interference create difficulties to determine an accurate real-time transmission rate. MPEG-4 codec is an “object-oriented” compression system and demands a high bandwidth. It is therefore difficult to avoid excessive delay, image quality degradation or data loss during MPEG-4 video transmission over IEEE 802.15.1 standard. Two buffers have been implemented at the input of the IEEE 802.15.1 device and at the output respectively. These buffers are controlled by a rule based fuzzy logic controller at the input and a neural fuzzy controller at the output. These rules manipulate and supervise the flow of video over the IEEE 802.15.1 standard. The computer simulation results illustrate the comparison between a non-AI video transmission over IEEE 802.15.1 and the proposed design, confirming that the applications of intelligent technique improve the image quality and reduce the data loss.

Knowledge discovery consists of finding new knowledge from data bases where dimension, complexity or amount of data is prohibitively large for human observation alone. The Self Organizing Map (SOM) is a powerful neural network method for the analysis and visualization of high-dimensional data. The need for efficient data visualization and clustering is often faced, for instance, in the analysis, monitoring, fault detection, or prediction of various engineering plants. In this paper, the use of a SOM based method for prediction of disruptions in experimental devices for nuclear fusion is investigated. The choice of the SOM size is firstly faced, which heavily affects the performance of the mapping. Then, the ASDEX Upgrade Tokamak high dimensional operational space is mapped onto the 2-dimensional SOM, and, finally, the current process state and its history in time has been visualized as a trajectory on the map, in order to predict the safe or disruptive state of the plasma.

Supervised neural networks belong to the class of parameterised non-linear models whose optimum values are determined by a best fit procedure to training data. The problem of over-fitting can occur when more parameters than needed are included in the model. David MacKay’s Occam factor deploys a Bayesian approach to the investigation of how model order can be rationally restrained. This paper uses a case study to show how the Occam factor might be used to discriminate on model order and how it compares with similar indices e.g. Schwarz’s Bayesian information criterion and Akaike’s information criterion.

The Biofield Diagnostic System (BDS) is an adjunct breast cancer detection modality that uses recorded skin surface electropotentials for differentiating benign and malignant lesions. The main objective of this paper is to apply data mining techniques to two BDS clinical trial datasets to improve the disease detection accuracy. Both the datasets are pre-processed to remove outliers and are then used for feature selection. Wrapper and filter feature selection techniques are employed and the selected features are used for classification using supervised techniques like Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA), Support Vector Machines (SVM) and Back Propagation Neural Network (BPNN). It was observed that the LDA classifier using the feature subset selected via wrapper technique significantly improved the sensitivity and accuracy of one of the datasets. Also, the key observation is that this feature subset reduced the mild subjective interpretation associated with the current prediction methodology of the BDS device, thereby opening up new development avenues for the BDS device.

Epileptic seizures seem to result from an abnormal synchronization of different areas of the brain, as if a kind of recruitment occurred from a critical area towards other areas of the brain, until the brain itself can no longer bear the extent of this recruitment and triggers the seizure in order to reset this abnormal condition. In order to catch these recruitment phenomena, a technique based on entropy is introduced to study the synchronization of the electric activity of neuronal sources in the brain and tested over three EEG dataset from patients affected by partial epilepsy. Entropy showed a very steady spatial distribution and appeared linked to the region of seizure onset. Entropy mapping was compared with the standard power mapping that was much less stable and selective. A SOM based spatial clustering of entropy topography showed that the critical electrodes were coupled together long time before the seizure onset.

In recent years, Artificial Neural Networks have emerged as a powerful data driven approach of modelling and predicting complex physical and biological systems. The approach has several advantages over other traditional data driven approaches. Particularly among them are the facts that they can be used to model non-linear processes and that they do not require

’a priori’

understanding of the detailed mechanics of the processes involved. Because of the parallel nature of the data processing, the approach is also quite robust and insensitive to noise present in the data. Several riverflow applications of ANN’s are presented in this paper.

Earlier studies have shown that it is feasible to apply ANN to categorise user recorded trajectory data such that the travelled road types can be revealed. This approach can be used to automatically detect, classify and report new roads and other road related information to GIS map vendor based on a user travel behavior. However, the effect of trajectory variability caused by varying road traffic conditions for the proposed approach was not presented; this is addressed in this paper. The results show that the variability encapsulated within the dataset is important for this approach since it aids the categorisation of the road types. Overall the SDNN achieved categorisation result of about 71% for original dataset and 55% for the variability pruned dataset.

Security is one of the major challenges in the design and implementation of protocols for mobile

ad hoc

networks (MANETs). ‘Cooperation for corporate well-being’ is one of the major principles being followed in current research to formulate various security protocols. In such systems, nodes establish trust-based interactions based on their reputation which is determined by node activities in the past. In this paper we propose the use of a Radial Basis Function-Neural Network (RBF-NN) to estimate the reputation of nodes based on their internal attributes as opposed to their observed activity, e.g., packet traffic. This technique is conducive to prediction of the reputation of a node before it portrays any activities, for example, malicious activities that could be potentially predicted before they actually begin. This renders the technique favorable for application in trust-based MANET defense systems to enhance their performance. In this work we were able to achieve an average prediction performance of approximately 91% using an RBF-NN to predict the reputation of the nodes in the MANET.