Engineering Applications of Neural Networks

Frontmatter

Fuzzy Inference ANN Ensembles for Air Pollutants Modeling in a Major Urban Area: The Case of Athens

Abstract

All over the globe, major urban centers face a significant air pollution problem, which is becoming worse every year. This research effort aims to contribute towards real time monitoring of air quality, which is a target of great importance for people’s health. However, a serious obstacle is the high percentage of erroneous or missing data which is highly prolonged in many of the cases. To overcome this problem and due to the individuality of each residential area of Athens, separate local ANN had to be developed, capable of performing reliable interpolation of missing data vectors on an hourly basis. Also due to the need for hourly overall estimations of pollutants in the wider area of a major city, ANN ensembles were additionally developed by employing four existing methods and an innovative fuzzy inference approach.

Ilias Bougoudis, Lazaros Iliadis, Antonis Papaleonidas

Remarks on Computational Facial Expression Recognition from HOG Features Using Quaternion Multi-layer Neural Network

Abstract

Facial expression recognition is an important technology in human-computer interaction. This study investigates a method for facial expression recognition using quaternion neural networks. A multi-layer quaternion neural network that conducts its learning using a quaternion back-propagation algorithm is employed to design the facial expression recognition system. The input feature vector of the recognition system is composed of histograms of oriented gradients calculated from an input facial expression image, and the output vector of the quaternion neural network indicates the class of facial expressions such as happiness, anger, sadness, fear, disgust, surprise and neutral. Computational experimental results show the feasibility of the proposed method for recognising human facial expressions.

Kazuhiko Takahashi, Sae Takahashi, Yunduan Cui, Masafumi Hashimoto

Classification of Database by Using Parallelization of Algorithms Third Generation in a GPU

Abstract

This manuscript is focused on the efficiency analysis of Artificial Neural Networks (ANN) that belongs to the third generation, which are Spiking Neural Networks (SNN) and Support Vector Machine (SVM). The main issue of scientific community have been to improve the efficiency of ANN. So, we applied architecture GPU (Graphical Processing Unit) from NVIDIA model GeForce 9400M. On the other hand, the results of QP method for SVM depends on computational complexity of the algorithm, which is proportional to the volume and attributes of the data. Moreover, SNN was selected because it is a method that has not been explored fully. Despite the economic cost is very high in parallel programming, this is compensated with the large number of real applications such as clustering and pattern recognition. In the state of the art, nobody of authors has coded Quadratic Programming (QP) of SVM in a GPU. In case of SNN, it has been developed by using a specific software as MATLAB, FPGA or sequential circuits but it have never been coded in a GPU. Finally, it is necessary to reduce the grade of parallelization caused by limitations of hardware.

Israel Tabarez Paz, Neil Hernández Gress, Miguel González Mendoza

An Iterative Feature Filter for Sensor Timeseries in Pervasive Computing Applications

Abstract

The paper discusses an efficient feature selection approach for multivariate timeseries of heterogeneous sensor data within a pervasive computing scenario. An iterative filtering procedure is devised to reduce information redundancy measured in terms of timeseries cross-correlation. The algorithm is capable of identifying non-redundant sensor sources in an unsupervised fashion even in presence of a large proportion of noisy features. A comparative experimental analysis on real-world data from pervasive computing applications is provided, showing that the algorithm addresses major limitations of unsupervised filters in literature when dealing with sensor timeseries.

Davide Bacciu

Exploiting Evolution on UAV Control Rules for Spraying Pesticides on Crop Fields

Abstract

The application of chemicals in agricultural areas is of crucial importance for crop production. The use of aircrafts is becoming increasingly common in carrying out this task mainly because of their speed and effectiveness. Nonetheless, some factors may reduce the yield, or even cause damage, like areas not covered in the spraying process or overlapped spraying areas. Weather conditions add further complexity to the problem. Sets of control rules, to be employed in an autonomous Unmanned Aerial Vehicles (UAV), are very hard to develop and harder to fine-tune to each environment characteristics. Hence, a fine-tuning phase must involves the parameters of the algorithm, due to the mechanical characteristics of each UAV and also must take into account the type of crop being handled and the type of pesticide to be used. In this paper we present an evolutionary algorithm to fine-tune sets of control rules, to be employed in a simulated autonomous UAV. We describe the proposed architecture and investigations about changing in the evolutionary parameters. The results show that the proposed evolutionary method can fine-tune the parameters of the UAV control rules to support environment and weather changes in the simulated environment, encouraging the deployment of the system with real hardware.

Bruno S. Faiçal, Gustavo Pessin, Geraldo P. R. Filho, Gustavo Furquim, André C. P. L. F. de Carvalho, Jó Ueyama

Fuzzy-Logic Decision Fusion for Nonintrusive Early Detection of Driver Fatigue or Drowsiness

Abstract

Traffic accidents due to falling asleep at the wheel are a longstanding problem in many countries. This paper presents a novel solution based on fuzzy-logic decision fusion that prevents accidents by detecting driver fatigue or drowsiness early. The proposed method is based on analyzing and inferring about certain biological and behavioral measurements that enable detection of reduced alertness preceding driver-sleep onset. Because wakeful or sleep activity is reflected in several physiological conditions in human beings, such as cardiac, breathing, movement, and skin galvanic conductance, captured bioelectric signal features were extracted and fuzzy decision-fusion logic was tuned to make inferences about oncoming driver fatigue or drowsiness. The proposed method improves the performance by applying the fuzzy logic inference to fuse decisions from independent modules that infer about features measured on the sensed physiologic and/or behavioral information. The method reduces the complexity of the signal processing and of the pattern matching model. Tests have been executed on clinical and in field physiologic and behavioral data. A prototype based on a 32 bit microcontroller and a highly integrated analog front-end has been developed to support the in field tests.

Mario Malcangi

Neural Trade-Offs among Specialist and Generalist Neurons in Pattern Recognition

Abstract

The olfactory system of insects has two types of neurons based on the conditional response to odorants. Neurons that respond to a few odor classes are called specialists, while generalist neurons code for a wide range of input classes. The function of these neurons is intriguing. Specialist neurons are perhaps essential for odor discrimination, while generalist neurons may extract general properties of the odor space to be able to generalize to new odor spaces. Our goal is to shed light on this issue by analyzing the relevance of these neurons for pattern recognition purposes. The computational model is based on the olfactory system of insects. The model contains an approximation to the antennal lobe (AL) and mushroom body (MB) using a single-hidden-layer neural network. To determine the optimal balance between specialists and generalists we measure the classification error of the pattern recognition task. The mechanism to achieve the optimal balance is synaptic pruning to select the optimal synaptic configuration. The results show that specialists play an important role in odor classification, which is not observed for generalists. Furthermore, proper classification requires low neural activity in Kenyon cells, KC, which is consistent with the sparseness condition observed in MB neurons. Moreover, we also observe that the model is robust against noise to input patterns showing better resilience for low connection probabilities between AL and MB.

Aarón Montero, Ramón Huerta, Francisco B. Rodríguez

Classification of Events in Switch Machines Using Bayes, Fuzzy Logic System and Neural Network

Abstract

The Railroad Switch denotes a set of parts in concordance with two lines in order to allow the passage of railway vehicles from one line to another. The Switch Machines are equipments used for handling Railroad Switches. Among all possible defects that can occur in a electromechanical Switch Machine, this work emphasizes the three main ones: the defect related to lack of lubrication, the defect related to lack of adjustment and the defect related to some component of Switch Machine. In addition, this work includes the normal operation of these equipments. The proposal in question makes use of real data provided by a company of the railway sector. Observing these four events, it is proposed the use of Signal Processing and Computational Intelligence techniques to classify the mentioned events, generating benefits that will be discussed and thus providing solutions for the company to reach the top of operational excellence.

Eduardo Aguiar, Fernando Nogueira, Renan Amaral, Diego Fabri, Sérgio Rossignoli, José Geraldo Ferreira, Marley Vellasco, Ricardo Tanscheit, Moisés Ribeiro, Pedro Vellasco

An Accurate Flood Forecasting Model Using Wireless Sensor Networks and Chaos Theory: A Case Study with Real WSN Deployment in Brazil

Abstract

Monitoring natural environments is a challenging task on account of their hostile features. The use of wireless sensor networks (WSN) for data collection is a viable method since these domains lack any infrastructure. Further studies are required to handle the data collected to provide a better modeling of behavior and make it possible to forecast impending disasters. These factors have led to this paper which conducts an analysis of the use of data gathered from urban rivers to forecast future flooding with a view to reducing the damage they cause. The data were collected by means of a WSN in São Carlos, São Paulo State, Brazil and were handled by employing the Immersion Theorem. The WSN were deployed by our group in the city of São Carlos due to numerous problems with floods. After discovering the data interdependence, artificial neural networks were employed to establish more accurate forecasting models.

Gustavo Furquim, Rodrigo Mello, Gustavo Pessin, Bruno S. Faiçal, Eduardo M. Mendiondo, Jó Ueyama

Regenerative Braking Control Strategy for Hybrid and Electric Vehicles Using Artificial Neural Networks

Abstract

One of the fundamental advantages of hybrid and electric vehicles compared to conventional vehicles is the regenerative braking mechanism. Some portion of the kinetic energy of the vehicle can be recovered during regenerative braking by using the electric drive system as a generator with the appropriate control strategy. The control requires distribution of the brake forces between front and rear axles of the vehicle and also between regenerative braking and frictional braking. In this paper, we propose solving the optimal brake force distribution problem using an Artificial Neural Network based methodology in order to maximize the available energy for recovery while following the rules for stability. Using the proposed approach, we find that for urban driving pattern, UDDS, up to 37 % of the total energy demand can be recovered. Then we compare the amount of recovered energy for different driving cycles and show that aggressive driving reduces recoverable energy up to 7%. An increase in the energy recovery rate directly translates into improvements in fuel economy and reductions in emissions.

Sanketh S. Shetty, Orkun Karabasoglu

Automatic Screening and Classification of Diabetic Retinopathy Fundus Images

Abstract

Eye screening is essential for the early detection and treatment of the diabetic retinopathy. This paper presents an automatic screening system for diabetic retinopathy to be used in the field of retinal ophthalmology. The paper first explores the existing systems and applications related to diabetic retinopathy screening and detection methods that have been previously reported in the literature. The proposed ophthalmic decision support system consists of an automatic acquisition, screening and classification of diabetic retinopathy fundus images, which will assist in the detection and management of the diabetic retinopathy. The developed system contains four main parts, namely the image acquisition, the image preprocessing, the feature extraction, and the classification by using several machine learning techniques.

Sarni Suhaila Rahim, Vasile Palade, James Shuttleworth, Chrisina Jayne

Brain Neural Data Analysis Using Machine Learning Feature Selection and Classification Methods

Abstract

The Electroencephalogram (EEG) is a powerful instrument to collect vast quantities of data about human brain activity. A typical EEG experiment can produce a two-dimensional data matrix related to the human neuronal activity every millisecond, projected on the head surface at a spatial resolution of a few centimeters. As in other modern empirical sciences, the EEG instrumentation has led to a flood of data and a corresponding need for new data analysis methods. This paper summarizes the results of applying supervised machine learning (ML) methods to the problem of classifying emotional states of human subjects based on EEG. In particular, we compare six ML algorithms to distinguish event-related potentials, associated with the processing of different emotional valences, collected while subjects were viewing high arousal images with positive or negative emotional content. 98% inter-subject classification accuracy based on the majority of votes between all classifiers is the main achievement of this paper, which outperforms previous published results.

Lachezar Bozhkov, Petia Georgieva, Roumen Trifonov

Application of Neural Networks Solar Radiation Prediction for Hybrid Renewable Energy Systems

Abstract

In this paper a Recurrent Neural Network (RNN) for solar radiation prediction is proposed for the enhancement of the Power Management Strategies (PMSs) of Hybrid Renewable Energy Systems (HYRES). The presented RNN can offer both daily and hourly prediction concerning solar irradiation forecasting. As a result, the proposed model can be used to predict the Photovoltaic Systems output of the HYRES and provide valuable feedback for PMSs of the understudy autonomous system. To do so a flexible network based design of the HYRES is used and, moreover, applied to a specific system located on Olvio, near Xanthi, Greece, as part of SYSTEMS SUNLIGHT S.A. facilities. As a result, the RNN after training with meteorological data of the aforementioned area is applied to the specific HYRES and successfully manages to enhance and optimize its PMS based on the provided solar radiation prediction.

P. Chatziagorakis, C. Elmasides, G. Ch. Sirakoulis, I. Karafyllidis, I. Andreadis, N. Georgoulas, D. Giaouris, A. I. Papadopoulos, C. Ziogou, D. Ipsakis, S. Papadopoulou, P. Seferlis, F. Stergiopoulos, S. Voutetakis

A New User Similarity Computation Method for Collaborative Filtering Using Artificial Neural Network

Abstract

A User-User Collaborative Filtering (CF) algorithm predicts the rating of a particular item for a given user based on the judgment of other users, who are similar to the given user. Hence, measuring similarity between two users turns out to be a crucial and challenging task as the similarity function is the core component of the item rating prediction function for a particular user. In this paper, we investigate the effectiveness of a multilayer feed-forward artificial neural network as a similarity measurement function. We model similarity between two users as a function that consists of a set of adaptive weights and attempt to train a neural network to optimize the weights. Specifically, our contribution lies in designing an error function for the neural network, which optimizes the network and sets weights in such a way that enables the neural network to produce a reasonable similarity value between two users as its output. Through experimentation on Movielens dataset, we conclude that neural network, as a similarity function, gains more accuracy and coverage compared to the Genetic Algorithm (GA) based similarity architecture proposed by Bobadilla et al.

Noman Bin Mannan, Sheikh Muhammad Sarwar, Najeeb Elahi

Probabilistic Models Based Intrusion Detection Using Sequence Characteristics in Control System Communication

Abstract

The importance of cyber security has increased with the networked and highly complex structure of computer systems, and the increased value of information. In this paper, we compare Conditional Random Field based intrusion detection with the other probabilistic models based intrusion detection. Theses methods uses the sequence characteristics of network traffic in the control system communication. The learning only utilizes normal data, assuming that there is no prior knowledge on attacks in the system. We applied these two probabilistic models to intrusion detection in DARPA data and an experimental control system network, and compared the differences in the performance.

Takashi Onoda

Compressive ELM: Improved Models through Exploiting Time-Accuracy Trade-Offs

Abstract

In the training of neural networks, there often exists a trade-off between the time spent optimizing the model under investigation, and its final performance. Ideally, an optimization algorithm finds the model that has best test accuracy from the hypothesis space as fast as possible, and this model is efficient to evaluate at test time as well. However, in practice, there exists a trade-off between training time, testing time and testing accuracy, and the optimal trade-off depends on the user’s requirements. This paper proposes the Compressive Extreme Learning Machine, which allows for a time-accuracy trade-off by training the model in a reduced space. Experiments indicate that this trade-off is efficient in the sense that on average more time can be saved than accuracy lost. Therefore, it provides a mechanism that can yield better models in less time.

Mark van Heeswijk, Amaury Lendasse, Yoan Miche

Detecting Port Scans against Mobile Devices with Neural Networks and Decision Trees

Abstract

Recently, mobile devices such as smartphones and tablets have emerged as one of the most popular forms of communication. This trend raises the question about the security of the private data and communication of the people using those devices. With increased computational resources and versatility the number of security threats on such devices is growing rapidly. Therefore, it is vital for security specialists to find adequate anti-measures against the threats. Machine Learning approaches with their ability to learn from and adapt to their environments provide a promising approach to modelling and protecting against security threats on mobile devices. This paper presents a comparative study and implementation of Decision Trees and Neural Network models for the detection of port scanning showing the differences between the responses on a desktop platform and a mobile device and the ability of the Neural Network model to adapt to the different environment and computational resource available on a mobile platform.

Christo Panchev, Petar Dobrev, James Nicholson

Categorization and Construction of Rule Based Systems

Abstract

Expert systems have been increasingly popular for commercial importance. A rule based system is a special type of an expert system, which consists of a set of ‘if-then’ rules and can be applied as a decision support system in many areas such as healthcare, transportation and security. Rule based systems can be constructed based on both expert knowledge and data. This paper aims to introduce the theory of rule based systems especially on categorization and construction of such systems from a conceptual point of view. This paper also introduces rule based systems for classification tasks in detail.

Han Liu, Alexander Gegov, Frederic Stahl

Tiling of Satellite Images to Capture an Island Object

Abstract

This study proposes a novel tiling approach to capture an image of an entire object. Multi-spectral and multi-temporal satellite images are obtained a priori, and these individual image pieces can then be joined together at a later date to form an image of the entire object. The effectiveness of the proposed technique has been studied by tiling partially overlapping satellite mosaic images of the Island of Cyprus. The images were captured by the recently-launched LandSat-8 satellite.

Ahmet Sayar, Süleyman Eken, Umit Mert

Learning User Models in Multi-criteria Recommender Systems

Abstract

Whenever people have to choose seeing or buying an item among many others, they are based on their own ways of evaluating its characteristics (criteria) to understand better which one of the items meets their needs. Based on this argument, in this paper we develop personalized models for each user, according to their ratings on specific criteria, and we use them in multi-criteria recommender systems. We assume the overall ranking, which indicates users’ final decision, is closely related to their given value in each criterion separately. We compare user models created using neural networks and linear regression and we show, as expected from the implicit nonlinear combination of criteria, that neural networks based models achieve better performance. In continue we investigate several different approaches of collaborative filtering and matrix factorization to make recommendations. For this purpose we estimate users’ similarity by comparing their models. Experimental justification is obtained using the Yahoo! Movie dataset.

Marilena Agathokleous, Nicolas Tsapatsoulis

Fault Classification System for Computer Networks Using Fuzzy Probabilistic Neural Network Classifier (FPNNC)

Abstract

Over the last decade, the world has witnessed the rapid development of networking applications of different kinds, and network domains have become more and more advanced regarding with their level of heterogeneity, complexity and the size. Some obstacles such as availability, flexibility and insufficient scalability have affected the existing centralized network management systems, as networks become more distributed. In this work a Fuzzy Probabilistic Neural Network Classifier (FPNNC) is proposed, comprising a hybrid fault classification algorithm based on Fuzzy Cluster Mean (FCM) with Probabilistic Neural Network (PNN) to classify the detected fault datasets. These results will assist network administrators with a highly effective tool to classify faults that occur in computer network systems, enabling them to take well-informed decisions pertaining to security, faults and performance.

Karwan Qader, Mo Adda

Estimation of the Electric Field across Medium Voltage Surge Arresters Using Artificial Neural Networks

Abstract

Artificial neural networks (ANNs) are addressed in order to estimate the electric field across medium voltage surge arresters, information which is very useful for diagnostic tests and design procedures. Actual input and output data collected from hundreds of measurements carried out in the High Voltage Laboratory of the National Technical University of Athens (NTUA) are used in the training, validation and testing process. The developed ANN method can be used by laboratories and manufacturing/retail companies dealing with medium voltage surge arresters which either face a lack of suitable measuring equipment or want to compare/verify their own measurements.

Lambros Ekonomou, Christos A. Christodoulou, Valeri Mladenov

Decoding Hand Trajectory from Primary Motor Cortex ECoG Using Time Delay Neural Network

Abstract

Brain-machines - also termed neural prostheses, could potentially increase substantially the quality of life for people suffering from motor disorders or even brain palsy. In this paper we investigate the non-stationary continuous decoding problem associated to the rat’s hand position. To this aim, intracortical data (also named ECoG for electrocorticogram) are processed in successive stages: spike detection, spike sorting, and intention extraction from the firing rate signal.

The two important questions to answer in our experiment are (i) is it realistic to link time events from the primary motor cortex with some time-delay mapping tool and are some inputs more suitable for this mapping (ii) shall we consider separated channels or a special representation based on multidimensional statistics. We propose our own answers to these questions and demonstrate that a nonlinear representation might be appropriate in a number of situations.

Abdessalam Kifouche, Vincent Vigneron, Mohammad B. Shamsollahi, Abderrezak Guessoum

Backmatter