Engineering Applications of Neural Networks

A special phenomenological model of induction motor speed estimation in the drive system is derived. The basis of approximation is calculated from the system mathematical model as a set of transformed, easily measured input variables. It is demonstrated analytically that the set suits well to speed approximation if the approximated signal is a constant or changes linearly. It is then demonstrated numerically that this set is also quite effective under non-zero jerk. Such a system could easily be implemented by widely experienced feedforward neural networks. Illustrative examples and simulation results are attached.

The behavior of Steam Generators Water Level from nuclear power plants is highly nonlinear. Its parameters change when facing different operational conditions. Simulating this system can be very useful to train people involved in the real plant operation. However, in order to simulate this system the identification process must be performed. Echo State Networks are a special type of Recurrent Neural Networks that are well suited to nonlinear dynamic systems identification, with the advantage of having a simpler and faster training algorithm than conventional Recurrent Neural Networks. Echo State Networks have an additional advantage over other conventional methods of dynamic systems identification, since it is not necessary to specify the model’s structure. However, some other parameters of the Echo State Network must be tuned in order to attain its best performance. Therefore, this study proposes the use of an Echo State Network, automatically tuned by Genetic Algorithms, to a closed loop identification of a nuclear steam generator water level process. The results obtained demonstrate that the proposed Echo State Networks can correctly model dynamical nonlinear system in a large range of operation.

Grinding is a key process in high-added value sectors due to its capacity for producing high surface quality and high precision parts. One of the most important parameters that indicate the grinding quality is the surface roughness (

R

a

). Analytical models developed to predict surface finish are not easy to apply in the industry. Therefore, many researchers have made use of Artificial Neural Networks. However, all the approaches provide a particular solution for a wheel-workpiece pair. Besides, these solutions do not give surface roughness values related to the grinding wheel status. Therefore, in this work the prediction of the surface roughness (

R

a

) evolution based on Recurrent Neural Networks is presented with the capability to generalize to new grinding wheels and conditions. Results show excellent prediction of the surface finish evolution. The absolute maximum error is below 0.49µm, being the average error around 0.32µm.

The reliability analysis of complex structural systems requires utilization of approximation methods for calculation of reliability measures with the view of reduction of computational efforts to an acceptable level. The aim is to replace the original limit state function by an approximation, the so-called response surface, whose function values can be computed more easily. In the paper, an artificial neural network based response surface method in the combination with the small-sample simulation technique is introduced. An artificial neural network is used as a surrogate model for approximation of original limit state function. Efficiency is emphasized by utilization of the stratified simulation for the selection of neural network training set elements. The proposed method is employed for reliability assessment of post-tensioned composite bridge. Response surface obtained is independent of the type of distribution or correlations among the basic variables.

In the era of Big Data in Life Sciences, efficient processing and analysis of vast amounts of sequence data is becoming an ever daunting challenge. Among such analyses, sequence alignment is one of the most commonly used procedures, as it provides useful insights on the functionality and relationship of the involved entities. Sequence alignment is one of the most common computational bottlenecks in several bioinformatics workflows. We have designed and implemented a time-efficient distributed modular application for sequence alignment, phylogenetic profiling and clustering of protein sequences, by utilizing the European Grid Infrastructure. The optimal utilization of the Grid with regards to the respective modules, allowed us to achieve significant speedups to the order of 1400%.

This work develops data interpretation by elastic nets based on statistical mechanics, for the detection of clusters of data structures in an n-dimensional space. The problem is to find patterns of activity in genes of E. coli, which are stressed by different carbon sources that allow them to activate various expressiveness. The results of the study show that for a string node using distinct temperatures distinct phase transitions are shown. As result of phase transitions, changed centroids of respective groupings are observed. At each node, different behaviors revealed by carbon sources applied to genes are observed.

Fuzzy image processing was proven to help improve the image quality for both medical and non-medical images. This paper presents a fuzzy techniques-based eye screening system for the detection of one of the most important visible signs of diabetic retinopathy; microaneurysms, small red spot on the retina with sharp margins. The proposed ophthalmic decision support system consists of an automatic acquisition, screening and classification of eye fundus images, which can assist in the diagnosis of the diabetic retinopathy. The developed system contains four main parts, namely the image acquisition, the image preprocessing with fuzzy techniques, the microaneurysms localisation and detection, and finally the image classification. The fuzzy image processing approach provides better results in the detection of microaneurysms.

Endotracheal intubation is a complex medical procedure in which a ventilating tube is inserted into the human trachea. Improper positioning carries potentially fatal consequences and therefore confirmation of correct positioning is mandatory. In this paper we report the results of using a neural network-based image classification system for endotracheal tube position confirmation. The proposed system comprises a miniature complementary metal oxide silicon sensor (CMOS) attached to the tip of a semi rigid stylet and connected to a digital signal processor (DSP) with an integrated video acquisition component. Video signals are acquired and processed by a confirmation algorithm implemented on the processor. The performance of the proposed algorithm was evaluated using two datasets: a dataset of 250 images of the upper airways. The results, obtained using a leave-one-case-out method, show that the system correctly classified 240 out of 250 (96.0%).

Climate change combined with the increase of extreme weather phenomena, has significantly influenced marine ecosystems, resulting in water overheating, increase of sea level and rising of the acidity of surface waters. The potential impacts in the biodiversity of sensitive ecosystems (such as Mediterranean sea) are obvious. Many organisms are under extinction, whereas other dangerous invasive species are multiplied and thus they are destroying the ecological equilibrium. This research paper presents the development of a sophisticated, fast and accurate Food Pathogen Detection (FPD) system, which uses the biologically inspired Artificial Intelligence algorithm of Extreme Learning Machines. The aim is the automated identification and control of the extremely dangerous for human health invasive fish species “Lagocephalus Sceleratus”. The matching is achieved through extensive comparisons of protein and DNA sequences, known also as DNA barcodes following an ensemble learning approach.

Nitrogen oxides (NO

x

) are one of the main pollutants produced by combustion processes. New European emission regulations (IED) extent emission monitoring requirements to smaller boilers. Heating grid operators may have a notable number of such boilers and therefore appreciate affordable monitoring solutions. This paper studies several types of regression models for estimating NO

x

emissions in natural gas fired boilers. The objective is to predict the emissions utilising the existing process measurements for monitoring, without an external NO

x

analyser. The performance of linear regression is compared with three nonlinear methods: multilayer perceptron, support vector regression and fuzzy inference system. The focus is on generalisation ability. The results on the two boilers in the study suggest that linear regression and multilayer perceptron network outperform the others in predicting with new, unseen data.

The studied inverse problem is determination of ionic composition of inorganic salts (concentrations of up to 10 ions) in multi-component water solutions by their Raman spectra. The regression problem was solved in two ways: 1) by a multilayer perceptron trained on the large dataset, composed of spectra of all possible mixing options of ions in water; 2) dividing the data set into compact clusters and creating regression models for each cluster separately. Within the first approach, we used supervised training of neural network, achieving good results. Unfortunately, this method isn’t stable enough; the results depend on data subdivision into training, test, and out-of-sample sets. In the second approach, we used algorithms of unsupervised learning for data clustering: Kohonen networks, k-means, k-medoids and hierarchical clustering, and built partial least squares regression models on the small datasets of each cluster. Both approaches and their results are discussed in this paper.

Soil Total Nitrogen (TN) can be measured with on-line visible and near infrared spectroscopy (vis-NIRS), whose calibration method may considerably affect the measurement accuracy. The aim of this study was to compare Principal Component Regression (PCR) with Supervised Self organizing Maps (SSOM) for the calibration of a visible and near infrared (vis-NIR) spectrophotometer for the on-line measurement of TN in a field in a German farm. A mobile, fiber type, vis-NIR spectrophotometer (AgroSpec from tec5 Technology for Spectroscopy, Germany) mounted in an on-line sensor platform, comprising of measurement range of 305–2200 nm was utilized so as to obtain soil spectra in diffuse reflectance mode. Both PCR and SSOM calibration models of TN were validated with independent validation sets. The obtain root mean square error (rmse) was equal to 0.0313.The component maps of SSOM allow for a visualization of different correlations between spectral components and nitrogen content.

We explore four local learning versions of regularization networks. While global learning algorithms create a global model for all testing points, the local learning algorithms use neighborhoods to learn local parameters and create on the fly a local model specifically designed for any particular testing point. This approach delivers breakthrough performance in many application domains. Usually however the computational overhead is substantial, and in some cases prohibited. For speeding up the online predictions we exploit both multithreaded parallel implementations as well as interplay between locally optimized parameters and globally optimized parameters. The multithreaded local learning regularization neural networks are implemented with OpenMP. The accuracy of the algorithms is tested against several benchmark datasets. The parallel efficiency and speedup is evaluated on a multi-core system.

Semi-supervised classification methods are based on the use of unlabeled data in combination with a smaller set of labeled examples, in order to increase the classification rate compared with the supervised methods, in which the total training is executed only by the usage of labeled data. In this work, a self-train Logitboost algorithm is presented. The self-train process improves the results by using the accurate class probabilities for which the Logitboost regression tree model is more confident at the unlabeled instances. We performed a comparison with other well-known semi-supervised classification methods on standard benchmark datasets and the presented technique had better accuracy in most cases.

Supervised learning algorithm for Spiking Neural Networks (SNN) based on Remote Supervised Method (ReSuMe) uses spike timing dependent plasticity (STDP) to adjust the synaptic weights. In this work, we present an optimal network configuration amenable to digital CMOS implementation and show that just 5 bits of resolution for the synaptic weights is sufficient to achieve fast convergence. We estimate that the implementation of this optimal network architecture in

$$65\,$$

nm and a futuristic

$$10\,$$

nm digital CMOS could result in systems with close to 0.85 and 30 Million Synaptic Updates Per Second (MSUPS)/Watt.

In this paper, a new classification method based on k-Nearest Neighbor (kNN) lazy classifier is proposed. This method leverages the clustering concept to reduce the size of the training set in kNN classifier and also in order to enhance its performance in terms of time complexity. The new approach is called Modified Nearest Neighbor Classifier Based on Clustering (MNNCBC). Inspiring the traditional lazy k-NN algorithm, the main idea is to classify a test instance based on the tags of its k nearest neighbors. In MNNCBC, the training set is first grouped into a small number of partitions. By obtaining a number of partitions employing several runnings of a simple clustering algorithm, MNNCBC algorithm extracts a large number of clusters out of those partitions. Then, a label is assigned to the center of each cluster produced in the previous step. The assignment is determined with use of the majority vote mechanism between the class labels of the patterns in each cluster. MNNCBC algorithm iteratively inserts a cluster into a pool of the selected clusters that are considered as the training set of the final 1-NN classifier as long as the accuracy of 1-NN classifier over a set of patterns included the training set and the validation set improves. The selected set of the most accurate clusters are considered as the training set of proposed 1-NN classifier. After that, the class label of a new test sample is determined according to the class label of the nearest cluster center. While kNN lazy classifier is computationally expensive, MNNCBC classifier reduces its computational complexity by a multiplier of 1/k. So MNNCBC classifier is about k times faster than kNN classifier. MNNCBC is evaluated on some real datasets from UCI repository. Empirical results show that MNNCBC has an excellent improvement in terms of both accuracy and time complexity in comparison with kNN classifier.

In big data analytics and machine learning applications on telecom network measurement data, accuracy of findings during the analysis phase greatly depends on the quality of the training data set. If the training data set contains data from Network Elements (NEs) with high number of failures and high failure rates, such behavior will be assumed as normal. As a result, the analysis phase will fail to detect NEs with such behavior. High failure ratios have traditionally been considered as signs of faults in NEs. Operators use well-known Key Performance Indicators (KPIs), such as, e.g., Drop Call Ratio and Handover failure ratio to identify misbehaving NEs. The main problem with these KPIs based on failure ratios is their unstable nature. This paper proposes a method of measuring the significance of failures and its use in training set filtering.

An important property of the competitive neural models for data clustering is autonomous discovery of the data structure without a need of a priori knowledge. Growing Neural Gas (GNG) is one of the commonly used incremental clustering models that aims at preserving the topology and the distribution of the input data. Keeping the data distribution unchanged has already been recognized as a problem leading to bias sampling of input data. This is undesired for the use cases such as mobile network management and troubleshooting where is important to capture all the relevant network states uniformly. In this paper we propose a novel incremental clustering approach called Fixed Resolution GNG (FRGNG) that keeps the input data representation at the fixed resolution avoiding the oversampling and undersampling problems of original GNG algorithm. Furthermore, FRGNG introduces a native stopping criteria by terminating the run once the input data is represented with the desired fixed resolution. Additionally, the FRGNG has a potential of the algorithm acceleration which is especially important when large input data set is applied. We apply the FRGNG model to analyze the mobile network performance data and evaluate its benefits compared to GNG approach.

In this study, we investigate the potential of using a novel neural-fuzzy network to predict the coastal erosion from bathymetry field data taken from the Eresos beach located at the SW coastline of Lesvos island, Greece. The bathymetry data were collected using specialized experimental devices deployed in the study area. To elaborate the data and predict the coastal erosion, we have developed a neural-fuzzy network implemented in three phases. The first phase defines the rule antecedent parts and includes three layers of hidden nodes. The second phase employs truncated Hermite polynomial series to form the rule consequent parts. Finally, the third phase intertwines the information coming from the above phases and infers the network’s output. The performance of the network is compared to other two relative approaches. The simulation study shows that the network achieves an accurate behavior, while outperforming the other methods.

RBF neural networks are an efficient tool for acquisition and representation of functional relations reflected in empirical data. The interpretation of acquired knowledge is, however, generally difficult because the knowledge is encoded into values of the parameters of the network. Contrary to neural networks, fuzzy systems allow a more convenient interpretation of the stored knowledge in the form of IF-THEN rules. This paper contributes to the fusion of these two concepts. Namely, we show that a RBF neural network can be interpreted as the radial fuzzy system. The proposed approach is based on the study of conjunctive and implicative representations of the rule base in radial fuzzy systems. We present conditions under which both representations are computationally close and, as the consequence, a reasonable syntactic interpretation of RBF neural networks can be introduced.

Improving automatic speech recognition systems is one of the hottest topics in speech-signal processing, especially if such systems are to operate in noisy environments. This paper proposes a multimodal evolutionary neuro-fuzzy approach to developing an automatic speech-recognition system. To make inferences at the decision stage about audiovisual information for speech-to-text conversion, the EFuNN paradigm was applied. Two independent feature extractors were developed, one for the speech phonetics (speech listening) and the other for the speech visemics (lip reading). The EFuNN network has been trained to fuse decisions on audio and decisions on video. This soft computing approach proved robust in harsh conditions and, at the same time, less complex than hard computing, pattern-matching methods. Preliminary experiments confirm the reliability of the proposed method for developing a robust, automatic, speech-recognition system.

Since the lateral superior olive (LSO) is the first nucleus in the auditory pathway where binaural inputs converge, it is thought to be involved in azimuthal localization of sounds by calculating the interaural level difference (ILD). The electronic LSO can be used for azimuthal localization in robotics. Thus, in this paper we demonstrate the design, fabrication and test results from a silicon chip which performs azimuthal localization based on the Reed and Blum’s model of the population response of the LSO in brain.

A desirable feature in smart cameras is the ability to autonomously detect any tampering event/attack that would prevent a clear view over the monitored scene. No matter whether tampering is due to atmospheric phenomena (e.g., few rain drops over the camera lens) or to malicious attacks (e.g., occlusions or device displacements), these have to be promptly detected to possibly activate countermeasures. Tampering detection is particularly challenging in battery-powered cameras, where it is not possible to acquire images at full-speed frame-rates, nor use sophisticated image-analysis algorithms.

We here introduce a tampering-detection algorithm specifically designed for low-power smart cameras. The algorithm leverages very simple indicators that are then monitored by an outlier-detection scheme: any frame yielding an outlier is detected as tampered. Core of the algorithm is the partitioning of the scene into adaptively defined regions, that are preliminarily defined by segmenting the image during the algorithm-configuration phase, and which shows to improve the detection of camera displacements. Experiments show that the proposed algorithm can successfully operate on sequences acquired at very low-frame rate, such as one frame every minute, with a very small computational complexity.

In this paper we propose the use of several feature extraction methods, which have been shown before to perform well for object recognition, for recognizing handwritten characters. These methods are the histogram of oriented gradients (HOG), a bag of visual words using pixel intensity information (BOW), and a bag of visual words using extracted HOG features (HOG-BOW). These feature extraction algorithms are compared to other well-known techniques: principal component analysis, the discrete cosine transform, and the direct use of pixel intensities. The extracted features are given to three different types of support vector machines for classification, namely a linear SVM, an SVM with the RBF kernel, and a linear SVM using L2-regularization. We have evaluated the six different feature descriptors and three SVM classifiers on three different handwritten character datasets: Bangla, Odia and MNIST. The results show that the HOG-BOW, BOW and HOG method significantly outperform the other methods. The HOG-BOW method performs best with the L2-regularized SVM and obtains very high recognition accuracies on all three datasets.

Emotions are important and meaningful aspects of human behaviour. Analyzing facial expressions and recognizing their emotional state is a challenging task with wide ranging applications. In this paper, we present an emotion recognition system, which recognizes basic emotional states in facial expressions. Initially, it detects human faces in images using the Viola-Jones algorithm. Then, it locates and measures characteristics of specific regions of the facial expression such as eyes, eyebrows and mouth, and extracts proper geometrical characteristics form each region. These extracted features represent the facial expression and based on them a classification schema, which consists of a Support Vector Machine (SVM) and a Multilayer Perceptron Neural Network (MLPNN), recognizes each expression’s emotional content. The classification schema initially recognizes whether the expression is emotional and then recognizes the specific emotions conveyed. The evaluation conducted on JAFFE and Kohn Kanade databases, revealed very encouraging results.

In this work, a method is presented for the continuous estimation of pain intensity based on fusion of bio-physiological and video features. The focus of the paper is to analyse which modalities and feature sets are suited best for the task of recognizing pain levels in a person-independent setting. A large set of features is extracted from the available bio-physiological channels (ECG, EMG and skin conductivity) and the video stream. Experimental validation demonstrates which modalities contribute the most to a robust prediction and the effects when combining them to improve the continuous estimation given unseen persons.

The present article deals with sexual cyberbullying, a serious subject that has gained significant attention throughout recent years of emerging social media platforms. The detection of sexual predation is one of the most important and challenging tasks in our days. Using real-world data, we follow a time series modeling approach, in which predator’s posts (i.e. questions) are associated with numeric labels, according to the style of the attack (e.g. attempts for physical approach, grooming, retrieval of personal information, etc.). Upon modeling the domain as time series, in order to allow for forecasting the severity of a future question of a predator (i.e. the class label), a sliding window method was adopted. Two well-known methods that have been traditionally applied in time series problems, namely Support Vector Machines and Neural Networks, were utilized for forecasting. Simultaneously, since text processing is almost certain to derive a large number of input features, an additional method for reducing dimensionality of the original dataset was applied, implemented with Singular Value Decomposition. We demonstrate that the use of SVM classifier is more appropriate for our data and we show that it is able to provide accurate results that surpass current state-of-the-art outcomes, by using both the original feature set as well as the reduced SVD dimensions.

This paper presents a new method to deal with classification of imbalanced data. A Bayesian ensemble of neural network classifiers is proposed. Several individual neural classifiers are trained to minimize a Bayesian cost function with different decision costs, thus working at different points of the Receiver Operating Characteristic (ROC). Decisions of the set of individual neural classifiers are fused using a Bayesian rule that introduces a “balancing” parameter allowing to compensate the imbalance of available data.

This paper examines the correlation of implicit and explicit user behaviour indicators in a task specific domain. An experiment was conducted and data was collected from 77 undergraduate students of Computer science. Users’ implicit features and explicit ratings of document relevance were captured and logged through a plugin in Firefox browser. A number of implicit indicators were correlated with user explicit ratings and a predictive function model was derived. Classification algorithms were also used to classify documents according to how relevant they are to the current task. It was found that implicit indicators could be used successfully to predict the user rating. These findings can be utilised in building individual and group profile for users of a context-based recommender system.

Microarray data classification is a critical challenge for computational techniques due to its inherent characteristics, mainly small sample size and high dimension of the input space. For this type of data two-class classification techniques have been widely applied while one-class learning is considered as a promising approach. In this paper, we study the suitability of employing the one-class classification for microarray datasets while the role played by feature selection is analyzed. The superiority of this approach is demonstrated by comparison with the classical approach, with two classes, on different benchmark data sets.

Intuitionistic fuzzy inference systems (IFISs) incorporate imprecision in the construction of membership functions present in fuzzy inference systems. In this paper we design intuitionistic fuzzy neural networks to adapt the antecedent and consequent parameters of IFISs. We also propose a mean of maximum defuzzification method for a class of Takagi-Sugeno IFISs and this method is compared with the center of area and basic defuzzification distribution operator. On credit scoring data, we show that the intuitionistic fuzzy neural network trained with gradient descent and Kalman filter algorithms outperforms the traditional ANFIS method.

Fuzzy Cognitive Maps is a well established decision making technique that combines Artificial Neural Networks and Fuzzy Logic. In this paper we present Fuzzy Cognitive Maps for making decisions regarding Container based Logistics, based on the knowledge extracted by a domain expert. Based on this knowledge, a model of the interactions and causal relations among various key Logistics factors is created. Having the FCM created, it is examined both statically and dynamically. A number of scenarios are introduced and the decision making capabilities of the technique are presented by simulating these scenarios and finding the predicted outcomes according to the model and expert’s knowledge. FCM’s predicted consequences of specific decisions can be valuable to Decision Makers since they can test their decisions and proceed with them only if the results are desirable.

For financial institutions, the ability to predict or forecast business failures is crucial, as incorrect decisions can have direct financial consequences. Credit prediction and credit scoring are the two major research problems in the accounting and finance domain. A variety of pattern recognition techniques including neural networks, decision trees and support vector machines have been applied to predict whether borrowers are in danger of bankruptcy and whether they should be considered a good or bad credit risk.

In this paper a clustering and unsupervised method named Self Organizing Map (SOM) is used. We propose to label each cluster with voted method and improve labeling process by training a feedforward Neural Network (NN). The approach uses transfer of learning via SOM to the NN, is tested on the Australian Credit Approval financial data set. We compare both approaches and we will discuss which one is the best prediction model for financial data.

This paper employs an Echo State Network (ESN) to classify unscripted, real-world, punctual activity using inertial sensor data collected from horse riders. ESN has been shown to be an effective black-box classifier for spatio-temporal data and so we suggest that ESN could be useful as a classifier for punctual human activities and as a result a potential tool for wearable technologies. The aim of this study is to provide an example classifier, illustrating the applicability of ESN as a punctual activity classifier for the chosen problem domain. This is part of a wider set of work to build a wearable coach for equestrian sport.

Soft sensor technology has been increasingly used in industry. Its importance is magnified when the process variable to be estimated is key to control and monitoring processes and the respective sensor either has a high probability of failure or is unreliable due to harsh environment conditions. This is the case for permanent downhole gauge (PDG) sensors in the oil and gas industry, which measure pressure and temperature in deepwater oil wells. In this paper, historical data obtained from an actual offshore oil well is used to build a black box model that estimates the PDG downhole pressure from platform variables, using Echo State Networks (ESNs), which are a class of recurrent networks with powerful modeling capabilities. These networks, differently from other neural networks models used by most soft sensors in literature, can model the nonlinear dynamical properties present in the noisy real-world data by using a two-layer structure with efficient training: a recurrent nonlinear layer with fixed randomly generated weights and a linear adaptive readout output layer. Experimental results show that ESNs are a promising technique to model soft sensors in an industrial setting.

Classification of cell types in context of the architecture in tissue specimen is the basis of diagnostic pathology and decisions for comprehensive investigations rely on a valid interpretation of tissue morphology. Especially visual examination of bone marrow cells takes a considerable amount of time and inter-observer variability can be remarkable. In this work, we propose a novel rotation-invariant learning scheme for multi-class Echo State Networks (ESNs), which achieves very high performance in automated bone marrow cell classification. Based on representing static images as temporal sequence of rotations, we show how ESNs robustly recognize cells of arbitrary rotations by taking advantage of their short-term memory capacity.