Skip to main content
main-content
Top

About this book

This book constitutes the refereed proceedings of the 19th International Conference on Engineering Applications of Neural Networks, EANN 2019, held in Xersonisos, Crete, Greece, in May 2019.

The 35 revised full papers and 5 revised short papers presented were carefully reviewed and selected from 72 submissions. The papers are organized in topical sections on AI in energy management - industrial applications; biomedical - bioinformatics modeling; classification - learning; deep learning; deep learning - convolutional ANN; fuzzy - vulnerability - navigation modeling; machine learning modeling - optimization; ML - DL financial modeling; security - anomaly detection; 1st PEINT workshop.

Table of Contents

Frontmatter

Invited Paper

Frontmatter

The Power of the “Pursuit” Learning Paradigm in the Partitioning of Data

Traditional Learning Automata (LA) work with the understanding that the actions are chosen purely based on the “state” in which the machine is. This modus operandus completely ignores any estimation of the Random Environment’s (RE’s) (specified as $$\mathbb {E}$$ ) reward/penalty probabilities. To take these into consideration, Estimator/Pursuit LA utilize “cheap” estimates of the Environment’s reward probabilities to make them converge by an order of magnitude faster. This concept is quite simply the following: Inexpensive estimates of the reward probabilities can be used to rank the actions. Thereafter, when the action probability vector has to be updated, it is done not on the basis of the Environment’s response alone, but also based on the ranking of these estimates. While this phenomenon has been utilized in the field of LA, until recently, it has not been incorporated into solutions that solve partitioning problems. In this paper (The second author gratefully acknowledges the partial support of NSERC, the Natural Sciences and Engineering Council of Canada), we will submit a complete survey of how the “Pursuit” learning paradigm can be and has been used in Object Partitioning. The results demonstrate that incorporating this paradigm can hasten the partitioning by a order of magnitude.

Abdolreza Shirvani, B. John Oommen

AI in Energy Management - Industrial Applications

Frontmatter

A Benchmark Framework to Evaluate Energy Disaggregation Solutions

Energy Disaggregation is the task of decomposing a single meter aggregate energy reading into its appliance level subcomponents. The recent growth of interest in this field has lead to development of many different techniques, among which Artificial Neural Networks have shown remarkable results. In this paper we propose a categorization of experiments that should serve as a benchmark, along with a baseline of results, to efficiently evaluate the most important aspects for this task. Furthermore, using this benchmark we investigate the application of Stacking on five popular ANNs. The models are compared on three metrics and show that Stacking can help improve or ensure performance in certain cases, especially on 2-state devices.

Nikolaos Symeonidis, Christoforos Nalmpantis, Dimitris Vrakas

Application of Deep Learning Long Short-Term Memory in Energy Demand Forecasting

The smart metering infrastructure has changed how electricity is measured in both residential and industrial application. The large amount of data collected by smart meter per day provides a huge potential for analytics to support the operation of a smart grid, an example of which is energy demand forecasting. Short term energy forecasting can be used by utilities to assess if any forecasted peak energy demand would have an adverse effect on the power system transmission and distribution infrastructure. It can also help in load scheduling and demand side management. Many techniques have been proposed to forecast time series including Support Vector Machine, Artificial Neural Network and Deep Learning. In this work we use Long Short Term Memory architecture to forecast 3-day ahead energy demand across each month in the year. The results show that 3-day ahead demand can be accurately forecasted with a Mean Absolute Percentage Error of 3.15%. In addition to that, the paper proposes way to quantify the time as a feature to be used in the training phase which is shown to affect the network performance.

Nameer Al Khafaf, Mahdi Jalili, Peter Sokolowski

Modelling of Compressors in an Industrial CO-Based Operational Cooling System Using ANN for Energy Management Purposes

Large scale cooling installations usually have high energy consumption and fluctuating power demands. There are several ways to control energy consumption and power requirements through intelligent energy and power management, such as utilizing excess heat, thermal energy storage and local renewable energy sources. Intelligent energy and power management in an operational setting is only possible if the time-varying performance of the individual components of the energy system is known. This paper presents an approach to model the compressors in an industrial, operational two-stage cooling system, with CO $$_2$$ as the working fluid, located in an advanced food distribution warehouse in Norway. An artificial neural network is adopted to model the compressors using the operational data. The model is trained with cooling medium evaporation and condensation temperature, suction gas temperature and compressor operating frequency, and outputs electrical power load and cooling load. The best results are found by using a single hidden layer with 45 hidden neurons and a hyperbolic tangent activation function trained with the Adam optimizer, with a resulting mean squared error as low as 0.08% for both the training and validation data sets. The trained model will be part of a system implemented in a real-world setting to determine the cooling load, compressor power load, and coefficient of performance. An intelligent energy management system will utilize the model for energy and power optimization of the cooling system by storing energy in a thermal energy storage, using predictions of energy demand and cooling system performance.

Sven Myrdahl Opalic, Morten Goodwin, Lei Jiao, Henrik Kofoed Nielsen, Mohan Lal Kolhe

Outlier Detection in Temporal Spatial Log Data Using Autoencoder for Industry 4.0

Industry is changing rapidly under industry 4.0. The manufacturing process and its cyber-physical systems (CPSs) produce large amounts of data with many relationships and dependencies in the data. Outlier detection and problem solving is difficult in such an environment. We present an unsupervised outlier detection method to find outliers in temporal spatial log data without domain-specific knowledge. Our method is evaluated with real-world unlabeled CPS log data extracted from a quality glass inspection machine used in production. As a measurement metric for success, we set reasonable outlier areas in cooperation with a domain expert. Using our proposed method, we were able to find all known outlier areas. In addition, we found outliers that were not previously known and have been verified as outliers by a domain expert ex post.

Lukas Kaupp, Ulrich Beez, Jens Hülsmann, Bernhard G. Humm

Reservoir Computing Approaches Applied to Energy Management in Industry

Echo-State Neural Networks represent a very efficient solution for modelling of dynamic systems, thanks to their particular structure, which allows faithful reproduction of the behavior of the system to model with a usually limited computational burden for a training phase. This aspect favors the deployment of Echo-State Neural networks in the industrial field. In this paper, a novel application of such approach is proposed for the modelling of industrial processes. The developed models are part of a complex system for optimizing the exploitation of process off-gases in an integrated steelwork. Two models are presented and discussed, where both shallow Echo-State Neural Networks and Deep Echo State Neural networks are applied. The achieved results are presented and discussed, by comparing advantages and drawbacks of both approaches.

Valentina Colla, Ismael Matino, Stefano Dettori, Silvia Cateni, Ruben Matino

Signal2Vec: Time Series Embedding Representation

The rise of Internet-of-Things (IoT) and the exponential increase of devices using sensors, has lead to an increasing interest in data mining of time series. In this context, several representation methods have been proposed. Signal2vec is a novel framework, which can represent any time-series in a vector space. It is unsupervised, computationally efficient, scalable and generic. The framework is evaluated via a theoretical analysis and real world applications, with a focus on energy data. The experimental results are compared against a baseline using raw data and two other popular representations, SAX and PAA. Signal2vec is superior not only in terms of performance, but also in efficiency, due to dimensionality reduction.

Christoforos Nalmpantis, Dimitris Vrakas

Biomedical - Bioinformatics Modeling

Frontmatter

Classification of Sounds Indicative of Respiratory Diseases

This work presents a system achieving classification of respiratory sounds directly related to various diseases of the human respiratory system, such as asthma, COPD, and pneumonia. We designed a feature set based on wavelet packet analysis characterizing data coming from four sound classes, i.e. crack, wheeze, normal, crack+wheeze. Subsequently, the captured temporal patterns are learned by hidden Markov models (HMMs). Finally, classification is achieved via a directed acyclic graph scheme limiting the problem space while based on decisions made by the available HMMs. Thorough experiments following a well-established protocol demonstrate the efficacy of the proposed solution.

Stavros Ntalampiras, Ilyas Potamitis

Eye Disease Prediction from Optical Coherence Tomography Images with Transfer Learning

Optical Coherence Tomography (OCT) of the human eye are used by optometrists to analyze and detect various age-related eye abnormalities like Choroidal Neovascularization, Drusen (CNV), Diabetic Macular Odeama (DME), Drusen. Detecting these diseases are quite challenging and requires hours of analysis by experts, as their symptoms are somewhat similar. We have used transfer learning with VGG16 and Inception V3 models which are state of the art CNN models. Our solution enables us to predict the disease by analyzing the image through a convolutional neural network (CNN) trained using transfer learning. Proposed approach achieves a commendable accuracy of 94% on the testing data and 99.94% on training dataset with just 4000 units of data, whereas to the best of our knowledge other researchers have achieved similar accuracies using a substantially larger (almost 10 times) dataset.

Arka Bhowmik, Sanjay Kumar, Neeraj Bhat

Severe Asthma Exacerbations Prediction Using Neural Networks

This work introduces a classification model using neural networks on the severity for asthma exacerbations [1] from patients who seek the Brazilian healthcare system in order to get a first treatment of their condition. Healthcare specialists, who work on related databases, usually have access to the information about whether these events need a critical handling or not. However, in many situations, these databases present missing data, which usually demands a manual evaluation of this data and, therefore, time.Hence, the aim of this work is to automate the classification process on the asthma emergency cases – by training and testing neural networks – and compare its performance with other classifiers. The results will be part of the analysis and assessments routines performed by specialists of a healthcare company.

Arthur Silveira, Cristian Muñoz, Leonardo Mendoza

A New Generalized Neuron Model Applied to DNA Microarray Classification

The DNA Microarray classification played an important role in bioinformatics and medicine area. By means of the genetic expressions obtained from a DNA microarrays, it is possible to identify which genes are correlated to a particular disease, in order solve different tasks such as tumor detection, best treatment selection, etc. In the last years, several computational intelligence techniques have been proposed to identify different groups of genes associated with a particular disease; one popular example is the application of artificial neural networks (ANN). The main disadvantage of using this technique is that ANN require a representative number of samples to provide acceptable results. However, the enormous quantity of genes and the few samples available for any disease, demand the use of more robust artificial neural models, capable of providing acceptable results using few samples during the learning process. In this research, we described a new type of generalized neuron model (GNM) applied to the DNA microarray classification task. The proposed methodology selects the set of genes that better describe the disease applying the artificial bee colony algorithm; after that, the GNM is trained using the discovered genes by means of a differential evolution algorithm. Finally, the accuracy of the proposed methodology is evaluated classifying two types of cancer using DNA microarrays: the acute lymphocytic leukemia and the acute myeloid leukemia.

Beatriz A. Garro, Roberto A. Vazquez

Classification - Learning

Frontmatter

A Hybrid Approach for the Fighting Game AI Challenge: Balancing Case Analysis and Monte Carlo Tree Search for the Ultimate Performance in Unknown Environment

The challenging nature of the Fighting Game AI Challenge originates from the short instant of response time which is a typical requirement in real-time fighting games. Handling such real-time constraint requires either tremendous computing power or a clever algorithm design. The former is uncontrollable by the participants, as for the latter, the competition has received a variety of submissions, ranging from the naivest case analysis approach to those using highly advanced computing techniques such as Genetic Algorithms (GA), Reinforcement Learning (RL) or Monte Carlo Tree Search (MCTS), but none could provide a stable solution, especially in the LUD division, where the environment setting is unknown in advance. Our study presents our submission to this challenge in which we designed a winning solution in the LUD division which, for the first time, stably outperformed all players in all competition categories. Our results demonstrate that a proper blend of case analysis and advanced algorithms could result in an ultimate performance.

Lam Gia Thuan, Doina Logofătu, Costin Badică

A Probabilistic Graph-Based Method to Improve Recommender System Accuracy

The last two decades have seen a surge of data on the Web which causes overwhelming users with huge amount of information. Recommender systems (RSs) help users to efficiently find desirable items among a pool of items. RSs often rely on collaborating filtering (CF), where history of transactions are analyzed in order to recommend items. High accuracy, and low time and implementation complexity are most important factors for evaluating the performance algorithms which current methods have the shortage of all or some of them. In this paper, a probabilistic graph-based recommender system (PGB) is proposed based on graph theory and Markov chain with improved accuracy and low complexity. In the proposed method, selecting each item for recommendation is conditioned by considering recommended items in the previous steps. This approach uses a probabilistic model to consider the items which are likely to be preferred by users in the future. Experimental results performed on two real-world datasets including Movielens and Jester, demonstrate that the proposed method significantly outperforms several traditional and state-of-the-art recommender systems.

Nima Joorabloo, Mahdi Jalili, Yongli Ren

A Robust Deep Ensemble Classifier for Figurative Language Detection

Recognition and classification of Figurative Language (FL) is an open problem of Sentiment Analysis in the broader field of Natural Language Processing (NLP) due to the contradictory meaning contained in phrases with metaphorical content. The problem itself contains three interrelated FL recognition tasks: sarcasm, irony and metaphor which, in the present paper, are dealt with advanced Deep Learning (DL) techniques. First, we introduce a data prepossessing framework towards efficient data representation formats so that to optimize the respective inputs to the DL models. In addition, special features are extracted in order to characterize the syntactic, expressive, emotional and temper content reflected in the respective social media text references. These features aim to capture aspects of the social network user’s writing method. Finally, features are fed to a robust, Deep Ensemble Soft Classifier (DESC) which is based on the combination of different DL techniques. Using three different benchmark datasets (one of them containing various FL forms) we conclude that the DESC model achieves a very good performance, worthy of comparison with relevant methodologies and state-of-the-art technologies in the challenging field of FL recognition.

Rolandos-Alexandros Potamias, Georgios Siolas, Andreas Stafylopatis

Enhanced Feature Selection for Facial Expression Recognition Systems with Genetic Algorithms

Humans use voice and facial expressions to infer their state of emotions. Key expressions include being happy, angry, sad and neutral. The expressions accounts for a third of non-verbal communication. The study presents an efficient method to identify facial expressions in images based on artificial neural networks enhanced by genetic algorithms. We use Viola Jones for facial detections and PCA, a statistical method to reduce the dimensionality and extract the features with a local variant of local binary patterns called CS-LBP for static images which is a local algorithm that reduces feature set by comparing symmetrical pixels halving the feature set. The features are then optimally selected using genetic algorithm before classification using artificial neural networks. It is also crucial to note that with these emotions being natural reactions, recognition of feature selection and edge detection from the images can increase accuracy and reduce the error rate. This can be achieved by removing unimportant information from the facial images. The genetic algorithm (GA) chooses a subset of image features based on a reduced-dimensional dataset. The study proposes local binary pattern variant central symmetric local directional pattern (CS-LDP), central symmetric LBP (CS-LBP) and artificial neural networks aided by genetic algorithms for feature selection. The study used the Japanese Female Facial Expression, JAFFE database. The approach outperformed other traditional approaches and proved that with added feature selection and optimization the processing time is reduced and accuracy improved.

Kennedy Chengeta

Imaging Time-Series for NILM

Non Intrusive Load Monitoring is the field that encompasses energy disaggregation and appliance detection. In recent years, Deep Neural Networks have improved the classification performance, using the standard data representation that most datasets provide; that being low-frequency or high-frequency data. In this paper, we explore the NILM problem from the scope of transfer learning. We propose a way of changing the feature space with the use of an image representation of the low-frequency data from UK-Dale and REDD datasets and the pretrained Convolutional Neural Network VGG16. We then train some basic classifiers and use the metric F1 score to test the performance of this representation. Multiple tests are performed to test the adaptability of the models to unseen houses and different datasets. We find that the performance is on par and in some cases outperforms that of popular deep NN algorithms.

Lamprini Kyrkou, Christoforos Nalmpantis, Dimitris Vrakas

Learning Meaningful Sentence Embedding Based on Recursive Auto-encoders

Learning meaningful representations for different granularities of texts is a challenging and on-going area of research in natural language processing. Recently, neural sentence modeling that learns continuous valued vector representations for sentences in a low dimensional latent semantic space has gained increasing attention. In this work, we propose a novel method to learn meaning representation for variable-sized sentence based on recursive auto-encoders. The key difference between our model and others is that we embed the sentence meaning while jointly learning evolved word representation in unsupervised manner and without using any parse or dependency tree. Our deep compositional model is not only able to construct meaningful sentence representation but also to keep pace with the words meanings evolving. We evaluate our obtained embeddings on semantic similarity task. The experimental results show the effectiveness of our proposed model and demonstrate that it can achieve a competitive performance without any feature engineering.

Amal Bouraoui, Salma Jamoussi, Abdelmajid Ben Hamadou

Pruning Extreme Wavelets Learning Machine by Automatic Relevance Determination

Extreme learning machines are used for various contexts in artificial intelligence, such as for classifying patterns, performing time series prediction and regression problems, and being a more viable solution for training hidden layer weights to determine values of the learning model. However, the essence, the model determines that these weights should be determined randomly, and the Moore Penrose pseudoinverse will define only the weights that will act in the output layer. Random weights make this learning a black box because there is no relationship between the hidden layer weights and the problem data. This paper proposes the initialization of weights and bias in the hidden layer through the Wavelets transform that allows the two parameters, previously initialized at random, to be more representative about the problem domain, allowing the frequency range of the input patterns of the network to aid in the definition of weights of the ELM hidden layer. To assist in the representativeness of the data, a technique of selection of characteristics based on automatic relevance determination will be applied to the selection of the most characteristic dimensions of the problem. To compose the network structure, activation functions of the type rectified linear unit, also called ReLU, were used. The proposed model was submitted to the classification test of binary patterns in real classes, and the results show that the proposition of this work assists in bringing better accuracy to the classification results, and thus can be considered a feasible proposition to the training of neural networks that use extreme learning machine.

Paulo V. de Campos Souza, Vinicius J. Silva Araujo, Vanessa S. Araujo, Lucas O. Batista, Augusto J. Guimaraes

Students’ Performance Prediction Model Using Meta-classifier Approach

Students’ performance is vitally important at all stages of education, particularly for Higher Education Institutions. One of the most important issues is to improve the performance and quality of students enrolled. The initial symptom of at-risks’ students need to be observed and earlier preventive measures are required to be carried out so as to determine the cause of students’ dropout rate. Hence, the purpose of this research is to identify factors influencing students’ performance using educational data mining techniques. In order to achieve this, data from different sources is employed into a single platform for pre-processing and modelling. The design of the study is divided into 6 different phases (data collection, data integration, data pre-processing such as cleaning, normalization, and transformation, feature selection, patterns extraction and model optimization as well as evaluation. The datasets were collected from a students’ information system and e-learning system from a public university in Malaysia, while sample data from the Faculty of Engineering were used accordingly. This study also employed the use of academic, demographical, economical and behaviour e-learning features, in which 8 different group models were developed using 3 base-classifiers; Decision Tree, Artificial Neural Network and Support Vector Machine, and 5 multi-classifiers; Random Forest, Bagging, AdaBoost, Stacking and Majority Vote classifier. Finally, the highest accuracy of the classifier model was optimized. At the end, new Students’ Performance Prediction Model was developed. The result proves that combination demographics with behaviour using a meta-classifier model with optimized hyper parameter produced better accuracy to predict students’ performance.

Hasniza Hassan, Syahid Anuar, Nor Bahiah Ahmad

Deep Learning

Frontmatter

A Deep Network System for Simulated Autonomous Driving Using Behavioral Cloning

This paper studies the performance of a convolutional neural network (CNN) trained to learn the behavior of a vehicle using data from a simulator that allows real-time information gathering from vehicle chassis, machine position and speed. The network uses information from the front-facing, right and right cameras, the car’s position on the lane and its speed. This approach proves to be quite effective: with a minimum of driving time taken directly from proper driving simulations in the form of a game, the system learns to drive on a marked strip road. The network automatically learns the internal representations of the necessary processing steps, such as the detection of useful road features, required speed, and track position. Different types of activation functions are used, and it is noticed that the exponential linear unit (ELU) activation function leads to improved learning compared to other activation functions.

Andreea-Iulia Patachi, Florin Leon, Doina Logofătu

A Machine Hearing Framework for Real-Time Streaming Analytics Using Lambda Architecture

Disruptions to the earth’s biosphere and to the natural environment stemming from the indiscreet human activity, have caused serious environmental problems which are tantamount to an extended and prolonged ecological crisis. Climate change is clearly reflected in the increase of the global average air and ocean temperatures, in the excessive melting of snow-ice, and in the rise of the global average sea level. One of the most serious impacts of climate change is the complex interaction of species in relation to their corresponding climatic survival factors, which favors the spread of invasive species (INSP). These species constitute a very serious and rapidly deteriorating threat to the natural biodiversity of the native environment, but also to the flora, fauna, and even to the local human population. This research proposes a Machine Hearing (MH) framework for real-time streaming analytics, employing Lambda Architecture (LARC). The hybrid modeling effort is based on timely and advanced Computational Intelligence (COIN) approaches. The Framework for Lambda Architecture Machine Hearing (FLAME_H) uses a combination of batch and streaming data. The FLAME_H applies the EL_GROSEMMARI (Extreme Learning Graph Regularized Online Sequential Multilayer Multiencoder Algorithm) to classify the batch data and the Adaptive Random Forest (ARF) in order to control the data streams in real time. The aim of the proposed framework is the intelligent identification and classification of invasive alien species, based on the sounds they produce. This would contribute to the protection of biodiversity and biosecurity in a certain area.

Konstantinos Demertzis, Lazaros Iliadis, Vardis-Dimitris Anezakis

Deep Learning and Change Detection for Fall Recognition

Early fall detection is a crucial research challenge since the time delay from fall to first aid is a key factor that determines the consequences of a fall. Wearable sensors allow a reliable way for daily-life activities tracking, able to detect immediately a high-risk fall via a machine learning framework. Towards this direction, accelerometer devices are used widely for the assessment of fall risk. Although there is a plethora of studies under this perspective with promising results, several challenges still remain such as the extremely demanding data and power management as well as the discovery of false positive falls. In this work we propose a complete methodology based on the combination of the computationally demanding convolutional neural networks along with a lightweight change detection method. Our basic assumption is that it is possible to control computational resources for the operation of a classifier, suffice to be activated when a strong change in user’s movements is identified. The proposed methodology was applied to real experimental data providing reliable results that justify the original hypothesis.

Sotiris K. Tasoulis, Georgios I. Mallis, Spiros V. Georgakopoulos, Aristidis G. Vrahatis, Vassilis P. Plagianakos, Ilias G. Maglogiannis

Image Classification Using Deep Neural Networks: Transfer Learning and the Handling of Unknown Images

Deep learning is a subset of machine learning that is powerful at recognizing patterns and extensively used for image classification. However, it typically requires a large amount of data and it is computationally expensive for training an application from scratch. ImageNet database has millions of images pertaining to different categories that are acquired by years of hard work. Getting such a database for every application is tough and time consuming. Transfer learning is an alternative to conventional training. Transfer learning results in much faster and easier training of a network. This research set out to evaluate the effect of transfer learning on the performance of a Deep Neural Network (DNN). Pre-trained AlexNet was selected, modified and retrained for 3 image classification applications (gears, connectors and coins) with a modest database. This approach gave 99% classification accuracy using transfer learning. To test the robustness of the network, unknown images were added to one of the classes and the accuracy was reinforced using a probability threshold. This approach succeeded in compensating for the effect of unknowns in the accuracy.

Vedang Chauhan, Keyur D. Joshi, Brian Surgenor

Learnae: Distributed and Resilient Deep Neural Network Training for Heterogeneous Peer to Peer Topologies

Learnae is a framework proposal for decentralized training of Deep Neural Networks (DNN). The main priority of Learnae is to maintain a fully distributed architecture, where no participant has any kind of coordinating role. This solid peer-to-peer concept covers all aspects: Underlying network protocols, data acquiring/distribution and model training. The result is a resilient DNN training system with no single point of failure. Learnae focuses on use cases where infrastructure heterogeneity and network unreliability result to an always changing environment of commodity-hardware nodes. In order to achieve this level of decentralization, new technologies had to be utilized. The main pillars of this implementation are the ongoing projects of IPFS and IOTA. IPFS is a platform for a purely decentralized filesystem, where each node contributes local data storage. IOTA aims to be the networking infrastructure of the upcoming IoT reality. On top of these, we propose a management algorithm for training a DNN model collaboratively, by optimal exchange of data and model weights, always using distribution-friendly gossip protocols.

Spyridon Nikolaidis, Ioannis Refanidis

Predicting Customer Churn Using Artificial Neural Network

Switching of customers from one service provider to another service provider is known as customer churn. With the surge of the technologies and increased customer awareness, retaining customers has become vital for a company’s growth. Several studies have been carried out to keep a check on the customer churn of companies and analyze churn prediction but the accuracy rate is not up to the mark. Recently with the extensive research in the field of Artificial Intelligence it has become possible to dig to the core of the factor responsible for customer churn. We present an effective solution to this challenging problem of customer churn prediction using the data set of telecommunication industry and Artificial Neural Networks to determine the factors influencing the customer churn and optimize the solutions by experimenting with different activation functions.

Sanjay Kumar, Manish Kumar

Virtual Sensor Based on a Deep Learning Approach for Estimating Efficiency in Chillers

Intensive use of heating, ventilation and air conditioning (HVAC) systems in buildings entails an analysis and monitoring of their efficiency. Cooling systems are key facilities in large buildings, and particularly critical in hospitals, where chilled water production is needed as an auxiliary resource for a large number of devices. A chiller plant is often composed of several HVAC units running at the same time, being impossible to assess the individual cooling production and efficiency, since a sensor is seldom installed due to the high cost. We propose a virtual sensor that provides an estimation of the cooling production, based on a deep learning architecture that features a 2D CNN (Convolutional Neural Network) to capture relevant features on two-way matrix arrangements of chiller data involving thermodynamic variables and the refrigeration circuits of the chiller unit. Our approach has been tested on an air-cooled chiller in the chiller plant at a hospital, and compared to other state-of-the-art methods using 10-fold cross-validation. Our results report the lowest errors among the tested methods and include a comparison of the true and estimated cooling production and efficiency for a period of several days.

Serafín Alonso, Antonio Morán, Daniel Pérez, Perfecto Reguera, Ignacio Díaz, Manuel Domínguez

Deep Learning - Convolutional ANN

Frontmatter

Canonical Correlation Analysis Framework for the Reduction of Test Time in Industrial Manufacturing Quality Tests

In industrial manufacturing processes, quality control tests are performed in order to measure product characteristics which help assess and classify the product’s quality. In this work, we focus on quality tests during which a signal is recorded for each product. We propose the usage of data-driven methods for a potential reduction of the test duration, without inducing loss in the quality classification performance. While in industrial practice most features extracted from the signals are still often hand crafted by domain experts and used as input to shallow classifiers, more advanced classification methods such as Convolutional Neural Networks (CNNs) are able to combine the feature extraction, selection and classification into a single process.In this paper we first use CNNs to determine whether an excerpt of the recorded signal exists which, starting at time 0, contains already enough information so as to match the classification performance reached with the usage of the complete signal. Second, we apply the Canonical Correlation Analysis (CCA) framework to investigate how the features extracted and selected from multiple, successively increasing excerpts relate to the features the quality test was originally designed to measure. Third, we analyze the presence of noise among the classification-relevant features extracted from the increasing excerpts. The suitability of the proposed framework is validated using a real-world dataset from the automotive industry, showing that the test time of the corresponding vibroacoustical quality test can be reduced by 77.78%, thus ensuring a high practical relevance of the findings.

Paul Alexandru Bucur, Philipp Hungerländer

Convolutional Neural Network for Detection of Building Contours Using Multisource Spatial Data

Building reconstruction from aerial photographs and other multi-source urban spatial data is a task endeavored using a plethora of automated and semi-automated methods ranging from point processes, classic image processing and laser scanning. Here, we describe a convolutional neural network (CNN) method for the detection of building borders. In particular, the network is based on the state of the art super-resolution model SRCNN and accepts aerial photographs depicting densely populated urban area data as well as their corresponding digital elevation maps (DEM). Training is performed using three variations of this urban data set and aims at detecting building contours through a novel super-resolved heteroassociative mapping. Another novelty of our approach is the design of a modified custom loss layer, named Top-N, whereby the mean square error (MSE) between the reconstructed output image and the provided ground truth (GT) image of building contours is computed on the 2N image pixels with highest values, where N is the number of contour pixels in GT. Assuming that most of the N contour pixels of the GT image are also in the top 2N pixels of the reconstruction, this modification balances the two pixel categories and improves the generalization behavior of the CNN model. It is shown, in our experiments, that the Top-N cost function offers performance gains in comparison to standard MSE. Further improvement in generalization ability of the network is achieved by using dropout.

George Papadopoulos, Nikolaos Vassilas, Anastasios Kesidis

Fuzzy - Vulnerability - Navigation Modeling

Frontmatter

A Meta-multicriteria Approach to Estimate Drought Vulnerability Based on Fuzzy Pattern Recognition

The objective of this paper is to explore a new integrated approach to estimate drought vulnerability taking into account the characteristics of a system that make it likely to be affected by an external risk. A meta-multicriteria approach is adopted since the problem itself modulates the multiple criteria method. Firstly, relevant information is grouped into drought sensitivity and adaptive capacity criteria. Ιnstead of the estimation of a unique score for the vulnerability, a classification of the vulnerability to drought into several categories is proposed. Based on the maximum and minimum values of the above criteria initially, four non-ordered categories are established initially to characterize the vulnerability to drought. In order to classify water-scarce countries into the four or more categories the fuzzy pattern recognition is exploited. The proposed approach is applied to estimate drought vulnerability in selected Mediterranean countries. A choice that strengthens the meta-multicriteria character of the proposed approaches is that the categories are not ordered, but they are modulated from all the combination of the extreme points.

M. Spiliotis, A. Iglesias, L. Garrote

Bioinspired Early Prediction of Earthquakes Inferred by an Evolving Fuzzy Neural Network Paradigm

Earthquakes could be early predicted as demonstrated by animal’s behavior that are able to detect the leading wave part of the seismic wave (the P-wave). P-waves travel faster than S-wave wave (the shaking wave), so they reach the seismic sensors early (tens of seconds to minutes in advance) compared to the P-wave.A bioinspired framework could be implemented mimicking the animal’s behaviour related to the event of an incoming earthquake.Training a Fuzzy Neural Network to recognize the P-waves, early prediction of earthquakes is feasible and an adequate recovery strategy could be implemented. A technological motivation is the availability of OTS (off-the-shelf) vibration sensors and the fast development of IoT (Internet of Things) toward the new paradigm IoE (Internet of Everything).

Mario Malcangi, Marco Malcangi

Enhancing Disaster Response for Hazardous Materials Using Emerging Technologies: The Role of AI and a Research Agenda

Despite all efforts like the introduction of new training methods and personal protective equipment, the need to reduce the number of First Responders (FRs) fatalities and injuries remains. Reports show that advances in technology have not yet resulted in protecting FRs from injuries, health impacts, and odorless toxic gases effectively. Currently, there are emerging technologies that can be exploited and applied in emergency management settings to improve FRs protection. The aim of this paper is threefold: First, to conduct scenario analysis and situations that currently threat the first responders. Second, to conduct gap analysis concerning the new technology needs in relations to the proposed scenarios. Third, to propose a research agenda and to discuss the role of Artificial Intelligence within it.

Jaziar Radianti, Ioannis Dokas, Kees Boersma, Nadia Saad Noori, Nabil Belbachir, Stefan Stieglitz

Machine Learning Modeling - Optimization

Frontmatter

Evolutionary Optimization on Artificial Neural Networks for Predicting the User’s Future Semantic Location

Location prediction has gained enormously in importance in the recent years. For this reason, there exists a great variety of research work carried out at both the academia and the industry. At the same time, there is an increasing trend towards utilizing additional semantic information aiming at building more accurate algorithms. Existing location prediction approaches rely mostly on data-driven models, such as Hidden Markov Chains, Bayes Networks and Artificial Neural Networks (ANN), with the latter achieving usually the best results. Most ANN-based solutions apply Grid Parameter Search and Stochastic Gradient Descent for training their models, that is, for identifying the optimal structure and weights of the network. In this work, motivated by the promising results of genetic algorithms in optimizing neural networks in temporal sequence learning areas, such as the gene and the stock price index prediction, we propose and evaluate their use in optimizing our ANN-based semantic location prediction model. It can be shown that evolutionary algorithms can lead to a significant improvement with respect to its predictive performance, as well as to the time needed for the model’s optimization.

Antonios Karatzoglou

Global Minimum Depth in Edwards-Anderson Model

In the literature the most frequently cited data are quite contradictory, and there is no consensus on the global minimum value of 2D Edwards-Anderson (2D EA) Ising model. By means of computer simulations, with the help of exact polynomial Schraudolph-Kamenetsky algorithm, we examined the global minimum depth in 2D EA-type models. We found a dependence of the global minimum depth on the dimension of the problem N and obtained its asymptotic value in the limit N → ∞. We believe these evaluations can be further used for examining the behavior of 2D Bayesian models often used in machine learning and image processing.

Iakov Karandashev, Boris Kryzhanovsky

Imbalanced Datasets Resampling Through Self Organizing Maps and Genetic Algorithms

The paper presents a novel approach for the resampling of imbalanced datasets aiming at the improvement of classifiers performance. The method exploits two self–organizing–maps for the determinations of the clusters of majority and minority data. Clusters centroids are used to select the samples whose under–sampling or over–sampling is more convenient while the optimal resampling rates are determined through a genetic algorithm that maximizes the classifier performance. The algorithm is tested on several datasets coming from both the UCI repository and real industrial applications and compared to other widely used resampling methods.

Marco Vannucci, Valentina Colla

Improvement of Routing in Opportunistic Communication Networks of Vehicles by Unsupervised Machine Learning

This paper deals with the problem of an application of machine learning in order to improve routing in a special class of opportunistic networks of vehicles called cluster opportunistic networks. We have proposed the hierarchical routing algorithm which combines three strategies in order to improve routing in OPNs: metric based on the node affiliation with detected OPN geographic sector, metrics based on the node affiliation with the communication community constructed in the spatio-temporal domain with time constraints and metric based on the node local encounter measure. The proposed routing scheme combines these four metrics to make decisions on message forwarding. The proposed routing method performance has been evaluated on simulation scenario and compared to Epidemic routing.

Ladislava Smítková Janků, Kateřina Hyniová

Machine Learning Approach for Drone Perception and Control

This study focuses on the application of machine learning and neural networks for the action selection and better understanding of the environment for controlling unmanned aerial vehicles, instead of explicit models to achieve the same task. Implementation of machine learning and deep learning algorithms such as non-linear regression were combined with neural networks to learn the system dynamics of a drone for the prediction of future states. Behavior cloning method is applied to mimic the actions of autopilot and comparative study of the decisions of autopilot and learned model were conducted in a simulated environment. The deep convolutional neural network was utilized for the visual perception task in the forest environment by detecting trees as obstacles. The prediction of future states and mimicking the autopilot actions were realized with relatively small error to the data from explicit model and the tree detection was successful even in the low sunlight condition.

Yograj S. Mandloi, Yoshinobu Inada

ML - DL Financial Modeling

Frontmatter

A Deep Dense Neural Network for Bankruptcy Prediction

Bankruptcy prediction is a problem that is becoming more and more interesting. This problem concerns in particular financial and accounting researchers. Nevertheless, it is a field that gathers the focus of companies, creditors, investors and in general firms which are interested in investments or transactions. Because of a variety of parameters, such as multiple accounting ratios or many potential explanatory variables, the complexity of this problem is very high. For this reason, the probability for a company to go bankrupt or not is very difficult to be calculated. Moreover, the precise determination of the bankruptcy is a very important issue. All the above details constitute a complex problem and by taking into account the data that need to be processed, we conclude that machine learning techniques and reliable predictive models are necessary. In this paper, the effectiveness of a dense deep neural network in bankruptcy prediction relating to solvent Greek firms is tested. The experimental results showed that the provided scheme gives promising outcomes.

Stamatios-Aggelos N. Alexandropoulos, Christos K. Aridas, Sotiris B. Kotsiantis, Michael N. Vrahatis

Stock Price Movements Classification Using Machine and Deep Learning Techniques-The Case Study of Indian Stock Market

Stock price movements forecasting is an important topic for traders and stock analyst. Timely prediction in stock yields can get more profits and returns. The predicting stock price movement on a daily basis is a difficult task due to more ups and down in the financial market. Therefore, there is a need for a more powerful predictive model to predict the stock prices. Most of the existing work is based on machine learning techniques and considered very few technical indicators to predict the stock prices. In this paper, we have extracted 33 technical indicators based on daily stock price such as open, high, low and close price. This paper addresses the two problems, first is the technical indicator feature selection and identification of the relevant technical indicators by using Boruta feature selection technique. The second is an accurate prediction model for stock price movements. To predict stock price movements we have proposed machine learning techniques and deep learning based model. The performance of the deep learning model is better than the machine learning techniques. The experimental results are significant improves the classification accuracy rate by 5% to 6%. National Stock Exchange, India (NSE) stocks are considered for the experiment.

Nagaraj Naik, Biju R. Mohan

Study of Stock Return Predictions Using Recurrent Neural Networks with LSTM

Stock price returns forecasting is challenging task for day traders to yield more returns. In the past, most of the literature was focused on machine learning algorithm to predict the stock returns. In this work, the recurrent neural network (RNN) with long short term memory (LSTM) is studied to forecast future stock returns. It has the ability to keep the memory of historical stock returns in order to forecast future stock return output. RNN with LSTM is used to store recent stock information than old related stock information. We have considered a recurrent dropout in RNN layers to avoid overfitting in the model. To accomplish the task we have calculated stock return based on stock closing prices. These stock returns are given as input to the recurrent neural network. The objective function of the prediction model is to minimize the error in the model. To conduct the experiment, data is collected from the National Stock Exchange, India (NSE). The proposed RNN with LSTM model outperforms compared to an feed forward artificial neural network.

Nagaraj Naik, Biju R. Mohan

Security - Anomaly Detection

Frontmatter

Comparison of Network Intrusion Detection Performance Using Feature Representation

Intrusion detection is essential for the security of the components of any network. For that reason, several strategies can be used in Intrusion Detection Systems (IDS) to identify the increasing attempts to gain unauthorized access with malicious purposes including those base on machine learning. Anomaly detection has been applied successfully to numerous domains and might help to identify unknown attacks. However, there are existing issues such as high error rates or large dimensionality of data that make its deployment difficult in real-life scenarios. Representation learning allows to estimate new latent features of data in a low-dimensionality space. In this work, anomaly detection is performed using a previous feature learning stage in order to compare these methods for the detection of intrusions in network traffic. For that purpose, four different anomaly detection algorithms are applied to recent network datasets using two different feature learning methods such as principal component analysis and autoencoders. Several evaluation metrics such as accuracy, F1 score or ROC curves are used for comparing their performance. The experimental results show an improvement for two of the anomaly detection methods using autoencoder and no significant variations for the linear feature transformation.

Daniel Pérez, Serafín Alonso, Antonio Morán, Miguel A. Prada, Juan José Fuertes, Manuel Domínguez

Cyber Security Incident Handling, Warning and Response System for the European Critical Information Infrastructures (CyberSANE)

This paper aims to enhance the security and resilience of Critical Information Infrastructures (CIIs) by providing a dynamic collaborative, warning and response system (CyberSANE system) supporting and guiding security officers and operators (e.g. Incident Response professionals) to recognize, identify, dynamically analyse, forecast, treat and respond to their threats and risks and handle their daily cyber incidents. The proposed solution provides a first of a kind approach for handling cyber security incidents in the digital environments with highly interconnected, complex and diverse nature.

Spyridon Papastergiou, Haralambos Mouratidis, Eleni-Maria Kalogeraki

Fault Diagnosis in Direct Current Electric Motors via an Artificial Neural Network

The combined problem of fault detection and classification (referred to as fault diagnosis) of Direct Current (DC) electric motors via a simple, yet powerful, technique based on an Artificial Neural Network (ANN) is proposed. The ability of an ANN in identifying patterns with high fidelity—without the need of any rigorous mathematical model of the system under investigation—leads to an excellent diagnosis performance, even for faults that result in almost indistinguishable output system responses (both in time and in frequency domain). The flexibility and speed of the presented method indicate that it can easily be applied to on-line fault diagnosis as well.

Theofanis I. Aravanis, Tryfon-Chrysovalantis I. Aravanis, Polydoros N. Papadopoulos

1st PEINT Workshop

Frontmatter

On Predicting Bottlenecks in Wavefront Parallel Video Coding Using Deep Neural Networks

Video coding incurs high computational complexity particularly at the encoder side. For this reason, parallelism is used at the various encoding steps. One of the popular coarse grained parallelization tools offered by many standards is wavefront parallelism. Under the scheme, each row of blocks is assigned to a separate thread for processing. A thread might commence encoding a particular block once certain precedence constraints are met, namely, it is required that the left block of the same row and the top and top-right block of the previous row have finished compression. Clearly, the imposed constraints result in processing delays. Therefore, in order to optimize performance, it is of paramount importance to properly identify potential bottlenecks before the compression of a frame starts, in order to alleviate them through better resource allocation. In this paper we present a simulation model that predicts bottlenecks based on the estimated block compression times produced from a regression neural network. Experiments with datasets obtained using the reference encoder of HEVC (High Efficiency Video Coding) illustrate the merits of the proposed model.

Natalia Panagou, Panagiotis Oikonomou, Panos K. Papadopoulos, Maria Koziri, Thanasis Loukopoulos, Dimitris Iakovidis

Recognizing Human Actions Using 3D Skeletal Information and CNNs

In this paper we present an approach for the recognition of human actions targeting at activities of daily living (ADLs). Skeletal information is used to create images capturing the motion of joints in the 3D space. These images are then transformed to the spectral domain using 4 well-known image transforms. A deep Convolutional Neural Network is trained on those images. Our approach is thoroughly evaluated using a well-known, publicly available challenging dataset and for a set of actions that resembles to common ADLs, covering both cross-view and cross-subject cases.

Antonios Papadakis, Eirini Mathe, Ioannis Vernikos, Apostolos Maniatis, Evaggelos Spyrou, Phivos Mylonas

Staircase Detection Using a Lightweight Look-Behind Fully Convolutional Neural Network

Staircase detection in natural images has several applications in the context of robotics and visually impaired navigation. Previous works are mainly based on handcrafted feature extraction and supervised learning using fully annotated images. In this work we address the problem of staircase detection in weakly labeled natural images, using a novel Fully Convolutional neural Network (FCN), named LB-FCN light. The proposed network is an enhanced version of our recent Look-Behind FCN (LB-FCN), suitable for deployment on mobile and embedded devices. Its architecture features multi-scale feature extraction, depthwise separable convolutions and residual learning. To evaluate its computational and classification performance, we have created a weakly-labeled benchmark dataset from publicly available images. The results from the experimental evaluation of LB-FCN light indicate its advantageous performance over the relevant state-of-the-art architectures.

Dimitrios E. Diamantis, Dimitra-Christina C. Koutsiou, Dimitris K. Iakovidis

Obstacle Detection Based on Generative Adversarial Networks and Fuzzy Sets for Computer-Assisted Navigation

Obstacle detection addresses the detection of an object, of any kind, that interferes with the canonical trajectory of a subject, such as a human or an autonomous robotic agent. Prompt obstacle detection can become critical for the safety of visually impaired individuals (VII). In this context, we propose a novel methodology for obstacle detection, which is based on a Generative Adversarial Network (GAN) model, trained with human eye fixations to predict saliency, and the depth information provided by an RGB-D sensor. A method based on fuzzy sets are used to translate the 3D spatial information into linguistic values easily comprehensible by VII. Fuzzy operators are applied to fuse the spatial information with the saliency information for the purpose of detecting and determining if an object may interfere with the safe navigation of the VII. For the evaluation of our method we captured outdoor video sequences of 10,170 frames in total, with obstacles including rocks, trees and pedestrians. The results showed that the use of fuzzy representations results in enhanced obstacle detection accuracy, reaching 88.1%.

George Dimas, Charis Ntakolia, Dimitris K. Iakovidis

Backmatter

Additional information

Premium Partner

    Image Credits