Advances in Hybridization of Intelligent Methods

Table of Contents

1. Frontmatter

2. Deep Learning Approaches for Facial Emotion Recognition: A Case Study on FER-2013

   Panagiotis Giannopoulos, Isidoros Perikos, Ioannis Hatzilygeroudis

   Abstract

   Emotions constitute an innate and important aspect of human behavior that colors the way of human communication. The accurate analysis and interpretation of the emotional content of human facial expressions is essential for the deeper understanding of human behavior. Although a human can detect and interpret faces and facial expressions naturally, with little or no effort, accurate and robust facial expression recognition by computer systems is still a great challenge. The analysis of human face characteristics and the recognition of its emotional states are considered to be very challenging and difficult tasks. The main difficulties come from the non-uniform nature of human face and variations in conditions such as lighting, shadows, facial pose and orientation. Deep learning approaches have been examined as a stream of methods to achieve robustness and provide the necessary scalability on new type of data. In this work, we examine the performance of two known deep learning approaches (GoogLeNet and AlexNet) on facial expression recognition, more specifically the recognition of the existence of emotional content, and on the recognition of the exact emotional content of facial expressions. The results collected from the study are quite interesting.

3. Analysis of Biologically Inspired Swarm Communication Models

   Musad Haque, Electa Baker, Christopher Ren, Douglas Kirkpatrick, Julie A. Adams

   Abstract

   The biological swarm literature presents communication models that attempt to capture the nature of interactions among the swarm’s individuals. The reported research derived algorithms based on the metric, topological, and visual biological swarm communication models. The evaluated hypothesis is that the choice of a biologically inspired communication model can affect the swarm’s performance for a given task. The communication models were evaluated in the context of two swarm robotics tasks: search for a goal and avoid an adversary. The general findings demonstrate that the swarm agents had the best overall performance when using the visual model for the search for a goal task and performed the best for the avoid an adversary task when using the topological model. Further analysis of the performance metrics by the various experimental parameters provided insights into specific situations in which the models will be the most or least beneficial. The importance of the reported research is that the task performance of a swarm can be amplified through the deliberate selection of a communications model.

4. Target-Dependent Sentiment Analysis of Tweets Using Bidirectional Gated Recurrent Neural Networks

   Mohammed Jabreel, Fadi Hassan, Antonio Moreno

   Abstract

   The task of target-dependent sentiment analysis aims to identify the sentiment polarity towards a certain target in a given text. All the existing models of this task assume that the target is known. This fact has motivated us to develop an end-to-end target-dependent sentiment analysis system. To the extent of our knowledge, this is the first system that identifies and extract the target of the tweets. The proposed system is composed of two main steps. First, the targets of the tweet to be analysed are extracted. Afterwards, the system identifies the polarities of the tweet towards each extracted target. We have evaluated the effectiveness of the proposed model on a benchmark dataset from Twitter. The experiments show that our proposed system outperforms the state-of-the-are methods for target-dependent sentiment analysis.

5. Traffic Modelling, Visualisation and Prediction for Urban Mobility Management

   Tomasz Maniak, Rahat Iqbal, Faiyaz Doctor

   Abstract

   Smart city combines connected services from different disciplines offering a promise of increased efficiency in transport and mobility in urban environment. This has been enabled through many important advancements in fields like machine learning, big data analytics, hardware manufacturing and communication technology. Especially important in this context is big data which is fueling the digital revolution in an increasingly knowledge driven society by offering intelligence solutions for the smart city. In this paper, we discuss the importance of big data analytics and computational intelligence techniques for the problem of taxi traffic modelling, visualisation and prediction. This work provides a comprehensive survey of computational intelligence techniques appropriate for the effective processing and analysis of big data. A brief description of many smart city projects, initiatives and challenges in the UK is also presented. We present a hybrid data modelling approach used for the modelling and prediction of taxi usage. The approach introduces a novel biologically inspired universal generative modelling technique called Hierarchical Spatial-Temporal State Machine (HSTSM). The HSTSM modelling approach incorporates many soft computing techniques including: deep belief networks, auto-encoders, agglomerative hierarchical clustering and temporal sequence processing. A case study for the modelling and prediction of traffic based on taxi movements is described, where HSTSM is used to address the computational challenges arising from analysing and processing large volumes of varied data.

6. Assurance in Reinforcement Learning Using Quantitative Verification

   George Mason, Radu Calinescu, Daniel Kudenko, Alec Banks

   Abstract

   Reinforcement learning (RL) agents converge to optimal solutions for sequential decision making problems. Although increasingly successful, RL cannot be used in applications where unpredictable agent behaviour may have significant unintended negative consequences. We address this limitation by introducing an assured reinforcement learning (ARL) method which uses quantitative verification (QV) to restrict the agent behaviour to areas that satisfy safety, reliability and performance constraints specified in probabilistic temporal logic. To this end, ARL builds an abstract Markov decision process (AMDP) that models the problem to solve at a high level, and uses QV to identify a set of Pareto-optimal AMDP policies that satisfy the constraints. These formally verified abstract policies define areas of the agent behaviour space where RL can occur without constraint violations. We show the effectiveness of our ARL method through two case studies: a benchmark flag-collection navigation task and an assisted-living planning system.

7. Distillation of Deep Learning Ensembles as a Regularisation Method

   Alan Mosca, George D. Magoulas

   Abstract

   Ensemble methods are among the most commonly utilised algorithms that construct a group of models and combine their predictions to provide improved generalisation. They do so by aggregating multiple diverse versions of models learned using machine learning algorithms, and it is this diversity that enables the ensemble to perform better than any of its members taken individually. This approach can be extended to produce ensembles of deep learning methods that combine various good performing models, which are between them very diverse because they have reached different local minima and make different prediction errors. It has been shown that a large, cumbersome deep neural network can be approximated by a smaller network through a process of distillation, and that it is possible to approximate an ensemble of other learning algorithms by using a single neural network, with the help of additional artificially generated pseudo-data. We extend this work to show that an ensemble of deep neural networks can indeed be approximated by a single deep neural network with size and capacity equal to the single ensemble member, and we develop a recipe that shows how this can be achieved without using any artificial training data or any other special provisions, such as using the soft output targets during the distillation process. We also show that, under particular circumstances, the distillation process can be used as a form of regularisation, through its implicit reduction in learning capacity. We corroborate our findings with an experimental analysis on some common benchmark datasets in computer vision and deep learning.

8. Heuristic Constraint Answer Set Programming for Manufacturing Problems

   Erich C. Teppan, Gerhard Friedrich

   Abstract

   Constraint answer set programming (CASP) is a family of hybrid approaches integrating answer set programming (ASP) and constraint programming (CP). These hybrid approaches have already proven to be successful in various domains. In this paper we present the CASP solver ASCASS (A Simple Constraint Answer Set Solver) which provides novel methods for defining and exploiting search heuristics. Beyond the possibility of using already built-in problem-independent heuristics, ASCASS allows on the ASP level the definition of problem-dependent variable selection, value selection and pruning strategies which guide the search of the CP solver. In this context, we investigate the applicability and performance of CASP in general and ASCASS in particular in two important manufacturing problem domains: system configuration and job scheduling.