Artificial Intelligence XXXVII

Information overload is a well-known problem that generally occurs when searching for information online. To reduce this effect having prior knowledge on the domain and also a searching strategy is critical. Obtaining such qualities can be challenging for students since they are still learning about various domains and might not be familiar with the domain-specific keywords. In this paper, we are proposing a framework that aims to assist students to have a richer list of keyphrases that are pertinent to a domain under study and provide a mechanism for lectures to understand what search strategies their students are adopting. The proposed framework includes a Google Chrome Extension, a background and a remote server. The Google Chrome Extension is utilized to collect, process browsing data and generate reports containing keyphrases searched by students. The results of the user evaluation were compared with a similar framework (TextRank). The results indicate that our framework performed better in terms of accuracy of keyphrases and response time.

In recent years, pattern classification has started to move from computing models with outstanding prediction rates to models able to reach a suitable trade-off between accuracy and interpretability. Fuzzy Cognitive Maps (FCMs) and their extensions are recurrent neural networks that have been partially exploited towards fulfilling such a goal. However, the interpretability of these neural systems has been confined to the fact that both neural concepts and weights have a well-defined meaning for the problem being modeled. This rather naive assumption oversimplifies the complexity behind an FCM-based classifier. In this paper, we propose a symbolic explanation module that allows extracting useful insights and patterns from a trained FCM-based classifier. The proposed explanation module is implemented in Prolog and can be seen as a reverse symbolic reasoning rule that infers the inputs to be provided to the model to obtain the desired output.

This paper focuses on the problem of inconsistent predictions of modern convolutional neural networks (CNN) at patch (i.e. sub-image) boundaries. Limited by the graphics processing unit (GPU) resources, image tiling and stitching countermeasure have been applied for most megapixel images, that is, cutting images into overlapping tiles as CNN input, and then stitching CNN outputs together. However, we found that stitched (i.e. recovered) predictions have discontinuous grid-like noise. We propose a simple yet efficient overlap training framework to mitigate the inconsistent prediction at patch boundaries without changing the model architecture while improving the stability, robustness of the model. We have applied our solution to various CNNs (such as U-Net, DeepLab, RCF) and tested them on two real-world datasets. Extensive experiments suggest that the new framework is sufficient in reducing inconsistency and outperform these countermeasures. The source code and coloured figures are made publicly available online at: https://github.com/anyuzoey/Overlap-Training.git .

The ABC system repairs faulty Datalog theories using a combination of abduction, belief revision and conceptual change via reformation. Abduction and Belief Revision add/delete axioms or delete/add preconditions to rules, respectively. Reformation repairs them by changing the language of the faulty theory. Unfortunately, the ABC system overproduces repair suggestions. Our aim is to prune these suggestions to leave only a Pareto front of the optimal ones. We apply an algorithm for solving Max-Sat problems, which we call the Partial Max-Sat algorithm, to form this Pareto front.

Tsetlin Machines (TMs) are an interpretable pattern recognition approach that captures patterns with high discriminative power from data. Patterns are represented as conjunctive clauses in propositional logic, produced using bandit-learning in the form of Tsetlin Automata. In this work, we propose a TM-based approach to two common Natural Language Processing (NLP) tasks, viz. Sentiment Analysis and Semantic Relation Categorization. By performing frequent itemset mining on the patterns produced, we show that they follow existing expert-verified rule-sets or lexicons. Further, our comparison with other widely used machine learning techniques indicates that the TM approach helps maintain interpretability without compromising accuracy – a result we believe has far-reaching implications not only for interpretable NLP but also for interpretable AI in general.

State-of-the-art methods of Human Activity Recognition (HAR) rely on a considerable amount of labelled data to train deep architectures. This becomes prohibitive when tasked with creating models that are sensitive to personal nuances in human movement, explicitly present when performing exercises and when it is infeasible to collect training data to cover the whole target population. Accordingly, learning personalised models with few data remains an open challenge in HAR research. We present a meta-learning methodology for learning-to-learn personalised models for HAR; with the expectation that the end-user only need to provide a few labelled data. These personalised HAR models benefit from the rapid adaptation of a generic meta-model using provided few end-user data. We implement the personalised meta-learning methodology with two algorithms, Personalised MAML and Personalised Relation Networks. A comparative study shows significant performance improvements against state-of-the-art deep learning algorithms and other personalisation algorithms in multiple HAR domains. Also, we show how personalisation improved meta-model training, to learn a generic meta-model suited for a wider population while using a shallow parametric model.

Reinforcement Learning (RL) is a general framework concerned with an agent that seeks to maximize rewards in an environment. The learning typically happens through trial and error using explorative methods, such as $$\epsilon $$ ϵ -greedy. There are two approaches, model-based and model-free reinforcement learning, that show concrete results in several disciplines. Model-based RL learns a model of the environment for learning the policy while model-free approaches are fully explorative and exploitative without considering the underlying environment dynamics. Model-free RL works conceptually well in simulated environments, and empirical evidence suggests that trial and error lead to a near-optimal behavior with enough training. On the other hand, model-based RL aims to be sample efficient, and studies show that it requires far less training in the real environment for learning a good policy.A significant challenge with RL is that it relies on a well-defined reward function to work well for complex environments and such a reward function is challenging to define. Goal-Directed RL is an alternative method that learns an intrinsic reward function with emphasis on a few explored trajectories that reveals the path to the goal state.This paper introduces a novel reinforcement learning algorithm for predicting the distance between two states in a Markov Decision Process. The learned distance function works as an intrinsic reward that fuels the agent’s learning. Using the distance-metric as a reward, we show that the algorithm performs comparably to model-free RL while having significantly better sample-efficiently in several test environments.

Due to the high energy consumption and scalability challenges of deep learning, there is a critical need to shift research focus towards dealing with energy consumption constraints. Tsetlin Machines (TMs) are a recent approach to machine learning that has demonstrated significantly reduced energy usage compared to neural networks alike, while performing competitively accuracy-wise on several benchmarks. However, TMs rely heavily on energy-costly random number generation to stochastically guide a team of Tsetlin Automata (TA) to a Nash Equilibrium of the TM game. In this paper, we propose a novel finite-state learning automaton that can replace the TA in TM learning, for increased determinism. The new automaton uses multi-step deterministic state jumps to reinforce sub-patterns. Simultaneously, flipping a coin to skip every d’th state update ensures diversification by randomization. The d-parameter thus allows the degree of randomization to be finely controlled. E.g., $$d=1$$ d = 1 makes every update random and $$d=\infty $$ d = ∞ makes the automaton completely deterministic. Our empirical results show that, overall, only substantial degrees of determinism reduces accuracy. Energy-wise, random number generation constitutes switching energy consumption of the TM, saving up to 11 mW power for larger datasets with high d values. We can thus use the new d-parameter to trade off accuracy against energy consumption, to facilitate low-energy machine learning.

This paper presents a learning speedup method based on the relationship between the support vectors and the within-class Mahalanobis distances among the training set. We explain how statistical properties of the data can be used to pre-rank the training set. Then we explain the relationship among the pre-ranked training set indices, convex hull indices, and the support vector indices. We also explain how this method has better efficiency than those approaches based on the convex hull, especially at large-scale problems. At the end of the paper we conclude by explaining the findings of the experimental results over the speedup alternative.

An exploratory study of learning a neural network for categorisation shows that commonly used leaky integrate and fire neurons and Hebbian learning can be effective. The system learns with a standard spike timing dependent plasticity Hebbian learning rule. A two layer feed forward topology is used with a presentation mechanism of inputs followed by outputs a simulated ms. later to learn Iris flower and Breast Cancer Tumour Malignancy categorisers. An exploration of parameters indicates how this may be applied to other tasks.

Telemetry is an automatic system for monitoring environments in a remote or inaccessible area and transmitting data via various media. Data from telemetry stations can be used to produce early warning or decision supports in risky situations. However, sometimes a device in a telemetry system may not work properly and generates some errors in the data, which lead to false alarms or miss true alarms for disasters. We then developed two types of ensembles: (1) simple and (2) complex ensembles for automatically detecting the anomaly data. The ensembles were tested on the data collected from 9 telemetry water level stations and the results clearly show that the complex ensembles are the most accurate and also reliable in detecting anomalies.

Most interactions or relationships among objects or entities can be modelled as graphs. Some classes of entity relationships have their own name due to their popularity; social graphs look at people’s relationships, computer networks show how computers (devices) communicate with each other and molecules represent the chemical bonds between atoms. Some graphs can also be dynamic in the sense that, over time, relationships change. Since the entities can, to a certain extent, manage their relationships, we say any changes in relationships reflect a change in entity behaviour. By comparing the relationships of an entity at different points in time, we can say there has been a change in behaviour. In this paper, we attempt to detect malicious devices in a network by showing a significant change in behaviour through analysing traffic data.

Despite the great success of neural networks in recent years, they are not providing useful extrapolation. In regression tasks, the popular Rectified Linear Units do enable unbounded linear extrapolation by neural networks, but their extrapolation behaviour varies widely and is largely independent of the training data. Our goal is instead to continue the local linear trend at the margin of the training data. Here we introduce ReLEx, a regularising method composed of a set of loss terms design to achieve this goal and reduce the variance of the extrapolation. We present a ReLEx implementation for single input, single output, and single hidden layer feed-forward networks. Our results demonstrate that ReLEx has little cost in terms of standard learning, i.e. interpolation, but enables controlled univariate linear extrapolation with ReLU neural networks.

Fleet optimisation can significantly reduce the time vehicles spend traversing road networks leading to lower costs and increased capacity. Moreover, reduced road use leads to lower emissions and improved air quality. Heuristic approaches such as Ant Colony Optimisation (ACO) are effective at solving fleet optimisation but scale poorly when dealing with larger fleets. The Partial-ACO technique has substantially improved ACO’s capacity to optimise large scale vehicle fleets but there is still much scope for improvement. A method to achieve this could be to integrate simple mutation with Partial-ACO as used by other heuristic methods. This paper explores a range of mutation strategies for Partial-ACO to both improve solution quality and reduce computational costs. It is found that substituting a majority of ant simulations with simple mutation operations instead improves both the accuracy and efficiency of Partial-ACO. For real-world fleet optimisation problems of up to 45 vehicles and 437 jobs reductions in fleet traversal of approximately 50% are achieved with much less computational cost enabling larger scale problems to be tackled. Moreover, CO $$_{2}$$ 2 and NO $$_{\text {x}}$$ x emissions are cut by 3.75 Kg and 1.71 g per vehicle a day respectively improving urban air quality.

In many transport companies, one of the main objectives is to optimize the travel cost of their fleet. Other objectives are related to delivery time, fuel savings, etc. However warehouse stock management is not properly considered. Warehouse stock control is based on the correct allocation of resources to each order. In this paper, we combine the warehouse stock management problem and the routing problem to be applied in a real company that allows negative stock in their warehouses. The proposed multi-objective problem is modeled and solved by the greedy randomized adaptive search (GRASP) algorithm. The results shows that the proposed algorithm outperforms the current search technique used by the company mainly in stock balancing, improving the negative average stock by up to 82%.

This paper focuses on the specific scenario of capturing data in the South African agricultural industry; an industry where it can be difficult, expensive and time consuming to gather information, yet the need for information is critical. The aim is to conduct an introductory study into determining which aspects: location, irrigation, fertilizer application, temperature, or type of growing medium, has the most significant impact on the yield of edible garnish and then to predict the yield of a specific plant. A dataset collected over a three year period and supplemented with empirical knowledge and expert opinion, is analysed and a number of classifiers are applied to select the best strategy for predicting future yield of edible garnish. A random forest classifier showed the most promise and location on the farm was shown to have the largest influence on yield.

Even though the Artificial Intelligence (AI) has overtaken many decision-making responsibilities in recent years, there still exists legitimate concerns of relying on AI due to their potentially incorrect, unjustified or even unfair results. Consequently, in order to overcome these issues, additional approaches are needed to make AI systems accountable, governable, and in general, trustworthy. In this article, Semantic Technologies are suggested towards making AI systems accountable. The proposed approach enables a fine-grained traceability of the predictions generated by the AI system. Summarising, the presented proposal is expected to make an initial step in the use of Semantic Technologies for addressing different aspects that may contribute to the trustworthiness of AI systems.

Demand Response (DR) can contribute towards the energy efficiency in buildings, which is one of the major concerns among governments, scientists, and researchers. DR programs rely on the anticipation to electric demand peaks, for which the development of short-term electric demand forecasting models may be valuable. This article presents two different variants of the KNN algorithm to predict short-term electric demand for apartments located in Madrid (Spain). On the one hand, the use of an approach based on the estimation of a Machine Learning model (KNFTS) is studied. In this method, time-related and date-related features are used as exploratory variables. On the other hand, a method based on the recognition of similar patterns in the time series (KNPTS) is analysed. The Edit Distance for Real Sequences (EDR), Root Mean Square Error (RMSE) and Dynamic Time Warping (DTW) are used to measure the accuracy of forecasts for both approaches. The experiments demonstrate that the KNPTS has a higher accuracy over the KNFTS when predicting the short-term electric demand. Furthermore, the models’ adaptation to unusual situations is showcased in this article. The impact of the COVID-19 pandemic derived in a worldwide electric demand drop due to the lockdown and other confinement measures, and the retraining method proposed for the KNPTS model has been demonstrated to be valid, as it improves the forecasting accuracy.

Sea ports act as a gateway for a country’s imports and exports. Delays of vessels at the anchorage due to adverse weather events are becoming increasingly problematic. This paper investigates using weather data to accurately predict delays experienced by ships at the port anchorage by means of both regression (delay duration) and classification (delay impact). The data sets consist of five years of weather information and vessel weather delay data obtained for a South African port. The weather information consist of three data sources, including rainfall, wind and wave data. An artificial neural network was found to perform the best in the prediction of vessel weather delay duration for both three day and weekly data sets and a random forest performed the best in predicting likelihood of weekly vessel weather delays.

Software development is a complex activity requiring intelligent action. This paper explores the use of an AI technique for one step in software development, viz. detecting the location of a fault in a program. A measure of program progress is proposed, which uses a Naïve Bayes model to measure how useful the information that has been produced by the program to the task that the program is tackling. Then, deviations in that measure are used to find the location of faults in the code. Experiments are carried out to test the effectiveness of this measure.

Dynamic Time Warping (DTW) coupled with k Nearest Neighbour classification, where $$k=1$$ k = 1 , is the most common classification algorithm in time series analysis. The fact that the complexity of DTW is quadratic, and therefore computationally expensive, is a disadvantage; although DTW has been shown to be more accurate than other distance measures such as Euclidean distance. This paper presents a hybrid, Euclidean and DTW time series analysis similarity metric approach to improve the performance of DTW coupled with a candidate reduction mechanism. The proposed approach results in better performance than alternative enhanced Sub-Sequence-Based DTW approaches, and the standard DTW algorithm, in terms of runtime, accuracy and F1 score.

We performed an investigation of how several data relationship discovery algorithms can be combined to improve performance. We investigated eight relationship discovery algorithms like Cosine similarity, Soundex similarity, Name similarity, Value range similarity, etc., to identify potential links between database tables in different ways using different categories of database information. We proposed voting system and hierarchical clustering ensemble methods to reduce the generalization error of each algorithm. Voting scheme uses a given weighting metric to combine the predictions of each algorithm. Hierarchical clustering groups predictions into clusters based on similarities and then combine a member from each cluster together. We run experiments to validate the performance of each algorithm and compare performance with our ensemble methods and the state-of-the-art algorithms (FaskFK, Randomness and HoPF) using Precision, Recall and F-Measure evaluation metrics over TPCH and AdvWork datasets. Results show that performance of each algorithm is limited, indicating the importance of combining them to consolidate their strengths.

It is hoped that the traffic in the cities will be almost optimal when autonomous vehicles will dominate the traffic. We investigate the route selection of autonomous vehicles. We extend, implement and apply a formal model to support the trustworthy route selection of real-world autonomous agents. We trust a model, if the route selection strategy of the model selects routes which are close to the possible fastest all the time. The formal model extends the intention-aware online routing game model with parallel lanes, traffic lights and give way intersections. These extensions are needed for real-world applications. The actual parameters of the formal model are derived from real-world OpenStreetMap data. The large-scale real-world testing of the model uses the SUMO (Simulation of Urban MObility) open source simulator. The implemented intention-aware online routing game model can execute the route selection for each vehicle faster than real-time. Our hypothesis is that the extended intention-aware online routing game model produces at least as good traffic as the dynamic equilibrium route assignment. This hypothesis is confirmed in a real-world scenario.

Helping users to find pertinent documents within a big corpus through the use of simple queries on a search engine is a major concern in the information retrieval field. The work presented in this article combines the use of standard natural language processing methods to estimate the relevance of a document to a query with an online preference learning method to infer such kind of pertinence by analyzing the past behavior of other users making similar searches. The first contribution of this article is the proposition of a specific heuristic method, conceived for an open access online juridical corpus, to filter and interpret data collected from the user behavior while navigating on the search engine’s query interface, on the list of results, and on the documents themselves. The second contribution is an original way for combining multiarmed bandit algorithms for learning pertinence from the user implicit feedback with natural language processing techniques in order to define a unique ranking for the search results.

Recent studies on sleep reveal its impact on the well-being of humans. Monitoring of in-bed body postures can provide clinicians with early indicators of a wide range of musculoskeletal disorders. Current work on sleep pose classification is directed at non-wearable technologies, with issues associated to limited body observability and concerns over personal privacy; or on wearable sensors that consider only a small number of sleep poses and thus have limited generalisation. This paper proposes a novel method for wearable-based human pose classification capable of classifying twelve benchmark sleeping poses. To overcome the scarcity of labelled inertial data, a new data augmentation technique is proposed to generate realistic synthetic datasets emulating real-world conditions. An Error-Correcting Output Codes model is used to employ a multi-class classifier based on an ensemble of Support Vector Machine based classifiers. For system validation, a computer graphics simulator was used to accurately emulate data recording of in-bed body postures, leveraging on a standard articulated body file format commonly used by commercial motion-capture technologies. Experiments show superior performance (as high as 100% classification accuracy), and resilience to noise contamination beyond what could be encountered in reality.

Curated Document Databases play a critical role in helping researchers find relevant articles in available literature. One such database is the ORRCA (Online Resource for Recruitment research in Clinical trials) database. The ORRCA database brings together published work in the field of clinical trials recruitment research into a single searchable collection. Document databases, such as ORRCA, require year-on-year updating as further relevant documents become available on a continuous basis. The updating of curated databases is a labour intensive and time consuming task. Machine learning techniques can help to automate the update process and reduce the workload needed for screening articles for inclusion. This paper presents an automated approach to the updating of ORRCA documents repository. The proposed automated approach is a learning to rank model. The approach is evaluated using the documents in the ORRCA database. Data from the ORRCA original systematic review was used to train the learning to rank model, and data from the ORRCA 2015 and 2017 updates was used to evaluate performance of the model. The evaluation demonstrated that significant resource savings can be made using the proposed approach.

The outbreak of coronavirus disease 2019 (COVID-19) recently has affected human life to a great extent. Besides direct physical and economic threats, the pandemic also indirectly impact people’s mental health conditions, which can be overwhelming but difficult to measure. The problem may come from various reasons such as unemployment status, stay-at-home policy, fear for the virus, and so forth. In this work, we focus on applying natural language processing (NLP) techniques to analyze tweets in terms of mental health. We trained deep models that classify each tweet into the following emotions: anger, anticipation, disgust, fear, joy, sadness, surprise and trust. We build the EmoCT (Emotion-Covid19-Tweet) dataset for the training purpose by manually labeling 1,000 English tweets. Furthermore, we propose an approach to find out the reasons that are causing sadness and fear, and study the emotion trend in both keyword and topic level.

The growth of textual content in various languages and the advancement of automatic translation systems has led to an increase of cases of translated plagiarism. When a text is translated into another language, word order will change and words may be substituted by synonyms, and as a result detection will be more challenging. The purpose of this paper is to introduce a new technique for English-Arabic cross-language plagiarism detection. This method combines word embedding, term weighting techniques, and universal sentence encoder models, in order to improve detection of sentence similarity. The proposed model has been evaluated based on English-Arabic cross-lingual datasets, and experimental results show improved performance when compared with other Arabic-English cross-lingual evaluation methods presented at SemEval-2017.

Anomaly detection is one of the popular research fields in Machine Learning. Also, this is one of the key techniques in system and application monitoring in Industry. Anomaly detection comprises of outlier detection and identifying novelty from the data - it is a process to understand the deviation of an observation from existing observations [12] and identifying the new observations. Carrying out anomaly detection in an enterprise application is a challenge as there are complex processes to gather and analyze functional and non-functional logs of unlabeled data. In this paper we are proposing an unsupervised learning process with log featurization incorporating time window to detect outliers and novel errors from enterprise application logs.

Videoconferencing is becoming ubiquitous, especially so during the COVID-19 pandemic. However, user experience of a videoconference call can be variable. To better understand and classify the performance of videoconference call systems, this paper reports a case study in which active learning - an interactive form of machine learning in which system engineers provide labels for instances of feature data - is applied to videoconference call logs. Investigations reveal that although system engineers have differing videoconference domain knowledge and so provide a wide range of labels, the active learning approach produces promising results in terms of model scale, accuracy and confidence reflecting the subjectivity of engineers’ experience.

Enterprise Resource management is crucial to the success of any service organizations. Having right resource at the right time at the right place can make a big difference to the quality of their service offering. This paper focuses on spare parts management in a telecom industry as part of the enterprise resource management problem. The traditional way of moving the spare parts within the network is done manually by expert planners. However, this is not efficient as they may not have a global view of supply and demand, considering a large number of spares and potential locations that have to be taken into account when making distribution decisions. We investigate two evolutionary algorithms to solve this problem. The objective is twofold: 1) to identify and implement a permutation based Estimation of Distribution Algorithm for this problem, 2) to perform detail experimental analysis and compare the performance EDA to that of GA, with the goal of enhancing existing spare management software.

In this paper we introduce a conversational agent reminiscence companion system called Betty. Betty increases subjective wellbeing (SWB) by engaging with people and by collecting information during one-to-one conversations, then using this data in further personalised conversations with each individual. Results have shown a positive effect on SWB, and improvement in normal age-associated memory loss (NM) was evident through increased everyday interaction with Betty.