Artificial Intelligence and Soft Computing

In this paper the model of a signal transmission in a synapse of the neuron is studied. The model is based on partial differential equations. The three-dimensional simulations based on the model are presented and discussed in details. The simulations enabled to estimate the value of the coefficient of diffusion transmission of neurotransmitters in the presynaptic bouton.

The paper presents a parallel approach to the Levenberg-Marquardt algorithm (also called LM or LMA). The first section contains the mathematical basics of the classic LMA. Then the parallel modification to LMA is introduced. The classic Levenberg-Marquardt algorithm is sufficient for a training of small neural networks. For bigger networks the algorithm complexity becomes too big for the effective teaching. The main scope of this paper is to propose more complexity efficient approach to LMA by parallel computation. The proposed modification to LMA has been tested on a few function approximation problems and has been compared to the classic LMA. The paper concludes with the resolution that the parallel modification to LMA could significantly improve algorithm performance for bigger networks. Summary also contains a several proposals for the possible future work directions in the considered area.

In the paper we study global (integral) properties of the Parzen-type recursive algorithm dealing with streaming data in the presence of the time-varying noise. The mean integrated squared error of the regression estimate is shown to converge under several conditions. Simulations results illustrate asymptotic properties of the algorithm and its convergence for a wide spectrum of a time-varying noise.

In this contribution, we investigate the relationship between speakers and speech utterance, and propose a speaker normalization/adaptation model that incorporates correlation amongst the utterance classes produced by male and female speakers of varying age categories (children: 0–15; youths: 16–30; adults: 31–50; seniors: $${>}50$$). Using Principal Component Analysis (PCA), a speaker space was constructed, and based on the speaker covariance matrix obtained directly from the speech data signals, a visualisation of the first three principal components (PCs) was achieved. For effective covariance learning, a component-wise normalisation of each vector weights of the covariance matrix was performed, and a machine learning algorithm (the SOM: self organising map) implemented to model selected speaker features (F0, intensity, pulse) variability. Results obtained reveal that, for the features selected, F0 gave the most variance, as both genders exhibited high variability. For male speakers, PC1 captured the most variance of 87%, while PC2 and PC3 captured the least variances of 7% and 3%, respectively. For female speakers, PC1 captured the most variance of 97%, while PC2 and PC3 captured the least variances of 2% and 1%, respectively. Further, intensity and pulse features show close similarity patterns between the speech features, and are not most relevant for speaker variability modelling. Component planes visualisation of the respective speech patterns learned from the features covariance revealed consistent patterns, and hence, useful in speaker recognition systems.

We present here a new model and algorithm which performs an efficient Natural gradient descent for multilayer perceptrons. Natural gradient descent was originally proposed from a point of view of information geometry, and it performs the steepest descent updates on manifolds in a Riemannian space. In particular, we extend an approach taken by the “Whitened Neural Networks” model. We make the whitening process not only in the feed-forward direction as in the original model, but also in the back-propagation phase. Its efficacy is shown by an application of this “Bidirectional Whitened Neural Networks” model to a handwritten character recognition data (MNIST data).

Unmanned aerial vehicles (UAVs) are becoming very popular now. They have a variety of applications: search and rescue missions, crop inspection, 3D mapping, surveillance and military applications. However, many of the lower-end UAV do not have obstacle avoidance systems installed, which can lead to broken equipment or people may get injured. In this paper, we describe the design of low-cost UAV with computer vision based obstacle avoidance system. We used Block Match (BM) and Semi Global Block Match (SGBM) algorithms for detection of obstacles in stereo images. We constructed custom UAV platform, and demonstrated the effectiveness of UAV with an obstacle avoidance system in real-world field testing conditions.

Extreme learning machines or radial basis function networks depends on kernel functions. If the kernel set is too small or not adequate (for the problem/learning data) the learning can be fruitless and generalization capabilities of classifiers do not become rewarding.The article presents a method of automatic stochastic selection of kernels. Thanks to the proposed scheme of kernel function selection we obtain the proper number of kernels and proper placements of kernels. Evaluation results clearly show that this methodology works very well and is superior to standard extreme learning machine, support vector machine or k nearest neighbours method.

This paper introduces an innovative approach for supervised learning systems in cases when we do not have initially defined training data sets, but we need to develop them gradually during training process on the basis of the motivation factors that come from the given environment. We suppose to gradually develop and update knowledge about the environment and use it for supervision of training MLP. In the beginning, the gradually gained knowledge does not have to be correct, but it allows to adapt a neural network still better and more efficiently in time. It is illustrated on the problem of acquiring the ability to return to the starting position optimally by a virtual robot from anywhere in an initially unknown and gradually explored maze. The proposed approach focuses on the attempt to reflect human cognitive abilities and motivation factors in an introduced model using artificial neural networks. This article presents a new approach in which the decision-making method arises from the supervised learning process controlled by the knowledge gained during maze exploration. This paper presents a model of maze exploration and knowledge-based adaptation of the neural network. The experimental results of the classical supervised learning approach and the proposed modified approach will be compared to demonstrate significant improvements.

In online handwriting authentication, it is difficult to forge handwriting because stroke characteristics cannot be reproduced using only a handwriting trajectory. However, it is difficult to completely reproduce registered stroke characteristics, even when signers attempt to reproduce their own signatures. For this reason, the principal criteria for authentication must be lowered. In this study, we use a recurrent neural network to model the behavior of the musculoskeletal function for handwriting. The proposed model can represent the handwriting stroke process for a character visualized by the authenticator. This research is an anti-counterfeit effort to reduce the error between autoassociative stroke information and handwriting stroke information.

Despite existing solutions for accurate translation between written and spoken language, sign language is still not well-studied area. A reliable, robust and working in real-time translator of American Sign Language is a crucial bridge to facilitate communication between deaf and hearing people. In this paper we propose a method of sign language fingerspelling recognition using a modern architecture of convolutional neural network called Wide Residual Network trained with Snapshot Learning procedure. The model was trained on augmented datasets available at Surrey University and Massey University web pages using transfer learning. The final result is a robust classifier of all alphabet letters, which beats current state-of-the-art results. The outcomes encourage further research in this field for creating fully usable sign language translator.

The saturation of particular neuron and a whole neural network is one of the reasons for problems with training effectiveness. The paper shows neural network saturation analysis, proposes a method for detection of saturated neurons and its reduction to achieve better training performance. The proposed approach has been confirmed by several experiments.

In the paper we analyze convergence and rates of convergence of the normalized radial basis function networks by relating their $$L_2$$ error to the $$L_2$$ error of the Wolverton-Wagner regression estimate. The network parameters are learned by minimizing the empirical risk and are applied in function learning and classification.

The paper describes a unique approach to optoelectronic elements application in artificial intelligence. Previously we considered molecular neural networks on the base of the functional porous silica thin films. But, for the successful molecular neural network design, we need efficient connections among them. Therefore we are presenting a material with tuneable non-linear optical (NLO) properties to be used for the optical signal transfer. The idea is briefly described and then followed by an experimental part to validate its feasibility. Promising results show that it is possible to design and synthesize the material with tuneable NLO properties.

Convolutional neural networks (CNN) are popularly used for applications in natural language processing, video analysis and image recognition. However, the max-pooling layer used in CNNs discards most of the data, which is a drawback in applications, such as, prediction of video frames. With this in mind, we propose an adaptive prediction and classification network (APCN) based on a data-dependent pooling architecture. We formulate a combined cost function for minimizing prediction and classification errors. During testing, we identify a new class in an unsupervised fashion. Simulation results over a synthetic data set show that the APCN algorithm is able to learn the spatio-temporal information to predict and classify the video frames, as well as, identify a new class during testing.

Multi-step forecasting is very challenging and there are a lack of studies available that consist of machine learning algorithms and methodologies for multi-step forecasting. It has also been found that lack of collaborations between these different fields is creating a barrier to further developments. In this paper, multi-step time series forecasting are performed on three nonlinear electric load datasets extracted from Open-Power-System-Data.org using two machine learning models. Multi-step forecasting performance of Auto-Regressive Integrated Moving Average (ARIMA) and Long-Short-Term-Memory (LSTM) based Recurrent Neural Networks (RNN) models are compared. Comparative analysis of forecasting performance of the two models reveals that the LSTM model has superior performance in comparison to the ARIMA model for multi-step electric load forecasting.

In this paper, we propose a Deep Q-Network using reward distribution. Deep Q-Network is based on the convolutional neural network which is a representative method of Deep Learning and the Q Learning which is a representative method of reinforcement learning. In the Deep Q-Network, when the game screen (observation) is given as an input to the convolutional neural network, the action value in Q Learning for each action is output. This method can realize learning that acquires a score equal to or higher than that of a human in plural games. The Q Learning learns using the greatest value in the next action, so a positive reward is propagated. However, since negative rewards can not be of greatest value, they are not propagated in learning. Therefore, by distributing negative rewards in the same way as Profit Sharing, the proposed method learn to not take wrong actions. Computer experiments were carried out, and it was confirmed that the proposed method can learn with almost the same speed and accuracy as the conventional Deep Q-Network. Moreover, by introducing reward distribution, we confirmed that learning can be performed so as not to acquire negative reward in the proposed method.

This paper describes the development of a cognitive agent using motivated reinforcement learning. The conducted research was based on the example of a virtual robot, that placed in an unknown maze, was learned to reach a given goal optimally. The robot should expand knowledge about the surroundings and learn how to move in it to achieve a given target. The built-in motivation factors allow it to focus initially on collecting experiences instead of reaching the goal. In this way, the robot gradually broadens its knowledge with the advancement of exploration of its surroundings. The correctly formed knowledge is used for effective controlling the reinforcement learning routine to reach the target by the robot. In such a way, the motivation factors allow the robot to adapt and control its motivated reinforcement learning routine automatically and autonomously.

Bankruptcy prediction is a very important issue in business financing. Raising availability of financial data makes it more and more viable. We use large data concerning the health of Polish companies to predict their possible bankruptcy in a relatively short period. To this end, we utilize feedforward neural networks.

Successful training of artificial neural networks depends primarily on used architecture and suitable algorithm that is able to train given network. During training process error for many patterns reach low level very fast while for other patterns remains on relative high level. In this case already trained patterns make impossible to adjust all trainable network parameters and overall training error is unable to achieve desired level. The paper proposes soft pattern reduction mechanism that allows to reduce impact of already trained patterns which helps in getting better results for all training patterns. Suggested approach has been confirmed by several experiments.

Computer vision applications have been largely incorporated into robotics and industrial automation, improving quality and safety of processes. Such systems involve pattern classifiers for specific functions that, many times, demand high processing time and large data memory. Robotics applications usually deal with restricted resources platforms, in order to preserve battery time and to reduce weight and costs. To assist those applications, this paper presents an investigation on GLCM (Gray Level Co-occurrence Matrix) features and image size for an SVM (Support Vector Machines) classifier that can reduce computer resources utilization while preserving high classifier accuracy. Experimental results show a computing time on the embedded platform of 80.5 ms, with an accuracy above to 99%, to classify images of 80 $$\times $$ 60 pixels.

The usual way to learn radial basis function (RBF) networks consists of two stages: first, select reasonable weights between input and hidden layers, and then optimize weights between hidden and output layers. When we learn multilayer perceptrons (MLPs), we usually employ the stochastic descent called backpropagation (BP) algorithm or 2nd-order methods such as pseudo-Newton method and conjugate gradient method. Recently new learning methods called singularity stairs following (SSF) methods have been proposed for learning real-valued or complex-valued MLPs by making good use of singular regions. SSF can monotonically decrease training error along with the increase of hidden units, and stably find a series of excellent solutions. This paper proposes a completely new method for learning RBF networks by introducing the SSF paradigm, and compares its performance with those of existing learning methods.

The reservoir computation (RC) is a recurrent neural network architecture that is very suitable for time series prediction tasks. Its implementation in specific hardware can be very useful in relation to software approaches, especially when low consumption is an essential requirement. However, the hardware realization of RC systems is expensive in terms of circuit area and power dissipation, mainly due to the need of a large number of multipliers at the synapses. In this paper, we present an implementation of an RC network with cyclic topology (simple cyclic reservoir) in which we limit the available synapses’ weights, which makes it possible to replace the multiplications with simple addition operations. This design is evaluated to implement the equalization of a non-linear communication channel, and allows significant savings in terms of hardware resources, presenting an accuracy comparable to previous works.

Pooling operations for convolutional neural networks provide the opportunity to greatly reduce network parameters, leading to faster training time and less data overfitting. Unfortunately, many of the common pooling methods such as max pooling and mean pooling lose information about the data (i.e., they are lossy methods). Recently, spectral pooling has been utilized to pool data in the spectral domain. By doing so, greater information can be retained with the same network parameter reduction as spatial pooling. Spectral pooling is currently implemented in the discrete Fourier domain, but it is found that implementing spectral pooling in the discrete cosine domain concentrates energy in even fewer spectra. Although Discrete Cosine Transforms Spectral Pooling Layers (DCTSPL) require extra computation compared to normal spectral pooling, the overall time complexity does not change and, furthermore, greater information preservation is obtained, producing networks which converge faster and achieve a lower misclassification error.

This paper presents a method of model aggregation using multivariate decompositions where the main problem is to properly identify the components that carry noise. We develop a volatility measure which uses generalized extreme value decomposition. It is applied to destructive and constructive latent component classification. A practical experiment was conducted in order to validate the effectiveness of the introduced method.

We propose a simple way to increase the robustness of deep neural network models to adversarial examples. The new architecture obtained by stacking deep neural network and RBF network is proposed. It is shown on experiments that such architecture is much more robust to adversarial examples than the original one while its accuracy on legitimate data stays more or less the same.

In this work, a new algorithm for the structure optimization of stacked autoencoder deep network (SADN) is introduced. It relies on the search for the numbers of the neurons in the first and the second layer of SADN through an approach based on reinforcement learning (RL). The Q(0)-learning based agent is constructed, which according to received reinforcement signal, picks appropriate values for the neurons. Considered network, with the architecture adjusted by the proposed algorithm, is applied to the task of MNIST digit database recognition. The classification quality is computed for SADN to determine its performance. It is shown that, using the proposed algorithm, the semi-optimal configuration of the number of hidden neurons can be achieved much faster than the successive exploration of the entire space of layers’ arrangement.

In this work, a new efficient evolutionary algorithm to enhance the global optimization search is presented, which applies double populations, each population divided into several groups. The first population is original and the second one is a copy of the first one but with different operators are applied to it. The operators used in this paper are dynamic schema, dynamic dissimilarity, dissimilarity, similarity and a random generation of new chromosomes. This algorithm is called Multi-Dynamic Schema with Dissimilarity and Similarity of Chromosomes (MDSDSC) which is a more elaborate version of our previous DSC and DSDSC algorithms. We have applied this algorithm to 20 test functions in 2 and 10 dimensions. Comparing the MDSDSC with the classical GA, DSC, DSDSC and, for some functions, BA and PSO algorithms, we have found that, in most cases, our method is better than the GA, BA and DSC.

Many bio-inspired algorithms have been proposed to solve optimization problems. However, there is still no conclusive evidence of superiority of particular algorithms in different problems, diverse experimental situations and varied testing scenarios. Here, eight methods are investigated through extensive experimentation in three problems: (1) benchmark functions optimization, (2) wind energy forecasting and (3) data clustering. Genetic algorithms, ant colony optimization, particle swarm optimization, artificial bee colony, firefly algorithm, cuckoo search algorithm, bat algorithm and self-adaptive cuckoo search algorithm are compared, concerning, the quality of solutions according to several performance metrics and convergence to best solution. A bio-inspired technique for automatic parameter tuning was developed to estimate the optimal values for each algorithm, allowing consistent performance comparison. Experiments with thousands of configurations, 12 performance metrics and Friedman and Nemenyi statistical tests consistently evidenced that cuckoo search works efficiently, robustly and superior to the other methods in the vast majority of experiments.

This paper proposes a covariance matrix adaptation evolution strategy (CMAES) based algorithm for a robotic flow shop scheduling problem with multiple robots and parallel machines. The algorithm is compared to three popular scheduling rules as well as existing schedules at a South African anodising plant. The CMAES algorithm statistically significantly outperformed all other algorithms for the size of problems currently scheduled by the anodising plant. A sensitivity analysis was also conducted on the number of tanks required at critical stages in the process to determine the effectiveness of the CMAES algorithm in assisting the anodising plant to make business decisions.

Ten variants of migration model are compared with six adaptive differential evolution (DE) algorithms on real-world problems. Two parameters of migration model are studied experimentally. The results of experiments demonstrate the superiority of the migration models in first stages of the search process. A success of adaptive DE algorithms employed by migration model is systematically influenced by the studied parameters. The most efficient algorithm in the comparison is proposed migration model P15x50. The worst performing algorithm is adaptive DE.

Demand-side management is a useful and necessary strategy in the context of smart grids, as it allows to reduce electricity consumption in periods of increased demand, ensuring system reliability and minimizing resources wastage. In its range of activities, Demand Response programs have received great attention in recent years due to their potential impact measured in several studies. In this work, different approaches of the Particle Swarm Optimization algorithm are applied to the autonomous and distributed demand response optimization model based on energy price. In addition, a stochastic mechanism is proposed to mitigate the structural bias problem that such algorithm presents, boosting its application in the analyzed problem. Results provided by computational simulations show that the proposed approach contributes significantly to reduce the energy consumption costs in relation to tariff variations, as well as minimizing the use of residential equipment during peak hours of a group of consumers.

Visualization techniques used to visualize the optimization process of multi-objective evolutionary algorithms (MOEAs) have been discussed in the literature, predominantly in the context of aiding domain experts in decision making and in improving the effectiveness of the design optimization process. These techniques provide the decision maker with the ability to directly observe the performance of individual solutions, as well as their distribution in the approximated Pareto-optimal front. In this paper a visualization technique to study the mechanics of a MOEA, as it is solving multi-objective optimization problems (MOOPs), is discussed. The visualization technique uses a scatterplot animation to visualize the evolutionary process of the algorithms search, focusing on the changes in the population of non-dominated solutions obtained for each generation. The ability to visualize the optimization process of the algorithm provides the means to evaluate the performance of the algorithm, as well as visually observing the trade-offs between objectives.

When considering real-world optimization problems the possibility of encountering problems having constraints is quite high. Constraint handling approaches such as the penalty function and others have been researched and developed to incorporate an optimization problem’s constraints into the optimization process. With regards to multi-objective optimization, in this paper the two main approaches of incorporating constraints are explored, namely: Penalty functions and dominance based selection operators. This paper aims to measure the effectiveness of these two approaches by comparing the empirical results produced by each approach. Each approach is tested using a set of ten benchmark problems, where each problem has certain constraints. The analysis of the results in this paper showed no overall statistical difference between the effectiveness of penalty functions and dominance based selection operators. However, significant statistical differences between the constraint handling approaches were found with regards to specific performance indicators.

Breast cancer is the second cause of death of adult women in Mexico. Some of the risk factors for breast cancer that are visible in a mammography are the masses, calcifications, and the levels of mammographic density. While the first two have been studied extensively through the use of digital mammographies, this is not the case for the last one. In this paper, we address the automatic classification problem for the levels of mammographic density based on an evolutionary approach. Our solution comprises the following stages: thresholding, feature extractions, and the implementation of a genetic program. We performed experiments to compare the accuracy of our solution with other conventional classifiers. Experimental results show that our solution is very competitive and even outperforms the other classifiers in some cases.

Due to the accelerated growth of unlabeled data, unsupervised classification methods have become of great importance, and clustering is one of the main approaches among these methods. However, the performance of any clustering algorithm is highly dependent on the quality of the features used for the task. This work presents a Differential Evolution algorithm for maximizing an unsupervised clustering measure. Results are evaluated using unsupervised clustering metrics, suggesting that the Differential Evolution algorithm can achieve higher scores when compared to other feature selection methods.

The short term scheduling of batch processes is an active research field of chemical engineering, that has been addressed by many different techniques over the last decades. These approaches, however, are unable to solve long-term scheduling problems due their size, and the vast number of discrete decisions they entail. Evolutionary algorithms already proved to be efficient for some classes of large scheduling problems, and recently, the utilization of bacterial mutations has shown promising results on other fields.In this paper, an evolutionary algorithm featuring bacterial mutation is introduced to solve a case study of a single stage product scheduling problem. The solution performance of the algorithm was compared to a method from the literature. The results indicate that the proposed approach can find the optimal solution under relatively short execution times.

Open ended evolution seeks computational structures whereby creation of unbounded diversity and novelty are possible. However, research has run into a problem known as the “novelty plateau” where further creation of novelty is not observed. Using standard algorithmic information theory and Chaitin’s Incompleteness Theorem, we prove no algorithm can detect unlimited novelty. Therefore observation of unbounded novelty in computer evolutionary programs is nonalgorithmic and, in this sense, unknowable.

In this paper, the two hybrid swarm-based metaheuristic algorithms are tested and compared. The first hybrid is already existing Firefly Particle Swarm Optimization (FFPSO), which is based, as the name suggests, on Firefly Algorithm (FA) and Particle Swarm Optimization (PSO). The secondly proposed hybrid is an algorithm using the multi-swarm method to merge FA and PSO. The performance of our developed algorithm is tested and compared with the FFPSO and canonical FA. Comparisons have been conducted on five selected benchmark functions, and the results have been evaluated for statistical significance using Friedman rank test.

This paper examines the performance of the Bison Algorithm with a new running technique. The Bison Algorithm was inspired by the typical behavior of bison herds: the swarming movement of endangered bison as the exploitation factor and the running as the exploration phase of the optimization.While the original running procedure allowed the running group to scatter throughout the search space, the new approach proposed in this paper preserves the initial formation of the running group throughout the optimization process.At the beginning of the paper, we introduce the Bison Algorithm and explain the new running technique procedure. Later the performance of the adjusted algorithm is tested and compared to the Particle Swarm Optimization and the Cuckoo Search algorithm on the IEEE CEC 2017 benchmark set, consisting of 30 functions. Finally, we evaluate the meaning of the experiment outcomes for future research.

The dynamics of neural networks and evolutionary algorithms share common attributes and based on many research papers it seems to be that from dynamic point of view are both systems indistinguishable. In order to compare them mutually from this point of view, artificial neural networks, as similar as possible to natural one, are needed. In this paper is described part of our research that is focused on the synthesis of artificial neural networks. Since most current ANN structures are not common in nature, we introduce a method of a complex network synthesis using network growth model, considered as a neural network. Synaptic weights of the synthesized ANN are then trained by an evolutionary algorithm to respond to an input training set successfully.

Collective computational intelligence can be used in several ways, for example as taking the decision together by some form of a bagging ensemble or as finding the solutions by multi-objective evolutionary algorithms. In this paper we examine and compare the application of the two approaches to instance selection for creating the Pareto front of the selected subsets, where the two objectives are classification accuracy and data size reduction. As the bagging ensemble members we use DROP5 algorithms. The evolutionary algorithm is based on NSGA-II. The findings are that the evolutionary approach is faster (contrary to the popular belief) and usually provides better quality solutions, with some exceptions, were the outcome of the DROP5 ensemble is better.

There are many different varieties of population-based algorithms. They are interesting techniques for investigating of the search space of solutions and can be used, among others, to solve optimization problems. They usually start from initialization of a population of individuals, each of which encodes parameters of a single solution to the problem under consideration. After initialization, the preselected individuals are processed in a way that depends on the specifics of the algorithm. Therefore, properly implemented population initialization can significantly improve the algorithm’s operation and increase the quality of obtained results. This article describes a new population initialization algorithm. Its characteristic feature is the marginalization of those areas of the search space, in which once localized individuals were assessed as not satisfying. The proposed algorithm is of particular importance for problems in which no information is available that can improve the search procedure (black-box optimization). To test the proposed algorithm simulations were carried out using well-known benchmark functions.

Genetic Programming is a specialized form of genetic algorithms which evolve trees. This paper proposes an approach to evolve an expression tree, which is an N-Ary tree that represents a mathematical equation and that describes a given set of points in some space. The points are a set of trade-off solutions of a multi-objective optimization problem (MOOP), referred to as the Pareto Optimal Front (POF). The POF is a curve in a multi-dimensional space that describes the boundary where a single objective in a set of objectives cannot improve more without sacrificing the optimal value of the other objectives. The algorithm, proposed in this paper, will thus find the mathematical function that describes a POF after a multi-objective optimization algorithm (MOA) has solved a MOOP. Obtaining the equation will assist in finding other points on the POF that was not discovered by the MOA. Results indicate that the proposed algorithm matches the general curve of the points, although the algorithm sometimes struggles to match the points perfectly.

Emotions may not only be perceived by humans, but could also be identified and recognized by a machine. Emotion recognition through pattern analysis can be used in expert systems, lie detectors, medical emergencies, as well as during rescue operations to quickly identify people in distress. This paper describes a system capable of recognizing emotions based on the arm movement. Features extracted from 3D skeleton using Kinect sensor are classified by five commonly used machine learning techniques: K nearest neighbors, SVM, Decision tree, Neural Network and Naive Bayes. A genetic algorithm is then invoked to find the best system parameters to achieve the higher recognition rate. The system achieved 98.96% average accuracy on the experimental dataset.

This work presents a simple but effective modification of the velocity updating formula in the Particle Swarm Optimization algorithm to improve the performance of the algorithm on multi-modal problems. The well-known issue of premature swarm convergence is addressed by a repulsive mechanism implemented on a single-particle level where each particle in the population is partially repulsed from a different particle. This mechanism manages to prolong the exploration phase and helps to avoid many local optima. The method is tested on well-known and typically used benchmark functions, and the results are further tested for statistical significance.

The article presents an Evolutionary System designed to solve optimization problems. The system consists of Genetic Algorithm and Evolutionary Strategy, working together to improve the efficiency of optimization and increase the resistance to stuck to suboptimal solutions. In the system, we combined the ability of the Genetic Algorithm to explore the search space and the ability of the Evolutionary Strategy to exploit the search space. The system maintains the right balance between the ability to explore and exploit the search space. Genetic Algorithm and Evolutionary Strategy can exchange information about the solutions found till now and periodically migrate the best individuals between populations. The efficiency of the system has been investigated by an example of function optimization. The results of the experiments suggest that the proposed system can be an effective tool in solving complex optimization problems.

A horizontal (or lateral) gene transfer, well known in biology is used as an additional mutation factor in genetic algorithms used for optimization. Numerical results indicate the usefulness of this concept for problems of moderate size.

Evolutionary-based approaches have recently been increasingly proposed for data mining tasks, but their real applicability depends on efficiency and scalability for large-scale data. It is clear that parallel and distributed processing support is indispensable herein. Apache Spark is one of the most promising cluster-computing engines for Big Data. In this paper, we investigate the application of Spark to speed up an evolutionary induction of classification trees in the Global Decision Tree (GDT) system. The system simultaneously searches for the tree structure and tests in non-terminal nodes due to specialized genetic operators. As the original GDT system is implemented in C++, the Java-based module is developed for Spark-based acceleration of the most computationally demanding fitness evaluation. The training dataset is transformed to Resilient Distributed Dataset, which enables in-memory processing of dataset’s parts on workers. Preliminary experimental validation on large-scale artificial and real-life datasets shows that the proposed solution is efficient and scales well.

This research focuses on the modern hybridization of the discrete chaotic dynamics and the evolutionary computation. It is aimed at the influence of chaotic sequences on the population diversity as well as at the algorithm performance of the simple parameter adaptive Differential Evolution (DE) strategy: jDE. Experiments are focused on the extensive investigation of totally ten different randomization schemes for the selection of individuals in DE algorithm driven by the default pseudo random generator of Java environment and nine different two-dimensional discrete chaotic systems, as the chaotic pseudo-random number generators. The population diversity and jDE convergence are recorded for 15 test functions from the CEC 2015 benchmark set in 30D.

Due to the growing need for metaheuristics with features that allow their implementation for real-time problems, this paper proposes an adaptive individual inertia weight in each iteration considering global and individual analysis, i.e., the best, worst and individual particles’ performance. As a result, the proposed adaptive individual inertia weight presents faster convergence for the Particle Swarm Optimization (PSO) algorithm when compared to other inertia mechanisms. The proposed algorithm is also suitable for real-time problems when the actual optimum is difficult to be attained, since a feasible and optimized solution is found in comparison to an initial solution. In this sense, the PSO with the proposed adaptive individual inertia weight was tested using eight benchmark functions in the continuous domain. The proposed PSO was compared to other three algorithms, reaching better optimized results in six benchmark functions at the end of 2000 iterations. Moreover, it is noteworthy to mention that the proposed adaptive individual inertia weight features rapid convergence for the PSO algorithm in the first 1000 iterations.

Manufacturing scheduling strategies have historically ignored the availability of the machines. The more realistic the schedule, more accurate the calculations and predictions. Availability of machines will play a crucial role in the Industry 4.0 smart factories. In this paper, a mixed integer linear programming model (MILP) and a discrete firefly algorithm (DFA) are proposed for an extended multi-objective FJSP with availability constraints (FJSP-FCR). Several standard instances of FJSP have been used to evaluate the performance of the model and the algorithm. New FJSP-FCR instances are provided. Comparisons among the proposed methods and other state-of-the-art reported algorithms are also presented. Alongside the proposed MILP model, a Genetic Algorithm is implemented for the experiments with the DFA. Extensive investigations are conducted to test the performance of the proposed model and the DFA. The comparisons between DFA and other recently published algorithms shows that it is a feasible approach for the stated problem.

In this paper, a prolonged exploration ability of distance based parameter adaptation is subject to a test via clustering analysis of the population in Success-History based Adaptive Differential Evolution (SHADE). The comparative study is done on the CEC 2015 benchmark set in two dimensional settings – 10D and 30D. It is shown, that the exploration phase of distance based adaptation in SHADE (Db_SHADE) lasts for more generations and therefore avoids the premature convergence into local optima.

This paper investigates the use of evolutionary and other meta-heuristic algorithms for routing problems where vehicle operating cost (VOC) and specifically, road roughness, has a significant impact. Three algorithms were implemented, namely a greedy heuristic, simulated annealing and CMA-ES. Simulated annealing delivered statistically significant results and was used to evaluate routes with and without VOC.

One of the most important steps in the formation of multiple classifier systems is the fusion process. The fusion process may be applied either to class labels or confidence levels (discriminant functions). In this paper, we propose an integration process which takes place in the geometry space. It means that the fusion of base classifiers is done using decision boundaries. In our approach, the final decision boundary is calculated by using the harmonic mean. The algorithm presented in the paper concerns the case of 3 basic classifiers and two-dimensional features space. The results of the experiment based on several data sets show that the proposed integration algorithm is a promising method for the development of multiple classifiers systems.

We propose a method to measure similarity of users based on their sparse location history such as geo-tagged photos or check-in activity of user. The method is useful when complete movement trajectories are not available. We map each activity point into the nearest location in a predefined set of fixed places. The problem is then formulated as histogram comparison. We compare the performance of similarity measures such as L₁, L₂, L_∞, ChiSquared, Bhattacharyya and Kullback and Leibler divergence using both crisp and fuzzy histograms. Results show that user can be recognized with fair accuracy, and that all similarity measures are suitable except L₂ and L_∞, which perform poorly.

We propose to use medoid-shift to reduce the noise in data prior to clustering. The method processes every point by calculating its k-nearest neighbors (k-NN), and then replacing the point by the medoid of its neighborhood. The process can be iterated. After the data cleaning process, any clustering algorithm can be applied that is suitable for the data.

The article presents a sales lead classification method using an adapted version of the Bag-of-Words algorithm. The data collected on the website of a financial institution and evaluated by that institution undergo a classification process. It is expected that the customer submitting data through a web form should be a person interested in a particular financial product. It often happens that instead of a person, i.e. a human user, it is a bot – a computer program that simulates human behavior. However, bots deliver lower quality sales leads. The way in which a web form is handled by a bot differs from the way in which it is completed by a human user. It is therefore possible to analyze the behavior on the website and to link it with the evaluation of the submitted data. The Bag-of-Words algorithm has been adapted to deal with this particular task. Experimental research based on the real-life data obtained from a bank shows how effective this algorithm is in the sales leads quality classification.

In machine learning, Case Based Reasoning is a prominent technique for harvesting knowledge from past experiences. The past experiences are represented in the form of a repository of cases having a set of features. But each feature may not have the equal relevancy in describing a case. Measuring the relevancy of each feature is always a prime issue. A subset of relevant features describes a case with adequate accuracy. An appropriate subset of relevant features should be selected for improving the performance of the system and to reduce dimensionality. In case based domain, feature selection is a process of selecting an appropriate subset of relevant features. There are various real domains which are inherently case based and features are expressed in terms of linguistic variables. To assign a numerical weight to each linguistic feature, a lot of feature subset selection algorithms have been proposed. But the weighting values are usually determined using subjective judgement or a trial and error basis.This work presents an alternative concept in this direction. It can be efficiently applied to select the relevant linguistic features by measuring the probability in term of numerical values. It can also rule out irrelevant and noisy features. Applications of this approach in various real world domain show an excellent performance.

Text classification is an important problem in Natural Language Processing. It differs from many other classification tasks by the large number of features that have to be used during training. One of the solution for reducing dimensionality of feature space, is the usage of Latent Dirichlet Allocation. After this step, the smaller problem can be solved using standard classifiers. In [11], authors propose combination of LDA and Softmax classifier called Multi-class sLDA, that does both tasks simultaneously. However, to use the method, we have to choose a number of topics - hyperparameter of the model. This step requires analysis and human supervision. In this paper, we propose Boost Multi-class sLDAmodel, based on ensemble of many Multi-class sLDA models, that does not require the choice of topic number. Moreover, our model achieves significantly better classification accuracy, than Multi-class sLDA for any number of topics.

This paper presents the results of an in-depth analysis of the impact of aggregation of different parts of the face to its recognition process. A novel approach is based on the aggregation of distances determined between histograms, which describe different parts of the face as well as various color channels. In addition, we propose to include thresholding to local descriptors and demonstrate that this type of image processing highly improves the accuracy of classification process. This paper also describes a new approach to converting color images to grayscale images using the variation of each channel in the neighborhood of a given pixel.

Frequently, high-dimensional features are used to represent data to be classified. This paper proposes a new approach to learn interpretable classification models from such high-dimensional data representation. To this end, we extend a popular prototype-based classification algorithm, the matrix learning vector quantization, to incorporate an enhanced feature selection objective via $$L_1$$-regularization. In contrast to previous work, we propose a framework that directly optimizes this objective using the alternating direction method of multipliers (ADMM) and manifold optimization. We evaluate our method on synthetic data and on real data for speech-based emotion recognition. Particularly, we show that our method achieves state-of-the-art results on the Berlin Database of Emotional speech and show its abilities to select relevant dimensions from the eGeMAPS set of audio features.

We propose a modified classifier that is based on the maximum a posteriori probability principle that is applied to images having the matrix normal distributions. These distributions have a special covariance structure, which is interpretable and easier to estimate than general covariance matrices. The modification is applicable when the estimated covariance matrices are still not well-conditioned. The proposed classifier is tested on synthetic images and on images of gas burner flames. The results of comparisons with other classifiers are also provided.

We study how much the k-means can be improved if initialized by random projections. The first variant takes two random data points and projects the points to the axis defined by these two points. The second one uses furthest point heuristic for the second point. When repeated 100 times, cluster level errors of a single run of k-means reduces from CI = 4.5 to 0.8, on average. We also propose simple projective indicator that predicts when the projection-heuristic is expected to work well.

The relational mountain clustering method (RMCM) is a simple and effective algorithm that can be used to obtain cluster centers and partitions for a relational data set. However, the performance of RMCM heavily depends on the choice of parameters of relational mountain function. In order to solve this problem, we propose a modified RMCM (M-RMCM) by using the correlation self-comparison method to estimate the parameters of the modified relational mountain function, and then applied a validity index to estimate the number of clusters. The proposed M-RMCM can provide good cluster centers, partitions and the number of clusters for most relational data sets in which the results will not be sensitive to parameters. The simulations and comparisons show the superiority and effectiveness of the proposed M-RMCM.

Our aim is to show that randomly generated transformation of high-dimensional data vectors, for example, images, could provide low dimensional features which are stable and suitable for classification tasks. We examine two types of projections: (a) global random projections, i.e., projections of the whole images, and (b) concatenated local projections of spatially-organized parts of an image (for example rectangular image blocks). In both cases, the transformed images provide good features for correct classification. The computational complexity of designing the transformation is linear with respect to the size of images and in case (b) it does not depend on the form of image partition. We have analyzed the stability of classification results with respect to random projection and to different randomly generated training sets. Experiments on the images of ten persons taken from the Extended Yale Database B demonstrate that the methods of classification based on Gaussian random projection are effective and positively comparable with PCA-based methods, both from the point of view of stability and classification accuracy.

The paper presents the application of classification based approach to software quality domain. In particular it deals with the issue of reducing the cost of mutation testing. The presented approach is based on the similarity of mutants represented at the bytecode level. The distance matrix for mutants is used in kNN algorithm to predict if a given test set detects a mutant or not. Experimental results are also presented in this paper on the basis of two systems. The obtained results show the usefulness of the proposed method.

Classification learning by prototype based approaches is an attractive strategy to achieve interpretable classification models. Frequently, those models optimize the classification error or an approximation thereof. Current deep network approaches use the cross entropy maximization instead. Therefore, we propose a prototype based classifier based on cross-entropy as a probabilistic classifier. As we deduce, the proposed probabilistic classifier is a generalization of the robust soft-learning vector quantizer and allows to handle label noise in training data, i.e. the classifier is able to take into account probabilistic class assignments during learning.

The evaluation of cluster and classification models in comparison to ground truth information or other models is still an objective for many applications. Frequently, this leads to controversy debates regarding the informative content. This particularly holds for cluster evaluations. Yet, for imbalanced class cardinalities, similar problems occur. One possibility to handle evaluation tasks in a more natural way is to consider comparisons in terms of shared or non-shared information. Information theoretic quantities like mutual information and divergence are designed to answer respective questions. Besides formulations based on the most prominent Shannon-entropy, alternative definitions based on relaxed entropy definitions are known. Examples are Rényi- and Tsallis-entropies. Obviously, the use of those entropy concepts result in an readjustment of mutual information etc. and respective evaluation measures thereof. In the present paper we consider several information theoretic evaluation measures based on different entropy concepts and compare them theoretically as well as regarding their performance in applications.

In the paper, we discuss IBL - Instance-Based Learning - as a method of acquiring knowledge, and apply it to the verification of the shape symmetry/asymmetry of the skin lesions. The test verifying whether the asymmetry of the lesion presented in PH2 dataset is conducted using IB3 algorithms. We also verify the construction of the DASMShape asymmetry measure and its results. We achieved classification ratio on DAS values from PH2 around 59% in comparison to 84% achieved on the defined DASMShape measure. It implies that the data verification using IBL algorithms is very vital in order to design reliable dermatological diagnosis supporting systems.

In this paper, we propose and compare a few methods of representing a stroke as a vector of numbers. For each method, we describe, how to calculate the fuzzy measure of two strokes similarity. Vectors are determined on the basis of polynomials calculated by a stroke approximation.