Learning and Intelligent Optimization

The misclassification error which is usually used in tests to compare classification algorithms, does not make a distinction between the sources of error, namely, false positives and false negatives. Instead of summing these in a single number, we propose to collect multivariate statistics and use multivariate tests on them. Information retrieval uses the measures of precision and recall, and signal detection uses true positive rate (tpr) and false positive rate (fpr) and a multivariate test can also use such two values instead of combining them in a single value, such as error or average precision. For example, we can have bivariate tests for (precision, recall) or (tpr, fpr). We propose to use the pairwise test based on Hotelling’s multivariate

T

2

test to compare two algorithms or multivariate analysis of variance (MANOVA) to compare

L

 > 2 algorithms. In our experiments, we show that the multivariate tests have higher power than the univariate error test, that is, they can detect differences that the error test cannot, and we also discuss how the decisions made by different multivariate tests differ, to be able to point out where to use which. We also show how multivariate or univariate pairwise tests can be used as post-hoc tests after MANOVA to find cliques of algorithms, or order them along separate dimensions.

Hyperheuristics have successfully been used in the past for a number of search and optimization problems. To the best of our knowledge, they have not been used for financial forecasting. In this paper we use a simple hyperheuristics framework to investigate whether we can improve the performance of a financial forecasting tool called EDDIE 8. EDDIE 8 allows the GP (Genetic Programming) to search in the search space of indicators for solutions, instead of using pre-specified ones; as a result, its search area is quite big and sometimes solutions can be missed due to ineffective search. We thus use two different heuristics and two different mutators combined under a simple hyperheuristics framework. We run experiments under five datasets from FTSE 100 and discover that on average, the new version can return improved solutions. In addition, the rate of missing opportunities reaches it’s minimum value, under all datasets tested in this paper. This is a very important finding, because it indicates that thanks to the hyperheuristics EDDIE 8 has the potential of missing less forecasting opportunities. Finally, results suggest that thanks to the introduction of hyperheuristics, the search has become more effective and more areas of the space have been explored.

Recently, the property of connectedness has been claimed to give a strong motivation on the design of local search techniques for multiobjective combinatorial optimization. Indeed, when connectedness holds, a basic Pareto local search, initialized with at least one non-dominated solution, allows to identify the efficient set exhaustively. However, this becomes quickly infeasible in practice as the number of efficient solutions typically grows exponentially with the instance size. As a consequence, we generally have to deal with a limited-size approximation, ideally a representative sample of efficient solutions. In this paper, we propose the biobjective long and multiple path problems. We show experimentally that, on the first problem, even if the efficient set is connected, a local search may be outperformed by a simple evolutionary algorithm in the sampling of the efficient set. At the opposite, on the second problem, a local search algorithm may successfully approximate a disconnected efficient set. Then, we argue that connectedness is not the single property to study for the design of multiobjective local search algorithms. This work opens new discussions on a proper definition of multiobjective fitness landscapes.

In this work, our objective is to study the impact of knowledge sharing on the performance of portfolio-based parallel local search algorithms. Our work is motivated by the demonstrated importance of clause-sharing in the performance of complete parallel SAT solvers. Unlike complete solvers, state-of-the-art local search algorithms for SAT are not able to generate redundant clauses during their execution. In our settings, each member of the portfolio shares its best configuration (i.e., one which minimizes conflicting clauses) in a common structure. At each restart point, instead of classically generating a random configuration to start with, each algorithm aggregates the shared knowledge to carefully craft a new starting point. We present several aggregation strategies and evaluate them on a large set of problems.

In this paper we present a variable neighborhood search for time-dependent vehicle routing problems with time windows. Unlike the well-studied routing problems with constant travel times, in the time-dependent case the travel time depends on the time of the day. This assumption approaches reality, in particular for urban areas where travel times typically vary during the day, e.g., because of traffic congestion due to rush hours. An experimental evaluation for the vehicle routing problem with soft time windows with and without time-dependent travel times is performed and it is shown that taking time-dependent travel times into account provides substantial improvements of the considered objective function.

The two-dimensional bin packing problem (2BP) consists in packing a set of rectangular items into rectangular, equally-sized bins. The problem is NP-hard and has a multitude of real world applications. We consider the case where the items are oriented and guillotine cutting is free. In this paper we first present a review of well-know heuristics for the 2BP and then propose a new ILP model for the problem. Moreover, we develop a multi-start algorithm based on a probabilistic version of the LGFi heuristic from the literature. Results are compared to other well-known heuristics, using data sets provided in the literature. The obtained experimental results show that the proposed algorithm returns excellent solutions. With an average percentage deviation of 1.8% from the best know lower bounds it outperformes the other algorithms by 1.1% − 5.7%. Also for 3 of the 500 instances we tested a new upper bound was found.

In this work, we analyze genetic diversity of Pareto optimal solutions (POS) and study effective crossover operators in evolutionary many-objective optimization. First we examine the diversity of genes in the true POS on many-objective 0/1 knapsack problems with up to 20 items (bits), showing that genes in POS become noticeably diverse as we increase the number of objectives. We also verify the effectiveness of conventional two-point crossover, Local Recombination that selects mating parents based on proximity in objective space, and two-point and uniform crossover operators Controlling the maximum number of Crossed Genes (CCG). We use NSGA-II, SPEA2, IBEA

ε

 + 

and MSOPS, which adopt different selection methods, and many-objective 0/1 knapsack problems with

n

 = {100,250,500,750,1000} items (bits) and

m

 = {2,4,6,8,10} objectives to verify the search performance of each crossover operator. Simulation results reveal that Local Recombination and CCG operators significantly improve search performance especially for NSGA-II and MSOPS, which have high diversity of genes in the population. Also, results show that CCG operators achieve higher search performance than Local Recombination for

m

 ≥ 4 objectives and that their effectiveness becomes larger as the number of objectives

m

increases.

This paper formalizes the problem of choosing online the number of explorations in a local search algorithm as a last-success problem. In this family of stochastic problems the events of interest belong to two categories (success or failure) and the objective consists in predicting when the last success will take place. The application to a local search setting is immediate if we identify the success with the detection of a new local optimum. Being able to predict when the last optimum will be found allows a computational gain by reducing the amount of iterations carried out in the neighborhood of the current solution. The paper proposes a new algorithm for online calibration of the number of iterations during exploration and assesses it with a set of continuous optimisation tasks.

In multiobjective combinatorial optimization, there exists two main classes of metaheuristics, based either on multiple aggregations, or on a dominance relation. As in the single-objective case, the structure of the search space can explain the difficulty for multiobjective metaheuristics, and guide the design of such methods. In this work we analyze the properties of multiobjective combinatorial search spaces. In particular, we focus on the features related the efficient set, and we pay a particular attention to the correlation between objectives. Few benchmark takes such objective correlation into account. Here, we define a general method to design multiobjective problems with correlation. As an example, we extend the well-known multiobjective

NK

-landscapes. By measuring different properties of the search space, we show the importance of considering the objective correlation on the design of metaheuristics.

This paper is concerned with automated tuning of parameters in local-search based meta-heuristics. Several generic approaches have been introduced in the literature that returns a ”one-size-fits-all” parameter configuration for all instances. This is unsatisfactory since different instances may require the algorithm to use very different parameter configurations in order to find good solutions. There have been approaches that perform instance-based automated tuning, but they are usually problem-specific. In this paper, we propose CluPaTra, a generic (problem-independent) approach to perform parameter tuning, based on CLUstering instances with similar PAtterns according to their search TRAjectories. We propose representing a search trajectory as a directed sequence and apply a well-studied sequence alignment technique to cluster instances based on the similarity of their respective search trajectories. We verify our work on the Traveling Salesman Problem (TSP) and Quadratic Assignment Problem (QAP). Experimental results show that CluPaTra offers significant improvement compared to ParamILS (a one-size-fits-all approach). CluPaTra is statistically significantly better compared with clustering using simple problem-specific features; and in comparison with the tuning of QAP instances based on a well-known distance and flow metric classification, we show that they are statistically comparable.

The main drawback of most metaheuristics is the absence of effective stopping criteria. Most implementations stop after performing a given maximum number of iterations or a given maximum number of consecutive iterations without improvement in the best known solution value, or after the stabilization of the set of elite solutions found along the search. We propose probabilistic stopping rules for randomized metaheuristics such as GRASP and VNS. We first show experimentally that the solution values obtained by GRASP fit a Normal distribution. Next, we use this approximation to obtain an online estimation of the number of solutions that might be at least as good as the best known at the time of the current iteration. This estimation is used to implement effective stopping rules based on the trade off between solution quality and the time needed to find a solution that might improve the best found to date. This strategy is illustrated and validated by a computational study reporting results obtained with some GRASP heuristics.

Real-world engineering design optimization problems often rely on computationally-expensive simulations to replace laboratory experiments. A common optimization approach is to approximate the expensive simulation with a computationally cheaper model resulting in a model-assisted optimization algorithm. A prevalent issue in such optimization problems is that the simulation may crash for some input vectors, a scenario which increases the optimization difficulty and results in wasted computer resources. While a common approach to handle such vectors is to assign them a penalized fitness and incorporate them in the model training set this can result in severe model deformation and degrade the optimization efficacy. As an alternative we propose a classifier-assisted framework where a classifier is incorporated into the optimization search and biases the optimizer away from vectors predicted to crash to simulator and with no model deformation. Performance analysis shows the proposed framework improves performance with respect to the penalty approach and that it may be possible to ‘knowledge-mine’ the classifier as a post-optimization stage to gain new insights into the problem being solved.

We consider the problem of optimizing a real-valued continuous function

f

, which is supposed to be expensive to evaluate and, consequently, can only be evaluated a limited number of times. This article focuses on the Bayesian approach to this problem, which consists in combining evaluation results and prior information about

f

in order to efficiently select new evaluation points, as long as the budget for evaluations is not exhausted.

The algorithm called efficient global optimization (EGO), proposed by Jones, Schonlau and Welch (

J. Global Optim.

, 13(4):455–492, 1998), is one of the most popular Bayesian optimization algorithms. It is based on a sampling criterion called the expected improvement (EI), which assumes a Gaussian process prior about

f

. In the EGO algorithm, the parameters of the covariance of the Gaussian process are estimated from the evaluation results by maximum likelihood, and these parameters are then plugged in the EI sampling criterion. However, it is well-known that this plug-in strategy can lead to very disappointing results when the evaluation results do not carry enough information about

f

to estimate the parameters in a satisfactory manner.

We advocate a fully Bayesian approach to this problem, and derive an analytical expression for the EI criterion in the case of Student predictive distributions. Numerical experiments show that the fully Bayesian approach makes EI-based optimization more robust while maintaining an average loss similar to that of the EGO algorithm.

Hidden Markov Models (HMMs) are very popular generative models for time series data. Recent work, however, has shown that for many tasks Conditional Random Fields (CRFs), a type of discriminative model, perform better than HMMs. Information extraction is the task of automatically extracting instances of specified classes or relations from text. A method for information extraction using Hierarchical Hidden Markov Models (HHMMs) has already been proposed. HHMMs, a generalization of HMMs, are generative models with a hierarchical state structure. In previous research, we developed the Hierarchical Hidden Conditional Random Field (HHCRF), a discriminative model corresponding to HHMMs. In this paper, we propose information extraction using HHCRFs, and then compare the performance of HHMMs and HHCRFs through an experiment.

Non convex mixed integer non-linear programming problems (MINLPs) are the most general form of global optimization problems. Such problems involve both discrete and continuous variables with several active non-linear equality and inequality constraints. In this paper, a new approach for solving MINLPs is presented using adaptive resolution based micro genetic algorithms with local search. Niching is incorporated in the algorithm by using a technique inspired from the tabu search algorithm. The proposed algorithm adaptively controls the intensity of the genetic search in a given sub-solution space, i.e. promising regions are searched more intensely as compared to other regions. The algorithm reduces the chances of convergence to a local minimum by maintaining a list of already visited minima and penalizing their neighborhoods. This technique is inspired from the tabu list strategy used in the tabu search algorithm. The proposed technique was able to find the best-known solutions to extremely difficult MINLP/NLP problems in a competitive amount of time. The results section discusses the performance of the algorithm and the effect of different operators by using a variety of MINLP/NLPs from different problem domains.

In this paper we propose a self-adaptive algorithm for noise robust abnormality detection on ECG data. For extracting features from ECG signals, we propose a feature extraction method by characterizing the magnitude, frequency and phase information of ECG signal as well as the temporal dynamics in time and frequency domains. At abnormality detection stage, we employ the subspace method for adaptively modeling the principal pattern subspace of ECG signal in unsupervised manner. Then, we measure the dissimilarity between the test signal and the trained major pattern subspace. The atypical periods can be effectively discerned based on such dissimilarity degree. The experimental results validate the effectiveness of the proposed approach for mining abnormalities of ECG signal including promising performance, high efficiency and robust to noise.

Evolutionary computation is inspired by nature in order to formulate metaheuristics capable to optimize several kinds of problems. A family of algorithms has emerged based on this idea; e.g. genetic algorithms, evolutionary strategies, particle swarm optimization (PSO), ant colony optimization (ACO), etc. In this paper we show a population-based metaheuristic inspired on the gravitational forces produced by the interaction of the masses of a set of bodies. We explored the physics knowledge in order to find useful analogies to design an optimization metaheuristic. The proposed algorithm is capable to find the optima of unimodal and multimodal functions commonly used to benchmark evolutionary algorithms. We show that the proposed algorithm (Gravitational Interactions Optimization - GIO) works and outperforms PSO with niches in both cases. Our algorithm does not depend on a radius parameter and does not need to use niches to solve multimodal problems. We compare GIO with other metaheuristics with respect to the mean number of evaluations needed to find the optima.

Solving efficiently complex problems using metaheuristics, and in particular local search algorithms, requires incorporating knowledge about the problem to solve. In this paper, the permutation flowshop problem is studied. It is well known that in such problems, several solutions may have the same fitness value. As this neutrality property is an important issue, it should be taken into account during the design of search methods. Then, in the context of the permutation flowshop, a deep landscape analysis focused on the neutrality property is driven and propositions on the way to use this neutrality in order to guide the search efficiently are given.

In this work we present a Reinforcement Learning approach for the Flexible Job Shop Scheduling problem. The proposed approach follows the ideas of the hierarchical approaches and combines learning and optimization in order to achieve better results. Several problem instances were used to test the algorithm and to compare the results with those reported by previous approaches.

This paper introduces a framework in which dispatching rules for job-shop scheduling problems are discovered by analysing the characteristics of optimal solutions. Training data is created via randomly generated job-shop problem instances and their corresponding optimal solution. Linear classification is applied in order to identify good choices from worse ones, at each dispatching time step, in a supervised learning fashion. The method is purely data-driven, thus less problem specific insights are needed from the human heuristic algorithm designer. Experimental studies show that the learned linear priority dispatching rules outperforms common single priority dispatching rules, with respect to minimum makespan.

Optimizing parameter settings is an important task in algorithm design. Several automated parameter tuning procedures/configurators have been proposed in the literature, most of which work effectively when given a good initial range for the parameter values. In the Design of Experiments (DOE), a good initial range is known to lead to an optimum parameter setting. In this paper, we present a framework based on DOE to find a good initial range of parameter values for automated tuning. We use a factorial experiment design to first screen and rank all the parameters thereby allowing us to then focus on the parameter search space of the important parameters. A model based on the Response Surface methodology is then proposed to define the promising initial range for the important parameter values. We show how our approach can be embedded with existing automated parameter tuning configurators, namely ParamILS and RCS (Randomized Convex Search), to tune target algorithms and demonstrate that our proposed methodology leads to improvements in terms of the quality of the solutions.

We propose a metamodel approach to the approximation of functions gradients within a hybrid genetic algorithm. The underlying structure is implemented in order to support parallel execution of the code: a genetic and a SQP algorithm run in different threads and can ask designs evaluations independently, but keeping all the available resources always working. A common archive collects the results and generates the population for the GA and the starting points for the SQP runs. A particular attention is dedicated to elitism and to constraints. The hybridization is performed through a modified

ε

 −constrained method. The general philosophy of the algorithm is to concentrate on not wasting information: metamodels, archiving and elitism, steady-state parallel evolution are key elements for this scope and they will be discussed in details. A preliminary but explanatory row of tests concludes the paper highlighting the benefits of this new approach.

Genetic programming is a good technique for finding near-global optimal solutions for complex problems, by finding the program used to solve the problems. One of these complex problems is designing stream cipher systems automatically. Steam cipher is an important encryption technique used to protect private information from an unauthorized access, and it plays an important role in the communication and storage systems. In this work, we propose a new approach for designing stream cipher systems of good properties, such as high degree of security and efficiency. The proposed approach is based on the genetic programming. Three algorithms are presented here, which are simple genetic programming, simulated annealing programming, and adaptive genetic programming. Experiments were performed to study the effectiveness of these algorithms in solving the underlying problem.

In practice, combinatorial optimization problems are complex and computationally time-intensive. Local search algorithms are powerful heuristics which allow to significantly reduce the computation time cost of the solution exploration space. In these algorithms, the multi-start model may improve the quality and the robustness of the obtained solutions. However, solving large size and time-intensive optimization problems with this model requires a large amount of computational resources. GPU computing is recently revealed as a powerful way to harness these resources. In this paper, the focus is on the multi-start model for local search algorithms on GPU. We address its re-design, implementation and associated issues related to the GPU execution context. The preliminary results demonstrate the effectiveness of the proposed approaches and their capabilities to exploit the GPU architecture.

We address the problem of automated discovery of preferred solutions by an interactive optimization procedure. The algorithm iteratively learns a utility function modeling the quality of candidate solutions and uses it to generate novel candidates for the following refinement. We focus on combinatorial utility functions made of weighted conjunctions of Boolean variables. The learning stage exploits the sparsity-inducing property of 1-norm regularization to learn a combinatorial function from the power set of all possible conjunctions up to a certain degree. The optimization stage uses a stochastic local search method to solve a weighted MAX-SAT problem. We show how the proposed approach generalizes to a large class of optimization problems dealing with satisfiability modulo theories. Experimental results demonstrate the effectiveness of the approach in focusing towards the optimal solution and its ability to recover from suboptimal initial choices.

This paper introduces a genetic algorithm enhanced with a trap escaping strategy derived from the dual information presented as discrete Lagrange multipliers. When the genetic algorithm is trapped into a local optima, the Discrete Lagrange Multiplier method is called for the best individual found. The information provided by the Lagrangian method is unified, in the form of recombination, with the one from the last population of the genetic algorithm. Then the genetic algorithm is restarted with this new improved configuration. The proposed algorithm is tested on the winner determination problem. Experiments are conducted using instances generated with the combinatorial auction test suite system. The results show that the method is viable.

We consider a multi-location inventory system where inventory choices at each location are centrally coordinated through the use of lateral Transshipments. This cooperation between different locations of the same echelon level often leads to cost reduction and service level improvement. However, when some locations face embarrassing storage capacity limits, inventory sharing through transshipment may cause undesirable lead time. In this paper, we propose a more realistic multiobjective transshipment model which optimizes three conflicting objectives: (1) minimizing the aggregate cost, (2) maximizing the fill rate and (3) minimizing the transshipment lead time, in the presence of different storage capacity constraints. We improve the performance of the well-known evolutionary multiobjective algorithm SPEA2 by adequately applying a multiobjective quasi-gradient local search to some candidate solutions that have lower density estimation. The resulting hybrid evolutionary algorithm outperforms NSGA-II and the original SPEA2 in both spread and convergence. It is also shown that lateral transshipments constitute an efficient inventory repairing mechanism in a wide range of system configurations.

This paper proposes a hybrid evolutionary algorithm (EA) dealing with population-based incremental learning (PBIL) and some efficient local search strategies. A simple PBIL using real codes is developed. The evolutionary direction and approximate gradient operators are integrated to the main procedure of PBIL. The method is proposed for single objective global optimization. The search performance of the developed hybrid algorithm for box-constrained optimization is compared with a number of well-established and newly developed evolutionary algorithms and meta-heuristics. It is found that, with the given optimization settings, the proposed hybrid optimizer outperforms the other EAs. The new derivative-free algorithm can maintain outstanding abilities of EAs.

This paper presents a study for the dynamic selection of operators in a local search process. The main purpose is to propose a generic autonomous local search method which manages operator selection from a set of available operators, built on neighborhood relations and neighbor selection functions, using the concept of Pareto dominance with respect to quality and diversity. The latter is measured using two different metrics. This control method is implemented using the

Comet

language in order to be easily introduced in various constraint local search algorithms. Focusing on permutation-based problems, experimental results are provided for the QAP and ATSP to assess the method’s effectiveness.

Reformulation is one of the most useful and widespread activities in mathematical modeling, in that finding a “good” formulation is a fundamental step in being able so solve a given problem. Currently, this is almost exclusively a human activity, with next to no support from modeling and solution tools. In this paper we show how the reformulation system defined in [13] allows to automatize the task of exploring the formulation space of a problem, using a specific example (the Hyperplane Clustering Problem). This nonlinear problem admits a large number of both linear and nonlinear formulations, which can all be generated by defining a relatively small set of general Atomic Reformulation Rules (ARR). These rules are not problem-specific, and could be used to reformulate many other problems, thus showing that a general-purpose reformulation system based on the ideas developed in [13] could be feasible.

How learning heuristic policies may be formulated as a reinforcement learning problem is discussed. Reinforcement learning algorithms are commonly centred around estimating value functions. Here a value function represents the average performance of the learned heuristic algorithm over a problem domain. Heuristics correspond to actions and states to solution instances. The problem of bin packing is used to illustrate the key concepts. Experimental studies show that the reinforcement learning approach is compatible with the current techniques used for learning heuristics. The framework opens up further possibilities for learning heuristics by exploring the numerous techniques available in the reinforcement learning literature.

Upper Confidence Trees are a very efficient tool for solving Markov Decision Processes; originating in difficult games like the game of Go, it is in particular surprisingly efficient in high dimensional problems. It is known that it can be adapted to continuous domains in some cases (in particular continuous action spaces). We here present an extension of Upper Confidence Trees to continuous stochastic problems. We (i) show a deceptive problem on which the classical Upper Confidence Tree approach does not work, even with arbitrarily large computational power and with progressive widening (ii) propose an improvement, termed double-progressive widening, which takes care of the compromise between variance (we want infinitely many simulations for each action/state) and bias (we want sufficiently many nodes to avoid a bias by the first nodes) and which extends the classical progressive widening (iii) discuss its consistency and show experimentally that it performs well on the deceptive problem and on experimental benchmarks. We guess that the double-progressive widening trick can be used for other algorithms as well, as a general tool for ensuring a good bias/variance compromise in search algorithms.

The analysis of complex engineering systems can often be expensive thereby necessitating the use of surrogate models within any design optimization. However, the time variant response of quantities of interest can be non-stationary in nature and therefore difficult to represent effectively with traditional surrogate modelling techniques. The following paper presents the application of partial non-stationary kriging to the prediction of time variant responses where the definition of the non-linear mapping scheme is based upon prior knowledge of either the inputs to, or the nature of, the engineering system considered.

The Portfolio Selection Problem [7] is amongst the most studied issues in finance. In this problem, given a universe of assets (shares, options, bonds, . . . ), we are concerned in finding out a portfolio (i.e., which asset to invest in and by how much) which minimizes the risk while ensuring a given minimum return. In the most common formulation it is required that all the asset shares have to be non-negative. Even though this requirement is a common assumption behind theoretical approaches, it is not enforced in real-markets, where the presence of short positions (i.e., assets with negative shares corresponding to speculations on falling prices) is intertwined to long positions (i.e., assets with positive shares).

Using the recently proposed model of combinatorial landscapes:

local optima networks

, we study the distribution of local optima in two classes of instances of the

quadratic assignment problem

. Our results indicate that the two problem instance classes give rise to very different configuration spaces. For the so-called real-like class, the optima networks possess a clear modular structure, while the networks belonging to the class of random uniform instances are less well partitionable into clusters. We briefly discuss the consequences of the findings for heuristically searching the corresponding problem spaces.

The introduction of learning to the search mechanisms of optimization algorithms has been nominated as one of the viable approaches when dealing with complex optimization problems, in particular with multi-objective ones. One of the forms of carrying out this hybridization process is by using multi-objective optimization estimation of distribution algorithms (MOEDAs). However, it has been pointed out that current MOEDAs have a intrinsic shortcoming in their model-building algorithms that hamper their performance.

In this work we argue that error-based learning, the class of learning most commonly used in MOEDAs is responsible for current MOEDA underachievement. We present adaptive resonance theory (ART) as a suitable learning paradigm alternative and present a novel algorithm called multi-objective ART-based EDA (MARTEDA) that uses a Gaussian ART neural network for model-building and an hypervolume-based selector as described for the HypE algorithm. In order to assert the improvement obtained by combining two cutting-edge approaches to optimization an extensive set of experiments are carried out. These experiments also test the scalability of MARTEDA as the number of objective functions increases.

Differential Evolution (DE) is a simple yet powerful evolutionary algorithm, whose performance highly depends on the setting of some parameters. In this paper, we propose an adaptive DE algorithm for multi-objective optimization problems. Firstly, a novel tree neighborhood density estimator is proposed to enforce a higher spread between the non-dominated solutions, while the Pareto dominance strength is used to promote a higher convergence to the Pareto front. These two metrics are then used by an original replacement mechanism based on a three-step comparison procedure; and also to port two existing adaptive mechanisms to the multi-objective domain, one being used for the autonomous selection of the operators, and the other for the adaptive control of DE parameters CR and F. Experimental results confirm the superior performance of the proposed algorithm, referred to as Adap-MODE, when compared to two state-of-the-art baseline approaches, and to its static and partially-adaptive variants.

MOEA/D is a multi-objective optimization algorithm based on decomposition, which consists in dividing a multi-objective problem into a number of single-objective sub-problems. This work presents two variants, called pMOEA/Dv1 and pMOEA/Dv2, of a new parallel model of MOEA/D that have been developed under the observation that different sub-problems may require different computational effort, and thus, demand different number of evaluations. Our interest in this paper is to analyze whether the proposed models are able of outperforming the MOEA/D in terms of the quality of the computed fronts. To cope with this issue, our proposals have been evaluated using a benchmark composed of eight problems and the obtained results have been compared against MOEA/D-DE, an extension of the original MOEA/D where new individuals are generated by an operator taken from differential evolution. Our experiments show that some configurations of pMOEA/Dv1 and pMOEA/Dv2 have been able to compute fronts of higher quality than MOEA/D-DE in many of the evaluated problems, giving room for further research in this line.

This work introduces a new technique for features construction in classification problems by means of multi objective genetic programming (MOGP). The final goal is to improve the generalization ability of the final classifier. MOGP can help in finding solutions with a better generalization ability with respect to standard genetic programming as stated in [1]. The main issue is the choice of the criteria that must be optimized by MOGP. In this work the construction of new features is guided by two criteria: the first one is the entropy of the target classes as in [7] while the second is inspired by the concept of margin used in support vector machines.

State-of-the-art algorithms for hard computational problems often expose many parameters that can be modified to improve empirical performance. However, manually exploring the resulting combinatorial space of parameter settings is tedious and tends to lead to unsatisfactory outcomes. Recently, automated approaches for solving this

algorithm configuration

problem have led to substantial improvements in the state of the art for solving various problems. One promising approach constructs explicit regression models to describe the dependence of target algorithm performance on parameter settings; however, this approach has so far been limited to the optimization of few numerical algorithm parameters on single instances. In this paper, we extend this paradigm for the first time to general algorithm configuration problems, allowing many categorical parameters and optimization for sets of instances. We experimentally validate our new algorithm configuration procedure by optimizing a local search and a tree search solver for the propositional satisfiability problem (SAT), as well as the commercial mixed integer programming (MIP) solver

CPLEX

. In these experiments, our procedure yielded state-of-the-art performance, and in many cases outperformed the previous best configuration approach.

The ability to visualise how algorithm performance varies across the feature space of possible instance, both real and synthetic, is critical to algorithm selection. Generalising algorithm performance, based on learning from a subset of instances, creates a “footprint” in instance space. This paper shows how self-organising maps can be used to visualise the footprint of algorithm performance, and illustrates the approach using a case study from university course timetabling. The properties of the timetabling instances, viewed from this instance space, are revealing of the differences between the instance generation methods, and the suitability of different algorithms.

A hybrid fish swarm algorithm has been proposed to solve exam timetabling problems where the movement of the fish is simulated when searching for food inside water (refer as a search space). The search space is categorised into three categories which are crowded, not crowded and empty areas. The movement of fish (where the fish represents the solution) is determined based on a Nelder-Mead simplex search algorithm. The quality of the solution is enhanced using a great deluge algorithm or a steepest descent algorithm. The proposed hybrid approach is tested on a set of benchmark examination timetabling problems in comparison with a set of state-of-the-art methods from the literature. The experimental results show that the proposed hybrid approach is able to produce promising results for the test problem.

This paper describes an alternative mutation control scheme for Genetic Algorithms (GAs) inspired by the Self-Organized Criticality (SOC) theory. The strategy, which mimics a SOC system known as

sandpile

, is able to generate mutation rates that, unlike those given by other methods of adaptive parameter control, oscillate between very low values and cataclysmic mutations. In order to attain the desired behaviour, the sandpile is not just attached to a GA; it is also modified in an attempt to link its rates to the stage of the search, i.e., the fitness distribution of the population. Due to its characteristics, the sandpile mutation arises as a promising candidate for efficient and yet simple and context-independent approach to dynamic optimization. An experimental study confirms this assumption: a GA with sandpile mutation outperforms a recently proposed SOC-based GA for dynamic optimization. Furthermore, the proposed method does not increase traditional GAs’ parameter set.

In spite of the success of evolutionary algorithms for dealing with multi-objective optimization problems (the so-called multi-objective evolutionary algorithms (MOEAs)), their main drawback is the fine-tuning of their parameters, which is normally done in an empirical way (using a trial-and-error process for each problem at hand), and usually has a significant impact on their performance. In this paper, we present a self-adaptation methodology that can be incorporated into any MOEA, in order to allow an automatic fine-tuning of parameters, without any human intervention. In order to validate the proposed mechanism, we incorporate it into the NSGA-II, which is a well-known elitist MOEA and we analyze the performance of the resulting approach. The results reported here indicate that the proposed approach is a viable alternative to self-adapt the parameters of a MOEA.

The Evolvable Agent model is a Peer-to-Peer Evolutionary Algorithm [4] which focuses on distributed optimisation over Peer-to-Peer infrastructures [7]. The main idea of the model is that every agent (i.e. individual) is designated as a peer (i.e. network node) and adopts a decentralised population structure defined by the underlying Peer-to-Peer protocol newscast [3]. That way, the population structure acquires a small network diameter which allows a fast dissemination of the best solutions. Additionally, speed of propagation holds with scaling network sizes due to the logarithmic growth of the network diameter.

The design, execution and analysis of experiments using heuristic algorithms can be a very time consuming task in the development of an algorithm. There are a lot of problems that have to be solved throughout this process. To speed up this process we have designed and implemented a framework called EDACC, which supports all the tasks that arise throughout the experimentation with algorithms. A graphical user interface together with a database facilitates archiving and management of solvers and problem instances. It also enables the creation of complex experiments and the generation of the computation jobs needed to perform the experiment. The task of running the jobs on an arbitrary computer system (or computer cluster or grid) is taken by a compute client, which is designed to increase computation throughput to a maximum. Real-time monitoring of running jobs can be done with the GUI or with a web frontend, both of which provide a wide variety of descriptive statistics and statistic testing to analyze the results. The web frontend also provides all the tools needed for the organization and execution of solver competitions.

Sophisticated empirical methods drive the development of high-performance solvers for an increasing range of problems from industry and academia. However, automated tools implementing these methods are often difficult to develop and to use. We address this issue with two contributions. First, we develop a formal description of

meta-algorithmic problems

and use it as the basis for an automated algorithm analysis and design framework called the High-performance Algorithm Laboratory. Second, we describe HAL 1.0, an implementation of the core components of this framework that provides support for distributed execution, remote monitoring, data management, and analysis of results. We demonstrate our approach by using HAL 1.0 to conduct a sequence of increasingly complex analysis and design tasks on state-of-the-art solvers for

SAT

and mixed-integer programming problems.

Hyper-heuristics are methodologies used to search the space of heuristics for solving computationally difficult problems. We describe an object-oriented domain analysis for hyper-heuristics that orthogonally decomposes the domain into generative policy components. The framework facilitates the recursive instantiation of hyper-heuristics over hyper-heuristics, allowing further exploration of the possibilities implied by the hyper-heuristic concept. We describe

Hyperion

, a Java

TM

class library implementation of this domain analysis.

The first

Cross-domain Heuristic Search Challenge

(CHeSC 2011) seeks to bring together practitioners from operational research, computer science and artificial intelligence who are interested in developing more generally applicable search methodologies. The challenge is to design a search algorithm that works well, not only across different instances of the same problem, but also across different problem domains. This article overviews the main features of this challenge.