Advances in Computational Intelligence

In the not so distant future, we expect analytic models to become a commodity. We envision having access to a large number of data-driven models, obtained by a combination of crowdsourcing, crowdservicing, cloud-based evolutionary algorithms, outsourcing, in-house development, and legacy models. In this new context, the critical question will be model ensemble selection and fusion, rather than model generation. We address this issue by proposing customized model ensembles on demand, inspired by Lazy Learning. In our approach, referred to as Lazy Meta-Learning, for a given query we find the most relevant models from a DB of models, using their meta-information. After retrieving the relevant models, we select a subset of models with highly uncorrelated errors. With these models we create an ensemble and use their meta-information for dynamic bias compensation and relevance weighting. The output is a weighted interpolation or extrapolation of the outputs of the models ensemble. Furthermore, the confidence interval around the output is reduced as we increase the number of uncorrelated models in the ensemble. We have successfully tested this approach in a power plant management application.

Neuroevolution is a promising approach for constructing intelligent agents in many complex tasks such as games, robotics, and decision making. It is also well suited for evolving team behavior for many multiagent tasks. However, new challenges and opportunities emerge in such tasks, including facilitating cooperation through reward sharing and communication, accelerating evolution through social learning, and measuring how good the resulting solutions are. This paper reviews recent progress in these three areas, and suggests avenues for future work.

The goal of reverse-engineering the human brain, starting with the auditory pathway, requires three essential ingredients: Neuroscience knowledge, a sufficiently capable computing platform, and a long-term funding source. By 2003, the neuroscience community had a good understanding of the characterization of sound which is carried out in the cochlea and auditory brainstem, and 1.4 GHz single-core computers with XGA displays were fast enough that it was possible to build computer models capable of running and visualizing these processes in isolation at near biological resolution in real-time, and it was possible to raise venture capital funding to begin the project.  By 2008, these advances had permitted the development of products in the area of two-microphone noise reduction for mobile phones, leading to viable business by 2010, thus establishing a self-sustaining funding source to continue the work into the next decade 2010-2020. By 2011, advances in fMRI, multi-electrode, and behavioral studies have illuminated the cortical brain regions responsible for separating sounds in mixtures, understanding speech in quiet and in noisy environments, producing speech, recognizing speakers, and understanding and responding emotionally to music. 2GHz computers with 8 virtual cores and HD displays now permit models of these advanced auditory brain processes to be simulated and displayed simultaneously in real-time, giving a rich perspective on the concurrent and interacting representations of sound and meaning which are developed and maintained in the brain, and exposing a deeper generality to brain architecture than was evident a decade earlier.  While there is much still to be discovered and implemented in the next decade, we can show demonstrable progress on the scientifically ambitious and commercially important goal of reverse-engineering the human auditory pathway.

Theoretical analysis of evolutionary algorithms (EAs) has made significant progresses in the last few years. There is an increased understanding of the computational time complexity of EAs on certain combinatorial optimisation problems. Complementary to the traditional time complexity analysis that focuses exclusively on the problem, e.g., the notion of NP-hardness, computational time complexity analysis of EAs emphasizes the relationship between algorithmic features and problem characteristics. The notion of EA-hardness tries to capture the essence of when and why a problem instance class is hard for what kind of EAs. Such an emphasis is motivated by the practical needs of insight and guidance for choosing different EAs for different problems. This chapter first introduces some basic concepts in analysing EAs. Then the impact of different components of an EA will be studied in depth, including selection, mutation, crossover, parameter setting, and interactions among them. Such theoretical analyses have revealed some interesting results, which might be counter-intuitive at the first sight. Finally, some future research directions of evolutionary computation will be discussed.

The representation of a problem for evolutionary computation is the choice of the data structure used for solutions and the variation operators that act upon that data structure. For a difficult problem, choosing a good representation can have an enormous impact on the performance of the evolutionary computation system. To understand why this is so, one must consider the search space and the fitness landscape induced by the representation. If someone speaks of the fitness landscape of a problem, they have committed a logical error: problems do not have a fitness landscape. The data structure used to represent solutions for a problem in an evolutionary algorithm establishes the set of points in the search space. The topology or connectivity that joins those points is induced by the variation operators, usually crossover and mutation. Points are connected if they differ by one application of the variation operators. Assigning fitness values to each point makes this a fitness landscape. The question of the type of fitness landscape created when a representation is chosen is a very difficult one, and we will explore it in this chapter.

At the Workshop on Evolutionary Algorithms, organized by the Institute for Mathematics and Its Applications, University of Minnesota, Minneapolis, Minnesota, October 21 – 25, 1996, one of the invited speakers, Dave Davis made an interesting claim. As the most recognised practitioner of Evolutionary Algorithms at that time he said that all theoretical results in the area of Evolutionary Algorithms were of no use to him – actually, his claim was a bit stronger. He said that if a theoretical result indicated that, say, the best value of some parameter was such-and-such, he would never use the recommended value in any real-world implementation of an evolutionary algorithm! Clearly, there was – in his opinion – a significant gap between theory and practice of Evolutionary Algorithms.

Fifteen years later, it is worthwhile revisiting this claim and to answer some questions; these include: What are the practical contributions coming from the theory of Evolutionary Algorithms? Did we manage to close the gap between the theory and practice? How do Evolutionary Algorithms compare with Operation Research methods in real-world applications? Why do so few papers on Evolutionary Algorithms describe real-world applications? For what type of problems are Evolutionary Algorithms “the best” method? In this article, I’ll attempt to answer these questions – or at least to provide my personal perspective on these issues.

Multi-objective optimization is widely found in many fields, such as logistics, economics, engineering, or whenever optimal decisions need to be made in the presence of tradeoff between two or more conflicting objectives. The synergy of probabilistic graphical approaches in evolutionary mechanism may enhance the iterative search process when interrelationships of the archived data has been learned, modeled, and used in the reproduction. This paper presents the implementation of probabilistic graphical approaches in solving multi-objective optimization problems under the evolutionary paradigm. First, the existing work on the synergy between probabilistic graphical models and evolutionary algorithms in the multi-objective framework will be presented. We will then show that the optimization problems can be solved using a restricted Boltzmann machine (RBM). The learning, modeling as well as sampling mechanisms of the RBM will be highlighted. Lastly, five studies that implement the RBM for solving multi-objective optimization problems will be discussed.

We present several relational frameworks for expressing similarities and preferences in a quantitative way. The main focus is on the occurrence of various types of transitivity in these frameworks. The first framework is that of fuzzy relations; the corresponding notion of transitivity is C-transitivity, with C a conjunctor. We discuss two approaches to the measurement of similarity of fuzzy sets: a logical approach based on biresidual operators and a cardinal approach based on fuzzy set cardinalities. The second framework is that of reciprocal relations; the corresponding notion of transitivity is cycle-transitivity. It plays a crucial role in the description of different types of transitivity arising in the comparison of (artificially coupled) random variables in terms of winning probabilities. It also embraces the study of mutual rank probability relations of partially ordered sets.

This chapter presents different notions used for fuzzy modelling that formalize fundamental concepts used in cognitive psychology. From a cognitive point of view, the tasks of categorization, pattern recognition or generalization lie in the notions of similarity, resemblance or prototypes. The same tasks are crucial in Artificial Intelligence to reproduce human behaviors. As most real world concepts are messy and open-textured, fuzzy logic and fuzzy set theory can be the relevant framework to model all these key notions.

On the basis of the essential works of Rosch and Tversky, and on the critics formulated on the inadequacy of fuzzy logic to model cognitive concepts, we study a formal and computational approach of the notions of similarity, typicality and prototype, using fuzzy set theory. We propose a framework to understand the different properties and possible behaviors of various families of similarities. We highlight their semantic specifics and we propose numerical tools to quantify these differences, considering different views. We propose also an algorithm for the construction of fuzzy prototypes that can be extended to a classification method.

The invited lecture in 2012 IEEE World Congress on Computational Intelligence (WCCI 2012) presents an overview of a unified fuzzy model-based framework for modeling and control of complex systems. A number of practical applications, ranging from flying vehicles control (including micro helicopter control) to brain-machine cooperative control, are provided in the lecture. The theory and applications have been developed in our laboratory [1] at the University of Electro-Communications (UEC), Tokyo, Japan, in collaboration with Prof. Hua O. Wang and his laboratory [2] at Boston University, Boston, USA. Due to lack of space, this chapter focuses on a unified fuzzy model-based framework for modeling and control of a micro helicopter that is a key application in our research.

Various disciplines, such as machine learning, statistics, data mining and artificial neural networks, are concerned with estimation of data-analytic models. A common theme among all these methodologies is estimation of predictive models from data. In our digital age, an abundance of data and cheap computing power offers hope of knowledge discovery via application of statistical and machine learning algorithms to empirical data. This data-analytic knowledge has similarities and differences with classical scientific knowledge. For example, any scientific theory can be viewed as an inductive theory because it generalizes over a finite number of observations (or experiments). The philosophical aspects of induction and knowledge discovery have been thoroughly explored in Western philosophy of science. This philosophical analysis dates back to Kant and Hume. Any knowledge involves a combination of hypotheses/ideas and empirical data. In the modern digital age, the balance between ideas (mental constructs) and observed data (facts) has completely shifted. Classical scientific knowledge was produced mainly by a stroke of genius (e.g., Newton, Maxwell, and Einstein). In contrast, much of modern knowledge in life sciences and social sciences is derived via data-analytic modeling. We argue that such data-driven knowledge can be properly described following the methodology of predictive learning originally developed in VC-theory. This paper presents a brief survey of the philosophical concepts related to inductive inference, and then extends these ideas to predictive data-analytic knowledge discovery. We contrast the differences between classical first-principle knowledge, data-analytic knowledge and beliefs. Several application examples are used to illustrate the differences between classical statistical and predictive learning approaches to data-analytic modeling. Finally, we discuss interpretation of data-analytic models under predictive learning framework.

Spatio- and spectro-temporal data (SSTD) are the most common types of data collected in many domain areas, including engineering, bioinformatics, neuroinformatics, ecology, environment, medicine, economics, etc. However, there is lack of methods for the efficient analysis of such data and for spatio-temporal pattern recognition (STPR). The brain functions as a spatio-temporal information processing machine and deals extremely well with spatio-temporal data. Its organisation and functions have been the inspiration for the development of new methods for SSTD analysis and STPR. The brain-inspired spiking neural networks (SNN) are considered the third generation of neural networks and are a promising paradigm for the creation of new intelligent ICT for SSTD. This new generation of computational models and systems are potentially capable of modelling complex information processes due to their ability to represent and integrate different information dimensions, such as time, space, frequency, and phase, and to deal with large volumes of data in an adaptive and self-organising manner. The paper reviews methods and systems of SNN for SSTD analysis and STPR, including single neuronal models, evolving spiking neural networks (eSNN) and computational neuro-genetic models (CNGM). Software and hardware implementations and some pilot applications for audio-visual pattern recognition, EEG data analysis, cognitive robotic systems, BCI, neurodegenerative diseases, and others are discussed.

How neuronal firing activities encode meaningful behavior is an ultimate challenge to neuroscientists. To make the problem tractable, we use a rat model to elucidate how an ensemble of single neuron firing events leads to conscious, goal-directed movement and control. This study discusses findings based on single unit, multi-channel simultaneous recordings from rats frontal areas while they learned to perform a decision and control task. To study neural firing activities, first and foremost we needed to identify single unit firing action potentials, or perform spike sorting prior to any analysis on the ensemble of neural activities. After that, we studied cortical neural firing rates to characterize their changes as rats learned a directional paddle control task. Single units from the rat’s frontal areas were inspected for their possible encoding mechanism of directional and sequential movement parameters. Our results entail both high level statistical snapshots of the neural data and more detailed neuronal roles in relation to rat’s learning control behavior.