Computational Intelligence: Research Frontiers

The last five years have seen the emergence of a powerful new framework for building sophisticated real-world applications based on machine learning. The cornerstones of this approach are (i) the adoption of a Bayesian viewpoint, (ii) the use of graphical models to represent complex probability distributions, and (iii) the development of fast, deterministic inference algorithms, such as variational Bayes and expectation propagation, which provide efficient solutions to inference and learning problems in terms of local message passing algorithms. This paper reviews the key ideas behind this new framework, and highlights some of its major benefits. The framework is illustrated using an example large-scale application.

We examine the interplay of optimization and machine learning. Great progress has been made in machine learning by cleverly reducing machine learning problems to convex optimization problems with one or more hyper-parameters. The availability of powerful convex-programming theory and algorithms has enabled a flood of new research in machine learning models and methods. But many of the steps necessary for successful machine learning models fall outside of the convex machine learning paradigm. Thus we now propose framing machine learning problems as Stackelberg games. The resulting bilevel optimization problem allows for efficient systematic search of large numbers of hyper-parameters. We discuss recent progress in solving these bilevel problems and the many interesting optimization challenges that remain. Finally, we investigate the intriguing possibility of novel machine learning models enabled by bilevel programming.

Two intelligent abilities and three inverse problems are re-elaborated from a probability theory based two pathway perspective, with challenges of statistical learning and efforts towards the challenges overviewed. Then, a detailed introduction is provided on the Bayesian Ying-Yang (BYY) harmony learning. Proposed firstly in (Xu,1995) and systematically developed in the past decade, this approach consists of a two pathway featured BYY system as a general framework for unifying a number of typical learning models, and a best Ying-Yang harmony principle as a general theory for parameter learning and model selection. The BYY harmony learning leads to not only a criterion that outperforms typical model selection criteria in a two-phase implementation, but also model selection made automatically during parameter learning for several typical learning tasks, with computing cost saved significantly. In addition to introducing the fundamentals, several typical learning approaches are also systematically compared and re-elaborated from the BYY harmony learning perspective. Moreover, a further brief is made on the features and applications of a particular family called Gaussian manifold based BYY systems.

The Berlin Brain-Computer Interface (BBCI) uses a machine learning approach to extract subject-specific patterns from high-dimensional EEG-features optimized for revealing the user’s mental state. Classical BCI application are brain actuated tools for patients such as prostheses (see Section 4.1) or mental text entry systems ([2] and see [3,4,5,6] for an overview on BCI). In these applications the BBCI uses natural motor competences of the users and specifically tailored pattern recognition algorithms for detecting the user’s intent. But beyond rehabilitation, there is a wide range of possible applications in which BCI technology is used to monitor other mental states, often even covert ones (see also [7] in the fMRI realm). While this field is still largely unexplored, two examples from our studies are exemplified in Section 4.3 and 4.4.

To promote international cooperation in the new field of Neuroinformatics (NI), the Neuroinformatics Japan Center at RIKEN Brain Science Institute (BSI) has been established in 2005 as the Japan-Node (J-Node) for coordination with the International Neuroinformatics Coordinating Facility. The Laboratory for Neuroinformatics was established in 2002 at RIKEN BSI, and created the NI base-platform “XooNIps” following the concepts and experience acquired from the Visiome Platform, which was developed under the project of the Neuroinformatics Research in Vision. XooNIps features better scalability, extensibility, and customizability to operate under various site policies supporting different databases and portals. Utilizing XooNIps, eight J-Node platforms have been developed by each platform committee which were selected from active research areas in Japan. XooNIps contributes not only in NI field but in diverse areas such as library repositories and university research resources.

There are evident and profoundly articulated needs to deal with distributed sources of data (such as e.g., sensors and sensor networks, web sites, distributed databases). While recognizing limited accessibility of such data at a global level (which could be associated with technical constraints and/or privacy issues) and fully acknowledging benefits and potentials of collaborative processing, we introduce a concept of Collaborative Computational Intelligence (CI), and collaborative fuzzy models, in particular. Collaboration is realized in different ways by engaging a host of bidirectional interactions between all local processing sites (models) or by proceeding with unidirectional communication in which we establish some mechanisms of developing experience consistency of fuzzy modeling. We offer a coherent taxonomy of various schemes of interaction which in the sequel implies a certain development of a suite of algorithms. In this setting, we highlight a pivotal role of granular information in the establishing of the mechanisms of interaction. In the realm of collaborative fuzzy models and fuzzy modeling we elaborate on the concept of knowledge sharing. We also bring forward a concept of experience–consistent fuzzy system identification showing how fuzzy models built on a basis of limited data can benefit from taking advantage of the past experience conveyed in the form of previously constructed fuzzy models. Proceeding with a more detailed algorithmic framework, we elaborate on the key design issues concerning fuzzy rule-based systems which constitute a dominant category of fuzzy models. Collaboration invokes some mechanisms of aggregation and reconciliation of local findings. We emphasize that the resulting findings such as specific components of models can be quantified in terms of type-2 fuzzy sets – a pursuit which offers an interesting motivation behind this higher type of fuzzy sets.

We first note that since humans communicate using linguistic terms central to man–machine cooperation is the availability of a common vocabulary understandable by both parties. Here we draw upon structures from granular computing, particularly fuzzy sets, to provide this capability. Having this capability allows the machine to use the types of information humans commonly provide. We then focus on some tools useful for the fusion of information and question answering in the context of man-machine cooperation. We describe methods for fusing information from multiple sources and we provide the capability to have multiple fused values. We also investigate the fusion of probabilistic and possibilistic information.

Since the SOM visualizes the similarity of raw information on the competitive layer, it can be utilized in the field of pattern classification, data analysis, and so on. However, it cannot model the input-output characteristics of the system of interest. In order to squeeze out the input-output relationship from the data set with evaluation obtained by trial and error, the novel modeling tool was developed by the author (1999), which is the extension of SOM and in which the input-output relationship of the system is mapped onto the competitive layer. The system is named as self-organizing relationship network (SOR network). A set of units on the competitive layer of the SOR network after learning exhibits a set of typical input-output characteristics of the system of interest and thus the network achieves the knowledge acquisition (IF-THEN rules) from the raw data with evaluation and the effective fuzzy inference with defuzzification. The plenary talk presents the tutorial aspects of the SOR network and an application to an intelligent control.

Type-1 Fuzzy Logic Controllers (FLCs) have been applied to date with great success to different applications. However, for many real-world applications, there is a need to cope with large amounts of uncertainties. The traditional type-1 FLCs that use crisp type-1 fuzzy sets cannot directly handle such uncertainties. Type-2 FLCs that use type-2 fuzzy sets can handle such uncertainties to produce a better performance. Hence, type-2 FLCs will have the potential to overcome the limitations of type-1 FLCs and produce a new generation of fuzzy controllers with improved performance for many applications which require handling high levels of uncertainty. This chapter will provide an overview of the interval type-2 FLCs and their advantages over type-1 FLCs. We will also present different techniques to avoid the computational overheads and thus enabling the type-2 FLCs to produce a good real time response. Furthermore, we will present various successful real world applications of type-2 FLCs.

In this document we examine the evolutionary methods that may lead to the emergence of altruistic cooperation in robot collectives. We present four evolutionary algorithms that derive from biological theories on the evolution of altruism in nature and compare them systematically in two experimental scenarios where altruistic cooperation can lead to a performance increment. We discuss the relative merits and drawbacks of the four methods and provide recommendations for the choice of the most suitable method for evolving altruistic robots.

Evolutionary Algorithms (EAs) as one important subdomain of Computational Intelligence (CI) have conquered the field of experimental as well as difficult numerical optimization despite the lack of addresses of welcome half a century ago. Meanwhile, they go without saying into the toolboxes of most practitioners who have to solve real-world problems. And an overwhelming number of theoretical results underpin at least parts of the practice. More recently, even vector optimization problems can be tackled by means of specialized EAs. These multiobjective evolutionary algorithms (MOEAs or EMOAs) help decision makers to reduce the number of design possibilities to the subsets that make the best of the situation in case of conflicting objectives. This article briefly describes the problem setting, the most important solution approaches, and the challenges that still lie ahead in their improvement. Most sophisticated algorithms in this domain have somehow lost their character of mimicking natural mechanisms found in organic evolution. That is why a couple of more bio-inspired aspects are mentioned in the second part of this contribution that may help to diversify further research and practice in multiobjective optimization (MOO) without forgetting to foster the interdisciplinary dialogue with natural scientists.

Evolutionary algorithms are stochastic search methods that are efficient and effective for solving sophisticated multi-objective (MO) problems. Advances made in the field of evolutionary multi-objective optimization (EMO) are the results of two decades worth of intense research, studying various topics that are unique to MO optimization. However many of these studies assume that the problem is deterministic and static, and the EMO performance generally deteriorates in the presence of uncertainties. In certain situations, the solutions found may not even be implementable in practice. In this chapter, the challenges faced in handling three different forms of uncertainties in EMO will be discussed, including 1) noisy objective functions, 2) dynamic MO fitness landscape, and 3) robust MO optimization. Specifically, the impact of these uncertainties on MO optimization will be described and the approaches/modifications to basic algorithm design for better and robust EMO performance will be presented.

All clustering algorithms partition data into a specified or algorithmically determined number of clusters, whether or not that number of clusters actually exists in the data. Therefore, identifying a "best" solution amongst a set of candidate partitions is an important step in the clustering process. This paper presents a visual technique for comparing found partitions with a pre-clustering VAT (Visual Assessment of cluster Tendency) image of the unlabeled input data. The method is developed independent of any particular clustering algorithm, and then illustrated with numerical examples that use the fuzzy c-means clustering method. The experiments use samples from mixtures of bivariate normals, a bivariate uniform, and a small real data set to illustrate the efficacy of the method.

The self-organizing map (SOM) is an automatic data- analysis method. It is widely applied to clustering problems and data exploration in industry, finance, natural sciences, and linguistics. The most extensive applications, exemplified in this paper, can be found in the management of massive textual data bases. The SOM is related to the classical vector quantization (VQ), which is used extensively in digital signal processing and transmission. Like in VQ, the SOM represents a distribution of input data items using a finite set of models. In the SOM, however, these models are automatically associated with the nodes of a regular (usually two-dimensional) grid in an ordered fashion such that more similar models become automatically associated with nodes that are adjacent in the grid, whereas less similar models are situated farther away from each other in the grid. This organization, a kind of similarity diagram of the models, makes it possible to obtain an insight into the topographic relationships of data, especially of high-dimensional data items. If the data items belong to certain predetermined classes, the models (and the nodes) can be calibrated according to these classes. An unknown input item is then classified according to that node, the model of which is most similar with it in some metric used in the construction of the SOM. A new finding introduced in this paper is that an input item can even more accurately be represented by a linear mixture of a few best-matching models. This becomes possible by a least-squares fitting procedure where the coefficients in the linear mixture of models are constrained to nonnegative values.

Speech segregation, or the cocktail party problem, has proven to be an extremely challenging problem. This chapter describes a computational auditory scene analysis (CASA) approach to the cocktail party problem. This monaural approach performs auditory segmentation and grouping in a two-dimensional time-frequency representation that encodes proximity in frequency and time, periodicity, amplitude modulation, and onset/offset. In segmentation, our model decomposes the input mixture into contiguous time-frequency segments. Grouping is first performed for voiced speech where detected pitch contours are used to group voiced segments into a target stream and the background. In grouping voiced speech, resolved and unresolved harmonics are dealt with differently. Grouping of unvoiced segments is based on the Bayesian classification of acoustic-phonetic features. This CASA approach has led to major advances towards solving the cocktail party problem.

Similarity is a key concept for all attempts to construct human-like automated systems or assistants to human task solving since they are very natural in the human process of categorization, underlying many natural capabilities such as language understanding, pattern recognition or decision-making. In this paper, we study the use of similarities in data mining, basing our discourse on cognitive approaches of similarity stemming for instance from Tversky’s and Rosch’s seminal works, among others. We point out a general framework for measures of comparison compatible with these cognitive foundations, and we show that measures of similarity can be involved in all steps of the data mining process. We then focus on fuzzy logic that provides interesting tools for data mining mainly because of its ability to represent imperfect information, which is of crucial importance when databases are complex, large, and contain heterogeneous, imprecise, vague, uncertain or incomplete data. We eventually illustrate our discourse by examples of similarities used in real-world data mining problems.

Self-adaptive systems autonomously change their behavior to compensate for faults or to improve their performance. Evolvable hardware, which combines evolutionary algorithms with reconfigurable hardware, is often proposed as the cornerstone for systems that use self-adaption for fault recovery. Although evolvable hardware was first introduced over 15 years ago, there are few, if any, fault tolerant self-adaptive systems in operation today. One primary reason why these unfortunate circumstances have arisen is many designers—and not limited to just designers from the computational intelligence community—do not really understand how to build a basic fault tolerant system, let alone a self-adaptive fault tolerant system. This chapter describes how fault tolerant systems are built. A model for designing fault tolerant systems that rely on evolvable hardware for fault recovery is presented.