Computational Intelligence | springerprofessional.de

Springer Professional

nach oben

2013 | Buch

Kapitel lesen Erstes Kapitel lesen

Computational Intelligence

A Methodological Introduction

verfasst von: Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Verlag: Springer London

Buchreihe : Texts in Computer Science

Enthalten in: Springer Professional "Wirtschaft+Technik" , Springer Professional "Technik" , Springer Professional "Wirtschaft"

Einloggen, um Zugang zu erhalten

Über dieses Buch

This clearly-structured, classroom-tested textbook/reference presents a methodical introduction to the field of CI. Providing an authoritative insight into all that is necessary for the successful application of CI methods, the book describes fundamental concepts and their practical implementations, and explains the theoretical background underpinning proposed solutions to common problems. Only a basic knowledge of mathematics is required. Features: provides electronic supplementary material at an associated website, including module descriptions, lecture slides, exercises with solutions, and software tools; contains numerous examples and definitions throughout the text; presents self-contained discussions on artificial neural networks, evolutionary algorithms, fuzzy systems and Bayesian networks; covers the latest approaches, including ant colony optimization and probabilistic graphical models; written by a team of highly-regarded experts in CI, with extensive experience in both academia and industry.

Anzeige

Inhaltsverzeichnis

Frontmatter

Chapter 1. Introduction

Abstract

In this chapter we give a very brief introduction to intelligent systems. In order to design such a system, it is necessary to efficiently represent and process knowledge about the given problem setting. For certain types of problems, techniques inspired by natural or biological processes proved successful. These techniques belong to the field of computational intelligence. Our main objective with this textbook is to give a methodical introduction to this field. Among such methods we find artificial neural networks, evolutionary algorithms, and fuzzy systems. Finally, we mention how to use this book and where additional material can be found the Internet.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Neural Networks

Frontmatter

Chapter 2. Introduction

Abstract

(Artificial) neural networks are information processing systems, whose structure and operation principles are inspired by the nervous system and the brain of animals and humans. They consist of a large number of fairly simple units, the so-called neurons, which are working in parallel. These neurons communicate by sending information in the form of activation signals, along directed connections, to each other. A commonly used synonym for “neural network” is the term “connectionist model.” The research area that is devoted to the study of connectionist models is called “connectionism.” Furthermore, the expression “parallel distributed processing” can often be found in relation to (artificial) neural networks.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 3. Threshold Logic Units

Abstract

The description of biological neural networks in the preceding chapter makes it natural to model neurons as threshold logic units: if a neuron receives enough excitatory input that is not compensated by equally strong inhibitory input, it becomes active and sends a signal to other neurons. Threshold logic units are also known as McCulloch–Pitts neurons. Another name which is commonly used for a threshold logic unit is perceptron.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 4. General Neural Networks

Abstract

In this chapter we introduce a general model of (artificial) neural networks that captures (more or less) all special forms, which we consider in the following chapters. We start by defining the structure of an (artificial) neural network and then describe generally the operation and finally the training of an (artificial) neural network.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 5. Multi-Layer Perceptrons

Abstract

Having described the structure, the operation and the training of (artificial) neural networks in a general fashion in the preceding chapter, we turn in this and the subsequent chapters to specific forms of (artificial) neural networks. We start with the best-known and most widely used form, the so-called multi-layer perceptron (MLP), which is closely related to the networks of threshold logic units we studied in a previous chapter. They exhibit a strictly layered structure and may employ other activation functions than a step at a crisp threshold.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 6. Radial Basis Function Networks

Abstract

Like multi-layer perceptrons, radial basis function networks are feed-forward neural networks with a strictly layered structure. However, the number of layers is always three, that is, there is exactly one hidden layer. In addition, radial basis function networks differ from multi-layer perceptrons in the network input and activation functions, especially in the hidden layer. In this hidden layer radial basis functions are employed, which are responsible for the name of this type of neural network. With these functions a kind of “catchment region” is assigned to each neuron, in which it mainly influences the output of the neural network.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 7. Self-organizing Maps

Abstract

Self-organizing maps are closely related to radial basis function networks. They can be seen as radial basis function networks without an output layer, or, rather, the hidden layer of a radial basis function network is already the output layer of a self-organizing map. This output layer also has an internal structure, since the neurons are arranged in a grid. The neighborhood relationships resulting from this grid are exploited in the training process in order to determine a topology preserving map.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 8. Hopfield Networks

Abstract

In the preceding Chaps. 5 to 7 we studied so-called feed forward networks, that is, networks with an acyclic graph (no directed cycles). In this and the next chapter, however, we turn to so-called recurrent networks, that is, networks, the graph of which may contain (directed) cycles. We start with one of the simplest forms, the so-called Hopfield networks, which originated as physical models to describe magnetism. Hopfield networks are indeed closely related to the Ising model of magnetism.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 9. Recurrent Networks

Abstract

The Hopfield networks that we discussed in the preceding chapter are special recurrent networks, which have a very constrained structure. In this chapter, however, we lift all restrictions and consider recurrent networks without any constraints. Such general recurrent networks are well suited to represent differential equations and to solve them (approximately) in a numerical fashion. If the type of differential equation is known that describes a given system, but the values of the parameters appearing in it are unknown, one may also try to train a suitable recurrent network with the help of example patterns in order to determine the system parameters.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 10. Mathematical Remarks

Abstract

The following sections treat mathematical topics that were presupposed in the text (e.g. about straight line equations and about regression), or side remarks, which would have disturbed the flow of the exposition (e.g. about activation transformation in a Hopfield network).

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Evolutionary Algorithms

Frontmatter

Chapter 11. Introduction to Evolutionary Algorithms

Abstract

Evolutionary algorithms comprise a class of optimization techniques that imitate principles of biological evolution. They belong to the family of metaheuristics, which also includes, for example, particle swarm and ant colony optimization, which are inspired by other biological structures and processes, as well as classical methods like simulated annealing, which is inspired by a thermodynamical process. The core principle of evolutionary algorithms is to apply evolution principles like mutation and selection to populations of candidate solutions in order to find a (sufficiently good) solution for a given optimization problem.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 12. Elements of Evolutionary Algorithms

Abstract

Evolutionary algorithms are not fixed procedures, but contain several elements that must be adapted to the optimization problem to be solved. In particular, the encoding of the candidate solution needs to be chosen with care. Although there is no generally valid rule or recipe, we discuss some important properties a good encoding should have. We also turn to the fitness function and review the most common selection techniques as well as how certain undesired effects can be avoided by adapting the fitness function or the selection method. The last section of this chapter is devoted to genetic operators, which serve as tools to explore the search space, and covers sexual and asexual recombination and other variation techniques.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 13. Fundamental Evolutionary Algorithms

Abstract

The preceding chapter presented all relevant elements of evolutionary algorithms, namely guidelines of how to choose an encoding for the solution candidates, procedures how to select individuals based on their fitness, and genetic operators with which modified solution candidates can be obtained. Equipped with these ingredients we proceed in this chapter to introducing basic forms of evolutionary algorithms, including classical genetic algorithms (in which solution candidates are encoded as bit strings), evolution strategies (which focus on numerical optimization) and genetic programming (which tries to derive function expressions or even (simple) program structures with evolutionary principles). Finally, we take a look at related population-based approaches (like ant colony and particle swarm optimization).

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 14. Special Applications and Techniques

Abstract

With this chapter we close our discussion of evolutionary algorithms by giving an overview of an application of and two special techniques for this kind of metaheuristics. In the first section we consider behavioral simulation for the iterated prisoners dilemma with an evolutionary algorithm. In the next section we study evolutionary algorithms for multi-criteria optimization, especially in the presence of conflicting criteria, which instead of returning a single solution try to map out the so-called Pareto-frontier with several solution candidates. Finally, we take a look at parallelized versions of evolutionary algorithms.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Fuzzy Systems

Frontmatter

Chapter 15. Fuzzy Sets and Fuzzy Logic

Abstract

Many propositions about the real world are not either true or false, rendering classical logic inadequate for reasoning with such propositions. Furthermore, most concepts used in human communication do not have crisp boundaries, rendering classical sets inadequate to represent such concept. The main aim of fuzzy logic and fuzzy sets is to overcome the disadvantages of classical logic and classical sets.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 16. The Extension Principle

Abstract

We have already discussed how set theoretic operations like intersection, union and complement can be generalized to fuzzy sets. This chapter is devoted to the issue of extending the concept of mappings or functions to fuzzy sets. These ideas allow us to define operations like addition, subtraction, multiplication, division or taking squares as well as set theoretic concepts like the composition of relations for fuzzy sets.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 17. Fuzzy Relations

Abstract

Relations can be used to model dependencies, correlations or connections between variables, quantities or attributes. Technically speaking, a (binary) relation over the universes of discourse X and Y is a subset R of the Cartesian product X×Y of X and Y. The pairs (x,y)∈X×Y belonging to the relation R are linked by a connection described by the relation R. Therefore, a common notation for (x,y)∈R is also xRy.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 18. Similarity Relations

Abstract

In this chapter we discuss a special type of fuzzy relations, called similarity relations. They play an important role in interpreting fuzzy controllers and, more generally, can be used to characterize the inherent indistinguishability or vagueness of a fuzzy systems.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 19. Fuzzy Control

Abstract

The biggest success of fuzzy systems in the field of industrial and commercial applications has been achieved with fuzzy controllers. Fuzzy control is a way of defining a nonlinear table-based controller whereas its nonlinear transition function can be defined without specifying every single entry of the table individually. Fuzzy control does not result from classical control engineering approaches. In fact, its roots can be found in the area of rule-based systems. Fuzzy controllers simply comprise a set of vague rules that can be used for knowledge-based interpolation of a vaguely defined function.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 20. Fuzzy Clustering

Abstract

After a brief overview of fuzzy methods in data analysis, this chapter focuses on fuzzy cluster analysis as the oldest fuzzy approach to data analysis. Fuzzy clustering comprises a family of prototype-based clustering methods that can be formulated as the problem of minimizing an objective function. These methods can be seen as “fuzzifications” of, for example, the classical c-means algorithm, which strives to minimize the sum of the (squared) distances between the data points and the cluster centers to which they are assigned. However, in order to “fuzzify” such a crisp clustering approach, it is not enough to merely allow values from the unit interval for the variables encoding the assignments of the data points to the clusters: the minimum is still obtained for a crisp data point assignment. As a consequence, additional means have to be employed in the objective function in order to obtain actual degrees of membership. This chapter surveys the most common fuzzification means and examines and compares their properties.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Bayes Networks

Frontmatter

Chapter 21. Introduction to Bayes Networks

Abstract

Relational database systems are amongst the most wide-spread data management systems in today’s businesses. A database typically consists of several tables that contain data about business objects such as customer data, sales orders or product information. Each table row represents a description of a single object with each table column representing an attribute of that object. Relations between these objects are also modeled via tables. Please note that we use the notions table and relation interchangeably. A major part of database theory is concerned with the task to represent data with as little redundancy as possible.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 22. Elements of Probability and Graph Theory

Abstract

This chapter introduces required theoretical concepts for the definition of Bayes and Markov networks. After important elements of probability theory—especially (conditional) independences—are discussed, we present relevant graph-theoretic notions with emphasis on so-called separation criteria. These criteria will later allow us to capture probabilistic independences with an undirected or directed graph.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 23. Decompositions

Abstract

The objective of this chapter is to connect the concepts of conditional independence with the separation in graphs. Both can be represented by a ternary relation

https://static-content.springer.com/image/chp%3A10.1007%2F978-1-4471-5013-8_23/302305_1_En_23_IEq1_HTML.gif

on either the set of attributes or nodes and it seems to be promising to investigate how to represent the probabilistic properties of a distribution by the means of a graph. The idea then is to use only graph-theoretic criteria (separations) to draw inferences about (conditional) independences because it is them what enables us to decompose a high-dimensional distribution and propagate evidence.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 24. Evidence Propagation

Abstract

After having discussed efficient representations for expert and domain knowledge, we intent to exploit them to draw inferences when new information (evidence) becomes known. Using the Volkswagen example from the last chapter, an inference could be the update of the probabilities of certain car parts combinations when the customer has chosen, say, the engine type to be m ^∗. The objective is to propagate the evidence through the underlying network to reach all relevant attributes. Obviously, the graph structure will play an important role.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Chapter 25. Learning Graphical Models

Abstract

In this chapter we will address how graphical models can be learned from given data. So far we were given the graphical structure. Now, we will introduce heuristics that allow us to induce these structures.

Rudolf Kruse, Christian Borgelt, Frank Klawonn, Christian Moewes, Matthias Steinbrecher, Pascal Held

Backmatter

Titel: Computational Intelligence
verfasst von: Rudolf Kruse
Christian Borgelt
Frank Klawonn
Christian Moewes
Matthias Steinbrecher
Pascal Held
Copyright-Jahr: 2013
Verlag: Springer London
Electronic ISBN: 978-1-4471-5013-8
Print ISBN: 978-1-4471-5012-1
DOI: https://doi.org/10.1007/978-1-4471-5013-8

Weitere Formate
Fachgebiete
Bücher
Zeitschriften
Themenseiten
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024