Skip to main content
main-content

Über dieses Buch

The aim of this book is to discuss the fundamental ideas which lie behind the statistical theory of learning and generalization. It considers learning from the general point of view of function estimation based on empirical data. Omitting proofs and technical details, the author concentrates on discussing the main results of learning theory and their connections to fundamental problems in statistics. These include: - the general setting of learning problems and the general model of minimizing the risk functional from empirical data - a comprehensive analysis of the empirical risk minimization principle and shows how this allows for the construction of necessary and sufficient conditions for consistency - non-asymptotic bounds for the risk achieved using the empirical risk minimization principle - principles for controlling the generalization ability of learning machines using small sample sizes - introducing a new type of universal learning machine that controls the generalization ability.

Inhaltsverzeichnis

Frontmatter

Introduction: Four Periods in the Research of the Learning Problem

Abstract
In the history of research of the learning problem one can extract four periods that can be characterized by four bright events:
(i)
Constructing the first learning machines,
 
(ii)
constructing the fundamentals of the theory,
 
(iii)
constructing neural networks,
 
(iv)
constructing the alternatives to neural networks.
 
Vladimir N. Vapnik

Chapter 1. Setting of the Learning Problem

Abstract
In this book we consider the learning problem as a problem of finding a desired dependence using a limited number of observations.
Vladimir N. Vapnik

Chapter 2. Consistency of Learning Processes

Abstract
The goal of this part of the theory is to describe the conceptual model for learning processes that are based on the Empirical Risk Minimization inductive principle. This part of the theory has to explain when a learning machine that minimizes empirical risk can achieve a small value of actual risk (can generalize) and when it can not. In other words, the goal of this part is to describe the necessary and sufficient conditions for the consistency of learning processes that minimizes the empirical risk.
Vladimir N. Vapnik

Chapter 3. Bounds on the Rate of Convergence of Learning Processes

Abstract
In this chapter we consider bounds on the rate of uniform convergence. We consider upper bounds (there exist lower bounds as well (Vapnik and Chervonenkis, 1974), however, they are not as important for controlling the learning processes as the upper bounds).
Vladimir N. Vapnik

Chapter 4. Controlling the Generalization Ability of Learning Processes

Abstract
The theory for controlling the generalization ability of learning machines is devoted to constructing an inductive principle for minimizing the risk functional using a small sample of training instances.
Vladimir N. Vapnik

Chapter 5. Constructing Learning Algorithms

Abstract
To implement the SRM inductive principle in learning algorithms one has to minimize the risk in a given set of functions by controlling two factors: the value of the empirical risk and the value of the confidence interval.
Vladimir N. Vapnik

Conclusion: What is Important in Learning Theory?

Abstract
In the beginning of this book we postulated (without any discussion) that learning is a problem of function estimation on the basis of empirical data. To solve this problem we used a classical inductive principle — the ERM principle. Later, however, we introduced a new principle — the SRM principle. Nevertheless, the general understanding of the problem remains based on the statistics of large samples: the goal is to derive the rule that possesses the lowest risk. The goal of obtaining the “lowest risk” reflects the philosophy of large sample size statistics: the rule with low risk is good because if we use this rule for a large test set, with high probability, the means of losses will be small.
Vladimir N. Vapnik

Backmatter

Weitere Informationen