nach oben

2012 | Buch

Kapitel lesen Erstes Kapitel lesen

Ensemble Machine Learning

Methods and Applications

herausgegeben von: Cha Zhang, Yunqian Ma

Verlag: Springer New York

Enthalten in: Springer Professional "Wirtschaft+Technik" , Springer Professional "Technik" , Springer Professional "Wirtschaft"

Einloggen, um Zugang zu erhalten

Über dieses Buch

It is common wisdom that gathering a variety of views and inputs improves the process of decision making, and, indeed, underpins a democratic society. Dubbed “ensemble learning” by researchers in computational intelligence and machine learning, it is known to improve a decision system’s robustness and accuracy. Now, fresh developments are allowing researchers to unleash the power of ensemble learning in an increasing range of real-world applications. Ensemble learning algorithms such as “boosting” and “random forest” facilitate solutions to key computational issues such as face recognition and are now being applied in areas as diverse as object tracking and bioinformatics.

Responding to a shortage of literature dedicated to the topic, this volume offers comprehensive coverage of state-of-the-art ensemble learning techniques, including the random forest skeleton tracking algorithm in the Xbox Kinect sensor, which bypasses the need for game controllers. At once a solid theoretical study and a practical guide, the volume is a windfall for researchers and practitioners alike.

Inhaltsverzeichnis

Frontmatter

Chapter 1. Ensemble Learning

Abstract

Over the last couple of decades, multiple classifier systems, also called ensemble systems have enjoyed growing attention within the computational intelligence and machine learning community. This attention has been well deserved, as ensemble systems have proven themselves to be very effective and extremely versatile in a broad spectrum of problem domains and real-world applications. Originally developed to reduce the variance—thereby improving the accuracy—of an automated decision-making system, ensemble systems have since been successfully used to address a variety of machine learning problems, such as feature selection, confidence estimation, missing feature, incremental learning, error correction, class-imbalanced data, learning concept drift from nonstationary distributions, among others. This chapter provides an overview of ensemble systems, their properties, and how they can be applied to such a wide spectrum of applications.

Robi Polikar

Chapter 2. Boosting Algorithms: A Review of Methods, Theory, and Applications

Abstract

Boosting is a class of machine learning methods based on the idea that a combination of simple classifiers (obtained by a weak learner) can perform better than any of the simple classifiers alone. A weak learner (WL) is a learning algorithm capable of producing classifiers with probability of error strictly (but only slightly) less than that of random guessing (0.5, in the binary case). On the other hand, a strong learner (SL) is able (given enough training data) to yield classifiers with arbitrarily small error probability.

Artur J. Ferreira, Mário A. T. Figueiredo

Chapter 3. Boosting Kernel Estimators

Abstract

A boosting algorithm [1, 2] could be seen as a way to improve the fit of statistical models. Typically, M predictions are operated by applying a base procedure—called a weak learner—to M reweighted samples. Specifically, in each reweighted sample an individual weight is assigned to each observation. Finally, the output is obtained by aggregating through majority voting. Boosting is a sequential ensemble scheme, in the sense the weight of an observation at step m depends (only) on the step m − 1. It appears clear that we obtain a specific boosting scheme when we choose a loss function, which orientates the data re-weighting mechanism, and a weak learner.

Marco Di Marzio, Charles C. Taylor

Chapter 4. Targeted Learning

Abstract

Suppose we observe n i.i.d. copies O₁, …, O_n of a random variable O with probability distribution P₀, and assume that it is known that \({P}_{0} \in \mathcal{M}\) for some set of probability distributions \(\mathcal{M}\). One refers to \(\mathcal{M}\) as the statistical model for P₀. We consider so called semiparametric models that cannot be parameterized by a finite dimensional Euclidean vector. In addition, suppose that our target parameter of interest is a parameter \(\Psi : \mathcal{M}\rightarrow \mathcal{F} =\{ \Psi (P) : P \in \mathcal{M}\}\), so that ψ₀ = Ψ(P₀) denotes the parameter value of interest.

Mark J. van der Laan, Maya L. Petersen

Chapter 5. Random Forests

Abstract

Random Forests were introduced by Leo Breiman [6] who was inspired by earlier work by Amit and Geman [2]. Although not obvious from the description in [6], Random Forests are an extension of Breiman’s bagging idea [5] and were developed as a competitor to boosting. Random Forests can be used for either a categorical response variable, referred to in [6] as “classification,” or a continuous response, referred to as “regression.” Similarly, the predictor variables can be either categorical or continuous.

Adele Cutler, D. Richard Cutler, John R. Stevens

Chapter 6. Ensemble Learning by Negative Correlation Learning

Abstract

This chapter investigates a specific ensemble learning approach by negative correlation learning (NCL) [21, 22, 23]. NCL is an ensemble learning algorithm which considers the cooperation and interaction among the ensemble members. NCL introduces a correlation penalty term into the cost function of each individual learner so that each learner minimizes its mean-square-error (MSE) error together with the correlation with other ensemble members.

Huanhuan Chen, Anthony G. Cohn, Xin Yao

Chapter 7. Ensemble Nyström

Abstract

A crucial technique for scaling kernel methods to very large datasets reaching or exceeding millions of instances is based on low-rank approximation of kernel matrices. The Nyström method is a popular technique to generate low-rank matrix approximations but it requires sampling of a large number of columns from the original matrix to achieve good accuracy. This chapter describes a new family of algorithms based on mixtures of Nyström approximations, Ensemble Nyström algorithms, that yield more accurate low-rank approximations than the standard Nyström method. We give a detailed study of variants of these algorithms based on simple averaging, an exponential weight method, and regression-based methods. A theoretical analysis of these algorithms, including novel error bounds guaranteeing a better convergence rate than the standard Nyström method is also presented. Finally, experiments with several datasets containing up to 1 M points are presented, demonstrating significant improvement over the standard Nyström approximation.

Sanjiv Kumar, Mehryar Mohri, Ameet Talwalkar

Chapter 8. Object Detection

Abstract

Over the past twenty years, data-driven methods have become a dominant paradigm for computer vision, with numerous practical successes. In difficult computer vision tasks, such as the detection of object categories (for example, the detection of faces of various gender, age, race, and pose, under various illumination and background conditions), researchers generally learn a classifier that can distinguish an image patch that contains the object of interest from all other image patches. Ensemble learning methods have been very successful in learning classifiers for object detection.

Jianxin Wu, James M. Rehg

Chapter 9. Classifier Boosting for Human Activity Recognition

Abstract

The ability to visually infer human activities happening in an environment is becoming increasingly important due to the tremendous practical applications it offers [1]. Systems that can automatically recognize human activities can potentially help us in monitoring people’s health as they age [7], and to fight crime through improved surveillance [26]. They have tremendous medical applications in terms of helping surgeons perform better by identifying and evaluating crucial parts of the surgical procedures, and providing the medical specialists with useful feedback [2]. Similarly, these systems can help us improve our productivity in office environments by detecting various interesting and important events around us to enhance our involvement in important office tasks [21].

Raffay Hamid

Chapter 10. Discriminative Learning for Anatomical Structure Detection and Segmentation

Abstract

There is an emerging trend of using machine learning approaches to solve the tasks in medical image analysis. In this chapter, we summarize several discriminative learning methods for detection and segmentation of anatomical structures. In particular, we propose innovative detector structures, namely Probabilistic Boosting Network (PBN) and Marginal Space Learning (MSL), to address the challenges in anatomical structure detection. We also present a regression approach called Shape Regression Machine (SRM) for anatomical structure detection. For anatomical structure segmentation, we propose discriminative formulations, explicit and implicit, that are based on classification, regression and ranking.

S. Kevin Zhou, Jingdan Zhang, Yefeng Zheng

Chapter 11. Random Forest for Bioinformatics

Abstract

Modern biology has experienced an increased use of machine learning techniques for large scale and complex biological data analysis. In the area of Bioinformatics, the Random Forest (RF) [6] technique, which includes an ensemble of decision trees and incorporates feature selection and interactions naturally in the learning process, is a popular choice. It is nonparametric, interpretable, efficient, and has high prediction accuracy for many types of data. Recent work in computational biology has seen an increased use of RF, owing to its unique advantages in dealing with small sample size, high-dimensional feature space, and complex data structures.

Yanjun Qi

Backmatter

Titel: Ensemble Machine Learning
herausgegeben von: Cha Zhang
Yunqian Ma
Verlag: Springer New York
Electronic ISBN: 978-1-4419-9326-7
Print ISBN: 978-1-4419-9325-0
DOI: https://doi.org/10.1007/978-1-4419-9326-7