Accuracy–diversity based pruning of classifier ensembles

Classification ensemble methods have recently drawn serious attention due to their ability to appreciably pull up prediction performance. Since smaller ensembles are preferred because of storage and efficiency reasons, ensemble pruning is an important step for construction of classifier ensembles. In this paper, we propose a heuristic method to obtain an optimal ensemble from a given pool of classifiers. The proposed accuracy–diversity based pruning algorithm takes into account the accuracy of individual classifiers as well as the pairwise diversity amongst these classifiers. The algorithm performs a systematic bottom-up search and conditionally grows sub-ensembles by adding diverse pairs of classifiers to the candidates with relatively higher accuracies. The ultimate aim is to deliver the smallest ensemble with highest achievable accuracy in the pool. The performance study on UCI datasets demonstrates that the proposed algorithm rarely misses the optimal ensemble, thus establishing confidence in the quality of heuristics employed by the algorithm.