On-line boosting-based car detection from aerial images

Abstract

Car detection from aerial images has been studied for years. However, given a large-scale aerial image with typical car and background appearance variations, robust and efficient car detection is still a challenging problem. In this paper, we present a novel and robust framework for automatic car detection from aerial images. The main contribution is a new on-line boosting algorithm for efficient car detection from large-scale aerial images. Boosting with interactive on-line training allows the car detector to be trained and improved efficiently. After training, detection is performed by exhaustive search. For post processing, a mean shift clustering method is employed, improving the detection rate significantly. In contrast to related work, our framework does not rely on any priori knowledge of the image like a site-model or contextual information, but if necessary this information can be incorporated. An extensive set of experiments on high resolution aerial images using the new UltraCamD shows the superiority of our approach.

Introduction

Building an efficient and robust framework for object detection from aerial images has drawn the attention of research community in computer vision for years (e.g., Ruskone et al., 1996, Rajagopalan et al., 1999, Zhao and Nevatia, 2003, Hinz, 2003, Alba-Flores, 2005). An aerial image contains a lot of objects with a complicated background of the urban scene. The UltraCamD camera from Microsoft-Vexcel can deliver large format panchromatic images as well as multi spectral images (Leberl et al., 2003). The high resolution images have a size of 11,500 pixels across-track and 7500 pixels along-track. Thus, a panchromatic image has a size of 84 MB and a RGB or NIR (near infrared) image has a size of 252 MB. These large images need automatic methods for efficient processing.

Car detection from aerial images has a variety of civil and military applications, such as transportation control, road verification to support land use classification for urban planning, military reconnaissance, etc.

Aerial images are usually taken from vertical direction. Although with some constraints on the viewpoint, the appearance of the cars in the image is varying widely. Cars appear as small objects, which vary in intensity and many details are not visible. Depending on the resolution a typical car has a size between 13 and 26 pixels (Zhao and Nevatia, 2003). The appearance of cars may have parts occluded by the shadow of buildings or trees, or may be dominated by the shadow of the car. Moreover, the urban scene comprises a complicated background with a variety of objects that look like cars such as windows, roofs, corners of streets, or buildings. All these issues make it difficult to characterize the features of a car and impose challenges in recognition of cars from aerial images. Therefore, although a lot of efforts have been made, it is still an open problem to build an efficient and robust algorithm for automatic car detection from aerial images.

In recent years, boosting, a machine learning method, has become popular. Referring to the overview given in Schapire (Schapire, 2003), boosting has been used for text recognition, routing, medical diagnostic, segmentation, etc. Various boosting frameworks have been developed for solving machine learning problems (Schapire, 2003, Demiriz et al., 2002, Freund and Schapire, 1997, Stojmenovic, 2006). Following the remarkable success of the face detector introduced by Viola and Jones (Viola and Jones, 2001), boosting techniques have been widely used for different problems in the computer vision community. The detection problem is formulated as a binary classification problem, discriminating the object from the background. The learned classifier is evaluated on the whole image. In order to speed up the exhaustive search, in the classical work of Viola and Jones (2001) integral images were employed, which allow very fast computation of simple image features for object representation. Additionally, a cascade structure makes the detector simultaneously fast and accurate. This framework allows to proceed efficiently on large image data and has been successfully applied for various object detection problems.

Most of the above work uses Adaboost for the detection of objects in terrestrial images. None of them (up to our knowledge) uses boosting methods for object (car) detection from aerial images. In this paper, we propose a robust boosting-based system for car detection from aerial images. The main goal is high quality detection by using novel machine learning methods with an efficient training mechanism.

First, we use boosting and particularly an efficient integral image representation for fast calculation of cars' features. In addition to the commonly used Haar wavelets (Viola and Jones, 2001), we employ local orientation histograms (Dalal and Triggs, 2005) and local binary patterns (Ojala et al., 2002) as features.

Second, we use a novel on-line version of Adaboost to train the detector. It performs on-line updating on the ensembles of features during the training process. By on-line training, we can update the classifier as new samples arrive, and therefore we can minimize the tedious work of hand labeling of training samples.

The developed framework results in a robust and automatic car detection system from aerial images achieving high performance. The system is flexible since it does not require any site-model or contextual knowledge or other information influencing the appearance of cars in images.

The paper is organized as follows. Section 2 gives a brief review of related work. Section 3 presents our approach for car detection from aerial images. Section 4 is dedicated to experiments and results. It also discusses the suitability data delivered by UltraCamD to integrate our system with related applications. Finally, Section 5 ends up with discussion and future work.