Adaptive color space switching for tracking under varying illumination

https://doi.org/10.1016/j.imavis.2004.09.005Get rights and content

Abstract

Many studies use color space models (CSM) and color distribution models (CDM) for detection of faces in an image. We develop a procedure that adaptively switches CSMs throughout the processing of a video. We show that this works in environments with varying types of illumination. In addition, a new performance measure for evaluating tracking algorithms is proffered. Extensive testing of the procedure found that switching between the color spaces resulted in increased tracking performance when compared to using single CSMs throughout. The methodology developed can be used to find the optimal CSM–CDM combination in the design of adaptive color tracking systems. The adaptive color space switching algorithm has linear computational time complexity O(S), at each iteration, where S is the picture size in pixels.

Introduction

Many interesting and useful applications have been developed using facial digital images. The face detection problem is of prime importance and involves the determination of whether or not there is a human face present in an image, and if present to return its location. Detection of faces is often preceded by the extraction of various cues. Typically, such cues take the form of face features such as; shape, color, motion, and relational position to the rest of the human body or other objects in the scene (such as desks or doorways). In this paper, we develop a skin color based tracking system. We have found that in environments where the face moves through non-stable illumination conditions involving multiple light sources, it is advantageous to employ a dynamic approach. Such an approach adaptively switches between a number of color space models (CSM) as a function of the state of the environment, as well as, dynamically updating the corresponding color distribution model (CDM). Although, adaptively updating the CDM has received some attention in the literature, we believe this is the first attempt to adaptively select CSMs throughout a tracking sequence.

Color space and color distribution models are described in Section 2. In 3 Adaptive color distribution models, 4 Adaptive color space switching, adaptive CDM's and color space switching, are discussed, respectively. In Section 5, a new color space quality measure is proffered. In Section 6, our adaptive color space switching architecture and algorithm is presented, along with its computational complexity. In Section 7, new face tracking performance measures are proffered. Section 8 is devoted to the experimental design for testing the performance of switching vs. non-switching. Experimental results are presented and analyzed in 9 Experimental results, 10 Analysis of switching results, respectively. Section 11 is devoted to the explanation of a case where a single switch occurs. Conclusions and future research are the subjects of Section 13.

Section snippets

Color space model selection

Numerous types of CSMs are used for segmentation and tracking of objects in a scene. Most CSMs represent colors in a three coordinate color space such as RGB and HSV. Conversion formulas are used to transform from one space to another (see Appendix for some examples). Because color components are correlated, they are often reduced to two or even one-dimension. This is also done to reduce computational load and because of the desire to eliminate phenomena such as brightness from the scene. See

Adaptive color distribution models

To adjust the CDM to changing illumination adaptive update schemes have been developed, in which pixel colors from the face region are collected and used as input to build a new temporal CDM. Let us denote it as CDMtnew. The new CDMtnew and current CDMt are used to construct the updated CDMt+1, which is to be used for the next detection, and kept until the next CDM-update operation. Let the smoothing parameter to be α, then given a color represented in a two-dimensional color space binned

Adaptive color space switching

There are many researchers who use CSMs for the selection or segmentation of objects. The main objective is to achieve the best segmentation of an object possessing a certain color gamut from the background. There are two cases with respect to knowledge about the object and the background.

  • Case (a)—There exists some knowledge (given by samples) about the background as well as the object. The solution is to construct a color model for each of them. The best color space is then chosen using some

Color space quality measure

We now want to find a measure to select a CSM, which is more likely to separate the human face from the background. This means that face pixels in the FPI, should display brighter values (values closer to 1) and the area close to the face (actually the background) should display a minimum amount of intensity values. To achieve this we separate the area of attention into two prime parts using the tracking window as a base; (i) an internal rectangular window containing the face, and (ii) an

Adaptive color space switching algorithm

We combine a number of procedures to construct an enriched face tracking approach. We not only update our CDM as others do [3], [13], but switch in and out different color spaces throughout the tracking sequence. In our initial investigations a histogram CDM is used rather than a gaussian type model, for real time considerations, but the system is general enough to accommodate any CDM. In fact in Section 12, a comparison of various CDMs with and without CSM switching is performed.

Face tracking performance measures

A review of the literature revealed little discussion of well-defined measures for face tracking quality. Many authors where satisfied with simple ‘successful/unsuccessful’ classifications. We propose a two level performance measure: (i) primary objective—avoid loosing the tracking window, i.e. the disjunction of the search window region and face area should not to be empty at any time; and (ii) secondary objective—bring the tacking window Wt as close as possible to the ‘minimal face-enclosing

Experimental design

The performance of the CSM switching face tracking system was tested under different input environments. Since our main objective is to develop a face tracking system, using an adaptive color model, for unstable backgrounds and illumination, the test cases were chosen accordingly. A well-designed experiment should be able to test the tracking robustness under different types of object motion, background conditions and illumination. Video sequences were designed to include typical object

Experimental results

The results are initially presented in Section 9.1 below in qualitative terms (success, failure). This is followed by Section 9.2 where quantitative tracking performance results are presented for the stable light, unstable brightness and multicolored environment categories.

Summary of overall color-switching results

The results for color switching are presented in Table 5. The entries in the table represent the percent of frames for which each CSM was selected.

Statistical validation

To perform a comparative evaluation of switching versus non-switching, we performed system runs using single fixed CSMs. The qualitative tracking results (primary objective) are shown in Table 6. For each case, single CSM runs were made for only those CSMs that participated in the switching history (shown in Table 5). Cases 1 and 7 where not examined

A single switch example

Case 5 was used to help explain the meaning of color-space switching under unstable brightness (see Table 5). Note, here no fixed CSM alone provides successful tracking (see Table 6). A sample history of the adaptive color space-switching model can be seen in Fig. 6. The horizontal axis represents time (frame number). The vertical axis represents the quality measure, rk, under each color space k at each given time epoch. If there is at least one intersection of the upper line, tracking with

Comparison of CDMs with and without CSM switching

In the previous results a single CDM was used. In order to demonstrate the use of the system to select a CDM among a collection of candidate CDMs we used the tracking performance measure to evaluate five different CDM types: Probability Model (Pr) [7], Possibility Model (Ps) [5], Gaussian with one component (G1), Gaussian with three components (G3) and Gaussian with dynamic number of components (GD) [7]. However, first we present qualitative results in terms of success and failure in Table 8,

Conclusions and future work

In this work an adaptive color space-switching algorithm is proffered. The adaptive color space switching algorithm has linear computational time complexity, O(S), in each iteration where S is the picture size (number of pixels in image). In addition, we have proposed a new performance measure for evaluating tracking algorithms, which includes both accuracy and robustness (lost tracking window) of the tracking window. These measures and statistical tools were used to validate the algorithm

Acknowledgements

This research was partially supported by the Paul Ivanier Center for Robotics Research and Production Management, Ben-Gurion University of the Negev.

References (19)

There are more references available in the full text version of this article.

Cited by (0)

View full text