Advances in Low-Level Color Image Processing

Surveillance and inspection have an important role in security and industry applications and are often carried out with line-scan cameras. The advantages of line-scan cameras include hyper-resolution (larger than 50 Megapixels), continuous image generation, and low cost, to mention a few. However, due to the physical separation of line CCD sensors for the red (R), green (G), and blue (B) color channels, the color images acquired by multi-line CCD cameras intrinsically exhibit a color misalignment defect, such that the edges of objects in the scene are separated by a certain number of pixels in the R, G, B color planes in the scan direction. This defect, if not corrected properly, can severely degrade the quality of multi-line CCD images and hence impairs the functionality of the cameras. Current techniques for correcting such color misalignments are typically not fully automated, which is undesirable in applications such as inspection and surveillance that depend on fast unmanned responses. This chapter introduces an algorithm to automatically correct the color misalignments in multi-line CCD images for rotational scans as well as for translational scans. Results are presented for two different configurations of multi-line CCD imaging systems: (a) a close-range multi-line CCD imaging system for inspection applications and (b) a long-range imaging system for surveillance applications. Experimental results show that the two imaging systems are able to acquire hyper-resolution images and the color misalignment correction algorithm can automatically and accurately correct those images for their respective applications.

Generally, the color filter array (CFA) image is interpolated considering the correlation between color channels. Previous works first interpolate the green (G) signal, and then obtain the differences between the R/B signal and the reference signal (the initial interpolated G signal). To determine the direction of interpolation, the proposed method computes the horizontal/vertical absolute inter-channel differences directly computed from the CFA image. Then, three color components (R/G/B) are interpolated along the estimated horizontal/vertical directions considering the differences of absolute inter-channel differences. Comparative experiments using 24 test images with six conventional demosaicing algorithms show the effectiveness of the proposed demosaicing algorithm in terms of the peak signal to noise ratio, structural similarity, and subjective visual quality.

Color is an effective cue for identifying regions of interest or objects for a wide range of applications in computer vision and digital image processing research. However, color information in recorded image data, typically represented in RGB format, is not always an intrinsic property of an object itself, but rather it also depends on the illumination condition and sensor characteristic. When these factors are not properly taken into consideration, the performance of color analysis system can deteriorate significantly. This chapter investigates two common methodologies to attain reliable color description of recorded image data, color constancy and color invariance. Comprehensive overview of existing techniques are presented. Further, fundamental physical models of light reflection, and a color image formation process in typical imaging devices are discussed, which provide important underlying concepts for various color constancy and invariance algorithms. Finally, two experiments are demonstrated to evaluate the performance of representative color constancy and invariance algorithms.

The von Kries model is widely employed to describe the color variation between two pictures portraying the same scene but captured under two different lights. Simple but effective, this model has been proved to be a good approximation of such a color variation and it underpins several color constancy algorithms. Here we present three recent research results: an efficient histogram-based method to estimate the parameters of the von Kries model, and two theoretical advances, that clarify the dependency of these parameters on the physical cues of the varied lights and on the photometric properties of the camera used for the acquisition. We illustrate many applications of these results: color correction, illuminant invariant image retrieval, estimation of color temperature and intensity of a light, and photometric characterization of a device. We also include a wide set of experiments carried out on public datasets, in order to allow the reproducibility and the verification of the results, and to enable further comparisons with other approaches.

Although the statistical depth functions have been studied in nonparametric inference for multivariate data for more than a decade, the results of these studies have thus far been mostly theoretical. Out of numerous statistical depth functions, the halfspace depth function behaves very well overall in comparison with various competitors, and is one of the few statistical depth functions for which a small number of algorithms for computation in real Euclidean spaces have been proposed. In this chapter a new approach for removal of impulse and mixed multichannel noise based on a modified version of the only proposed algorithm for higher dimensional computation of the deepest location, i.e. a set of points with maximal halfspace depth, is discussed. A survey of experimental results shows that even in its baseline nonlinear spatial domain form, this filtering method gives excellent results in comparison to currently used state-of-the-art filters in elimination of wide range of powers of impulse and mixed multichannel noise from various benchmark image datasets. Multivariate nature of the implemented algorithm ensures the preservation of spectral correlation between channels and consequently, fine image details. Also, since the presented filter is independent of the source or distribution of the noise, it can be potentially used for removal of other types of multichannel noise.

This chapter is focused on spatially adaptive image processing for color images in the context of the General Adaptive Neighborhood Image Processing (GANIP) approach. The GANIP was first defined for gray-tone images and is here extended to color images. A set of local adaptive neighborhoods is defined for each image point, depending on the color intensity function of the image. These adaptive neighborhoods are then used as spatially adaptive operational windows for defining adaptive Choquet filters and adaptive morphological filters. The resulting adaptive operators are successfully applied and compared with the classical operators for image restoration, enhancement and segmentation of color images.

This chapter illustrates the suitability of recent multivariate ordering approaches to morphological analysis of colour and multispectral images working on their vector representation. On the one hand, supervised ordering renders machine learning notions and image processing techniques, through a learning stage to provide a total ordering in the colour/multispectral vector space. On the other hand, anomaly-based ordering, automatically detects spectral diversity over a majority background, allowing an adaptive processing of salient parts of a colour/multispectral image. These two multivariate ordering paradigms allow the definition of morphological operators for multivariate images, from algebraic dilation and erosion to more advanced techniques as morphological simplification, decomposition and segmentation. A number of applications are reviewed and implementation issues are discussed in detail.

Template matching is a fundamental problem in image analysis and computer vision. It has been addressed very early by Mathematical Morphology, through the well-known Hit-or-Miss Transform. In this chapter, we review most of the existing works on this morphological template matching operator, from the standard case of binary images to the (not so standard) case of grayscale images and the very recent extensions to color and multivariate data. We also discuss the issues raised by the application of the HMT operator to the context of template matching and provide guidelines to the interested reader. Various use cases in different application domains have been provided to illustrate the potential impact of this operator.

This chapter proposes two robust color edge detection methods based on tensor voting. The first method is a direct adaptation of the classical tensor voting to color images where tensors are initialized with either the gradient or the local color structure tensor. The second method is based on an extension of tensor voting in which the encoding and voting processes are specifically tailored to robust edge detection in color images. In this case, three tensors are used to encode local CIELAB color channels and edginess, while the voting process propagates both color and edginess by applying perception-based rules. Unlike the classical tensor voting, the second method considers the context in the voting process. Recall, discriminability, precision, false alarm rejection and robustness measurements with respect to three different ground-truths have been used to compare the proposed methods with the state-of-the-art. Experimental results show that the proposed methods are competitive, especially in robustness. Moreover, these experiments evidence the difficulty of proposing an edge detector with a perfect performance with respect to all features and fields of application.

In 1969, Brent Berlin and Paul Kay presented a classic study of color naming where experimentally demonstrated that all languages share a universal color assignment system of 11 basic color categories. Based on this work, new color categorization models have appeared in order to confirm this theory. Some of these models assign one category to each color in a certain color space, while other models assign a degree of membership to each category. The degree of membership can be interpreted as the probability of a color to belong to a color category. In the first part of this work we review some color categorization models: discrete and fuzzy based models. Then, we pay special attention to a recent color categorization model that provides a probabilistic partition of a color space, which was proposed by Alarcon and Marroquin in 2009. The proposal combines the color categorization model with a probabilistic segmentation algorithm and also generalizes the probabilistic segmentation algorithm so that one can include interaction between categories. We present some experiments of color image segmentation and applications of color image segmentation to image and video recolourization and tracking.

This chapter presents an overview of existing methods for human skin detection and segmentation. First of all, the skin color modeling schemes are outlined, and their limitations are discussed based on the presented experimental study. Then, we explain the techniques which were reported helpful in improving the efficacy of color-based classification, namely (1) textural features extraction, (2) model adaptation schemes, and (3) spatial analysis of the skin blobs. The chapter presents meaningful qualitative and quantitative results obtained during our study, which demonstrate the benefits of exploiting particular techniques for improving the skin detection outcome.

Face detection has been considered as one of the most active areas of research due to its wide range of applications in computer vision and digital image processing technology. In order to build a robust face detection system, several cues, such as motion, shape, color, and texture have been considered. Among available cues, color is one of the most effective ones due to its computational efficiency, high discriminative power, as well as robustness against geometrical transform. This chapter investigates the role of skin color cue in automatic face detection systems. General overview of existing face detection techniques and skin pixel classification solutions are provided. Further, illumination adaptation strategies for skin color detection are discussed to overcome the sensitivity of skin color analysis against illumination variation. Finally, two case studies are presented to provide more realistic view of contribution of skin color cue in face detection frameworks.

This chapter presents the saliency evaluation approach for visual design of video games, where visual saliency is an important factor to evaluate the impact of visual design on user experience of video games. To introduce visual saliency into game design, we carried out an investigation on several state-of-art saliency estimation methods, and studied on three approaches for saliency estimation: color-based, histogram-based, and information theory based methods. In experiments, these approaches were evaluated on a public saliency dataset and compared with the state-of-art technologies, and it was shown that the proposed information theoretic saliency model can attain a better performance in comparison with several state-of-art methods. Then we applied the information theoretic saliency model to visual game design with image and video examples and demonstrated on how to help game designers to evaluate their visual design with respect to the salience awareness of human visual perception systems.