Approximately symmetrical face images for image preprocessing in face recognition and sparse representation based classification

Highlights

• Developed a novel method to automatically produce approximately axis-symmetrical virtual face images.

• Treated as an effective image preprocessing method.

• Used as a virtual image dictionary learning method for image classification.

• Extensive experiments on different face databases show its effectiveness as an image preprocessing algorithm.

• The strong identification capability of our method is verified in comparison with state-of-the-art dictionary learning algorithms.

Abstract

Though most of the faces are axis-symmetrical objects, few real-world face images are axis-symmetrical images. In the past years, there are many studies on face recognition, but only little attention is paid to this issue and few studies to explore and exploit the axis-symmetrical property of faces for face recognition are conducted. In this paper, we take the axis-symmetrical nature of faces into consideration and design a framework to produce approximately axis-symmetrical virtual dictionary for enhancing the accuracy of face recognition. It is noteworthy that the novel algorithm to produce axis-symmetrically virtual face images is mathematically very tractable and quite easy to implement. Extensive experimental results demonstrate the superiority in face recognition of the virtual face images obtained using our method to the original face images. Moreover, experimental results on different databases also show that the proposed method can achieve satisfactory classification accuracy in comparison with state-of-the-art image preprocessing algorithms. The MATLAB code of the proposed method can be available at http://www.yongxu.org/lunwen.html.

Introduction

In the face recognition community, various algorithms have been proposed to remedy difficulties caused by the small sample size (SSS) problem and variations of poses, illuminations and facial expressions [1], [2]. It should be pointed out that the problem of varying poses, illuminations and facial expressions usually has the following characteristics. Each face usually has severe variations of poses, illuminations and facial expressions but a face has only a limited number of available face images [3]. This of course causes a great intra-subject difference and the difference may even be greater than the inter-subject difference. This may incur the result that different faces have similar patterns [4], [5]. As a consequence, the performance of face recognition algorithms will be seriously affected.

Many efforts have been made to address these difficulties from the viewpoint of methodology. For example, when linear discriminant analysis (LDA) [6], [8] is applied to face recognition, it usually suffers from the SSS problem. In other words, because the number of face samples for training is almost always smaller than the dimension of the face sample, LDA cannot be directly implemented. A number of variants of the original LDA algorithm have been proposed to overcome this problem. Moreover, various methods have been proposed to perform face recognition across poses, illuminations and facial expressions and to reduce disadvantageous effect on recognition of faces of variations of poses, illuminations and facial expressions [7].

Besides special algorithms are designed to overcome above difficulties, to obtain more virtual face samples is another important way to improve the performance of face recognition [9], [10]. In recent years researchers have made notable advances in this way. For example, symmetrical face images proposed in [11] are very beneficial to overcome the problem of varying poses and illuminations. The mirror face image based face recognition method proposed in [12] has the following merits. The used mirror face images not only can provide possible poses of faces that are not shown in the original face images but also offer us natural face images. Li et al. [13] proposed to enlarge the number of the training samples by exploiting the inter-class relationship and combined the original samples and obtained virtual samples to perform face recognition. The simultaneous use of these synthesized images and original face images can lead to very good performance. Besides the above symmetrical face images and mirror face images, noised or corrupted face images are also available virtual face samples [14], [15].

We see that a few methods take the facial geometry into account when generating more available face samples. However, when humans recognize faces, they often exploit the facial geometry information. A common property of most faces is that they have a symmetrically geometric structure. In other words, the left-half face is almost always the “mirror image” of the right-half face. This motivates us to explore the way to exploit the symmetrical structure of faces when designing automatic face recognition algorithms. It has been proved that the performance of some tasks such as face detection can be promoted by using the symmetrical structure of faces [16], [17], [18]. The facial symmetry can also be applied for shadow compensation and analysis [19], [20]. Moreover, the symmetry of faces has also been shown to be partially beneficial to 3D face recognition [21], [22], [23]. However, it seems that how to effectively exploit the facial symmetry to perform 2D face recognition is not studied in-depth. Especially, in the 2D face recognition community, there are only few studies on how to design an algorithm of automatically generating symmetrical face images. For 2D face recognition, the literature showed that if the 2D face image is completely an axis-symmetrical image, then we can use only one-half of the face image for the classification of faces [24]. However, most of the real 2D face images are not axis-symmetrical. Thus, if only one-half of the face image is used for matching and classification of faces, there will be much information loss and the classification performance will be degraded. Previous research also demonstrates that enhancing the symmetry of a face image is a meaningful and effective strategy [25], [26].

In the past, there were some studies on preprocessing of face images [29], [30]. For example, some methods to reduce effects on face recognition of illuminations have been proposed [27], [28]. With the development of the time–frequency theory, frequency domain features such as discrete cosine transform (DCT) feature [31] were extensively used for face recognition. Because the small-scale feature is more robust to illumination variations in recognition tasks, the logarithmic total variation (LTV) model [32] and the logarithmic wavelet transform (LWT) method [33] based on the multi-scale facial structure representation were proposed as image preprocessing methods for face recognition. In the light of the fact that each face image can be constructed by the integration of the large- and small-scale features, Xie et al. [34] proposed to integrate the large- and small-scale features (LSSF) of face images to perform normalization of face illumination. Zhang et al. [35] proposed to use gradientfaces (GRF) and Šrtuc and Pavešić [36] presented a non-local means based normalization method (NLM) as image preprocessing techniques for robust face recognition. Cyganek et al. [37] proposed a very impressive method, which used Support Vector Machines with specific tensor kernel and achieved high accuracy for processing multidimensional images (among which are face datasets) without any need for sophisticated pre-processing or posing restrictions on the face pose. In addition, the Lambertian reflectance model also has received much attention in remedying the problems caused by unbalanced illuminations [39]. However, a real face image is usually not completely subject to the assumption of this model. The illuminating light field method also seems to be applicable for face recognition across illuminations and poses [40]. However, this method is still established on the basis of the Lambertain reflectance model.

In this paper, we propose a novel face image preprocessing method and also use it to generate a virtual dictionary for image classification. The proposed method aims at producing virtual face images with an approximately symmetrical structure, which will be utilized in both image preprocessing and image classification. Our method has the following notable merits. (1) Our method can well reduce the negative effect on face images of heterogeneous illuminations and it can be used as a face image preprocessing method. (2) The proposed method can automatically produce an approximately axis-symmetrical virtual face image from an original face image. Because the virtual face image is approximately axis-symmetrical, it can effectively reduce negative effects on face recognition of varying face appearance. As a result, the use of the virtual face images enables the accuracy of a face recognition algorithm to be improved. Thus, this not only provides a good solution to the face image preprocessing problem, but also can generate geometrically symmetrical and more attractive face images. To evaluate the performance of our method in two different applications, i.e. face image preprocessing and generation of virtual dictionary, we design experiments composed of two parts. We first applied our method as an image preprocessing method to all images including training samples and test samples from different datasets, and then exploited the nearest neighbor classifier for classification. In addition, we also applied our method on different face databases to generate virtual training dictionaries and then employed different sparse presentation based classification algorithms for classification. It should be pointed out that in the second part of experiments, our method was applied only to the original training sample and the test samples were not used to generate virtual samples. This makes use of our method partially similar to the dictionary learning method in face recognition. In summary, the paper has the following important contributions:

• It proposes an originally creative idea and algorithm to automatically produce approximately axis-symmetrical virtual face images.

• The proposed method can be treated as an image preprocessing method for face recognition, and extensive experiments on different face databases show the effectiveness of our method.

• The images synthesized by our method can be used as a virtual image dictionary, and the strong recognition capability of our method is verified in comparison with state-of-the-art dictionary learning algorithms in image classification.

• As an unsupervised image classification and image preprocessing method, our method can work much better than the benchmark that uses the original image datasets for classification.

The rest of our paper is organized as follows. Section 2 presents the description of the proposed method in detail. The analysis and advantages of the proposed method and the obtained virtual images are shown in Section 3. In Section 4, we introduce the experimental evaluation of our method as the image preprocessing method on some widely used databases. In Section 5, the experimental evaluation of generating the virtual dictionary is reported. The analysis and discussion are explicitly presented in Section 6. Finally, we give our conclusions in Section 7.