Discriminative kernel-based metric learning for face verification

Highlights

• Simplistic doublet constraint metric learning framework that leverages kernel machine learning and discriminative analysis.

• Recast a metric learning design problem as a classification/regression problem with kernel machines.

• Reformulated as a quadratic kernel function through SVM, LSSVM and KRR.

• The proposed framework suggests a generic way to generate efficient closed form solution.

• Solves the unconstrained face problem by minimizing the dependency of the outside information.

Abstract

This paper outlines a simplistic formulation for doublet constrained discriminative metric learning framework for face verification. The Mahalanobis distance metric of the framework is formulated by leveraging the within-class scatter matrix of the doublet and a quadratic kernel function. Unlike existing metric learning methods, the proposed framework admits efficient solution attributed to the convexity nature of the kernel machines. We demonstrate three realizations of the proposed framework based on the well-known kernel machine instances, namely Support Vector Machine, Kernel Ridge Regression and Least Squares Support Vector Machine. Due to wide availability of off-the-shelf kernel learner solvers, the proposed method can be easily trained and deployed. We evaluate the proposed discriminative kernel-based metric learning with two types of face verification setup: standard and unconstrained face verification through three benchmark datasets. The promising experimental results corroborate the feasibility and robustness of the proposed framework.

Introduction

Over the years, face verification has captivated huge interest among the researchers. Various techniques of face verification have been actively studied and invented due to the application demands such as nonintrusive access control in security systems, verification for business transactions, military and law-enforcement applications and others. A face verification system will decide whether a pair of test faces is from the client (correct identity to be claimed as) or the imposter (fault identity to be assigned to). Human faces are immensely challenging due to the significant variations in appearance, which may be caused by varying poses, aging, lighting, expression and others. Moreover, face images captured in an unconstrained environment (the factors of variations in appearance are not controlled such as photos taken from news articles) increase the difficulty in verification task, not to mention the situation of applying restricted protocol [51], which usually consist of large intra-class variations and no additional information about the person.

In addition, limited training data are available in the real-world environment and it may be impractical for those face verification techniques that rely heavily on full label training information. The main reasons are the waste in the computational costs and storage and the difficulty of obtaining full information from a given individual in certain situations. Contrary to current deep learning approach that exploits gigantic outside training data [2], [30], the restricted protocol place the emphasis to examine the learning capacity of the innovated algorithms rather than the performance gain attributed to the training data volume. Hence, it is important to better simulate the real-world scenarios by inventing the face verification techniques without relying on the facial side information. In practice, the unconstrained yet restricted face verification remains beneficial, but not limited to the applications of, (1) Alzheimer's patient face verification: Alzheimer’s patients who lost their way home do not have the ability to remember their personal information due to memory lost. The facial image is the only data to verify the person against the government database. (2) Surveillance face verification: The facial image of a murder/robber suspect caught on a CCTV camera with no extra information of the individual is verified against the government database. The Boston Marathon bombing case in 2013 [66] is an example where the law enforcement failed to match the unconstrained face image of the bombing suspect against the government database without any additional information of the bombing suspect.

Metric learning plays an imperative role in contemporary face verification as well as in many machine learning problems [1], [2], [3]. Metric learning dedicates to learn a Mahalanobis distance metric $d_M={\mathbf{x}}_j^T\mathbf{M}{\mathbf{x}}_j$ where M is the Mahalanobis matrix, from the training samples in measuring the similarity score between ${\mathbf{x}}_i$ and ${\mathbf{x}}_j$ by enhancing the similarity of matched pairs and suppressing the similarity of the mismatched pairs. Various metric learning formulations can be established based on the objective functions that designed for the specific task [4], [5], [6], [7], [8].

The problem of learning a good metric from training data is of critically essential. A good metric learning method should be able to reduce the influence of non-informative dimensions while emphasize on relevant dimensions [9]. There are two criteria to be considered when learning the Mahalanobis matrix. The first criterion is that the labels of the training data should be as weak as possible. Due to the fact that it is always difficult to obtain strong label such as class label of the training data in practice, it is preferable to design a metric learning algorithm that is based on the data pair labels (similarity or dissimilarity), which is more commonly available. For methods that depend much on the additional information, for example, the identity information of a person would be impractical in some applications such as verifying a stranger whose identity is not stored in the data bank or an intruder who repeatedly abuses the system. The second criterion in designing a good metric learning algorithm is that the algorithm should be computationally efficient. In this regard, the metric learning algorithm should not be heavy in computation.

On the other hand, kernel machines [10], [11], [12], [13] for classification and regression have had a huge impact to machine learning community over the past several decades [14], [15]. Kernel machines have been analyzed from the metric learning perspective lately and the intrinsic relationship of them is explored. A number of efficient algorithms have emerged, such as using metric learning notion in classifier design [16] and integrating support vector machine in metric learning [17].

Distance metric learning for face verification is the focus of this paper. In order to mitigate the problems of the existing distance metric learning algorithms for face verification, specifically with unconstrained images, a new distance metric learning framework is put forward. In this paper, the metric learning algorithm and the widely available kernel machines had coalesced into a simplistic metric learning framework for face verification to garner the advantages of both worlds. The Mahalanobis distance metric of this work is framed by leveraging the within-class scatter matrix of the doublet and a quadratic kernel function. To be specific, doublets are constructed from the weak label training pairs and reformulated as a quadratic kernel function through the popular kernel machine instances such as SVM, LSSVM and KRR. As such, the metric learning problem is transformed to classification or regression problem. Three realizations of the proposed framework are developed based on the well-known kernel machine instances such as Support Vector Machine [10], Least Squares Support Vector Machine [11] and Kernel Ridge Regression [12]. The decision to propose three different realizations is mainly due to the weaknesses and strengths of the kernel machine instances, which can be further improved and absorbed respectively, as discussed in Section 4.7.

In Section 2, we will briefly discuss the existing metric learning algorithms and kernel machines designed for face domain. It is followed by the motivation and contributions of this work in Section 3. The formulation of the three proposed realizations based on the proposed framework is given in Section 4. Section 5 reports the experimental results through the use of three publicly available datasets, including two standard datasets and one unconstrained dataset. Finally, a concluding remark and future work are presented in Section 6.