10. Document to Live Facial Identification

The National Institute for Standards and Technology (NIST) highlights that facial recognition (FR) has improved significantly for ideal cases such, where face photographs are full frontal, of good quality, and pose and illumination variations are not significant. However, there are automated face recognition scenarios that involve comparing degraded facial photographs of subjects against their high-resolution counterparts. Such non-ideal scenarios can be encountered in situations where the need is to be able to identify legacy face photographs acquired by a government agency, including examples such as matching of scanned, but degraded, face images present in drivers licenses, refugee documents, and visas against their live photographs for the purpose of establishing or verifying a subject’s identity. The factors impacting the quality of such degraded face photographs include hairstyle, pose and expression variations, lamination and security watermarks, and other artifacts such as camera motion, camera resolution, and compression. In this work, we focus on investigating a set of methodological approaches in order to be able to overcome most of the aforementioned limitations and achieve high identification rate. Thus, we incorporate a combination of preprocessing and heterogeneous face-matching techniques, where comparisons are made between the original (degraded) photograph, the restored photograph, and the high-quality photograph (the mug shot of the live subject). For the purpose of this study, we, first, introduce the restorative building blocks that include threshold-based (TB) denoising, total variational (TV) wavelet inpainting, and exemplar-based inpainting. Next, we empirically assess improvement in image quality, when the aforementioned inpainting methods are applied separately and independently, coupled with TB denoising. Finally, we compare the face-matching performance achieved when using the original degraded, restored, and live photographs and a set of academic and commercial face matchers, including the local binary patterns (LBP) and local ternary patterns (LTP) texture-based operators, combined with different distance metric techniques, as well as a state-of-the-art commercial face matcher. Our results show that the combination of TB denoising, coupled with either of the two inpainting methods selected for the purpose of this study, illustrates significant improvement in rank-1 identification accuracy. It is expected that the proposed restoration approaches discussed in this work can be directly applied to operational scenarios that include border-crossing stations and various transit centers.