Securing Social Identity in Mobile Platforms

Social networks have become an indispensable part of everyday life by providing users with different types of interaction. However, sharing different types of data, such as text, image, video and etc., on social networks, gives rise to user privacy concerns and risks, while the user is not aware of that. In this chapter, we show how the images shared by users can be applied to fingerprint the acquisition devices and link user profiles on social networks.

Active authentication allows an individual’s identity to be continuously verified in a transparent fashion. For devices centered on user convenience, active authentication using behavioral biometrics is an appealing solution to user authentication since behavioral data can be captured as consumers naturally interact with their devices. However, while such implementations are user-friendly and help to counter some of the challenges associated with knowledge-based authentication methods (e.g., easily guessed passcodes), an adversarial attack must be carefully considered. In this regard, to gain unauthorized access to a secured device, an adversary may falsify biometric information through impersonating the legitimate user. This attack is often referred to as a presentation attack or biometric spoofing. Throughout this chapter, various attack scenarios on mobile devices are discussed for gait, keystroke and touch dynamics, and user-device interaction modalities. Presentation attacks are categorized according to the biometric modality, which may differ given the context of the sensor component involved. This chapter exposes multiple research gaps and challenges which could significantly strengthen adversary detection once addressed, while discussing novel research in which no sensor information is required.

Mobile identity applications allow people to use a mobile phone as a form of secure digital identity (ID) card for identification purposes. In this paper, we present a novel transferring method for identity data such as electronic passport or other identification document data between two mobile devices, i.e. mobile identity holder and reader, over a BLE channel and propose the definition of a new GATT (Generic Attributes) profile suitable for mobile identity applications. Using the proposed approach, we show that BLE standard profiles can simplify and speed up mobile identity data exchange for several use cases.

Due to the overlap between the fields of forensic investigation and biometric recognition, including face recognition, there have been several interesting applications that bridge the gap between the two sciences and better connect the associated communities. These applications have been developed with the intent to assist law enforcement officers with computer assisted and biometrics related capabilities. Thus, utilizing biometric algorithms within the forensics field can support law enforcement investigations in a wide array of applications, including fingerprint comparisons, sketch-to-photo face comparisons, and even find persons of interest via soft biometrics such as scars, marks, and tattoos. In this book chapter, we focus on facial recognition, which can help provide clues when other forensic evidence is not present or available and, most importantly, help investigators eliminate the time consuming processes of interviewing potential witnesses or manually searching through thousands of mugshots to determine a suspect’s identity. To aid in this mission, we propose a software toolkit to automatically and hierarchically categorize face images with a set of binary classifiers using three different attributes, which depending on their true label/condition can affect facial recognition performance. These attributes are: based on facial photo, (1) determining whether a subject’s eyes are open or closed, (2) whether the subject is wearing glasses or not, and (3) whether the facial pose of the subject is either frontal or non-frontal. Our toolkit offers batch processing and therefore can aid forensic operators with a capability to rapidly categorize large scale face datasets in terms of the aforementioned attributes, and thus, determine, which individuals have a higher chance to be identified based on their face information. The proposed forensic toolkit will allow the operators to analyze, enhance, group, or exclude face data before being used for face matching.

In this chapter, we investigate the advantages and limitations of the heterogeneous problem of matching Near-Infrared (NIR) long-range, night time, face images against their visible counterparts. Image quality degradation can result due to a variety of factors including low illumination, variable standoff distance, and is responsible for performance degradation of conventional face recognition (FR) systems. In addition to intra-spectral matching (i.e. NIR vs. NIR face images), cross-spectral matching (i.e. matching NIR face images against their visible counterparts) is a challenging matching scenario that increases system complexity. In this work, we propose the usage of a set of FR algorithms when working with operational-based face matching scenarios, namely, where the face images used are collected by a night vision, long range (from 30 to 120 m), NIR-based face imaging system. First, we establish a system identification baseline using a set of commercial and academic face matchers. To improve baseline performance, we propose a scenario dependent convolutional neural network (CNN) to, first, categorize the face images of our challenging face dataset, in terms of gender, ethnicity, and facial hair. For each of the aforementioned generated categories, we apply our proposed algorithmic pipeline including, image restoration and a multi-feature based fusion scheme. Then, a set of FR algorithms are used before and after image restoration and data categorization. Based on the experimental results, we conclude that our proposed image restoration and fusion schemes, as well as the usage of demographic-based face categories, result in improved identification performance. For example, for the 30 m vs. 30 m NIR face matching scenario, the rank-1 identification rate is improved from 48% (all vs. all) using a commercial face matching system to 73% (all vs. all) and to 82% (if we use only the male with beards face data category). Experimental results suggest that our proposed methodological approach can improve system performance (i.e. efficiently identifying the subject of interest) on various cross-spectral face matching scenarios.

Face recognition performance in operational scenarios is can be improved by using cameras that capture multispectral or hyperspectral images at specific bands within the visible spectrum. Band-selected images have shown promise to improve face recognition performance, but the requisite camera systems needed to achieve multi-filter or hyperspectral imaging are often to complex and cost-prohibitive for many law enforcement applications. In order to find a more cost-effective solution, the work presented here aims to determine if simple band-filtered images, captured by placing bandpass filters on conventional RGB imagers, show any application advantages over broad-spectrum visible facial imagery. After data collection was completed, matching studies were performed to determine what performance enhancement, if any, is gained using band-filtered imaging. Results indicate that image quality may play a bigger role in the facial recognition performance of band-filtered images rather than simple band-filtering alone, warranting further study in this area.

The ever growing field of face recognition is constantly expanding to tackle new and more challenging, problems as the advances in algorithms yield higher accuracy results. The most recent advances have opened up the possibility of conducting high accuracy face recognition on faces from completely uncontrolled sources, such as search engines, social-media, and other online sources. Conducting face recognition in this area is usually deemed as faces-in-the-wild, given the unbounded nature in which faces are collected. While performing face recognition on faces-in-the-wild datasets has many advantages, it can make it difficult to determine the limitations of the face recognition algorithm in terms of the scenarios in which the faces were collected. In this work, we will collect a simulated faces-in-the-wild dataset using four cell phones (common sources for faces-in-the-wild) in varying scenarios (distance, lighting, background, etc.) to fully demonstrate the capability of newly proposed deep learning based methods of face recognition. Furthermore, we will contrast this with previous, standard, methods of face recognition in the same scenarios to see how recent improvements in the filed have opened up new capabilities.

The capability to perform face recognition in the visible and thermal spectra is of prime interest in many law enforcement and military organizations. Face detection is an important pre-processing step for face recognition. Though many algorithms are available for face detection in the visible spectrum, an assessment of how these algorithms can be retrained for the thermal spectrum is an important study. Current available visible-based face detection algorithms are very effective in daytime conditions, however, when there are extreme changes in illumination conditions like very low-light to no light (night-time), these become challenging. Due to limited amount of data available for researchers from sensors in the thermal band (due to the increased cost of having and operating state of the art thermal sensors), there are only a few proposed algorithms. In this work, we conducted a study to determine the impact of factors such as indoor/outdoor environment, distance from the camera, application of sunscreen, training set size, etc. on training deep-learning models for a face detection system in the thermal spectrum that simultaneously performs face detection and frontal/non-frontal classification. Existing deep learning models such as SSD (Single Shot Multi-box Detector), R-FCN (Region Based Fully Convolutional Network) and R-CNN (Region Based Convolutional Neural Network), are re-trained using thermal images for face detection and pose estimation tasks. Results from each model are compared, and the model with the best performance is further trained and tested on different datasets, including indoor, outdoor at different stand-off distances. The highest accuracy is achieved using a Faster R-CNN model with ResNet-101 and the accuracy is 99.4% after a 10-fold cross-validation. More experiments are performed to further study the efficiency and limitations of this model. The data set we use was collected under constrained indoor and unconstrained outdoor conditions.

In this chapter we evaluate mobile active authentication based on an ensemble of biometrics and behavior-based profiling signals. We consider seven different data channels and their combination. Touch dynamics (touch gestures and keystroking), accelerometer, gyroscope, WiFi, GPS location and app usage are all collected during human-mobile interaction to authenticate the users. We evaluate two approaches: one-time authentication and active authentication. In one-time authentication, we employ the information of all channels available during one session. For active authentication we take advantage of mobile user behavior across multiple sessions by updating a confidence value of the authentication score. Our experiments are conducted on the semi-uncontrolled UMDAA-02 database. This database comprises of smartphone sensor signals acquired during natural human-mobile interaction. Our results show that different traits can be complementary in terms of mobile user authentication and multimodal systems clearly increase the performance when compared to individual biometrics systems with accuracies ranging from 82.2% to 98.0% depending on the authentication scenario.

In this chapter, we investigate how to detect intruders with low latency for Active Authentication (AA) systems with multiple-users. We extend The Quickest Change Detection (QCD) framework is extended to the multiple-user case and the Multiple-user Quickest Intruder Detection (MQID) algorithm is formulated. Furthermore, the algorithm is extended to the data-efficient scenario where intruder detection is carried out with fewer observation samples. The effectiveness of the method is evaluated on two publicly available AA datasets on the face modality.

Methods of biometric recognition are becoming an essential part of various mobile applications. Their usability is determined by the accuracy and the speed of recognition in a highly variable environment. Complex textural features make the human iris one of the most reliable biometric traits. The changing environment and limited computational power of mobile devices give rise to a need for robust and fast feature extraction techniques. A method for iris feature extraction and matching is here proposed. It uses deep and element-wise representations of the discriminative features in combination with characteristics describing the environment. The model outperforms state-of-the-art methods in terms of both accuracy and speed. It has also been tested on a specially collected dataset that contains two-second videos simulating the natural enrollment and verification attempts of the user of the device. The dataset was collected considering the changes in environment and possible behavior of the user. The testing was performed in two scenarios: image-to-image and also video-to-video. A method for iris fusion (both eyes) is also proposed in this paper. Several such methods are studied and compared.

Most user authentication methods and identity proving systems rely on centralized databases. Such information storage presents a single point of compromise from a security perspective. If this system is compromised, it poses a direct threat to a significant number of users’ digital identities. A recent example of compromised data includes the Equifax breach, which affected 140 million people. The other issue with these centralized systems that individuals don’t have a control of how much of their Personal Identifying information (PII) is shared in different contexts.

This chapter discusses a decentralized biometric-based authentication protocol for identity ecosystems, called the Horcrux (The term “Horcrux” comes from the Harry Potter book series in which the antagonist (Lord Voldemort) places copies of his soul into physical objects. Each object is scattered and/or hidden to disparate places around the world. He cannot be killed until all Horcruxes are found and destroyed.) protocol, in which there is no such single point of compromise. The Horcrux protocol is founded on the principle that an individual should have a control over the use of their own PII. The decentralization of control over the components of individual identities will allow them proof of their PII – secured by blockchains and cryptography – to governmental and private-sector entities. Meanwhile, BOPS will enable these entities to undertake an advanced risk assessment, verify identities and provide seamless access through secure mobile biometric recognition technology. All of this can be achieved without the need to store PII in one central database and pose too great a risk for stakeholders. Horcrux protocol relies on decentralized identifiers (DIDs) under development by the W3C Verifiable Claims Community Group and the concept of self-sovereign identity. In this chapter, we discuss the specification and implementation of a decentralized biometric credential storage option via blockchains using DIDs and DID documents within the IEEE 2410–2017 Biometric Open Protocol Standard (BOPS).

Continuous Authentication (CA) is the process of constantly checking for the authorized user’s presence, which brings unique advantages and disadvantages. CA is more secure and facilitates schemes with multiple levels of authentication security; however, it can consume more resources and cause user anxiety about privacy. In this chapter we seek to understand the practical aspects of CA; in particular, user perception and resource consumption. To gauge user perception towards CA, we conducted a survey with roughly 500 respondents. We found that users desire multiple levels of authentication security. Furthermore, users are willing to adopt CA for mobile devices. We then analyzed factors like security awareness, gender, and mobile device OS, to draw statistically significant conclusions regarding their effect on users’ willingness to adopt CA, and user perceptions about CA. We also compare between biometric modalities based on their resource consumption, as measured by their Resource Profile Curve (RPC). This Curve reveals the trade-off between authentication accuracy and resource usage, and is helpful for different usage scenarios in which a CA system needs to operate. In particular, we explain how a CA system can intelligently switch between RPCs to conserve battery power, memory usage, or to maximize authentication accuracy. We argue for the importance of understanding user perceptions and using RPCs to guide the development of practical CA systems.