Human gait recognition based on matching of body components

doi:10.1016/j.patcog.2006.11.012

Pattern Recognition

Volume 40, Issue 6, June 2007, Pages 1763-1770

https://doi.org/10.1016/j.patcog.2006.11.012 Get rights and content

Abstract

This paper presents a novel approach for gait recognition based on the matching of body components. The human body components are studied separately and are shown to have unequal discrimination power. Several approaches are presented for the combination of the results obtained from different body components into a common distance metric for the evaluation of similarity between gait sequences. A method is also proposed for the determination of the weighting of the various body components based on their contribution to recognition performance. Using the best performing of the proposed methods, improved recognition performance is achieved.

Introduction

Gait recognition [1] is a fairly new technique for biometric identification based on the walking style of individuals. Recognition based on human gait has several advantages related to the unobtrusiveness and the ease with which gait information can be captured. Unlike other biometrics, such as face, iris, fingerprints, gait can be captured from a distant camera, without drawing the attention of the observed subject.

A few early attempts on gait recognition were presented in Refs. [2], [3], [4]. In Ref. [2], gait patterns were extracted from spatiotemporal image volumes in order to outline the contours of walking subjects. These contours were used for gait recognition. In Ref. [3], a template matching method based on an eigenspace representation of gait was proposed. In Ref. [4], phase information of gait features was extracted, based on optical flow, and was subsequently used for gait recognition.

In Ref. [5], gait is represented as a two-dimensional stick figure constructed using the prior knowledge of human body structure, and leg angles calculated from the contour of the binary silhouette. The behavior of several joint angles from the stick figure within one walking cycle is studied and shown to be consistent with known medical data. In Ref. [6], a gait recognition method was proposed using recovered static body parameters. The static parameters that were used in Ref. [6] were the height, the distance between head and pelvis, the maximum distance between pelvis and feet and the distance between the feet. However, gait dynamics were not used and no attempt was made to quantify the contribution of each of these parameter in recognition performance.

One of the first methods that attempted to divide the human body into components that are treated separately was presented in Ref. [7]. In that work, a silhouette is considered to consist of seven ellipses. They study the behavior of each of these ellipses throughout the walking period in terms of the change of the magnitude component and the phase component obtained from Fourier transform, as well as the discrimination power of each of these components. Feature vectors derived from the magnitude and phase components were used in person identification and gender classification tests, and produced good results based on a relatively small data set.

In Ref. [8], manually extracted and labelled silhouettes were used based on the USF Human ID Gait Challenge data set [9]. Although the manual silhouettes provide much more accurate representations of human shape, the results reported in Ref. [8] are inferior to those obtained using automatically extracted binary silhouettes. This is due to the fact that, in the automatically extracted binary silhouettes, there are correlated errors which contribute to the recognition performance. Consequently, more research needs to be conducted using the manually extracted and labelled silhouettes, which do not suffer from noise and background interference, in order to reliably determine what type of information carries discrimination power.

In this paper, we use the manual silhouettes of Ref. [8] in order to investigate the contribution of each body component of a walking person to the recognition performance of a gait recognition system. We provide a detailed analysis of the role and the contribution of each body component by reporting recognition results of systems based on each one of the body components. We also propose several ways to combine the results obtained using the independent body components, and we show that the best performing combination achieves better results than the system in Ref. [8] on the same manual silhouette data set.

The paper is organized as follows. Section 2 describes the component-wise comparison between two manual silhouettes. Section 3 presents the matching of gait styles for two silhouette sequences. Section 4 reports the detailed results using the proposed system. Finally, conclusions are drawn in Section 5.

Section snippets

Component-wise comparison

The manual silhouettes consist of body components which can be clearly distinguished from one another, such as those presented in Fig. 1. On the other hand, the binary silhouettes contain limited information and are also plagued by error pixels due to the imperfect background subtraction. The availability of manually segmented and labelled silhouettes allows a more reliable investigation of issues related to the discrimination power of body components in gait recognition.

As we can see in the

Gait style matching using human body component weighting

After calculating the distances between the corresponding body components, we formed a component distance vector $d_{ij} = [\begin{matrix} d_{1} (p_{1 i}, g_{1 j}) \\ d_{2} (p_{2 i}, g_{2 j}) \\ ⋮ \\ d_{8} (p_{8 i}, g_{8 j}) \end{matrix}] .$ We tested several approaches for the combination of the individual component distances into a single distance that quantify the dissimilarity between two silhouettes. Specifically, the total distance between two silhouettes was taken to be equal to the median, min, max, and a weighted sum of the distances corresponding to each body part. Such

Optimal determination of weighting coefficients

For the purpose of determination of the weights in Eq. (5), we use the gallery set as the training set, since the gallery set contains all the subjects that appear in the probe sets. We define a component-wise distance between the nth and the rth subject $D_{α} (n, r) = \frac{1}{N_{F}} \sum_{(i, j) \in F} d_{α} (p_{α i}, g_{α j}) .$ Note that the index $α$ in Eq. (7) is included in order to emphasize that only one body component is involved in the determination of the component-wise distance between two subjects. This distance, i.e. $D_{α} (n, r)$ , is

Experimental results

For the experimental evaluation of our method, we used the manual silhouettes¹ provided by the University of South Florida (USF). This database contains human gait sequences captured under different conditions, such as camera viewing angle, shoe type, walking surface, and walking with/without carrying a briefcase. The gallery (reference) set of gait sequences was used as the system database and the probe (test) sets B, D, H, K were considered to contain

Conclusion

We presented a new approach for gait recognition using manually extracted and labelled silhouettes. The human body components were studied separately and were shown to carry different discrimination power. Several approaches were presented for the combination of the results of the different body components into a common distance metric for the evaluation of similarity between gait sequences. By combining the results from all body components, improved recognition performance was achieved.

Acknowledgment

The authors would like to thank Professor Sudeep Sarkar for providing the manual silhouettes of the database of the University of South Florida.

References (12)

H. Murase et al.
Moving object recognition in eigenspace representation: gait analysis and lip reading
Pattern Recognition Lett.
(1996)
N.V. Boulgouris et al.
Gait recognition using linear time normalization
Pattern Recognition
(2006)
N.V. Boulgouris et al.
Gait recognition: a challenging signal processing technology for biometric identification
IEEE Signal Process. Mag.
(2005)
S.A. Niyogi, E.H. Adelson, Analyzing and recognizing walking figures in xyt, in: Proceedings of Computer Vision and...
J. Little et al.
Recognizing people by their gait: the shape of motion
Videre: J. Computer Vision Res.
(1998)
J.H. Yoo et al.
Extracting human gait signatures by body segment properties

There are more references available in the full text version of this article.

Cited by (79)

Vision-based approaches towards person identification using gait
2021, Computer Science Review
Visual surveillance has exponentially increased the growth of security devices and systems in the digital era. Gait-based person identification is an emerging biometric modality for automatic visual surveillance and monitoring as the walking patterns highly correlate to the subject’s identity. The scientific research on person identification using gait has grown dramatically over the past two decades due to its several benefits. It does not require active collaboration from users and can be performed without their cooperation. It is difficult to be impersonated and identification can be validated from low-resolution videos and with simple instrumentation. This paper presents a comprehensive overview of the exiting techniques, their key stages, and recent developments in vision-based person identification using gait. We reviewed the historical research on gait locomotion and explain that how it is used to recognize the identity. The article summarizes the different types of features that have been proposed to encode the biomechanics of gait and also groups them into different categories and subcategories based upon the similarity in their implementation. We also present the impact of different covariate factors that affect the performance of gait recognition systems and also discuss the recent works to cope with these challenges. Furthermore, a comparison of the recognition accuracies reported by the existing algorithms to assess their performance under verification and identification mode is also presented. A detailed summary of publicly available vision-based gait databases is also provided. Finally, it offers insight into the challenges and open problems for future perspectives in the field of gait recognition that can help to set the directions for future research in this field.
Forensic and Legal Medicine
2020, Neale’s Disorders of the Foot and Ankle
Joint intensity transformer network for gait recognition robust against clothing and carrying status
2020, Materials Today: Proceedings
Citation Excerpt :
Spatial metric learning based approaches concentrate on improving performance by learning a feature space from the original appearance based features that is more discriminant and robust against the covariates. There are two further categories within this family: whole based [10–12,25–27,46] and part based approaches [28–30,47]. For the whole based approaches, the holistic appearance based features are projected into a discriminative space to make them more robust against the covariate conditions.
Biometric systems are becoming increasingly important, as they provide a more reliable and efficient means of identity verification. Human identification at a distance has recently gained enormous interest among computer vision researchers. This process describes a simple but practical approach to gait recognition by detection and classification of the gait posture in image. On the other hand, background and foreground extraction maintenance model is also proposed for preventing any false information and to avoid overlapping results, poor segmentation and classification. Hence the features are extracted through the Gabor, CNN and then the classification is done by the multi SVM classifier. Index Terms— Joint intensity transformer network, joint intensity metric learning, gait recognition.
Person re-identification by unsupervised video matching
2017, Pattern Recognition
Citation Excerpt :
To handle often-occurring occlusion, Hofmann et al. [36] propose a specific dataset for evaluating their negative influence on gait recognition performance. Meanwhile, a number of part-based methods [37–39] are developed by assuming that matched people share common observed parts (COPs). For relaxing this assumption, Muramatsu et al. [40] reconstruct complete gait features from partially observed body parts without sharing COPs.
Most existing person re-identification (ReID) methods rely only on the spatial appearance information from either one or multiple person images, whilst ignore the space-time cues readily available in video or image-sequence data. Moreover, they often assume the availability of exhaustively labelled cross-view pairwise data for every camera pair, making them non-scalable to ReID applications in real-world large scale camera networks. In this work, we introduce a novel video based person ReID method capable of accurately matching people across views from arbitrary unaligned image-sequences without any labelled pairwise data. Specifically, we introduce a new space-time person representation by encoding multiple granularities of spatio-temporal dynamics in form of time series. Moreover, a Time Shift Dynamic Time Warping (TS-DTW) model is derived for performing automatically alignment whilst achieving data selection and matching between inherently inaccurate and incomplete sequences in a unified way. We further extend the TS-DTW model for accommodating multiple feature-sequences of an image-sequence in order to fuse information from different descriptions. Crucially, this model does not require pairwise labelled training data (i.e. unsupervised) therefore readily scalable to large scale camera networks of arbitrary camera pairs without the need for exhaustive data annotation for every camera pair. We show the effectiveness and advantages of the proposed method by extensive comparisons with related state-of-the-art approaches using two benchmarking ReID datasets, PRID2011 and iLIDS-VID.
GADER: GAit DEtection and Recognition in the Wild
2023, arXiv
Proposed Security System for Cities Based on Animal Recognition Using IOT and Clouds
2023, Proceedings - International Conference on Developments in eSystems Engineering, DeSE

View all citing articles on Scopus

About the Author—NIKOLAOS V. BOULGOURIS received the Diploma and the Ph.D. degrees from the Electrical and Computer Engineering department of the University of Thessaloniki, Greece, in 1997 and 2002, respectively. Since December 2004 he has been a Lecturer with the department of Electronic Engineering, Division of Engineering, at King's College London, UK. From September 2003 to November 2004 he was a Post-Doctoral Fellow with the University of Toronto, Canada. Previously, he was affiliated with the Informatics and Telematics Institute, Thessaloniki, Greece. Dr. Boulgouris has participated in several research projects in the areas of pattern recognition, image/video communication, multimedia security, and content-based indexing and retrieval. Dr. Boulgouris is a member of the British Machine Vision Association and a member of the IEEE.

About the Author—ZHIWEI CHI received his M.Sc. in Digital Signal Processing with Distinction from King's College London, UK, in 2005, and his B.Eng. in Communications Engineering from the Department of Information Engineering, Northeast China Institute of Electric Power Engineering, China, in 2004. Since October 2005, he has been a Ph.D. student with the Centre for Digital Signal Processing Research at King's College London.

View full text

Human gait recognition based on matching of body components

Abstract

Introduction

Section snippets

Component-wise comparison

Gait style matching using human body component weighting

Optimal determination of weighting coefficients

Experimental results

Conclusion

Acknowledgment

Pattern Recognition Lett.

Pattern Recognition

Gait recognition: a challenging signal processing technology for biometric identification

IEEE Signal Process. Mag.

Recognizing people by their gait: the shape of motion

Videre: J. Computer Vision Res.

Extracting human gait signatures by body segment properties