Elsevier

Pattern Recognition Letters

Volume 78, 15 July 2016, Pages 56-63
Pattern Recognition Letters

Human gait recognition based on deterministic learning through multiple views fusion

https://doi.org/10.1016/j.patrec.2016.04.004Get rights and content

Highlights

  • Robust gait recognition via deterministic learning and multiple views fusion.

  • Multiple views fusion strategy is employed to form synthesized silhouette.

  • Human gait can be characterized with four time-varying gait features.

  • Gait variability are effectively modeled via deterministic learning.

  • A rapid recognition scheme is chosen for encouraging recognition accuracy.

Abstract

Gait characteristics extracted from one single camera are limited and not comprehensive enough to develop a robust recognition system. This paper proposes a robust gait recognition method using multiple views fusion and deterministic learning. First, a multiple-views fusion strategy is introduced, in which gaits collected under different views are synthesized as a kind of synthesized silhouette images. Second, the synthesized silhouettes are characterized with four kinds of time-varying gait features, including three width features of the silhouette and one silhouette area feature. Third, gait variability underlying different individuals’ time-varying gait features is effectively modeled by using deterministic learning algorithm. This kind of variability reflects the change of synthesized silhouettes while preserving temporal dynamics information of human walking. Gait patterns are represented as the gait variability underlying time-varying gait features and a rapid recognition scheme is presented in published gait databases. Experimental results show that encouraging recognition accuracy can be achieved.

Introduction

Gait is an attractive biometric feature for human identification at a distance. Compared with other biometrics, gait has great prominent advantages of being non-contact, non-invasive, unobvious and low resolution requirement [22]. Gait is also difficult to hide or imitate and has great potential applications in video surveillance [9], [47].

Despite that much progress has been made for gait recognition [1], [16], [19], [26], [36], most of them depend on only one specific view angle and thus have low recognition rate due to the susceptibility to clothes, background, carrying status, shoe type, walking speed, etc. [41]. Gait characteristics extracted with one single camera are relatively limited and not comprehensive enough to develop a robust recognition system [12]. Based on this assumption, one possible method is to get gait sequences from multiple cameras fixed at different view angles and present a recognition scheme based on multi-view gait characteristics fusion. In fact, multiple cameras are now widely used and needed in real-world surveillance environments.

Zhao et al. [48] set up a human 3D model using video sequences captured by multiple cameras. Static and dynamic features were extracted for gait recognition. Lu et al. [32] operated on binary silhouettes and introduced one multiple views fusion recognition scheme on the decision level based on product of sum rule. Kusakunniran et al. [20] used Singular Value Decomposition (SVD) technique to create a new View Transformation Model (VTM), in which the viewing angles of gallery gait data and probe gait data were transformed into the same view angle. Jeong et al. [18] proposed canonical view synthesis based on planar homograpy. Havasi et al. [11] extracted gait characteristics from video-image sequences based on symmetry algorithm for multiple-camera registration. More recently, techniques based on extend SVM [22], correlated motion regression [23], support vector regression (SVR) [21], canonical correlation analysis (CCA) [3] were proposed for multiple views gait recognition.

A robust gait recognition method based on multiple views fusion and deterministic learning is presented in this paper. First, multi-view fusion is employed to synthesize human gait information from two different views images, which is inspired by Huang and Xu [15]. The synthesized silhouettes make full use of two cameras’ silhouettes to dispel the influence caused by walking conditions. In contrast to other multi-view methods [48], our method directly fuse human gait information from different views images and extract gait features from the fusion result silhouettes images.

In our previous work [43], [45], we successfully applied deterministic learning algorithm to human gait recognition. Deterministic learning theory is capable of capturing the dynamics information underlying the temporal gait features, which contains more in-depth information and is robust to different walking status or occlusion problem compared with the static features of gait signals [39]. Following this idea, we attempt to capture the gait variability via deterministic learning from multi-view silhouettes. This kind of variability reflects the change of body poses or silhouettes between consecutive frame sequences while preserving temporal dynamics information of human walking. Gait patterns are represented as the gait variability underlying time-varying gait features. Based on the modeling results, a set of dynamical estimators are constructed to represent the training gait patterns. By comparing the set of estimators with a test gait pattern, a set of synchronization(recognition) errors are generated. A test gait pattern can be rapidly recognized according to the smallest error principle. Fig. 1 shows the flow-chart of our method.

Section snippets

Feature extraction and representation

Before training and recognition, gait sequences from two different views are synthesized and converted into a sequence of gait feature signals at this preprocessing stage.

Gait pattern extraction and representation

The median width feature of the holistic silhouette Wd1, the median width features of regions 3 and 4, Wd2 and Wd3, and the synthesized silhouette area Ad are selected to capture the frame-to-frame variability between consecutive frame sequences. Training gait patterns can be represented by sequences variability related to these four features.

Training gait patterns can be represented by variability between consecutive frame sequences: x˙=F(x;p)+v(x;p),x(t0)=x0where x=[x1,,x4]TR4 is the state

Rapid recognition using gait pattern (gait variability)

Consider a training set containing dynamical human gait patterns φζk,k=1,,M, with the kth gait training pattern φζk generated from x˙=Fk(x;pk)+vk(x;pk),x(t0)=xζ0where Fk(x; pk) denotes the gait sequences variability, vk(x; pk) denotes the modeling uncertainty, pk is the system parameter.

First, similar to the training phase, gait features are extracted. To achieve rapid recognition of a test gait pattern from a set of training gait patterns, it is preferred not to model the gait sequences

Experiments on CASIA-B gait database

This paper reports experimental results on CASIA-B database, which includes sequences from 124 different subjects under 11 different views [42]. Each subject walks along the straight line 10 times (6 for normal walking, 2 for walking in a coat and 2 for walking with a bag) and 11 video sequences are collected each time. The view angle between the camera optical axis direction and the walking direction takes on the values of 0°, 18°, 36°, , 180°.

Conclusion

The gait variability modeling and rapid recognition through multiple views fusion are investigated in this paper by the need of human identification in real-world surveillance environments. There are some conclusions in below.

The direct multiple views images fusion can be used to obtain different gait information in different views, which overcomes the limitation of recognition with one single camera. The deterministic learning theory can be used to describe the gait variability with RBF

Acknowledgments

This work was supported by the National Science Fund for Distinguished Young Scholars (Grant no. 61225014), by the National R&D Program for Major Research Instruments (Grant no. 61527811).

References (48)

  • K. Bashir et al.

    Gait recognition using gait entropy image

    International Conference on Crime Detection and Prevention

    (2009)
  • K. Bashir et al.

    Cross view gait recognition using correlation strength

    Proceeding of British Machine Vision Conf.

    (2010)
  • W. Bian et al.

    Max-min distance analysis by using sequential sdp relaxation for dimension reduction

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2011)
  • S. Choudhury et al.

    Robust view-invariant multiscale gait recognition

    Pattern Recognit.

    (2015)
  • S.D. Choudhury et al.

    Silhouette-based gait recognition using procrustes shape analysis and elliptic fourier descriptors

    Pattern Recognit.

    (2012)
  • M. Goffredo et al.

    Self-calibrating view-invariant gait biometrics

    IEEE Trans. Syst. Man Cybern. B, Cybern.

    (2010)
  • R. Gross et al.

    The cmu motion of body (mobo) database

    Technical report

    (2001)
  • J. Gu et al.

    Action and gait recognition from recovered 3-d human joints

    IEEE Trans. Cybern.

    (2010)
  • J. Han et al.

    Individual recognition using gait energy image

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2006)
  • L. Havasi et al.

    Detection of gait characteristics for scene registration in video surveillance system

    IEEE Trans. Image Process.

    (2007)
  • E. Hossain et al.

    Multi-view multi-modal gait based human identity recognition from surveillance videos

    Proceeding of International Conference on Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction

    (2013)
  • HuM. et al.

    Gait-based gender classification using mixed conditional random field

    IEEE Trans. Syst. Man Cybern. B Cybern.

    (2011)
  • HuM. et al.

    Incremental learning for video-based gait recognition with lbp flow

    IEEE Trans. Cybern.

    (2013)
  • HuangF.-Y. et al.

    Viewpoint independent action recognition

    J. Softw.

    (2008)
  • Cited by (0)

    “This paper has been recommended for acceptance by Egon L. van den Broek.”

    View full text