Human gait recognition based on deterministic learning through multiple views fusion☆
Introduction
Gait is an attractive biometric feature for human identification at a distance. Compared with other biometrics, gait has great prominent advantages of being non-contact, non-invasive, unobvious and low resolution requirement [22]. Gait is also difficult to hide or imitate and has great potential applications in video surveillance [9], [47].
Despite that much progress has been made for gait recognition [1], [16], [19], [26], [36], most of them depend on only one specific view angle and thus have low recognition rate due to the susceptibility to clothes, background, carrying status, shoe type, walking speed, etc. [41]. Gait characteristics extracted with one single camera are relatively limited and not comprehensive enough to develop a robust recognition system [12]. Based on this assumption, one possible method is to get gait sequences from multiple cameras fixed at different view angles and present a recognition scheme based on multi-view gait characteristics fusion. In fact, multiple cameras are now widely used and needed in real-world surveillance environments.
Zhao et al. [48] set up a human 3D model using video sequences captured by multiple cameras. Static and dynamic features were extracted for gait recognition. Lu et al. [32] operated on binary silhouettes and introduced one multiple views fusion recognition scheme on the decision level based on product of sum rule. Kusakunniran et al. [20] used Singular Value Decomposition (SVD) technique to create a new View Transformation Model (VTM), in which the viewing angles of gallery gait data and probe gait data were transformed into the same view angle. Jeong et al. [18] proposed canonical view synthesis based on planar homograpy. Havasi et al. [11] extracted gait characteristics from video-image sequences based on symmetry algorithm for multiple-camera registration. More recently, techniques based on extend SVM [22], correlated motion regression [23], support vector regression (SVR) [21], canonical correlation analysis (CCA) [3] were proposed for multiple views gait recognition.
A robust gait recognition method based on multiple views fusion and deterministic learning is presented in this paper. First, multi-view fusion is employed to synthesize human gait information from two different views images, which is inspired by Huang and Xu [15]. The synthesized silhouettes make full use of two cameras’ silhouettes to dispel the influence caused by walking conditions. In contrast to other multi-view methods [48], our method directly fuse human gait information from different views images and extract gait features from the fusion result silhouettes images.
In our previous work [43], [45], we successfully applied deterministic learning algorithm to human gait recognition. Deterministic learning theory is capable of capturing the dynamics information underlying the temporal gait features, which contains more in-depth information and is robust to different walking status or occlusion problem compared with the static features of gait signals [39]. Following this idea, we attempt to capture the gait variability via deterministic learning from multi-view silhouettes. This kind of variability reflects the change of body poses or silhouettes between consecutive frame sequences while preserving temporal dynamics information of human walking. Gait patterns are represented as the gait variability underlying time-varying gait features. Based on the modeling results, a set of dynamical estimators are constructed to represent the training gait patterns. By comparing the set of estimators with a test gait pattern, a set of synchronization(recognition) errors are generated. A test gait pattern can be rapidly recognized according to the smallest error principle. Fig. 1 shows the flow-chart of our method.
Section snippets
Feature extraction and representation
Before training and recognition, gait sequences from two different views are synthesized and converted into a sequence of gait feature signals at this preprocessing stage.
Gait pattern extraction and representation
The median width feature of the holistic silhouette the median width features of regions 3 and 4, and and the synthesized silhouette area Ad are selected to capture the frame-to-frame variability between consecutive frame sequences. Training gait patterns can be represented by sequences variability related to these four features.
Training gait patterns can be represented by variability between consecutive frame sequences: where is the state
Rapid recognition using gait pattern (gait variability)
Consider a training set containing dynamical human gait patterns with the kth gait training pattern generated from where Fk(x; pk) denotes the gait sequences variability, vk(x; pk) denotes the modeling uncertainty, pk is the system parameter.
First, similar to the training phase, gait features are extracted. To achieve rapid recognition of a test gait pattern from a set of training gait patterns, it is preferred not to model the gait sequences
Experiments on CASIA-B gait database
This paper reports experimental results on CASIA-B database, which includes sequences from 124 different subjects under 11 different views [42]. Each subject walks along the straight line 10 times (6 for normal walking, 2 for walking in a coat and 2 for walking with a bag) and 11 video sequences are collected each time. The view angle between the camera optical axis direction and the walking direction takes on the values of 0°, 18°, 36°, 180°.
Conclusion
The gait variability modeling and rapid recognition through multiple views fusion are investigated in this paper by the need of human identification in real-world surveillance environments. There are some conclusions in below.
The direct multiple views images fusion can be used to obtain different gait information in different views, which overcomes the limitation of recognition with one single camera. The deterministic learning theory can be used to describe the gait variability with RBF
Acknowledgments
This work was supported by the National Science Fund for Distinguished Young Scholars (Grant no. 61225014), by the National RD Program for Major Research Instruments (Grant no. 61527811).
References (48)
- et al.
Frontal-view gait recognition by intra- and inter- frame rectangle size distribution
Pattern Recognit. Lett.
(2009) - et al.
Cross-view and multi-view gait recognitions based on view transformation model using multi-layer perceptron
Pattern Recognit. Lett.
(2012) - et al.
Gait flow image: a silhouette-based gait representation for human identification
Pattern Recognit.
(2011) - et al.
Gait flow image: a silhouette-based gait representation for human identification
Pattern Recognit.
(2011) - et al.
Uncorrelated discriminant simplex analysis for view-invariant gait signal computing
Pattern Recognit. Lett.
(2010) - et al.
Gait recognition for human identification based on ica and fuzzy svm through multiple views fusion
Pattern Recognit. Lett.
(2007) - et al.
Model-based human gait recognition using leg and arm movements
Eng. Appl. Artif. Intell.
(2010) - et al.
Human gait recognition via deterministic learning
Neural Netw.
(2012) - et al.
Silhouette-based gait recognition via deterministic learning
Pattern Recognit.
(2014) - et al.
Gait recognition across different walking speeds via deterministic learning
Neurocomputing
(2015)
Gait recognition using gait entropy image
International Conference on Crime Detection and Prevention
Cross view gait recognition using correlation strength
Proceeding of British Machine Vision Conf.
Max-min distance analysis by using sequential sdp relaxation for dimension reduction
IEEE Trans. Pattern Anal. Mach. Intell.
Robust view-invariant multiscale gait recognition
Pattern Recognit.
Silhouette-based gait recognition using procrustes shape analysis and elliptic fourier descriptors
Pattern Recognit.
Self-calibrating view-invariant gait biometrics
IEEE Trans. Syst. Man Cybern. B, Cybern.
The cmu motion of body (mobo) database
Technical report
Action and gait recognition from recovered 3-d human joints
IEEE Trans. Cybern.
Individual recognition using gait energy image
IEEE Trans. Pattern Anal. Mach. Intell.
Detection of gait characteristics for scene registration in video surveillance system
IEEE Trans. Image Process.
Multi-view multi-modal gait based human identity recognition from surveillance videos
Proceeding of International Conference on Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction
Gait-based gender classification using mixed conditional random field
IEEE Trans. Syst. Man Cybern. B Cybern.
Incremental learning for video-based gait recognition with lbp flow
IEEE Trans. Cybern.
Viewpoint independent action recognition
J. Softw.
Cited by (0)
- ☆
“This paper has been recommended for acceptance by Egon L. van den Broek.”