Robust Gait Recognition Based on Spatio-Temporal Fusion Network

The human gait sequence contains both spatial and temporal information, and the spatial and temporal information are restricted to different degrees under different views and walking conditions. In this paper, we propose a method based on spatio-temporal fusion network by combining temporal and spatial information. The method improves the performance of gait recognition under clothing change and view change. Frame difference color energy image (FDCEI) is proposed in this paper to extract the temporal information in the gait sequence, which complements the gait entropy image (GEnI). A two-branch network is used to process the energy image information, in which the temporal branch network processes the temporal information in FDCEI and the spatial branch network processes the spatial information in GEnI. Another contribution is that a Balanced spatio-temporal feature module (BSTFM) has been proposed, which integrates spatio-temporal information. The experimental performances indicate that our proposed methods contribute to increasing the robustness of gait recognition in external environments with view variances and clothes changes.