GMNet: an action recognition network with global motion representation

In recent years, an astonishing progress has been made in action recognition. However, the traditional spatio-temporal convolution kernels cannot learn sufficient motion information, which is the key step in action recognition. Therefore, a more effective motion representation approach is required to reason the motion cues in videos. In this light, we propose GMNet, an action recognition network with global motion representation to fulfill such task. It includes a short-term motion feature extraction module and a motion feature aggregation module. The former one is capable of capturing local motion features from adjacent frames, while the latter one excels at aggregating the above features to yield global motion representations. GMNet is easily compatible to any mainstream backbones to realize end-to-end training without additional supervision. Extensive experiments have been carried out on popular benchmarks (Something-Something V1 & V2, Diving-48, Jester and Kinetics 400) to testify its effectiveness. It turns out that GMNet surpasses most of the state-of-the-art methods.