Compressed-Domain Based Camera Motion Estimation for Realtime Action Recognition

Camera motions seriously affect the accuracy of action recognition. Traditional methods address this issue through estimating and compensating camera motions based on optical flow in pixel-domain. But the high computational complexity of optical flow hinders these methods from applying to realtime scenarios. In this paper, we advance an efficient camera motion estimation and compensation method for realtime action recognition by exploiting motion vectors in video compressed-domain (a.k.a. compressed-domain global motion estimation, CGME). Taking advantage of geometric symmetry and differential theory of motion vectors, we estimate the parameters of camera affine transformation. These parameters are then used to compensate the initial motion vectors to retain crucial object motions. Finally, we extract video features for action recognition based on compensated motion vectors. Experimental results show that our method improves the speed of camera motion estimation by over 100 times with a minor reduction of about \(4\,\%\) in recognition accuracy compared with iDT.