NMF with local constraint and Deep NMF with temporal dependencies constraint for action recognition

In order to improve action recognition accuracy, a new nonnegative matrix factorization with local constraint (LC-NMF) is firstly presented. By applying it for effective trajectory clustering, complex backgrounds are removed and then the motion salient regions are obtained. Secondly, a nonnegative matrix factorization with temporal dependencies constraint (TD-NMF) is proposed, which fully mines the spatiotemporal relationship in a video not only between adjacent frames, but also between multi-interval frames. Meanwhile, the introduction of \( l_{2,1} \)-norm makes the spatiotemporal features possess better sparseness and robustness. In addition, these features are directly learned from data and thus have an inherent generalization ability. Finally, a Deep NMF method is established, which takes the proposed TD-NMF as the unit algorithm of each layer. By introducing the hierarchical feature extraction strategy, the base matrix of the first layer is gradually decomposed; then, it is supplemented and completed layer by layer. Consequently, the more complete and accurate local feature estimations are obtained, and then the discriminative and expressive abilities of features are effectively enhanced and recognition performance is further improved. Adequate and extensive experiments verify the effectiveness of the proposed methods. Moreover, the update rules and convergence proofs for LC-NMF and TD-NMF are also given.