Natural Calling Gesture Recognition in Crowded Environments

Most existing gesture recognition algorithms use fixed postures in simple environments. In natural calling, the user may perform gestures in various positions and the environment may be occupied by many people with many hand motions. This paper presents an algorithm for natural calling gesture recognition by detecting gaze, the position of the hand-wrist, and fingertips. A challenge to solve is how to make the natural calling gesture recognition work in crowded environments with randomly moving objects. The approach taken here is to get the key-points of individual people using a real-time detector by using a camera and detect gaze and hand-wrist positions. Then, zooming into the hand-wrist part and getting the key-points of fingertips, we calculate the positions of the fingertips to recognize calling gestures. We tested the proposed approach in video under different conditions: from one person to over four people that sit and walk around. We obtained an average recognition accuracy of 87.12%, thus showing the effectiveness of our approach.