Detection, tracking, and recognition of isolated multi-stroke gesticulated characters

The detection and tracking of the bare hand are the most vital stages in the bare hand gesticulated character recognition system. Applying detection and tracking in an uncontrolled environment is quite challenging due to the variations in pose, position, illumination, occlusion, scale, rotation, speed, and bare hand-like impostors in the background or foreground. The motion blur due to speed variation makes it more complex. A computationally efficient object localization approach, i.e., region-based convolutional neural network on the selective region (RCNN-SR), is introduced in this work to detect the bare hand by overcoming these challenges at a certain level. To handle the motion blur and computational complexity, the paper presents a motion information-based detection and tracking approach using RCNN-SR, point tracker, and Kalman filter. The trajectory of the gesticulated characters is formed by mapping all the centroid points of the localized bare hand. To recognize this gesticulated trajectory, the original-existence-based gesticulated character recognition model is designed in this work by utilizing prior information about the characters. In this, we have tried to overcome the variations in pattern, style, scale, rotation, and illumination. In addition, we have addressed the case sensitivity between the English lowercase and uppercase alphabets by incorporating the boundary information. We have also proposed NITS R-Net, NITS hand gesture database VIIIB, and NITS Gesture image databases in this work. To evaluate proposed models, several benchmark databases are considered.