Utilizing Invariant Descriptors for Finger Spelling American Sign Language Using SVM

For an effective vision-based HCI system, inference from natural means of sources (i.e. hand) is a crucial challenge in unconstrained environment. In this paper, we have aimed to build an interaction system through hand posture recognition for static finger spelling American Sign Language (ASL) alphabets and numbers. Unlike the interaction system based on speech, the coarticulation due to hand shape, position and movement influences the different aspects of sign language recognition. Due to this, we have computed the features which are invariant to translation, rotation and scaling. Considering these aspects as the main objectives of this research, we have proposed a three-step approach: first, features vector are computed using two moment based approaches namely Hu-Moment along with geometrical features and Zernike moment. Second, the categorization of symbols according to the fingertip is performed to avoid mis-classification among the symbols. Third, the extracted set of two features vectors (i.e. Hu-Moment with geometrical features and Zernike moment) are trained by Support Vector Machines (SVM) for the classification of the symbols. Experimental results of the proposed approaches achieve recognition rate of 98.5% using Hu-Moment with geometrical features and 96.2% recognition rate using Zernike moment for ASL alphabets and numbers demonstrating the dominating performance of Hu-Moment with geometrical features over Zernike moments.