Pat Jangyodsuk
ABSTRACT
Recognizing sign language is a very challenging task in computer vision. One of the more popular approaches, Dynamic Time Warping (DTW), utilizes hand trajectory information to compare a query sign with those in a database of examples. In this work, we conducted an American Sign Language (ASL) recognition experiment on Kinect sign data using DTW for sign trajectory similarity and Histogram of Oriented Gradient (HoG) [5] for hand shape representation. Our results show an improvement over the original work of [14], achieving an 82% accuracy in ranking signs in the 10 matches. In addition to our method that improves sign recognition accuracy, we propose a simple RGB-D alignment tool that can help roughly approximate alignment parameters between the color (RGB) and depth frames.
- J. Alon, V. Athitsos, Q. Yuan, and S. Sclaroff. A unified framework for gesture recognition and spatiotemporal gesture segmentation. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 31(9):1685--1699, 2009. Google Scholar
Digital Library
- F.-S. Chen, C.-M. Fu, and C.-L. Huang. Hand gesture recognition using a real-time tracking method and hidden markov models. Image and Vision Computing, 21(8):745--758, 2003.Google ScholarCross Ref
- C. Conly, P. Doliotis, P. Jangyodsuk, R. Alonzo, and V. Athitsos. Toward a 3d body part detection video dataset and hand tracking benchmark. In Proceedings of the 6th International Conference on PErvasive Technologies Related to Assistive Environments, PETRA '13, pages 2:1--2:6, New York, NY, USA, 2013. ACM. Google ScholarDigital Library
- A. Corradini. Dynamic time warping for off-line recognition of a small gesture vocabulary. In Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, 2001. Proceedings. IEEE ICCV Workshop on, pages 82--89. IEEE, 2001. Google ScholarDigital Library
- N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, volume 1, pages 886--893. IEEE, 2005. Google ScholarDigital Library
- S. R. Eddy. Hidden markov models. Current opinion in structural biology, 6(3):361--365, 1996.Google Scholar
- A. Kar. Skeletal tracking using microsoft kinect. Methodology, 1: 1--11, 2010.Google Scholar
- L. Karlinsky, M. Dinerstein, D. Harari, and S. Ullman. The chains model for detecting parts by their context. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pages 25--32, 2010.Google ScholarCross Ref
- J. Lafferty, A. McCallum, and F. C. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. 2001.Google ScholarDigital Library
- A. Mittal, A. Zisserman, and P. H. S. Torr. Hand detection using multiple proposals. In British Machine Vision Conference, 2011.Google ScholarCross Ref
- T. Starner and A. Pentland. Real-time american sign language recognition from video using hidden markov models. In Motion-Based Recognition, pages 227--243. Springer, 1997.Google ScholarCross Ref
- G. Ten Holt, M. Reinders, and E. Hendriks. Multi-dimensional dynamic time warping for gesture recognition. In Thirteenth annual conference of the Advanced School for Computing and Imaging, volume 300, 2007.Google Scholar
- e. a. Vondrick Carl. Hoggles: Visualizing object detection features, Jan. 2013.Google Scholar
- H. Wang, R. Stefan, S. Moradi, V. Athitsos, C. Neidle, and F. Kamangar. A system for large vocabulary sign search.Google Scholar
- S. B. Wang, A. Quattoni, L.-P. Morency, D. Demirdjian, and T. Darrell. Hidden conditional random fields for gesture recognition. In Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, volume 2, pages 1521--1527. IEEE, 2006. Google ScholarDigital Library
- A. D. Wilson and A. F. Bobick. Parametric hidden markov models for gesture recognition. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 21(9):884--900, 1999. Google ScholarDigital Library
- R. Yang and S. Sarkar. Gesture recognition using hidden markov models from fragmented observations. In Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, volume 1, pages 766--773. IEEE, 2006. Google ScholarDigital Library
- V. M. Z. Zhang, W. H. Liu and V. Athitsos. A viewpoint-independent statistical method for fall detection. In International Conference on Pattern Recognition, pages 3626--3630, Nov 2012.Google Scholar
- Z. Zhang, R. Alonzo, and V. Athitsos. Experiments with computer vision methods for hand detection. In Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments, PETRA '11, pages 21:1--21:6, New York, NY, USA, 2011. ACM. Google ScholarDigital Library
- Z. Zhang, R. Alonzo, and V. Athitsos. Experiments with computer vision methods for hand detection. In Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments, pages 21:1--21:6, 2011. Google ScholarDigital Library
Index Terms
- Sign language recognition using dynamic time warping and hand shape distance based on histogram of oriented gradient features
Recommendations
Isolated Arabic Sign Language Recognition Using a Transformer-based Model and Landmark Keypoints
Pose-based approaches for sign language recognition provide light-weight and fast models that can be adopted in real-time applications. This article presents a framework for isolated Arabic sign language recognition using hand and face keypoints. We ...
Kinect-based Taiwanese sign-language recognition system
Gesture-recognition is an important component for many intelligent human---computer interaction applications. For example, a realtime sign-language recognition system would detect and interpret hand gestures. Many vision-based sign-language recognition ...
Kinect Based Dynamic Hand Gesture Recognition Algorithm Research
IHMSC '12: Proceedings of the 2012 4th International Conference on Intelligent Human-Machine Systems and Cybernetics - Volume 01In dynamic gesture track recognition, gesture recognition rate has always been a problem, Because the traditional hand gesture recognition in how to separate the hand type from the surrounding environment, and then determine the palm position has been ...
Comments