Skip to main content

Advertisement

SpringerLink
Log in

Design of hand gesture recognition system for human-computer interaction

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Human-Computer interaction (HCI) with gesture recognition is designed to recognize a number of meaningful human expressions, and has become a valuable and intuitive computer input technique. Hand gestures are one of the most intuitive and common forms of communication, and can communicate a wide range of meaning. Vision-based hand gesture recognition has received a significant amount of research attention in recent years. However, the field still presents a number of challenges for researchers. In the vision-based hand gesture interaction process between humans and computers, gesture interpretation must be performed quickly and with high accuracy. In this paper, a low-cost HCI system with hand gesture recognition is proposed. This system uses several vision techniques. Skin and motion detection is used for capturing the region-of-interest from the background regions. A connected component labeling algorithm is proposed to identify the centroid of an object. To identify the exact area of hand gesture, the arm area is removed with the aid of a convex hull algorithm. Moreover, a real-time demonstration system is developed, based on a single-camera mechanism which allows for the use of wearable devices. Simulation results show that the recognition rate is still high, although some interference is encountered in the simulated environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Aksaç A, Öztürk O, Özyer T (2011) Real-time multi-objective hand posture/gesture recognition by using distance classifiers and finite state machine for virtual mouse operations. IEEE International Conference on Electrical and Electronics Engineering (ELECO) 7th, pp. 457–461

  2. Aviles-Arriaga HH, Sucar LE, Mendoza CE, Vargas B (2003) Visual recognition of gestures using dynamic naive Bayesian classifiers. Robot and Human Interactive Communication, Proceedings. The 12th IEEE International Workshop on Robot and Human Interactive Communication, pp. 133–138

  3. Bellarbi A, Benbelkacem S, Henda NZ, Belhocine M (2011) Hand gesture interaction using color-based method for tabletop interfaces. IEEE International Symposium on Intelligent Signal Processing (WISP):1–6

  4. Berman S, Stern S (2012) Sensors for gesture recognition systems. IEEE Transactions on Systems, Man, and Cybernetics, Part C:277–290. https://doi.org/10.1109/TSMCC.2011.2161077

  5. Burger T, Caplier A, Mancini S (2005) Cued speech hand gestures recognition tool. IEEE European Signal Processing Conference:1–4

  6. Chen Q, Georganas ND, Petriu EM (2008) Hand gesture recognition using haar-like features and a stochastic context-free grammar. IEEE Trans Instrum Meas 57(8):1562–1571

    Article  Google Scholar 

  7. Cheng LT, Chih WK, Tsai A, Chih WC (2009) Hand posture recognition using hidden conditional random fields. IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp.1828–1833, pp. 14–17

  8. Chiang T, Fan CP (2018) 3D Depth Information Based 2D Low-Complexity Hand Posture and Gesture Recognition Design for Human Computer Interactions. International Conference on Computer and Communication Systems (ICCCS). https://doi.org/10.1109/CCOMS.2018.8463327

  9. Deyou X (2006) A neural approach for hand gesture recognition in virtual reality driving training system of SPG. Proc. of International Conference on Pattern Recognition, ICPR’06, pp. 519–522

  10. Dias DB, Madeo RCB, Rocha T, Biscaro HH, Peres SM (2009) Hand movement recognition for Brazilian Sign Language: A study using distance-based neural networks. Neural Networks, IEEE - INNS - ENNS International Joint Conference on, pp. 697–704

  11. Dipietro L, Sabatini AM, Dario P (2008) A survey of glove-based systems and their Applications. IEEE Transactions on Systems, Man, and Cybernetics, Part C:461–482. https://doi.org/10.1109/TSMCC.2008.923862

  12. Duan HX, Zhang QY, Ma W (2011) An approach to dynamic hand gesture modeling and real-time extraction. IEEE International Conference on Communication Software and Networks (ICCSN):139–142

  13. Elmezain M, Al-Hamadi A, Michaelis B (2008) Real-time capable system for hand motion detection, labeling, data association and tracking gesture recognition using hidden Markov models in stereo color image sequences. The Journal of WSCG’08 16:65–72

    Google Scholar 

  14. Erol A, Bebis G, Nicolescu M, Boyle RD, Twombly X (2007) Vision-based hand pose estimation: A review. Comput Vis Image Understanding 108(1–2):52–73

    Article  Google Scholar 

  15. Foxlin E (2002) Motion tracking requirements and technologies. Handbook of Virtual Environment Technology, pp. 163–210

  16. Ghosh DK, Ari S (2011) A static hand gesture recognition algorithm using k-mean based radial basis function neural network. IEEE International Conference on Information, Communications and Signal Processing:1–5

  17. Han J, Shao L, Xu D, Shotton J (2013) Enhanced computer vision with Microsoft Kinect sensor: A review. IEEE Transactions on Systems, Man, and Cybernetics:1318–1334. https://doi.org/10.1109/TCYB.2013.2265378

  18. Heung-Il S, Kee SB, Whan LS (2010) Hand gesture recognition based on dynamic Bayesian network framework. Pattern Recogn 43(9):3059–3072

    Article  Google Scholar 

  19. Hsieh CC, Liou DH, Lee D (2010) A real time hand gesture recognition system using motion history image. IEEE International Conference on Singal Processing Systems (ICSPS) 2:394–398

    Google Scholar 

  20. Kukharev G, Nowosielski A (2004) Visitor identification: elaborating real time face recognition system. in Proceedings of the 12th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, Plzen-Bory, Czech Republic, pp. 157–164

  21. Kumar S, Kaurav A (2018) Hand Gesture through Geometric Moments (HCI Based). International Conference on Inventive Systems and Control (ICISC), DOI: https://doi.org/10.1109/ICISC.2018.8398862

  22. Kumar P, Rautaray SS, Agrawal A (2012) Hand data glove: A new generation real-time mouse for human-computer interaction. International Conference on Recent Advances in Information Technology (RAIT), pp. 750–755

  23. Lacassagne L, Milgram M, Garda P (1999) Motion detection, labeling, data association and tracking, in real-time on RISC computer. IEEE Image Analysis and Processing, Proceedings. International Conference on, pp. 520–525

  24. Lee C, Xu Y (1996) Online Interactive learning of gestures for human/robot interfaces. IEEE International Conference on Robotics and Automation 4:2982–2987

    Article  Google Scholar 

  25. Lin L, Cong Y, Tang Y (2012) Hand gesture recognition using RGB-D cue. IEEE International Conference on Information and Automation (ICIA):311–316

  26. Lu Z, Chen X, Li Q, Zhang X, Zhou P (2014) A Hand Gesture Recognition Framework and Wearable Gesture-Based Interaction Prototype for Mobile Devices. IEEE Transactions on Human-Machine Systems:293–299. https://doi.org/10.1109/THMS.2014.2302794

  27. Mitra S, Acharya T (2007) Gesture recognition: A survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C:311–324. https://doi.org/10.1109/TSMCC.2007.893280

  28. Modler P, Myatt T (2008) Recognition of separate hand gestures by Time-Delay Neural Networks based on multistate spectral image patterns from cyclic hand movements. Systems, Man and Cybernetics, 2008. SMC 2008. IEEE International Conference on, pp. 1539–1544

  29. Nguyen DB, Enokida S, Toshiaki E (2005) Real-time hand tracking and gesture recognition system. IGVIP’05, pp. 362–368

  30. Panwar M, Mehra PS (2011) Hand tracking and gesture recognition for human-computer interaction. Image Information Processing (ICIIP), 2011 International Conference on

  31. Rahmat RW, Al-Tairi ZH, Saripan MI, Sulaiman PS (2012) Removing shadow for hand segmentation based on background subtraction. International Conference on Advanced Computer Science Applications and Technologies (ACSAT), pp. 481–485

  32. Rautaray SS, Agrawal A (2012) Design of gesture recognition system for dynamic user interface analysis. IEEE International Conference on Technology Enhanced Education (ICTEE):1–6

  33. Rossol N, Cheng I, Basu A (2016) A Multisensor Technique for Gesture Recognition Through Intelligent Skeletal Pose Analysis. IEEE Transactions on Human-Machine Systems:350–359. https://doi.org/10.1109/THMS.2015.2467212

  34. Sahoo JP, Ari S, Ghosh DK (2018) Hand gesture recognition using DWT and F-ratio based feature descriptor. IET Image Process 12(10):1780–1787

    Article  Google Scholar 

  35. Song S, Yan D, Xie Y (2018) Design of control system based on hand gesture recognition. International Conference on Networking, Sensing and Control (ICNSC). DOI:https://doi.org/10.1109/ICNSC.2018. 8361351

  36. Takahashi T, Kishino F (1991) Hand gesture coding based on experiments using a hand gesture interface device. SIGCHI Bull 23(2):67–74

    Article  Google Scholar 

  37. Tang C, Ou Y, Jiang G, Xie Q, Xu Y (2012) Hand tracking and pose recognition via depth and color information. IEEE International Conference on Robotics and Biomimetics (ROBIO):1104–1109

  38. Turk M (2001) Handbook of Virtual Environment Technology. Lawrence Erlbaum Associates, Gesture Recognition, Chap. 9

  39. Wachs JP, Kolsch M, Stern H, Edan Y (2011) Vision-based hand gesture applications. Commun ACM 54(2):60–71

    Article  Google Scholar 

  40. Wan M (2014) Feature extraction using two-dimensional maximum embedding difference. Inf Sci 274:55–69

    Article  Google Scholar 

  41. Wan M, Yang G, Gai S, Yang Z (2017) Two-dimensional Discriminant Locality Preserving Projections (2DDLPP) and Its Application to Feature Extraction via Fuzzy Set. Multimed Tools Appl 76:355–371

    Article  Google Scholar 

  42. Wan M et al (2017) Local graph embedding based on maximum margin criterion via fuzzy set. Fuzzy Sets Syst 318:120–131

    Article  MathSciNet  Google Scholar 

  43. Yi B, Harris FC, Wang L, Yan Y (2005) Real-time natural hand gestures. Proceedings of IEEE Computing in Science & Engineering and the American Institute of Physics 7(3):92–97

    Google Scholar 

  44. Zaletelj J, Perhavc J, Tasic JF (2007) Vision-based human-computer interface using hand gestures. International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS'07)

  45. Zhan X, Chen X, Li Y, Lantz V, Wang K, Yang J (2011) A framework for hand gesture recognition based on accelerometer and EMG sensors. IEEE Transactions on Systems, Man, and Cybernetics, Part A:1064–1076. https://doi.org/10.1109/TSMCA.2011.2116004

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, National Central University, No.300, Jung-Da Rd., Jung-Li City, Taiwan, 320, Republic of China

    Tsung-Han Tsai, Chih-Chi Huang & Kung-Long Zhang

Authors
  1. Tsung-Han Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chih-Chi Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kung-Long Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tsung-Han Tsai.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tsai, TH., Huang, CC. & Zhang, KL. Design of hand gesture recognition system for human-computer interaction. Multimed Tools Appl 79, 5989–6007 (2020). https://doi.org/10.1007/s11042-019-08274-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08274-w

Keywords

Search

Navigation