Skip to main content
Top

2023 | OriginalPaper | Chapter

Convexity Defects-Based Fingertip Detection and Hand Gesture Recognition

Authors : Soumi Paul, Shrouti Gangopadhyay, Ayatullah Faruk Mollah, Subhadip Basu, Mita Nasipuri

Published in: Proceedings of International Conference on Frontiers in Computing and Systems

Publisher: Springer Nature Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Vision-based hand gesture recognition is perhaps the most widely used technique for human–computer interaction technology with special application toward sign languages used by differently abled people. Perhaps the most important step in such a gesture recognition system is feature extraction. In this paper, we use a novel fingertip detection mechanism based on convexity defects and use nine geometrical features that are translation and rotation invariant. We use seven different classifiers on two different public hand digit datasets (NTU hand digit dataset and SP-EMD color-depth hand gesture dataset) and find that for both the datasets, the random forest classifier gives the best classification accuracy (94.2% and 90.1%, respectively).

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Mitra S, Acharya T (2007) Gesture recognition: a survey. IEEE Trans Syst Man Cybern Part C (Appl Rev) 37(3):311–324CrossRef Mitra S, Acharya T (2007) Gesture recognition: a survey. IEEE Trans Syst Man Cybern Part C (Appl Rev) 37(3):311–324CrossRef
2.
go back to reference Shi B, Brentari D, Shakhnarovich G, Livescu K (2021) Fingerspelling detection in American sign language. arXiv preprint arXiv:2104.01291 Shi B, Brentari D, Shakhnarovich G, Livescu K (2021) Fingerspelling detection in American sign language. arXiv preprint arXiv:​2104.​01291
3.
go back to reference Hussain SMA, Rashid AH (2012) User independent hand gesture recognition by accelerated DTW. In: 2012 international conference on informatics, electronics and vision (ICIEV). IEEE, 1033–1037 Hussain SMA, Rashid AH (2012) User independent hand gesture recognition by accelerated DTW. In: 2012 international conference on informatics, electronics and vision (ICIEV). IEEE, 1033–1037
4.
go back to reference Choi J, Seo BK, Lee D, Park H, Park JI (2013) Rgb-d camera-based hand shape recognition for human-robot interaction. In: IEEE ISR 2013. IEEE, 1–2 Choi J, Seo BK, Lee D, Park H, Park JI (2013) Rgb-d camera-based hand shape recognition for human-robot interaction. In: IEEE ISR 2013. IEEE, 1–2
5.
go back to reference Palacios JM, Sagüés C, Montijano E, Llorente S (2013) Human-computer interaction based on hand gestures using rgb-d sensors. Sensors 13(9):11842–11860CrossRef Palacios JM, Sagüés C, Montijano E, Llorente S (2013) Human-computer interaction based on hand gestures using rgb-d sensors. Sensors 13(9):11842–11860CrossRef
6.
go back to reference Tiantian L, Jinyuan S, Runjie L, Yingying G (2015) Hand gesture recognition based on improved histograms of oriented gradients. In: The 27th Chinese control and decision conference (2015 CCDC), 4211–4215 Tiantian L, Jinyuan S, Runjie L, Yingying G (2015) Hand gesture recognition based on improved histograms of oriented gradients. In: The 27th Chinese control and decision conference (2015 CCDC), 4211–4215
7.
go back to reference Jiang F, Wang C, Gao Y, Wu S, Zhao D (2015) Discriminating features learning in hand gesture classification. IET Comput Vis 9(5):673–680CrossRef Jiang F, Wang C, Gao Y, Wu S, Zhao D (2015) Discriminating features learning in hand gesture classification. IET Comput Vis 9(5):673–680CrossRef
8.
go back to reference Panwar M (2012) Hand gesture recognition based on shape parameters. In: 2012 international conference on computing, communication and applications, 1–6 Panwar M (2012) Hand gesture recognition based on shape parameters. In: 2012 international conference on computing, communication and applications, 1–6
9.
go back to reference Mesbahi SC, Mahraz MA, Riffi J, Tairi H (2018) Hand gesture recognition based on convexity approach and background subtraction. In: 2018 international conference on intelligent systems and computer vision (ISCV). IEEE, 1–5 Mesbahi SC, Mahraz MA, Riffi J, Tairi H (2018) Hand gesture recognition based on convexity approach and background subtraction. In: 2018 international conference on intelligent systems and computer vision (ISCV). IEEE, 1–5
10.
go back to reference Ganapathyraju S (2013) Hand gesture recognition using convexity hull defects to control an industrial robot. In: 2013 3rd international conference on instrumentation control and automation (ICA). IEEE, 63–67 Ganapathyraju S (2013) Hand gesture recognition using convexity hull defects to control an industrial robot. In: 2013 3rd international conference on instrumentation control and automation (ICA). IEEE, 63–67
11.
go back to reference Maisto M, Panella M, Liparulo L, Proietti A (2013) An accurate algorithm for the identification of fingertips using an rgb-d camera. IEEE J Emerg Sel Top Circ Syst 3(2):272–283CrossRef Maisto M, Panella M, Liparulo L, Proietti A (2013) An accurate algorithm for the identification of fingertips using an rgb-d camera. IEEE J Emerg Sel Top Circ Syst 3(2):272–283CrossRef
12.
go back to reference Plouffe G, Cretu AM (2015) Static and dynamic hand gesture recognition in depth data using dynamic time warping. IEEE Trans Instrum Measur 65(2):305–316CrossRef Plouffe G, Cretu AM (2015) Static and dynamic hand gesture recognition in depth data using dynamic time warping. IEEE Trans Instrum Measur 65(2):305–316CrossRef
13.
go back to reference Ma X, Peng J (2018) Kinect sensor-based long-distance hand gesture recognition and fingertip detection with depth information. J Sens Ma X, Peng J (2018) Kinect sensor-based long-distance hand gesture recognition and fingertip detection with depth information. J Sens
14.
go back to reference Xu Y, Park DW, Pok G (2017) Hand gesture recognition based on convex defect detection. Int J Appl Eng Res 12(18):7075–7079 Xu Y, Park DW, Pok G (2017) Hand gesture recognition based on convex defect detection. Int J Appl Eng Res 12(18):7075–7079
15.
go back to reference Xu Y, Dai Y (2018) Convex defect detection and density distribution based hand gesture recognition. J Comput 29(5):29–40 Xu Y, Dai Y (2018) Convex defect detection and density distribution based hand gesture recognition. J Comput 29(5):29–40
16.
go back to reference Kakkoth SS, Gharge, S (2018) Visual descriptors based real time hand gesture recognition. In: International conference on advances in communication and computing technology (ICACCT). IEEE, 361–367 Kakkoth SS, Gharge, S (2018) Visual descriptors based real time hand gesture recognition. In: International conference on advances in communication and computing technology (ICACCT). IEEE, 361–367
17.
go back to reference Hartanto R, Kartikasari A (2016) Android based real-time static Indonesian sign language recognition system prototype. In: 2016 8th international conference on information technology and electrical engineering (ICITEE). IEEE, 1–6 Hartanto R, Kartikasari A (2016) Android based real-time static Indonesian sign language recognition system prototype. In: 2016 8th international conference on information technology and electrical engineering (ICITEE). IEEE, 1–6
18.
go back to reference Li Y (2012) Multi-scenario gesture recognition using kinect. In: 2012 17th international conference on computer games (CGAMES). IEEE, 126–130 Li Y (2012) Multi-scenario gesture recognition using kinect. In: 2012 17th international conference on computer games (CGAMES). IEEE, 126–130
19.
go back to reference Saha HN, Tapadar S, Ray S, Chatterjee SK, Saha S (2018) A machine learning based approach for hand gesture recognition using distinctive feature extraction. In: IEEE 8th annual computing and communication workshop and conference (CCWC). IEEE, 91–98 Saha HN, Tapadar S, Ray S, Chatterjee SK, Saha S (2018) A machine learning based approach for hand gesture recognition using distinctive feature extraction. In: IEEE 8th annual computing and communication workshop and conference (CCWC). IEEE, 91–98
20.
go back to reference Ren Z, Meng J, Yuan J (2011) Depth camera based hand gesture recognition and its applications in human-computer-interaction. In: 2011 8th international conference on information, communications and signal processing (ICICS). IEEE, 1–5 Ren Z, Meng J, Yuan J (2011) Depth camera based hand gesture recognition and its applications in human-computer-interaction. In: 2011 8th international conference on information, communications and signal processing (ICICS). IEEE, 1–5
21.
go back to reference Ren Z, Yuan J, Meng J, Zhang Z (2013) Robust part-based hand gesture recognition using kinect sensor. IEEE Trans Multimedia 15(5):1110–1120CrossRef Ren Z, Yuan J, Meng J, Zhang Z (2013) Robust part-based hand gesture recognition using kinect sensor. IEEE Trans Multimedia 15(5):1110–1120CrossRef
22.
go back to reference Wang C, Chan S (2014) A new hand gesture recognition algorithm based on joint color-depth superpixel earth mover’s distance. In: 4th international workshop on cognitive information processing (CIP). IEEE, 1–6 Wang C, Chan S (2014) A new hand gesture recognition algorithm based on joint color-depth superpixel earth mover’s distance. In: 4th international workshop on cognitive information processing (CIP). IEEE, 1–6
23.
go back to reference Wang C, Liu Z, Chan SC (2014) Superpixel-based hand gesture recognition with kinect depth camera. IEEE Trans Multimedia 17(1):29–39CrossRef Wang C, Liu Z, Chan SC (2014) Superpixel-based hand gesture recognition with kinect depth camera. IEEE Trans Multimedia 17(1):29–39CrossRef
24.
go back to reference Paul S, Basu S, Nasipuri M (2015) Microsoft kinect in gesture recognition: a short review. Int J Control Theory Appl 8(5):20712076 Paul S, Basu S, Nasipuri M (2015) Microsoft kinect in gesture recognition: a short review. Int J Control Theory Appl 8(5):20712076
25.
go back to reference Paul S, Bhattacharyya A, Mollah AF, Basu S, Nasipuri M (2019) Hand segmentation from complex background for gesture recognition. In: Emerging technology in modelling and graphics. Springer Singapore, 775–782 Paul S, Bhattacharyya A, Mollah AF, Basu S, Nasipuri M (2019) Hand segmentation from complex background for gesture recognition. In: Emerging technology in modelling and graphics. Springer Singapore, 775–782
Metadata
Title
Convexity Defects-Based Fingertip Detection and Hand Gesture Recognition
Authors
Soumi Paul
Shrouti Gangopadhyay
Ayatullah Faruk Mollah
Subhadip Basu
Mita Nasipuri
Copyright Year
2023
Publisher
Springer Nature Singapore
DOI
https://doi.org/10.1007/978-981-19-0105-8_21