Skip to main content
Top

2013 | OriginalPaper | Chapter

Control with Hand Gestures in Home Environment: A Review

Author : Sheau-Farn Max Liang

Published in: Proceedings of the Institute of Industrial Engineers Asian Conference 2013

Publisher: Springer Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

With many advances made in the area of automatic gesture recognition, gestural control has gradually gained its acceptance and popularity. However, the research to date has tended to focus on recognition technologies rather than human behaviors. With an emphasis on users, this paper reviews recent research literature on hand gestural control in home environment. The aim is to summarize and analyze current development processes of gesture vocabularies for commanding home appliances. A semiotic dual triadic model is proposed for the review of the control commands for the appliances, the types of hand gestures, the derivation processes of designer-defined and user-defined gestures. A typical derivation process for a user-defined gesture set was first collecting raw data by inquiring potential users to perform the most suitable gestures that they thought for triggering the given commands or functions, and then applying algorithms for the selection of final gesture vocabularies. A brief comparison among the research results of user-defined freehand gestures for TV control commands was provided as an example to show a need of research in this area. Further research direction includes the exploration of broader user population and the refinement of current gesture selection algorithms.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
go back to reference Aoki R, Karatsu Y, Ihara M, Maeda A, Kobayashi M, Kagami S (2011a) Gesture identification based on zone entry and axis crossing. In: Paper presented at the 14th international conference on human-computer interaction. Orlando, USA, 9–14 July 2011 LNCS 6762-2: 194–203 Aoki R, Karatsu Y, Ihara M, Maeda A, Kobayashi M, Kagami S (2011a) Gesture identification based on zone entry and axis crossing. In: Paper presented at the 14th international conference on human-computer interaction. Orlando, USA, 9–14 July 2011 LNCS 6762-2: 194–203
go back to reference Aoki R, Karatsu Y, Ihara M, Maeda A, Kobayashi M, Kagami S (2011b) Unicursal gesture interface for TV remote with touch screens. In: Paper presented at the 2011 IEEE international conference on consumer electronics (ICCE). Las Vegas, USA, 9–12 Jan 2011, pp 99–100 Aoki R, Karatsu Y, Ihara M, Maeda A, Kobayashi M, Kagami S (2011b) Unicursal gesture interface for TV remote with touch screens. In: Paper presented at the 2011 IEEE international conference on consumer electronics (ICCE). Las Vegas, USA, 9–12 Jan 2011, pp 99–100
go back to reference Aoki R, Karatsu Y, Ihara M, Maeda A, Kobayashi M, Kagami S (2011c) Expanding kinds of gestures for hierarchical menu selection by unicursal gesture interface. IEEE T Consum Electr 57–2:731–737CrossRef Aoki R, Karatsu Y, Ihara M, Maeda A, Kobayashi M, Kagami S (2011c) Expanding kinds of gestures for hierarchical menu selection by unicursal gesture interface. IEEE T Consum Electr 57–2:731–737CrossRef
go back to reference Aoki R, Ihara M, Kobayashi T, Kobayashi M, Chan B, Kagami S (2012) A gesture recognition algorithm for vision-based unicursal gesture interfaces. In: Paper presented at the 10th European conference on interactive TV and video (EuroITV’12). Berlin, Germany, 4–6 July 2012, pp 53–56 Aoki R, Ihara M, Kobayashi T, Kobayashi M, Chan B, Kagami S (2012) A gesture recognition algorithm for vision-based unicursal gesture interfaces. In: Paper presented at the 10th European conference on interactive TV and video (EuroITV’12). Berlin, Germany, 4–6 July 2012, pp 53–56
go back to reference Bailly G, Vo D-B, Lecolinet E, Guiard Y (2011a) Gesture-aware remote controls: Guidelines and interaction techniques. In: Paper presented at the 13th international conference on multimodal interfaces (ICMI’11). Alicante, Spain, 14–18 Nov 2011, pp 263–270 Bailly G, Vo D-B, Lecolinet E, Guiard Y (2011a) Gesture-aware remote controls: Guidelines and interaction techniques. In: Paper presented at the 13th international conference on multimodal interfaces (ICMI’11). Alicante, Spain, 14–18 Nov 2011, pp 263–270
go back to reference Bailly G, Walter R, Müller J, Ning T, Lecolinet E (2011b) Comparing free hand menu techniques for distant displays using linear, marking and finger-count menus. In: Paper presented at the 13th IFIP TC 13 international conference on human-computer interaction (INTERACT’11). Lisbon, Portugal, 5–9 Sept 2011 LNCS 6947-2: 248–262 Bailly G, Walter R, Müller J, Ning T, Lecolinet E (2011b) Comparing free hand menu techniques for distant displays using linear, marking and finger-count menus. In: Paper presented at the 13th IFIP TC 13 international conference on human-computer interaction (INTERACT’11). Lisbon, Portugal, 5–9 Sept 2011 LNCS 6947-2: 248–262
go back to reference Bobeth J, Schmehl S, Kruijff E, Deutsch S, Tscheligi M (2012) Evaluating performance and acceptance of older adults using freehand gestures for TV menu control. In: Paper presented at the 10th European conference on interactive TV and video (EuroITV’12). Berlin, Germany, 4–6 July 2012, pp 35–44 Bobeth J, Schmehl S, Kruijff E, Deutsch S, Tscheligi M (2012) Evaluating performance and acceptance of older adults using freehand gestures for TV menu control. In: Paper presented at the 10th European conference on interactive TV and video (EuroITV’12). Berlin, Germany, 4–6 July 2012, pp 35–44
go back to reference Choi E, Kwon S, Lee D, Lee H, Chung MK (2012) Can user-defined gesture be considered as the best gesture for a command? Focusing on the commands for smart home system. In: Paper presented at the human factors and ergonomics society 56th annual meeting. Boston, USA, 22–26 Oct 2012, pp 1253–1257 Choi E, Kwon S, Lee D, Lee H, Chung MK (2012) Can user-defined gesture be considered as the best gesture for a command? Focusing on the commands for smart home system. In: Paper presented at the human factors and ergonomics society 56th annual meeting. Boston, USA, 22–26 Oct 2012, pp 1253–1257
go back to reference De Miranda LC, Hornung HH, Baranauskas C (2010) Adjustable interactive rings for iDTV. IEEE T Consum Electr 56(3):1988–1996CrossRef De Miranda LC, Hornung HH, Baranauskas C (2010) Adjustable interactive rings for iDTV. IEEE T Consum Electr 56(3):1988–1996CrossRef
go back to reference De Miranda LC, Hornung HH, Baranauskas C (2011a) Adjustable interactive rings for iDTV: first results of an experiment with end-users. In: Paper presented at the 14th international conference on human-computer interaction. Orlando, USA, 9–14 July 2011 LNCS 6776: 262–271 De Miranda LC, Hornung HH, Baranauskas C (2011a) Adjustable interactive rings for iDTV: first results of an experiment with end-users. In: Paper presented at the 14th international conference on human-computer interaction. Orlando, USA, 9–14 July 2011 LNCS 6776: 262–271
go back to reference De Miranda LC, Hornung HH, Baranauskas C (2011b) Prospecting a new physical artifact of interaction for iDTV: results of participatory practices. In: Paper presented at the 14th international conference on human-computer interaction. Orlando, USA, 9–14 July 2011 LNCS 6770: 167–176 De Miranda LC, Hornung HH, Baranauskas C (2011b) Prospecting a new physical artifact of interaction for iDTV: results of participatory practices. In: Paper presented at the 14th international conference on human-computer interaction. Orlando, USA, 9–14 July 2011 LNCS 6770: 167–176
go back to reference Dezfuli N, Khalilbeigi M, Huber J, Müller FB, Mühlhäuser M (2012) PalmRC: imaginary palm-based remote control for eyes-free television interaction. In: Paper presented at the 10th European conference on interactive TV and video (EuroITV’12). Berlin, Germany, 4–6 July 2012, pp 27–34 Dezfuli N, Khalilbeigi M, Huber J, Müller FB, Mühlhäuser M (2012) PalmRC: imaginary palm-based remote control for eyes-free television interaction. In: Paper presented at the 10th European conference on interactive TV and video (EuroITV’12). Berlin, Germany, 4–6 July 2012, pp 27–34
go back to reference Freeman D, Vennelakanti R, Madhvanath S (2012) Freehand pose-based gestural interaction: Studies and implications for interface design. In: Paper presented at the 4th international conference on intelligent human computer interaction. IIT Kharagpur, India, 27–29 Dec 2012, pp 1–6 Freeman D, Vennelakanti R, Madhvanath S (2012) Freehand pose-based gestural interaction: Studies and implications for interface design. In: Paper presented at the 4th international conference on intelligent human computer interaction. IIT Kharagpur, India, 27–29 Dec 2012, pp 1–6
go back to reference Guesgen H, Kessell D (2012) Gestural control of household appliances for the physically impaired. In: Paper presented at the 25th international Florida artificial intelligence research society conference. Macro Island, USA, 23–25 May 2012, pp 353–358 Guesgen H, Kessell D (2012) Gestural control of household appliances for the physically impaired. In: Paper presented at the 25th international Florida artificial intelligence research society conference. Macro Island, USA, 23–25 May 2012, pp 353–358
go back to reference Ishikawa T, Horry Y, Hoshino T (2005) Touchless input device and gesture commands. In: Paper presented at the IEEE international conference on consumer electronics. Las Vegas, USA, 8–12 Jan 2005, pp 205–206 Ishikawa T, Horry Y, Hoshino T (2005) Touchless input device and gesture commands. In: Paper presented at the IEEE international conference on consumer electronics. Las Vegas, USA, 8–12 Jan 2005, pp 205–206
go back to reference Jeong S, Jin J, Song T, Kwon K, Jeon JW (2012) Single-camera dedicated television control system using gesture drawing. IEEE T Consum Electr 58–4:1129–1137CrossRef Jeong S, Jin J, Song T, Kwon K, Jeon JW (2012) Single-camera dedicated television control system using gesture drawing. IEEE T Consum Electr 58–4:1129–1137CrossRef
go back to reference Jing L, Yamagishi K, Wang J, Zhou Y, Huang T, Cheng Z (2011) A unified method for multiple home appliances control through static finger gestures. In: Paper presented at the IEEE/IPSJ 11th international symposium on applications and the internet. Munich, Germany, 18–21 July 2011, pp 82–90 Jing L, Yamagishi K, Wang J, Zhou Y, Huang T, Cheng Z (2011) A unified method for multiple home appliances control through static finger gestures. In: Paper presented at the IEEE/IPSJ 11th international symposium on applications and the internet. Munich, Germany, 18–21 July 2011, pp 82–90
go back to reference Kela J, Korpipää P, Mäntyjärvi J, Kallio S, Savino G, Jozzo L, Di Marca S (2006) Accelerometer-based gesture control for a design environment. Pers Ubiquit Comput 10–5:285–299CrossRef Kela J, Korpipää P, Mäntyjärvi J, Kallio S, Savino G, Jozzo L, Di Marca S (2006) Accelerometer-based gesture control for a design environment. Pers Ubiquit Comput 10–5:285–299CrossRef
go back to reference Kühnel C, Westermann T, Hemmert F, Kratz S, Müller A, Möller S (2011) I’m home: defining and evaluating a gesture set for smart-home control. Int J Hum Comput St 69:693–704CrossRef Kühnel C, Westermann T, Hemmert F, Kratz S, Müller A, Möller S (2011) I’m home: defining and evaluating a gesture set for smart-home control. Int J Hum Comput St 69:693–704CrossRef
go back to reference Lee S-S, Maeng S, Kim D, Lee K-P, Lee W, Kim S, Jung S (2011) FlexRemote: exploring the effectiveness of deformable user interface as an input device for TV. In: Paper presented at the 14th international conference on human-computer interaction. Orlando, USA, 9–14 July 2011 CCIS 174: 62–65 Lee S-S, Maeng S, Kim D, Lee K-P, Lee W, Kim S, Jung S (2011) FlexRemote: exploring the effectiveness of deformable user interface as an input device for TV. In: Paper presented at the 14th international conference on human-computer interaction. Orlando, USA, 9–14 July 2011 CCIS 174: 62–65
go back to reference Liang SFM (2006) Analyzing icon design by the axiomatic method: a case study of alarm icons in process control displays. Asian J Ergon, 7(1–2):11–28 Liang SFM (2006) Analyzing icon design by the axiomatic method: a case study of alarm icons in process control displays. Asian J Ergon, 7(1–2):11–28
go back to reference Löcken A, Hesselmann T, Pielot M, Henze N, Boll S (2012) User-centred process for the definition of free-hand gestures applied to controlling music playback. Multimedia Syst 18:15–31CrossRef Löcken A, Hesselmann T, Pielot M, Henze N, Boll S (2012) User-centred process for the definition of free-hand gestures applied to controlling music playback. Multimedia Syst 18:15–31CrossRef
go back to reference Morris MR (2012) Web on the wall: insights from a multimodal interaction elicitation study. In: Paper presented at the ACM international conference on interactive tabletops and surfaces. Cambridge, USA, 11–14 Nov 2012, pp 95–104 Morris MR (2012) Web on the wall: insights from a multimodal interaction elicitation study. In: Paper presented at the ACM international conference on interactive tabletops and surfaces. Cambridge, USA, 11–14 Nov 2012, pp 95–104
go back to reference Norman DA (1983) Some observations on mental models. In: Gentner D, Stevens AL (eds) Mental models. Erlbaum, Hillsdale, pp 7–14 Norman DA (1983) Some observations on mental models. In: Gentner D, Stevens AL (eds) Mental models. Erlbaum, Hillsdale, pp 7–14
go back to reference Obaid M, Häring M, Kistler F, Bühling R, André E (2012) User-defined body gestures for navigational control of a humanoid robot. In: Paper presented at the international conference on social robotics. Chengdu, China, 29–31 Oct 2012 LNAI 7621: 367–377 Obaid M, Häring M, Kistler F, Bühling R, André E (2012) User-defined body gestures for navigational control of a humanoid robot. In: Paper presented at the international conference on social robotics. Chengdu, China, 29–31 Oct 2012 LNAI 7621: 367–377
go back to reference Pan G, Wu J, Zhang D, Wu Z, Yang Y, Li S (2010) GeeAir: a universal multimodal remote control device for home appliances. Pers Ubiquit Comput 14:723–735CrossRef Pan G, Wu J, Zhang D, Wu Z, Yang Y, Li S (2010) GeeAir: a universal multimodal remote control device for home appliances. Pers Ubiquit Comput 14:723–735CrossRef
go back to reference Shimada A, Yamashita T, Taniguchi R (2013) Hand gesture based TV control system. In: Paper presented at the 19th Korea-Japan joint workshop on frontiers of computer vision. Incheon, Korea, 30 Jan–01 Feb 2013, pp 121–126 Shimada A, Yamashita T, Taniguchi R (2013) Hand gesture based TV control system. In: Paper presented at the 19th Korea-Japan joint workshop on frontiers of computer vision. Incheon, Korea, 30 Jan–01 Feb 2013, pp 121–126
go back to reference Stern HI, Wachs JP, Edan Y (2008) Designing hand gesture vocabularies for natural interaction by combining psycho-physiological and recognition factors. Int J Semantic Comput 2(1):137–160CrossRef Stern HI, Wachs JP, Edan Y (2008) Designing hand gesture vocabularies for natural interaction by combining psycho-physiological and recognition factors. Int J Semantic Comput 2(1):137–160CrossRef
go back to reference Tahir M, Bailly G, Lecolinet E (2007) ARemote: A tangible interface for selecting TV channels. In: Paper presented at the 17th international conference on artificial reality and telexistence. Esbjerg, Denmark, 28–30 Nov 2007, pp 298–299 Tahir M, Bailly G, Lecolinet E (2007) ARemote: A tangible interface for selecting TV channels. In: Paper presented at the 17th international conference on artificial reality and telexistence. Esbjerg, Denmark, 28–30 Nov 2007, pp 298–299
go back to reference Vatavu R-D (2012) User-defined gestures for free-hand TV control. In: Paper presented at the 10th European conference on interactive TV and video (EuroITV’12). Berlin, Germany, 4–6 July 2012, pp 45–48 Vatavu R-D (2012) User-defined gestures for free-hand TV control. In: Paper presented at the 10th European conference on interactive TV and video (EuroITV’12). Berlin, Germany, 4–6 July 2012, pp 45–48
go back to reference Wobbrock JO, Aung HH, Rothrock B, Myers BA (2005) Maximizing the guessability of symbolic input. In: Paper presented at the conference on human factors in computing (CHI). Oregon, USA, 2–7 April 2005 1869-1872 Wobbrock JO, Aung HH, Rothrock B, Myers BA (2005) Maximizing the guessability of symbolic input. In: Paper presented at the conference on human factors in computing (CHI). Oregon, USA, 2–7 April 2005 1869-1872
go back to reference Wu H, Wang J (2012) User-defined body gestures for TV-based applications. In: Paper presented at the 4th international conference on digital home. Guangzhou, China, 23–25 Nov 2012, pp 415–420 Wu H, Wang J (2012) User-defined body gestures for TV-based applications. In: Paper presented at the 4th international conference on digital home. Guangzhou, China, 23–25 Nov 2012, pp 415–420
go back to reference Wu H, Wang J (2013) Understanding user preferences for freehand gestures in the TV viewing environment. AISS 5–4:709–717 Wu H, Wang J (2013) Understanding user preferences for freehand gestures in the TV viewing environment. AISS 5–4:709–717
go back to reference Wu H, Chen X, Li G (2012) Simultaneous tracking and recognition of dynamic digit gestures for smart TV systems. In: Paper presented at the 4th international conference on digital home. Guangzhou, China, 23–25 Nov 2012, pp 351–356 Wu H, Chen X, Li G (2012) Simultaneous tracking and recognition of dynamic digit gestures for smart TV systems. In: Paper presented at the 4th international conference on digital home. Guangzhou, China, 23–25 Nov 2012, pp 351–356
Metadata
Title
Control with Hand Gestures in Home Environment: A Review
Author
Sheau-Farn Max Liang
Copyright Year
2013
Publisher
Springer Singapore
DOI
https://doi.org/10.1007/978-981-4451-98-7_100