Skip to main content
Top
Published in: Journal on Multimodal User Interfaces 4/2019

25-04-2019 | Original Paper

Object acquisition and selection using automatic scanning and eye blinks in an HCI system

Authors: Hari Singh, Jaswinder Singh

Published in: Journal on Multimodal User Interfaces | Issue 4/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

This paper presents an object acquisition and selection approach in human computer interaction systems. In this approach, objects placed over computer screen are automatically scanned and the user performs voluntary eye blinks for object selection when the focus comes over the object of interest. Here, scanning means moving the focus over objects placed on the computer screen one by one and the scanning time is the time taken to move focus from one object to the next object. The user is not required to perform any physical movement, the moving part is only the eye lids. A low cost webcam and MATLAB software with computer vision toolbox are required to implement the proposed approach. The performance of the proposed approach has been compared with the Camera Mouse for selection of text and graphic objects. The Camera Mouse utilizes facial feature tracking for mouse cursor control and dwell time for object selection. Three experiments were performed for evaluation of the proposed method in which ten healthy users voluntarily participated. The proposed method has given significantly better performance than the Camera Mouse when selection of text objects was performed in an html file. For selection of graphic objects placed on computer screen, where page scrolling is not required, no significant difference has been found in the performance of both the systems. The proposed method has also been evaluated for performing mouse analogous operations using eye blinks and a performance comparison has been made with state-of-the-art methods.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Hegde VN, Ullagaddimath RS, Kumuda S (2016) Low cost eye based human computer interface system. In: Proceedings of 2016 IEEE annual India conference (INDICON), pp 1–6 Hegde VN, Ullagaddimath RS, Kumuda S (2016) Low cost eye based human computer interface system. In: Proceedings of 2016 IEEE annual India conference (INDICON), pp 1–6
2.
go back to reference Zhao XA, Guestrin ED, Sayenko D et al (2012) Typing with eye-gaze and tooth-clicks. In: Proceedings of ETRA’12 the symposium on eye tracking research and applications, pp 341–344 Zhao XA, Guestrin ED, Sayenko D et al (2012) Typing with eye-gaze and tooth-clicks. In: Proceedings of ETRA’12 the symposium on eye tracking research and applications, pp 341–344
4.
go back to reference Hao Z, Lei Q (2008) Vision-based interface: using face and eye blinking tracking with camera. In: Proceedings of IITA’08 second international symposium on intelligent information technology applications, pp 308–310 Hao Z, Lei Q (2008) Vision-based interface: using face and eye blinking tracking with camera. In: Proceedings of IITA’08 second international symposium on intelligent information technology applications, pp 308–310
6.
go back to reference Quain R, Khan MM (2014) Portable tongue-supported human computer interaction system design and implementation. In: Proceedings of 2014 36th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp 6302–6307 Quain R, Khan MM (2014) Portable tongue-supported human computer interaction system design and implementation. In: Proceedings of 2014 36th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp 6302–6307
8.
go back to reference Biswas P, Langdon P (2013) A new interaction technique involving eye gaze tracker and scanning system. In: Proceedings of the 2013 conference on eye tracking, pp 67–70 Biswas P, Langdon P (2013) A new interaction technique involving eye gaze tracker and scanning system. In: Proceedings of the 2013 conference on eye tracking, pp 67–70
9.
go back to reference Grauman K, Betke M, Lombardi J et al (2003) Communication via eye blinks and eyebrow raises: video-based human-computer interfaces. Univ Access Inf Soc 2:359–373CrossRef Grauman K, Betke M, Lombardi J et al (2003) Communication via eye blinks and eyebrow raises: video-based human-computer interfaces. Univ Access Inf Soc 2:359–373CrossRef
11.
go back to reference Gips J, Olivieri P, Tecce J (1993) Direct control of the computer through electrodes placed around the eyes. In: Proceedings of the fifth international conference on human-computer interaction: applications and case studies, pp 630–635 Gips J, Olivieri P, Tecce J (1993) Direct control of the computer through electrodes placed around the eyes. In: Proceedings of the fifth international conference on human-computer interaction: applications and case studies, pp 630–635
12.
go back to reference Deepika SS, Murugesan G (2015) A novel approach for human computer interface on eye movements for disabled people. In: Proceedings of 2015 IEEE international conference on electrical, computer and communication technologies (ICECCT 2015) Deepika SS, Murugesan G (2015) A novel approach for human computer interface on eye movements for disabled people. In: Proceedings of 2015 IEEE international conference on electrical, computer and communication technologies (ICECCT 2015)
13.
go back to reference Kumar M, Paepcke A, Winograd T (2007) EyePoint: practical pointing and selection using gaze and keyboard. In: CHI’07 proceedings of the SIGCHI conference on human factors in computing systems, pp 421–430 Kumar M, Paepcke A, Winograd T (2007) EyePoint: practical pointing and selection using gaze and keyboard. In: CHI’07 proceedings of the SIGCHI conference on human factors in computing systems, pp 421–430
15.
go back to reference Singh JV, Prasad G (2015) Enhancing an eye-tracker based human-computer interface with multi-modal accessibility applied for text entry. Int J Comput Appl 130:16–22 Singh JV, Prasad G (2015) Enhancing an eye-tracker based human-computer interface with multi-modal accessibility applied for text entry. Int J Comput Appl 130:16–22
16.
go back to reference Urbina MH, Weimar B (2010) Alternatives to single character entry and dwell time selection on eye typing. In: Proceedings of ETRA’10 the 2010 symposium on eye-tracking research and applications, pp 315–322 Urbina MH, Weimar B (2010) Alternatives to single character entry and dwell time selection on eye typing. In: Proceedings of ETRA’10 the 2010 symposium on eye-tracking research and applications, pp 315–322
17.
go back to reference Vertegaal R (2008) A Fitts’ law comparison of eye tracking and manual input in the selection of visual targets. In: Proceedings of ICMI’08 the 10th international conference on multimodal interfaces, pp 241–248 Vertegaal R (2008) A Fitts’ law comparison of eye tracking and manual input in the selection of visual targets. In: Proceedings of ICMI’08 the 10th international conference on multimodal interfaces, pp 241–248
18.
go back to reference Arai K, Mardiyanto R (2011) Eye-based HCI with full specification of mouse and keyboard using pupil knowledge in the gaze estimation. In: Proceedings of 2011 8th international conference on information technology: new generations, ITNG 2011, pp 423–428 Arai K, Mardiyanto R (2011) Eye-based HCI with full specification of mouse and keyboard using pupil knowledge in the gaze estimation. In: Proceedings of 2011 8th international conference on information technology: new generations, ITNG 2011, pp 423–428
20.
go back to reference Siriluck W, Kamolphiwong S, Kamolphiwong T (2007) Blink and click. In: Proceedings of the 1st international convention on rehabilitation engineering and assistive technology: in conjunction with 1st Tan Tock Seng Hospital Neurorehabilitation meeting, pp 43–46 Siriluck W, Kamolphiwong S, Kamolphiwong T (2007) Blink and click. In: Proceedings of the 1st international convention on rehabilitation engineering and assistive technology: in conjunction with 1st Tan Tock Seng Hospital Neurorehabilitation meeting, pp 43–46
21.
go back to reference Huckauf A, Urbina MH, Weimar B (2008) On object selection in gaze controlled environments. J Eye Mov Res 2:1–7 Huckauf A, Urbina MH, Weimar B (2008) On object selection in gaze controlled environments. J Eye Mov Res 2:1–7
22.
go back to reference Surakka V, Illi M, Isokoski P (2004) Gazing and frowning as a new human–computer interaction technique. ACM Trans Appl Perception 1:40–56CrossRef Surakka V, Illi M, Isokoski P (2004) Gazing and frowning as a new human–computer interaction technique. ACM Trans Appl Perception 1:40–56CrossRef
24.
go back to reference Posusta A, Sporka AJ, Polacek O et al (2015) Control of word processing environment using myoelectric signals. J Multimodal User Interfaces 9:299–311CrossRef Posusta A, Sporka AJ, Polacek O et al (2015) Control of word processing environment using myoelectric signals. J Multimodal User Interfaces 9:299–311CrossRef
25.
go back to reference Gizatdinova Y, Spakov O, Surakka V (2012) Comparison of video-based pointing and selection techniques for hands-free text entry. In: Proceedings of AVI’12 the international working conference advanced visual interfaces, pp 132–139 Gizatdinova Y, Spakov O, Surakka V (2012) Comparison of video-based pointing and selection techniques for hands-free text entry. In: Proceedings of AVI’12 the international working conference advanced visual interfaces, pp 132–139
26.
go back to reference Rantanen V, Verho J, Lekkala J et al (2012) The effect of clicking by smiling on the accuracy of head-mounted gaze tracking. In: Proceedings of ETRA’12 the symposium on eye tracking research and applications, pp 345–348 Rantanen V, Verho J, Lekkala J et al (2012) The effect of clicking by smiling on the accuracy of head-mounted gaze tracking. In: Proceedings of ETRA’12 the symposium on eye tracking research and applications, pp 345–348
28.
go back to reference Zhang X, Ren X, Zha H (2008) Improving eye cursor’s stability for eye pointing tasks. In: Proceedings of SIGCHI conference on human factors in computing systems, pp 525–534 Zhang X, Ren X, Zha H (2008) Improving eye cursor’s stability for eye pointing tasks. In: Proceedings of SIGCHI conference on human factors in computing systems, pp 525–534
29.
go back to reference Kim H, Suh KH, Lee EC (2017) Multi-modal user interface combining eye tracking and hand gesture recognition. J Multimodal User Interfaces 11:241–250CrossRef Kim H, Suh KH, Lee EC (2017) Multi-modal user interface combining eye tracking and hand gesture recognition. J Multimodal User Interfaces 11:241–250CrossRef
31.
go back to reference Konig WA, Radle R, Reiterer H (2010) Interactive design of multimodal user interfaces. J Multimodal User Interfaces 3:197–213CrossRef Konig WA, Radle R, Reiterer H (2010) Interactive design of multimodal user interfaces. J Multimodal User Interfaces 3:197–213CrossRef
35.
go back to reference Venkataramanan S, Prabhat P, Chaudhury SR et al (2005) Biomedical instrumentation based on electrooculogram (EOG) signal processing and application to a hospital alarm system. In: Proceedings of the 2005 international conference on intelligent sensing and information processing, pp 535–540 Venkataramanan S, Prabhat P, Chaudhury SR et al (2005) Biomedical instrumentation based on electrooculogram (EOG) signal processing and application to a hospital alarm system. In: Proceedings of the 2005 international conference on intelligent sensing and information processing, pp 535–540
36.
go back to reference Kumar D, Poole E (2002) Classification of EOG for human computer interface. In: Proceedings of the second joint EMBS/BMES conference, pp 64–67 Kumar D, Poole E (2002) Classification of EOG for human computer interface. In: Proceedings of the second joint EMBS/BMES conference, pp 64–67
37.
go back to reference Pander T (2008) An application of detection function for the eye blinking detection. In: Proceedings of human-computer systems interaction part of the advances in intelligent and soft-computing—book series, pp 181–191 Pander T (2008) An application of detection function for the eye blinking detection. In: Proceedings of human-computer systems interaction part of the advances in intelligent and soft-computing—book series, pp 181–191
38.
go back to reference Drutarovsky T, Fogelton A (2014) Eye blink detection using variance of motion vectors. In: Proceedings of ECCV 2014 workshop on computer vision, pp 1–12 Drutarovsky T, Fogelton A (2014) Eye blink detection using variance of motion vectors. In: Proceedings of ECCV 2014 workshop on computer vision, pp 1–12
39.
go back to reference Divjak M, Bischof H (2009) Eye blink based fatigue detection for prevention of computer vision syndrome. In: Proceedings of IAPR conference on machine vision applications, pp 350–353 Divjak M, Bischof H (2009) Eye blink based fatigue detection for prevention of computer vision syndrome. In: Proceedings of IAPR conference on machine vision applications, pp 350–353
40.
go back to reference Danisman T, Bilasco IM, Djeraba C, Ihaddadene N (2010) Drowsy driver detection system using eye blink patterns. In: Proceedings of international conference on machine and web intelligence. Algiers, Algeria, 3–5 October, pp 230–233 Danisman T, Bilasco IM, Djeraba C, Ihaddadene N (2010) Drowsy driver detection system using eye blink patterns. In: Proceedings of international conference on machine and web intelligence. Algiers, Algeria, 3–5 October, pp 230–233
42.
go back to reference Missimer E, Betke M (2010) Blink and wink detection for mouse pointer control. In: Proceedings of PETRA’10 the 3rd international conference on pervasive technologies related to assistive environments. Samos, Greece, June 23–25, pp 1–8 Missimer E, Betke M (2010) Blink and wink detection for mouse pointer control. In: Proceedings of PETRA’10 the 3rd international conference on pervasive technologies related to assistive environments. Samos, Greece, June 23–25, pp 1–8
43.
go back to reference Chareonsuk W, Kanhaun S, Khawkam K, Wongsawang D (2016) Face and eyes mouse for ALS patients. In: Proceedings of 2016 fifth ICT international student project conference (ICT-ISPC), pp 1–4 Chareonsuk W, Kanhaun S, Khawkam K, Wongsawang D (2016) Face and eyes mouse for ALS patients. In: Proceedings of 2016 fifth ICT international student project conference (ICT-ISPC), pp 1–4
46.
go back to reference Palleja T, Rubión E, Tresanchez M, Fernández A (2008) Using the optical flow to implement a relative virtual mouse controlled by head movements. J Univ Comput Sci 14:3127–3141 Palleja T, Rubión E, Tresanchez M, Fernández A (2008) Using the optical flow to implement a relative virtual mouse controlled by head movements. J Univ Comput Sci 14:3127–3141
47.
go back to reference Pimplaskar D, Nagmode MS, Borkar A (2013) Real time eye blinking detection and tracking using OpenCV. Int J Eng Res Appl 3:1780–1787 Pimplaskar D, Nagmode MS, Borkar A (2013) Real time eye blinking detection and tracking using OpenCV. Int J Eng Res Appl 3:1780–1787
Metadata
Title
Object acquisition and selection using automatic scanning and eye blinks in an HCI system
Authors
Hari Singh
Jaswinder Singh
Publication date
25-04-2019
Publisher
Springer International Publishing
Published in
Journal on Multimodal User Interfaces / Issue 4/2019
Print ISSN: 1783-7677
Electronic ISSN: 1783-8738
DOI
https://doi.org/10.1007/s12193-019-00303-0

Other articles of this Issue 4/2019

Journal on Multimodal User Interfaces 4/2019 Go to the issue

Premium Partner