Thiago Santini
ABSTRACT
Pervasive eye-tracking applications such as gaze-based human computer interaction and advanced driver assistance require real-time, accurate, and robust pupil detection. However, automated pupil detection has proved to be an intricate task in real-world scenarios due to a large mixture of challenges - for instance, quickly changing illumination and occlusions. In this work, we introduce the <u>Pu</u>pil <u>Re</u>constructor with <u>S</u>ubsequent <u>T</u>racking (PuReST), a novel method for fast and robust pupil tracking. The proposed method was evaluated on over 266,000 realistic and challenging images acquired with three distinct head-mounted eye tracking devices, increasing pupil detection rate by 5.44 and 29.92 percentage points while reducing average run time by a factor of 2.74 and 1.1. w.r.t. state-of-the-art 1) pupil detectors and 2) vendor provided pupil trackers, respectively. Overall, PuReST outperformed other methods in 81.82% of use cases.
- Reuben Aronson et al. 2018. Eye-Hand Behavior in Human-Robot Shared Manipulation. In Proceedings of the 13th Annual ACM/IEEE International Conference on Human Robot Interaction (To appear). Google Scholar
Digital Library
- John Canny. 1986. A computational approach to edge detection. IEEE Transactions on pattern analysis and machine intelligence 6 (1986), 679--698. Google ScholarDigital Library
- Byoung Sun Chu et al. 2010. The effect of presbyopic vision corrections on nighttime driving performance. Investigative ophthalmology & visual science 51, 9 (2010), 4861--4866.Google Scholar
- David H Douglas and Thomas K Peucker. 1973. Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica: The International Journal for Geographic Information and Geovisualization 10, 2 (1973), 112--122.Google ScholarCross Ref
- Andrew W. Fitzgibbon and Robert B. Fisher. 1995. A Buyer's Guide to Conic Fitting. In Proceedings of the 6th British Conference on Machine Vision (Vol. 2) (BMVC '95). BMVA Press, Surrey, UK, UK, 513--522. http://dl.acm.org/citation.cfm?id=243124.243148 Google ScholarDigital Library
- Tom Foulsham, Esther Walker, and Alan Kingstone. 2011. The where, what and when of gaze allocation in the lab and the natural environment. Vision research 51, 17 (2011), 1920--1931.Google Scholar
- Michael Frigge, David C Hoaglin, and Boris Iglewicz. 1989. Some implementations of the boxplot. The American Statistician 43, 1 (1989), 50--54.Google Scholar
- Wolfgang Fuhl et al. 2015. Excuse: Robust pupil detection in real-world scenarios. In International Conference on Computer Analysis of Images and Patterns. Springer, 39--51.Google Scholar
- Wolfgang Fuhl et al. 2017. PupilNet v2.0: Convolutional Neural Networks for CPU based real time Robust Pupil Detection. (2017).Google Scholar
- Wolfgang Fuhl, Thiago Santini, Gjergji Kasneci, and Enkelejda Kasneci. 2016a. PupilNet: convolutional neural networks for robust pupil detection. arXiv preprint arXiv:1601.04902 (2016).Google Scholar
- Wolfgang Fuhl, Thiago C Santini, Thomas Kübler, and Enkelejda Kasneci. 2016b. Else: Ellipse selection for robust pupil detection in real-world environments. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications. ACM, 123--130. Google ScholarDigital Library
- Wolfgang Fuhl, Marc Tonsen, Andreas Bulling, and Enkelejda Kasneci. 2016c. Pupil detection for head-mounted eye tracking in the wild: an evaluation of the state of the art. Machine Vision and Applications 27, 8 (2016), 1275--1288. Google ScholarDigital Library
- Dan Witzner Hansen and Riad I Hammoud. 2007. An improved likelihood model for eye tracking. Computer Vision and Image Understanding 106, 2 (2007), 220--230. Google ScholarDigital Library
- Dan Witzner Hansen and Arthur EC Pece. 2005. Eye tracking in the wild. Computer Vision and Image Understanding 98, 1 (2005), 155--181. Google ScholarDigital Library
- João F Henriques, Rui Caseiro, Pedro Martins, and Jorge Batista. 2015. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 3 (2015), 583--596.Google ScholarDigital Library
- Amir-Homayoun Javadi, Zahra Hakimi, Morteza Barati, Vincent Walsh, and Lili Tcheang. 2015. SET: a pupil detection method using sinusoidal approximation. Frontiers in neuroengineering 8 (2015).Google Scholar
- Zdenek Kalal et al. 2012. Tracking-learning-detection. IEEE transactions on pattern analysis and machine intelligence 34, 7 (2012), 1409--1422. Google ScholarDigital Library
- Enkelejda Kasneci. 2013. Towards the automated recognition of assistance need for drivers with impaired visual field. Ph.D. Dissertation. Universität Tübingen, Germany.Google Scholar
- Enkelejda Kasneci et al. 2014. Homonymous Visual Field Loss and Its Impact on Visual Exploration: A Supermarket Study. Translational vision science & technology 3, 6 (2014), 2--2.Google Scholar
- Moritz Kassner, William Patera, and Andreas Bulling. 2014. Pupil: an open source platform for pervasive eye tracking and mobile gaze-based interaction. In Proceedings of the 2014 ACM international joint conference on pervasive and ubiquitous computing: Adjunct publication. ACM, 1151--1160. Google ScholarDigital Library
- Thomas C Kübler et al. 2015. Driving with glaucoma: task performance and gaze movements. Optometry & Vision Science 92, 11 (2015), 1037--1046.Google ScholarCross Ref
- Dongheng Li, David Winfield, and Derrick J Parkhurst. 2005. Starburst: A hybrid algorithm for video-based eye tracking combining feature-based and model-based approaches. In Computer Vision and Pattern Recognition-Workshops, 2005. CVPR Workshops. IEEE Computer Society Conference on. IEEE, 79--79. Google ScholarDigital Library
- Yang Li and Jianke Zhu. 2014. A Scale Adaptive Kernel Correlation Filter Tracker with Feature Integration.. In ECCV Workshops (2). 254--265.Google Scholar
- Carlos H Morimoto and Marcio RM Mimica. 2005. Eye gaze tracking techniques for interactive applications. Computer vision and image understanding 98, 1 (2005), 4--24. Google ScholarDigital Library
- Mohammad Othman et al. 2017. CrowdEyes: Crowdsourcing for Robust Real-World Mobile Eye Tracking. (2017).Google Scholar
- Dilip K Prasad and Maylor KH Leung. 2012. Methods for ellipse detection from edge maps of real images. In Machine Vision-Applications and Systems. InTech.Google Scholar
- Pupil Labs. 2017a. Accessed in 2017-12-26. (2017). https://pupil-labs.com/Google Scholar
- Pupil Labs. 2017b. Accessed in 2017-12-26.(2017). https://pupil-labs.com/blog/2016-03/pupil-v0-7-release-notes/Google Scholar
- Thiago Santini, Hanna Brinkmann, Luise Reitstätter, Helmut Leder, Raphael Rosenberg, Wolfgang Rosenstiel, and Enkelejda Kasneci. 2018a. The Art of Pervasive Eye Tracking. In Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications (ETRA) - Adjunct: PETMEI. ACM. Google ScholarDigital Library
- Thiago Santini, Wolfgang Fuhl, David Geisler, and Enkelejda Kasneci. 2017b. EyeRecToo: Open-Source Software for Real-Time Pervasive Head-Mounted Eye-Tracking. In Proceedings of the 12th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications.Google ScholarCross Ref
- Thiago Santini, Wolfgang Fuhl, and Enkelejda Kasneci. 2017a. CalibMe: Fast and Unsupervised Eye Tracker Calibration for Gaze-Based Pervasive Human-Computer Interaction. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 2594--2605. Google ScholarDigital Library
- Thiago Santini, Wolfgang Fuhl, and Enkelejda Kasneci. 2018b. PuRe: Robust pupil detection for real-time pervasive eye tracking. Computer Vision and Image Understanding (Feb 2018).Google Scholar
- Thiago Santini, Wolfgang Fuhl, Thomas Kübler, and Enkelejda Kasneci. 2016. Bayesian identification of fixations, saccades, and smooth pursuits. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications. ACM, 163--170. Google ScholarDigital Library
- Jürgen Schmidt et al. 2017. Eye blink detection for different driver states in conditionally automated driving and manual driving using EOG and a driver camera. Behavior Research Methods (2017), 1--14.Google Scholar
- Jack Sklansky. 1982. Finding the convex hull of a simple polygon. Pattern Recognition Letters 1, 2 (1982), 79--83. Google ScholarDigital Library
- Yusuke Sugano and Andreas Bulling. 2015. Self-calibrating head-mounted eye trackers using egocentric visual saliency. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. ACM, 363--372. Google ScholarDigital Library
- Satoshi Suzuki et al. 1985. Topological structural analysis of digitized binary images by border following. Computer vision, graphics, and image processing 30, 1 (1985), 32--46.Google Scholar
- Lech Šwirski, Andreas Bulling, and Neil Dodgson. 2012. Robust real-time pupil tracking in highly off-axis images. In Proceedings of the Symposium on Eye Tracking Research and Applications. ACM, 173--176. Google ScholarDigital Library
- Lech Šwirski and Neil A. Dodgson. 2013. A fully-automatic, temporal approach to single camera, glint-free 3D eye model fitting {Abstract}. In Proceedings of ECEM 2013.Google Scholar
- Tony Tien et al. 2015. Differences in gaze behaviour of expert and junior surgeons performing open inguinal hernia repair. Surgical endoscopy 29, 2 (2015), 405--413.Google Scholar
- Fabian Timm and Erhardt Barth. 2011. Accurate Eye Centre Localisation by Means of Gradients. Visapp 11 (2011), 125--130.Google Scholar
- Marc Tonsen, Xucong Zhang, Yusuke Sugano, and Andreas Bulling. 2016. Labelled pupils in the wild: a dataset for studying pupil detection in unconstrained environments. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications. ACM, 139--142. Google ScholarDigital Library
- Sandra Trösterer, Alexander Meschtscherjakov, David Wilfinger, and Manfred Tscheligi. 2014. Eye Tracking in the Car: Challenges in a Dual-Task Scenario on a Test Track. In Proceedings of the 6th AutomotiveUI. ACM. Google ScholarDigital Library
- FJ Vera-Olmos and N Malpica. 2017. Deconvolutional Neural Network for Pupil Detection in Real-World Environments. In International Work-Conference on the Interplay Between Natural and Artificial Computation. Springer, 223--231.Google Scholar
- Joanne M Wood, Richard A Tyrrell, Philippe Lacherez, and Alex A Black. 2017. Night-time pedestrian conspicuity: effects of clothing on drivers' eye movements. Ophthalmic and physiological optics 37, 2 (2017), 184--190.Google Scholar
Index Terms
- PuReST: robust pupil tracking for real-time pervasive eye tracking
Recommendations
Get a grip: slippage-robust and glint-free gaze estimation for real-time pervasive head-mounted eye tracking
ETRA '19: Proceedings of the 11th ACM Symposium on Eye Tracking Research & ApplicationsA key assumption conventionally made by flexible head-mounted eye-tracking systems is often invalid: The eye center does not remain stationary w.r.t. the eye camera due to slippage. For instance, eye-tracker slippage might happen due to head ...
ElSe: ellipse selection for robust pupil detection in real-world environments
ETRA '16: Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & ApplicationsFast and robust pupil detection is an essential prerequisite for video-based eye-tracking in real-world settings. Several algorithms for image-based pupil detection have been proposed in the past, their applicability, however, is mostly limited to ...
The art of pervasive eye tracking: unconstrained eye tracking in the Austrian Gallery Belvedere
PETMEI '18: Proceedings of the 7th Workshop on Pervasive Eye Tracking and Mobile Eye-Based InteractionPervasive mobile eye tracking provides a rich data source to investigate human natural behavior, providing a high degree of ecological validity in natural environments. However, challenges and limitations intrinsic to unconstrained mobile eye tracking ...
Comments