Thiago Santini
ABSTRACT
Smooth pursuit eye movements provide meaningful insights and information on subject's behavior and health and may, in particular situations, disturb the performance of typical fixation/saccade classification algorithms. Thus, an automatic and efficient algorithm to identify these eye movements is paramount for eye-tracking research involving dynamic stimuli. In this paper, we propose the Bayesian Decision Theory Identification (I-BDT) algorithm, a novel algorithm for ternary classification of eye movements that is able to reliably separate fixations, saccades, and smooth pursuits in an online fashion, even for low-resolution eye trackers. The proposed algorithm is evaluated on four datasets with distinct mixtures of eye movements, including fixations, saccades, as well as straight and circular smooth pursuits; data was collected with a sample rate of 30 Hz from six subjects, totaling 24 evaluation datasets. The algorithm exhibits high and consistent performance across all datasets and movements relative to a manual annotation by a domain expert (recall: μ = 91.42%, σ = 9.52%; precision: μ = 95.60%, σ = 5.29%; specificity μ = 95.41%, σ = 7.02%) and displays a significant improvement when compared to I-VDT, an state-of-the-art algorithm (recall: μ = 87.67%, σ = 14.73%; precision: μ = 89.57%, σ = 8.05%; specificity μ = 92.10%, σ = 11.21%). Algorithm implementation and annotated datasets are openly available at www.ti.uni-tuebingen.de/perception
Supplemental Material
- Ben-David, A. 2007. A lot of randomness is hiding in accuracy. Engineering Applications of Artificial Intelligence 20, 7, 875--885. Google Scholar
Digital Library
- Berg, D. J., Boehnke, S. E., Marino, R. A., Munoz, D. P., and Itti, L. 2009. Free viewing of dynamic stimuli by humans and monkeys. Journal of Vision 9, 5, 19.Google ScholarCross Ref
- Braunagel, C., Kasneci, E., Stolzmann, W., and Rosenstiel, W. 2015. Driver-activity recognition in the context of conditionally autonomous driving. In IEEE 18th International Conference on Intelligent Transportation Systems (ITSC). Google ScholarDigital Library
- Fletcher, W. A., and Sharpe, J. A. 1988. Smooth pursuit dysfunction in alzheimer's disease. Neurology 38, 2, 272--272.Google ScholarCross Ref
- Fuhl, W., Kübler, T., Sippel, K., Rosenstiel, W., and Kasneci, E. 2015. Excuse: Robust pupil detection in real-world scenarios. In Computer Analysis of Images and Patterns 2015. CAIP 2015. 16th International Conference, IEEE.Google Scholar
- Galar, M., Fernández, A., Barrenechea, E., Bustince, H., and Herrera, F. 2011. An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes. Pattern Recognition 44, 8, 1761--1776. Google ScholarDigital Library
- Gyllensten, O. C. 2014. Evaluating current algorithms for smooth pursuit detection on Tobii Eye Trackers. Master's thesis, Royal Institute of Technology, Sweden.Google Scholar
- Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., and Van de Weijer, J. 2011. Eye tracking: A comprehensive guide to methods and measures. Oxford University Press.Google Scholar
- Kasneci, E., Kasneci, G., Kübler, T., and Rosenstiel, W. 2015. Online recognition of fixations, saccades, and smooth pursuits for automated analysis of traffic hazard perception. In Artificial Neural Networks, P. Koprinkova-Hristova, V. Mladenov, and N. K. Kasabov, Eds., vol. 4 of Springer Series in Bio-/Neuroinformatics. Springer International Publishing, 411--434.Google Scholar
- Kasneci, E. 2013. Towards the automated recognition of assistance need for drivers with impaired visual field. PhD thesis, Universität Tübingen, Germany.Google Scholar
- Kleiner, M., Brainard, D., Pelli, D., Ingling, A., Murray, R., and Broussard, C. 2007. Whats new in psychtoolbox-3. Perception 36, 14, 1.Google Scholar
- Knuth, D. E. 1992. Two notes on notation. American Mathematical Monthly, 403--422. Google ScholarDigital Library
- Komogortsev, O. V., and Karpov, A. 2013. Automated classification and scoring of smooth pursuit eye movements in the presence of fixations and saccades. Behavior research methods 45, 1, 203--215.Google Scholar
- Komogortsev, O. V., and Khan, J. I. 2007. Kalman filtering in the design of eye-gaze-guided computer interfaces. In Human-Computer Interaction. HCI Intelligent Multimodal Interaction Environments. Springer, 679--689. Google ScholarDigital Library
- Komogortsev, O. V., Gobert, D. V., Jayarathna, S., Koh, D. H., and Gowda, S. M. 2010. Standardization of automated analyses of oculomotor fixation and saccadic behaviors. Biomedical Engineering, IEEE Transactions on 57, 11, 2635--2645.Google ScholarCross Ref
- Larsson, L., Nystrom, M., and Stridh, M. 2013. Detection of saccades and postsaccadic oscillations in the presence of smooth pursuit. Biomedical Engineering, IEEE Transactions on 60, 9, 2484--2493.Google ScholarCross Ref
- Larsson, L., Nyström, M., Andersson, R., and Stridh, M. 2015. Detection of fixations and smooth pursuit movements in high-speed eye-tracking data. Biomedical Signal Processing and Control 18, 145--152.Google ScholarCross Ref
- Larsson, L. 2010. Event detection in eye-tracking data. Master's thesis, Lund University, Sweden.Google Scholar
- Leigh, R. J., and Zee, D. S. 2015. The neurology of eye movements. Oxford University Press.Google Scholar
- Lopez, J. S. A. 2009. Off-the-shelf Gaze Interaction. PhD thesis, IT University of Copenhagen, Denmark.Google Scholar
- Meyer, C. H., Lasker, A. G., and Robinson, D. A. 1985. The upper limit of human smooth pursuit velocity. Vision Research 25, 4, 561--563.Google ScholarCross Ref
- ODriscoll, G. A., and Callahan, B. L. 2008. Smooth pursuit in schizophrenia: a meta-analytic review of research since 1993. Brain and cognition 68, 3, 359--370.Google Scholar
- Salvucci, D. D., and Goldberg, J. H. 2000. Identifying fixations and saccades in eye-tracking protocols. In Proceedings of the 2000 symposium on Eye tracking research & applications, ACM, 71--78. Google ScholarDigital Library
- Santini, T., Fuhl, W., Kübler, T., and Kasneci, E. 2016. Eyerec: An open-source data acquisition software for head-mounted eye-tracking. In International Conference on Vision Theory and Applications (VISAPP). Forthcoming.Google Scholar
- Sauter, D., Martin, B., Di Renzo, N., and Vomscheid, C. 1991. Analysis of eye tracking movements using innovations generated by a kalman filter. Medical and biological Engineering and Computing 29, 1, 63--69.Google Scholar
- Stampe, D. M. 1993. Heuristic filtering and reliable calibration methods for video-based pupil-tracking systems. Behavior Research Methods, Instruments, & Computers 25, 2, 137--142.Google ScholarCross Ref
- Tafaj, E., Kasneci, G., Rosenstiel, W., and Bogdan, M. 2012. Bayesian online clustering of eye movement data. In Proceedings of the Symposium on Eye Tracking Research and Applications, ACM, New York, NY, USA, ETRA '12, 285--288. Google ScholarDigital Library
- Vidal, M., Bulling, A., and Gellersen, H. 2011. Analysing eog signal features for the discrimination of eye movements with wearable devices. In Proceedings of the 1st international workshop on pervasive eye tracking & mobile eye-based interaction, ACM, 15--20. Google ScholarDigital Library
- Vidal, M., Bulling, A., and Gellersen, H. 2012. Detection of smooth pursuits using eye movement shape features. In Proceedings of the symposium on eye tracking research and applications, ACM, 177--180. Google ScholarDigital Library
- Vidal, M., Bulling, A., and Gellersen, H. 2013. Pursuits: spontaneous interaction with displays based on smooth pursuit eye movement and moving targets. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing, ACM, 439--448. Google ScholarDigital Library
Index Terms
- Bayesian identification of fixations, saccades, and smooth pursuits
Recommendations
The applicability of probabilistic methods to the online recognition of fixations and saccades in dynamic scenes
ETRA '14: Proceedings of the Symposium on Eye Tracking Research and ApplicationsIn many applications involving scanpath analysis, especially when dynamic scenes are viewed, consecutive fixations and saccades, have to be identified and extracted from raw eye-tracking data in an online fashion. Since probabilistic methods can adapt ...
Comparing Dwell time, Pursuits and Gaze Gestures for Gaze Interaction on Handheld Mobile Devices
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing SystemsGaze is promising for hands-free interaction on mobile devices. However, it is not clear how gaze interaction methods compare to each other in mobile settings. This paper presents the first experiment in a mobile setting that compares three of the most ...
Hierarchical HMM for Eye Movement Classification
Computer Vision – ECCV 2020 WorkshopsAbstractIn this work, we tackle the problem of ternary eye movement classification, which aims to separate fixations, saccades and smooth pursuits from the raw eye positional data. The efficient classification of these different types of eye movements ...
Comments