Top

Published in:

2021 | OriginalPaper | Chapter

Eye Movement Classification with Temporal Convolutional Networks

Authors : Carlos Elmadjian, Candy Gonzales, Carlos H. Morimoto

Published in: Pattern Recognition. ICPR International Workshops and Challenges

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Recently, deep learning approaches have been proposed to detect eye movements such as fixations, saccades, and smooth pursuits from eye tracking data. These are end-to-end methods that have shown to surpass traditional ones, requiring no ad hoc parameters. In this work we propose the use of temporal convolutional networks (TCNs) for automated eye movement classification and investigate the influence of feature space, scale, and context window sizes on the classification results. We evaluated the performance of TCNs against a state-of-the-art 1D-CNN-BLSTM model using GazeCom, a public available dataset. Our results show that TCNs can outperform the 1D-CNN-BLSTM, achieving an F-score of 94.2% for fixations, 89.9% for saccades, and 73.7% for smooth pursuits on sample level, and 89.6%, 94.3%, and 60.2% on event level. We also state the advantages of TCNs over sequential networks for this problem, and how these scores can be further improved by feature space extension.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Synthetic Gaze Data Augmentation for Improved User Calibration

next chapter A Web-Based Eye Tracking Data Visualization Tool

Agtzidis, I., Startsev, M., Dorr, M.: In the pursuit of (ground) truth: a hand-labelling tool for eye movements recorded during dynamic scene viewing. In: 2016 IEEE Second Workshop on Eye Tracking and Visualization (ETVIS), pp. 65–68 (2016)

Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. CoRR abs/1803.01271 (2018). http://arxiv.org/abs/1803.01271

Berg, D.J., Boehnke, S.E., Marino, R.A., Munoz, D.P., Itti, L.: Free viewing of dynamic stimuli by humans and monkeys. J. Vis. 9(5), 19 (2009). https://doi.org/10.1167/9.5.19CrossRef

Berndt, S., Kirkpatrick, D., Taviano, T., Komogortsev, O.: Tertiary eye movement classification by a hybrid algorithm. CoRR abs/1904.10085 (2019). http://arxiv.org/abs/1904.10085

Campbell, C.S., Maglio, P.P.: A robust algorithm for reading detection. In: Proceedings of the 2001 workshop on Perceptive User Interfaces, pp. 1–7 (2001)

Cassin, B., Rubin, M.L., Solomon, S.: Dictionary of Eye Terminology, vol. 10. Triad Publishing Company, Gainsville (1984)

Dorr, M., Martinetz, T., Gegenfurtner, K.R., Barth, E.: Variability of eye movements when viewing dynamic natural scenes. J. Vision 10(10), 28 (2010). https://doi.org/10.1167/10.10.28CrossRef

Duchowski, A.T.: Gaze-based interaction: a 30 year retrospective. Comput. Graph. 73, 59–69 (2018). https://doi.org/10.1016/j.cag.2018.04.002CrossRef

Fuhl, W.: Fully convolutional neural networks for raw eye tracking data segmentation, generation, and reconstruction (2020)

10.

Goodfellow, I.J., Bengio, Y., Courville, A.C.: Deep Learning: Adaptive computation and Machine Learning. MIT Press, Cambridge (2016). http://www.deeplearningbook.org/

11.

Hooge, I., Niehorster, D., Nyström, M., Andersson, R., Hessels, R.: Is human classification by experienced untrained observers a gold standard in fixation detection? Behav. Res. Methods 50(5), 1864–1881 (2018). https://doi.org/10.3758/s13428-017-0955-xCrossRef

12.

Hoppe, S., Bulling, A.: End-to-end eye movement detection using convolutional neural networks. CoRR abs/1609.02452 (2016). http://arxiv.org/abs/1609.02452

13.

Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015). http://arxiv.org/abs/1412.6980

14.

Komogortsev, O., Gobert, D., Jayarathna, S., Koh, D., Gowda, S.: Standardization of automated analyses of oculomotor fixation and saccadic behaviors. IEEE Trans. Biomed. Eng. 57, 2635–2645 (2010). https://doi.org/10.1109/TBME.2010.2057429CrossRef

15.

Komogortsev, O., Karpov, A.: Automated classification and scoring of smooth pursuit eye movements in the presence of fixations and saccades. Behav. Res. Methods 45 (2012). https://doi.org/10.3758/s13428-012-0234-9

16.

Komogortsev, O.V., Karpov, A.: Automated classification and scoring of smooth pursuit eye movements in the presence of fixations and saccades. Behav. Res. Methods 45(1), 203–215 (2013). https://doi.org/10.3758/s13428-012-0234-9CrossRef

17.

Komogortsev, O.V., Khan, J.I.: Kalman filtering in the design of eye-gaze-guided computer interfaces. In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4552, pp. 679–689. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-73110-8_74. http://dl.acm.org/citation.cfm?id=1769590.1769667CrossRef

18.

Larsson, L., Nyström, M., Andersson, R., Stridh, M.: Detection of fixations and smooth pursuit movements in high-speed eye-tracking data. Biomed. Signal Process. Control 18, 145–152 (2015). https://doi.org/10.1016/j.bspc.2014.12.008, http://www.sciencedirect.com/science/article/pii/S1746809414002031

19.

Leigh, R.J., Zee, D.S.: The neurology of eye movements. OUP USA (2015)

20.

Nyström, M., Holmqvist, K.: An adaptive algorithm for fixation, saccade, and glissade detection in eyetracking data. Behav. Res. Methods 42(1), 188–204 (2010). https://doi.org/10.3758/BRM.42.1.188

21.

Peters, C., Pelachaud, C., Bevacqua, E., Mancini, M., Poggi, I.: A model of attention and interest using gaze behavior. In: Panayiotopoulos, T., Gratch, J., Aylett, R., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS (LNAI), vol. 3661, pp. 229–240. Springer, Heidelberg (2005). https://doi.org/10.1007/11550617_20CrossRef

22.

Salvucci, D.D., Goldberg, J.H.: Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of the 2000 Symposium on Eye Tracking Research & Applications, ETRA 2000, pp. 71–78. ACM, New York (2000). https://doi.org/10.1145/355017.355028, http://doi.acm.org/10.1145/355017.355028

23.

Santini, T., Fuhl, W., Kübler, T., Kasneci, E.: Bayesian identification of fixations, saccades, and smooth pursuits. In: Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications, ETRA 2016, pp. 163–170. ACM, New York (2016). https://doi.org/10.1145/2857491.2857512, http://doi.acm.org/10.1145/2857491.2857512

24.

Sauter, D., Martin, B.J., Di Renzo, N., Vomscheid, C.: Analysis of eye tracking movements using innovations generated by a kalman filter. Med. Biol. Eng. Comput. 29(1), 63–69 (1991). https://doi.org/10.1007/BF02446297

25.

Shepherd, S.: Following gaze: Gaze-following behavior as a window into social cognition. Front. Integr. Neurosci. 4, 5 (2010). https://doi.org/10.3389/fnint.2010.00005, https://www.frontiersin.org/articles/10.3389/fnint.2010.00005

26.

Startsev, M., Agtzidis, I., Dorr, M.: 1D CNN with BLSTM for automated classification of fixations, saccades, and smooth pursuits. Behav. Res. Methods 51(2), 556–572 (2019). https://doi.org/10.3758/s13428-018-1144-2

27.

Startsev, M., Agtzidis, I., Dorr, M.: Sequence-to-sequence deep learning for eye movement classification. In: Perception, vol. 48, pp. 200–200. Sage Publications LTD., London (2019)

28.

Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, Montreal, Quebec, Canada, 8–13 December 2014, pp. 3104–3112 (2014). http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks

29.

Vidal, M., Bulling, A., Gellersen, H.: Detection of smooth pursuits using eye movement shape features. In: Proceedings of the Symposium on Eye Tracking Research and Applications, ETRA 2012, pp. 177–180. ACM, New York (2012). https://doi.org/10.1145/2168556.2168586, http://doi.acm.org/10.1145/2168556.2168586

30.

Zemblys, R., Niehorster, D.C., Holmqvist, K.: gazenet: End-to-end eye-movement event detection with deep neural networks. Behav. Res. Methods 51, 840–864 (2018)CrossRef

31.

Zemblys, R., Niehorster, D.C., Komogortsev, O., Holmqvist, K.: Using machine learning to detect events in eye-tracking data. Behav. Res. Methods 50(1), 160–181 (2018)CrossRef

Title: Eye Movement Classification with Temporal Convolutional Networks
Authors: Carlos Elmadjian
Candy Gonzales
Carlos H. Morimoto
Publisher: Springer International Publishing
Book: Pattern Recognition. ICPR International Workshops and Challenges
Print ISBN: 978-3-030-68795-3

Electronic ISBN: 978-3-030-68796-0

Copyright Year: 2021
DOI: https://doi.org/10.1007/978-3-030-68796-0_28

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner