Skip to main content
Top

2017 | OriginalPaper | Chapter

Test-Retest Stability of EEG and Eye Tracking Metrics as Indicators of Variations in User State—An Analysis at a Group and an Individual Level

Authors : Jessica Schwarz, Sven Fuchs

Published in: Advances in Neuroergonomics and Cognitive Engineering

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

EEG- and eye tracking metrics have been investigated for their potential as indicators of user state in a variety of studies. However, their stability over time has rarely been assessed and findings are reported predominantly on a group level. In this paper, we report a test-retest analysis that aimed to investigate—at group and individual levels—the temporal stability of fixation duration, pupil dilation, and two built-in metrics from the Emotiv EPOC EEG sensor, namely Engagement and Frustration. The retest confirmed the temporal stability of most physiological metrics at the group level. But analysis at an individual level revealed that outcomes differ strongly between and also within individuals from test to retest. The divergent results between individual and group level illustrate that group level findings are of limited value for applications such as adaptive systems requiring individual user state diagnosis.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Footnotes
1
In accordance with Tomarken [6], we refer to the term stability for test-retest analyses across sessions and reliability for test-retest-analyses within sessions.
 
Literature
1.
go back to reference Dirican, C., Göktürk, M.: Psychophysiological measures of human cognitive states applied in human computer interaction. Procedia Comput. Sci. 3, 1361–1367 (2011)CrossRef Dirican, C., Göktürk, M.: Psychophysiological measures of human cognitive states applied in human computer interaction. Procedia Comput. Sci. 3, 1361–1367 (2011)CrossRef
2.
go back to reference Kruse, A.A., Schmorrow, D.D.: Session Overview: foundations of augmented cognition. In Proceedings of the 11th International Conference on Human-Computer Interaction (Augmented Cognition International) (2005) Kruse, A.A., Schmorrow, D.D.: Session Overview: foundations of augmented cognition. In Proceedings of the 11th International Conference on Human-Computer Interaction (Augmented Cognition International) (2005)
3.
go back to reference Berka, C., Levendowski, D.J., Davis, G., Whitmoyer, M., Hale, K., Fuchs, S.: Objective measures of situational awareness using neurophysiology technology. In: Schmorrow, D.D., Stanney, K.M., Reeves, L.M. (eds.) Foundations of Augmented Cognition, pp. 145–154. Strategic Analysis Inc, Arlington, VA (2006) Berka, C., Levendowski, D.J., Davis, G., Whitmoyer, M., Hale, K., Fuchs, S.: Objective measures of situational awareness using neurophysiology technology. In: Schmorrow, D.D., Stanney, K.M., Reeves, L.M. (eds.) Foundations of Augmented Cognition, pp. 145–154. Strategic Analysis Inc, Arlington, VA (2006)
4.
go back to reference Schwarz, J., Fuchs, S., Flemisch, F.: Towards a more holistic view on user state assessment in adaptive human-computer interaction. In: IEEE International Conference on Systems, Man, and Cybernetics (SMC 2014), pp. 1228–1234 (2014) Schwarz, J., Fuchs, S., Flemisch, F.: Towards a more holistic view on user state assessment in adaptive human-computer interaction. In: IEEE International Conference on Systems, Man, and Cybernetics (SMC 2014), pp. 1228–1234 (2014)
5.
go back to reference Faulstich, M.E., Williamson, D.A., McKenzie, S.J., Duchmann, E.G., Hutchinson, K.M., Blouin, D.C.: Temporal stability of psychophysiological responding: a comparative analysis of mental and physical stressors. Int. J. Neurosci. 30(1–2), 65–72 (1986)CrossRef Faulstich, M.E., Williamson, D.A., McKenzie, S.J., Duchmann, E.G., Hutchinson, K.M., Blouin, D.C.: Temporal stability of psychophysiological responding: a comparative analysis of mental and physical stressors. Int. J. Neurosci. 30(1–2), 65–72 (1986)CrossRef
6.
go back to reference Tomarken, A.J.: A psychometric perspective on psychophysiological measures. Psychol. Assess. 7(3), 387–395 (1995)CrossRef Tomarken, A.J.: A psychometric perspective on psychophysiological measures. Psychol. Assess. 7(3), 387–395 (1995)CrossRef
7.
go back to reference Schwarz, J., Fuchs, S.: Efficacy of metrics from a low-cost EEG sensor for multi-dimensional user state assessment. In: Wienrich, C., Zander, T. Gramann, K. (eds.) Trends in Neuroergonomics: Proceedings of the 11. Berlin Workshop Human-Machine Systems, pp. 20–21. Universitätsverlag der TU Berlin, Berlin (2015) Schwarz, J., Fuchs, S.: Efficacy of metrics from a low-cost EEG sensor for multi-dimensional user state assessment. In: Wienrich, C., Zander, T. Gramann, K. (eds.) Trends in Neuroergonomics: Proceedings of the 11. Berlin Workshop Human-Machine Systems, pp. 20–21. Universitätsverlag der TU Berlin, Berlin (2015)
8.
go back to reference Neerincx, M.A.: Cognitive task load design: model, methods and examples. In: Hollnagel, E. (ed.) Handbook of Cognitive Task Design, pp. 283–305. Lawrence Erlbaum Associates, Mahwah, NJ (2003)CrossRef Neerincx, M.A.: Cognitive task load design: model, methods and examples. In: Hollnagel, E. (ed.) Handbook of Cognitive Task Design, pp. 283–305. Lawrence Erlbaum Associates, Mahwah, NJ (2003)CrossRef
9.
go back to reference Reuderink, B., Nijholt, A., Poel, M.: Affective pacman: a frustrating game for brain-computer interface experiments. In: Third International Conference on Intelligent Technologies for Interactive Entertainment, INTETAIN 2009, pp. 221—227. Amsterdam, The Netherlands (2009) Reuderink, B., Nijholt, A., Poel, M.: Affective pacman: a frustrating game for brain-computer interface experiments. In: Third International Conference on Intelligent Technologies for Interactive Entertainment, INTETAIN 2009, pp. 221—227. Amsterdam, The Netherlands (2009)
10.
go back to reference Diener, H., Oertel, K.: Experimental approach to affective interaction in games. In: Pan, Z., et al. (eds.) Edutainment 2006, LNCS, vol. 3942, pp. 507–518. Springer, Heidelberg (2006) Diener, H., Oertel, K.: Experimental approach to affective interaction in games. In: Pan, Z., et al. (eds.) Edutainment 2006, LNCS, vol. 3942, pp. 507–518. Springer, Heidelberg (2006)
11.
go back to reference Pomplun, M., Reingold, E.M., Shen, J.: Investigating the visual span in comparative search: the effects of task difficulty and divided attention. Cognition 81(2), B57–B67 (2001)CrossRef Pomplun, M., Reingold, E.M., Shen, J.: Investigating the visual span in comparative search: the effects of task difficulty and divided attention. Cognition 81(2), B57–B67 (2001)CrossRef
12.
go back to reference King, L.A.: Visual navigation patterns and cognitive load. In: Foundations of augmented cognition. Neuroergonomics and operational neuroscience, pp. 254–259. Springer, Berlin (2009) King, L.A.: Visual navigation patterns and cognitive load. In: Foundations of augmented cognition. Neuroergonomics and operational neuroscience, pp. 254–259. Springer, Berlin (2009)
14.
go back to reference Duchowski, A.T.: A breadth-first survey of eye tracking applications. Behav. Res. Methods Instrum. Comput. 34, 455–471 (2002)CrossRef Duchowski, A.T.: A breadth-first survey of eye tracking applications. Behav. Res. Methods Instrum. Comput. 34, 455–471 (2002)CrossRef
15.
go back to reference De Rivecourt, M., Kuperus, M.N., Post, W.J., Mulder, L.J.M.: Cardiovascular and eye activity measures as indices for momentary changes in mental effort during simulated flight. Ergonomics 51(9), 1295–1319 (2008)CrossRef De Rivecourt, M., Kuperus, M.N., Post, W.J., Mulder, L.J.M.: Cardiovascular and eye activity measures as indices for momentary changes in mental effort during simulated flight. Ergonomics 51(9), 1295–1319 (2008)CrossRef
16.
go back to reference Richer, F., Beatty, J.: Pupillary dilations in movement preparation and execution. Psychophysiology 22(2), 204–207 (1985)CrossRef Richer, F., Beatty, J.: Pupillary dilations in movement preparation and execution. Psychophysiology 22(2), 204–207 (1985)CrossRef
17.
go back to reference Van Orden, K.F., Limbert, W., Makeig, S., Jung, T.P.: Eye activity correlates of workload during a visuospatial memory task. Hum. Factors 43(1), 111–121 (2001)CrossRef Van Orden, K.F., Limbert, W., Makeig, S., Jung, T.P.: Eye activity correlates of workload during a visuospatial memory task. Hum. Factors 43(1), 111–121 (2001)CrossRef
18.
go back to reference Iqbal, S.T., Zheng, X.S., Bailey, B.P.: Task-evoked pupillary response to mental workload in human computer interaction. In: Ext. Abstracts CHI 2004, pp. 1477–1480. ACM Press (2004) Iqbal, S.T., Zheng, X.S., Bailey, B.P.: Task-evoked pupillary response to mental workload in human computer interaction. In: Ext. Abstracts CHI 2004, pp. 1477–1480. ACM Press (2004)
19.
go back to reference Jiang, X., Zheng, B., Bednarik, R., Atkins, M.S.: Pupil responses to continuous aiming movements. Int. J. Hum Comput. Stud. 83, 1–11 (2015)CrossRef Jiang, X., Zheng, B., Bednarik, R., Atkins, M.S.: Pupil responses to continuous aiming movements. Int. J. Hum Comput. Stud. 83, 1–11 (2015)CrossRef
20.
go back to reference Kramer, A.F.: Physiological metrics of mental workload: a review of recent progress. In: Damos, D. (ed) Multiple Task Performance, pp. 279—328. Taylor and Francis, London (1991) Kramer, A.F.: Physiological metrics of mental workload: a review of recent progress. In: Damos, D. (ed) Multiple Task Performance, pp. 279—328. Taylor and Francis, London (1991)
21.
go back to reference Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental Workload, pp. 139–183. NorthHolland, Amsterdam (1988)CrossRef Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental Workload, pp. 139–183. NorthHolland, Amsterdam (1988)CrossRef
22.
go back to reference Bradley, M.M.: Lang. P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)CrossRef Bradley, M.M.: Lang. P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)CrossRef
Metadata
Title
Test-Retest Stability of EEG and Eye Tracking Metrics as Indicators of Variations in User State—An Analysis at a Group and an Individual Level
Authors
Jessica Schwarz
Sven Fuchs
Copyright Year
2017
DOI
https://doi.org/10.1007/978-3-319-41691-5_13

Premium Partner