Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Advanced Nuclear Interface Modeling Environment (ANIME): A Tool for Developing Human-Computer Interfaces for Experimental Process Control Systems Read chapter Read first chapter

2017 | OriginalPaper | Chapter

Challenges in Research Generalizability: The Need for Standardization of Performance Metrics and Methodology

Authors : Kathryn A. Feltman, Kyle A. Bernhardt, Amanda M. Kelley

Published in: HCI in Business, Government and Organizations. Interacting with Information Systems

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The proliferation of psychological and psychophysiological metrics, data collection techniques, and data analysis strategies used throughout psychological research of operator performance presents cross-study synthesis complications. Currently, the lack of defined and established standardizations in psychological and psychophysiological research continues to present challenges to researchers studying an array of interrelated constructs. Without standardizations, differences in measurement implementation, data reduction techniques, and the interpretation of results make it difficult to directly compare studies and reach unequivocal conclusions while synthesizing literature and transfer laboratory findings to field-ready applications.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Virtual World Versus Real World: An Economic Study of the Cyber Games Participation
next chapter The Adoption of Physiological Measures as an Evaluation Tool in UX
Literature
1.
2.
Button, K.S., Ioannidis, J.P.A., Mokrysz, C., Nosek, B.A., Flint, J., Robinson, E.S.J., Munafò, M.R.: Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14, 365–376 (2013)CrossRef
3.
Larson, M.J.: Commitment to cutting-edge research with rigor and replication in psychophysiological science. Int. J. Psychophysiol. 102, ix–x (2016)
4.
Pendlebury, S.T., Mariz, J., Bull, L., Mehta, Z., Rothwell, P.M.: Impact of different operational definitions on mild cognitive impairment rate and MMSE and MoCA performance in transient ischaemic attack and stroke. Cerebrovasc. Dis. 36(5–6), 355–362 (2013)CrossRef
5.
Kristman, V.L., Borg, J., Godbolt, A.K., Salmi, L.R., Cancelliere, C., Carroll, L.J., Holm, L.W., Nygren-de Boussard, C., Hartvigsen, J., Abara, U., Donovan, J., Cassidy, J.D. Methodological issues and research recommendations for prognosis after mild traumatic brain injury: results of the international collaboration on mild traumatic brain injury prognosis. Arch. Phys. Med. Rehab. 95(3), S265–S277 (2014)
6.
Berntson, G.G., Bigger, J.T., Eckberg, D.L., Grossman, P., Kaufmann, P.G., Malik, M., Nagaraja, H.N., Porges, S.W., Saul, J.P., Stone, P.H., van der Molen, M.W.: Heart rate variability: origins, methods, and interpretive caveats. Psychophysiology 34, 623–648 (1997)CrossRef
7.
Pivik, R.T., Broughton, R.J., Coppola, R., Davidson, R.J., Fox, N., Nuwer, M.R.: Guidelines for the recording and quantitative analysis of electroencephalographic activity in research contexts. Psychophysiology 30, 547–558 (1993)CrossRef
8.
Ritz, T., Dahme, B., Dubois, A.B., Folgering, H., Fritz, G.K., Harver, A., Kotses, H., Lehrer, P.M., Ring, C., Steptoe, A., Van de Woestijne, K.P.: Guidelines for mechanical lung function measures in psychophysiology. Psychophysiology 39, 546–567 (2002)
9.
Ryu, K., Myung, R.: Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic. Int. J. Ind. Ergon. 35, 991–1009 (2005)CrossRef
10.
Bonner, M.A., Wilson, G.F.: Heart rate measures of flight test and evaluation. Int. J. Aviat. Psychol. 12(1), 63–77 (2002)CrossRef
11.
Durantin, G., Gagnon, J.F., Tremblay, S., Dehais, F.: Using near infrared spectroscopy and heart rate variability to detect mental overload. Behav. Brain Res. 259, 16–23 (2014)CrossRef
12.
Hsu, B.W., Wang, M.J.J., Chen, C.Y.: Effective indices for monitoring mental workload while performing multiple tasks. Percept. Mot. Skills 121(1), 94–117 (2015)CrossRef
13.
Berntson, G.G., Quigley, K.S., Lozano, D.: Cardiovascular psychophysiology. In: Cacioppo, J.T., Tassinary, L.G., Berntson, G.G. (eds.) Handbook of Psychophysiology, pp. 182–210. Cambridge University Press, New York (2007)CrossRef
14.
Kligfield, P., Gettes, L.S., Bailey, J.J., Childers, R., Deal, B.J., Hancock, W., van Herpen, G., Kors, J., Macfarlane, P., Mirvis, D., Pahlm, O., Rautaharju, P., Wagner, G.S.: Recommendations for the standardization and interpretation of the electrocardiogram. Circulation 115, 1306–1324 (2007)
15.
Strube, M.J., Newman, L.C.: Psychometrics. In: Cacioppo, J.T., Tassinary, L.G., Berntson, G.G. (eds.) Handbook of Psychophysiology, pp. 789–811. Cambridge University Press, New York (2007)
16.
Weippert, M., Kumar, M., Kreuzfeld, S., Arndt, D., Rieger, A., Stoll, R.: Comparison of three mobile devices for measuring R-R intervals and heart rate variability: Polar S810i, Suunto t6 and an ambulatory ECG system. Eur. J. Appl. Physiol. 109(4), 779–786 (2010)CrossRef
17.
Di Stasi, L.L., Diaz-Piedra, C., Suárez, J., McCamy, M.B., Martinez-Conde, S., Roca-Dorda, J., Catena, A.: Task complexity modulates pilot electroencephalographic activity during real flights. Psychophysiology 52(7), 951–956 (2015)CrossRef
18.
Fairclough, S.H., Venables, L., Tattersall, A.: The influence of task demand and learning on the psychophysiological response. Int. J. Psychophysiol. 56, 171–184 (2005)CrossRef
19.
Cohen, M.X.: Rigor and replication in time-frequency analyses of cognitive electrophysiology data. Int. J. Psychophysiol. 111, 80–87 (2017)CrossRef
20.
Cacioppo, J.T., Tassinary, L.G.: Inferring psychological significance from physiological signals. Am. Psychol. 45(1), 16–28 (1990)CrossRef
21.
Pizzagalli, D.A.: Electroencephalography and high-density electrophysiological source localization. In: Cacioppo, J.T., Tassinary, L.G., Berntson, G.G. (eds.) Handbook of Psychophysiology, pp. 56–84. Cambridge University Press, New York (2007)CrossRef
22.
Ekman, P., Levenson, R.W., Friesen, W.V.: Autonomic nervous system activity distinguishes among emotions. Science 221, 1208–1210 (1983)CrossRef
23.
Pattyn, N., Neyt, X., Henderickx, D., Soetens, E.: Psychophysiological investigation of vigilance decrement: Boredom or cognitive fatigue? Physiol. Behav. 93, 369–378 (2008)CrossRef
24.
Stern, R.M., Ray, W.J., Quigley, K.S.: Psychophysiological Recording. University Press Inc., New York (2001)
25.
Pollak, M.H.: Heart rate reactivity to laboratory tasks and ambulatory heart rate in daily life. Psychosom. Med. 53, 25–35 (1991)CrossRef
26.
Jennings, J.R., Kamarck, T., Steward, C., Eddy, M., Johnson, P.: Alternate cardiovascular baseline assessment techniques: vanilla or resting baseline. Psychophysiology 29(6), 742–750 (1992)CrossRef
27.
Fishel, S.R., Muth, E.R., Hoover, A.W.: Establishing appropriate physiological baseline procedures for real-time physiological measurement. J. Cogn. Eng. Decis. Making 1, 286–308 (2007). doi:10.​1518/​15553407X255636 CrossRef
28.
29.
Davidson, R.J., Marshall, J.R., Tomarken, A.J., Henriques, J.B.: While a phobic waits: regional brain electrical and autonomic activity in social phobics during anticipation of public speaking. Biol. Psychol. 47(2), 85–95 (2000). doi:10.​1016/​S0006-3223(99)00222-X
30.
Hart, S.G., Bortolussi, M.R.: Pilot errors as a source of workload. Hum. Factors 26(5), 545–556 (1984)
31.
32.
33.
34.
Keil, A., Debener, S., Gratton, G., Junghofer, M., Kappenman, E.S., Luck, S.J., Luu, P., Miller, G.A., Yee, C.M.: Committee report: publication guidelines and recommendations for studies using electroencephalography and magnetoencephalograph. Psychophysiology 51(1), 1–21 (2014)CrossRef
35.
Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., Bablioni, F.: Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue, and drowsiness. Neurosci. Biobehav. Rev. 44, 58–75 (2014). doi:10.​1016/​jneubiorev.​2012.​10.​003 CrossRef
36.
Mehler, B., Reimer, B., Coughlin, J.F., Dusek, J.A.: Impact of incremental increases in cognitive workload on physiological arousal and performance in young adult drivers. Transp. Res. Rec. 2138, 6–12 (2009)CrossRef
37.
Riniolo, T., Porges, S.W.: Inferential and descriptive influences on measures of respiratory sinus arrthymia: sampling rate, R-wave trigger accuracy, and variance estimates. Psychophysiology 34, 613–621 (1997)CrossRef
38.
Nyquist, H.: Certain topics in telegraph transmission theory. IEEE Trans. Commun. 47, 617–644 (1928)
39.
Bolek, J.E.: Digital sampling, bits, and psychophysiological data: A primer, with cautions. Appl. Psychophys. Biofeedback 38(4), 303–308 (2013)CrossRef
40.
Weiergräber, M., Papazoglou, A., Broich, K., Muller, R.: Sampling rate, signal bandwidth and related pitfalls in EEG analysis. J. Neurosci. Methods 268, 53–55 (2016)CrossRef
41.
Parasuraman, R.: Neuroergonomic perspectives on human systems integration: mental workload, vigilance, adaptive automation, and training. In: Boehm-Davis, D.A., Durso, F.T., Lee, J.D. (eds.) APA Handbook of Human Systems Integration, pp. 163–176. American Psychological Association, Washington, DC (2015)CrossRef
42.
Warm, J.S., Parasuraman, R., Matthews, G.: Vigilance requires hard mental work and is stressful. Hum. Factors 50(3), 433–441 (2008)CrossRef
43.
RTO human factors, medicine panel task group: operator functional state assessment. Technical report, Research and Technology Organization (2004)
44.
Borghini, G., Aricò, P., Graziani, I., Salinari, S., Sun, Y., Taya, F., Bezerianos, A., Thakor, N.V., Babiloni, F.: Quantitative assessment of the training improvement in a motor-cognitive task by using EEG, ECG, and EOG signals. Brain Topogr. 29, 149–161 (2016)CrossRef
45.
Stuiver, A., Brookhuis, K.A., de Waard, D., Mulder, B.: Short-term cardiovascular measures for driver support: increasing sensitivity for detecting changes in mental workload. Int. J. Psychophysiol. 92, 35–41 (2014)CrossRef
Metadata
Title
Challenges in Research Generalizability: The Need for Standardization of Performance Metrics and Methodology
Authors
Kathryn A. Feltman
Kyle A. Bernhardt
Amanda M. Kelley
Copyright Year
2017
Book
HCI in Business, Government and Organizations. Interacting with Information Systems

Print ISBN: 978-3-319-58480-5

Electronic ISBN: 978-3-319-58481-2

Copyright Year: 2017

DOI
https://doi.org/10.1007/978-3-319-58481-2_7

Premium Partner

    Image Credits