Top

Cognitive Neurodynamics

Published in:

09-03-2020 | Research Article

A brain–computer interface for the continuous, real-time monitoring of working memory load in real-world environments

Authors: Aldo Mora-Sánchez, Alfredo-Aram Pulini, Antoine Gaume, Gérard Dreyfus, François-Benoît Vialatte

Published in: Cognitive Neurodynamics | Issue 3/2020

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

We developed a brain–computer interface (BCI) able to continuously monitor working memory (WM) load in real-time (considering the last 2.5 s of brain activity). The BCI is based on biomarkers derived from spectral properties of non-invasive electroencephalography (EEG), subsequently classified by a linear discriminant analysis classifier. The BCI was trained on a visual WM task, tested in a real-time visual WM task, and further validated in a real-time cross task (mental arithmetic). Throughout each trial of the cross task, subjects were given real or sham feedback about their WM load. At the end of the trial, subjects were asked whether the feedback provided was real or sham. The high rate of correct answers provided by the subjects validated not only the global behaviour of the WM-load feedback, but also its real-time dynamics. On average, subjects were able to provide a correct answer 82% of the time, with one subject having 100% accuracy. Possible cognitive and motor confounding factors were disentangled to support the claim that our EEG-based markers correspond indeed to WM.

previous article Design exploration predicts designer creativity: a deep learning approach

next article Intranasal insulin administration may be highly effective in improving cognitive function in mice with cognitive dysfunction by reversing brain insulin resistance

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

Allison BZ, Neuper C (2010) Could anyone use a BCI? In: Tan D, Nijholt A (eds) Brain–computer interfaces. Springer, Berlin, pp 35–54CrossRef

Antonenko P, Paas F, Grabner R, Van Gog T (2010) Using electroencephalography to measure cognitive load. Educ Psychol Rev 22(4):425–438CrossRef

Awh E, Vogel EK, Oh S-H (2006) Interactions between attention and working memory. Neuroscience 139(1):201–208PubMedCrossRef

Baddeley A (1992) Working memory. Science 255(5044):556–559PubMedCrossRef

Baddeley A (2003) Working memory: looking back and looking forward. Nat Rev Neurosci 4(10):829–839PubMedCrossRef

Baddeley AD (2017) The concept of working memory: a view of its current state and probable future development. In: Exploring working memory. Routledge, Abingdon, pp 99–106

Baldwin CL, Penaranda B (2012) Adaptive training using an artificial neural network and EEG metrics for within-and cross-task workload classification. NeuroImage 59(1):48–56PubMedCrossRef

Berka C, Levendowski DJ, Cvetinovic MM, Petrovic MM, Davis G, Lumicao MN, Zivkovic VT, Popovic MV, Olmstead R (2004) Real-time analysis of EEG indexes of alertness, cognition, and memory acquired with a wireless EEG headset. Int J Hum-Comput Interact 17(2):151–170CrossRef

Borghini G, Astolfi L, Vecchiato G, Mattia D, Babiloni F (2014) Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci Biobehav Rev 44:58–75PubMedCrossRef

Cavanagh JF, Frank MJ (2014) Frontal theta as a mechanism for cognitive control. Trends Cogn Sci 18(8):414–421PubMedPubMedCentralCrossRef

Cheney W, Kincaid D (2009) Linear algebra: theory and applications. The Australian Mathematical Society, Canberra, p 110

Cheng M, Lu Z, Wang H (2017) Regularized common spatial patterns with subject-to-subject transfer of EEG signals. Cogn Neurodyn 11(2):173–181PubMedCrossRef

Comstock Jr JR, Arnegard RJ (1992) The multi-attribute task battery for human operator workload and strategic behavior research

Cowan N (1999) An embedded-processes model of working memory. In: Miyake A, Shah P (eds) Models of working memory: mechanisms of active maintenance and executive control, vol 20. Cambridge University Press, Cambridge, p 506

Cragg L, Richardson S, Hubber PJ, Keeble S, Gilmore C (2017) When is working memory important for arithmetic? The impact of strategy and age. PLoS ONE 12(12):e0188693PubMedPubMedCentralCrossRef

de Fockert JW, Rees G, Frith CD, Lavie N (2001) The role of working memory in visual selective attention. Science 291(5509):1803–1806PubMedCrossRef

Dehais F, Duprès A, Blum S, Drougard N, Scannella S, Roy RN, Lotte F (2019) Monitoring pilot’s mental workload using ERPs and spectral power with a six-dry-electrode EEG system in real flight conditions. Sensors 19(6):1324CrossRefPubMedCentral

Di Flumeri G, Borghini G, Aricò P, Sciaraffa N, Lanzi P, Pozzi S, Vignali V, Lantieri C, Bichicchi A, Simone A et al (2018) EEG-based mental workload neurometric to evaluate the impact of different traffic and road conditions in real driving settings. Front Hum Neurosci 12:509PubMedPubMedCentralCrossRef

Downing PE (2000) Interactions between visual working memory and selective attention. Psychol Sci 11(6):467–473PubMedCrossRef

Erdogmus D, Adami A, Pavel M, Lan T, Mathan S, Whitlow S, Dorneich M (2005) Cognitive state estimation based on EEG for augmented cognition. In: Conference proceedings. 2nd international IEEE EMBS conference on neural engineering, 2005. IEEE, pp 566–569

Ericsson KA, Delaney PF (1999) Working memory in everyday skilled performance. In: Miyake A, Shah P (eds) Models of working memory: mechanisms of active maintenance and executive control, vol 20. Cambridge University Press, Cambridge, p 274

Farwell LA, Donchin E (1988) Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol 70(6):510–523PubMedCrossRef

Fisher RA (1936) The use of multiple measurements in taxonomic problems. Ann Eugen 7(2):179–188CrossRef

Gaume A, Vialatte A, Mora-Sánchez A, Ramdani C, Vialatte F (2016) A psychoengineering paradigm for the neurocognitive mechanisms of biofeedback and neurofeedback. Neurosci Biobehav Rev 68:891–910PubMedCrossRef

Gaume A, Dreyfus G, Vialatte F-B (2019) A cognitive brain–computer interface monitoring sustained attentional variations during a continuous task. Cogn Neurodyn 13(3):257–269PubMedPubMedCentralCrossRef

Gerjets P, Walter C, Rosenstiel W, Bogdan M, Zander TO (2014) Cognitive state monitoring and the design of adaptive instruction in digital environments: lessons learned from cognitive workload assessment using a passive brain–computer interface approach. Front Neurosci 8:385PubMedPubMedCentral

Gevins A, Smith ME, Leong H, McEvoy L, Whitfield S, Du R, Rush G (1998) Monitoring working memory load during computer-based tasks with EEG pattern recognition methods. Hum Factors J Hum Factors Ergon Soc 40(1):79–91CrossRef

Grimes D, Tan DS, Hudson SE, Shenoy P, Rao RP (2008) Feasibility and pragmatics of classifying working memory load with an electroencephalograph. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, pp 835–844

Haapalainen E, Kim S, Forlizzi JF, Dey AK (2010) Psycho-physiological measures for assessing cognitive load. In: Proceedings of the 12th ACM international conference on Ubiquitous computing. ACM, New York, pp 301–310

Haenschel C, Baldeweg T, Croft RJ, Whittington M, Gruzelier J (2000) Gamma and beta frequency oscillations in response to novel auditory stimuli: a comparison of human electroencephalogram (EEG) data with in vitro models. Proc Natl Acad Sci 97(13):7645–7650PubMedCrossRefPubMedCentral

Hart SG, Staveland LE (1988) Development of NASA-TLX (task load index): results of empirical and theoretical research. Adv Psychol 52:139–183CrossRef

Heger D, Putze F, Schultz T (2010) Online workload recognition from EEG data during cognitive tests and human–machine interaction. In: Annual conference on artificial intelligence. Springer, Berlin, pp 410–417

Hinterberger T, Kübler A, Kaiser J, Neumann N, Birbaumer N (2003) A brain–computer interface (BCI) for the locked-in: comparison of different EEG classifications for the thought translation device. Clin Neurophysiol 114(3):416–425PubMedCrossRef

Jensen O, Tesche CD (2002) Frontal theta activity in humans increases with memory load in a working memory task. Eur J Neurosci 15(8):1395–1399PubMedCrossRef

Jensen O, Gelfand J, Kounios J, Lisman JE (2002) Oscillations in the alpha band (9–12 Hz) increase with memory load during retention in a short-term memory task. Cereb Cortex 12(8):877–882PubMedCrossRef

Khasnobish A, Datta S, Bose R, Tibarewala D, Konar A (2017) Analyzing text recognition from tactually evoked EEG. Cogn Neurodyn 11(6):501–513PubMedPubMedCentralCrossRef

Kintsch W, Patel VL, Ericsson KA (1999) The role of long-term working memory in text comprehension. Psychologia 42(4):186–198

Kleiner M, Brainard D, Pelli D, Ingling A, Murray R, Broussard C (2007) What’s new in psychtoolbox-3. Perception 36(14):1

Kohavi R et al (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. IJCAI 14:1137–1145

Kohlmorgen J, Dornhege G, Braun M, Blankertz B, Müller K-R, Curio G, Hagemann K, Bruns A, Schrauf M, Kincses W et al (2007) Improving human performance in a real operating environment through real-time mental workload detection. In: Dornhege G, Millán JR, Hinterberger T, McFarland D (eds) Toward brain–computer interfacing. MIT Press, Cambridge, pp 409–422

Koles ZJ, Lazar MS, Zhou SZ (1990) Spatial patterns underlying population differences in the background EEG. Brain Topogr 2(4):275–284PubMedCrossRef

Lotte F, Larrue F, Mühl C (2013) Flaws in current human training protocols for spontaneous brain–computer interfaces: lessons learned from instructional design. Front Hum Neurosci 7:568PubMedPubMedCentralCrossRef

Lutz A (2002) Toward a neurophenomenology as an account of generative passages: a first empirical case study. Phenomenol Cogn Sci 1(2):133–167CrossRef

Lutz A, Thompson E (2003) Neurophenomenology integrating subjective experience and brain dynamics in the neuroscience of consciousness. J Conscious Stud 10(9–10):31–52

Mizuno K, Tanaka M, Yamaguti K, Kajimoto O, Kuratsune H, Watanabe Y (2011) Mental fatigue caused by prolonged cognitive load associated with sympathetic hyperactivity. Behav Brain Funct 7(1):17PubMedPubMedCentralCrossRef

Mohanchandra K, Saha S (2016) A communication paradigm using subvocalized speech: translating brain signals into speech. Augment Hum Res 1(1):3CrossRef

Mora-Sánchez A, Gaume A, Dreyfus G, Vialatte F-B (2015) A cognitive brain–computer interface prototype for the continuous monitoring of visual working memory load. In: 2015 IEEE 25th international workshop on machine learning for signal processing (MLSP). IEEE, pp 1–5

Mora-Sánchez A, Dreyfus G, Vialatte F-B (2019) Scale-free behaviour and metastable brain-state switching driven by human cognition, an empirical approach. Cogn Neurodyn 13:1–16CrossRef

Müller K-R, Tangermann M, Dornhege G, Krauledat M, Curio G, Blankertz B (2008) Machine learning for real-time single-trial EEG-analysis: from brain–computer interfacing to mental state monitoring. J Neurosci Methods 167(1):82–90PubMedCrossRef

Muller-Putz GR, Pfurtscheller G (2008) Control of an electrical prosthesis with an SSVEP-based BCI. IEEE Trans Biomed Eng 55(1):361–364PubMedCrossRef

Mumtaz W, Vuong PL, Xia L, Malik AS, Rashid RBA (2017) An EEG-based machine learning method to screen alcohol use disorder. Cogn Neurodyn 11(2):161–171PubMedCrossRef

Nasrabadi NM (2007) Pattern recognition and machine learning. J Electron Imaging 16(4):049901CrossRef

Paas F, Renkl A, Sweller J (2003) Cognitive load theory and instructional design: recent developments. Educ Psychol 38(1):1–4CrossRef

Sanchez G, Daunizeau J, Maby E, Bertrand O, Bompas A, Mattout J (2014) Toward a new application of real-time electrophysiology: online optimization of cognitive neurosciences hypothesis testing. Brain Sci 4(1):49–72PubMedPubMedCentralCrossRef

Sauseng P, Klimesch W, Doppelmayr M, Pecherstorfer T, Freunberger R, Hanslmayr S (2005a) EEG alpha synchronization and functional coupling during top-down processing in a working memory task. Hum Brain Mapp 26(2):148–155PubMedCrossRefPubMedCentral

Sauseng P, Klimesch W, Schabus M, Doppelmayr M (2005b) Fronto-parietal EEG coherence in theta and upper alpha reflect central executive functions of working memory. Int J Psychophysiol 57(2):97–103PubMedCrossRef

Shorrock ST (2005) Errors of memory in air traffic control. Saf Sci 43(8):571–588CrossRef

Stoppiglia H, Dreyfus G, Dubois R, Oussar Y (2003) Ranking a random feature for variable and feature selection. J Mach Learn Res 3:1399–1414

Strijkstra AM, Beersma DG, Drayer B, Halbesma N, Daan S (2003) Subjective sleepiness correlates negatively with global alpha (8–12 Hz) and positively with central frontal theta (4–8 Hz) frequencies in the human resting awake electroencephalogram. Neurosci Lett 340(1):17–20PubMedCrossRef

Sweller J (2011) Cognitive load theory. In: Mestre JP, Ross BH (eds) Psychology of learning and motivation, vol 55. Elsevier, New-York, pp 37–76CrossRef

Swets JA (2014) Signal detection theory and ROC analysis in psychology and diagnostics: collected papers. Psychology Press, HoveCrossRef

Tafreshi TF, Daliri MR, Ghodousi M (2019) Functional and effective connectivity based features of EEG signals for object recognition. Cogn Neurodyn 13(6):555–566PubMedCrossRefPubMedCentral

Varela FJ (1996) Neurophenomenology: a methodological remedy for the hard problem. J Conscious Stud 3(4):330–349

Varela FJ, Thompson E, Rosch E (2017) The embodied mind: cognitive science and human experience. MIT Press, CambridgeCrossRef

Vialatte F-B, Cichocki A (2008) Split-test Bonferroni correction for QEEG statistical maps. Biol Cybern 98(4):295–303PubMedCrossRef

Vogel EK, McCollough AW, Machizawa MG (2005) Neural measures reveal individual differences in controlling access to working memory. Nature 438(7067):500–503PubMedCrossRef

Wilson GF, Russell CA (2003) Real-time assessment of mental workload using psychophysiological measures and artificial neural networks. Hum Factors J Hum Factors Ergon Soc 45(4):635–644CrossRef

Wolpaw JR, Birbaumer N, Heetderks WJ, McFarland DJ, Peckham PH, Schalk G, Donchin E, Quatrano LA, Robinson CJ, Vaughan TM et al (2000) Brain–computer interface technology: a review of the first international meeting. IEEE Trans Rehabil Eng 8(2):164–173PubMedCrossRef

Zander TO, Kothe C (2011) Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general. J Neural Eng 8(2):025005PubMedCrossRef

Zarjam P, Epps J, Chen F (2011) Characterizing working memory load using EEG delta activity. In: Signal processing conference, 2011 19th European. IEEE, pp 1554–1558

Zeng H, Yang C, Dai G, Qin F, Zhang J, Kong W (2018) EEG classification of driver mental states by deep learning. Cogn Neurodyn 12(6):597–606PubMedPubMedCentralCrossRef

Title: A brain–computer interface for the continuous, real-time monitoring of working memory load in real-world environments
Authors: Aldo Mora-Sánchez
Alfredo-Aram Pulini
Antoine Gaume
Gérard Dreyfus
François-Benoît Vialatte
Publication date: 09-03-2020
Publisher: Springer Netherlands
Published in: Cognitive Neurodynamics / Issue 3/2020
Print ISSN: 1871-4080
Electronic ISSN: 1871-4099
DOI: https://doi.org/10.1007/s11571-020-09573-x

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 3/2020

Effects of synaptic integration on the dynamics and computational performance of spiking neural network

Prediction of numbers of the accumulative confirmed patients (NACP) and the plateau phase of 2019-nCoV in China

Machine classification of spatiotemporal patterns: automated parameter search in a rebounding spiking network

Various firing activities and finite-time synchronization of an improved Hindmarsh–Rose neuron model under electric field effect

An adaptive decoder design based on the receding horizon optimization in BMI system

Intranasal insulin administration may be highly effective in improving cognitive function in mice with cognitive dysfunction by reversing brain insulin resistance