Skip to main content
SpringerLink
Log in

Brain–computer interfacing: more than the sum of its parts

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

The performance of non-invasive electroencephalogram-based (EEG) brain–computer interfaces (BCIs) has improved significantly in recent years. However, remaining challenges include the non-stationarity and the low signal-to-noise ratio of the EEG, which limit the bandwidth and hence the available applications. Optimization of both individual components of BCIs and the interrelationship between them is crucial to enhance bandwidth. In other words, neuroscientific knowledge and machine learning need to be optimized by considering concepts from human–computer interaction research and usability. In this paper, we present results of ongoing relevant research in our lab that addresses several important issues for BCIs based on the detection of transient changes in oscillatory EEG activity. First, we report on the long-term stability and robustness of detection of oscillatory EEG components modulated by distinct mental tasks, and show that the use of mental task pairs “mental subtraction versus motor imagery” achieves robust and reliable performance (Cohen’s κ > 0.6) in seven out of nine subjects over a period of 4 days. Second, we report on restricted Boltzmann machines (RBMs) as promising tools for the recognition of oscillatory EEG patterns. In an off-line BCI simulation we computed average peak accuracies, averaged over ten subjects, of 80.8 ± 7.2 %. Third, we present the basic framework of the context-aware hybrid Graz-BCI that allows interacting with the massive multiplayer online role playing game World of Warcraft. We show how a more integrated design approach that considers all components of BCIs, their interrelationships, other input signals and contextual information can increase interaction efficacy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Allison BZ, Brunner C, Kaiser V, Müller-Putz GR, Neuper C, Pfurtscheller G (2010) Toward a hybrid brain–computer interface based on imagined movement and visual attention. J Neural Eng 7:026,007. doi:10.1088/1741-2560/7/2/026007

    Article  Google Scholar 

  • Allison BZ, Leeb R, Brunner C, Müller-Putz GR, Bauernfeind G, Kelly JW, Neuper C (2012) Toward smarter BCIs: extending BCIs through hybridization and intelligent control. J Neural Eng 9(1):013001. doi:10.1088/1741-2560/9/1/013001

    Google Scholar 

  • Balderas D, Zander T, Bachl F, Neuper C, Scherer R (2011) Restricted boltzmann machines as useful tool for detecting oscillatory eeg components. In: Proc. of the 5th international brain–computer interface conference, Graz, Austria, pp 68–71

  • Blankertz B, Dornhege G, Krauledat M, Müller KR, Curio G (2007) The non-invasive Berlin brain–computer interface: fast acquisition of effective performance in untrained subjects. NeuroImage 37:539–550. doi:10.1016/j.neuroimage.2007.01.051

    Article  Google Scholar 

  • Blankertz B, Tomioka R, Lemm S, Kawanabe M, Müller KR (2008) Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Process Mag 25:41–56. doi:10.1109/MSP.2008.4408441

    Article  Google Scholar 

  • Carreira-Perpinan MA, Hinton GE (2005) On contrastive divergence learning. In: Proceedings of the tenth international workshop on artificial intelligence and statistics, pp 33–40

  • Chung M, Cheung W, Scherer R, Rao RPN (2011) A hierarchical architecture for adaptive brain–computer interfacing. In: Proceedings of the 22nd international joint conference on artificial intelligence, IJCAI 2011, Barcelona, Catalonia, Spain, July 16–22, 2011 pp 1647–1652

  • Cohen J (1960) A coefficient of agreement for nominal scales. Psychol Meas 20:37–46

    Article  Google Scholar 

  • Darvas F, Scherer R, Ojemann JG, Rao RP, Miller KJ, Sorensen LB (2009) High gamma mapping using EEG. NeuroImage 49:930–938. doi:10.1016/j.neuroimage.2009.08.041

    Article  Google Scholar 

  • Fatourechi M, Bashashati A, Ward RK, Birch GE (2007) EMG and EOG artifacts in brain computer interface systems: a survey. Clin Neurophysiol 118:480–494

    Article  Google Scholar 

  • Fazli S, Mehnert J, Steinbrink J, Curio G, Villringer A, Müller KR, Blankertz B (2012) Enhanced performance by a hybrid NIRSEEG brain computer interface. NeuroImage 59:519–529. doi:10.1016/j.neuroimage.2011.07.084

    Article  Google Scholar 

  • Friedrich EVC, Scherer R, Neuper C (2012) The effect of distinct mental strategies on classification performance for brain–computer interfaces. Int J Psychophysiol 84(1):86–94

    Google Scholar 

  • Gevins A, Smith ME, McEvoy L, Yu D (1997) High-resolution eeg mapping of cortical activation related to working memory: effects of task difficulty, type of processing, and practice. Cereb Cortex 7(4):374–385

    Article  Google Scholar 

  • Grozea C, Voinescu CD, Fazli S (2011) Bristle-sensors—low-cost flexible passive dry EEG electrodes for neurofeedback and BCI applications. J Neural Eng 8:025,008. doi:10.1088/1741-2560/8/2/025008

    Article  Google Scholar 

  • Hinton GE (2002) Training products of experts by minimizing contrastive divergence. Neural Comput 14(8):1771–1800

    Google Scholar 

  • Hinton GE (2010) A practical guide to training restricted boltzmann machines. http://www.cs.toronto.edu/hinton/absps/guideTR.pdf

  • Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18:1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  • Hjorth B (1975) An on-line transformation of EEG scalp potentials into orthogonal source derivations. Electroencephalogr Clin Neurophysiol 39:526–530. doi:10.1016/0013-4694(75)90056-5

    Article  Google Scholar 

  • Johnson RR, Popovic DP, Olmstead RE, Stikic M, Levendowski DJ, Berka C (2011) Drowsiness/alertness algorithm development and validation using synchronized eeg and cognitive performance to individualize a generalized model. Biol Psychol 87(2):241–250. doi:10.1016/j.biopsycho.2011.03.003

    Google Scholar 

  • Krauledat M, Tangermann M, Blankertz B, Müller KR (2008) Towards zero training for brain–computer interfacing. PLoS ONE 3:e2967. doi:10.1371/journal.pone.0002967

    Article  Google Scholar 

  • Larochelle H, Bengio Y (2008) Classification using discriminative restricted boltzmann machines. In: ICML 08: proceedings of the 25th international conference on machine learning. ACM

  • Leeb R, Friedman D, Müller-Putz GR, Scherer R, Slater M, Pfurtscheller G (2007) Self-paced (asynchronous) BCI control of a wheelchair in virtual environments: a case study with a tetraplegics. Comput Intell Neurosci 2007:79,642. doi:10.1155/2007/79642

    Article  Google Scholar 

  • Leeb R, Sagha H, Chavarriaga R, Milln JDR (2011) A hybrid brain–computer interface based on the fusion of electroencephalographic and electromyographic activities. J Neural Eng 8(2):025011. doi:10.1088/1741-2560/8/2/025011

    Google Scholar 

  • Lotte F, Congedo M, Lcuyer A, Lamarche F, Arnaldi B (2007) A review of classification algorithms for EEG-based brain–computer interfaces. J Neural Eng 4:R1–R13. doi:10.1088/1741-2560/4/2/R01

    Article  Google Scholar 

  • Mason SG, Bashashati A, Fatourechi M, Navarro KF, Birch GE (2007) A comprehensive survey of brain interface technology designs. Ann Biomed Eng 35:137–169. doi:10.1007/s10439-006-9170-0

    Article  Google Scholar 

  • Millán J, Mourino J (2003) Asynchronous BCI and local neural classifiers: an overview of the adaptive brain interface project. IEEE Trans Neural Syst Rehabil Eng 11:159–161

    Article  Google Scholar 

  • Müller-Gerking J, Pfurtscheller G, Flyvbjerg H (1999) Designing optimal spatial filters for single-trial EEG classification in a movement task. Clin Neurophysiol 110:787–798

    Article  Google Scholar 

  • Müller-Putz GR, Eder E, Wriessnegger SC, Pfurtscheller G (2008) Comparison of DFT and lock-in amplifier features and search for optimal electrode positions in SSVEP-based BCI. J Neurosci Methods 168:174–181. doi:10.1016/j.jneumeth.2007.09.024

    Article  Google Scholar 

  • Müller-Putz GR, Scherer R, Pfurtscheller G, Neuper C (2010) Temporal coding of brain patterns for direct limb control in humans. Front Neurosci 4. doi:10.3389/fnins.2010.00034

  • Navarro NA, Ceccaroni L, Velickovski F, Torrellas S, Miralles F, Allison BZ, Scherer R, Faller J (2011) Context-awareness as an enhancement of brain–computer interfaces. In: International workshop on ambient assisted living, Malaga, Spain. Lecture notes in computer science, pp 216–223

  • Navdeep J, Hinton GE (2011) Learning a better representation of speech sound waves using restricted boltzmann machines. In: Proceedings of the 12th international conference on artificial intelligence and statistics (AISTATS)

  • Neuper C, Scherer R, Reiner M, Pfurtscheller G (2005) Imagery of motor actions: differential effects of kinesthetic versus visual-motor mode of imagery on single-trial EEG. Brain Res Cognit Brain Res 25:668–677

    Article  Google Scholar 

  • Neuper C, Scherer R, Wriessnegger S, Pfurtscheller G (2009) Motor imagery and action observation: modulation of sensorimotor brain rhythms during mental control of a brain–computer interface. Clin Neurophysiol 120:239–247. doi:10.1016/j.clinph.2008.11.015

    Article  Google Scholar 

  • Pfurtscheller G, Lopes da Silva FH (1999) Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol 110:1842–1857. doi:10.1016/S1388-2457(99)00141-8

    Article  Google Scholar 

  • Pfurtscheller G, Scherer R, Müller-Putz GR, Lopes da Silva FH (2008) Short-lived brain state after cued motor imagery in naive subjects. Eur J Neurosci 28:1419–1426. doi:10.1111/j.1460-9568.2008.06441.x

    Article  Google Scholar 

  • Pfurtscheller G, Allison BZ, Brunner C, Bauernfeind G, Solis-Escalante T, Scherer R, Zander TO, Müller-Putz G, Neuper C, Birbaumer N (2010) The hybrid BCI. Front Neurosci 4:30. doi:10.3389/fnpro.2010.00003

    Google Scholar 

  • Popescu F, Fazli S, Badower Y, Blankertz B, Müller KR (2007) Single trial classification of motor imagination using 6 dry EEG electrodes. PLoS ONE 2:e637. doi:10.1371/journal.pone.0000637

    Article  Google Scholar 

  • Pregenzer M, Pfurtscheller G (1999) Frequency component selection for an EEG-based brain to computer interface. IEEE Trans Neural Syst Rehabil Eng 7:413–419

    Article  Google Scholar 

  • Pregenzer M, Pfurtscheller G, Flotzinger D (1996) Automated feature selection with a distinction sensitive learning vector quantizer. Neurocomputing 11:19–29. doi:10.1016/0925-2312(94)00071-9

    Article  Google Scholar 

  • Ramoser H, Müller-Gerking J, Pfurtscheller G (2000) Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans Rehabil Eng 8:441–446

    Article  Google Scholar 

  • Rao RPN, Scherer R (2010) Brain–computer interfacing. IEEE Signal Proc Mag 27(4):150–152

    Article  Google Scholar 

  • Scherer R, Müller GR, Neuper C, Graimann B, Pfurtscheller G (2004) An asynchronously controlled EEG-based virtual keyboard: improvement of the spelling rate. IEEE Trans Neural Syst Rehabil Eng 51:979–984

    Google Scholar 

  • Scherer R, Schlögl A, Lee F, Bischof H, Jansa J, Pfurtscheller G (2007) The self-paced Graz brain–computer interface: methods and applications. Comput Intell Neurosci 2007:79,826

    Article  Google Scholar 

  • Scherer R, Lee F, Schlögl A, Leeb R, Bischof H, Pfurtscheller G (2008) Toward self-paced brain–computer communication: navigation through virtual worlds. IEEE Trans Biomed Eng 55:675–682. doi:10.1109/TBME.2007.903709

    Article  Google Scholar 

  • Scherer R, Pfurtscheller G, Neuper C (2008) Motor imagery induced changes in oscillatory ee components: speed vs. accuracy. In: Proceedings of the Graz brain–computer interface workshop 2008

  • Scherer R, Müller-Putz GR, Pfurtscheller G (2009) Flexibility and practicality: Graz brain–computer interface approach. Int Rev Neurobiol 86:119–131. doi:10.1016/S0074-7742(09)86009-1

    Article  Google Scholar 

  • Scherer R, Friedrich EVC, Allison BZ, Pröll M, Chung M, Cheung W, Rao RPN, Neuper C (2011) Non-invasive brain–computer interfaces: enhanced gaming and robotic control. Lect Notes Comput Sci 6691:362–369

    Article  Google Scholar 

  • Schlögl A, Kronegg J, Huggins JE, Mason SG (2007) Evaluation criteria for BCI research. In: Toward brain–computer interfacing. MIT Press, Cambridge

  • Shenoy P, Krauledat M, Blankertz B, Rao RP, Müller KR (2006) Towards adaptive classification for BCI. J Neural Eng 3:13–23

    Article  Google Scholar 

  • Trejo L, Kochavi R, Kubitz K, Montgomery L, Rosipal R, Matthews B (2005) EEG-based estimation of cognitive fatigue. In: SPIE conference proceedings

  • Usakli AB, Gurkan S, Aloise F, Vecchiato G, Babiloni F (2009) A hybrid platform based on EOG and EEG signals to restore communication for patients afflicted with progressive motor neuron diseases. In: Conference proceedings of the IEEE engineering in medicine and biology society

  • Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM (2002) Brain–computer interfaces for communication and control. Clin Neurophysiol 113:767–791. doi:10.1016/S1388-2457(02)00057-3

    Article  Google Scholar 

  • Zander TO, Jatzev S (2012) Context-aware brain–computer interfaces: exploring the information space of user, technical system and environment. J Neural Eng 9(1):016003. doi:10.1088/1741-2560/9/1/016003

  • Zander TO, Kothe C (2011) Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general. J Neural Eng 8:025,005. doi:10.1088/1741-2560/8/2/025005

    Google Scholar 

Download references

Acknowledgments

This work was supported in part by the ICT Collaborative Project BrainAble (247447), the GaLA project (258169), the Wings for Life Spinal Cord Foundation, and ARO award W911NF-11-1-0307.

Author information

Authors and Affiliations

  1. Institute for Knowledge Discovery, Graz University of Technology, Krenngasse 37, 8010, Graz, Austria

    Reinhold Scherer, Josef Faller, David Balderas, Markus Pröll & Gernot Müller-Putz

  2. Department of Psychology, University of Graz, Universitätsplatz 2/III, 8010, Graz, Austria

    Elisabeth V. C. Friedrich

  3. Cognitive Science Department, University of California San Diego, 9500 Gilman Drive, 92122, La Jolla, CA, USA

    Brendan Allison

Authors
  1. Reinhold Scherer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Josef Faller
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Balderas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Elisabeth V. C. Friedrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Markus Pröll
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Brendan Allison
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Gernot Müller-Putz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Reinhold Scherer.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Scherer, R., Faller, J., Balderas, D. et al. Brain–computer interfacing: more than the sum of its parts. Soft Comput 17, 317–331 (2013). https://doi.org/10.1007/s00500-012-0895-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-012-0895-4

Keywords

Search

Navigation