Brain-Computer Interface Research

Brain-computer interface (BCI) research has been advancing quickly, and novel directions with both invasive and non-invasive BCIs could help new patient groups. Each year, the annual BCI Research Award recognizes the top projects in BCI research. This book includes chapters that review these different BCI projects, and this chapter presents more information about the award process in 2017 and the nominated projects.

RATIONALE: Electrocorticography-based functional language mapping (ECoG-FLM) utilizes an ECoG signal paired with simultaneous language task presentation to create functional maps of the eloquent language cortex in patients selected for resective epilepsy or tumor surgery. At present, the concordance of functional maps derived by ECoG-FLM and electrical cortical stimulation mapping (ESM) remains rather low. This impedes the transition of ECoG-FLM into an independent functional mapping modality. As ESM is considered the gold standard of functional mapping, we aimed to use it in combination with machine learning (ML) approaches (“ESM-ML guide”), to improve the accuracy of ECoG-FLM. METHODS: The ECoG data was collected from 6 patients (29.67 ± 12.5 yrs; 19–52 yrs; 3 males, 3 females). Patient ECoG activity was recorded (g.USBamp, g.tec, Austria) during administration of language tasks. For data analysis: (1) All ECoG sites were divided into ESM positive [ESM(+)] and ESM negative [ESM(−)]; (2) Features of ESM(+) and ESM(−) sites in the ECoG signal were determined by analyzing the signal in the frequency domain; (3) ML classifiers [Random Forest (RF) and Deep Learning (DL)] were trained to identify these features in language-related ECoG activity; (4) The accuracy of the ESM-ML guided classification was compared with the accuracy of the conventional ECoG-FLM. RESULTS: The conventional approach demonstrated: 58% accuracy, 22% sensitivity, and 78% specificity. The “ESM-ML guide” approach with RF classifier demonstrated: 76.2% accuracy, 73.6% sensitivity and 78.78% specificity. The DL classifier achieved the highest performances compared to all others with 83% accuracy, 84% sensitivity and 83% specificity. CONCLUSION: ECoG-FLM accuracy can be improved by using an “ESM-ML guide”, making the use of ECoG-FLM feasible as a stand-alone methodology. The long-term goal is to create a tool-box with “ready to use an ESM-ML guide” algorithm trained to provide high accuracy ECoG-FLM results by classifying between ESM(+) and ESM(−) contacts in prospective sets of language-related ECoG data and, thus, contribute towards improved surgical outcomes.

In real-life scenarios, outside of the laboratory setting, the performance of brain-computer interface (BCI) systems is influenced by the user’s mental state such as attentional diversion. Here, we propose a novel online BCI system able to adapt with variations in the users’ attention during real-time movement execution. Electroencephalography (EEG) signals were recorded from twelve channels in twelve healthy participants and two stroke patients while performing 50 trials of ankle dorsiflexion simultaneously with an auditory oddball task. For each participant, the selected channels, classifiers and features from the offline mode were used in the online mode to predict the attention status. For both healthy controls and subacute stroke patients, feedback to the user on attentional status reduced the amount of attentional diversion created by the oddball task. The findings presented here demonstrate that the users’ attention can be monitored in a fully online BCI system, and further, that real-time neurofeedback on the attentional state of the user can be implemented to focus the attention of the user back onto the main task of the BCI for neuromodulation. Monitoring the users’ attention status will have a major impact in the BCI for neurorehabilitation area in the future.

Phantom limb pain is neuropathic pain that occurs after the amputation of a limb and partial or complete deafferentation. The underlying cause has been attributed to maladaptive plasticity of the sensorimotor cortex, and evidence suggests that experimental induction of further reorganization should affect the pain. Here, we use a brain–computer interface (BCI) based on real-time magnetoencephalography signals to reconstruct affected hand movements with a robotic hand. BCI training successfully induced some plastic alteration in the sensorimotor representation of the phantom hand movements. If a patient tried to control the robotic hand by associating the representation of phantom hand movement, it increased the pain while improving classification accuracy of the phantom hand movements. However, if the patient tried to control the robotic hand by associating the representation of the intact hand, it decreased the pain while decreasing the classification accuracy of the phantom hand movements. These results demonstrate that the BCI training controls the phantom limb pain depending on the induced sensorimotor plasticity. Moreover, these results strongly suggest that a reorganization of the sensorimotor cortex is the underlying cause of phantom limb pain.

Controlling advanced robotic systems with brain signals promises substantial improvements in health care, for example, to restore intuitive control of hand movements after severe stroke or spinal cord injuries (SCI). However, such integrated, brain- or neural-controlled robotic systems have yet to enter broader clinical use or daily life environments. The main challenge to integrate such systems in everyday life environments relates to the reliability of brain-control, particularly when brain signals are recorded non-invasively. Using a non-invasive, hybrid EEG-EOG-based brain/neural hand exoskeleton (B/NHE), we demonstrate full restoration of activities of daily living (ADL), such as eating and drinking, across six paraplegic individuals (five males, 30 ± 14 years) outside the laboratory. In a second set of experiments, we show that even whole-arm exoskeleton control is feasible and safe by combining hybrid brain/neural control with vision-guided and context-sensitive autonomous robotics. Given that recent studies indicate neurological recovery after chronic stroke or SCI when brain-controlled assistive technology is repeatedly used for 1–12 months, we suggest that combining an assistive and rehabilitative approach may further promote brain-machine interface (BMI) technology as a standard therapy option after stroke and SCI. In such scenario, brain/neural-assistive technology would not only have an immediate impact on the quality of life and autonomy of individuals with brain or spinal cord lesions but would also foster neurological recovery by stimulating functional and structural neuroplasticity.

In the past, the decoding quality of brain-computer interface (BCI) systems was often enhanced by independently improving either the machine learning algorithms or the BCI paradigms. We propose to take a novel perspective instead by optimizing the whole system, paradigm and decoder, jointly. To exemplify this holistic idea, we introduce learning from label proportions (LLP) as a new classification approach and prove its value for visual event-related potential (ERP) signals of the EEG. LLP utilizes the existence of subgroups with different label proportions in the data. This leads to a conceptually simple BCI system which combines previously unseen capabilities: (1) it does not require calibration and learns from unlabeled data, (2) under i.i.d. conditions, LLP is guaranteed to obtain the optimal decoder for online data, (3) under violation of stationarity assumptions, LLP can continuously adapt to the changing data, and (4) it can, in practice, replace a traditional supervised decoder when combined with an expectation-maximization algorithm.

Impairment of shoulder elevation in poststroke hemiplegia is a debilitating condition with no evidence-based, accessible treatment. This study evaluated the safety and efficacy of direct brain control of advanced exoskeleton robotics as a physiotherapeutic intervention. Poststroke patients with severe chronic hemiplegia participated in a physiotherapeutic intervention with movement support aided by online decoding of contralesional primary motor cortex activity and exoskeleton shoulder robotics. Participants engaged in 1 h of daily exercise for 7 consecutive days, which promoted lateralized motor-related electroencephalogram (EEG) responses to the contralesional side and the appearance of a transcranial magnetic stimulation-evoked potential in the paralyzed shoulder muscle. Participants gained active range-of-motion in the affected arm, with a flexion movement beyond the standardized minimal clinically important difference. These results suggest that an EEG-based brain-machine interface could facilitate targeted up-conditioning of contralesional corticospinal pathways, resulting in the clinically relevant functional recovery of movement.

In this study, we evaluated the ability to identify individual words in a binary word classification task during imagined speech, using high frequency activity (HFA; 70–150 Hz) features in the time domain. For this, we used an imagined word repetition task cued with a word perception stimulus, and followed by an overt word repetition, and compared the results across the three conditions. We used support-vector machines, and introduced a non-linear time-realignment in the classification framework—in order to deal with speech temporal irregularities. As expected, high classification accuracy was obtained in the listening (mean = 89%) and overt speech conditions (mean = 86%), where speech stimuli were directly observed. In the imagined speech condition, where speech is generated internally by the patient, results show for the first time that individual words in single trials were classified with statistically significant accuracy. Classification accuracy reached 88% in a two-class classification framework, and average classification accuracy across fifteen word-pairs was significant across five subjects (mean = 58%). The majority of electrodes carrying discriminative information were located in the superior temporal gyrus, inferior frontal gyrus and sensorimotor cortex, regions commonly associated with speech processing. These data represent a proof of concept study for basic decoding of speech imagery, and delineate a number of key challenges to usage of speech imagery neural representations for clinical applications.

Brain-computer interfaces (BCIs) have been employed to provide different patient groups with communication and control that does not require the use of limbs that have been damaged. In this study, we explored BCI-based navigation in three long term amputees. Each participant attempted motor execution with the affected limb, and performed motor execution with the intact limb, while fMRI activity was recorded. Participants attempted, and executed, one of four tasks to direct the movement of an avatar on a monitor. Classification accuracy was very high across both cue-based and free-choice conditions. Results support the use of this fMRI BCI approach for virtual navigation, which could improve BCIs based on fMRI as well as other approaches such as EEG.

The sense of agency is the attribution of an action to ourselves, which allows us to distinguish our own actions from those of other people and gives us a feeling of control and responsibility for their outcomes. Under physiological conditions, the sense of agency typically accompanies all our actions. Further, it can even be experienced over an illusory owned body—that is, a surrogate body perceived as if it were our own. However, the extent to which actions controlled through a brain–computer interface (BCI) also induce feelings of agency and responsibility is not well known. In the following chapter, we will review the relevant literature on body ownership and agency in virtual reality (VR) embodiment and outline an experiment in which participants controlled a virtual body through different BCI protocols based either on sensorimotor activity or on visually evoked potentials. Our findings show that BCI protocols can induce feelings of agency and that those BCI protocols based on sensorimotor activity have an advantage over those based on activity in visual areas. We further show that BCI protocols based on sensorimotor activity can even induce feelings of responsibility over the outcomes of that action, a finding that raises important ethical implications. We give particular focus to subjective reports from the debriefing after the experiment about the experience of BCI-induced agency over the action of a virtual body.

This book reviews the Seventh Annual BCI Research Award, with chapters that review the most promising new BCI research. As with prior years, we announced the first, second, and third place winners as part of a major international BCI conference. The Gala Awards ceremony for the 2017 BCI Research Award was part of the Seventh International BCI Conference in Graz, Austria. This conference series occurs every two years, and we already plan to host the ceremony for the 2019 award with the Eighth International BCI Conference.