Skip to main content

Über dieses Buch

For generations, humans have fantasized about the ability to create devices that can see into a person’s mind and thoughts, or to communicate and interact with machines through thought alone. Such ideas have long captured the imagination of humankind in the form of ancient myths and modern science fiction stories. Recent advances in cognitive neuroscience and brain imaging technologies have started to turn these myths into a reality, and are providing us with the ability to interface directly with the human brain. This ability is made possible through the use of sensors that monitor physical processes within the brain which correspond with certain forms of thought. Brain-Computer Interfaces: Applying our Minds to Human-Computer Interaction broadly surveys research in the Brain-Computer Interface domain. More specifically, each chapter articulates some of the challenges and opportunities for using brain sensing in Human-Computer Interaction work, as well as applying Human-Computer Interaction solutions to brain sensing work. For researchers with little or no expertise in neuroscience or brain sensing, the book provides background information to equip them to not only appreciate the state-of-the-art, but also ideally to engage in novel research. For expert Brain-Computer Interface researchers, the book introduces ideas that can help in the quest to interpret intentional brain control and develop the ultimate input device. It challenges researchers to further explore passive brain sensing to evaluate interfaces and feed into adaptive computing systems. Most importantly, the book will connect multiple communities allowing research to leverage their work and expertise and blaze into the future.



Overview and Techniques


Chapter 1. Brain-Computer Interfaces and Human-Computer Interaction

Advances in cognitive neuroscience and brain imaging technologies have started to provide us with the ability to interface directly with the human brain. This ability is made possible through the use of sensors that can monitor some of the physical processes that occur within the brain that correspond with certain forms of thought. Researchers have used these technologies to build brain-computer interfaces (BCIs), communication systems that do not depend on the brain’s normal output pathways of peripheral nerves and muscles. In these systems, users explicitly manipulate their brain activity instead of using motor movements to produce signals that can be used to control computers or communication devices.
Human-Computer Interaction (HCI) researchers explore possibilities that allow computers to use as many sensory channels as possible. Additionally, researchers have started to consider implicit forms of input, that is, input that is not explicitly performed to direct a computer to do something. Researchers attempt to infer information about user state and intent by observing their physiology, behavior, or the environment in which they operate. Using this information, systems can dynamically adapt themselves in order to support the user in the task at hand.
BCIs are now mature enough that HCI researchers must add them to their tool belt when designing novel input techniques. In this introductory chapter to the book we present the novice reader with an overview of relevant aspects of BCI and HCI, so that hopefully they are inspired by the opportunities that remain.
Desney Tan, Anton Nijholt

Chapter 2. Neural Control Interfaces

The control interface is the primary component of a Brain-Computer Interface (BCI) system that provides user interaction. The control interface supplies cues for performing mental tasks, reports system status and task feedback, and often displays representations of the user’s brain signals. Control interfaces play a significant role in determining the usability of a BCI, and some of the traditional human-computer interaction design methods apply. However, the very specialized input methods and display paradigms of a BCI require consideration to create optimal usability for a BCI system. This chapter outlines some of the issues and challenges that make designing control interfaces for BCIs unique.
Melody Moore Jackson, Rudolph Mappus

Chapter 3. Could Anyone Use a BCI?

Brain-computer interface (BCI) systems can provide communication and control for many users, but not all users. This problem exists across different BCI approaches; a “universal” BCI that works for everyone has never been developed. Instead, about 20% of subjects are not proficient with a typical BCI system. Some groups have called this phenomenon “BCI illiteracy”. Some possible solutions have been explored, such as improved signal processing, training, and new tasks or instructions. These approaches have not resulted in a BCI that works for all users, probably because a small minority of users cannot produce detectable patterns of brain activity necessary to a particular BCI approach. We also discuss an underappreciated solution: switching to a different BCI approach. While the term “BCI illiteracy” elicits interesting comparisons between BCIs and natural languages, many issues are unclear. For example, comparisons across different studies have been problematic since different groups use different performance thresholds, and do not account for key factors such as the number of trials or size of the BCI’s alphabet. We also discuss challenges inherent in establishing widely used terms, definitions, and measurement approaches to facilitate discussions and comparisons among different groups.
Brendan Z. Allison, Christa Neuper

Chapter 4. Using Rest Class and Control Paradigms for Brain Computer Interfacing

The use of Electroencephalography (EEG) for Brain Computer Interface (BCI) provides a cost-efficient, safe, portable and easy to use BCI for both healthy users and the disabled. This chapter will first briefly review some of the current challenges in BCI research and then discuss two of them in more detail, namely modeling the “no command” (rest) state and the use of control paradigms in BCI. For effective prosthetic control of a BCI system or when employing BCI as an additional control-channel for gaming or other generic man machine interfacing, a user should not be required to be continuously in an active state, as is current practice. In our approach, the signals are first transduced by computing Gaussian probability distributions of signal features for each mental state, then a prior distribution of idle-state is inferred and subsequently adapted during use of the BCI. We furthermore investigate the effectiveness of introducing an intermediary state between state probabilities and interface command, driven by a dynamic control law, and outline the strategies used by two subjects to achieve idle state BCI control.
Siamac Fazli, Márton Danóczy, Florin Popescu, Benjamin Blankertz, Klaus-Robert Müller

Chapter 5. EEG-Based Navigation from a Human Factors Perspective

In this chapter we discuss Brain-Computer Interfaces (BCIs) as navigation devices from a Human Factors point of view. We argue that navigation is more than only steering a car or a wheelchair. It involves three levels: planning, steering and control, linked to cognition, perception and sensation, respectively. We structure the existing BCIs along those three levels. Most existing BCIs focus on the steering level of navigation. This is a remarkable observation from a Human Factors perspective because steering requires a very specific subclass of control devices that have a high bandwidth and a very low latency like joysticks or steering wheels; requirements that can not be met with current BCIs. We recommend exploring the potential of BCIs for the planning level, e.g. to select a route, and for the control level, e.g. based on possible collision-related potentials.
Marieke E. Thurlings, Jan B. F. van Erp, Anne-Marie Brouwer, Peter J. Werkhoven



Chapter 6. Applications for Brain-Computer Interfaces

Brain-computer Interfaces (BCIs) have been studied for nearly thirty years, with the primary motivation of providing assistive technologies for people with very severe motor disabilities. The slow speeds, high error rate, susceptibility to artifact, and complexity of BCI systems have been challenges for implementing workable real-world systems. However, recent advances in computing and bio-sensing technologies have improved the outlook for BCI applications, making them promising not only as assistive technologies but also for mainstream applications. This chapter presents a survey of applications for BCI systems, both historical and recent, in order to characterize the broad range of possibilities for neural control.
Melody Moore Jackson, Rudolph Mappus

Chapter 7. Direct Neural Control of Anatomically Correct Robotic Hands

This chapter presents a potential method of achieving dexterous control of a prosthetic hand using a brain-computer interface (BCI). Major control successes with invasive BCIs have been achieved by recording the activity of small populations of neurons in motor areas of the cortex. Even the activity of single neurons can be used to directly control computer cursors or muscle stimulators. The combination of this direct neural control with anthropomorphic hand prostheses has great promise for the restoration of dexterity. Based on users’ requirements for a functional hand prosthesis, a fully anthropomorphic robot hand is required. Recent work in our laboratories has developed two new technologies, the Neurochip and the Anatomically Correct Testbed (ACT) Hand. These technologies are described and some examples of their performance are given. We conclude by describing the advantages of merging these approaches, with the goal of achieving dexterous control of a prosthetic hand.
Alik S. Widge, Chet T. Moritz, Yoky Matsuoka

Chapter 8. Functional Near-Infrared Sensing (fNIR) and Environmental Control Applications

Functional near-infrared (fNIR) sensing is a relatively young brain imaging technique, yet one that holds great promise for brain-computer interfaces. Measuring essentially the same signals as functional magnetic resonance imaging (fMRI), fNIR acts as a single-point monitor of oxy- and deoxy-hemoglobin concentrations for localized sensing with greatly lowered costs and hardware requirements. As an optical sensing technique, fNIR is more robust to ambient electrical noise that affects the electroencephalogram (EEG) signal. The reduced hardware requirements and robustness in noisy environments make fNIR well-suited for brain-computer interface systems as it poses few physical restrictions on the operator and can be implemented in a wide range of applications and scenarios.
Erin M. Nishimura, Evan D. Rapoport, Peter M. Wubbels, Traci H. Downs, J. Hunter Downs

Chapter 9. Cortically-Coupled Computer Vision

We have developed EEG-based BCI systems which couple human vision and computer vision for speeding the search of large images and image/video databases. We term these types of BCI systems “cortically-coupled computer vision” (C3Vision). C3Vision exploits (1) the ability of the human visual system to get the “gist” of a scene with brief (10’s–100’s of ms) and rapid serial (10 Hz) image presentations and (2) our ability to decode from the EEG whether, based on the gist, the scene is relevant, informative and/or grabs the user’s attention. In this chapter we describe two system architectures for C3Vision that we have developed. The systems are designed to leverage the relative advantages, in both speed and recognition capabilities, of human and computer, with brain signals serving as the medium of communication of the user’s intentions and cognitive state.
Paul Sajda, Eric Pohlmeyer, Jun Wang, Barbara Hanna, Lucas C. Parra, Shih-Fu Chang

Chapter 10. Brain-Computer Interfacing and Games

Recently research into Brain-Computer Interfacing (BCI) applications for healthy users, such as games, has been initiated. But why would a healthy person use a still-unproven technology such as BCI for game interaction? BCI provides a combination of information and features that no other input modality can offer. But for general acceptance of this technology, usability and user experience will need to be taken into account when designing such systems. Therefore, this chapter gives an overview of the state of the art of BCI in games and discusses the consequences of applying knowledge from Human-Computer Interaction (HCI) to the design of BCI for games. The integration of HCI with BCI is illustrated by research examples and showcases, intended to take this promising technology out of the lab. Future research needs to move beyond feasibility tests, to prove that BCI is also applicable in realistic, real-world settings.
Danny Plass-Oude Bos, Boris Reuderink, Bram van de Laar, Hayrettin Gürkök, Christian Mühl, Mannes Poel, Anton Nijholt, Dirk Heylen

Brain Sensing in Adaptive User Interfaces


Chapter 11. Enhancing Human-Computer Interaction with Input from Active and Passive Brain-Computer Interfaces

This chapter introduces a formal categorization of BCIs, according to their key characteristics within HCI scenarios. This comprises classical approaches, which we group into active and reactive BCIs, and the new group of passive BCIs. Passive BCIs provide easily applicable and yet efficient interaction channels carrying information on covert aspects of user state, while adding little further usage cost. All of these systems can also be set up as hybrid BCIs, by incorporating information from outside the brain to make predictions, allowing for enhanced robustness over conventional approaches. With these properties, passive and hybrid BCIs are particularly useful in HCI. When any BCI is transferred from the laboratory to real-world situations, one faces new types of problems resulting from uncontrolled environmental factors—mostly leading to artifacts contaminating data and results. The handling of these situations is treated in a brief review of training and calibration strategies. The presented theory is then underpinned by two concrete examples. First, a combination of Event Related Desynchronization (ERD)-based active BCI with gaze control, defining a hybrid BCI as solution for the midas touch problem. And second, a passive BCI based on human error processing, leading to new forms of automated adaptation in HCI. This is in line with the results from other recent studies of passive BCI technology and shows the broad potential of this approach.
Thorsten O. Zander, Christian Kothe, Sabine Jatzev, Matti Gaertner

Chapter 12. Brain-Based Indices for User System Symbiosis

The future generation user system interfaces need to be user-centric which goes beyond user-friendly and includes understanding and anticipating user intentions. We introduce the concept of operator models, their role in implementing user-system symbiosis, and the usefulness of brain-based indices on for instance effort, vigilance, workload and engagement to continuously update the operator model. Currently, the best understood parameters in the operator model are vigilance and workload. An overview of the currently employed brain-based indices showed that indices for the lower workload levels (often based on power in the alpha and theta band of the EEG) are quite reliable, but good indices for the higher workload spectrum are still missing. We argue that this is due to the complex situation when performance stays optimal despite increasing task demands because the operator invests more effort. We introduce a model based on perceptual control theory that provides insight into what happens in this situations and how this affects physiological and brain-based indices. We argue that a symbiotic system only needs to intervene directly in situations of under and overload, but not in a high workload situation. Here, the system must leave the option to adapt on a short notice exclusively to the operator. The system should lower task demands only in the long run to reduce the risk of fatigue or long recovery times. We end by indicating future operator model parameters that can be reflected by brain-based indices.
Jan B. F. van Erp, Hans (J. A. ) Veltman, Marc Grootjen

Chapter 13. From Brain Signals to Adaptive Interfaces: Using fNIRS in HCI

Functional near-infrared spectroscopy (fNIRS) is an emerging non-invasive, lightweight imaging tool which can measure blood oxygenation levels in the brain. In this chapter, we describe the fNIRS device and its potential within the realm of human-computer interaction (HCI). We discuss research that explores the kinds of states that can be measured with fNIRS, and we describe initial research and prototypes that can use this objective, real time information about users’ states as input to adaptive user interfaces.
Audrey Girouard, Erin Treacy Solovey, Leanne M. Hirshfield, Evan M. Peck, Krysta Chauncey, Angelo Sassaroli, Sergio Fantini, Robert J. K. Jacob



Chapter 14. MATLAB-Based Tools for BCI Research

We first discuss two MATLAB-centered solutions for real-time data streaming, the environments FieldTrip (Donders Institute, Nijmegen) and DataSuite (Data- River, Producer, MatRiver) (Swartz Center, La Jolla). We illustrate the relative simplicity of coding BCI feature extraction and classification under MATLAB (The Mathworks, Inc.) using a minimalist BCI example, and then describe BCILAB (Team PhyPa, Berlin), a new BCI package that uses the data structures and extends the capabilities of the widely used EEGLAB signal processing environment. We finally review the range of standalone and MATLAB-based software currently freely available to BCI researchers.
Arnaud Delorme, Christian Kothe, Andrey Vankov, Nima Bigdely-Shamlo, Robert Oostenveld, Thorsten O. Zander, Scott Makeig

Chapter 15. Using BCI2000 for HCI-Centered BCI Research

BCI2000 is a general-purpose software suite designed for brain-computer interface (BCI) and related research. BCI2000 has been in development since 2000 and is currently used in close to 500 laboratories around the world. BCI2000 can provide stimulus presentation while simultaneously recording brain signals and subject responses from a number of data acquisition and input devices, respectively. Furthermore, BCI2000 provides a number of services (such as a generic data format that can accommodate any hardware or experimental setup) that can greatly facilitate research. In summary, BCI2000 is ideally suited to support investigations in the area of human-computer interfaces (HCI), in particular those that include recording and processing of brain signals. This chapter provides an overview of the BCI2000 system, and gives examples of its utility for HCI research.
Adam Wilson, Gerwin Schalk


Weitere Informationen