Surfing the internet with a BCI mouse

Brain–computer interfaces (BCIs) provide non-muscular communication and control by directly translating brain activities recorded from the scalp into computer control signals and thus enable users with motor disabilities to convey their intent to the external world (Pfurtscheller et al 2000, Wolpaw et al 2002). Significant progress, especially of non-invasive BCI, has been made in recent decades (see Birbaumer and Cohen (2007) and Wolpaw (2007) for comprehensive survey). Diverse types of real-world applications, such as word spelling (Farwell and Donchin 1988, Birbaumer et al 1999), environment control (Gao et al 2003), neural prothesis control (McFarland and Wolpaw 2008), wheelchair control (Galan et al 2008, Rebsamen et al 2010) and gaming (Lalor et al 2005), have been developed.

Currently, the internet has become an important part of our daily lives. Interactive access to the World Wide Web (WWW) is one of the most promising applications of BCI, since it enables the users with limited motor functions to get information (e.g. news) from the internet, do shopping and even run a business, etc. Indeed, a BCI-based browser can greatly improve the quality of lives for disabled people (Mugler et al 2010). In the past decade, several BCI browsers have been developed. For instance, Mankoff et al (2002) proposed a web browser working with four switch neural signals; Moore et al developed BCI browsers with slow cortical potentials (SCP) (Moore 2003, Tomori and Moore 2003); Karim et al (2006) proposed a BCI browser based on μ rhythm; Bensch et al (2007) introduced a web browser Nessi based on SCP. These browsers extract and divide the selectable targets in a web page into groups, and then use a linear tabbing method or a decision-tree method to select the target of interest. We call them group-based browsers in this paper. Later, Mugler et al (2008, 2010) extended browser Nessi using the P300 event-related potential as a control signal, which has the ability to select a target from multiple options. It maps the flashing buttons of a P300 stimulation matrix onto different targets, so that the P300 potential detected on a specified button will activate its corresponding target. This type of browser is called map-based here.

Note that most real web browsers and pages have been designed to work with a mouse. In web surfing, a mouse is commonly used to click on hyperlinks, scroll the pages and control navigation forward and backward in the history. Although all of these actions based on a mouse have keyboard counterparts (for example, tab, return, page up, page down, alt left and alt right, in Netscape^TM), they are much easier to do with a mouse (Mankoff et al 2002). BCI mouse emulation was introduced by McFarland et al (2008), based on μ and β rhythms. Citi et al (2008) also developed a BCI mouse with a P300-based design. However, to our knowledge, internet surfing controlled by BCI mouse has not yet appeared in the existing literature. Therefore, we design and implement a BCI mouse-controlled web browser in this study, which is different from the group-based and map-based methods. This browser is based on the two-dimensional cursor control and target selection methods in our previous studies (Li et al 2010, Long et al 2011). The BCI mouse inherits the move-and-click nature of a computer mouse. While a single-target condition has been considered in Li et al (2010) and Long et al (2011), here we focus on the emulation of a computer mouse in a rich-target environment. This is necessary because a web page usually contains tens or even hundreds of targets. These so-called targets can be classified as targets of interest and targets of no interest according to the user's intention. We first introduce a target filter to exclude most of the targets of no interest. Then through a series of mouse movements and selections/rejections, the intended target can be selected. Hence, the special requirements of mouse-based web browsing are accommodated.

The remaining part of this paper is organized as follows. The methodology and implementation for mouse control and web browsing are discussed in section 2. The experimental paradigms and results are presented in section 3. Further discussions on the system and results are involved in section 4. Finally, section 5 concludes the paper.

In this section, the system paradigm is first described. Next, the target-filtering technique is elaborated. Then, the 2D cursor control and target selection are summarized and extended to accommodate the multi-target environment. Finally, implementation of the web browser based on the BCI mouse is discussed in detail.

2.1. System paradigm

The system is composed of two components: an electroencephalographic (EEG) acquisition device and a computer system. The EEG device from Neuroscan captures 32-channel electric potentials from the scalp of the user wearing an EEG cap referenced to the right ear. The EEG signals are digitized at a sampling rate of 250 Hz. The EEG acquisition device and the computer system are connected via a USB port and a parallel port, with the USB port for EEG data transmission and the parallel port for synchronization. In our system, the two channels 'HEOG' and 'VEOG' recording eye movements are excluded, and the remaining 30 channels are used for signal processing without further channel selection. The distribution of these channels is depicted in figure 1 (Li et al 2010).

Zoom In Zoom Out Reset image size

Figure 2 shows the graphical user interface (GUI). There are eight buttons around the client area of the GUI, with three labeled 'UP' at the top, three labeled 'DOWN' at the bottom and two labeled 'STOP' at the left and right sides. During mouse control, these buttons flash in a random order to elicit P300 potentials when the user focuses attention on one of them. The web browser is embedded in the central client area of the GUI. The navigation bar on the top of the browser contains several navigation buttons: a 'Home' button that makes the browser return to the home page, a 'Reload' button that reloads the current page, a 'Backward' button and a 'Forward' button that show the previous page and the next page, respectively, an edit box for URL address input, a 'Go' button to access the web page with the given address and a filter edit box for filtering out targets of no interest.

Zoom In Zoom Out Reset image size

The user controls the vertical movement of the mouse by paying attention to a specified 'UP'/'DOWN' button. That is, he/she moves the mouse up/down by attending to one of the three 'UP'/'DOWN' buttons. If no vertical movement is needed, the user can focus on one of the two 'STOP' buttons. Meanwhile, the horizontal movement of the mouse is controlled by the user's motor imagery. Specifically, the user moves the mouse toward right by imagining moving his right hand and vice versa (Li et al 2010). In this way, the user can move the mouse into a target box that is a local area containing a target. For target selection or rejection, the motor imagery and P300 features work collaboratively as follows. If the target is an intended one, the user can make a selection by paying attention to the 'STOP' button on the left side and stopping motor imagery for a while (i.e. in a so-called idle state of motor imagery). Otherwise, if the target is of no interest, he/she can reject it by continuing motor imagery without paying attention to the 'STOP' button (i.e. in a so-called idle state of P300) (Long et al 2011).

2.2. Target filtering

A target filter located at the right end of the browser's navigation bar (see figure 2) is used to extract and keep those targets of interest while disabling the others. Regarding target selection with the BCI mouse in a rich-target web page, a preprocessing of target filtering is important to effectively save working time for selecting a target of interest through reducing the number of targets of no interest. Specifically, if a web page is found to contain tens or even hundreds of targets, the user can move the mouse toward the target filter and select it. Then a P300 spelling interface shown in figure 3 is activated and covers the browsing interface. Using the speller, the user inputs keywords and then focuses on the 'OK' button. Once a P300 potential is detected at the 'OK' button, the P300 spelling interface disappears and the browsing interface with these input keywords in the filter edit box is activated again. At the same time, the target filter highlights those targets in the loaded page with their text content matching the given keywords and disables those unmatched targets. By doing this, the number of targets is greatly reduced, usually from tens or hundreds to a small number (e.g. about 10 in our experiments). This makes the following target selection based on our BCI mouse feasible.

Zoom In Zoom Out Reset image size

2.3. Mouse control in a multi-target environment

In general, several targets will remain after the target filtering. In the following, we show the mouse control method for selecting one of interest from these targets.

2.3.1. Mouse movement control

The vertical movement and the horizontal movement of the mouse are controlled by P300 and motor imagery, respectively. In the following, we briefly describe this method, and the details can be found in Li et al (2010).

In our system, the position of the mouse is updated every 200 ms. Regarding the vertical movement control of the mouse, the velocity is fixed and the direction is controlled through P300 detection. For the kth update, if the P300 potential is detected by a SVM classifier at one of the three 'UP' buttons, the parameter c(k) that determines the direction of vertical movement is set to 1, and the mouse moves upward; if the P300 potential is detected at one of the three 'DOWN' buttons, the parameter c(k) is set to −1, and the mouse moves down; if the P300 potential is detected at one of the two 'STOP' buttons, the parameter c(k) is set to 0, and the mouse has no vertical movement; if no P300 potential is detected at any button, the parameter c(k) keeps its value, and the direction of the vertical movement of the mouse does not change. The details of the P300 detection algorithm can be found in Li et al (2010). The vertical coordinate of the mouse in the kth update is given by

$$\begin{equation} \fl y(k) = y(k-1)+c(k)v_0,\quad \ c(k) \in \lbrace -1,0,1\rbrace , \end{equation} \tag{ 1 }$$

where c(k), determined by P300 detection as above, is used to control the direction of vertical movement and v₀ is a speed constant which can be tuned according to the user's performance (v₀ was set to 10 pixels in our experiments).

For horizontal movement control in the kth position update of the mouse, an epoch of EEG data was first spatially filtered with a common average filter (CAR) and then band-pass filtered in 8–13 Hz. A common spatial pattern (CSP) transform matrix W prepared in a training session is applied to the filtered data to extract a feature vector (Ramoser et al 2000, Blankertz et al 2008, Li and Guan 2008). Next, a SVM classifier also prepared in the training session is applied to the feature vector, and the corresponding SVM score f(k) is obtained. Finally, the horizontal coordinate of the mouse in the kth update is given by

$$\begin{eqnarray} &&x(k) = x(k-1)+a_x[f(k-2) + f(k-1) + f(k)] + b_x, \nonumber \\ \end{eqnarray} \tag{ 2 }$$

where the parameters a_x and b_x are calibrated just before an online experiment such that the absolute value of the difference (x(k) − x(k − 1)) is close to zero when the subject is in an idle state of motor imagery (see Li et al (2010)).

For many subjects, simultaneously executing the dual tasks of P300 and motor imagery may deteriorate the performance of P300 or motor imagery because both tasks need attention. This problem has been considered in our system as explained below. Based on the above principle for setting the parameters a_x and b_x in model (2), the horizontal displacement of the cursor is small when the subject focuses on a flashing button without motor imagery. Furthermore, we set a threshold θ in our P300 detection as in Li et al (2010). The principle for setting the threshold is as follows: the output of the P300 detection algorithm can exceed the threshold when the subject pays attention to a flashing button; however, the threshold is difficult to exceed when the subject does not pay attention to any flashing button. If the output of the P300 detection algorithm does not exceed this threshold, the direction of the vertical movement of the cursor will not change. The two principles for setting parameters lead to the following control strategy especially for those subjects who find it hard to perform the P300 and motor imagery tasks simultaneously: the subjects can choose to focus on one of P300 and motor imagery tasks to separately control the vertical and horizontal movements of the cursor. For instance, the subjects can mainly pay attention to a flashing button to elicit P300 at the beginning of a trial or when they want to change the direction of the vertical movement of the cursor. In this case, the cursor is stable in the horizontal dimension. Once the vertical movement direction of the cursor is correct, the subjects mainly pay attention to motor imagery for control of horizontal movement, while the vertical movement direction of the cursor generally does not change because of the above threshold for P300 detection. Such parameter-setting methods enable P300 and motor imagery to work separately or simultaneously, depending on the control strategy of each subject. Therefore, P300 and motor imagery need not work simultaneously all the time in our system.

Using the above control method based on both the P300 potential and mu/beta rhythm, users can move the mouse from an arbitrary initial position to an arbitrary target position.

2.3.2. Target selection or rejection

We adopted a hybrid approach combining both the P300 potential and mu/beta rhythm for target selection with its details given in Long et al (2011). Specifically, if a target of interest is reached by the mouse, the user can select it by focusing on the left flashing button 'STOP' in the browser GUI and making no motor imagery. Otherwise the user can reject it by continuing motor imagery and paying no attention to the 'STOP' button. In other words, one state with a P300 potential detected at the left 'STOP' button and no motor imagery implies a selection of the currently reached target, and the other state with left/right motor imagery and no P300 potential detected at the left 'STOP' buttons means a rejection. Given an epoch of EEG signals, we extract both P300 feature and mu/beta rhythm feature, and then concatenate them to construct a hybrid feature vector. An SVM classifier, which is established by a training data set with label 1 implying selection and label −1 implying rejection, is applied to the hybrid feature vector. If the predicted label of the hybrid feature vector is 1, then the target is selected; otherwise the target is rejected. This hybrid approach provides not only high accuracy of selection, but also high detection speed, as validated in Long et al (2011).

2.3.3. Target selection in a multi-target condition

With the above mouse control method, a single move-and-click operation can be done. However, there are always multiple targets in a real-world web page, even after the target filtering mentioned before. In such a case, the user first moves the mouse toward the target of interest. During this movement, it is very likely for the mouse to traverse targets of no interest. Once the mouse reaches a target, it will stop moving for a while to wait for the decision of selection or rejection. If a target box of interest is reached, the user needs to make a selection. Otherwise, if a target box of no interest is reached, the user needs to reject it and continue moving the mouse. This procedure is repeated until the target of interest is selected.

2.4. Implementations

The BCI mouse-based web browser here is developed based on the standard HTML control within the Windows 32bit Platform Development Kit of the Microsoft Windows system. Some common functions, such as history navigation, URL address input, text input, page scrolling, HTML parser, etc, are automatically included in our browser. The detailed implementations include the following three steps.

• (1)  
  Initialization. An initial home page is defined as shown in figure 4, which contains eight hyperlinks to eight popular sites of search engine, news, email, video, shopping and social network services, respectively. In this home page, the distance between any two hyperlinks is large, and thus each hyperlink can be easily selected with our BCI mouse. Our browser is initialized at this home page with which users can open an interesting or commonly used site quickly.
• (2)  
  Target visualization. After a web page is fully loaded, all selectable targets except those dynamically created by scripts are extracted based on the page content. Because it is inconvenient for a BCI mouse to move onto and select items as small as hyperlinks, we place an enlarged translucent target box over each extracted target, which is used as an in-place marker (see figure 5). To avoid cluttering the contents of the web page, these target boxes are visible only when the mouse reaches them.
• (3)  
  Target filtering, mouse movement and target selection. If the loaded page contains tens or even more targets as in figure 5, the user needs to move the mouse to the target filter mentioned before, which is located in the upper-right corner of the browser interface, and select it. A P300 speller interface is then activated (see figure 3). By inputting keywords with the P300 speller and performing target filtering, the user can exclude most of the targets of no interest. Those remaining targets are highlighted with yellow (see figure 6). Next, the user moves the mouse toward the intended target and selects it. During this procedure, if the mouse reaches an unintended target, the user needs to reject it and continue moving the mouse. Once the intended target is selected, a web page of interest is opened as in figure 7. The user can keep the mouse in any blank area of the web page so that he/she has enough time to read the page content without worrying about any unintended activation of target.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Two online experiments were conducted in this study. One was for testing the BCI mouse by emulating a task of mouse movement and target selection in a multi-target environment. The other was for testing the BCI mouse-based web browser by asking users to perform a comprehensive task, i.e. carrying out most of the functions provided by the browser to open a real-world web page. Seven subjects from the local research unit, aged from 23 to 30, attended the online experiments. One of them became skilled in web surfing with the BCI mouse during the development of our system. Another three had experience in our early experiments of 2D cursor control (Li et al 2010), and the rest had little experience in BCI experiments.

Before the online experiments, three training data sets were collected from each subject to set parameters of three models as below. First, each subject attended a P300 training session of 20 trials with the GUI of the P300 speller (see figure 3). Specifically, in each trial, all 50 buttons flashed in a random order and each button flashed ten times. Simultaneously, the subject was instructed to focus his attention on a given button. Data were collected to construct a P300 classification model for text input and also for control of the mouse's vertical movement. Second, for each subject, data were collected in 60 trials of motor imagery to set model parameters for motor imagery detection. Each trial was composed of 4 s left- or right-hand motor imagery. The subject performed left- and right-hand motor imagery according to two types of cues, left and right arrows, respectively. Third, one data set was collected to construct a model for target selection. In this training session, there were three classes of trials performed by each subject. Each class contained 20 trials and each trial lasted 4 s. In each trial, a cue (left/right/upward arrow) appeared to instruct a task, while the eight buttons flashed in a random order (the buttons' positions can be referred to in figure 2). The P300 stimuli were synchronized with the appearance of a cue, i.e. the stimuli began once an arrow appeared and ended when it disappeared. When a left arrow/right arrow appeared, the subject imagined left-/right-hand motor without paying attention to any flashing button. On the other hand, the subject focused on the 'STOP' button without any motor imagery when an upward arrow appeared.

3.1. Experiment 1: testing the BCI mouse

In our previous studies (Li et al 2010, Long et al 2011), we presented 2D cursor control and target selection, respectively, in a single-target environment, which were based on the combination of P300 and motor imagery. Based on these methods, a BCI mouse was established here. In this experiment, the performance of the BCI mouse was assessed in a multi-target environment.

Figure 8 illustrates the GUI in this experiment. In each trial of this experiment, ten randomly positioned targets (solid circles) appeared in the client area. The red one was the intended target, while the green ones were the unintended targets. The initial position of the mouse was randomly given in each trial. The task for each subject was to move the mouse toward the intended target and select it with the above-mentioned mouse control method. Specifically, the subject was required to move the mouse toward the red target and make a selection once the mouse reached its target box. When a target of no interest (green one) lay on the way from the current mouse position to the intended target, the mouse might touch it. In this case, the subject should reject it by continuing his motor imagery. A trial was considered to be successful if the intended target was selected in 90 s, or failed if either a target of no interest was selected or the time limit of 90 s elapsed. This procedure was repeated 80 times for each subject.

Zoom In Zoom Out Reset image size

Table 1 shows the results of experiment 1, including the number of trials for each subject, the accuracy rates for successfully hitting and selecting the intended targets, the average time of mouse movement and the average time of selection or rejection decision. The average performance over all the subjects is illustrated in the last row. Additionally, figure 9 presents the trajectory of mouse movement from a sample trial in experiment 1, which shows that the user moved the mouse and passed through three targets of no interest, and finally reached and selected the intended one successfully.

Zoom In Zoom Out Reset image size

Table 1. Results of experiment 1. 'Accuracy': the ratio of successful trials over total 80 trials; 'Movement time': the average time of mouse movement; 'Selection time': the average time of selection or rejection decision.

Subject	No of trials	Accuracy (%)	Movement time (s)	Selection time (s)
S1	80	98.75	31.48	1.00
S2	80	92.50	34.58	1.02
S3	80	87.50	37.27	1.01
S4	80	95.00	32.13	1.64
S5	80	91.25	29.06	1.34
S6	80	92.50	32.80	1.37
S7	80	95.00	31.66	1.35
Average	80	93.21	32.71	1.25

Additionally, to check that target selection/rejection is actually driven by the hybrid feature, ERP waveforms and power spectra of the three discriminative channels are calculated using the calibration data for target selection. Figure 10(A) shows the EEG amplitudes of channel 'CPz' ('Oz') for subject S1 (S2), in which the red/blue (solid/dashed) curve implies that the subject selected/rejected a target. Furthermore, figure 10(B) shows three power spectra of the channels 'C3' and 'C4' for each of the two subjects. The red, blue and green (solid, dashed and dotted) curves were obtained when the subject performed left-hand motor imagery, right-hand motor imagery and was in the idle state of motor imagery, respectively. From figures 10(A) and 10(B), we can see that when the subject selected a target, a P300 potential was elicited (see the red solid curve in figure 10(A)), and no motor imagery was detected (see the green dotted curve in figure 10(B)). Contrarily, when he/she rejected a target, no P300 was elicited (see the blue dashed curve in figure 10(A)), but left-/right-hand motor imagery was detected (see the red solid or blue dashed curve in figure 10(B)). Figure 10(C) shows the topographies of two CSP filters, i.e. the first and the last rows of the CSP transformation matrix W. The CSP transformation matrix W is obtained from the calibration data set for CSP feature extraction. It follows from the topographies in figure 10(C) that channels 'C3' and 'C4' were associated with the left- and right-hand motor imagery in the experiment.

Zoom In Zoom Out Reset image size

3.2. Experiment 2: testing the BCI mouse-based browser

3.3. Workload evaluation

In experiment 1, all the subjects executed the same number of move-and-click tasks. Since the positions of the targets and the initial positions of the mouse were randomly given, the execution time of each trial varied. The average accuracy rate over the subjects for successfully selecting the intended target was 93.21%. The average time for mouse movement was 32.71 s, while the average time for selection decision making was 1.25 s (see table 1). In experiment 2, although the web page opening task including text input and multiple selections was quite complicated, all the subjects finished it on average in 247 s with few mistakes (see table 2). These results show that, using our BCI mouse-based browser, users can efficiently surf the internet with high accuracy.

The experimental results also show several benefits of our method and system, which are as follows.

• (1)  
  The access time and task complexity are not significantly increased with the number of targets. When there is a large number of targets, we perform target filtering and exclude most of the targets of no interest. The next task (series of mouse movement and target selection/rejection) is carried out within the environment of several targets (e.g. about ten in experiment 2 in this paper).
• (2)  
  In our system, we do not need an independent target list or a map between the output commands of the BCI system and the targets of a web page. Except text input for target filtering, all executions are based on a BCI mouse. This matches the mouse-based design of real-world web pages and is convenient for users.
• (3)  
  More importantly, there is no need to predefine a time interval in our system for reading a loaded page. If the mouse does not hit a target box, no selection of any target is carried out. Hence, the user can move the mouse to a non-target area (e.g. the margin between the page content and the edges of the browser) in order to fully access the content. If the user has no interest in the current page, he/she can move the mouse to the next target in order to open a new page.

In order to check whether P300 can be reliably detected when combined with motor imagery task (dual condition), a supplementary experiment for 2D cursor control is conducted. The GUI can be seen in the first subplot of figure 11 which has a dimension of 1166 pixels × 721 pixels. The ratio of the size of the cursor, the size of the target and the size of workspace is fixed to be 0.000 84:0.003:1. In each trial of this experiment, the target randomly appears at one of the four corners of the GUI, while the starting position of the cursor is fixed at the center of the GUI. The task for the subject is to move the cursor to the target through motor imagery and P300. During the cursor control, the subject is instructed to pay attention to the flashing button at the same corner with the target, e.g., subject keeps paying attention to the left 'UP' button when the target appears at the up-left corner (see the first subplot of figure 11). Seven subjects attended this experiment and each subject completed 40 trials. All the subjects succeeded in the control task with hit rate 100%. The last seven subplots in figure 11 show the trajectories of the cursor of the 40 trials for the seven subjects. From these subplots, we can see that subjects S1 and S4 presented better control performance than the others. This is because the two subjects had a lot of experience in using this system. The other subjects were less experienced so that in many cases, they chose the separate control strategy. For instance, the subjects might mainly pay attention to a flashing button to elicit P300 at the beginning of some trials. Once the vertical movement direction of the cursor was correct, the subjects turned to focus on motor imagery for the control of horizontal movement. In some other trials, the subjects might also focus attention on motor imagery at the beginning of the trials to move the cursor horizontally, and then shift their attention to the desired flashing button to control the vertical movement of the cursor.

Zoom In Zoom Out Reset image size

Using the data collected in this experiment involving dual tasks, we extract P300, and compare it with the P300 extracted from the training data collected when the odd ball task is presented in isolation (isolation condition). The P300 ERP waveforms were calculated using the data collected when the subjects were paying attention to a specific button. In figure 12, we present grand-average P300 ERP waveforms from three electrodes 'Fz', 'Cz' and 'Pz' for all subjects, where the two waveforms on each subplot were obtained by making a grand average of target and non-target epochs of raw EEG signal respectively from each subject. Specifically, the first row for each electrode shows ERP waveforms for the isolation condition, while the second row shows those for the dual condition, where the magenta/gray and black curves are for target and non-target flashing buttons, respectively. From figure 12, we can see that the P300 potentials can be elicited in both dual and isolation conditions, although different subjects show P300 at different electrode sites (e.g. subject S3 shows P300 mainly at electrode 'Pz', while subject S4 shows P300 mainly at electrode 'Cz'). There is no evidence showing that the P300 potentials have been deteriorated by the dual tasks, compared to those extracted in the isolation condition. This can be explained by the fact that the subjects often performed P300 task and motor imagery task separately in the dual condition, as described above.

Zoom In Zoom Out Reset image size

The scalp distributions of SVM weights for P300 detection for all subjects are also presented at the bottom of figure 12. These average weights are derived from the P300 calibration data (isolation condition) set as follows: in the calibration phase, we construct a P300 feature vector by concatenating the features from all 30 channels. Thus, the trained SVM classifier has a weight vector of which the entries can be partitioned into 30 subvectors corresponding to the 30 channels, respectively. The average weight corresponding to a channel is calculated by averaging all of the absolute entries of the subvector associated with this channel (Li et al 2010). The average weight corresponding to a channel reflects its importance in SVM classification/P300 detection (Rakotomamonjy and Guigue 2008, Sellers et al 2006). These scalp distributions of SVM weights presented at the bottom of figure 12 show those electrodes with large weights which play an important role in P300 detection. For instance, for subject S4, it follows from the scalp distributions of SVM weights that electrode 'Cz' has large weight. Meanwhile, this electrode shows significant P300 ERPs.

Although our experimental results show that free surfing the internet has been realized with our BCI mouse-based web browser, our system needs to be further improved and extended in the future. First, the browser itself can be extended to include several other functions such as accessibility to bookmarks, drop menus, list boxes, etc. Second, the algorithms for detecting the P300 potential and motor imagery can be further optimized to reduce the execution time of each task. Finally, besides P300 evoked potential and motor imagery, other modalities such as SSVEP can be incorporated into our system for more functions and better performance.

This study presented a BCI mouse-based web browser, which is a real-world application of BCI systems. Our major concern is target selection in a multi-target web page. We emulated the computer mouse using 2D cursor control and target selection methods, which were based on the combination of P300 evoked potential and mu/beta rhythm. Using this mouse, the user can select a target in a web page if the number of targets is not large. However, there may be a large number of targets in a real-world web page. In this case, a target filter based on a P300 speller was incorporated to exclude most of the targets of no interest. Online experimental results showed that users can surf the internet through our BCI mouse-based web browser and the effectiveness of our system was thus demonstrated.

This work was supported by the National Natural Science Foundation of China under grants 60825306, 91120305 and 61105121, and National High-tech R&D Program of China (863 Program) under grant 2012AA011601.