Frequency recognition in an SSVEP-based brain computer interface using empirical mode decomposition and refined generalized zero-crossing

https://doi.org/10.1016/j.jneumeth.2010.12.014Get rights and content

Abstract

This paper presents an empirical mode decomposition (EMD) and refined generalized zero crossing (rGZC) approach to achieve frequency recognition in steady-stated visual evoked potential (SSVEP)-based brain computer interfaces (BCIs). Six light emitting diode (LED) flickers with high flickering rates (30, 31, 32, 33, 34, and 35 Hz) functioned as visual stimulators to induce the subjects’ SSVEPs. EEG signals recorded in the Oz channel were segmented into data epochs (0.75 s). Each epoch was then decomposed into a series of oscillation components, representing fine-to-coarse information of the signal, called intrinsic mode functions (IMFs). The instantaneous frequencies in each IMF were calculated by refined generalized zero-crossing (rGZC). IMFs with mean instantaneous frequencies (f¯GZC) within 29.5 Hz and 35.5 Hz (i.e., 29.5f¯GZC35.5Hz) were designated as SSVEP-related IMFs. Due to the time-locked and phase-locked characteristics of SSVEP, the induced SSVEPs had the same frequency as the gazing visual stimulator. The LED flicker that contributed the majority of the frequency content in SSVEP-related IMFs was chosen as the gaze target. This study tests the proposed system in five male subjects (mean age = 25.4 ± 2.07 y/o). Each subject attempted to activate four virtual commands by inputting a sequence of cursor commands on an LCD screen. The average information transfer rate (ITR) and accuracy were 36.99 bits/min and 84.63%. This study demonstrates that EMD is capable of extracting SSVEP data in SSVEP-based BCI system.

Research highlights

▶ Using EMD to remove SSVEP unrelated noise. ▶ High-frequency induced SSVEPs avoid the low-frequency interferences of brain rhythm. ▶ High-frequency flickering design achieves a more comfortable visualization. ▶ Frequency recognition by refined generialized zero crossing.

Introduction

Patients with stroke trauma, poliomyelitis, amyotrophic lateral sclerosis (ALS), botulism, multiple sclerosis, and Guillain–Barré syndrome are suffering from motor disabilities, which can disrupt their communication with the external environment, resulted in so-called locked – in syndrome (Wolpaw et al., 2000). Researchers have developed brain computer interface (BCIs) to deal with this problem. A BCI allows patients to communicate by recognizing user's particular electroencephalography (EEG) pattern in performing elaborately designed tasks. Various BCI systems are designed based on different types of EEG signals, such as the Mu rhythm, slow cortical potential (SCP), P300, motor-related potential (MRP), and visual evoked potential (VEP) (Birbaumer et al., 1999, Donchin et al., 2000, Lee et al., 2005, Lee et al., 2006, Mason and Birch, 2000, Pfurtscheller et al., 2002, Sutter, 1992, Wang et al., 2006). The steady-state visual evoked potential (SSVEP) has attracted a lot of attention due to its phase-locked characteristic and excellent signal-to-noise (SNR) ratio, which has been implemented to achieve high information transfer rate (ITR) BCI systems.

SSVEP is an oscillatory activity that is synchronized and phase-locked to repetitive visual stimulation generated by the human visual cortex (Herrmann, 2001, Pastor et al., 2003). SSVEP can be elicited by visual stimuli with a flickering frequency ranging from 4 to 75 Hz. This type of stimulus is a powerful indicator in the diagnosis of visual pathway function, visual imperception in patients with cerebral lesions (Bodis-Wollner, 1972), loss of multifocal sensitivity in patients with multiple sclerosis (Kupersmith et al., 1984a, Kupersmith et al., 1984b), and neurological abnormalities in patients with schizophrenia (Krishnan et al., 2005) and other clinical diagnoses. Pastor et al. (2003) studied the relationship between visual stimulation and SSVEP-evoked amplitudes, showing that the amplitude of occipital SSVEPs peaks at ∼15 Hz, forms a lower plateau at 27 Hz, and declines further at higher frequencies (>30 Hz) (Herrmann, 2001, Pastor et al., 2003, Wang et al., 2006). Ding et al. (2006) demonstrated that a person's attention level modulates his/her SSVEP. Since the SSVEP depends directly on the stimulation frequency of visual flicker, user's attended target can be identified by analyzing the frequency contents in the induced SSVEP.

Several researchers have endeavored to develop SSVEP-based BCI. By tagging different flickers with distinct flickering frequencies, subjects can shift their gaze to their desired flickers. These gaze targets can then be identified using the Fourier spectrum of the measured SSVEP signals (Cheng et al., 2002, Kelly et al., 2005, Lalor et al., 2005, Muller-Putz et al., 2005, Muller-Putz et al., 2008, Srihari Mukesh et al., 2006). Middendorf et al. (2000) designed a flight simulator controlled by two flickering lights that controlled leftwards or rightwards movement with an accuracy of 92%. Cheng et al. (2002) implemented a SSVEP-based virtual keypad that achieved a mean information transfer rate (ITR) of 27.15 bits/min using twelve frequency-tagged flickering lights. Using two EEG electrodes positioned at the primary visual cortex, Lalor et al. (2005) developed a method allowing participants to interact with a computer game. Moreover, some visual BCIs have been developed independent of uers's eye gaze. Allison et al. (2008) investigated selective attention using overlapping stimulus to induce SSVEPs difference in an online control study. Zhang et al. (2009) also modulated the SSVEP amplitude and phase response by means of shifting covert attention on two sets of random dots with distinct colors, motion direction and flickering frequencies in the same visual field. Treder and Blankertz (2010) compare the performance of the Hex-o-Spell and matrix design using covert attention. Their results demonstrate that the Hex-o-Spell increase 50% than matrix design with covert attention.

The aforementioned SSVEP-based BCIs identify user's intended targets on calculated Fourier spectra. Nevertheless, the Fourier spectrum requires a time window, e.g., 1 or 2 s, for computation to achieve sufficient frequency resolution in identifying two distinct gaze targets. Data segment with insufficient length in Fourier spectrum computation usually results in reduction of frequency resolution, which can limit the number of available targets in SSVEP-based BCI. Besides, the lack of signal pre-processing for noise removal in some studies may inevitably usher task-related noise and lead to misleading results (Zhonglin et al., 2007). Since BCI performance depends on accuracy and speed, a reliable method for extracting SSVEPs and recognizing gaze targets in an appropriate data segments is crucial.

This paper presents empirical mode decomposition (EMD)-based approach to extract SSVEPs in EEG data segments. The experiments in this study use the proposed system to control six cursor functions (‘cursor left,’ ‘cursor up,’ ‘cursor right,’ ‘cursor down,’ ‘left click,’ and ‘right click’) on a personal computer. Huang et al. (1998a) proposed the EMD approach as an efficient method of analyzing nonlinear and non-stationary data. EMD decomposes a signal into a finite number of intrinsic mode functions (IMFs) by iteratively conducting a sifting process (Huang et al., 1998b, Huang et al., 1998c). EMD is a powerful data-driven approach for extracting meaningful stochastically modulated signals in many applications, including blood pressure measurement (Huang et al., 1998b), heart-rate monitoring and electrocardiogram (ECG) (Balocchi et al., 2004), pulmonary hypertension (Huang et al., 1998c), visual spatial attention (Liang et al., 2005) and brain–computer interfaces. This study uses the signal processing advantages of EMD in noise reduction. EMD decomposes each Oz EEG data segment into a series of IMFs. The instantaneous frequencies in each IMF were computed by refined generalized zero-crossing (rGZC) to identify SSVEP-related IMFs. The flicker that provided the majority of the frequency content of the SSVEP-related IMFs was the gaze target. The proposed BCI system achieves cursor control with acceptable accuracy and ITR. This system provides an efficient and reliable channel for patients with neuromuscular disabilities to communicate with the external environment.

Section snippets

System configuration

Fig. 1 presents a block diagram of the proposed SSVEP-based BCI cursor control system. This system includes six LED flickers, a liquid crystal display (LCD) monitor, an EEG acquisition system (bio-amplifier and A/D card), and a personal computer for data processing. SSVEPs evoked by the LED flickers, flickering at frequencies of f1 (30 Hz), f2 (31 Hz), f3 (32 Hz), f4 (33 Hz), f5 (34 Hz), and f6 (35 Hz), were used to control six cursor functions (←: ‘cursor left’, ↑: ‘cursor up’, →: ‘cursor right’, ↓:

Results

Fig. 4 shows an example of EMD processing on an Oz EEG epoch (0.75 s) in subject I, while he was gazing at LED ‘cursor left’. EMD first decomposed ten IMFs, representing fine-to-coarse information of the Oz EEG epoch. The mean instantaneous frequencies (f¯GZC) of the ten IMFs were calculated by rGZC as 626.22 Hz, 344.49 Hz, 183.27 Hz, 60.28 Hz, 42.14 Hz, 30.10 Hz, 16.42 Hz, 9.83 Hz, 6.51 Hz, and 1.86 Hz, respectively. The IMF6, with f¯GZC=30.10 Hz within 29.5 Hz and 35.5 Hz, was chosen as the SSVEP-related

Discussion

This study presents an EMD and rGZC combined approach to achieve the frequency recognition in SSVEP based BCI system. Each participant in this study completed a practical cursor control task to verify the performance of the proposed BCI. The proposed EMD approach decomposed the EEG signal induced from visual cortical area into a set of IMFs representing fine-to-coarse degrees of oscillatory waves. After examining the mean instantaneous frequency of each IMF, the SSVEP-related IMFs that

Conclusions

This study develops an SSVEP-based BCI using EMD-rGZC approach for SSVEP frequency recognition to control a computer mouse. The proposed BCI system has several distinct features: (1) SSVEP unrelated noise can be removed by deselecting SSVEP-unrelated IMFs; (2) high-frequency flickers induce SSVEPs to avoid the interferences of low-frequency noise; (3) the proposed high-frequency flickering design achieves a more comfortable visualization; (4) the frequency recognition in data segment is

Acknowledgments

This study was funded by the National Central University, National Science Council (96-2628-E-008-070-MY3, 96-2221-E-008-122-MY3, 96-2221-E-010-003-MY3, 96-2221-E-008-115-MY3, 99-2628-E-008-003-MY3, 99-2628-E-008-012-MY3), and Veterans General Hospital University System of Taiwan Joint Research Program (VGHUST96-P4-15, VGHUST97-P3-11, VGHUST98-P3-09, VGHUST99-P3-13), Taoyuan General Hospital Intramural Project (PTH-9819).

References (38)

  • T.N. Cornsweet

    Visual perception

    (1970)
  • Jian Ding et al.

    Attentional modulation of SSVEP power depends on the network tagged by the flicker frequency

    Cereb Cortex

    (2006)
  • E. Donchin et al.

    The mental prosthesis: assessing the speed of a P300-based brain–computer interface

    IEEE Trans Rehabil Eng

    (2000)
  • P. He et al.

    Removal of ocular artifacts from electro-encephalogram by adaptive filtering

    Med Biol Eng Comput

    (2004)
  • C.S. Herrmann

    Human EEG responses to 1–100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena

    Exp Brain Res

    (2001)
  • N.E. Huang et al.

    The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis

    Proc Royal Soc Lond A: Math Phys Eng Sci

    (1998)
  • Huang NEB, MD, US). Computing frequency by using generalized zero-crossing applied to intrinsic mode functions. The...
  • W. Huang et al.

    Engineering analysis of biological variables: an example of blood pressure over 1 day

    Proc Natl Acad Sci USA

    (1998)
  • W. Huang et al.

    Use of intrinsic modes in biology: examples of indicial response of pulmonary blood pressure to ± step hypoxia

    Proc Natl Acad Sci USA

    (1998)
  • Cited by (79)

    • A L1 normalization enhanced dynamic window method for SSVEP-based BCIs

      2022, Journal of Neuroscience Methods
      Citation Excerpt :

      Thus, SSVEPs are easily detected and its detection task is dependent less on training data. SSVEP-based BCI systems have been widely applied to control cursors (Wu et al., 2011; Allison et al., 2012; Bi et al., 2014), smart homes (Putze and Wei, 2019; Chai et al., 2020), spellers (Chen et al., 2015; Hwang et al., 2012; Nakanishi et al., 2014; Saravanakumar and Reddy, 2019) and unmanned aerial vehicles (Wang et al., 2018; Duan et al., 2019; Arpaia et al., 2020). There are two ways to improve the ITR of SSVEP-based BCI systems, including increasing the number of recognized targets and improving the efficiency of detection methods (Chen et al., 2015).

    • A novel hybrid BCI speller based on RSVP and SSVEP paradigm

      2020, Computer Methods and Programs in Biomedicine
      Citation Excerpt :

      The attention to the target stimulus increases the effect of the corresponding frequency in brain activity. Therefore, it is possible to determine which stimulus is given attention by the user, by identifying the main frequency of the power spectral density of the brain activity [13]. In [14,15], unlike conventional work done in the SSVEP BCIs, a single flicker stimulus was used to perform multi-class classification.

    • SSVEP recognition by modeling brain activity using system identification based on Box-Jenkins model

      2018, Computers in Biology and Medicine
      Citation Excerpt :

      In addition to correlation-based methods, various methods which convey feature extraction and classification steps for SSVEP recognition based on pattern recognition were proposed. Among these methods, one can refer to methods based on Hilbert-Huang transform (HHT) [40] [41], wavelet transform [42], empirical mode decomposition (EMD) [43], and multivariate linear regression (MLR) [29]. Most of these methods extract SSVEP features from each stimulation frequency individually.

    View all citing articles on Scopus
    View full text