Skip to main content

09.03.2024 | Research Article

The design and implementation of multi-character classification scheme based on EEG signals of visual imagery

verfasst von: Hongguang Pan, Wei Song, Li Li, Xuebin Qin

Erschienen in: Cognitive Neurodynamics

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In visual-imagery-based brain–computer interface (VI-BCI), there are problems of singleness of imagination task and insufficient description of feature information, which seriously hinder the development and application of VI-BCI technology in the field of restoring communication. In this paper, we design and optimize a multi-character classification scheme based on electroencephalogram (EEG) signals of visual imagery (VI), which is used to classify 29 characters including 26 lowercase English letters and three punctuation marks. Firstly, a new paradigm of randomly presenting characters and including preparation stage is designed to acquire EEG signals and construct a multi-character dataset, which can eliminate the influence between VI tasks. Secondly, tensor data is obtained by the Morlet wavelet transform, and a feature extraction algorithm based on tensor—uncorrelated multilinear principal component analysis is used to extract high-quality features. Finally, three classifiers, namely support vector machine, K-nearest neighbor, and extreme learning machine, are employed for classifying multi-character, and the results are compared. The experimental results demonstrate that, the proposed scheme effectively extracts character features with minimal redundancy, weak correlation, and strong representation capability, and successfully achieves an average classification accuracy 97.59% for 29 characters, surpassing existing research in terms of both accuracy and quantity of classification. The present study designs a new paradigm for acquiring EEG signals of VI, and combines the Morlet wavelet transform and UMPCA algorithm to extract the character features, enabling multi-character classification in various classifiers. This research paves a novel pathway for establishing direct brain-to-world communication.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Alom KM, Islam S (2020) Classification for the P300-based brain computer interface (BCI). In: 2020 2nd international conference on advanced information and communication technology (ICAICT), pp. 387–391 Alom KM, Islam S (2020) Classification for the P300-based brain computer interface (BCI). In: 2020 2nd international conference on advanced information and communication technology (ICAICT), pp. 387–391
Zurück zum Zitat Altan A, Karasu S (2020) Recognition of COVID-19 disease from X-ray images by hybrid model consisting of 2D curvelet transform, chaotic salp swarm algorithm and deep learning technique. Chaos Solit Fract 140:110071MathSciNetCrossRef Altan A, Karasu S (2020) Recognition of COVID-19 disease from X-ray images by hybrid model consisting of 2D curvelet transform, chaotic salp swarm algorithm and deep learning technique. Chaos Solit Fract 140:110071MathSciNetCrossRef
Zurück zum Zitat Babo RM, Buot A, Tallon BC (2019) Neural responses to heartbeats distinguish self from other during imagination. NeuroImage 191:10–20CrossRef Babo RM, Buot A, Tallon BC (2019) Neural responses to heartbeats distinguish self from other during imagination. NeuroImage 191:10–20CrossRef
Zurück zum Zitat Bang JS, Jeong JH, Won DO (2021) Classification of visual perception and imagery based EEG signals using convolutional neural networks. In: 2021 9th international winter conference on brain-computer interface (BCI), pp. 1–6 Bang JS, Jeong JH, Won DO (2021) Classification of visual perception and imagery based EEG signals using convolutional neural networks. In: 2021 9th international winter conference on brain-computer interface (BCI), pp. 1–6
Zurück zum Zitat Boashash B (2015) Time-frequency signal analysis and processing: a comprehensive reference Boashash B (2015) Time-frequency signal analysis and processing: a comprehensive reference
Zurück zum Zitat Christa N, Scherer R, Reiner M, Gert P (2005) Imagery of motor actions: differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. Brain Res Cogn Brain Res 25(3):668–677CrossRef Christa N, Scherer R, Reiner M, Gert P (2005) Imagery of motor actions: differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. Brain Res Cogn Brain Res 25(3):668–677CrossRef
Zurück zum Zitat Daniel B, Beata J, Nicolas MY, Sergey SD (2015) Neural point-and-click communication by a person with incomplete locked-in syndrome. Neurorehabil Neural Rep 29(5):462–471CrossRef Daniel B, Beata J, Nicolas MY, Sergey SD (2015) Neural point-and-click communication by a person with incomplete locked-in syndrome. Neurorehabil Neural Rep 29(5):462–471CrossRef
Zurück zum Zitat Dijkstra N, Mostert P, Lange FPd, Bosch S, Gerven MA (2018) Differential temporal dynamics during visual imagery and perception. Elife 7:33904CrossRef Dijkstra N, Mostert P, Lange FPd, Bosch S, Gerven MA (2018) Differential temporal dynamics during visual imagery and perception. Elife 7:33904CrossRef
Zurück zum Zitat Ehsan ET (2012) Classification of primitive shapes using brain computer interfaces. Comput Aided Des 44(10):1011–1019CrossRef Ehsan ET (2012) Classification of primitive shapes using brain computer interfaces. Comput Aided Des 44(10):1011–1019CrossRef
Zurück zum Zitat Esposito DM, Detre JA, Aguirre GK, Stallcup M, Alsop DC (1997) A functional MRI study of mental image generation. Neuropsychologia 35(5):725–30CrossRefPubMed Esposito DM, Detre JA, Aguirre GK, Stallcup M, Alsop DC (1997) A functional MRI study of mental image generation. Neuropsychologia 35(5):725–30CrossRefPubMed
Zurück zum Zitat Fabio LR, Gustavo P, Jose AM (2020) Convolutional neural networks and genetic algorithm for visual imagery classification. Austral Phys Eng Sci Med 43(3):973–983CrossRef Fabio LR, Gustavo P, Jose AM (2020) Convolutional neural networks and genetic algorithm for visual imagery classification. Austral Phys Eng Sci Med 43(3):973–983CrossRef
Zurück zum Zitat Filip S, Tom C (2020) Non-specific visuospatial imagery as a novel mental task for online EEG-based BCI control. Int J Neural Syst 30(6):2050026CrossRef Filip S, Tom C (2020) Non-specific visuospatial imagery as a novel mental task for online EEG-based BCI control. Int J Neural Syst 30(6):2050026CrossRef
Zurück zum Zitat Fu Y, Li Z, Gong A, Qian Q, Su L, Zhao L (2021) Identification of visual imagery by electroencephalography based on empirical mode decomposition and an auto-regressive model. Comput Intell Neuro Sci 30:203 Fu Y, Li Z, Gong A, Qian Q, Su L, Zhao L (2021) Identification of visual imagery by electroencephalography based on empirical mode decomposition and an auto-regressive model. Comput Intell Neuro Sci 30:203
Zurück zum Zitat Jelena M, Jeremy F, Smeety P, Jeremie M, Fabien L (2021) Towards identifying optimal biased feedback for various user states and traits in motor imagery BCI. IEEE Trans Biomed Eng 69(23):1101–1110 Jelena M, Jeremy F, Smeety P, Jeremie M, Fabien L (2021) Towards identifying optimal biased feedback for various user states and traits in motor imagery BCI. IEEE Trans Biomed Eng 69(23):1101–1110
Zurück zum Zitat Juneja K (2019) Individual and mutual feature processed ELM model for EEG signal based brain activity classification. Wirel Pers Commun Int J 108(2):659CrossRef Juneja K (2019) Individual and mutual feature processed ELM model for EEG signal based brain activity classification. Wirel Pers Commun Int J 108(2):659CrossRef
Zurück zum Zitat Kano N (2019) Communication of ALS patients with totally locked-in syndrome. J Nurs Sci Eng 6(2):63–69 Kano N (2019) Communication of ALS patients with totally locked-in syndrome. J Nurs Sci Eng 6(2):63–69
Zurück zum Zitat Ko W, Jeon E, Jeong S, Suk H-I (2021) Multi-scale neural network for EEG representation learning in BCI. IEEE Computat Intell Magaz 16(2):31–45CrossRef Ko W, Jeon E, Jeong S, Suk H-I (2021) Multi-scale neural network for EEG representation learning in BCI. IEEE Computat Intell Magaz 16(2):31–45CrossRef
Zurück zum Zitat Koizumi K, Ueda K, Nakao M (2018) Development of a cognitive brain-machine interface based on a visual imagery method. Annu Int Conf IEEE Eng Med Biol Soc 18:1062–1065 Koizumi K, Ueda K, Nakao M (2018) Development of a cognitive brain-machine interface based on a visual imagery method. Annu Int Conf IEEE Eng Med Biol Soc 18:1062–1065
Zurück zum Zitat Kumar P, Saini R, Roy PP, Sahu PK, Dogra DP (2018) Envisioned speech recognition using EEG sensors. Pers Ubiquit Comput 22:185–199CrossRef Kumar P, Saini R, Roy PP, Sahu PK, Dogra DP (2018) Envisioned speech recognition using EEG sensors. Pers Ubiquit Comput 22:185–199CrossRef
Zurück zum Zitat Kwon BH, Lee BH, Cho JH, Jeong JH (2022) Decoding visual imagery from EEG signals using visual perception guided network training method. In: 2022 10th international winter conference on brain-computer interface (BCI), pp. 1–5 Kwon BH, Lee BH, Cho JH, Jeong JH (2022) Decoding visual imagery from EEG signals using visual perception guided network training method. In: 2022 10th international winter conference on brain-computer interface (BCI), pp. 1–5
Zurück zum Zitat Laurens VDM, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(86):2579–2605 Laurens VDM, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(86):2579–2605
Zurück zum Zitat Lee S, Jang S, Jun SC (2022) Exploring the ability to classify visual perception and visual imagery EEG data: toward an intuitive BCI system. Electronics 11(17):2706CrossRef Lee S, Jang S, Jun SC (2022) Exploring the ability to classify visual perception and visual imagery EEG data: toward an intuitive BCI system. Electronics 11(17):2706CrossRef
Zurück zum Zitat Lee S, Lee M, Lee S (2020) Neural decoding of imagined speech and visual imagery as intuitive paradigms for BCI communication. IEEE Trans Neural Syst Rehabilit Eng 28(12):2647–2659CrossRef Lee S, Lee M, Lee S (2020) Neural decoding of imagined speech and visual imagery as intuitive paradigms for BCI communication. IEEE Trans Neural Syst Rehabilit Eng 28(12):2647–2659CrossRef
Zurück zum Zitat Li Y, Ning F, Jiang X, Yi Y (2022) Feature extraction of ship radiation signals based on wavelet packet decomposition and energy entropy. Math Probl Eng 2022:1–12CrossRef Li Y, Ning F, Jiang X, Yi Y (2022) Feature extraction of ship radiation signals based on wavelet packet decomposition and energy entropy. Math Probl Eng 2022:1–12CrossRef
Zurück zum Zitat Li M, Wang R, Xu D (2020) An improved composite multiscale fuzzy entropy for feature extraction of MI-EEG. Entropy (Basel) 22(12):1356ADSCrossRefPubMed Li M, Wang R, Xu D (2020) An improved composite multiscale fuzzy entropy for feature extraction of MI-EEG. Entropy (Basel) 22(12):1356ADSCrossRefPubMed
Zurück zum Zitat Llorella FR, AzorÃn JM, Patow G (2023) Black hole algorithm with convolutional neural networks for the creation of brain-computer interface based in visual perception and visual imagery. Neural Comput Appl 35(8):5631–5641CrossRef Llorella FR, Azorín JM, Patow G (2023) Black hole algorithm with convolutional neural networks for the creation of brain-computer interface based in visual perception and visual imagery. Neural Comput Appl 35(8):5631–5641CrossRef
Zurück zum Zitat Lu H, Konstantinos PN, Anastasios VN (2009) Uncorrelated multilinear discriminant analysis with regularization and aggregation for tensor object recognition. IEEE Trans Neural Netw 20(1):103–23CrossRefPubMed Lu H, Konstantinos PN, Anastasios VN (2009) Uncorrelated multilinear discriminant analysis with regularization and aggregation for tensor object recognition. IEEE Trans Neural Netw 20(1):103–23CrossRefPubMed
Zurück zum Zitat Lu H, Konstantinos PN, Anastasios VN (2009) Uncorrelated multilinear principal component analysis for unsupervised multilinear subspace learning. IEEE Trans Neural Netw 20(11):1820–1836CrossRefPubMed Lu H, Konstantinos PN, Anastasios VN (2009) Uncorrelated multilinear principal component analysis for unsupervised multilinear subspace learning. IEEE Trans Neural Netw 20(11):1820–1836CrossRefPubMed
Zurück zum Zitat Markus K, Jan K, Thomas M, Mark G (2000) Cortical activation evoked by visual mental imagery as measured by fMRI. Neuro Rep 11:3957–3962 Markus K, Jan K, Thomas M, Mark G (2000) Cortical activation evoked by visual mental imagery as measured by fMRI. Neuro Rep 11:3957–3962
Zurück zum Zitat Min B, Kim J, Park H-j, Lee B, et al. (2016) Vowel imagery decoding toward silent speech BCI using extreme learning machine with electroencephalogram. BioMed Res Int 2016 Min B, Kim J, Park H-j, Lee B, et al. (2016) Vowel imagery decoding toward silent speech BCI using extreme learning machine with electroencephalogram. BioMed Res Int 2016
Zurück zum Zitat Nataliya K, Lindgren JT, Anatole L (2018) Attending to visual stimuli versus performing visual imagery as a control strategy for EEG-based brain-computer interfaces. Sci Rep 8(1):13222CrossRef Nataliya K, Lindgren JT, Anatole L (2018) Attending to visual stimuli versus performing visual imagery as a control strategy for EEG-based brain-computer interfaces. Sci Rep 8(1):13222CrossRef
Zurück zum Zitat Pan H, Li Z, Tian C, Wang L, Fu Y, Qin X, Liu F (2022) The light GBM-based classification algorithm for Chinese characters speech development imagery BCI system. Cognit Neurodyn 17(2):373–384CrossRef Pan H, Li Z, Tian C, Wang L, Fu Y, Qin X, Liu F (2022) The light GBM-based classification algorithm for Chinese characters speech development imagery BCI system. Cognit Neurodyn 17(2):373–384CrossRef
Zurück zum Zitat Sousa T, Amaral C, Andrade J, Pires G, Nunes UJ, Castelo-Branco M (2017) Pure visual imagery as a potential approach to achieve three classes of control for implementation of BCI in non-motor disorders. J Neural Eng 14(4):1–11CrossRef Sousa T, Amaral C, Andrade J, Pires G, Nunes UJ, Castelo-Branco M (2017) Pure visual imagery as a potential approach to achieve three classes of control for implementation of BCI in non-motor disorders. J Neural Eng 14(4):1–11CrossRef
Zurück zum Zitat Srivastava G, Crottaz-Herbette S, Lau KM, Glover GH, Menon V (2005) ICA-based procedures for removing ballistocardiogram artifacts from EEG data acquired in the MRI scanner. Neuroimage 24(1):50–60CrossRefPubMed Srivastava G, Crottaz-Herbette S, Lau KM, Glover GH, Menon V (2005) ICA-based procedures for removing ballistocardiogram artifacts from EEG data acquired in the MRI scanner. Neuroimage 24(1):50–60CrossRefPubMed
Zurück zum Zitat Sun G, Song Z, Liu J, Zhu S, He Y (2017) Feature selection method based on maximum information coefficient and approximate markov blanket. Acta Automat Sin 43(5):795–805 Sun G, Song Z, Liu J, Zhu S, He Y (2017) Feature selection method based on maximum information coefficient and approximate markov blanket. Acta Automat Sin 43(5):795–805
Zurück zum Zitat Vinay C, Ankur V, Anand N, Eklas H (2020) Brain-computer interface-based humanoid control: a review. Sensors (Basel) 20(13):3620CrossRef Vinay C, Ankur V, Anand N, Eklas H (2020) Brain-computer interface-based humanoid control: a review. Sensors (Basel) 20(13):3620CrossRef
Zurück zum Zitat Williams AH, Ben P, Maheswaranathan N, Dhawale AK (2020) Discovering precise temporal patterns in large-scale neural recordings through robust and interpretable time warping. Neuron 105(2):246–259CrossRefPubMed Williams AH, Ben P, Maheswaranathan N, Dhawale AK (2020) Discovering precise temporal patterns in large-scale neural recordings through robust and interpretable time warping. Neuron 105(2):246–259CrossRefPubMed
Zurück zum Zitat Winlove C, Milton F, Ranson J, Fulford J, Mackisack M, Macpherson F, Zeman A (2018) The neural correlates of visual imagery: a co-ordinate-based meta-analysis. Cortex 105:4–25CrossRefPubMed Winlove C, Milton F, Ranson J, Fulford J, Mackisack M, Macpherson F, Zeman A (2018) The neural correlates of visual imagery: a co-ordinate-based meta-analysis. Cortex 105:4–25CrossRefPubMed
Zurück zum Zitat YaÄŸ I, Altan A (2022) Artificial intelligence-based robust hybrid algorithm design and implementation for real-time detection of plant diseases in agricultural environments. Biology 11(12):1732CrossRef YaÄŸ I, Altan A (2022) Artificial intelligence-based robust hybrid algorithm design and implementation for real-time detection of plant diseases in agricultural environments. Biology 11(12):1732CrossRef
Zurück zum Zitat Zhang Z, Sun J, Chen T (2022) A new dynamically convergent differential neural network for brain signal recognition. Biomed Sig Process Control 71(3):103130CrossRef Zhang Z, Sun J, Chen T (2022) A new dynamically convergent differential neural network for brain signal recognition. Biomed Sig Process Control 71(3):103130CrossRef
Zurück zum Zitat Zhang L, Wen D, Li C, Zhu R (2020) Ensemble classifier based on optimized extreme learning machine for motor imagery classification. J Neural Eng 17(2):1–12CrossRef Zhang L, Wen D, Li C, Zhu R (2020) Ensemble classifier based on optimized extreme learning machine for motor imagery classification. J Neural Eng 17(2):1–12CrossRef
Metadaten
Titel
The design and implementation of multi-character classification scheme based on EEG signals of visual imagery
verfasst von
Hongguang Pan
Wei Song
Li Li
Xuebin Qin
Publikationsdatum
09.03.2024
Verlag
Springer Netherlands
Erschienen in
Cognitive Neurodynamics
Print ISSN: 1871-4080
Elektronische ISSN: 1871-4099
DOI
https://doi.org/10.1007/s11571-024-10087-z

Neuer Inhalt