nach oben

Erschienen in:

2023 | OriginalPaper | Buchkapitel

DCLL—A Deep Network for Possible Real-Time Decoding of Imagined Words

verfasst von : Jerrin Thomas Panachakel, A. G. Ramakrishnan

Erschienen in: International Symposium on Intelligent Informatics

Verlag: Springer Nature Singapore

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

We present a novel architecture for classifying imagined words from electroencephalogram (EEG) captured during speech imagery. The proposed architecture employs a sliding window with overlap for data augmentation (DA) and common spatial pattern (CSP) in order to derive the features. The dimensionality of features is reduced using linear discriminant analysis (LDA). Long short-term memory (LSTM) along with majority voting is used as the classifier. We call the proposed architecture the DCLL (DA-CSP-LDA-LSTM) architecture. On a publicly available imagined word dataset, the DCLL architecture achieves an accuracy of 85.2% for classifying the imagined words “in” and “cooperate”. Although this is around 7% less than the best result in the literature on this dataset, the DCLL architecture is roughly 300 times faster than the latter, making it a potential candidate for imagined word-based online BCI systems where the EEG signal needs to be classified in real time.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nächstes Kapitel Towards Frugal Artificial Intelligence: Exploring Neural Network Pruning and Binarization

S.N. Abdulkader, A. Atia, M.S.M. Mostafa, Brain computer interfacing: applications and challenges. Egypt. Inf. J. 16(2), 213–230 (2015)

C. Herff, D. Heger, A. De Pesters, D. Telaar, P. Brunner, G. Schalk, T. Schultz, Brain-to-text: decoding spoken phrases from phone representations in the brain. Front. Neurosci. 9, 217 (2015)CrossRef

C. Herff, G. Johnson, L. Diener, J. Shih, D. Krusienski, T. Schultz, Towards direct speech synthesis from ECoG: A pilot study, in 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (IEEE, 2016), pp. 1540–1543

E.M. Mugler, C.A. Ruf, S. Halder, M. Bensch, A. Kubler, Design and implementation of a p300-based brain-computer interface for controlling an internet browser. IEEE Trans. Neural Syst. Rehabil. Eng. 18(6), 599–609 (2010)CrossRef

M. Xu, X. Xiao, Y. Wang, H. Qi, T.P. Jung, D. Ming, A brain-computer interface based on miniature-event-related potentials induced by very small lateral visual stimuli. IEEE Trans. Biomed. Eng. 65(5), 1166–1175 (2018)CrossRef

J.S. Brumberg, E.J. Wright, D.S. Andreasen, F.H. Guenther, P.R. Kennedy, Classification of intended phoneme production from chronic intracortical microelectrode recordings in speech motor cortex. Front. Neurosci. 5, 65 (2011)

P. Kennedy, A. Cervantes, C. Gambrell, P. Ehirim, Advances in the development of a speech prosthesis, in Direct and Indirect Benefits of Translingual Neurostimulation Technology for Neurorehabilitation of Chronic Stroke Symptoms (2017), p. 1

G.H. Wilson, S.D. Stavisky, F.R. Willett, D.T. Avansino, J.N. Kelemen, L.R. Hochberg, J.M. Henderson, S. Druckmann, K.V. Shenoy, Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus. J. Neural Eng. 17(6), 066007 (2020)CrossRef

D. Dash, A. Wisler, P. Ferrari, E.M. Davenport, J. Maldjian, J. Wang, MEG sensor selection for neural speech decoding. IEEE Access 8, 182320–182337 (2020)CrossRef

10.

F. Destoky, M. Philippe, J. Bertels, M. Verhasselt, N. Coquelet, M. Vander Ghinst, V. Wens, X. De Tiège, M. Bourguignon, Comparing the potential of MEG and EEG to uncover brain tracking of speech temporal envelope. Neuroimage 184, 201–213 (2019)CrossRef

11.

S.S. Yoo, T. Fairneny, N.K. Chen, S.E. Choo, L.P. Panych, H. Park, S.Y. Lee, F.A. Jolesz, Brain-computer interface using fMRI: spatial navigation by thoughts. Neuroreport 15(10), 1591–1595 (2004)CrossRef

12.

K. Abe, T. Takahashi, Y. Takikawa, H. Arai, S. Kitazawa, Applying independent component analysis to detect silent speech in magnetic resonance imaging signals. Eur. J. Neurosci. 34(8), 1189–1199 (2011)CrossRef

13.

C. Herff, F. Putze, D. Heger, C. Guan, T. Schultz, Speaking mode recognition from functional near infrared spectroscopy, in 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE, 2012), pp. 1715–1718

14.

E.N. Kamavuako, U.A. Sheikh, S.O. Gilani, M. Jamil, I.K. Niazi, Classification of overt and covert speech for near-infrared spectroscopy-based brain computer interface. Sensors 18(9), 2989 (2018)CrossRef

15.

A.R. Sereshkeh, R. Yousefi, A.T. Wong, T. Chau, Online classification of imagined speech using functional near-infrared spectroscopy signals. J. Neural Eng. 16(1), 016005 (2018)CrossRef

16.

I.A. Fouad, F.E.Z.M. Labib, M.S. Mabrouk, A.A. Sharawy, A.Y. Sayed, Improving the performance of P300 BCI system using different methods. Netw. Model. Anal. Health Inf. Bioinform. 9(1), 1–13 (2020)

17.

E.W. Sellers, D.J. Krusienski, D.J. McFarland, T.M. Vaughan, J.R. Wolpaw, A p300 event-related potential brain-computer interface (bci): the effects of matrix size and inter stimulus interval on performance. Biol. Psychol. 73(3), 242–252 (2006)CrossRef

18.

J. Kevric, A. Subasi, Comparison of signal decomposition methods in classification of EEG signals for motor-imagery BCI system. Biomed. Signal Process. Control 31, 398–406 (2017)CrossRef

19.

G. Onose, C. Grozea, A. Anghelescu, C. Daia, C. Sinescu, A. Ciurea, T. Spircu, A. Mirea, I. Andone, A. Spânu et al., On the feasibility of using motor imagery EEG-based brain-computer interface in chronic tetraplegics for assistive robotic arm control: a clinical test and long-term post-trial follow-up. Spinal Cord 50(8), 599–608 (2012)CrossRef

20.

M.K. Ojha, M.K. Mukul, Detection of target frequency from SSVEP signal using empirical mode decomposition for SSVEP based BCI inference system. Wireless Personal Commun. 1–13 (2020)

21.

G.R. Müller-Putz, R. Scherer, C. Brauneis, G. Pfurtscheller, Steady-state visual evoked potential (ssvep)-based communication: impact of harmonic frequency components. J. Neural Eng. 2(4), 123 (2005)CrossRef

22.

C. Han, G. Xu, J. Xie, C. Chen, S. Zhang, Highly interactive brain-computer interface based on flicker-free steady-state motion visual evoked potential. Sci. Rep. 8(1), 1–13 (2018)

23.

B. Allison, B. Graimann, A. Gräser, Why use a BCI if you are healthy, in ACE Workshop-Brain-Computer Interfaces and Games (2007), pp. 7–11

24.

R. Bogue, Brain-computer interfaces: control by thought. Indus. Robot Int. J. (2010)

25.

J.T. Panachakel, A. Ramakrishnan, Decoding covert speech from EEG-a comprehensive review. Front. Neurosci. 15, 392 (2021)CrossRef

26.

C.H. Nguyen, G.K. Karavas, P. Artemiadis, Adaptive multi-degree of freedom brain computer interface using online feedback: Towards novel methods and metrics of mutual adaptation between humans and machines for BCI. PloS One 14(3), e0212620 (2019)CrossRef

27.

A.R. Sereshkeh, R. Trott, A. Bricout, T. Chau, Online EEG classification of covert speech for brain-computer interfacing. Int. J. Neural Syst. 27(08), 1750033 (2017)CrossRef

28.

N. Naseer, M.J. Hong, K.S. Hong, Online binary decision decoding using functional near-infrared spectroscopy for the development of brain-computer interface. Exp. Brain Res. 232(2), 555–564 (2014)CrossRef

29.

G. Gallegos-Ayala, A. Furdea, K. Takano, C.A. Ruf, H. Flor, N. Birbaumer, Brain communication in a completely locked-in patient using bedside near-infrared spectroscopy. Neurology 82(21), 1930–1932 (2014)CrossRef

30.

C.H. Nguyen, G.K. Karavas, P. Artemiadis, Inferring imagined speech using EEG signals: a new approach using Riemannian manifold features. J. Neural Eng. 15(1), 016002 (2017)CrossRef

31.

P. He, G. Wilson, C. Russell, Removal of ocular artifacts from electro-encephalogram by adaptive filtering. Med. Biol. Eng. Comput. 42(3), 407–412 (2004)CrossRef

32.

E. Lashgari, D. Liang, U. Maoz, Data augmentation for deep-learning-based electroencephalography. J. Neurosci. Methods 108885 (2020)

33.

A. O’Shea, G. Lightbody, G. Boylan, A. Temko, Neonatal seizure detection using convolutional neural networks, in 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP) (IEEE, 2017), pp. 1–6

34.

N.S. Kwak, K.R. Müller, S.W. Lee, A convolutional neural network for steady state visual evoked potential classification under ambulatory environment. PloS One 12(2), e0172578 (2017)CrossRef

35.

I. Ullah, M. Hussain, H. Aboalsamh et al., An automated system for epilepsy detection using EEG brain signals based on deep learning approach. Expert Syst. Appl. 107, 61–71 (2018)CrossRef

36.

I. Majidov, T. Whangbo, Efficient classification of motor imagery electroencephalography signals using deep learning methods. Sensors 19(7), 1736 (2019)CrossRef

37.

Y. Luo, B.L. Lu, EEG data augmentation for emotion recognition using a conditional Wasserstein GAN, in 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (IEEE, 2018), pp. 2535–2538

38.

Z. Wei, J. Zou, J. Zhang, J. Xu, Automatic epileptic EEG detection using convolutional neural network with improvements in time-domain. Biomed. Signal Process. Control 53, 101551 (2019)CrossRef

39.

S. Chang, H. Jun, Hybrid deep-learning model to recognise emotional responses of users towards architectural design alternatives. J. Asian Architect. Build. Eng. 18(5), 381–391 (2019)CrossRef

40.

J.T. Panachakel, G.P. Kumar, K. Sharma, A. Ramakrishnan, Automated classification of EEG into meditation and non-meditation epochs using common spatial pattern, linear discriminant analysis, and LSTM, in TENCON 2021-2021 IEEE Region 10 Conference (IEEE, 2021), pp. 1–6

41.

P. Goel, R. Joshi, M. Sur, H.A. Murthy, A common spatial pattern approach for classification of mental counting and motor execution EEG, in International Conference on Intelligent Human Computer Interaction (Springer, 2018), pp. 26–35

42.

J.T. Panachakel, N.N. Vinayak, M. Nunna, A.G. Ramakrishnan, K. Sharma, An improved EEG acquisition protocol facilitates localized neural activation, in Advances in Communication Systems and Networks (Springer, 2020), pp. 267–281

43.

J.T. Panachakel, G.P. Kumar, K. Sharma, A. Ramakrishnan, Binary classification of meditative state from the resting state using EEG, in 2021 IEEE 18th India Council International Conference (INDICON) (IEEE, 2021), pp. 1–6

44.

J.T. Panachakel, A. Ramakrishnan, Decoding imagined speech from EEG using transfer learning. IEEE Access (2021)

45.

J.T. Panachakel, A. Ramakrishnan, T. Ananthapadmanabha, A novel deep learning architecture for decoding imagined speech from EEG. arXiv preprint arXiv:2003.09374 (2020)

46.

J.T. Panachakel, A. Ramakrishnan, A. Anusha, K. Sharma, Can we identify the category of imagined phoneme from EEG? in 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (IEEE, 2021)

47.

J.T. Panachakel, A. Ramakrishnan, Classification of phonological categories in imagined speech using phase synchronization measure, in 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (IEEE, 2021)

48.

V. Benzy, A. Vinod, R. Subasree, S. Alladi, K. Raghavendra, Motor imagery hand movement direction decoding using brain computer interface to aid stroke recovery and rehabilitation. IEEE Trans. Neural Syst. Rehabil. Eng. 28(12), 3051–3062 (2020)CrossRef

Titel: DCLL—A Deep Network for Possible Real-Time Decoding of Imagined Words
verfasst von: Jerrin Thomas Panachakel
A. G. Ramakrishnan
Verlag: Springer Nature Singapore
Buch: International Symposium on Intelligent Informatics
Print ISBN: 978-981-19-8093-0

Electronic ISBN: 978-981-19-8094-7

Copyright-Jahr: 2023
DOI: https://doi.org/10.1007/978-981-19-8094-7_1

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"