Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Cover of the book

2023 | OriginalPaper | Chapter

DCLL—A Deep Network for Possible Real-Time Decoding of Imagined Words

Authors : Jerrin Thomas Panachakel, A. G. Ramakrishnan

Published in: International Symposium on Intelligent Informatics

Publisher: Springer Nature Singapore

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

We present a novel architecture for classifying imagined words from electroencephalogram (EEG) captured during speech imagery. The proposed architecture employs a sliding window with overlap for data augmentation (DA) and common spatial pattern (CSP) in order to derive the features. The dimensionality of features is reduced using linear discriminant analysis (LDA). Long short-term memory (LSTM) along with majority voting is used as the classifier. We call the proposed architecture the DCLL (DA-CSP-LDA-LSTM) architecture. On a publicly available imagined word dataset, the DCLL architecture achieves an accuracy of 85.2% for classifying the imagined words “in” and “cooperate”. Although this is around 7% less than the best result in the literature on this dataset, the DCLL architecture is roughly 300 times faster than the latter, making it a potential candidate for imagined word-based online BCI systems where the EEG signal needs to be classified in real time.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
next chapter Towards Frugal Artificial Intelligence: Exploring Neural Network Pruning and Binarization
Literature
1.
S.N. Abdulkader, A. Atia, M.S.M. Mostafa, Brain computer interfacing: applications and challenges. Egypt. Inf. J. 16(2), 213–230 (2015)
2.
C. Herff, D. Heger, A. De Pesters, D. Telaar, P. Brunner, G. Schalk, T. Schultz, Brain-to-text: decoding spoken phrases from phone representations in the brain. Front. Neurosci. 9, 217 (2015)CrossRef
3.
C. Herff, G. Johnson, L. Diener, J. Shih, D. Krusienski, T. Schultz, Towards direct speech synthesis from ECoG: A pilot study, in 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (IEEE, 2016), pp. 1540–1543
4.
E.M. Mugler, C.A. Ruf, S. Halder, M. Bensch, A. Kubler, Design and implementation of a p300-based brain-computer interface for controlling an internet browser. IEEE Trans. Neural Syst. Rehabil. Eng. 18(6), 599–609 (2010)CrossRef
5.
M. Xu, X. Xiao, Y. Wang, H. Qi, T.P. Jung, D. Ming, A brain-computer interface based on miniature-event-related potentials induced by very small lateral visual stimuli. IEEE Trans. Biomed. Eng. 65(5), 1166–1175 (2018)CrossRef
6.
J.S. Brumberg, E.J. Wright, D.S. Andreasen, F.H. Guenther, P.R. Kennedy, Classification of intended phoneme production from chronic intracortical microelectrode recordings in speech motor cortex. Front. Neurosci. 5, 65 (2011)
7.
P. Kennedy, A. Cervantes, C. Gambrell, P. Ehirim, Advances in the development of a speech prosthesis, in Direct and Indirect Benefits of Translingual Neurostimulation Technology for Neurorehabilitation of Chronic Stroke Symptoms (2017), p. 1
8.
G.H. Wilson, S.D. Stavisky, F.R. Willett, D.T. Avansino, J.N. Kelemen, L.R. Hochberg, J.M. Henderson, S. Druckmann, K.V. Shenoy, Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus. J. Neural Eng. 17(6), 066007 (2020)CrossRef
9.
D. Dash, A. Wisler, P. Ferrari, E.M. Davenport, J. Maldjian, J. Wang, MEG sensor selection for neural speech decoding. IEEE Access 8, 182320–182337 (2020)CrossRef
10.
F. Destoky, M. Philippe, J. Bertels, M. Verhasselt, N. Coquelet, M. Vander Ghinst, V. Wens, X. De Tiège, M. Bourguignon, Comparing the potential of MEG and EEG to uncover brain tracking of speech temporal envelope. Neuroimage 184, 201–213 (2019)CrossRef
11.
S.S. Yoo, T. Fairneny, N.K. Chen, S.E. Choo, L.P. Panych, H. Park, S.Y. Lee, F.A. Jolesz, Brain-computer interface using fMRI: spatial navigation by thoughts. Neuroreport 15(10), 1591–1595 (2004)CrossRef
12.
K. Abe, T. Takahashi, Y. Takikawa, H. Arai, S. Kitazawa, Applying independent component analysis to detect silent speech in magnetic resonance imaging signals. Eur. J. Neurosci. 34(8), 1189–1199 (2011)CrossRef
13.
C. Herff, F. Putze, D. Heger, C. Guan, T. Schultz, Speaking mode recognition from functional near infrared spectroscopy, in 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE, 2012), pp. 1715–1718
14.
E.N. Kamavuako, U.A. Sheikh, S.O. Gilani, M. Jamil, I.K. Niazi, Classification of overt and covert speech for near-infrared spectroscopy-based brain computer interface. Sensors 18(9), 2989 (2018)CrossRef
15.
A.R. Sereshkeh, R. Yousefi, A.T. Wong, T. Chau, Online classification of imagined speech using functional near-infrared spectroscopy signals. J. Neural Eng. 16(1), 016005 (2018)CrossRef
16.
I.A. Fouad, F.E.Z.M. Labib, M.S. Mabrouk, A.A. Sharawy, A.Y. Sayed, Improving the performance of P300 BCI system using different methods. Netw. Model. Anal. Health Inf. Bioinform. 9(1), 1–13 (2020)
17.
E.W. Sellers, D.J. Krusienski, D.J. McFarland, T.M. Vaughan, J.R. Wolpaw, A p300 event-related potential brain-computer interface (bci): the effects of matrix size and inter stimulus interval on performance. Biol. Psychol. 73(3), 242–252 (2006)CrossRef
18.
J. Kevric, A. Subasi, Comparison of signal decomposition methods in classification of EEG signals for motor-imagery BCI system. Biomed. Signal Process. Control 31, 398–406 (2017)CrossRef
19.
G. Onose, C. Grozea, A. Anghelescu, C. Daia, C. Sinescu, A. Ciurea, T. Spircu, A. Mirea, I. Andone, A. Spânu et al., On the feasibility of using motor imagery EEG-based brain-computer interface in chronic tetraplegics for assistive robotic arm control: a clinical test and long-term post-trial follow-up. Spinal Cord 50(8), 599–608 (2012)CrossRef
20.
M.K. Ojha, M.K. Mukul, Detection of target frequency from SSVEP signal using empirical mode decomposition for SSVEP based BCI inference system. Wireless Personal Commun. 1–13 (2020)
21.
G.R. Müller-Putz, R. Scherer, C. Brauneis, G. Pfurtscheller, Steady-state visual evoked potential (ssvep)-based communication: impact of harmonic frequency components. J. Neural Eng. 2(4), 123 (2005)CrossRef
22.
C. Han, G. Xu, J. Xie, C. Chen, S. Zhang, Highly interactive brain-computer interface based on flicker-free steady-state motion visual evoked potential. Sci. Rep. 8(1), 1–13 (2018)
23.
B. Allison, B. Graimann, A. Gräser, Why use a BCI if you are healthy, in ACE Workshop-Brain-Computer Interfaces and Games (2007), pp. 7–11
24.
R. Bogue, Brain-computer interfaces: control by thought. Indus. Robot Int. J. (2010)
25.
J.T. Panachakel, A. Ramakrishnan, Decoding covert speech from EEG-a comprehensive review. Front. Neurosci. 15, 392 (2021)CrossRef
26.
C.H. Nguyen, G.K. Karavas, P. Artemiadis, Adaptive multi-degree of freedom brain computer interface using online feedback: Towards novel methods and metrics of mutual adaptation between humans and machines for BCI. PloS One 14(3), e0212620 (2019)CrossRef
27.
A.R. Sereshkeh, R. Trott, A. Bricout, T. Chau, Online EEG classification of covert speech for brain-computer interfacing. Int. J. Neural Syst. 27(08), 1750033 (2017)CrossRef
28.
N. Naseer, M.J. Hong, K.S. Hong, Online binary decision decoding using functional near-infrared spectroscopy for the development of brain-computer interface. Exp. Brain Res. 232(2), 555–564 (2014)CrossRef
29.
G. Gallegos-Ayala, A. Furdea, K. Takano, C.A. Ruf, H. Flor, N. Birbaumer, Brain communication in a completely locked-in patient using bedside near-infrared spectroscopy. Neurology 82(21), 1930–1932 (2014)CrossRef
30.
C.H. Nguyen, G.K. Karavas, P. Artemiadis, Inferring imagined speech using EEG signals: a new approach using Riemannian manifold features. J. Neural Eng. 15(1), 016002 (2017)CrossRef
31.
P. He, G. Wilson, C. Russell, Removal of ocular artifacts from electro-encephalogram by adaptive filtering. Med. Biol. Eng. Comput. 42(3), 407–412 (2004)CrossRef
32.
E. Lashgari, D. Liang, U. Maoz, Data augmentation for deep-learning-based electroencephalography. J. Neurosci. Methods 108885 (2020)
33.
A. O’Shea, G. Lightbody, G. Boylan, A. Temko, Neonatal seizure detection using convolutional neural networks, in 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP) (IEEE, 2017), pp. 1–6
34.
N.S. Kwak, K.R. Müller, S.W. Lee, A convolutional neural network for steady state visual evoked potential classification under ambulatory environment. PloS One 12(2), e0172578 (2017)CrossRef
35.
I. Ullah, M. Hussain, H. Aboalsamh et al., An automated system for epilepsy detection using EEG brain signals based on deep learning approach. Expert Syst. Appl. 107, 61–71 (2018)CrossRef
36.
I. Majidov, T. Whangbo, Efficient classification of motor imagery electroencephalography signals using deep learning methods. Sensors 19(7), 1736 (2019)CrossRef
37.
Y. Luo, B.L. Lu, EEG data augmentation for emotion recognition using a conditional Wasserstein GAN, in 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (IEEE, 2018), pp. 2535–2538
38.
Z. Wei, J. Zou, J. Zhang, J. Xu, Automatic epileptic EEG detection using convolutional neural network with improvements in time-domain. Biomed. Signal Process. Control 53, 101551 (2019)CrossRef
39.
S. Chang, H. Jun, Hybrid deep-learning model to recognise emotional responses of users towards architectural design alternatives. J. Asian Architect. Build. Eng. 18(5), 381–391 (2019)CrossRef
40.
J.T. Panachakel, G.P. Kumar, K. Sharma, A. Ramakrishnan, Automated classification of EEG into meditation and non-meditation epochs using common spatial pattern, linear discriminant analysis, and LSTM, in TENCON 2021-2021 IEEE Region 10 Conference (IEEE, 2021), pp. 1–6
41.
P. Goel, R. Joshi, M. Sur, H.A. Murthy, A common spatial pattern approach for classification of mental counting and motor execution EEG, in International Conference on Intelligent Human Computer Interaction (Springer, 2018), pp. 26–35
42.
J.T. Panachakel, N.N. Vinayak, M. Nunna, A.G. Ramakrishnan, K. Sharma, An improved EEG acquisition protocol facilitates localized neural activation, in Advances in Communication Systems and Networks (Springer, 2020), pp. 267–281
43.
J.T. Panachakel, G.P. Kumar, K. Sharma, A. Ramakrishnan, Binary classification of meditative state from the resting state using EEG, in 2021 IEEE 18th India Council International Conference (INDICON) (IEEE, 2021), pp. 1–6
44.
J.T. Panachakel, A. Ramakrishnan, Decoding imagined speech from EEG using transfer learning. IEEE Access (2021)
45.
J.T. Panachakel, A. Ramakrishnan, T. Ananthapadmanabha, A novel deep learning architecture for decoding imagined speech from EEG. arXiv preprint arXiv:​2003.​09374 (2020)
46.
J.T. Panachakel, A. Ramakrishnan, A. Anusha, K. Sharma, Can we identify the category of imagined phoneme from EEG? in 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (IEEE, 2021)
47.
J.T. Panachakel, A. Ramakrishnan, Classification of phonological categories in imagined speech using phase synchronization measure, in 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (IEEE, 2021)
48.
V. Benzy, A. Vinod, R. Subasree, S. Alladi, K. Raghavendra, Motor imagery hand movement direction decoding using brain computer interface to aid stroke recovery and rehabilitation. IEEE Trans. Neural Syst. Rehabil. Eng. 28(12), 3051–3062 (2020)CrossRef
Metadata
Title
DCLL—A Deep Network for Possible Real-Time Decoding of Imagined Words
Authors
Jerrin Thomas Panachakel
A. G. Ramakrishnan
Copyright Year
2023
Publisher
Springer Nature Singapore
Book
International Symposium on Intelligent Informatics

Print ISBN: 978-981-19-8093-0

Electronic ISBN: 978-981-19-8094-7

Copyright Year: 2023

DOI
https://doi.org/10.1007/978-981-19-8094-7_1