Skip to main content
Erschienen in: Cognitive Neurodynamics 6/2018

18.07.2018 | Research Article

EEG classification of driver mental states by deep learning

verfasst von: Hong Zeng, Chen Yang, Guojun Dai, Feiwei Qin, Jianhai Zhang, Wanzeng Kong

Erschienen in: Cognitive Neurodynamics | Ausgabe 6/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Driver fatigue is attracting more and more attention, as it is the main cause of traffic accidents, which bring great harm to society and families. This paper proposes to use deep convolutional neural networks, and deep residual learning, to predict the mental states of drivers from electroencephalography (EEG) signals. Accordingly we have developed two mental state classification models called EEG-Conv and EEG-Conv-R. Tested on intra- and inter-subject, our results show that both models outperform the traditional LSTM- and SVM-based classifiers. Our major findings include (1) Both EEG-Conv and EEG-Conv-R yield very good classification performance for mental state prediction; (2) EEG-Conv-R is more suitable for inter-subject mental state prediction; (3) EEG-Conv-R converges more quickly than EEG-Conv. In summary, our proposed classifiers have better predictive power and are promising for application in practical brain-computer interaction .

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Abdel-Hamid O, Mohamed AR, Jiang H, Deng L, Penn G, Yu D (2014) Convolutional neural networks for speech recognition. IEEE/ACM Trans Audio Speech Lang Pprocess 22(10):1533–1545CrossRef Abdel-Hamid O, Mohamed AR, Jiang H, Deng L, Penn G, Yu D (2014) Convolutional neural networks for speech recognition. IEEE/ACM Trans Audio Speech Lang Pprocess 22(10):1533–1545CrossRef
Zurück zum Zitat Bornas X, Fiolveny A, Balle M, Morillasromero A, Tortellafeliu M (2015) Long range temporal correlations in eeg oscillations of subclinically depressed individuals: their association with brooding and suppression. Cognit Neurodyn 9(1):53–62CrossRef Bornas X, Fiolveny A, Balle M, Morillasromero A, Tortellafeliu M (2015) Long range temporal correlations in eeg oscillations of subclinically depressed individuals: their association with brooding and suppression. Cognit Neurodyn 9(1):53–62CrossRef
Zurück zum Zitat Brookhuis KA, De WD (1993) The use of psychophysiology to assess driver status. Ergonomics 36(9):1099CrossRef Brookhuis KA, De WD (1993) The use of psychophysiology to assess driver status. Ergonomics 36(9):1099CrossRef
Zurück zum Zitat Cecotti H, Graser A (2011) Convolutional neural networks for p300 detection with application to brain-computer interfaces. IEEE Trans Pattern Anal Mach Intell 33(3):433–445CrossRef Cecotti H, Graser A (2011) Convolutional neural networks for p300 detection with application to brain-computer interfaces. IEEE Trans Pattern Anal Mach Intell 33(3):433–445CrossRef
Zurück zum Zitat Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):1–27CrossRef Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):1–27CrossRef
Zurück zum Zitat Chen LL, Zhao Y, Ye PF, Zhang J, Zou JZ (2017) Detecting driving stress in physiological signals based on multimodal feature analysis and kernel classifiers. Expert Syst Appl 85(C):279–291CrossRef Chen LL, Zhao Y, Ye PF, Zhang J, Zou JZ (2017) Detecting driving stress in physiological signals based on multimodal feature analysis and kernel classifiers. Expert Syst Appl 85(C):279–291CrossRef
Zurück zum Zitat Correa AG, Orosco L, Laciar E (2014) Automatic detection of drowsiness in eeg records based on multimodal analysis. Med Eng Phys 36(2):244CrossRef Correa AG, Orosco L, Laciar E (2014) Automatic detection of drowsiness in eeg records based on multimodal analysis. Med Eng Phys 36(2):244CrossRef
Zurück zum Zitat Domhan T, Springenberg JT, Hutter F (2015) Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves. In: IJCAI, pp 3460–3468 Domhan T, Springenberg JT, Hutter F (2015) Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves. In: IJCAI, pp 3460–3468
Zurück zum Zitat Fu R, Wang H (2014) Detection of driving fatigue by using noncontact emg and ecg signals measurement system. Int J Neural Syst 24(03):1450006CrossRef Fu R, Wang H (2014) Detection of driving fatigue by using noncontact emg and ecg signals measurement system. Int J Neural Syst 24(03):1450006CrossRef
Zurück zum Zitat Fu RR, Wang H, Zhao WB (2016) Dynamic driver fatigue detection using hidden markov model in real driving condition. Expert Syst Appl 63(C):397–411CrossRef Fu RR, Wang H, Zhao WB (2016) Dynamic driver fatigue detection using hidden markov model in real driving condition. Expert Syst Appl 63(C):397–411CrossRef
Zurück zum Zitat Gravier A, Quek C, Duch W, Wahab A, Gravier-Rymaszewska J (2016) Neural network modelling of the influence of channelopathies on reflex visual attention. Cognit Neurodyn 10(1):49–72CrossRef Gravier A, Quek C, Duch W, Wahab A, Gravier-Rymaszewska J (2016) Neural network modelling of the influence of channelopathies on reflex visual attention. Cognit Neurodyn 10(1):49–72CrossRef
Zurück zum Zitat Hajinoroozi M, Mao Z, Huang Y (2016) Prediction of driver’s drowsy and alert states from eeg signals with deep learning. In: IEEE international workshop on computational advances in multi-sensor adaptive processing, pp 493–496 Hajinoroozi M, Mao Z, Huang Y (2016) Prediction of driver’s drowsy and alert states from eeg signals with deep learning. In: IEEE international workshop on computational advances in multi-sensor adaptive processing, pp 493–496
Zurück zum Zitat He KM, Zhang XY, Ren SQ, Sun J (2016) Deep residual learning for image recognition. In: Computer vision and pattern recognition, pp 770–778 He KM, Zhang XY, Ren SQ, Sun J (2016) Deep residual learning for image recognition. In: Computer vision and pattern recognition, pp 770–778
Zurück zum Zitat Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780CrossRef Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780CrossRef
Zurück zum Zitat Hu JF (2017) Automated detection of driver fatigue based on adaboost classifier with eeg signals. Front Comput Neurosci 11:72CrossRef Hu JF (2017) Automated detection of driver fatigue based on adaboost classifier with eeg signals. Front Comput Neurosci 11:72CrossRef
Zurück zum Zitat Hu SH, Zheng GT (2009) Driver drowsiness detection with eyelid related parameters by support vector machine. Expert Syst Appl 36(4):7651–7658CrossRef Hu SH, Zheng GT (2009) Driver drowsiness detection with eyelid related parameters by support vector machine. Expert Syst Appl 36(4):7651–7658CrossRef
Zurück zum Zitat Idogawa K (2006) On the brain wave activity of professional drivers during monotonous work. Behaviormetrika 18(30):23–34CrossRef Idogawa K (2006) On the brain wave activity of professional drivers during monotonous work. Behaviormetrika 18(30):23–34CrossRef
Zurück zum Zitat Jap BT, Lal S, Fischer P, Bekiaris E (2009) Using eeg spectral components to assess algorithms for detecting fatigue. Expert Syst Appl 36(2):2352–2359CrossRef Jap BT, Lal S, Fischer P, Bekiaris E (2009) Using eeg spectral components to assess algorithms for detecting fatigue. Expert Syst Appl 36(2):2352–2359CrossRef
Zurück zum Zitat Jeong IC, Lee DH, Park SW, Ko JI, Yoon HR (2007) Automobile driver’s stress index provision system that utilizes electrocardiogram. In: Intelligent vehicles symposium, 2007 IEEE. IEEE, pp 652–656 Jeong IC, Lee DH, Park SW, Ko JI, Yoon HR (2007) Automobile driver’s stress index provision system that utilizes electrocardiogram. In: Intelligent vehicles symposium, 2007 IEEE. IEEE, pp 652–656
Zurück zum Zitat Jung TP, Makeig S, Humphries C, Lee TW, Mckeown MJ, Iragui V, Sejnowski TJ (2000) Removing electroencephalographic artifacts by blind source separation. Psychophysiology 37(2):163–178CrossRef Jung TP, Makeig S, Humphries C, Lee TW, Mckeown MJ, Iragui V, Sejnowski TJ (2000) Removing electroencephalographic artifacts by blind source separation. Psychophysiology 37(2):163–178CrossRef
Zurück zum Zitat Kar S, Bhagat M, Routray A (2010) Eeg signal analysis for the assessment and quantification of drivers fatigue. Transp Res Part F Traffic Psychol Behav 13(5):297–306CrossRef Kar S, Bhagat M, Routray A (2010) Eeg signal analysis for the assessment and quantification of drivers fatigue. Transp Res Part F Traffic Psychol Behav 13(5):297–306CrossRef
Zurück zum Zitat Khushaba RN, Kodagoda S, Lal S, Dissanayake G (2010) Driver drowsiness classification using fuzzy wavelet-packet-based feature-extraction algorithm. IEEE Trans Biomed Eng 58(1):121–131CrossRef Khushaba RN, Kodagoda S, Lal S, Dissanayake G (2010) Driver drowsiness classification using fuzzy wavelet-packet-based feature-extraction algorithm. IEEE Trans Biomed Eng 58(1):121–131CrossRef
Zurück zum Zitat Kong WZ, Zhou ZP, Jiang B, Babiloni F, Borghini G (2017) Assessment of driving fatigue based on intra/inter-region phase synchronization. Neurocomputing 219(5):474–482CrossRef Kong WZ, Zhou ZP, Jiang B, Babiloni F, Borghini G (2017) Assessment of driving fatigue based on intra/inter-region phase synchronization. Neurocomputing 219(5):474–482CrossRef
Zurück zum Zitat Lal SK, Craig A (2001) A critical review of the psychophysiology of driver fatigue. Biol Psychol 55(3):173–194CrossRef Lal SK, Craig A (2001) A critical review of the psychophysiology of driver fatigue. Biol Psychol 55(3):173–194CrossRef
Zurück zum Zitat Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ (2016) Eegnet: a compact convolutional network for EEG-based brain-computer interfaces. arXiv preprint arXiv:1611.08024 Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ (2016) Eegnet: a compact convolutional network for EEG-based brain-computer interfaces. arXiv preprint arXiv:​1611.​08024
Zurück zum Zitat LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444CrossRef LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444CrossRef
Zurück zum Zitat Liang SF, Wang HC, Chang WL (2010) Combination of eeg complexity and spectral analysis for epilepsy diagnosis and seizure detection. Eurasip J Adv Signal Process 2010(1):1–15 Liang SF, Wang HC, Chang WL (2010) Combination of eeg complexity and spectral analysis for epilepsy diagnosis and seizure detection. Eurasip J Adv Signal Process 2010(1):1–15
Zurück zum Zitat Lin CT, Huang KC, Chao CF, Chen JA, Chiu TW, Ko LW, Jung TP (2010) Tonic and phasic eeg and behavioral changes induced by arousing feedback. NeuroImage 52(2):633–642CrossRef Lin CT, Huang KC, Chao CF, Chen JA, Chiu TW, Ko LW, Jung TP (2010) Tonic and phasic eeg and behavioral changes induced by arousing feedback. NeuroImage 52(2):633–642CrossRef
Zurück zum Zitat Lin CT, Wang YK, Chen SA (2014) An eeg-based brain-computer interface for dual task driving detection. Neurocomputing 129(4):85–93 Lin CT, Wang YK, Chen SA (2014) An eeg-based brain-computer interface for dual task driving detection. Neurocomputing 129(4):85–93
Zurück zum Zitat Manor R, Geva AB (2015) Convolutional neural network for multi-category rapid serial visual presentation BCI. Front Comput Neurosci 9:146CrossRef Manor R, Geva AB (2015) Convolutional neural network for multi-category rapid serial visual presentation BCI. Front Comput Neurosci 9:146CrossRef
Zurück zum Zitat Mu ZD, Hu JF, Min JL (2017) Driver fatigue detection system using electroencephalography signals based on combined entropy features. Appl Sci 7(2):150CrossRef Mu ZD, Hu JF, Min JL (2017) Driver fatigue detection system using electroencephalography signals based on combined entropy features. Appl Sci 7(2):150CrossRef
Zurück zum Zitat Page A, Shea C, Mohsenin T (2016) Wearable seizure detection using convolutional neural networks with transfer learning. In: IEEE international symposium on circuits and systems (ISCAS). IEEE, pp 1086–1089 Page A, Shea C, Mohsenin T (2016) Wearable seizure detection using convolutional neural networks with transfer learning. In: IEEE international symposium on circuits and systems (ISCAS). IEEE, pp 1086–1089
Zurück zum Zitat Pal NR, Chuang CY, Ko LW, Chao CF, Jung TP, Liang SF, Lin CT (2008) Eeg-based subject-and session-independent drowsiness detection: an unsupervised approach. EURASIP J Adv Signal Process 2008(1):519480CrossRef Pal NR, Chuang CY, Ko LW, Chao CF, Jung TP, Liang SF, Lin CT (2008) Eeg-based subject-and session-independent drowsiness detection: an unsupervised approach. EURASIP J Adv Signal Process 2008(1):519480CrossRef
Zurück zum Zitat Puanhvuan D, Khemmachotikun S, Wechakarn P, Wijarn B, Wongsawat Y (2017) Navigation-synchronized multimodal control wheelchair from brain to alternative assistive technologies for persons with severe disabilities. Cognit Neurodyn 11(2):117–134CrossRef Puanhvuan D, Khemmachotikun S, Wechakarn P, Wijarn B, Wongsawat Y (2017) Navigation-synchronized multimodal control wheelchair from brain to alternative assistive technologies for persons with severe disabilities. Cognit Neurodyn 11(2):117–134CrossRef
Zurück zum Zitat Qin FW, Gao NN, Peng Y, Wu ZZ, Shen SY, Grudtsin A (2018) Fine-grained leukocyte classification with deep residual learning for microscopic images. Comput Methods Programs Biomed 162(8):243–252CrossRef Qin FW, Gao NN, Peng Y, Wu ZZ, Shen SY, Grudtsin A (2018) Fine-grained leukocyte classification with deep residual learning for microscopic images. Comput Methods Programs Biomed 162(8):243–252CrossRef
Zurück zum Zitat Raghu S, Sriraam N, Kumar GP (2017) Classification of epileptic seizures using wavelet packet log energy and norm entropies with recurrent elman neural network classifier. Cognit Neurodyn 11(1):51–66CrossRef Raghu S, Sriraam N, Kumar GP (2017) Classification of epileptic seizures using wavelet packet log energy and norm entropies with recurrent elman neural network classifier. Cognit Neurodyn 11(1):51–66CrossRef
Zurück zum Zitat Sahayadhas A, Sundaraj K, Murugappan M (2012) Detecting driver drowsiness based on sensors: a review. Sensors 12(12):16937CrossRef Sahayadhas A, Sundaraj K, Murugappan M (2012) Detecting driver drowsiness based on sensors: a review. Sensors 12(12):16937CrossRef
Zurück zum Zitat Sakhavi S, Guan CT, Yan SC (2015) Parallel convolutional-linear neural network for motor imagery classification. In: Signal processing conference (EUSIPCO). IEEE, pp 2736–2740 Sakhavi S, Guan CT, Yan SC (2015) Parallel convolutional-linear neural network for motor imagery classification. In: Signal processing conference (EUSIPCO). IEEE, pp 2736–2740
Zurück zum Zitat Schirrmeister RT, Springenberg JT, Fiederer LDJ, Glasstetter M, Eggensperger K, Tangermann M, Hutter F, Burgard W, Ball T (2017) Deep learning with convolutional neural networks for brain mapping and decoding of movement-related information from the human eeg. arXiv preprint arXiv:1703.05051 Schirrmeister RT, Springenberg JT, Fiederer LDJ, Glasstetter M, Eggensperger K, Tangermann M, Hutter F, Burgard W, Ball T (2017) Deep learning with convolutional neural networks for brain mapping and decoding of movement-related information from the human eeg. arXiv preprint arXiv:​1703.​05051
Zurück zum Zitat Schoenberg PLA, Speckens AEM (2015) Multi-dimensional modulations of alpha and gamma cortical dynamics following mindfulness-based cognitive therapy in major depressive disorder. Cognit Neurodyn 9(1):13–29CrossRef Schoenberg PLA, Speckens AEM (2015) Multi-dimensional modulations of alpha and gamma cortical dynamics following mindfulness-based cognitive therapy in major depressive disorder. Cognit Neurodyn 9(1):13–29CrossRef
Zurück zum Zitat Stein D, Orbach ISM, Har ED, Yaruslasky A, Roth D, Meged S, Apter A (2013) Eeg alpha band synchrony predicts cognitive and motor performance in patients with ischemic stroke. Behav Neurol 26(3):187CrossRef Stein D, Orbach ISM, Har ED, Yaruslasky A, Roth D, Meged S, Apter A (2013) Eeg alpha band synchrony predicts cognitive and motor performance in patients with ischemic stroke. Behav Neurol 26(3):187CrossRef
Zurück zum Zitat Tang ZC, Li C, Sun SQ (2017) Single-trial eeg classification of motor imagery using deep convolutional neural networks. Opt Int J Light Electron Opt 130:11–18CrossRef Tang ZC, Li C, Sun SQ (2017) Single-trial eeg classification of motor imagery using deep convolutional neural networks. Opt Int J Light Electron Opt 130:11–18CrossRef
Zurück zum Zitat Thodoroff P, Pineau J, Lim A (2016) Learning robust features using deep learning for automatic seizure detection. In: Machine learning for healthcare conference, pp 178–190 Thodoroff P, Pineau J, Lim A (2016) Learning robust features using deep learning for automatic seizure detection. In: Machine learning for healthcare conference, pp 178–190
Zurück zum Zitat Tsuchida A, Bhuiyan M, Oguri K (2009) Estimation of drowsiness level based on eyelid closure and heart rate variability. In: EMBC 2009 international conference of the IEEE Engineering in Medicine and Biology Society, 2009, pp 2543–2546 Tsuchida A, Bhuiyan M, Oguri K (2009) Estimation of drowsiness level based on eyelid closure and heart rate variability. In: EMBC 2009 international conference of the IEEE Engineering in Medicine and Biology Society, 2009, pp 2543–2546
Zurück zum Zitat Wali MK, Murugappan M, Ahmmad B (2013) Wavelet packet transform based driver distraction level classification using eeg. Math Probl Eng 2013(3):841–860 Wali MK, Murugappan M, Ahmmad B (2013) Wavelet packet transform based driver distraction level classification using eeg. Math Probl Eng 2013(3):841–860
Zurück zum Zitat Zeng H, Dai GJ, Kong WZ, Chen FY, Wang LY (2017) A novel nonlinear dynamic method for stroke rehabilitation effect evaluation using eeg. IEEE Trans Neural Syst Rehabil Eng 25(12):2488–2497CrossRef Zeng H, Dai GJ, Kong WZ, Chen FY, Wang LY (2017) A novel nonlinear dynamic method for stroke rehabilitation effect evaluation using eeg. IEEE Trans Neural Syst Rehabil Eng 25(12):2488–2497CrossRef
Zurück zum Zitat Zhang JH, Li SN, Wang RB (2017) Pattern recognition of momentary mental workload based on multi-channel electrophysiological data and ensemble convolutional neural networks. Front Neurosci 11:310CrossRef Zhang JH, Li SN, Wang RB (2017) Pattern recognition of momentary mental workload based on multi-channel electrophysiological data and ensemble convolutional neural networks. Front Neurosci 11:310CrossRef
Zurück zum Zitat Zhang JH, Cui XQ, Li JR, Wang RB (2017) Imbalanced classification of mental workload using a cost-sensitive majority weighted minority oversampling strategy. Cognit Technol Work 19(4):633–653CrossRef Zhang JH, Cui XQ, Li JR, Wang RB (2017) Imbalanced classification of mental workload using a cost-sensitive majority weighted minority oversampling strategy. Cognit Technol Work 19(4):633–653CrossRef
Zurück zum Zitat Zhao CL, Zheng CX, Zhao M, Liu JP (2010) Physiological assessment of driving mental fatigue using wavelet packet energy and random forests. Am J Biomed Sci 2(3):262–274CrossRef Zhao CL, Zheng CX, Zhao M, Liu JP (2010) Physiological assessment of driving mental fatigue using wavelet packet energy and random forests. Am J Biomed Sci 2(3):262–274CrossRef
Metadaten
Titel
EEG classification of driver mental states by deep learning
verfasst von
Hong Zeng
Chen Yang
Guojun Dai
Feiwei Qin
Jianhai Zhang
Wanzeng Kong
Publikationsdatum
18.07.2018
Verlag
Springer Netherlands
Erschienen in
Cognitive Neurodynamics / Ausgabe 6/2018
Print ISSN: 1871-4080
Elektronische ISSN: 1871-4099
DOI
https://doi.org/10.1007/s11571-018-9496-y

Weitere Artikel der Ausgabe 6/2018

Cognitive Neurodynamics 6/2018 Zur Ausgabe

Neuer Inhalt