Skip to main content
Top

10-04-2024 | Research Article

An improved CapsNet based on data augmentation for driver vigilance estimation with forehead single-channel EEG

Authors: Huizhou Yang, Jingwen Huang, Yifei Yu, Zhigang Sun, Shouyi Zhang, Yunfei Liu, Han Liu, Lijuan Xia

Published in: Cognitive Neurodynamics

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Various studies have shown that it is necessary to estimate the drivers’ vigilance to reduce the occurrence of traffic accidents. Most existing EEG-based vigilance estimation studies have been performed on intra-subject and multi-channel signals, and these methods are too costly and complicated to implement in practice. Hence, aiming at the problem of cross-subject vigilance estimation of single-channel EEG signals, an estimation algorithm based on capsule network (CapsNet) is proposed. Firstly, we propose a new construction method of the input feature maps to fit the characteristics of CapsNet to improve the algorithm accuracy. Meanwhile, the self-attention mechanism is incorporated in the algorithm to focus on the key information in feature maps. Secondly, we propose substituting the traditional multi-channel signals with the single-channel signals to improve the utility of algorithm. Thirdly, since the single-channel signals carry fewer dimensions of the information compared to the multi-channel signals, we use the conditional generative adversarial network to improve the accuracy of single-channel signals by increasing the amount of data. The proposed algorithm is verified on the SEED-VIG, and Root-mean-square-error (RMSE) and Pearson Correlation Coefficient (PCC) are used as the evaluation metrics. The results show that the proposed algorithm improves the computing speed while the RMSE is reduced by 3%, and the PCC is improved by 12% compared to the mainstream algorithm. Experiment results prove the feasibility of using forehead single-channel EEG signals for cross-subject vigilance estimation and offering the possibility of lightweight EEG vigilance estimation devices for practical applications.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
go back to reference Alioua N, Amine A, Rziza M (2014) Driver’s fatigue detection based on yawning extraction. Int J Veh Technol 2014:47–75 Alioua N, Amine A, Rziza M (2014) Driver’s fatigue detection based on yawning extraction. Int J Veh Technol 2014:47–75
go back to reference Bergasa LM, Nuevo J, Sotelo MA, Vhzquez M (2006) Real-time system for monitoring driver vigilance. IEEE Trans Intell Transp Syst 7:63–77CrossRef Bergasa LM, Nuevo J, Sotelo MA, Vhzquez M (2006) Real-time system for monitoring driver vigilance. IEEE Trans Intell Transp Syst 7:63–77CrossRef
go back to reference Buendia R, Forcolin F, Karlsson J, Sjöqvist BA, Anund A, Candefjord S (2019) Deriving heart rate variability indices from cardiac monitoring-an indicator of driver sleepiness. Traffic Inj Prev 20:249–254CrossRefPubMed Buendia R, Forcolin F, Karlsson J, Sjöqvist BA, Anund A, Candefjord S (2019) Deriving heart rate variability indices from cardiac monitoring-an indicator of driver sleepiness. Traffic Inj Prev 20:249–254CrossRefPubMed
go back to reference Cao Z, Chuang C-H, King J-K, Lin C-T (2019) Multi-channel eeg recordings during a sustained-attention driving task. Sci Data Cao Z, Chuang C-H, King J-K, Lin C-T (2019) Multi-channel eeg recordings during a sustained-attention driving task. Sci Data
go back to reference Chao H, Dong L, Liu Y, Lu B (2019) Emotion recognition from multiband eeg signals using capsnet. Sensors (Switzerland) 19 Chao H, Dong L, Liu Y, Lu B (2019) Emotion recognition from multiband eeg signals using capsnet. Sensors (Switzerland) 19
go back to reference Chen S, Kaili X, Yao X, Ge J, Li L, Zhu S, Li Z (2021) Information fusion and multi-classifier system for miner fatigue recognition in plateau environments based on electrocardiography and electromyography signals. Comput Methods Programs Biomed 211:106451CrossRefPubMed Chen S, Kaili X, Yao X, Ge J, Li L, Zhu S, Li Z (2021) Information fusion and multi-classifier system for miner fatigue recognition in plateau environments based on electrocardiography and electromyography signals. Comput Methods Programs Biomed 211:106451CrossRefPubMed
go back to reference Chuang CH, Ko LW, Lin YP, Jung TP, Lin CT (2014) Independent component ensemble of eeg for brain-computer interface. IEEE Trans Neural Syst Rehabil Eng 22:230–238CrossRefPubMed Chuang CH, Ko LW, Lin YP, Jung TP, Lin CT (2014) Independent component ensemble of eeg for brain-computer interface. IEEE Trans Neural Syst Rehabil Eng 22:230–238CrossRefPubMed
go back to reference Dong Y, Hu Z, Uchimura K, Murayama N (2011) Driver inattention monitoring system for intelligent vehicles: A review. IEEE Trans Intell Transp Syst 12:596–614CrossRef Dong Y, Hu Z, Uchimura K, Murayama N (2011) Driver inattention monitoring system for intelligent vehicles: A review. IEEE Trans Intell Transp Syst 12:596–614CrossRef
go back to reference Flores MJ, Armingol JM, Adl Escalera (2010) Driver drowsiness warning system using visual information for both diurnal and nocturnal illumination conditions. In: Ad Hoc networks Flores MJ, Armingol JM, Adl Escalera (2010) Driver drowsiness warning system using visual information for both diurnal and nocturnal illumination conditions. In: Ad Hoc networks
go back to reference Gao Z, Wang X, Yang Y, Chaoxu M, Cai Q, Dang W, Zuo S (2019) Eeg-based spatio-temporal convolutional neural network for driver fatigue evaluation. IEEE Trans Neural Netw Learn Syst 30:2755–2763CrossRefPubMed Gao Z, Wang X, Yang Y, Chaoxu M, Cai Q, Dang W, Zuo S (2019) Eeg-based spatio-temporal convolutional neural network for driver fatigue evaluation. IEEE Trans Neural Netw Learn Syst 30:2755–2763CrossRefPubMed
go back to reference Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Neural information processing systems Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Neural information processing systems
go back to reference Guarda L, Tapia J, Droguett EL, Ramos M (2022) A novel capsule neural network based model for drowsiness detection using electroencephalography signals. Expert Syst Appl 201 Guarda L, Tapia J, Droguett EL, Ramos M (2022) A novel capsule neural network based model for drowsiness detection using electroencephalography signals. Expert Syst Appl 201
go back to reference He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778
go back to reference Jianchao L, Zheng X, Tang L, Zhang T, Sheng QZ, Wang C, Jin J, Shui Yu, Zhou W (2021) Can steering wheel detect your driving fatigue? IEEE Trans Veh Technol 70:5537–5550CrossRef Jianchao L, Zheng X, Tang L, Zhang T, Sheng QZ, Wang C, Jin J, Shui Yu, Zhou W (2021) Can steering wheel detect your driving fatigue? IEEE Trans Veh Technol 70:5537–5550CrossRef
go back to reference Jiang Y, Zhang Y, Lin C, Dongrui W, Lin C-T (2021) Eeg-based driver drowsiness estimation using an online multi-view and transfer tsk fuzzy system. IEEE Trans Intell Transp Syst 22(3):1752–1764CrossRef Jiang Y, Zhang Y, Lin C, Dongrui W, Lin C-T (2021) Eeg-based driver drowsiness estimation using an online multi-view and transfer tsk fuzzy system. IEEE Trans Intell Transp Syst 22(3):1752–1764CrossRef
go back to reference Jiao Y, Deng Y, Luo Y, Lu BL (2020) Driver sleepiness detection from eeg and eog signals using gan and lstm networks. Neurocomputing 408:100–111CrossRef Jiao Y, Deng Y, Luo Y, Lu BL (2020) Driver sleepiness detection from eeg and eog signals using gan and lstm networks. Neurocomputing 408:100–111CrossRef
go back to reference Kamakura Y, Ohsuga M, Inoue Y, Noguchi Y (2007) Classification of blink waveforms towards the assessment of driver’s arousal level. Trans Soc Autom Engineers Jpn 38:173–178 Kamakura Y, Ohsuga M, Inoue Y, Noguchi Y (2007) Classification of blink waveforms towards the assessment of driver’s arousal level. Trans Soc Autom Engineers Jpn 38:173–178
go back to reference Kingma D, Ba J (2014) Adam: a method for stochastic optimization. Comput Sci Kingma D, Ba J (2014) Adam: a method for stochastic optimization. Comput Sci
go back to reference Ko LW, Komarov O, Lai WK, Liang WG, Jung TP (2020) Eyeblink recognition improves fatigue prediction from single-channel forehead eeg in a realistic sustained attention task. J Neural Eng 17:036015 ((12pp))CrossRefPubMed Ko LW, Komarov O, Lai WK, Liang WG, Jung TP (2020) Eyeblink recognition improves fatigue prediction from single-channel forehead eeg in a realistic sustained attention task. J Neural Eng 17:036015 ((12pp))CrossRefPubMed
go back to reference Kong W, Zhou Z, Jiang B, Babiloni F, Borghini G (2017) Assessment of driving fatigue based on intra/inter-region phase synchronization. Neurocomputing 219:474–482CrossRef Kong W, Zhou Z, Jiang B, Babiloni F, Borghini G (2017) Assessment of driving fatigue based on intra/inter-region phase synchronization. Neurocomputing 219:474–482CrossRef
go back to reference Ko W, Oh K, Jeon E, Suk H-I (2020) Vignet: a deep convolutional neural network for eeg-based driver vigilance estimation. In: 2020 8th International Winter Conference on Brain-Computer Interface (BCI), pp 1–3 Ko W, Oh K, Jeon E, Suk H-I (2020) Vignet: a deep convolutional neural network for eeg-based driver vigilance estimation. In: 2020 8th International Winter Conference on Brain-Computer Interface (BCI), pp 1–3
go back to reference Liu Y, Lan Z, Cui J, Sourina O, Muller-Wittig W (2019) Eeg-based cross-subject mental fatigue recognition. In: Proceedings—2019 international conference on cyberworlds, CW 2019, pp 247–252 Liu Y, Lan Z, Cui J, Sourina O, Muller-Wittig W (2019) Eeg-based cross-subject mental fatigue recognition. In: Proceedings—2019 international conference on cyberworlds, CW 2019, pp 247–252
go back to reference Lu BL, Li H, Zheng WL (2018) Multimodal vigilance estimation with adversarial domain adaptation networks. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp 1–6 Lu BL, Li H, Zheng WL (2018) Multimodal vigilance estimation with adversarial domain adaptation networks. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp 1–6
go back to reference Luo H, Qiu T, Liu C, Huang P (2019) Research on fatigue driving detection using forehead eeg based on adaptive multi-scale entropy. Biomed Signal Process Control 51:50–58CrossRef Luo H, Qiu T, Liu C, Huang P (2019) Research on fatigue driving detection using forehead eeg based on adaptive multi-scale entropy. Biomed Signal Process Control 51:50–58CrossRef
go back to reference Maaten L, Geoffrey H (2008) Visualizing data using t-sne. J Mach Learn Res 9:2579–2605 Maaten L, Geoffrey H (2008) Visualizing data using t-sne. J Mach Learn Res 9:2579–2605
go back to reference Ma B, Li H, Luo Y, Lu BL (2019) Depersonalized cross-subject vigilance estimation with adversarial domain generalization. In: 2019 International joint conference on neural networks (IJCNN), pp 1–8 Ma B, Li H, Luo Y, Lu BL (2019) Depersonalized cross-subject vigilance estimation with adversarial domain generalization. In: 2019 International joint conference on neural networks (IJCNN), pp 1–8
go back to reference Maskeliunas R, Damasevicius R, Martisius I, Vasiljevas M (2016) Consumer-grade eeg devices: Are they usable for control tasks? PeerJ Maskeliunas R, Damasevicius R, Martisius I, Vasiljevas M (2016) Consumer-grade eeg devices: Are they usable for control tasks? PeerJ
go back to reference Murphy-Chutorian E, Trivedi MM (2010) Head pose estimation and augmented reality tracking: an integrated system and evaluation for monitoring driver awareness. IEEE Trans Intell Transp Syst 11:300–311CrossRef Murphy-Chutorian E, Trivedi MM (2010) Head pose estimation and augmented reality tracking: an integrated system and evaluation for monitoring driver awareness. IEEE Trans Intell Transp Syst 11:300–311CrossRef
go back to reference Naeije M, Zorn H (1982) Relation between emg power spectrum shifts and muscle fibre action potential conduction velocity changes during local muscular fatigue in man. Eur J Appl Physiol 50:23–33CrossRef Naeije M, Zorn H (1982) Relation between emg power spectrum shifts and muscle fibre action potential conduction velocity changes during local muscular fatigue in man. Eur J Appl Physiol 50:23–33CrossRef
go back to reference Pei Z, Wang H, Bezerianos A, Li J (2021) Eeg-based multiclass workload identification using feature fusion and selection. IEEE Trans Instru Measure 70:4001108CrossRef Pei Z, Wang H, Bezerianos A, Li J (2021) Eeg-based multiclass workload identification using feature fusion and selection. IEEE Trans Instru Measure 70:4001108CrossRef
go back to reference Ratti E, Waninger S, Berka C, Ruffini G, Verma A (2017) Comparison of medical and consumer wireless eeg systems for use in clinical trials. Front Human Neurosci 11 Ratti E, Waninger S, Berka C, Ruffini G, Verma A (2017) Comparison of medical and consumer wireless eeg systems for use in clinical trials. Front Human Neurosci 11
go back to reference Rogado E, García JL, Barea R, Bergasa LM, López E (2009) Driver fatigue detection system. In: 2008 IEEE International Conference on Robotics and Biomimetics, ROBIO 2008, pp 1105–1110 Rogado E, García JL, Barea R, Bergasa LM, López E (2009) Driver fatigue detection system. In: 2008 IEEE International Conference on Robotics and Biomimetics, ROBIO 2008, pp 1105–1110
go back to reference Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. Proc Adv Neural Inf Process Syst 3856–3866 Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. Proc Adv Neural Inf Process Syst 3856–3866
go back to reference Sikander G, Anwar S (2019) Driver fatigue detection systems: a review. IEEE Trans Intell Transp Syst 20:2339–2352CrossRef Sikander G, Anwar S (2019) Driver fatigue detection systems: a review. IEEE Trans Intell Transp Syst 20:2339–2352CrossRef
go back to reference Trutschel U, Sirois B, Sommer D, Golz M, Edwards D (2011) Perclos: an alertness measure of the past. In: Driving Assessment: International Driving Symposium on Human Factors in Driver Assessment Trutschel U, Sirois B, Sommer D, Golz M, Edwards D (2011) Perclos: an alertness measure of the past. In: Driving Assessment: International Driving Symposium on Human Factors in Driver Assessment
go back to reference Tuncer T, Dogan S, Ertam F, Subasi A (2021) A dynamic center and multi threshold point based stable feature extraction network for driver fatigue detection utilizing eeg signals. Cogn Neurodyn 15:223–237CrossRefPubMed Tuncer T, Dogan S, Ertam F, Subasi A (2021) A dynamic center and multi threshold point based stable feature extraction network for driver fatigue detection utilizing eeg signals. Cogn Neurodyn 15:223–237CrossRefPubMed
go back to reference Vaswani A, Brain G, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need Vaswani A, Brain G, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need
go back to reference Wei W, JonathanWu QM, Sun W, Yang Y, Yuan X, Zheng WL, Lu BL (2021) A regression method with subnetwork neurons for vigilance estimation using eog and eeg. IEEE Trans Cogn Develop Syst 13:209–222CrossRef Wei W, JonathanWu QM, Sun W, Yang Y, Yuan X, Zheng WL, Lu BL (2021) A regression method with subnetwork neurons for vigilance estimation using eog and eeg. IEEE Trans Cogn Develop Syst 13:209–222CrossRef
go back to reference Wu W, Sun W, Jonathan Wu QM, Yang Y, Zhang H, Zheng WL, Lu BL (2022) Multimodal vigilance estimation using deep learning. IEEE Trans Cybern 52:3097–3110CrossRefPubMed Wu W, Sun W, Jonathan Wu QM, Yang Y, Zhang H, Zheng WL, Lu BL (2022) Multimodal vigilance estimation using deep learning. IEEE Trans Cybern 52:3097–3110CrossRefPubMed
go back to reference Yue W, Ji Q (2019) Facial landmark detection: a literature survey. Int J Comput Vision 127:115–142CrossRef Yue W, Ji Q (2019) Facial landmark detection: a literature survey. Int J Comput Vision 127:115–142CrossRef
go back to reference Zhang Y-F, Gao X-Y, Zhu J-Y, Zheng W-L, Lu B-L (2015) A novel approach to driving fatigue detection using forehead eog. In 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER), pp 707–710 Zhang Y-F, Gao X-Y, Zhu J-Y, Zheng W-L, Lu B-L (2015) A novel approach to driving fatigue detection using forehead eog. In 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER), pp 707–710
go back to reference Zhang G, Etemad A (2021) Capsule attention for multimodal eeg-eog representation learning with application to driver vigilance estimation. IEEE Trans Neural Syst Rehabil Eng 29:1138–1149CrossRefPubMed Zhang G, Etemad A (2021) Capsule attention for multimodal eeg-eog representation learning with application to driver vigilance estimation. IEEE Trans Neural Syst Rehabil Eng 29:1138–1149CrossRefPubMed
go back to reference Zhang C, Sun L, Cong F, Kujala T, Ristaniemi T, Parviainen T (2020) Optimal imaging of multi-channel eeg features based on a novel clustering technique for driver fatigue detection. Biomed Signal Process Control 62:102103CrossRef Zhang C, Sun L, Cong F, Kujala T, Ristaniemi T, Parviainen T (2020) Optimal imaging of multi-channel eeg features based on a novel clustering technique for driver fatigue detection. Biomed Signal Process Control 62:102103CrossRef
go back to reference Zhang Y, Guo R, Peng Y, Kong W, Nie F, Bao-Liang L (2022) An auto-weighting incremental random vector functional link network for eeg-based driving fatigue detection. IEEE Trans Instrum Meas 71:1–14 Zhang Y, Guo R, Peng Y, Kong W, Nie F, Bao-Liang L (2022) An auto-weighting incremental random vector functional link network for eeg-based driving fatigue detection. IEEE Trans Instrum Meas 71:1–14
go back to reference Zhang Y, Guo H, Zhou Y, Chengji X, Liao Y (2023) Recognising drivers’ mental fatigue based on eeg multi-dimensional feature selection and fusion. Biomed Signal Process Control 79:104237CrossRef Zhang Y, Guo H, Zhou Y, Chengji X, Liao Y (2023) Recognising drivers’ mental fatigue based on eeg multi-dimensional feature selection and fusion. Biomed Signal Process Control 79:104237CrossRef
go back to reference Zhao L, Li M, He Z, Ye S, Qin H, Zhu X, Dai Z (2022) Data-driven learning fatigue detection system: a multimodal fusion approach of ecg (electrocardiogram) and video signals. Measurement 201:111648CrossRef Zhao L, Li M, He Z, Ye S, Qin H, Zhu X, Dai Z (2022) Data-driven learning fatigue detection system: a multimodal fusion approach of ecg (electrocardiogram) and video signals. Measurement 201:111648CrossRef
go back to reference Zheng WL, Lu BL (2017) A multimodal approach to estimating vigilance using eeg and forehead eog. J Neural Eng 14 Zheng WL, Lu BL (2017) A multimodal approach to estimating vigilance using eeg and forehead eog. J Neural Eng 14
Metadata
Title
An improved CapsNet based on data augmentation for driver vigilance estimation with forehead single-channel EEG
Authors
Huizhou Yang
Jingwen Huang
Yifei Yu
Zhigang Sun
Shouyi Zhang
Yunfei Liu
Han Liu
Lijuan Xia
Publication date
10-04-2024
Publisher
Springer Netherlands
Published in
Cognitive Neurodynamics
Print ISSN: 1871-4080
Electronic ISSN: 1871-4099
DOI
https://doi.org/10.1007/s11571-024-10105-0