Skip to main content
Top

2022 | OriginalPaper | Chapter

Enhancing EEG-Based Emotion Recognition with Fast Online Instance Transfer

Authors : Hao Chen, Huiguang He, Ting Cai, Jinpeng Li

Published in: Integrating Artificial Intelligence and IoT for Advanced Health Informatics

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The Electroencephalogram (EEG)-based emotion detection relies on extensive training data. Generalization from training to testing is accomplished by collecting enormous labeled samples during training, leading to a time-consuming and laborious calibration procedure. In the last few years, numerous papers have proposed different semi-supervised and transfer learning approaches for reducing the demand for training data. However, most of them are iterative methods and require substantial training time, which is unfeasible in practice. To address this issue, we propose Fast Online Instance Transfer (FOIT) for enhancing the affective brain–computer interface (aBCI). FOIT heuristically selects auxiliary data from historical sessions and (or) other subjects, which are subsequently combined with the training data for supervised training. After that, a multi-classifier ensemble makes the predictions on the test trials. During the training, since FOIT is a one-shot algorithm, it avoids time-consuming iterations that satisfy the demand for fast response of BCIs. Experimental results show that FOIT (˜35 s) significantly decreases the time cost than iterative methods (˜45–900 s). Meanwhile, FOIT still maintains the accuracy, even improves ˜1–14% of accuracy in some settings. Our method provides a straightforward, fast, and practically feasible solution for enhancing the effectiveness of EEG-based emotion recognition, allowing for various choices of classifiers without constraints. The code is available at https://​github.​com/​JC-Journal-Club/​FOIT.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Dolan, R.J.: Emotion, cognition, and behavior. Science 298(5596), 1191–1194 (2002)CrossRef Dolan, R.J.: Emotion, cognition, and behavior. Science 298(5596), 1191–1194 (2002)CrossRef
2.
go back to reference Tyng, C.M., Amin, H.U., Saad, M.N., Malik, A.S.: The influences of emotion on learning and memory. Front. Psychol. 8, 1454 (2017)CrossRef Tyng, C.M., Amin, H.U., Saad, M.N., Malik, A.S.: The influences of emotion on learning and memory. Front. Psychol. 8, 1454 (2017)CrossRef
3.
go back to reference Jeon, M.: Emotions and affect in human factors and human-computer interaction: Taxonomy, theories, approaches, and methods. In: Emotions and Affect in Human Factors and Human-computer Interaction, pp. 3–26 (2017) Jeon, M.: Emotions and affect in human factors and human-computer interaction: Taxonomy, theories, approaches, and methods. In: Emotions and Affect in Human Factors and Human-computer Interaction, pp. 3–26 (2017)
4.
go back to reference Bucks, R.S., Radford, S.A.: Emotion processing in Alzheimer’s disease. Aging Mental Health 8(3), 222–232 (2004)CrossRef Bucks, R.S., Radford, S.A.: Emotion processing in Alzheimer’s disease. Aging Mental Health 8(3), 222–232 (2004)CrossRef
5.
go back to reference Joormann, J., Gotlib, I.H.: Emotion regulation in depression: relation to cognitive inhibition. Cogn. Emotion 24(2), 281–298 (2010)CrossRef Joormann, J., Gotlib, I.H.: Emotion regulation in depression: relation to cognitive inhibition. Cogn. Emotion 24(2), 281–298 (2010)CrossRef
6.
go back to reference Hu, W., Huang, G., Li, L., et al.: Video-triggered EEG-emotion public databases and current methods: A survey. Brain 6(3), 255–287 (2020) Hu, W., Huang, G., Li, L., et al.: Video-triggered EEG-emotion public databases and current methods: A survey. Brain 6(3), 255–287 (2020)
7.
go back to reference Ay, B., Yildirim, O., Talo, M., Baloglu, U.B., Aydin, G., Puthankattil, S.D., Acharya, U.R.: Automated depression detection using deep representation and sequence learning with EEG signals. J. Med. Syst. 43(7), 1–12 (2019)CrossRef Ay, B., Yildirim, O., Talo, M., Baloglu, U.B., Aydin, G., Puthankattil, S.D., Acharya, U.R.: Automated depression detection using deep representation and sequence learning with EEG signals. J. Med. Syst. 43(7), 1–12 (2019)CrossRef
8.
go back to reference Zhang, B., Yan, G., Yang, Z., Su, Y., Wang, J., Lei, T.: Brian functional networks based on resting-state EEG data for major depressive disorder analysis and classification. IEEE Trans. Neural Syst. Rehab. Eng. 29, 215–229 (2020)CrossRef Zhang, B., Yan, G., Yang, Z., Su, Y., Wang, J., Lei, T.: Brian functional networks based on resting-state EEG data for major depressive disorder analysis and classification. IEEE Trans. Neural Syst. Rehab. Eng. 29, 215–229 (2020)CrossRef
9.
go back to reference Lee, S.H., Lee, M., Jeong, J.H., Lee, S.W.: Towards an EEG-based intuitive BCI communication system using imagined speech and visual imagery. In: 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), pp. 4409–4414 (2019) Lee, S.H., Lee, M., Jeong, J.H., Lee, S.W.: Towards an EEG-based intuitive BCI communication system using imagined speech and visual imagery. In: 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), pp. 4409–4414 (2019)
10.
go back to reference Sanei, S., Chambers, J.A.: EEG Signal Processing. Wiley, Chichester (2013) Sanei, S., Chambers, J.A.: EEG Signal Processing. Wiley, Chichester (2013)
12.
go back to reference Hu, W., Huang, G., Li, L., et al.: Video-triggered EEG-emotion public databases and current methods: A survey. Brain 6(3), 255–287 (2020) Hu, W., Huang, G., Li, L., et al.: Video-triggered EEG-emotion public databases and current methods: A survey. Brain 6(3), 255–287 (2020)
13.
go back to reference Comito, C., Forestiero, A., Pizzuti, C.: Word embedding based clustering to detect topics in social media. In: 2019 IEEE/WIC/ACM International Conference on Web Intelligence (WI), pp. 192–199 (2019) Comito, C., Forestiero, A., Pizzuti, C.: Word embedding based clustering to detect topics in social media. In: 2019 IEEE/WIC/ACM International Conference on Web Intelligence (WI), pp. 192–199 (2019)
14.
go back to reference Comito, C.: How COVID-19 information spread in US The Role of Twitter as Early Indicator of Epidemics. IEEE Trans. Serv. Comput. (2021) Preprint Comito, C.: How COVID-19 information spread in US The Role of Twitter as Early Indicator of Epidemics. IEEE Trans. Serv. Comput. (2021) Preprint
15.
go back to reference Wang, X., Chen, X., Cao, C.: Human emotion recognition by optimally fusing facial expression and speech feature. Signal Process. Image Commun. 84, 115831 (2020)CrossRef Wang, X., Chen, X., Cao, C.: Human emotion recognition by optimally fusing facial expression and speech feature. Signal Process. Image Commun. 84, 115831 (2020)CrossRef
16.
go back to reference He, G., Liu, X., Fan, F., You, J.: Image2audio: Facilitating semi-supervised audio emotion recognition with facial expression image. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 912–913 (2020) He, G., Liu, X., Fan, F., You, J.: Image2audio: Facilitating semi-supervised audio emotion recognition with facial expression image. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 912–913 (2020)
17.
go back to reference Dzedzickis, A., Kaklauskas, A., Bucinskas, V.: Human emotion recognition: Review of sensors and methods. Sensors 20(3), 592 (2020)CrossRef Dzedzickis, A., Kaklauskas, A., Bucinskas, V.: Human emotion recognition: Review of sensors and methods. Sensors 20(3), 592 (2020)CrossRef
19.
go back to reference Hu, W., Huang, G., Li, L., et al.: Video-triggered EEG-emotion public databases and current methods: A survey. Brain 6(3), 255–287 (2020) Hu, W., Huang, G., Li, L., et al.: Video-triggered EEG-emotion public databases and current methods: A survey. Brain 6(3), 255–287 (2020)
20.
go back to reference Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., He, Q.: A comprehensive survey on transfer learning. Proc. IEEE 109(1), 43–76 (2020)CrossRef Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., He, Q.: A comprehensive survey on transfer learning. Proc. IEEE 109(1), 43–76 (2020)CrossRef
21.
go back to reference Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. IEEE Trans. Neural Netw. 22(2), 199–210 (2010)CrossRef Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. IEEE Trans. Neural Netw. 22(2), 199–210 (2010)CrossRef
22.
go back to reference Borgwardt, K.M., et al.: Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22(14), e49–e57 (2006)CrossRef Borgwardt, K.M., et al.: Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22(14), e49–e57 (2006)CrossRef
23.
go back to reference Sun, B., Saenko, K.: Deep coral: Correlation alignment for deep domain adaptation. In: European Conference on Computer Vision, pp. 443–450 (2016) Sun, B., Saenko, K.: Deep coral: Correlation alignment for deep domain adaptation. In: European Conference on Computer Vision, pp. 443–450 (2016)
24.
go back to reference Zheng, W.L., Lu, B.L.: Personalizing EEG-based affective models with transfer learning. In: Proceedings of the Twenty-fifth International Joint Conference on Artificial Intelligence, pp. 2732–2738 (2016) Zheng, W.L., Lu, B.L.: Personalizing EEG-based affective models with transfer learning. In: Proceedings of the Twenty-fifth International Joint Conference on Artificial Intelligence, pp. 2732–2738 (2016)
25.
go back to reference Li, H., et al.: Cross-subject emotion recognition using deep adaptation networks. In: International Conference on Neural Information Processing, Siem Reap, December, pp. 403–413 (2018) Li, H., et al.: Cross-subject emotion recognition using deep adaptation networks. In: International Conference on Neural Information Processing, Siem Reap, December, pp. 403–413 (2018)
26.
go back to reference Jin, Y.M., et al.: EEG-based emotion recognition using domain adaptation network. In: 2017 International Conference on Orange Technologies (ICOT), pp. 222–225 (2017) Jin, Y.M., et al.: EEG-based emotion recognition using domain adaptation network. In: 2017 International Conference on Orange Technologies (ICOT), pp. 222–225 (2017)
27.
go back to reference Long, M., et al.: Learning transferable features with deep adaptation networks. In: International Conference on Machine Learning, pp. 97–105 (2015) Long, M., et al.: Learning transferable features with deep adaptation networks. In: International Conference on Machine Learning, pp. 97–105 (2015)
28.
go back to reference Zhang, W., Wu, D.: Manifold embedded knowledge transfer for brain-computer interfaces. IEEE Trans. Neural Syst. Rehab. Eng. 28(5), 1117–1127 (2020)CrossRef Zhang, W., Wu, D.: Manifold embedded knowledge transfer for brain-computer interfaces. IEEE Trans. Neural Syst. Rehab. Eng. 28(5), 1117–1127 (2020)CrossRef
29.
go back to reference Li, J., Qiu, S., Du, C., Wang, Y., He, H.: Domain adaptation for EEG emotion recognition based on latent representation similarity. IEEE Trans. Cogn. Dev. Syst. 12(2), 344–353 (2019)CrossRef Li, J., Qiu, S., Du, C., Wang, Y., He, H.: Domain adaptation for EEG emotion recognition based on latent representation similarity. IEEE Trans. Cogn. Dev. Syst. 12(2), 344–353 (2019)CrossRef
30.
go back to reference Chen, H., Jin, M., Li, Z., Fan, C., Li, J., He, H.: MS-MDA: multisource marginal distribution adaptation for cross-subject and cross-session EEG emotion recognition. Preprint. arXiv: 2107.07740 (2021) Chen, H., Jin, M., Li, Z., Fan, C., Li, J., He, H.: MS-MDA: multisource marginal distribution adaptation for cross-subject and cross-session EEG emotion recognition. Preprint. arXiv: 2107.07740 (2021)
31.
go back to reference Li, J., et al.: Multisource transfer learning for cross-subject EEG emotion recognition. IEEE Trans. Cybern. 50(7), 3281–3293 (2019) Li, J., et al.: Multisource transfer learning for cross-subject EEG emotion recognition. IEEE Trans. Cybern. 50(7), 3281–3293 (2019)
32.
go back to reference Zhao, L.M., Yan, X., Lu, B.L.: Plug-and-play domain adaptation for cross-subject EEG-based emotion recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence (2021) Zhao, L.M., Yan, X., Lu, B.L.: Plug-and-play domain adaptation for cross-subject EEG-based emotion recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence (2021)
33.
go back to reference Liu, S., et al.: Incorporation of multiple-days information to improve the generalization of EEG-based emotion recognition over time. Front. Hum. Neurosci. 12, 267 (2018)CrossRef Liu, S., et al.: Incorporation of multiple-days information to improve the generalization of EEG-based emotion recognition over time. Front. Hum. Neurosci. 12, 267 (2018)CrossRef
35.
go back to reference Zheng, W.L., Lu, B.L.: Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans. Auton. Ment. Dev. 7(3), 162–175 (2015)CrossRef Zheng, W.L., Lu, B.L.: Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans. Auton. Ment. Dev. 7(3), 162–175 (2015)CrossRef
36.
go back to reference Duan, R.N., Zhu, J.Y., Lu, B.L.: Differential entropy feature for EEG-based emotion classification. In: 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), November, pp. 81–84 (2013) Duan, R.N., Zhu, J.Y., Lu, B.L.: Differential entropy feature for EEG-based emotion classification. In: 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), November, pp. 81–84 (2013)
37.
go back to reference Zheng, W.L., et al.: Emotionmeter: A multimodal framework for recognizing human emotions. IEEE Trans. Cybern. 49(3), 1110–1122 (2018)CrossRef Zheng, W.L., et al.: Emotionmeter: A multimodal framework for recognizing human emotions. IEEE Trans. Cybern. 49(3), 1110–1122 (2018)CrossRef
38.
go back to reference Soleynami, M., Asghari-Esfeden, S., Fu, Y., Pantic, M.: Analysis of EEG signals and facial expressions for continuous emotion detection. IEEE Trans. Affect. Comput. 7(1), 17–28 (2015)CrossRef Soleynami, M., Asghari-Esfeden, S., Fu, Y., Pantic, M.: Analysis of EEG signals and facial expressions for continuous emotion detection. IEEE Trans. Affect. Comput. 7(1), 17–28 (2015)CrossRef
Metadata
Title
Enhancing EEG-Based Emotion Recognition with Fast Online Instance Transfer
Authors
Hao Chen
Huiguang He
Ting Cai
Jinpeng Li
Copyright Year
2022
DOI
https://doi.org/10.1007/978-3-030-91181-2_9