Skip to main content

06.03.2024 | Research Article

Linear and nonlinear analysis of multimodal physiological data for affective arousal recognition

verfasst von: Ali Khaleghi, Kian Shahi, Maryam Saidi, Nafiseh Babaee, Razieh Kaveh, Amin Mohammadian

Erschienen in: Cognitive Neurodynamics

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Objective

In this work we intend to design a system to classify human arousal at five levels (i.e., five stress levels) using four peripheral bio signals including photo-plethysmography measurements (PPG), galvanic skin response (GSR), thorax respiration (TR) and abdominal respiration (AR).

Method

A total of 98 young people voluntarily participated in this study, including 65 men and 33 women with an average age of 24.48 ± 4.26 years. We induced five levels of mental stress in subjects through the Stroop test. A range of physiological features from different analysis domains, including statistical, frequency, and geometrical analyzes, as well as recurrence quantification analysis (RQA) and detrended fluctuation analysis (DFA) were extracted to find out the best arousal-related features and to correlate them with arousal states. Classification of the five arousal levels is performed by a simple naïve Bayes classifier.

Results

Accuracies of 58.45%, 57.1% and 69.13% were obtained using linear features, nonlinear features and a combination of them, respectively. The combination of linear and nonlinear features resulted in the largest average accuracy of 69.13%, ICC of 88.12% and F1 of 69.43% values in the classification of five levels of mental stress.

Conclusion

This paper suggested that combining linear and nonlinear dynamic methods for the analysis of physiological data could help improve the accuracy of the recognition of arousal levels. However, it should be noted that combining multiple modalities (here, PPG, GSR and respiration modalities) by equally weighting them may not always be a good approach to improve accuracy.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat CHEN Y, CUI Y (2017) Review of emotion recognition based on physiological signals. Syst Simul Technol 13(1):1–5 CHEN Y, CUI Y (2017) Review of emotion recognition based on physiological signals. Syst Simul Technol 13(1):1–5
Zurück zum Zitat Chen X, Hao X (2019) Feature reduction method for cognition and classification of IoT devices based on artificial intelligence. IEEE Access 7:103291–103298CrossRef Chen X, Hao X (2019) Feature reduction method for cognition and classification of IoT devices based on artificial intelligence. IEEE Access 7:103291–103298CrossRef
Zurück zum Zitat Dey S et al (2020) A comparative study of support vector machine and Naive Bayes classifier for sentiment analysis on Amazon product reviews. in. International Conference on Contemporary Computing and Applications (IC3A). 2020. IEEE Dey S et al (2020) A comparative study of support vector machine and Naive Bayes classifier for sentiment analysis on Amazon product reviews. in. International Conference on Contemporary Computing and Applications (IC3A). 2020. IEEE
Zurück zum Zitat Dzedzickis A, Kaklauskas A, Bucinskas V (2020) Hum Emot Recognition: Rev Sens Methods Sens 20(3):592 Dzedzickis A, Kaklauskas A, Bucinskas V (2020) Hum Emot Recognition: Rev Sens Methods Sens 20(3):592
Zurück zum Zitat Egger M, Ley M, Hanke S (2019) Emotion recognition from physiological signal analysis: a review. Electron Notes Theor Comput Sci 343:35–55CrossRef Egger M, Ley M, Hanke S (2019) Emotion recognition from physiological signal analysis: a review. Electron Notes Theor Comput Sci 343:35–55CrossRef
Zurück zum Zitat Ganzach Y, Yaor E (2019) The retrospective evaluation of positive and negative affect. Pers Soc Psychol Bull 45(1):93–104CrossRefPubMed Ganzach Y, Yaor E (2019) The retrospective evaluation of positive and negative affect. Pers Soc Psychol Bull 45(1):93–104CrossRefPubMed
Zurück zum Zitat Giannakakis G et al (2019) Review on psychological stress detection using biosignals. IEEE Trans Affect Comput 13(1):440–460CrossRef Giannakakis G et al (2019) Review on psychological stress detection using biosignals. IEEE Trans Affect Comput 13(1):440–460CrossRef
Zurück zum Zitat Gillespie SM, Jones A, Garofalo C (2022) Psychopathy and dangerousness: An umbrella review and meta-analysis Clinical psychology review, : p. 102240 Gillespie SM, Jones A, Garofalo C (2022) Psychopathy and dangerousness: An umbrella review and meta-analysis Clinical psychology review, : p. 102240
Zurück zum Zitat Greco A et al (2016) Arousal and valence recognition of affective sounds based on electrodermal activity. IEEE Sens J 17(3):716–725ADSCrossRef Greco A et al (2016) Arousal and valence recognition of affective sounds based on electrodermal activity. IEEE Sens J 17(3):716–725ADSCrossRef
Zurück zum Zitat Gruszczyńska I et al (2019) Epilepsy identification based on EEG signal using RQA method. Adv Med Sci 64(1):58–64CrossRefPubMed Gruszczyńska I et al (2019) Epilepsy identification based on EEG signal using RQA method. Adv Med Sci 64(1):58–64CrossRefPubMed
Zurück zum Zitat Hong W-C (2020) Phase Space Reconstruction and recurrence plot theory, in Hybrid Intelligent technologies in Energy demand forecasting. Springer, pp 153–179 Hong W-C (2020) Phase Space Reconstruction and recurrence plot theory, in Hybrid Intelligent technologies in Energy demand forecasting. Springer, pp 153–179
Zurück zum Zitat Jain A, Kumar L (2023) Subject-independent trajectory prediction using pre-movement EEG during grasp and lift task. Biomed Signal Process Control 86:105160CrossRef Jain A, Kumar L (2023) Subject-independent trajectory prediction using pre-movement EEG during grasp and lift task. Biomed Signal Process Control 86:105160CrossRef
Zurück zum Zitat Khaleghi A et al (2020) Applicable features of electroencephalogram for ADHD diagnosis. Res Biomedical Eng 36(1):1–11CrossRef Khaleghi A et al (2020) Applicable features of electroencephalogram for ADHD diagnosis. Res Biomedical Eng 36(1):1–11CrossRef
Zurück zum Zitat Khaleghi A et al (2021) A neuronal population model based on cellular automata to simulate the electrical waves of the brain. Waves Random Complex Media, : p. 1–20 Khaleghi A et al (2021) A neuronal population model based on cellular automata to simulate the electrical waves of the brain. Waves Random Complex Media, : p. 1–20
Zurück zum Zitat Khaleghi A et al (2022) Computational neuroscience approach to psychiatry: a review on theory-driven approaches. Clin Psychopharmacol Neurosci 20(1):26CrossRefPubMedPubMedCentral Khaleghi A et al (2022) Computational neuroscience approach to psychiatry: a review on theory-driven approaches. Clin Psychopharmacol Neurosci 20(1):26CrossRefPubMedPubMedCentral
Zurück zum Zitat Khaleghi A et al (2023) Possible neuropathological mechanisms underlying the increased complexity of Brain Electrical Activity in Schizophrenia: a computational study. Iran J Psychiatry, : p. 1–7 Khaleghi A et al (2023) Possible neuropathological mechanisms underlying the increased complexity of Brain Electrical Activity in Schizophrenia: a computational study. Iran J Psychiatry, : p. 1–7
Zurück zum Zitat Kumov V, Samorodov A (2020) Recognition of genetic diseases based on combined feature extraction from 2D face images. in 26th Conference of Open Innovations Association (FRUCT). 2020. IEEE Kumov V, Samorodov A (2020) Recognition of genetic diseases based on combined feature extraction from 2D face images. in 26th Conference of Open Innovations Association (FRUCT). 2020. IEEE
Zurück zum Zitat Li X et al (2016) Emotion recognition from multi-channel EEG data through convolutional recurrent neural network. in. IEEE international conference on bioinformatics and biomedicine (BIBM). 2016. IEEE Li X et al (2016) Emotion recognition from multi-channel EEG data through convolutional recurrent neural network. in. IEEE international conference on bioinformatics and biomedicine (BIBM). 2016. IEEE
Zurück zum Zitat Liu J et al (2016) Emotion detection from EEG recordings. in. 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD). 2016. IEEE Liu J et al (2016) Emotion detection from EEG recordings. in. 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD). 2016. IEEE
Zurück zum Zitat Mehrnam A et al (2017) Reprint of a new approach to analyze data from EEG-based concealed face recognition system. Int J Psychophysiol 122:17–23CrossRefPubMed Mehrnam A et al (2017) Reprint of a new approach to analyze data from EEG-based concealed face recognition system. Int J Psychophysiol 122:17–23CrossRefPubMed
Zurück zum Zitat Mohammadi MR et al (2016) EEG classification of ADHD and normal children using non-linear features and neural network. Biomed Eng Lett 6(2):66–73MathSciNetCrossRef Mohammadi MR et al (2016) EEG classification of ADHD and normal children using non-linear features and neural network. Biomed Eng Lett 6(2):66–73MathSciNetCrossRef
Zurück zum Zitat Ragot M et al (2017) Emotion recognition using physiological signals: laboratory vs. wearable sensors. in International Conference on Applied Human Factors and Ergonomics. Springer Ragot M et al (2017) Emotion recognition using physiological signals: laboratory vs. wearable sensors. in International Conference on Applied Human Factors and Ergonomics. Springer
Zurück zum Zitat Seleznov I, Tukaiev S, Popov A (2019) Detrended fluctuation analysis of the effects of advertisement placement on electroencephalogram. in 2019 Signal Processing Symposium (SPSympo). IEEE Seleznov I, Tukaiev S, Popov A (2019) Detrended fluctuation analysis of the effects of advertisement placement on electroencephalogram. in 2019 Signal Processing Symposium (SPSympo). IEEE
Zurück zum Zitat Shi Y et al (2022) Perception of driving simulations: can the level of Detail of virtual scenarios affect the driver’s behavior and emotions? IEEE Trans Veh Technol 71(4):3429–3442CrossRef Shi Y et al (2022) Perception of driving simulations: can the level of Detail of virtual scenarios affect the driver’s behavior and emotions? IEEE Trans Veh Technol 71(4):3429–3442CrossRef
Zurück zum Zitat Tushar AK, Kabir MA, Ahmed SI (2021) Mental health and sensing. Signal Processing Techniques for Computational Health Informatics, pp 247–260 Tushar AK, Kabir MA, Ahmed SI (2021) Mental health and sensing. Signal Processing Techniques for Computational Health Informatics, pp 247–260
Zurück zum Zitat Valdiviezo-Diaz P et al (2019) A collaborative filtering approach based on Naïve Bayes classifier. IEEE Access 7:108581–108592CrossRef Valdiviezo-Diaz P et al (2019) A collaborative filtering approach based on Naïve Bayes classifier. IEEE Access 7:108581–108592CrossRef
Zurück zum Zitat Valenza G, Lanata A, Scilingo EP (2011) The role of nonlinear dynamics in affective valence and arousal recognition. IEEE Trans Affect Comput 3(2):237–249CrossRef Valenza G, Lanata A, Scilingo EP (2011) The role of nonlinear dynamics in affective valence and arousal recognition. IEEE Trans Affect Comput 3(2):237–249CrossRef
Zurück zum Zitat Wallot S, Mønster D (2018) Calculation of average mutual information (AMI) and false-nearest neighbors (FNN) for the estimation of embedding parameters of multidimensional time series in matlab. Front Psychol 9:1679CrossRefPubMedPubMedCentral Wallot S, Mønster D (2018) Calculation of average mutual information (AMI) and false-nearest neighbors (FNN) for the estimation of embedding parameters of multidimensional time series in matlab. Front Psychol 9:1679CrossRefPubMedPubMedCentral
Zurück zum Zitat Yan Y et al (2022) Topological EEG nonlinear dynamics analysis for emotion recognition. IEEE Transactions on Cognitive and Developmental Systems Yan Y et al (2022) Topological EEG nonlinear dynamics analysis for emotion recognition. IEEE Transactions on Cognitive and Developmental Systems
Zurück zum Zitat Ziaratnia S et al (2023) Comparison of Physiological Responses to Stroop Word Color Test and IAPS Stimulation. in International Conference on Human-Computer Interaction. Springer Ziaratnia S et al (2023) Comparison of Physiological Responses to Stroop Word Color Test and IAPS Stimulation. in International Conference on Human-Computer Interaction. Springer
Metadaten
Titel
Linear and nonlinear analysis of multimodal physiological data for affective arousal recognition
verfasst von
Ali Khaleghi
Kian Shahi
Maryam Saidi
Nafiseh Babaee
Razieh Kaveh
Amin Mohammadian
Publikationsdatum
06.03.2024
Verlag
Springer Netherlands
Erschienen in
Cognitive Neurodynamics
Print ISSN: 1871-4080
Elektronische ISSN: 1871-4099
DOI
https://doi.org/10.1007/s11571-024-10090-4

Neuer Inhalt