Top

Published in:

2022 | OriginalPaper | Chapter

Affective Computing for eHealth Using Low-Cost Remote Internet of Things-Based EMG Platform

Authors : Žygintas Tamulis, Mindaugas Vasiljevas, Robertas Damaševičius, Rytis Maskeliunas, Sanjay Misra

Published in: Intelligent Internet of Things for Healthcare and Industry

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Research in emotion recognition is important in many domains, from industrial ergonomics and health diagnostics to neuromarketing and affective computing. Emotion recognition from facial expression can be performed using image processing of face photos or using directly attached sensors reading the physiological signals from the muscular system governing the facial expression. We describe our experiments with the recognition of affectively induced facial expressions using non-intrusive on-body electromyography (EMG) sensors from the MySignals biosignal acquisition platform. For induction of the affective reaction, we use three different visual stimuli (movie clips) aimed to induce neutral, positive, and negative reactions. The results from 14 subjects show that 97.6% accuracy of emotion recognition was achieved using the covariance matrix features and K-nearest neighbor (KNN) classifier.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Network Protocols for the Internet of Health Things

next chapter Application of the Internet of Things (IoT) to Fight the COVID-19 Pandemic

A.R. Javed, L.G. Fahad, A.A. Farhan, S. Abbas, G. Srivastava, R.M. Parizi, M.S. Khan, Automated cognitive health assessment in smart homes using machine learning. Sustain. Cities Soc., 65 (2021). https://doi.org/10.1016/j.scs.2020.102572

D. Połap, G. Srivastava, K. Yu, Agent architecture of an intelligent medical system based on federated learning and blockchain technology. J. Inf. Secur. Appl., 58 (2021). https://doi.org/10.1016/j.jisa.2021.102748

R.W. Picard, Affective Computing (MIT Press, Cambridge, MA, 1997)

A. Luneski, E. Konstantinidis, P.D. Bamidis, Affective medicine. Methods Inf. Med. 49(3), 207–218 (2010). https://doi.org/10.3414/me061 CrossRef

J.M. Garcia-Garcia, V.M.R. Penichet, M.D. Lozano, J.E. Garrido, E.L.-C. Law, Multimodal affective computing to enhance the user experience of educational software applications. Mob. Inf. Syst., 1–10 (2018). https://doi.org/10.1155/2018/8751426

F. Bevilacqua, H. Engström, P. Backlund, Game-calibrated and user-tailored remote detection of stress and boredom in games. Sensors 19, 2877 (2019)CrossRef

C. Filippini, E. Spadolini, D. Cardone, A. Merla, Thermal imaging based affective computing for educational robot. Proceedings 27, 27 (2019)

G. Liaudanskaitė, G. Saulytė, J. Jakutavičius, E. Vaičiukynaitė, L. Zailskaitė-Jakštė, R. Damaševičius, Analysis of affective and gender factors in image comprehension of visual advertisement. Adv. Intell. Syst. Comput. 764, 1–11 (2019)

P. Ekman, An argument for basic emotions. Cognit. Emot. 6(3–4), 169–200 (1992)CrossRef

10.

P.J. Lang, The emotion probe: Studies of motivation and attention. Am. Psychol. 50(5), 372 (1995)CrossRef

11.

I.B. Mauss, M.D. Robinson, Measures of emotion: A review. Cognit. Emot. 23(2), 209–237 (2009)CrossRef

12.

A. Ree, I. Morrison, H. Olausson, U. Sailer, M. Heilig, L.M. Mayo, Using facial electromyography to assess facial muscle reactions to experienced and observed affective touch in humans. J. Vis. Exp. 145 (2019). https://doi.org/10.3791/59228

13.

E. Coutinho, K. Gentsch, J. Van Peer, K.R. Scherer, B.W. Schuller, Evidence of emotion-antecedent appraisal checks in electroencephalography and facial electromyography. PLoS One 13(1) (2018). https://doi.org/10.1371/journal.pone.0189367

14.

A. Sahayadhas, K. Sundaraj, M. Murugappan, Detecting driver drowsiness based on sensors: A review. Sensors 12(12), 16937–16953 (2012). https://doi.org/10.3390/s121216937 CrossRef

15.

J. Šalkevicius, R. Damaševičius, R. Maskeliunas, I. Laukienė, Anxiety level recognition for virtual reality therapy system using physiological signals. Electronics 8, 1039 (2019)CrossRef

16.

J. Lee, Y. Kagamihara, S. Kakei, Quantitative evaluation of movement disorders in neurological diseases based on EMG signals. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2008, 181–184 (2008). https://doi.org/10.1109/IEMBS.2008.4649120 CrossRef

17.

J. Wijsman, B. Grundlehner, H. Liu, J. Penders, H. Hermens, Wearable physiological sensors reflect mental stress state in office-like situations, in 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, 2013, pp. 600–605

18.

V. Raudonis, R. Maskeliūnas, K. Stankevičius, R. Damaševičius, Gender, age, colour, position and stress: How they influence attention at workplace?, in International Conference on Computational Science and Its Applications, ICCSA 2017, Trieste, Italy, July 3–6 2017, pp. 248–264. https://doi.org/10.1007/978-3-319-62404-4_19

19.

R. Maskeliūnas, T. Blažauskas, R. Damaševičius, Depression behavior detection model based on participation in serious games, in International Joint Conference on Rough Sets, IJCRS 2017, 3–7 July 2017, Olsztyn, Poland, 2017, pp. 423–434

20.

S. Li, G. Walters, J. Packer, N. Scott, Using skin conductance and facial electromyography to measure emotional responses to tourism advertising. Curr. Issue Tour., 1–23 (2016). https://doi.org/10.1080/13683500.2016.1223023

21.

M. Vasiljevas, R. Turcinas, R. Damasevicius, EMG speller with adaptive stimulus rate and dictionary support. in Federated Conference on Computer Science and Information Systems, FedCSIS 2014, Warsaw, Poland, 2014, pp. 227–234. https://doi.org/10.15439/2014F338

22.

M. Vasiljevas, R. Turčinas, R. Damaševičius (2014). Development of EMG-based speller, in ACM International Conference Proceeding Series, 10–12 September 2014

23.

R. Damaševičius, M. Vasiljevas, T. Šumskas, Development of a concept-based EMG-based speller. DYNA (Colombia) 82(193), 170–179 (2015)CrossRef

24.

K. Erin, B. Boru, Real time control of industrial robot arm with EMG and gyroscope data. Sakarya Univ. J. Sci., 1 (2018). https://doi.org/10.16984/saufenbilder.339232

25.

L. Yang, B. Kang, T. Wang, T. Zhao, An ergonomic study of Firefighters’ postural comfort evaluation based on EMG method. Adv. Intell. Syst. Comput. 58(1), 2310–2314 (2014)

26.

V. Balasubramanian, G.S. Prasad, An EMG-based ergonomic evaluation of manual bar bending. Int. J. Ind. Syst. Eng. 2(3), 299 (2007)

27.

K.R. Balasubramanian, Fatigue evaluation in manual handling using surface EMG and ergonomic Design of Trolley. Ergon. Int. J. 2(3) (2018)

28.

M. Motamedzade, D. Afshari, A. Soltanian, The impact of ergonomically designed workstations on shoulder EMG activity during carpet weaving. Health Promot. Perspect. 4(2), 144–150 (2014). https://doi.org/10.5681/hpp.2014.019 CrossRef

29.

J. Kaiser, M.M. Crespo-Llado, C. Turati, E. Geangu, The development of spontaneous facial responses to others’ emotions in infancy: An EMG study. Sci. Rep. 7(1) (2017). https://doi.org/10.1038/s41598-017-17556-y

30.

M. Perusqua-Hernandez, M. Hirokawa, K. Suzuki, A wearable device for fast and subtle spontaneous smile recognition. IEEE Trans. Affect. Comput. 8(4), 522–533

31.

M. Riehle, T.M. Lincoln, Social consequences of subclinical negative symptoms: An EMG study of facial expressions within a social interaction. J. Behav. Ther. Exp. Psychiatry 55, 90–98 (2017). https://doi.org/10.1016/j.jbtep.2017.01.003 CrossRef

32.

Guendil, Z., Lachiri, Z., Maaoui, C., Pruski, A. (2016). Emotion recognition from physiological signals using fusion of wavelet based features, in Proceedings of 7th International Conference on Modelling, Identification and Control, ICMIC 2015, https://doi.org/10.1109/ICMIC.2015.7409485

33.

U. Hess, R. Arslan, H. Mauersberger, C. Blaison, M. Dufner, J.J.A. Denissen, M. Ziegler, Reliability of surface facial electromyography. Psychophysiology 54(1), 12–23 (2017)CrossRef

34.

M.D. Latham, N. Cook, J.G. Simmons, M.L. Byrne, J.W.L. Kettle, O. Schwartz, N.B. Allen, Physiological correlates of emotional reactivity and regulation in early adolescents. Biol. Psychol. (2017). https://doi.org/10.1016/j.biopsycho.2017.07.018

35.

M.S. Perez-Rosero, B. Rezaei, M. Akcakaya, S. Ostadabbas Decoding emotional experiences through physiological signal processing, in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2017, New Orleans, LA, USA, March 5–9, 2017, pp. 881–885. https://doi.org/10.1109/ICASSP.2017.7952282

36.

D. Thompson, I.G. Mackenzie, H. Leuthold, R. Filik, Emotional responses to irony and emoticons in written language: Evidence from EDA and facial EMG. Psychophysiology 53(7), 1054–1062 (2016). https://doi.org/10.1111/psyp.12642 CrossRef

37.

V. Kehri, R. Ingle, S. Patil, R.N. Awale, Analysis of facial EMG signal for emotion recognition using wavelet packet transform and SVM, in Advances in Intelligent Systems and Computing, (Springer, Singapore, 2019), pp. 247–257

38.

S. Orguc, H.S. Khurana, K.M.Stankovic, H.S. Leel, A.P. Chandrakasan, EMG-based real time facial gesture recognition for stress monitoring, in Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2018, pp. 2651–2654

39.

M.M.P. Lajante, O. Droulers, D. Amarantini, How reliable are “state-of-the-art” facial EMG processing methods?: Guidelines for improving the assessment of emotional valence in advertising research. J. Advert. Res. 57(1), 28–37 (2017)CrossRef

40.

Y. Guo, Y. Xia, J. Wang, H. Yu, R.-C. Chen, Real-time facial affective computing on Mobile devices. Sensors 20, 870 (2020)CrossRef

41.

M. Gazzoni, B. Afsharipour, R. Merletti, Surface EMG in ergonomics and occupational medicine, in Surface Electromyography?: Physiology, Engineering, and Applications, (Wiley, London, 2016), pp. 361–391CrossRef

42.

J. Perdiz, G. Pires, U.J. Nunes, Emotional state detection based on EMG and EOG biosignals: A short survey, in 5th Portuguese Meeting on Bioengineering, ENBENG 2017, 16–18 February 2017, Coimbra, Portugal, https://doi.org/10.1109/ENBENG.2017.7889451

43.

A.B. Jani, R. Bagree, A.K. Roy, Design of a low-power, low-cost ECG EMG sensor for wearable biometric and medical application. 2017 IEEE Sensors, 2017, https://doi.org/10.1109/icsens.2017.8234427

44.

M. Jiang, T.N. Gia, A. Anzanpour, A.-M. Rahmani, T. Westerlund, S. Salantera, P. Liljeberg, H. Tenhunen, IoT-based remote facial expression monitoring system with sEMG signal, in 2016 IEEE Sensors Applications Symposium (SAS), 2016

45.

E. De Buyser, E. De ConinckE., B. Dhoedt, P. Simoens, Exploring the potential of combining smart glasses and consumer-grade EEG/EMG headsets for controlling IoT appliances in the smart home, in 2nd IET International Conference on Technologies for Active and Assisted Living (TechAAL 2016), 2016, https://doi.org/10.1049/ic.2016.0053

46.

R. Rastogi, D.K. Chaturvedi, S. Satya, N. Arora, M. Gupta, V. Yadav, S. Chauhan, P. Sharma, Chronic TTH analysis by EMG and GSR biofeedback on various modes and various medical symptoms using IoT, in Big Data Analytics for Intelligent Healthcare Management, (Elsevier, 2019), pp. 87–150. https://doi.org/10.1016/b978-0-12-818146-1.00005-2 CrossRef

47.

M. Zanghieri, S. Benatti, A. Burrello, V. Kartsch, F. Conti, L. Benini, Robust real-time embedded EMG recognition framework using temporal convolutional networks on a multicore IoT processor. IEEE Trans. Biomed. Circuits Syst. 14(2), 244–256 (2020). https://doi.org/10.1109/tbcas.2019.2959160 CrossRef

48.

M. Ali, A.A. Ali, A.-E. Taha, I.B. Dhaou, T.N. Gia, Intelligent autonomous elderly patient home monitoring system, in ICC 2019–2019 IEEE International Conference on Communications (ICC). ICC, 2019. https://doi.org/10.1109/icc.2019.8761204

49.

Libelium MySignals. http://www.my-signals.com/. Accessed: 2/2/2021

50.

E. Lopez-Larraz, O.M. Mozos, , J.M. Antelis, J. Minguez, Syllable-based speech recognition using EMG, in 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, 2010, pp. 4699–4702. https://doi.org/10.1109/IEMBS.2010.5626426

51.

van Boxtel A., Facial EMG as a tool for inferring affective states, in Proceedings of measuring behavior (Noldus Information Technology, Wageningen, 2010), pp. 104–108

52.

R. Damasevicius, M. Vasiljevas, I. Martisius, V. Jusas, D. Birvinskas, M. Wozniak, BoostEMD: An extension of EMD method and its application for denoising of EMG signals. Elektronika ir Elektrotechnika 21(6), 57–61 (2015). https://doi.org/10.5755/j01.eee.21.6.13763 CrossRef

53.

N.E. Huang, Z. Shen, S.R. Long, M.C. Wu, H.H. Shih, Q. Zheng, H.H. Liu, The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc R Soc Lond A Math Phys Eng Sci 454(1971), 903–995 (1998). https://doi.org/10.1098/rspa.1998.0193 MathSciNetCrossRefMATH

54.

M.W. Fagerland, S. Lydersen, P. Laake, The McNemar test for binary matched-pairs data: Mid-p and asymptotic are better than exact conditional. BMC Med. Res. Methodol. 13, 91 (2013). https://doi.org/10.1186/1471-2288-13-91 CrossRef

55.

P. Barros, N. Churamani, E. Lakomkin, H. Siqueira, A. Sutherland, S. Wermter, The OMG-emotion behavior dataset. arXiv preprint arXiv:1803.05434, 2018

56.

K.J. Varcin, S.A. Grainger, J.L. Richmond, P.E. Bailey, J.D. Henry, A role for affectivity in rapid facial mimicry: An electromyographic study. Soc. Neurosci. 14(5), 608–617 (2019). https://doi.org/10.1080/17470919.2018.1564694 CrossRef

Title: Affective Computing for eHealth Using Low-Cost Remote Internet of Things-Based EMG Platform
Authors: Žygintas Tamulis
Mindaugas Vasiljevas
Robertas Damaševičius
Rytis Maskeliunas
Sanjay Misra
Publisher: Springer International Publishing
Book: Intelligent Internet of Things for Healthcare and Industry
Print ISBN: 978-3-030-81472-4

Electronic ISBN: 978-3-030-81473-1

Copyright Year: 2022
DOI: https://doi.org/10.1007/978-3-030-81473-1_3

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner