Skip to main content

2022 | OriginalPaper | Buchkapitel

Recognition of Facial Patterns Using Surface Electromyography—A Preliminary Study

verfasst von : M. R. Lima, J. J. A. Mendes Júnior, D. P.  Campos

Erschienen in: XXVII Brazilian Congress on Biomedical Engineering

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Facial expressions are considered as a universal language method with non-verbal communication between humans. The surface Electromyography (sEMG) is a technique to acquire and processing signals obtained from the electrical activity of the contraction of voluntary muscles. From that, this work presents a preliminary study of recognition of facial expression obtained from sEMG signals. Signals from 4 subjects were acquired for six basic facial expressions that expresses the following emotions: happiness, surprise, sadness, angry, disgust, and fear. The sEMG signals were acquired from two muscles: zygomaticus major and corrugator of the eyebrow. Three feature sets for sEMG and four different classifiers were evaluated. The best combination was found with the TD9 feature set and SVM with radial basis function kernel classifier, reaching 79.8% of accuracy. Among the expressions, the best recognition was happiness, with individual accuracy of 99.2%. The most difficult expression to classify was the expression of disgust.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
[1]
Zurück zum Zitat Cai Y, Guo Y, Jiang H, Huang M-C (2018) Machine-learning approaches for recognizing muscle activities involved in facial expressions captured by multi-channels surface electromyogram. Smart Health 5–6:15–25CrossRef Cai Y, Guo Y, Jiang H, Huang M-C (2018) Machine-learning approaches for recognizing muscle activities involved in facial expressions captured by multi-channels surface electromyogram. Smart Health 5–6:15–25CrossRef
[2]
Zurück zum Zitat Nasor M, Rahman KKM, Zubair MM, Ansari H, Mohamed F (2018) Eye-controlled mouse cursor for physically disabled individual. In: 2018 advances in science and engineering technology international conferences (ASET), 1–4 Nasor M, Rahman KKM, Zubair MM, Ansari H, Mohamed F (2018) Eye-controlled mouse cursor for physically disabled individual. In: 2018 advances in science and engineering technology international conferences (ASET), 1–4
[3]
Zurück zum Zitat Maria H, Harri P, Arto V (2015) Current state of digital signal processing in myoelectric interfaces and related applications. Biomed Sig Process Control 18:334–359CrossRef Maria H, Harri P, Arto V (2015) Current state of digital signal processing in myoelectric interfaces and related applications. Biomed Sig Process Control 18:334–359CrossRef
[4]
Zurück zum Zitat Nazmi N, Rahman MA, Yamamoto S-I, Ahmad S, Zamzuri H, Mazlan S (2016) A review of classification techniques of EMG signals during isotonic and isometric contractions Sensors 16:1304 Nazmi N, Rahman MA, Yamamoto S-I, Ahmad S, Zamzuri H, Mazlan S (2016) A review of classification techniques of EMG signals during isotonic and isometric contractions Sensors 16:1304
[5]
Zurück zum Zitat Enoka RM Bases neuromecânicas da cinesiologia. Manole 2000. Google-Books-ID: Az7JQwAACAAJ Enoka RM Bases neuromecânicas da cinesiologia. Manole 2000. Google-Books-ID: Az7JQwAACAAJ
[6]
Zurück zum Zitat Kamen G, Gabriel D (2010) Essentials of electromyography. Hum Kinet. Google-Books-ID: 9izzXnnKzJoC Kamen G, Gabriel D (2010) Essentials of electromyography. Hum Kinet. Google-Books-ID: 9izzXnnKzJoC
[7]
Zurück zum Zitat Maisarah S, Aizreena A, Azli Y (2020) Influence of individual physical activity on EMG muscle activation pattern. In: Hassan MHA, Che MAM, Fahriza MAN et al (eds) Enhancing health and sports performance by design. Springer, Lecture Notes in Bioengineering (Singapore), pp 215–222 Maisarah S, Aizreena A, Azli Y (2020) Influence of individual physical activity on EMG muscle activation pattern. In: Hassan MHA, Che MAM, Fahriza MAN et al (eds) Enhancing health and sports performance by design. Springer, Lecture Notes in Bioengineering (Singapore), pp 215–222
[8]
Zurück zum Zitat Aoi M, Kamijo M, Yoshida H (2011) Relationship between facial expression and facial electromyogram (f-EMG) analysis in the expression of drowsiness. In: 2011 international conference on biometrics and kansei engineering, 65–70 Aoi M, Kamijo M, Yoshida H (2011) Relationship between facial expression and facial electromyogram (f-EMG) analysis in the expression of drowsiness. In: 2011 international conference on biometrics and kansei engineering, 65–70
[9]
Zurück zum Zitat Ang LBP, Belen EF, Bernardo RA, Boongaling ER, Briones GH, Coronel JB (2004) Facial expression recognition through pattern analysis of facial muscle movements utilizing electromyogram sensors. In: 2004 IEEE region 10 conference TENCON 2004, vol 3; C:600–603 Ang LBP, Belen EF, Bernardo RA, Boongaling ER, Briones GH, Coronel JB (2004) Facial expression recognition through pattern analysis of facial muscle movements utilizing electromyogram sensors. In: 2004 IEEE region 10 conference TENCON 2004, vol 3; C:600–603
[10]
Zurück zum Zitat Hamedi M, Salleh S-H, Ting C-M, Astaraki M, Noor AM (2018) Robust facial expression recognition for MuCI: a comprehensive neuromuscular signal analysis. IEEE Transactions on Affective Computing. 2018;9:102–115. Conference Name Hamedi M, Salleh S-H, Ting C-M, Astaraki M, Noor AM (2018) Robust facial expression recognition for MuCI: a comprehensive neuromuscular signal analysis. IEEE Transactions on Affective Computing. 2018;9:102–115. Conference Name
[11]
Zurück zum Zitat Mithbavkar SA, Shah MS (2019) Recognition of emotion through facial expressions using EMG signal. In: 2019 international conference on nascent technologies in engineering (ICNTE), pp 1–6 Mithbavkar SA, Shah MS (2019) Recognition of emotion through facial expressions using EMG signal. In: 2019 international conference on nascent technologies in engineering (ICNTE), pp 1–6
[12]
Zurück zum Zitat Passardi S, Peyk P, Rufer M, Wingenbach Tanja SH, Pfaltz Monique C (2019) Facial mimicry, facial emotion recognition and alexithymia in post-traumatic stress disorder. Behav Res Therapy 122:CrossRef Passardi S, Peyk P, Rufer M, Wingenbach Tanja SH, Pfaltz Monique C (2019) Facial mimicry, facial emotion recognition and alexithymia in post-traumatic stress disorder. Behav Res Therapy 122:CrossRef
[13]
Zurück zum Zitat Kordsachia CC, Labuschagne I, Andrews SC, Stout JC (2018) Diminished facial EMG responses to disgusting scenes and happy and fearful faces in Huntington’s disease. Cortex 106:185–199CrossRef Kordsachia CC, Labuschagne I, Andrews SC, Stout JC (2018) Diminished facial EMG responses to disgusting scenes and happy and fearful faces in Huntington’s disease. Cortex 106:185–199CrossRef
[14]
Zurück zum Zitat Hermens HJ, Freriks B, Disselhorst-Klug C, Rau G (2000) Development of recommendations for SEMG sensors and sensor placement procedures. J Electromyogr Kinesiol 10:361–374CrossRef Hermens HJ, Freriks B, Disselhorst-Klug C, Rau G (2000) Development of recommendations for SEMG sensors and sensor placement procedures. J Electromyogr Kinesiol 10:361–374CrossRef
[15]
Zurück zum Zitat Künecke J, Hildebrandt A, Recio G, Sommer W, Wilhelm O (2014) Facial EMG responses to emotional expressions are related to emotion perception ability. PloS one 9:e84053–e84053. 24489647[pmid] Künecke J, Hildebrandt A, Recio G, Sommer W, Wilhelm O (2014) Facial EMG responses to emotional expressions are related to emotion perception ability. PloS one 9:e84053–e84053. 24489647[pmid]
[16]
Zurück zum Zitat Louisa K, Dennis F, Annekathrin S (2020) A comparison of the affectiva iMotions facial expression analysis software with EMG for identifying facial expressions of emotion. Front Psychol 11:329 Louisa K, Dennis F, Annekathrin S (2020) A comparison of the affectiva iMotions facial expression analysis software with EMG for identifying facial expressions of emotion. Front Psychol 11:329
[17]
Zurück zum Zitat Gordon BJ (2013) Anatotomy and phisiology: a case study approach. OpenStax Gordon BJ (2013) Anatotomy and phisiology: a case study approach. OpenStax
[18]
Zurück zum Zitat Hamedi M, Mohammad RI, Firoozabadi M (2011) Facial gesture recognition using two-channel bio-sensors configuration and fuzzy classifier: a pilot study. In: International conference on electrical, control and computer engineering (InECCE), pp 338–343 Hamedi M, Mohammad RI, Firoozabadi M (2011) Facial gesture recognition using two-channel bio-sensors configuration and fuzzy classifier: a pilot study. In: International conference on electrical, control and computer engineering (InECCE), pp 338–343
[19]
Zurück zum Zitat Hamedi M, Salleh S, Tan TS (2011) Surface electromyography-based facial expression recognition in bi-polar configuration. J Comput Sci 7:1407–1415CrossRef Hamedi M, Salleh S, Tan TS (2011) Surface electromyography-based facial expression recognition in bi-polar configuration. J Comput Sci 7:1407–1415CrossRef
[20]
Zurück zum Zitat Hamedi M, Sh-Hussain Salleh, Tan TS et al (2011) Human facial neural activities and gesture recognition for machine-interfacing applications. Int J Nanomed 6:3461–3472 Hamedi M, Sh-Hussain Salleh, Tan TS et al (2011) Human facial neural activities and gesture recognition for machine-interfacing applications. Int J Nanomed 6:3461–3472
[21]
Zurück zum Zitat Scott M (2006) Making comics: storytelling secrets of comics, manga and graphic novels, 1st edn. Harper, New York Scott M (2006) Making comics: storytelling secrets of comics, manga and graphic novels, 1st edn. Harper, New York
[22]
Zurück zum Zitat Ekman P (1992) An argument for basic emotions. Cognit Emotion 6:169–200CrossRef Ekman P (1992) An argument for basic emotions. Cognit Emotion 6:169–200CrossRef
[23]
Zurück zum Zitat Ekman P (2005) Basic emotions, ch. 3. Wiley, pp 45–60 Ekman P (2005) Basic emotions, ch. 3. Wiley, pp 45–60
[24]
Zurück zum Zitat Magda K, Monika W (2017) Basic emotions:1–6. Springer International Publishing, Cham Magda K, Monika W (2017) Basic emotions:1–6. Springer International Publishing, Cham
[25]
Zurück zum Zitat Dušan S, Miloš T, Charles F (2018) Facial expressions in comics: an empirical consideration of McCloud’s proposal. Vis Commun 17:407–432CrossRef Dušan S, Miloš T, Charles F (2018) Facial expressions in comics: an empirical consideration of McCloud’s proposal. Vis Commun 17:407–432CrossRef
[26]
Zurück zum Zitat Wang K, Hoosain R, Lee Tatia MC, Meng Y, Fu J, Yang R (2006) Perception of six basic emotional facial expressions by the Chinese. J Cross-Cult Psychol 37:623–629CrossRef Wang K, Hoosain R, Lee Tatia MC, Meng Y, Fu J, Yang R (2006) Perception of six basic emotional facial expressions by the Chinese. J Cross-Cult Psychol 37:623–629CrossRef
[27]
Zurück zum Zitat Murata A, Saito H, Schug J, Ogawa K, Kameda T (2016) Spontaneous facial mimicry is enhanced by the goal of inferring emotional states: evidence for moderation of “Automatic” mimicry by higher cognitive processes. PLoS ONE 11 Murata A, Saito H, Schug J, Ogawa K, Kameda T (2016) Spontaneous facial mimicry is enhanced by the goal of inferring emotional states: evidence for moderation of “Automatic” mimicry by higher cognitive processes. PLoS ONE 11
[28]
Zurück zum Zitat Oyedotun OK, Demisse G, Shabayek AER, Aouada D, Ottersten B (2017) Facial expression recognition via joint deep learning of RGB-depth map latent representations. In: 2017 IEEE international conference on computer vision workshops (ICCVW), pp 3161–3168 Oyedotun OK, Demisse G, Shabayek AER, Aouada D, Ottersten B (2017) Facial expression recognition via joint deep learning of RGB-depth map latent representations. In: 2017 IEEE international conference on computer vision workshops (ICCVW), pp 3161–3168
[29]
Zurück zum Zitat Spindler O (2003) Affective space interfaces Master’s thesisFakultät für Informatik der Technischen Universität WienA-1040 Wien Spindler O (2003) Affective space interfaces Master’s thesisFakultät für Informatik der Technischen Universität WienA-1040 Wien
[30]
Zurück zum Zitat Ruiliang Su, Chen Xiang, Cao Shuai, Zhang Xu (2016) Random forest-based recognition of isolated sign language subwords using data from accelerometers and surface electromyographic sensors. Sensors 16:100CrossRef Ruiliang Su, Chen Xiang, Cao Shuai, Zhang Xu (2016) Random forest-based recognition of isolated sign language subwords using data from accelerometers and surface electromyographic sensors. Sensors 16:100CrossRef
[31]
Zurück zum Zitat Hudgins B, Parker P, Scott RN (1993) A new strategy for multifunction myoelectric control. IEEE Trans Bio-Med Eng 40:82–94CrossRef Hudgins B, Parker P, Scott RN (1993) A new strategy for multifunction myoelectric control. IEEE Trans Bio-Med Eng 40:82–94CrossRef
[32]
Zurück zum Zitat Phinyomark A, Khushaba Rami N, Scheme E (2018) Feature extraction and selection for myoelectric control based on wearable EMG sensors, vol 18. Sensors (Basel, Switzerland) Phinyomark A, Khushaba Rami N, Scheme E (2018) Feature extraction and selection for myoelectric control based on wearable EMG sensors, vol 18. Sensors (Basel, Switzerland)
[33]
Zurück zum Zitat Theodoridis S, Pikrakis A, Koutroumbas K, Cavouras D (2010) Introduction to pattern recognition: a matlab approach, 1st edn. Academic Press, Burlington Theodoridis S, Pikrakis A, Koutroumbas K, Cavouras D (2010) Introduction to pattern recognition: a matlab approach, 1st edn. Academic Press, Burlington
[34]
Zurück zum Zitat Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications Neurocomputing 70:489–501 Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications Neurocomputing 70:489–501
[36]
Zurück zum Zitat Freitas MLB, Mendes Alves JJ, Campos DP, Stevan SL (2019) Hand gestures classification using multichannel sEMG armband. In: Costa-Felix R, Machado JC, Alvarenga AV (eds) XXVI Brazilian congress on biomedical engineering. IFMBE proceedings, Singapore, Springer, pp 239–246CrossRef Freitas MLB, Mendes Alves JJ, Campos DP, Stevan SL (2019) Hand gestures classification using multichannel sEMG armband. In: Costa-Felix R, Machado JC, Alvarenga AV (eds) XXVI Brazilian congress on biomedical engineering. IFMBE proceedings, Singapore, Springer, pp 239–246CrossRef
[37]
Zurück zum Zitat Mendes JJJA, Freitas Melissa LB, Siqueira Hugo V, Lazzaretti André E, Pichorim Sergio F, Stevan Sergio L (2020) Feature selection and dimensionality reduction: An extensive comparison in hand gesture classification by sEMG in eight channels armband approach. Biomed Sig Process Control 59:CrossRef Mendes JJJA, Freitas Melissa LB, Siqueira Hugo V, Lazzaretti André E, Pichorim Sergio F, Stevan Sergio L (2020) Feature selection and dimensionality reduction: An extensive comparison in hand gesture classification by sEMG in eight channels armband approach. Biomed Sig Process Control 59:CrossRef
[38]
Zurück zum Zitat Kehri V, Ingle R, Patil S, Awale RN (2019) Analysis of facial EMG signal for emotion recognition using wavelet packet transform and SVM. In: Tanveer M, Pachori RB (eds) Machine intelligence and signal analysis. Advances in intelligent systems and computing, Singapore, Springer, pp 247–257CrossRef Kehri V, Ingle R, Patil S, Awale RN (2019) Analysis of facial EMG signal for emotion recognition using wavelet packet transform and SVM. In: Tanveer M, Pachori RB (eds) Machine intelligence and signal analysis. Advances in intelligent systems and computing, Singapore, Springer, pp 247–257CrossRef
[39]
Zurück zum Zitat Jerritta S, Murugappan M, Wan K, Yaacob S (2014) Emotion recognition from facial EMG signals using higher order statistics and principal component analysis. J Chin Inst Eng 37:385–394CrossRef Jerritta S, Murugappan M, Wan K, Yaacob S (2014) Emotion recognition from facial EMG signals using higher order statistics and principal component analysis. J Chin Inst Eng 37:385–394CrossRef
Metadaten
Titel
Recognition of Facial Patterns Using Surface Electromyography—A Preliminary Study
verfasst von
M. R. Lima
J. J. A. Mendes Júnior
D. P.  Campos
Copyright-Jahr
2022
DOI
https://doi.org/10.1007/978-3-030-70601-2_300

Neuer Inhalt