Skip to main content
Erschienen in: Granular Computing 1/2022

17.02.2021 | Original Paper

Multiple classifiers fusion for facial expression recognition

verfasst von: Chuanjie Zhang, Changming Zhu

Erschienen in: Granular Computing | Ausgabe 1/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Human facial expression recognition has been treated as a multi-class classification problem in the field of artificial intelligence. The main difficulty lies in how to distinguish the different categories of expression features. In this paper, we identify common facial expressions by fusing multiple weak classifiers. It compensates for the disadvantage of single classifier in weak generalization ability and low recognition rate for different datasets and different environments. This paper integrates the prediction results of each classifier through improved weighted mean value method and proposes an expression feature extraction method based on keypoint detection. Classifier fusion methods enable each classifier to perform at its best in order to improve overall expression recognition. Keypoint detection is used to improve the model’s attention on the expression features. Convolution neural network is selected as the model for feature extraction and classification, and the model structure is adjusted. Experiments show that the recognition accuracy of this method used on datasets FER 2013 and CK+ are 70.7% and 95.4% respectively, which are better than that of a single classifier, which shows that the keypoint extraction feature and classifier fusion method used in this paper have a good effect on facial expression recognition.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Bera S, Roy SK (2020) Fuzzy rough soft set and its application to lattice. Granular Comput 5(2):217–223CrossRef Bera S, Roy SK (2020) Fuzzy rough soft set and its application to lattice. Granular Comput 5(2):217–223CrossRef
Zurück zum Zitat Dange AD, Momin B (2019) The cnn and dpm based approach for multiple object detection in images. In: 2019 International conference on intelligent computing and control systems (ICCS), IEEE, pp 1106–1109 Dange AD, Momin B (2019) The cnn and dpm based approach for multiple object detection in images. In: 2019 International conference on intelligent computing and control systems (ICCS), IEEE, pp 1106–1109
Zurück zum Zitat Egrioglu E, Yolcu U, Bas E (2019) Intuitionistic high-order fuzzy time series forecasting method based on pi-sigma artificial neural networks trained by artificial bee colony. Granular Comput 4(4):639–654CrossRef Egrioglu E, Yolcu U, Bas E (2019) Intuitionistic high-order fuzzy time series forecasting method based on pi-sigma artificial neural networks trained by artificial bee colony. Granular Comput 4(4):639–654CrossRef
Zurück zum Zitat Ejegwa PA (2020) Generalized triparametric correlation coefficient for pythagorean fuzzy sets with application to mcdm problems. Granular Comput (1–2) Ejegwa PA (2020) Generalized triparametric correlation coefficient for pythagorean fuzzy sets with application to mcdm problems. Granular Comput (1–2)
Zurück zum Zitat Ekman P, Friesen W (1978) A technique for the measurement of facial actions. Rivista DI Psichiatria 47(2):126–138 Ekman P, Friesen W (1978) A technique for the measurement of facial actions. Rivista DI Psichiatria 47(2):126–138
Zurück zum Zitat Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:170404861 Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:​170404861
Zurück zum Zitat Kido S, Hirano Y, Hashimoto N (2018) Detection and classification of lung abnormalities by use of convolutional neural network (cnn) and regions with cnn features (r-cnn). In: 2018 International workshop on advanced image technology (IWAIT), IEEE, pp 1–4 Kido S, Hirano Y, Hashimoto N (2018) Detection and classification of lung abnormalities by use of convolutional neural network (cnn) and regions with cnn features (r-cnn). In: 2018 International workshop on advanced image technology (IWAIT), IEEE, pp 1–4
Zurück zum Zitat Kong Q, Zhang X, Xu W (2019) Operation properties and algebraic properties of multi-covering rough sets. Granular Comput 4(3):377–390CrossRef Kong Q, Zhang X, Xu W (2019) Operation properties and algebraic properties of multi-covering rough sets. Granular Comput 4(3):377–390CrossRef
Zurück zum Zitat Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105 Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105
Zurück zum Zitat Lecun Y, Bottou L (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRef Lecun Y, Bottou L (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRef
Zurück zum Zitat Liang M, Mi J, Feng T (2019) Optimal granulation selection for multi-label data based on multi-granulation rough sets. Granular Comput 4(3):323–335CrossRef Liang M, Mi J, Feng T (2019) Optimal granulation selection for multi-label data based on multi-granulation rough sets. Granular Comput 4(3):323–335CrossRef
Zurück zum Zitat Lin JT (1997) Granular computing. Announcement of the BISC Special Interest Group on Granular Computing Lin JT (1997) Granular computing. Announcement of the BISC Special Interest Group on Granular Computing
Zurück zum Zitat Mehrabian A, Russell JA (1974) An approach to environmental psychology. The MIT Press, Chicago Mehrabian A, Russell JA (1974) An approach to environmental psychology. The MIT Press, Chicago
Zurück zum Zitat Mollahosseini A, Chan D, Mahoor MH (2016) Going deeper in facial expression recognition using deep neural networks. In: 2016 IEEE Winter conference on applications of computer vision (WACV), IEEE, pp 1–10 Mollahosseini A, Chan D, Mahoor MH (2016) Going deeper in facial expression recognition using deep neural networks. In: 2016 IEEE Winter conference on applications of computer vision (WACV), IEEE, pp 1–10
Zurück zum Zitat Shi D, Zhang X (2019) Probabilistic decision making based on rough sets in interval-valued fuzzy information systems. Granular Comput 4(3):391–405CrossRef Shi D, Zhang X (2019) Probabilistic decision making based on rough sets in interval-valued fuzzy information systems. Granular Comput 4(3):391–405CrossRef
Zurück zum Zitat Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:14091556 Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:​14091556
Zurück zum Zitat Tsai HH, Chang YC (2018) Facial expression recognition using a combination of multiple facial features and support vector machine. Soft Comput 22(13):4389–4405CrossRef Tsai HH, Chang YC (2018) Facial expression recognition using a combination of multiple facial features and support vector machine. Soft Comput 22(13):4389–4405CrossRef
Zurück zum Zitat Wu F, Yan S, Smith JS, Zhang B (2019) Deep multiple classifier fusion for traffic scene recognition. Granular Comput pp 1–12 Wu F, Yan S, Smith JS, Zhang B (2019) Deep multiple classifier fusion for traffic scene recognition. Granular Comput pp 1–12
Zurück zum Zitat Xu LL, Zhang SM, Zhao JL (2019) Expression recognition algorithm for parallel convolutional neural network. J Image Graph 24:227–236 Xu LL, Zhang SM, Zhao JL (2019) Expression recognition algorithm for parallel convolutional neural network. J Image Graph 24:227–236
Zurück zum Zitat Yang B, Cao JM, Jiang DP, Lv JD (2018) Facial expression recognition based on dual-feature fusion and improved random forest classifier. Multimed Tool Appl 77:20477–20499CrossRef Yang B, Cao JM, Jiang DP, Lv JD (2018) Facial expression recognition based on dual-feature fusion and improved random forest classifier. Multimed Tool Appl 77:20477–20499CrossRef
Zurück zum Zitat Yao J (2005a) Information granulation and granular relationships. In: 2005 IEEE international conference on granular computing, IEEE, vol 1, pp 326–329 Yao J (2005a) Information granulation and granular relationships. In: 2005 IEEE international conference on granular computing, IEEE, vol 1, pp 326–329
Zurück zum Zitat Yao JT, Vasilakos AV, Pedrycz W (2013) Granular computing: perspectives and challenges. IEEE Trans Cybern 43(6):1977–1989CrossRef Yao JT, Vasilakos AV, Pedrycz W (2013) Granular computing: perspectives and challenges. IEEE Trans Cybern 43(6):1977–1989CrossRef
Zurück zum Zitat Yao MZ, Huang GW (2020) Facial expression recognition based on convolutional neural network. Comput Knowl Techol 16(16):19–23 Yao MZ, Huang GW (2020) Facial expression recognition based on convolutional neural network. Comput Knowl Techol 16(16):19–23
Zurück zum Zitat Yao Y (2005b) Perspectives of granular computing. In: 2005 IEEE international conference on granular computing, IEEE, vol 1, pp 85–90 Yao Y (2005b) Perspectives of granular computing. In: 2005 IEEE international conference on granular computing, IEEE, vol 1, pp 85–90
Zurück zum Zitat Yurtkan K, Demirel H (2014) Entropy-based feature selection for improved 3d facial expression recognition. SIViP 8(2):267–277CrossRef Yurtkan K, Demirel H (2014) Entropy-based feature selection for improved 3d facial expression recognition. SIViP 8(2):267–277CrossRef
Zurück zum Zitat Zadeh LA (1979) Fuzzy sets and information granularity. Adv Fuzzy Set Theory Appl 11:3–18MathSciNet Zadeh LA (1979) Fuzzy sets and information granularity. Adv Fuzzy Set Theory Appl 11:3–18MathSciNet
Zurück zum Zitat Zadeh LA (1996) Key roles of information granulation and fuzzy logic in human reasoning, concept formulation and computing with words. In: Proceedings of IEEE 5th international fuzzy systems, IEEE, vol 1, pp 1–1 Zadeh LA (1996) Key roles of information granulation and fuzzy logic in human reasoning, concept formulation and computing with words. In: Proceedings of IEEE 5th international fuzzy systems, IEEE, vol 1, pp 1–1
Zurück zum Zitat Zadeh LA (1998) Some reflections on soft computing, granular computing and their roles in the conception, design and utilization of information/intelligent systems. Soft Comput 2(1):23–25CrossRef Zadeh LA (1998) Some reflections on soft computing, granular computing and their roles in the conception, design and utilization of information/intelligent systems. Soft Comput 2(1):23–25CrossRef
Zurück zum Zitat Zhang W, Wang X, Yang X, Chen X, Wang P (2019) Neighborhood attribute reduction for imbalanced data. Granular Comput 4(3):301–311CrossRef Zhang W, Wang X, Yang X, Chen X, Wang P (2019) Neighborhood attribute reduction for imbalanced data. Granular Comput 4(3):301–311CrossRef
Zurück zum Zitat Zhao H, Liu H (2020) Multiple classifiers fusion and cnn feature extraction for handwritten digits recognition. Granular Comput 5(3):411–418MathSciNetCrossRef Zhao H, Liu H (2020) Multiple classifiers fusion and cnn feature extraction for handwritten digits recognition. Granular Comput 5(3):411–418MathSciNetCrossRef
Metadaten
Titel
Multiple classifiers fusion for facial expression recognition
verfasst von
Chuanjie Zhang
Changming Zhu
Publikationsdatum
17.02.2021
Verlag
Springer International Publishing
Erschienen in
Granular Computing / Ausgabe 1/2022
Print ISSN: 2364-4966
Elektronische ISSN: 2364-4974
DOI
https://doi.org/10.1007/s41066-021-00258-2

Weitere Artikel der Ausgabe 1/2022

Granular Computing 1/2022 Zur Ausgabe