Skip to main content
Top
Published in: Granular Computing 1/2022

17-02-2021 | Original Paper

Multiple classifiers fusion for facial expression recognition

Authors: Chuanjie Zhang, Changming Zhu

Published in: Granular Computing | Issue 1/2022

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Human facial expression recognition has been treated as a multi-class classification problem in the field of artificial intelligence. The main difficulty lies in how to distinguish the different categories of expression features. In this paper, we identify common facial expressions by fusing multiple weak classifiers. It compensates for the disadvantage of single classifier in weak generalization ability and low recognition rate for different datasets and different environments. This paper integrates the prediction results of each classifier through improved weighted mean value method and proposes an expression feature extraction method based on keypoint detection. Classifier fusion methods enable each classifier to perform at its best in order to improve overall expression recognition. Keypoint detection is used to improve the model’s attention on the expression features. Convolution neural network is selected as the model for feature extraction and classification, and the model structure is adjusted. Experiments show that the recognition accuracy of this method used on datasets FER 2013 and CK+ are 70.7% and 95.4% respectively, which are better than that of a single classifier, which shows that the keypoint extraction feature and classifier fusion method used in this paper have a good effect on facial expression recognition.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
go back to reference Bera S, Roy SK (2020) Fuzzy rough soft set and its application to lattice. Granular Comput 5(2):217–223CrossRef Bera S, Roy SK (2020) Fuzzy rough soft set and its application to lattice. Granular Comput 5(2):217–223CrossRef
go back to reference Dange AD, Momin B (2019) The cnn and dpm based approach for multiple object detection in images. In: 2019 International conference on intelligent computing and control systems (ICCS), IEEE, pp 1106–1109 Dange AD, Momin B (2019) The cnn and dpm based approach for multiple object detection in images. In: 2019 International conference on intelligent computing and control systems (ICCS), IEEE, pp 1106–1109
go back to reference Egrioglu E, Yolcu U, Bas E (2019) Intuitionistic high-order fuzzy time series forecasting method based on pi-sigma artificial neural networks trained by artificial bee colony. Granular Comput 4(4):639–654CrossRef Egrioglu E, Yolcu U, Bas E (2019) Intuitionistic high-order fuzzy time series forecasting method based on pi-sigma artificial neural networks trained by artificial bee colony. Granular Comput 4(4):639–654CrossRef
go back to reference Ejegwa PA (2020) Generalized triparametric correlation coefficient for pythagorean fuzzy sets with application to mcdm problems. Granular Comput (1–2) Ejegwa PA (2020) Generalized triparametric correlation coefficient for pythagorean fuzzy sets with application to mcdm problems. Granular Comput (1–2)
go back to reference Ekman P, Friesen W (1978) A technique for the measurement of facial actions. Rivista DI Psichiatria 47(2):126–138 Ekman P, Friesen W (1978) A technique for the measurement of facial actions. Rivista DI Psichiatria 47(2):126–138
go back to reference Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:170404861 Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:​170404861
go back to reference Kido S, Hirano Y, Hashimoto N (2018) Detection and classification of lung abnormalities by use of convolutional neural network (cnn) and regions with cnn features (r-cnn). In: 2018 International workshop on advanced image technology (IWAIT), IEEE, pp 1–4 Kido S, Hirano Y, Hashimoto N (2018) Detection and classification of lung abnormalities by use of convolutional neural network (cnn) and regions with cnn features (r-cnn). In: 2018 International workshop on advanced image technology (IWAIT), IEEE, pp 1–4
go back to reference Kong Q, Zhang X, Xu W (2019) Operation properties and algebraic properties of multi-covering rough sets. Granular Comput 4(3):377–390CrossRef Kong Q, Zhang X, Xu W (2019) Operation properties and algebraic properties of multi-covering rough sets. Granular Comput 4(3):377–390CrossRef
go back to reference Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105 Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105
go back to reference Lecun Y, Bottou L (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRef Lecun Y, Bottou L (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRef
go back to reference Liang M, Mi J, Feng T (2019) Optimal granulation selection for multi-label data based on multi-granulation rough sets. Granular Comput 4(3):323–335CrossRef Liang M, Mi J, Feng T (2019) Optimal granulation selection for multi-label data based on multi-granulation rough sets. Granular Comput 4(3):323–335CrossRef
go back to reference Lin JT (1997) Granular computing. Announcement of the BISC Special Interest Group on Granular Computing Lin JT (1997) Granular computing. Announcement of the BISC Special Interest Group on Granular Computing
go back to reference Mehrabian A, Russell JA (1974) An approach to environmental psychology. The MIT Press, Chicago Mehrabian A, Russell JA (1974) An approach to environmental psychology. The MIT Press, Chicago
go back to reference Mollahosseini A, Chan D, Mahoor MH (2016) Going deeper in facial expression recognition using deep neural networks. In: 2016 IEEE Winter conference on applications of computer vision (WACV), IEEE, pp 1–10 Mollahosseini A, Chan D, Mahoor MH (2016) Going deeper in facial expression recognition using deep neural networks. In: 2016 IEEE Winter conference on applications of computer vision (WACV), IEEE, pp 1–10
go back to reference Shi D, Zhang X (2019) Probabilistic decision making based on rough sets in interval-valued fuzzy information systems. Granular Comput 4(3):391–405CrossRef Shi D, Zhang X (2019) Probabilistic decision making based on rough sets in interval-valued fuzzy information systems. Granular Comput 4(3):391–405CrossRef
go back to reference Tsai HH, Chang YC (2018) Facial expression recognition using a combination of multiple facial features and support vector machine. Soft Comput 22(13):4389–4405CrossRef Tsai HH, Chang YC (2018) Facial expression recognition using a combination of multiple facial features and support vector machine. Soft Comput 22(13):4389–4405CrossRef
go back to reference Wu F, Yan S, Smith JS, Zhang B (2019) Deep multiple classifier fusion for traffic scene recognition. Granular Comput pp 1–12 Wu F, Yan S, Smith JS, Zhang B (2019) Deep multiple classifier fusion for traffic scene recognition. Granular Comput pp 1–12
go back to reference Xu LL, Zhang SM, Zhao JL (2019) Expression recognition algorithm for parallel convolutional neural network. J Image Graph 24:227–236 Xu LL, Zhang SM, Zhao JL (2019) Expression recognition algorithm for parallel convolutional neural network. J Image Graph 24:227–236
go back to reference Yang B, Cao JM, Jiang DP, Lv JD (2018) Facial expression recognition based on dual-feature fusion and improved random forest classifier. Multimed Tool Appl 77:20477–20499CrossRef Yang B, Cao JM, Jiang DP, Lv JD (2018) Facial expression recognition based on dual-feature fusion and improved random forest classifier. Multimed Tool Appl 77:20477–20499CrossRef
go back to reference Yao J (2005a) Information granulation and granular relationships. In: 2005 IEEE international conference on granular computing, IEEE, vol 1, pp 326–329 Yao J (2005a) Information granulation and granular relationships. In: 2005 IEEE international conference on granular computing, IEEE, vol 1, pp 326–329
go back to reference Yao JT, Vasilakos AV, Pedrycz W (2013) Granular computing: perspectives and challenges. IEEE Trans Cybern 43(6):1977–1989CrossRef Yao JT, Vasilakos AV, Pedrycz W (2013) Granular computing: perspectives and challenges. IEEE Trans Cybern 43(6):1977–1989CrossRef
go back to reference Yao MZ, Huang GW (2020) Facial expression recognition based on convolutional neural network. Comput Knowl Techol 16(16):19–23 Yao MZ, Huang GW (2020) Facial expression recognition based on convolutional neural network. Comput Knowl Techol 16(16):19–23
go back to reference Yao Y (2005b) Perspectives of granular computing. In: 2005 IEEE international conference on granular computing, IEEE, vol 1, pp 85–90 Yao Y (2005b) Perspectives of granular computing. In: 2005 IEEE international conference on granular computing, IEEE, vol 1, pp 85–90
go back to reference Yurtkan K, Demirel H (2014) Entropy-based feature selection for improved 3d facial expression recognition. SIViP 8(2):267–277CrossRef Yurtkan K, Demirel H (2014) Entropy-based feature selection for improved 3d facial expression recognition. SIViP 8(2):267–277CrossRef
go back to reference Zadeh LA (1979) Fuzzy sets and information granularity. Adv Fuzzy Set Theory Appl 11:3–18MathSciNet Zadeh LA (1979) Fuzzy sets and information granularity. Adv Fuzzy Set Theory Appl 11:3–18MathSciNet
go back to reference Zadeh LA (1996) Key roles of information granulation and fuzzy logic in human reasoning, concept formulation and computing with words. In: Proceedings of IEEE 5th international fuzzy systems, IEEE, vol 1, pp 1–1 Zadeh LA (1996) Key roles of information granulation and fuzzy logic in human reasoning, concept formulation and computing with words. In: Proceedings of IEEE 5th international fuzzy systems, IEEE, vol 1, pp 1–1
go back to reference Zadeh LA (1998) Some reflections on soft computing, granular computing and their roles in the conception, design and utilization of information/intelligent systems. Soft Comput 2(1):23–25CrossRef Zadeh LA (1998) Some reflections on soft computing, granular computing and their roles in the conception, design and utilization of information/intelligent systems. Soft Comput 2(1):23–25CrossRef
go back to reference Zhang W, Wang X, Yang X, Chen X, Wang P (2019) Neighborhood attribute reduction for imbalanced data. Granular Comput 4(3):301–311CrossRef Zhang W, Wang X, Yang X, Chen X, Wang P (2019) Neighborhood attribute reduction for imbalanced data. Granular Comput 4(3):301–311CrossRef
go back to reference Zhao H, Liu H (2020) Multiple classifiers fusion and cnn feature extraction for handwritten digits recognition. Granular Comput 5(3):411–418MathSciNetCrossRef Zhao H, Liu H (2020) Multiple classifiers fusion and cnn feature extraction for handwritten digits recognition. Granular Comput 5(3):411–418MathSciNetCrossRef
Metadata
Title
Multiple classifiers fusion for facial expression recognition
Authors
Chuanjie Zhang
Changming Zhu
Publication date
17-02-2021
Publisher
Springer International Publishing
Published in
Granular Computing / Issue 1/2022
Print ISSN: 2364-4966
Electronic ISSN: 2364-4974
DOI
https://doi.org/10.1007/s41066-021-00258-2

Other articles of this Issue 1/2022

Granular Computing 1/2022 Go to the issue

Premium Partner