Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:

2018 | OriginalPaper | Chapter

Deep Learning Approaches for Facial Emotion Recognition: A Case Study on FER-2013

Authors : Panagiotis Giannopoulos, Isidoros Perikos, Ioannis Hatzilygeroudis

Published in: Advances in Hybridization of Intelligent Methods

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Emotions constitute an innate and important aspect of human behavior that colors the way of human communication. The accurate analysis and interpretation of the emotional content of human facial expressions is essential for the deeper understanding of human behavior. Although a human can detect and interpret faces and facial expressions naturally, with little or no effort, accurate and robust facial expression recognition by computer systems is still a great challenge. The analysis of human face characteristics and the recognition of its emotional states are considered to be very challenging and difficult tasks. The main difficulties come from the non-uniform nature of human face and variations in conditions such as lighting, shadows, facial pose and orientation. Deep learning approaches have been examined as a stream of methods to achieve robustness and provide the necessary scalability on new type of data. In this work, we examine the performance of two known deep learning approaches (GoogLeNet and AlexNet) on facial expression recognition, more specifically the recognition of the existence of emotional content, and on the recognition of the exact emotional content of facial expressions. The results collected from the study are quite interesting.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
next chapter Analysis of Biologically Inspired Swarm Communication Models
Literature
1.
Pantic, M.: Facial expression recognition. In: Encyclopedia of Biometrics, pp. 400–406. Springer, US (2009)
2.
Ekman, P., Rosenberg, E.L. (eds.): What the face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System. Oxford University Press, Oxford, UK (2005)
3.
Mehrabian, A.: Communication without words. Psychol. Today 2(4), 53–56 (1968)
4.
Heylen, D.: Head gestures, gaze and the principles of conversational structure. Int. J. Humanoid Rob. 3(03), 241–267 (2006)CrossRef
5.
Ochs, M., Niewiadomski, R., Pelachaud, C.: Facial Expressions of Emotions for Virtual Characters. The Oxford Handbook of Affective Computing, 261 (2014)
6.
Liebold, B., Richter, R., Teichmann, M., Hamker, F.H., Ohler, P.: Human capacities for emotion recognition and their implications for computer vision. i-com, 14(2), pp. 126–137 (2015)
7.
Clavel, C.: Surprise and human-agent interactions. Rev. Cogn. Linguist. 13(2), 461–477 (2015)CrossRef
8.
Liebold, B., Ohler, P.: Multimodal emotion expressions of virtual agents, mimic and vocal emotion expressions and their effects on emotion recognition. In: Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII), pp. 405–410. IEEE (2013)
9.
Bahreini, K., Nadolski, R., Westera, W.: Towards multimodal emotion recognition in e-learning environments. Interact. Learning Environ. 24(3), 590–605 (2016)CrossRef
10.
Akputu, K.O., Seng, K.P., Lee, Y.L.: Facial emotion recognition for intelligent tutoring environment. In: 2nd International Conference on Machine Learning and Computer Science (IMLCS’2013), pp. 9–13 (2013)
11.
Shen, L., Wang, M., Shen, R.: Affective e—learning: Using “emotional” data to improve learning in pervasive learning environment. Educ. Technol. Soc. 12(2), 176–189 (2009)
12.
Koutlas, A., Fotiadis, D.I.: An automatic region based methodology for facial expression recognition. In: IEEE International Conference on Systems Man and Cybernetics SMC, pp. 662–666 (2008)
13.
Pantic, M., Rothkrantz, L.J.M.: Automatic analysis of facial expressions: The state of the art. Pattern Anal. Mach. Intell. IEEE Trans. 22(12), 1424–1445 (2000)CrossRef
14.
Arca, S., Campadelli, P., Lanzarotti, R.: An automatic feature-based face recognition system. In: Proceedings of the 5th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS’04) (2004)
15.
Ekman, P.: Basic Emotions. Handbook of Cognition and Emotion, pp. 45–60 (1999)
16.
Ortony, A., Clore, G., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press, Cambridge (1988)CrossRef
17.
18.
Wang, W., Yang, J., Xiao, J., Li, S., Zhou, D.: Face recognition based on deep learning. In: International Conference on Human Centered Computing, pp. 812–820. International Publishing, Springer (2014)
19.
Salakhutdinov, R., Hinton, G.: Deep boltzmann machines. In: Artificial Intelligence and Statistics, pp. 448–455 (2009)
20.
Deng, L.: A tutorial survey of architectures, algorithms, and applications for deep learning. APSIPA Trans. Signal Inf. Process. 3, e2 (2014)CrossRef
21.
Srivastava, N., Salakhutdinov, R.R.: Multimodal learning with deep boltzmann machines. In: Advances in Neural Information Processing Systems, pp. 2222–2230 (2012)
22.
Căleanu, C.D.: Face expression recognition: A brief overview of the last decade. In: IEEE 8th International Symposium on Applied Computational Intelligence and Informatics (SACI), pp. 157–161 (2013)
23.
Sariyanidi, E., Gunes, H., Cavallaro, A.: Automatic analysis of facial affect: A survey of registration, representation, and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1113–1133 (2015)CrossRef
24.
Danelakis, A., Theoharis, T., Pratikakis, I.: A survey on facial expression recognition in 3D video sequences. Multimedia Tools Appl. 74(15), 5577–5615 (2015)CrossRef
25.
Aung, D.M., Aye, N.A.: Facial expression classification using histogram based method. In: International Conference on Signal Processing Systems (2012)
26.
Murthy, G.R.S., Jadon, R.S. Recognizing facial expressions using eigenspaces. In: IEEE International Conference on Computational Intelligence and Multimedia Applications. 3, pp. 201–207 (2007)
27.
Thai, L.H., Nguyen, N.D.T., Hai, T.S.: A facial expression classification system integrating canny, principal component analysis and artificial neural network.(2011) arXiv preprint arXiv:​1111.​4052
28.
Perikos, I., Ziakopoulos, E., Hatzilygeroudis, I.: Recognizing emotions from facial expressions using neural network. In: IFIP International Conference on Artificial Intelligence Applications and Innovations, pp. 236–245. Springer, Heidelberg (2014)
29.
Perikos, I., Ziakopoulos, E., & Hatzilygeroudis, I.: Recognize emotions from facial expressions using a SVM and neural network schema. In: Engineering Applications of Neural Networks, pp. 265–274. Springer International Publishing, (2015)
30.
Anderson, K., McOwan, P.W.: A real-time automated system for the recognition of human facial expressions. IEEE Trans. Syst. Man Cybern. Part B (Cybern.), 36(1), 96–105 (2006)
31.
Přinosil, J., Smékal, Z., Esposito, A.: Combining features for recognizing emotional facial expressions in static images. In: Esposito, A., Bourbakis, N.G., Avouris, N., Hatzilygeroudis, I. (eds.) Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction, pp. 56−69. Springer, Heidelberg (2008)
32.
Shinohara, Y., Otsu, N.: Facial expression recognition using fisher weight maps. In: Proceedings Sixth IEEE International Conference on Automatic Face and Gesture Recognition, IEEE. pp. 499–504 (2004)
33.
Yang, J., Zhang, D., Frangi, A.F., Yang, J.Y.: Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 131–137 (2004)CrossRef
34.
Oh, S.K., Yoo, S.H., Pedrycz, W.: Design of face recognition algorithm using PCA-LDA combined for hybrid data pre-processing and polynomial-based RBF neural networks: Design and its application. Expert Syst. Appl. 40(5), 1451–1466 (2013)CrossRef
35.
Mohammadi, M.R., Fatemizadeh, E., Mahoor, M.H.: PCA-based dictionary building for accurate facial expression recognition via sparse representation. J. Vis. Commun. Image Represent. 25(5), 1082–1092 (2014)CrossRef
36.
37.
Ionescu, R.T., Popescu, M., Grozea, C.: Local learning to improve bag of visual words model for facial expression recognition. In: Workshop on Challenges in Representation Learning, ICML (2013)
38.
Mollahosseini, A., Chan, D., & Mahoor, M.H. (2016, March). Going deeper in facial expression recognition using deep neural networks. In: IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE. pp. 1–10 (2016)
39.
Goodfellow, I.J., Erhan, D., Carrier, P.L., Courville, A., Mirza, M., Hamner, B., Zhou, Y.: Challenges in representation learning: A report on three machine learning contests. In: International Conference on Neural Information Processing pp. 117–124. Springer, Heidelberg (2013)
40.
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. (2014) arXiv preprint arXiv:​1409.​4842
41.
Dean, J., Corrado, G., Monga, R., Chen, K., Devin, M., Mao, M., Ng, A.Y.: Large scale distributed deep networks. In: Advances in Neural Information Processing Systems, pp. 1223−1231 (2012)
42.
Krizhevsky, A., Sutskever, I., Hinton. G.E.: ImageNet classification with deep convolutional neural networks. Part of: Adv. Neural Inf. Process. Syst. NIPS, 25 (2012)
43.
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Darrell, T.: Caffe: Convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675−678. ACM (2014)
44.
45.
I. J. Goodfellow, D. Erhan, P. L. Carrier, A. Courville, M. Mirza, B. Hamner, W. Cukierski, Y. Tang, D. Thaler, D.-H. Lee, Y. Zhou, C. Ramaiah, F. Feng, R. Li, X. Wang, D. Athanasakis, J. Shawe-Taylor, M. Milakov, J. Park, R. Ionescu, M. Popescu, C. Grozea, J. Bergstra, J. Xie, L. Romaszko, B. Xu, Z. Chuang, and Y. Bengio, Challenges in representation learning: A report on three machine learning contests, Neural Networks, vol. 64, pp. 59–63 (2015)
Metadata
Title
Deep Learning Approaches for Facial Emotion Recognition: A Case Study on FER-2013
Authors
Panagiotis Giannopoulos
Isidoros Perikos
Ioannis Hatzilygeroudis
Copyright Year
2018
Book
Advances in Hybridization of Intelligent Methods

Print ISBN: 978-3-319-66789-8

Electronic ISBN: 978-3-319-66790-4

Copyright Year: 2018

DOI
https://doi.org/10.1007/978-3-319-66790-4_1

Premium Partner

    Image Credits