Skip to main content
Top

2015 | OriginalPaper | Chapter

Landmark-Based Histograms of Oriented Gradients for Facial Emotion Recognition

Authors : Pablo Guerrero, Matías Pavez, Diego Chávez, Sergio F. Ochoa

Published in: Ambient Assisted Living. ICT-based Solutions in Real Life Situations

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The automatic recognition of human emotions is used to support several computing paradigms, like affective, positive and pervasive computing. Histograms of oriented gradients (HOG) have been successfully used with such a purpose, by processing facial images. However, the results of using HOG vary depending on the position of the facial components in the image used as input. This paper presents an extension to the HOG method, which was named Landmark-based Histograms of Oriented Gradients (LaHOG), that not only calculates HOG blocks in the whole face, but also in specific positions around selected facial landmarks. In this sense, the new method is more robust than its predecessor. In order to evaluate the capabilities and limitations of this proposal, we used it to recognize emotions in face images from the FACES database. In such a process we used two classification strategies: support vector machines and logistic regression. The results show that the extended method significantly surpasses the performance of HOG in the tested database.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Ahlberg, J.: Candide-3 - an updated parameterized face. Technical report, LiTH-ISY-R-2326, Department of Electrical Engineering, Linkping University, Sweden (2001) Ahlberg, J.: Candide-3 - an updated parameterized face. Technical report, LiTH-ISY-R-2326, Department of Electrical Engineering, Linkping University, Sweden (2001)
2.
go back to reference Anderson, K., McOwan, P.W.: A real-time automated system for the recognition of human facial expressions. IEEE Trans. Syst. Man Cybern. B Cybern. 36(1), 96–105 (2006)CrossRef Anderson, K., McOwan, P.W.: A real-time automated system for the recognition of human facial expressions. IEEE Trans. Syst. Man Cybern. B Cybern. 36(1), 96–105 (2006)CrossRef
3.
go back to reference Bai, Y., Guo, L., Jin, L., Huang, Q.: A novel feature extraction method using pyramid histogram of orientation gradients for smile recognition. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 3305–3308, November 2009 Bai, Y., Guo, L., Jin, L., Huang, Q.: A novel feature extraction method using pyramid histogram of orientation gradients for smile recognition. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 3305–3308, November 2009
4.
go back to reference Bartlett, M., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J.: Recognizing facial expression: machine learning and application to spontaneous behavior. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 568–573, June 2005 Bartlett, M., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J.: Recognizing facial expression: machine learning and application to spontaneous behavior. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 568–573, June 2005
5.
go back to reference Bettadapura, V.: Face expression recognition and analysis: the state of the art. Technical report, College of Computing, Georgia Institute of Technology (2012) Bettadapura, V.: Face expression recognition and analysis: the state of the art. Technical report, College of Computing, Georgia Institute of Technology (2012)
7.
go back to reference Chen, J., Chen, Z., Chi, Z., Fu, H.: Facial expression recognition based on facial components detection and hog features. In: Scientific Cooperations International Workshops on Electrical and Computer Engineering Subfields, Istanbul, Turkey, August 2014 Chen, J., Chen, Z., Chi, Z., Fu, H.: Facial expression recognition based on facial components detection and hog features. In: Scientific Cooperations International Workshops on Electrical and Computer Engineering Subfields, Istanbul, Turkey, August 2014
8.
go back to reference Dahmane, M., Meunier, J.: Emotion recognition using dynamic grid-based hog features. In: FG, pp. 884–888. IEEE (2011) Dahmane, M., Meunier, J.: Emotion recognition using dynamic grid-based hog features. In: FG, pp. 884–888. IEEE (2011)
9.
go back to reference Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: International Conference on Computer Vision & Pattern Recognition, vol. 2, pp. 886–893, June 2005 Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: International Conference on Computer Vision & Pattern Recognition, vol. 2, pp. 886–893, June 2005
10.
go back to reference Dhall, A., Asthana, A., Goecke, R., Gedeon, T.: Emotion recognition using phog and lpq features. In: FG, pp. 878–883. IEEE (2011) Dhall, A., Asthana, A., Goecke, R., Gedeon, T.: Emotion recognition using phog and lpq features. In: FG, pp. 878–883. IEEE (2011)
11.
go back to reference Ebner, N.C., Riediger, M., Lindenberger, U.: Faces. a database of facial expressions in young, middle-aged, and older women and men: development and validation. Behav. Res. Meth. 42(1), 351–362 (2010)CrossRef Ebner, N.C., Riediger, M., Lindenberger, U.: Faces. a database of facial expressions in young, middle-aged, and older women and men: development and validation. Behav. Res. Meth. 42(1), 351–362 (2010)CrossRef
12.
go back to reference Ekman, P., Friesen, W. (eds.): The Facial Action Coding System. Consulting Psychologists Press, Palo Alto (1978) Ekman, P., Friesen, W. (eds.): The Facial Action Coding System. Consulting Psychologists Press, Palo Alto (1978)
14.
go back to reference Kołakowska, A., Landowska, A., Szwoch, M., Szwoch, W., Wróbel, M.R.: Emotion recognition and its applications. In: Hippe, Z.S., Kulikowski, J.L., Mroczek, T., Wtorek, J. (eds.) Human-Computer Systems Interaction: Backgrounds and Applications 3. AISC, vol. 300, pp. 51–62. Springer, Heidelberg (2014) Kołakowska, A., Landowska, A., Szwoch, M., Szwoch, W., Wróbel, M.R.: Emotion recognition and its applications. In: Hippe, Z.S., Kulikowski, J.L., Mroczek, T., Wtorek, J. (eds.) Human-Computer Systems Interaction: Backgrounds and Applications 3. AISC, vol. 300, pp. 51–62. Springer, Heidelberg (2014)
15.
go back to reference Kotsia, I., Pitas, I.: Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans. Image Process. 16(1), 172–187 (2007)MathSciNetCrossRef Kotsia, I., Pitas, I.: Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans. Image Process. 16(1), 172–187 (2007)MathSciNetCrossRef
16.
go back to reference Li, Z., Ichi Imai, J., Kaneko, M.: Facial-component-based bag of words and phog descriptor for facial expression recognition. In: IEEE International Conference on Systems, Man and Cybernetics, SMC 2009, pp. 1353–1358, October 2009 Li, Z., Ichi Imai, J., Kaneko, M.: Facial-component-based bag of words and phog descriptor for facial expression recognition. In: IEEE International Conference on Systems, Man and Cybernetics, SMC 2009, pp. 1353–1358, October 2009
17.
go back to reference Murphy, K.P.: Machine Learning: A Probabilistic Perspective. The MIT Press, Cambridge (2012) Murphy, K.P.: Machine Learning: A Probabilistic Perspective. The MIT Press, Cambridge (2012)
18.
go back to reference Orrite, C., Gañán, A., Rogez, G.: HOG-based decision tree for facial expression classification. In: Araujo, H., Mendonça, A.M., Pinho, A.J., Torres, M.I. (eds.) IbPRIA 2009. LNCS, vol. 5524, pp. 176–183. Springer, Heidelberg (2009) CrossRef Orrite, C., Gañán, A., Rogez, G.: HOG-based decision tree for facial expression classification. In: Araujo, H., Mendonça, A.M., Pinho, A.J., Torres, M.I. (eds.) IbPRIA 2009. LNCS, vol. 5524, pp. 176–183. Springer, Heidelberg (2009) CrossRef
19.
go back to reference Pantic, M., Patras, I.: Detecting facial actions and their temporal segments in nearly frontal-view face image sequences. In: 2005 IEEE International Conference on Systems, Man and Cybernetics, vol. 4, pp. 3358–3363, October 2005 Pantic, M., Patras, I.: Detecting facial actions and their temporal segments in nearly frontal-view face image sequences. In: 2005 IEEE International Conference on Systems, Man and Cybernetics, vol. 4, pp. 3358–3363, October 2005
20.
go back to reference Pantic, M., Patras, I.: Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences. IEEE Trans. Syst. Man Cybern. Part B Cybern. 36(2), 433–449 (2006)CrossRef Pantic, M., Patras, I.: Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences. IEEE Trans. Syst. Man Cybern. Part B Cybern. 36(2), 433–449 (2006)CrossRef
22.
go back to reference Sinha, P.: Perceiving and recognizing three-dimensional forms. Ph.D. Thesis, Massachusetts Institute of Technology (1995) Sinha, P.: Perceiving and recognizing three-dimensional forms. Ph.D. Thesis, Massachusetts Institute of Technology (1995)
23.
go back to reference Steffens, J., Elagin, E., Neven, H.: Personspotter-fast and robust system for human detection, tracking and recognition. In: Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, pp. 516–521, April 1998 Steffens, J., Elagin, E., Neven, H.: Personspotter-fast and robust system for human detection, tracking and recognition. In: Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, pp. 516–521, April 1998
24.
go back to reference Tao, H., Huang, T.S.: A piecewise Bezier volume deformation model and its applications in facial motion capture. In: Advances in Image Processing and Understanding: A Festschrift for Thomas S. Huang (2002) Tao, H., Huang, T.S.: A piecewise Bezier volume deformation model and its applications in facial motion capture. In: Advances in Image Processing and Understanding: A Festschrift for Thomas S. Huang (2002)
25.
go back to reference Tivatansakul, S., Ohkura, M., Puangpontip, S., Achalakul, T.: Emotional healthcare system: emotion detection by facial expressions using japanese database. In: 2014 6th Computer Science and Electronic Engineering Conference (CEEC), pp. 41–46, September 2014 Tivatansakul, S., Ohkura, M., Puangpontip, S., Achalakul, T.: Emotional healthcare system: emotion detection by facial expressions using japanese database. In: 2014 6th Computer Science and Electronic Engineering Conference (CEEC), pp. 41–46, September 2014
26.
go back to reference Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)CrossRef Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)CrossRef
27.
go back to reference Yu, X., Huang, J., Zhang, S., Yan, W., Metaxas, D.: Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 1944–1951, December 2013 Yu, X., Huang, J., Zhang, S., Yan, W., Metaxas, D.: Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 1944–1951, December 2013
28.
go back to reference Yun, W., Kim, D., Park, C., Kim, J.: Hybrid facial representations for emotion recognition. ETRI J. 35(6), 1021–1028 (2013)CrossRef Yun, W., Kim, D., Park, C., Kim, J.: Hybrid facial representations for emotion recognition. ETRI J. 35(6), 1021–1028 (2013)CrossRef
29.
go back to reference Zheng, W., Zhou, X., Zou, C., Zhao, L.: Facial expression recognition using kernel canonical correlation analysis (KCCA). IEEE Trans. Neural Netw. 17(1), 233–238 (2006)CrossRef Zheng, W., Zhou, X., Zou, C., Zhao, L.: Facial expression recognition using kernel canonical correlation analysis (KCCA). IEEE Trans. Neural Netw. 17(1), 233–238 (2006)CrossRef
Metadata
Title
Landmark-Based Histograms of Oriented Gradients for Facial Emotion Recognition
Authors
Pablo Guerrero
Matías Pavez
Diego Chávez
Sergio F. Ochoa
Copyright Year
2015
DOI
https://doi.org/10.1007/978-3-319-26410-3_27

Premium Partner