Skip to main content
Top

2014 | OriginalPaper | Chapter

Dynamic Visual Time Context Descriptors for Automatic Human Expression Classification

Authors : Yi Ji, Shengrong Gong, Chunping Liu

Published in: Foundations and Practical Applications of Cognitive Systems and Information Processing

Publisher: Springer Berlin Heidelberg

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In this paper, we propose two fast dynamic descriptors Vertical-Time-Backward (VTB) and Vertical-Time-Forward (VTF) on spatial–temporal domain to catch the cues of essential facial movements. These dynamic descriptors are used in a two-step system to recognize human facial expression within image sequences. In the first step, the system classifies static images and then it identifies the whole sequence. After combining the visual-time context features with popular LBP, the system can efficiently recognize the expression in a single image, and is especially helpful in highly ambiguous ones. In the second step, we use the evaluation method through the weighted probabilities of all frames to predict the class of the whole sequence. The experiments were performed on 348 sequences from 95 subjects in Cohn–Kanade database and obtained good results as high as 97.6 % in seven-class recognition for frames and 95.7 % in six class for sequences.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Ekman P, Friesen W (1978) Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto Ekman P, Friesen W (1978) Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto
2.
go back to reference Pantic M, Rothkrantz L (2000) Automatic analysis of facial expressions: the state of the art. IEEE Trans Pattern Anal Mach Intell 22:1424–1445CrossRef Pantic M, Rothkrantz L (2000) Automatic analysis of facial expressions: the state of the art. IEEE Trans Pattern Anal Mach Intell 22:1424–1445CrossRef
3.
go back to reference Huang D, Shan C, Ardabilian M, Wang Y, Chen L (2011) Local binary patterns and its application to facial image analysis: a survey. IEEE Trans Syst Man Cybern Part C: Appl Rev 41(4):1–17 Huang D, Shan C, Ardabilian M, Wang Y, Chen L (2011) Local binary patterns and its application to facial image analysis: a survey. IEEE Trans Syst Man Cybern Part C: Appl Rev 41(4):1–17
4.
go back to reference Chang K, Liu T, Lai S (2009) Learning partially-observed hidden conditional random fields for facial expression recognition. In: CVPR, Miami, pp 533–540 Chang K, Liu T, Lai S (2009) Learning partially-observed hidden conditional random fields for facial expression recognition. In: CVPR, Miami, pp 533–540
5.
go back to reference Bartlett M, Littlewort G, Lainscsek C, Fasel I, Frank M, Movellan J (2005) Fully automatic facial action recognition in spontaneous behavior. In: Automatic Face and Gesture Recognition, pp 223–228 Bartlett M, Littlewort G, Lainscsek C, Fasel I, Frank M, Movellan J (2005) Fully automatic facial action recognition in spontaneous behavior. In: Automatic Face and Gesture Recognition, pp 223–228
6.
go back to reference Shan C, Gong S, McOwan P (May 2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis Comput 27(6):803–816 Shan C, Gong S, McOwan P (May 2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis Comput 27(6):803–816
7.
go back to reference Zhao G, Ahonen T, Matas J, Pietikainen M (2012) Rotation-invariant image and video description with local binary pattern features. Image Process, IEEE Trans 21(4):1465–1477 Zhao G, Ahonen T, Matas J, Pietikainen M (2012) Rotation-invariant image and video description with local binary pattern features. Image Process, IEEE Trans 21(4):1465–1477
8.
go back to reference Yeasin M, Bullot B, Sharma R (2004) From facial expression to level of interest: a spatio-temporal approach. CVPR 2:922–927 Yeasin M, Bullot B, Sharma R (2004) From facial expression to level of interest: a spatio-temporal approach. CVPR 2:922–927
9.
go back to reference Xiang T, Leung M, Cho S (2008) Expression recognition using fuzzy spatio-temporal modeling. Pattern Recognit 41(1):204–216CrossRefMATH Xiang T, Leung M, Cho S (2008) Expression recognition using fuzzy spatio-temporal modeling. Pattern Recognit 41(1):204–216CrossRefMATH
10.
go back to reference Zhao G, Pietikäinen M (2009) Boosted multi-resolution spatiotemporal descriptors for facial expression recognition. Pattern Recogn Lett 30(12):1117–1127CrossRef Zhao G, Pietikäinen M (2009) Boosted multi-resolution spatiotemporal descriptors for facial expression recognition. Pattern Recogn Lett 30(12):1117–1127CrossRef
11.
go back to reference Yang P, Liu Q, Metaxas DN (2009) Boosting encoded dynamic features for facial expression recognition. Pattern Recogn Lett 30(2):132–139CrossRef Yang P, Liu Q, Metaxas DN (2009) Boosting encoded dynamic features for facial expression recognition. Pattern Recogn Lett 30(2):132–139CrossRef
12.
go back to reference Buenaposada J, Munoz E, Baumela L (January 2008) Recognising facial expressions in video sequences. Pattern Anal Appl 1(2):101–116 Buenaposada J, Munoz E, Baumela L (January 2008) Recognising facial expressions in video sequences. Pattern Anal Appl 1(2):101–116
13.
go back to reference Rudovic O, Pavlovic V, Pantic M (2012) Multi-output laplacian dynamic ordinal regression for facial expression recognition and intensity estimation. CVPR '12 Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp 2634–2641 Rudovic O, Pavlovic V, Pantic M (2012) Multi-output laplacian dynamic ordinal regression for facial expression recognition and intensity estimation. CVPR '12 Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp 2634–2641
14.
go back to reference Kienzle W, Bakir G, Franz M, Schölkopf B (2004) Face detection—efficient and rank deficient. NIPS 17(2005):673–680 Kienzle W, Bakir G, Franz M, Schölkopf B (2004) Face detection—efficient and rank deficient. NIPS 17(2005):673–680
15.
go back to reference Ji Y, Idrissi K (2012) Automatic facial expression recognition based on spatiotemporal descriptors. Pattern Recognit Lett 33(10):1373–1380CrossRef Ji Y, Idrissi K (2012) Automatic facial expression recognition based on spatiotemporal descriptors. Pattern Recognit Lett 33(10):1373–1380CrossRef
16.
go back to reference Beveridge J, Bolme D, Draper B, Teixeira M (February 2005) The csu face identification evaluation system: its purpose, features, and structure. Mach Vis Appl 16(2):128–138 Beveridge J, Bolme D, Draper B, Teixeira M (February 2005) The csu face identification evaluation system: its purpose, features, and structure. Mach Vis Appl 16(2):128–138
17.
go back to reference Ojala T, Pietikäinen M, Harwood D (January 1996) A comparative study of texture measures with classification based on feature distributions. Pattern Recognit 29(1):51–59 Ojala T, Pietikäinen M, Harwood D (January 1996) A comparative study of texture measures with classification based on feature distributions. Pattern Recognit 29(1):51–59
18.
go back to reference Bassili JN (1979) Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. J Pers Soc Psychol 37(11):2049–2058CrossRef Bassili JN (1979) Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. J Pers Soc Psychol 37(11):2049–2058CrossRef
19.
go back to reference Kanade T, Cohn J, Tian Y-L (2000) Comprehensive database for facial expression analysis. In Proceedings of the 4th IEEE international conference on automatic face and gesture recognition (FG’00), Grenoble, pp 46–53 Kanade T, Cohn J, Tian Y-L (2000) Comprehensive database for facial expression analysis. In Proceedings of the 4th IEEE international conference on automatic face and gesture recognition (FG’00), Grenoble, pp 46–53
Metadata
Title
Dynamic Visual Time Context Descriptors for Automatic Human Expression Classification
Authors
Yi Ji
Shengrong Gong
Chunping Liu
Copyright Year
2014
Publisher
Springer Berlin Heidelberg
DOI
https://doi.org/10.1007/978-3-642-37835-5_28

Premium Partner