Skip to main content
Top

2016 | OriginalPaper | Chapter

In Search of Truth: Analysis of Smile Intensity Dynamics to Detect Deception

Authors : Michal Kawulok, Jakub Nalepa, Karolina Nurzynska, Bogdan Smolka

Published in: Advances in Artificial Intelligence - IBERAMIA 2016

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Detection of deceptive facial expressions, including estimating smile genuineness, is an important and challenging research topic that draws increasing attention from the computer vision and pattern recognition community. The state-of-the-art methods require localizing a number of facial landmarks to extract sophisticated facial characteristics. In this paper, we explore how to exploit fast smile intensity detectors to extract temporal features. This allows for real-time discrimination between posed and spontaneous expressions at the early smile onset phase. We report the results of experimental validation, which indicate high competitiveness of our method for the UvA-NEMO benchmark database.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Footnotes
1
This appears not trivial for patients with mental disorders, e.g., schizophrenia [1].
 
Literature
1.
go back to reference Barkhof, E., de Sonneville, L.M., Meijer, C.J., de Haan, L.: Specificity of facial emotion recognition impairments in patients with multi-episode schizophrenia. Schizophr. Res.: Cogn. 2(1), 12–19 (2015) Barkhof, E., de Sonneville, L.M., Meijer, C.J., de Haan, L.: Specificity of facial emotion recognition impairments in patients with multi-episode schizophrenia. Schizophr. Res.: Cogn. 2(1), 12–19 (2015)
2.
go back to reference Ross, E.D., Pulusu, V.K.: Posed versus spontaneous facial expressions are modulated by opposite cerebral hemispheres. Cortex 49(5), 1280–1291 (2013)CrossRef Ross, E.D., Pulusu, V.K.: Posed versus spontaneous facial expressions are modulated by opposite cerebral hemispheres. Cortex 49(5), 1280–1291 (2013)CrossRef
3.
go back to reference Trutoiu, L.C., Carter, E.J., Pollard, N., Cohn, J.F., Hodgins, J.K.: Spatial and temporal linearities in posed and spontaneous smiles. ACM Trans. Appl. Percept. 11(3), 12:1–12:15 (2014)CrossRef Trutoiu, L.C., Carter, E.J., Pollard, N., Cohn, J.F., Hodgins, J.K.: Spatial and temporal linearities in posed and spontaneous smiles. ACM Trans. Appl. Percept. 11(3), 12:1–12:15 (2014)CrossRef
4.
go back to reference Nalepa, J., Kawulok, M.: Adaptive memetic algorithm enhanced with data geometry analysis to select training data for SVMs. Neurocomputing 185, 113–132 (2016)CrossRef Nalepa, J., Kawulok, M.: Adaptive memetic algorithm enhanced with data geometry analysis to select training data for SVMs. Neurocomputing 185, 113–132 (2016)CrossRef
5.
go back to reference Dibeklioğlu, H., Salah, A.A., Gevers, T.: Recognition of genuine smiles. IEEE Trans. Multimedia 17(3), 279–294 (2015)CrossRef Dibeklioğlu, H., Salah, A.A., Gevers, T.: Recognition of genuine smiles. IEEE Trans. Multimedia 17(3), 279–294 (2015)CrossRef
6.
go back to reference Wu, P., Liu, H., Zhang, X.: Spontaneous versus posed smile recognition using discriminative local spatial-temporal descriptors. In: Proceedings of the IEEE ICASSP, pp. 1240–1244 (2014) Wu, P., Liu, H., Zhang, X.: Spontaneous versus posed smile recognition using discriminative local spatial-temporal descriptors. In: Proceedings of the IEEE ICASSP, pp. 1240–1244 (2014)
7.
go back to reference Calder, A.J., Young, A.W.: Understanding the recognition of facial identity and facial expression. Nature Rev.: Neurosci. 6, 641–651 (2005)CrossRef Calder, A.J., Young, A.W.: Understanding the recognition of facial identity and facial expression. Nature Rev.: Neurosci. 6, 641–651 (2005)CrossRef
8.
go back to reference Martinez, B., Valstar, M.F.: Advances, challenges, and opportunities in automatic facial expression recognition. In: Kawulok, M., Celebi, M.E., Smolka, B. (eds.) Advances in Face Detection and Facial Image Analysis, pp. 63–100. Springer, Heidelberg (2016)CrossRef Martinez, B., Valstar, M.F.: Advances, challenges, and opportunities in automatic facial expression recognition. In: Kawulok, M., Celebi, M.E., Smolka, B. (eds.) Advances in Face Detection and Facial Image Analysis, pp. 63–100. Springer, Heidelberg (2016)CrossRef
9.
go back to reference Krumhuber, E.G., Likowski, K.U., Weyers, P.: Facial mimicry of spontaneous and deliberate Duchenne and non-Duchenne smiles. J. Nonverbal Behav. 38(1), 1–11 (2014)CrossRef Krumhuber, E.G., Likowski, K.U., Weyers, P.: Facial mimicry of spontaneous and deliberate Duchenne and non-Duchenne smiles. J. Nonverbal Behav. 38(1), 1–11 (2014)CrossRef
10.
go back to reference Girard, J.M., Cohn, J.F., Jeni, L.A., Sayette, M.A., De la Torre, F.: Spontaneous facial expression in unscripted social interactions can be measured automatically. Behav. Res. Methods 47(4), 1136–1147 (2015)CrossRef Girard, J.M., Cohn, J.F., Jeni, L.A., Sayette, M.A., De la Torre, F.: Spontaneous facial expression in unscripted social interactions can be measured automatically. Behav. Res. Methods 47(4), 1136–1147 (2015)CrossRef
11.
go back to reference Valstar, M.F., Pantic, M., Ambadar, Z., Cohn, J.F.: Spontaneous vs. posed facial behavior: automatic analysis of brow actions. In: Proceedings of the ICMI, pp. 162–170 (2006) Valstar, M.F., Pantic, M., Ambadar, Z., Cohn, J.F.: Spontaneous vs. posed facial behavior: automatic analysis of brow actions. In: Proceedings of the ICMI, pp. 162–170 (2006)
12.
go back to reference Cohn, J.F., Schmidt, K.L.: The timing of facial motion in posed and spontaneous smiles. Int. J. Wavelets Multiresolut. Inf. Process. 02(02), 121–132 (2004)CrossRef Cohn, J.F., Schmidt, K.L.: The timing of facial motion in posed and spontaneous smiles. Int. J. Wavelets Multiresolut. Inf. Process. 02(02), 121–132 (2004)CrossRef
13.
go back to reference Sénéchal, T., Turcot, J., el Kaliouby, R.: Smile or smirk? Automatic detection of spontaneous asymmetric smiles to understand viewer experience. In: Proceedings of the IEEE FG, pp. 1–8 (2013) Sénéchal, T., Turcot, J., el Kaliouby, R.: Smile or smirk? Automatic detection of spontaneous asymmetric smiles to understand viewer experience. In: Proceedings of the IEEE FG, pp. 1–8 (2013)
14.
go back to reference Dibeklioğlu, H., Valenti, R., Salah, A.A., Gevers, T.: Eyes do not lie: spontaneous versus posed smiles. In: ACM International Conference on Multimedia, pp. 1–4 (2010) Dibeklioğlu, H., Valenti, R., Salah, A.A., Gevers, T.: Eyes do not lie: spontaneous versus posed smiles. In: ACM International Conference on Multimedia, pp. 1–4 (2010)
15.
go back to reference Gan, Q., Wu, C., Wang, S., Ji, Q.: Posed and spontaneous facial expression differentiation using deep boltzmann machines. In: 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 643–648. IEEE (2015) Gan, Q., Wu, C., Wang, S., Ji, Q.: Posed and spontaneous facial expression differentiation using deep boltzmann machines. In: 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 643–648. IEEE (2015)
16.
go back to reference Zhao, G., Pietikäinen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)CrossRef Zhao, G., Pietikäinen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)CrossRef
17.
go back to reference Smolka, B., Nurzynska, K.: Power LBP: a novel texture operator for smiling and neutral facial display classification. Procedia Comp. Sci. 51, 1555–1564 (2015)CrossRef Smolka, B., Nurzynska, K.: Power LBP: a novel texture operator for smiling and neutral facial display classification. Procedia Comp. Sci. 51, 1555–1564 (2015)CrossRef
18.
go back to reference Abouelenien, M., Pérez-Rosas, V., Mihalcea, R., Burzo, M.: Deception detection using a multimodal approach. In: Proceedings of the ICMI, pp. 58–65. ACM, New York (2014) Abouelenien, M., Pérez-Rosas, V., Mihalcea, R., Burzo, M.: Deception detection using a multimodal approach. In: Proceedings of the ICMI, pp. 58–65. ACM, New York (2014)
19.
go back to reference Cohn, J., Reed, L., Moriyama, T., Xiao, J., Schmidt, K., Ambadar, Z.: Multimodal coordination of facial action, head rotation, and eye motion during spontaneous smiles. In: Proceedings of the IEEE FG, pp. 129–138 (2004) Cohn, J., Reed, L., Moriyama, T., Xiao, J., Schmidt, K., Ambadar, Z.: Multimodal coordination of facial action, head rotation, and eye motion during spontaneous smiles. In: Proceedings of the IEEE FG, pp. 129–138 (2004)
20.
go back to reference Valstar, M.F., Gunes, H., Pantic, M.: How to distinguish posed from spontaneous smiles using geometric features. In: Proceedings of the ICMI, pp. 38–45. ACM (2007) Valstar, M.F., Gunes, H., Pantic, M.: How to distinguish posed from spontaneous smiles using geometric features. In: Proceedings of the ICMI, pp. 38–45. ACM (2007)
21.
go back to reference Rajoub, B.A., Zwiggelaar, R.: Thermal facial analysis for deception detection. IEEE Trans. Inf. Forensics Secur. 9(6), 1015–1023 (2014)CrossRef Rajoub, B.A., Zwiggelaar, R.: Thermal facial analysis for deception detection. IEEE Trans. Inf. Forensics Secur. 9(6), 1015–1023 (2014)CrossRef
22.
go back to reference Kawulok, M., Szymanek, J.: Precise multi-level face detector for advanced analysis of facial images. IET Image Process. 6(2), 95–103 (2012)MathSciNetCrossRef Kawulok, M., Szymanek, J.: Precise multi-level face detector for advanced analysis of facial images. IET Image Process. 6(2), 95–103 (2012)MathSciNetCrossRef
23.
go back to reference Kawulok, M., Nalepa, J.: Towards robust SVM training from weakly labeled large data sets. In: Proceedings of the ACPR, pp. 464–468 (2015) Kawulok, M., Nalepa, J.: Towards robust SVM training from weakly labeled large data sets. In: Proceedings of the ACPR, pp. 464–468 (2015)
24.
go back to reference Dibeklioğlu, H., Salah, A.A., Gevers, T.: Are you really smiling at me? Spontaneous versus posed enjoyment smiles. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 525–538. Springer, Heidelberg (2012)CrossRef Dibeklioğlu, H., Salah, A.A., Gevers, T.: Are you really smiling at me? Spontaneous versus posed enjoyment smiles. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 525–538. Springer, Heidelberg (2012)CrossRef
25.
go back to reference Pfister, T., Li, X., Zhao, G., Pietikäinen, M.: Differentiating spontaneous from posed facial expressions within a generic facial expression recognition framework. In: Proceedings of the IEEE ICCV, pp. 868–875 (2011) Pfister, T., Li, X., Zhao, G., Pietikäinen, M.: Differentiating spontaneous from posed facial expressions within a generic facial expression recognition framework. In: Proceedings of the IEEE ICCV, pp. 868–875 (2011)
26.
go back to reference Kawulok, M., Wu, J., Hancock, E.R.: Supervised relevance maps for increasing the distinctiveness of facial images. Pattern Recogn. 44(4), 929–939 (2011)CrossRef Kawulok, M., Wu, J., Hancock, E.R.: Supervised relevance maps for increasing the distinctiveness of facial images. Pattern Recogn. 44(4), 929–939 (2011)CrossRef
Metadata
Title
In Search of Truth: Analysis of Smile Intensity Dynamics to Detect Deception
Authors
Michal Kawulok
Jakub Nalepa
Karolina Nurzynska
Bogdan Smolka
Copyright Year
2016
DOI
https://doi.org/10.1007/978-3-319-47955-2_27

Premium Partner