Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Machine Vision and Applications
Published in: Machine Vision and Applications 2/2018

18-12-2017 | Original Paper

Facial expression analysis and expression-invariant face recognition by manifold-based synthesis

Authors: Yao Peng, Hujun Yin

Published in: Machine Vision and Applications | Issue 2/2018

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Over the last decades, expression classification and face recognition have received substantial attention in computer vision and pattern recognition with more recent efforts focusing on understanding and modelling expression variations. In this paper, we present an expression classification and expression-invariant face recognition method by synthesising photorealistic expression manifolds to expand the gallery set. By means of synthesising expression images from neutral faces, more within-subject variability can be obtained. Eigentransformation is utilised to generate both shape and expression details for novel subjects. Expression classification and face recognition are then performed on the extended training set with synthesised expressions. Experimental results on various datasets show that the proposed method is robust for recognising various expressions and faces with varying degrees of expression. Comprehensive experiments conducted and comparisons with the existing methods are reported. Cross-database synthesis and effect of landmark quality are also studied.
previous article Tracking using Numerous Anchor Points
next article Instance-based object recognition in 3D point clouds using discriminative shape primitives

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
Abboud, B., Davoine, F.: Bilinear factorisation for facial expression analysis and synthesis. IEEE Proc. Vis. Image Signal Process. 152(3), 327–333 (2005)CrossRef
2.
Abboud, B., Davoine, F., Dang, M.: Facial expression recognition and synthesis based on an appearance model. Signal Process. Image Commun. 19(8), 723–740 (2004)CrossRef
3.
Aifanti, N., Papachristou, C., Delopoulos, A.: The MUG facial expression database. In: Proceedings of IEEE International Conference on Image Analysis for Multimedia Interactive Services Workshop, pp. 1–4 (2010)
4.
Ambadar, Z., Schooler, J.W., Cohn, J.F.: Deciphering the enigmatic face the importance of facial dynamics in interpreting subtle facial expressions. Psychol. Sci. 16(5), 403–410 (2005)CrossRef
5.
Baudat, G., Anouar, F.: Generalized discriminant analysis using a kernel approach. Neural Comput. 12(10), 2385–2404 (2000)CrossRef
6.
Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces versus fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 711–720 (1997)CrossRef
7.
Chang, Y., Hu, C., Turk, M.: Manifold of facial expression. In: Proceedings of IEEE International Conference on Analysis and Modeling of Faces and Gestures Workshops, pp. 28–35 (2003)
8.
Chapman, R.E.: Conventional procrustes approaches. In: Proceedings of Michigan Morphometrics Workshop, pp. 251–267 (1990)
9.
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)CrossRef
10.
Dibeklioğlu, H., Gökberk, B., Akarun, L.: Nasal region-based 3d face recognition under pose and expression variations. In: Proceedings of International Conference on Advances in Biometrics, pp. 309–318 (2009)
11.
Dryden, I.L., Mardia, K.V.: Statistical Shape Analysis, vol. 4. Wiley, Chichester (1998)MATH
12.
Ekman, P., Friesen, W.V.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto (1978)
13.
Ekman, P., Rolls, E., Perrett, D., Ellis, H.: Facial expressions of emotion: an old controversy and new findings. Philos. T. Roy. Soc. B 335(1273), 63–69 (1992)CrossRef
14.
Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recognit. 36(1), 259–275 (2003)CrossRefMATH
15.
Gao, S., Zhang, Y., Jia, K., Lu, J., Zhang, Y.: Single sample face recognition via learning deep supervised auto-encoders. IEEE Trans. Inf. Forensic Sec. 10(10), 2108–2118 (2015)CrossRef
16.
Ghimire, D., Lee, J., Li, Z.N., Jeong, S.: Recognition of facial expressions based on salient geometric features and support vector machines. Multimedia Tools Appl. 76(6), 7921–7946
17.
18.
19.
Gu, W., Xiang, C., Venkatesh, Y.V., Huang, D., Lin, H.: Facial expression recognition using radial encoding of local Gabor features and classifier synthesis. Pattern Recognit. 45(1), 80–91 (2012)CrossRef
20.
Happy, S.L., Routray, A.: Automatic facial expression recognition using features of salient facial patches. IEEE Trans. Affect. Comput. 6(1), 1–12 (2015)CrossRef
21.
Huang, D., De la Torre, F.: Bilinear kernel reduced rank regression for facial expression synthesis. In: Proceedings of European Conference on Computer Vision, pp. 364–377 (2010)
22.
Jain, A.K., Li, S.Z.: Handbook of Face Recognition, vol. 1. Springer, Berlin (2005)MATH
23.
Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53 (2000)
24.
Lee, H., Kim, D.: Expression-invariant face recognition by facial expression transformations. Pattern Recogn. Lett. 29(13), 1797–1805 (2008)CrossRef
25.
Liu, Z., Shan, Y., Zhang, Z.: Expressive expression mapping with ratio images. In: Proceedings of Conference on Computer Graphics and Interactive Techniques, pp. 271–276 (2001)
26.
Lowe, D.G.: Object recognition from local scale-invariant features. In: Proceedings of IEEE International Conference on Computer Vision, vol. 2, pp. 1150–1157 (1999)
27.
Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn–Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition Workshop, pp. 94–101 (2010)
28.
Lyons, M., Akamatsu, S., Kamachi, M., Gyoba, J.: Coding facial expressions with gabor wavelets. In: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, pp. 200–205 (1998)
29.
Ma, W.C., Jones, A., Chiang, J.Y., Hawkins, T., Frederiksen, S., Peers, P., Vukovic, M., Ouhyoung, M., Debevec, P.: Facial performance synthesis using deformation-driven polynomial displacement maps. ACM Trans. Graph. 27(5), 121:1–121:10 (2008)
30.
Martinez, A., Benavente, R.: The AR face database. Technical Report, CVC Technical Report (1998)
31.
Martinez, A.M.: Recognizing imprecisely localized, partially occluded, and expression variant faces from a single sample per class. IEEE Trans. Pattern Anal. Mach. Intell. 24(6), 748–763 (2002)CrossRef
32.
Mohammadi, M.R., Fatemizadeh, E., Mahoor, M.H.: PCA-based dictionary building for accurate facial expression recognition via sparse representation. J. Vis. Commun. Image R. 25(5), 1082–1092 (2014)CrossRef
33.
Mohammadi, M.R., Fatemizadeh, E., Mahoor, M.H.: Simultaneous recognition of facial expression and identity via sparse representation. In: Proceedings of IEEE International Winter Conference on Applications of Computer Vision, pp. 1066–1073 (2014)
34.
Mohammadzade, H., Hatzinakos, D.: Projection into expression subspaces for face recognition from single sample per person. IEEE Trans. Affect. Comput. 4(1), 69–82 (2013)CrossRef
35.
Pantic, M., Patras, I.: Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences. IEEE Trans. Syst. Man Cybern. B Cybern. 36(2), 433–449 (2006)CrossRef
36.
Pantic, M., Rothkrantz, L.J.: Facial action recognition for facial expression analysis from static face images. IEEE Trans. Syst. Man Cybern. B Cybern. 34(3), 1449–1461 (2004)CrossRef
37.
Pantic, M., Valstar, M., Rademaker, R., Maat, L.: Web-based database for facial expression analysis. In: Proceedings of IEEE International Conference on Multimedia and Expo, pp. 317–321 (2005)
38.
Petpairote, C., Madarasmi, S.: Face recognition improvement by converting expression faces to neutral faces. In: Proceedings of International Symposium on Communications and Information Technologies, pp. 439–444 (2013)
39.
Pyun, H., Kim, Y., Chae, W., Kang, H.W., Shin, S.Y.: An example-based approach for facial expression cloning. In: Proceedings of EG/SIGGRAPH Symposium on Computer Animation, pp. 167–176 (2003)
40.
Rahulamathavan, Y., Phan, R.C.W., Chambers, J.A., Parish, D.J.: Facial expression recognition in the encrypted domain based on local fisher discriminant analysis. IEEE Trans. Affect. Comput. 4(1), 83–92 (2013)CrossRef
41.
Sandbach, G., Zafeiriou, S., Pantic, M., Rueckert, D.: A dynamic approach to the recognition of 3d facial expressions and their temporal models. In: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition Workshops, pp. 406–413 (2011)
42.
Savran, A., Alyüz, N., Dibeklioğlu, H., Çeliktutan, O., Gökberk, B., Sankur, B., Akarun, L.: Bosphorus database for 3d face analysis. In: Proceedings of European Workshop on Biometrics and Identity Management, pp. 47–56 (2008)
43.
Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)CrossRef
44.
Shojaeilangari, S., Yau, W.Y., Teoh, E.K.: Pose-invariant descriptor for facial emotion recognition. Mach. Vis. Appl. 27(7), 1063–1070 (2016)
45.
Smeets, D., Keustermans, J., Vandermeulen, D., Suetens, P.: meshSIFT: local surface features for 3D face recognition under expression variations and partial data. Comput. Vis. Image Und. 117(2), 158–169 (2013)CrossRef
46.
Song, M., Tao, D., Liu, Z., Li, X., Zhou, M.: Image ratio features for facial expression recognition application. IEEE Trans. Syst. Man Cybern. B Cybern. 40(3), 779–788 (2010)CrossRef
47.
Soyel, H., Demirel, H.: Localized discriminative scale invariant feature transform based facial expression recognition. Comput. Electr. Eng. 38(5), 1299–1309 (2012)CrossRef
48.
Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: Closing the gap to human-level performance in face verification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1701–1708 (2014)
49.
Tan, X., Chen, S., Zhou, Z., Zhang, F.: Recognizing partially occluded, expression variant faces from single training image per person with SOM and soft k-NN ensemble. IEEE Trans. Neural Netw. 16(4), 875–886 (2005)CrossRef
50.
Tang, X., Wang, X.: Face photo recognition using sketch. In: Proceedings of IEEE International Conference on Image Processing, vol. 1, pp. 257–260 (2002)
51.
Tang, X., Wang, X.: Face sketch synthesis and recognition. In: Proceedings of IEEE International Conference on Computer Vision, pp. 687–694 (2003)
52.
Tian, Y., Kanade, T., Cohn, J.F.: Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 97–115 (2001)CrossRef
53.
Valstar, M., Pantic, M.: Induced disgust, happiness and surprise: an addition to the MMI facial expression database. In: Proceedings of International Conference on Language Resources and Evaluation Workshop, pp. 65–70. Malta (2010)
54.
Wagner, A., Wright, J., Ganesh, A., Zhou, Z., Mobahi, H., Ma, Y.: Toward a practical face recognition system: robust alignment and illumination by sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 34(2), 372–386 (2012)CrossRef
55.
Wang, H., Ahuja, N.: Facial expression decomposition. In: Proceedings of IEEE International Conference on Computer Vision, pp. 958–965 (2003)
56.
Wang, R., Chen, X.: Manifold discriminant analysis. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pp. 429–436 (2009)
57.
Wang, R., Shan, S., Chen, X., Gao, W.: Manifold–manifold distance with application to face recognition based on image set. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)
58.
Zaman, F.K., Shafie, A.A., Mustafah, Y.M.: Robust face recognition against expressions and partial occlusions. Int. J. Autom. Comput. 13(4), 319–337 (2016)CrossRef
59.
Zhang, L., Tjondronegoro, D.: Facial expression recognition using facial movement features. IEEE Trans. Affect. Comput. 2(4), 219–229 (2011)
60.
Zhang, Q., Liu, Z., Quo, G., Terzopoulos, D., Shum, H.Y.: Geometry-driven photorealistic facial expression synthesis. IEEE Trans. Vis. Comput. Gr. 12(1), 48–60 (2006)CrossRef
61.
Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: a literature survey. ACM Comput. Surv. 35(4), 399–458 (2003)CrossRef
62.
Zhong, L., Liu, Q., Yang, P., Liu, B., Huang, J., Metaxas, D.N.: Learning active facial patches for expression analysis. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pp. 2562–2569 (2012)
Metadata
Title
Facial expression analysis and expression-invariant face recognition by manifold-based synthesis
Authors
Yao Peng
Hujun Yin
Publication date
18-12-2017
Publisher
Springer Berlin Heidelberg
Published in
Machine Vision and Applications / Issue 2/2018
Print ISSN: 0932-8092
Electronic ISSN: 1432-1769
DOI
https://doi.org/10.1007/s00138-017-0895-6

Other articles of this Issue 2/2018

Machine Vision and Applications 2/2018 Go to the issue

Original Paper

Sparse-then-dense alignment-based 3D map reconstruction method for endoscopic capsule robots

Original Paper

Exercise classification and event segmentation in Hammersmith Infant Neurological Examination videos

Original Paper

Example-based 3D inpainting of point clouds using metric tensor and Christoffel symbols

Original Paper

Geometric primitive refinement for structured light cameras

Original Paper

Beyond Eleven Color Names for Image Understanding

Original Paper

Tracking using Numerous Anchor Points

Premium Partner

    Image Credits