Skip to main content

2018 | OriginalPaper | Buchkapitel

RGB-D Sensor for Facial Expression Recognition in AAL Context

verfasst von : Andrea Caroppo, Alessandro Leone, Pietro Siciliano

Erschienen in: Sensors and Microsystems

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper investigates the use of a commercial and low-cost RGB-D sensor for real-time facial expression recognition in Ambient Assisted Living Context. Since head poses and light conditions could be very different in domestic environments, the methodology used was designed to handle such situations. The implemented framework is able to classify four different categories of facial expressions: (1) happy, (2) sad, (3) fear/surprise, and (4) disgust/anger. The classification is obtained through an hybrid-based approach, by combining appearance and geometric features. The HOG feature descriptor and a group of Action Units compose the feature vector that is given as input, in the classification step, to a group of Support Vector Machines. The robustness of the approach is highlighted by the results obtained: the average accuracy for fear/surprise is the lowest with 85.2%, while happy is the facial expression better recognized (93.6%). Sad and disgust/anger are the facial expression confused the most.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Hu, Y., Zeng, Z., Yin, L., Wei, X., Tu, J., Huang, T.S.: A study of non-frontal-view facial expressions recognition. In: 19th International Conference on Pattern Recognition, 2008 (ICPR 2008), IEEE, pp. 1–4 (2008) Hu, Y., Zeng, Z., Yin, L., Wei, X., Tu, J., Huang, T.S.: A study of non-frontal-view facial expressions recognition. In: 19th International Conference on Pattern Recognition, 2008 (ICPR 2008), IEEE, pp. 1–4 (2008)
2.
Zurück zum Zitat Rudovic, O., Pantic, M., Patras, I.: Coupled Gaussian processes for pose-invariant facial expression recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1357–1369 (2013)CrossRef Rudovic, O., Pantic, M., Patras, I.: Coupled Gaussian processes for pose-invariant facial expression recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1357–1369 (2013)CrossRef
3.
Zurück zum Zitat Zheng, W.: Multi-view facial expression recognition based on group sparse reduced-rank regression. IEEE Trans. Affect. Comput. 5(1), 71–85 (2014)CrossRef Zheng, W.: Multi-view facial expression recognition based on group sparse reduced-rank regression. IEEE Trans. Affect. Comput. 5(1), 71–85 (2014)CrossRef
4.
Zurück zum Zitat Cament, L.A., Galdames, F.J., Bowyer, K.W., Perez, C.A.: Face recognition under pose variation with local Gabor features enhanced by active shape and statistical models. Pattern Recogn. 48(11), 3371–3384 (2015)CrossRef Cament, L.A., Galdames, F.J., Bowyer, K.W., Perez, C.A.: Face recognition under pose variation with local Gabor features enhanced by active shape and statistical models. Pattern Recogn. 48(11), 3371–3384 (2015)CrossRef
5.
Zurück zum Zitat Sandbach, G., Zafeiriou, S., Pantic, M., Yin, L.: Static and dynamic 3D facial expression recognition: a comprehensive survey. Image Vis. Comput. 30(10), 683–697 (2012)CrossRef Sandbach, G., Zafeiriou, S., Pantic, M., Yin, L.: Static and dynamic 3D facial expression recognition: a comprehensive survey. Image Vis. Comput. 30(10), 683–697 (2012)CrossRef
6.
Zurück zum Zitat Malawski, F., Kwolek, B., Sako, S.: Using kinect for facial expression recognition under varying poses and illumination. In: International Conference on Active Media Technology, pp. 395–406, Springer International Publishing (2014) Malawski, F., Kwolek, B., Sako, S.: Using kinect for facial expression recognition under varying poses and illumination. In: International Conference on Active Media Technology, pp. 395–406, Springer International Publishing (2014)
7.
Zurück zum Zitat Andò, B., Siciliano, P., Marletta, V., Monteriù, A.: Ambient Assisted Living. (2015) Andò, B., Siciliano, P., Marletta, V., Monteriù, A.: Ambient Assisted Living. (2015)
8.
Zurück zum Zitat Chang, Y., Hu, C., Feris, R., Turk, M.: Manifold based analysis of facial expression. Image Vis. Comput. 24(6), 605–614 (2006)CrossRef Chang, Y., Hu, C., Feris, R., Turk, M.: Manifold based analysis of facial expression. Image Vis. Comput. 24(6), 605–614 (2006)CrossRef
9.
Zurück zum Zitat Shbib, R., Zhou, S.: Facial expression analysis using active shape model. Int J Signal Process., Image Process. Pattern Recognit. 8(1), 9–22 (2015) Shbib, R., Zhou, S.: Facial expression analysis using active shape model. Int J Signal Process., Image Process. Pattern Recognit. 8(1), 9–22 (2015)
10.
Zurück zum Zitat Cheon, Y., Kim, D.: Natural facial expression recognition using differential-AAM and manifold learning. Pattern Recognit. 42(7), 1340–1350 (2009)CrossRefMATH Cheon, Y., Kim, D.: Natural facial expression recognition using differential-AAM and manifold learning. Pattern Recognit. 42(7), 1340–1350 (2009)CrossRefMATH
11.
Zurück zum Zitat Chen, Y., Hua, C., Bai, R.: Regression-based active appearance model initialization for facial feature tracking with missing frames. Pattern Recognit. Lett. 38, 113–119 (2014)CrossRef Chen, Y., Hua, C., Bai, R.: Regression-based active appearance model initialization for facial feature tracking with missing frames. Pattern Recognit. Lett. 38, 113–119 (2014)CrossRef
12.
Zurück zum Zitat Soyel, H., Demirel, H.: Facial expression recognition based on discriminative scale invariant feature transform. Electron. Lett. 46(5), 343–345 (2010)CrossRef Soyel, H., Demirel, H.: Facial expression recognition based on discriminative scale invariant feature transform. Electron. Lett. 46(5), 343–345 (2010)CrossRef
13.
Zurück zum Zitat Gu, W., Xiang, C., Venkatesh, Y.V., Huang, D., Lin, H.: Facial expression recognition using radial encoding of local Gabor features and classifier synthesis. Pattern Recognit. 45(1), 80–91 (2012)CrossRef Gu, W., Xiang, C., Venkatesh, Y.V., Huang, D., Lin, H.: Facial expression recognition using radial encoding of local Gabor features and classifier synthesis. Pattern Recognit. 45(1), 80–91 (2012)CrossRef
14.
Zurück zum Zitat Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6): (2007) Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6): (2007)
15.
Zurück zum Zitat Dahmane, M., Meunier, J.: Emotion recognition using dynamic grid-based HoG features. In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), pp. 884–888, IEEE (2011) Dahmane, M., Meunier, J.: Emotion recognition using dynamic grid-based HoG features. In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), pp. 884–888, IEEE (2011)
16.
Zurück zum Zitat Jack, R.E., Garrod, O.G., Schyns, P.G.: Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time. Curr. Biol. 24(2), 187–192 (2014)CrossRef Jack, R.E., Garrod, O.G., Schyns, P.G.: Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time. Curr. Biol. 24(2), 187–192 (2014)CrossRef
17.
Zurück zum Zitat Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124 (1971)CrossRef Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124 (1971)CrossRef
19.
Zurück zum Zitat Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004)CrossRef Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004)CrossRef
20.
Zurück zum Zitat Yu, X., Huang, J., Zhang, S., Yan, W., Metaxas, D.N.: Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1944–1951 (2013) Yu, X., Huang, J., Zhang, S., Yan, W., Metaxas, D.N.: Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1944–1951 (2013)
21.
Zurück zum Zitat Dalal, N., & Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005 (CVPR 2005), vol. 1, pp. 886–893, IEEE (2005) Dalal, N., & Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005 (CVPR 2005), vol. 1, pp. 886–893, IEEE (2005)
22.
Zurück zum Zitat Ekman, P., Friesen, W.V., Hager, J.C.: Facial Action Coding System (FACS). A Technique for the Measurement of Facial Action. Consulting, Palo Alto, 22 (1978) Ekman, P., Friesen, W.V., Hager, J.C.: Facial Action Coding System (FACS). A Technique for the Measurement of Facial Action. Consulting, Palo Alto, 22 (1978)
23.
Zurück zum Zitat Knerr, S., Personnaz, L., Dreyfus, G.: Single-layer learning revisited: a stepwise procedure for building and training a neural network. In: Neurocomputing, pp. 41–50. Springer, Berlin (1990) Knerr, S., Personnaz, L., Dreyfus, G.: Single-layer learning revisited: a stepwise procedure for building and training a neural network. In: Neurocomputing, pp. 41–50. Springer, Berlin (1990)
24.
Zurück zum Zitat Hsu, C.W., Chang, C.C., Lin, C.J.: A Practical Guide to Support Vector Classification. (2003) Hsu, C.W., Chang, C.C., Lin, C.J.: A Practical Guide to Support Vector Classification. (2003)
25.
Zurück zum Zitat Aly, S., Trubanova, A., Abbott, L., White, S., Youssef, A.: VT-KFER: a Kinect-based RGBD+ time dataset for spontaneous and non-spontaneous facial expression recognition. In: 2015 International Conference on Biometrics (ICB), pp. 90–97. IEEE (2015) Aly, S., Trubanova, A., Abbott, L., White, S., Youssef, A.: VT-KFER: a Kinect-based RGBD+ time dataset for spontaneous and non-spontaneous facial expression recognition. In: 2015 International Conference on Biometrics (ICB), pp. 90–97. IEEE (2015)
Metadaten
Titel
RGB-D Sensor for Facial Expression Recognition in AAL Context
verfasst von
Andrea Caroppo
Alessandro Leone
Pietro Siciliano
Copyright-Jahr
2018
DOI
https://doi.org/10.1007/978-3-319-66802-4_39