Skip to main content
Top

2018 | OriginalPaper | Chapter

RGB-D Sensor for Facial Expression Recognition in AAL Context

Authors : Andrea Caroppo, Alessandro Leone, Pietro Siciliano

Published in: Sensors and Microsystems

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

This paper investigates the use of a commercial and low-cost RGB-D sensor for real-time facial expression recognition in Ambient Assisted Living Context. Since head poses and light conditions could be very different in domestic environments, the methodology used was designed to handle such situations. The implemented framework is able to classify four different categories of facial expressions: (1) happy, (2) sad, (3) fear/surprise, and (4) disgust/anger. The classification is obtained through an hybrid-based approach, by combining appearance and geometric features. The HOG feature descriptor and a group of Action Units compose the feature vector that is given as input, in the classification step, to a group of Support Vector Machines. The robustness of the approach is highlighted by the results obtained: the average accuracy for fear/surprise is the lowest with 85.2%, while happy is the facial expression better recognized (93.6%). Sad and disgust/anger are the facial expression confused the most.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Hu, Y., Zeng, Z., Yin, L., Wei, X., Tu, J., Huang, T.S.: A study of non-frontal-view facial expressions recognition. In: 19th International Conference on Pattern Recognition, 2008 (ICPR 2008), IEEE, pp. 1–4 (2008) Hu, Y., Zeng, Z., Yin, L., Wei, X., Tu, J., Huang, T.S.: A study of non-frontal-view facial expressions recognition. In: 19th International Conference on Pattern Recognition, 2008 (ICPR 2008), IEEE, pp. 1–4 (2008)
2.
go back to reference Rudovic, O., Pantic, M., Patras, I.: Coupled Gaussian processes for pose-invariant facial expression recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1357–1369 (2013)CrossRef Rudovic, O., Pantic, M., Patras, I.: Coupled Gaussian processes for pose-invariant facial expression recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1357–1369 (2013)CrossRef
3.
go back to reference Zheng, W.: Multi-view facial expression recognition based on group sparse reduced-rank regression. IEEE Trans. Affect. Comput. 5(1), 71–85 (2014)CrossRef Zheng, W.: Multi-view facial expression recognition based on group sparse reduced-rank regression. IEEE Trans. Affect. Comput. 5(1), 71–85 (2014)CrossRef
4.
go back to reference Cament, L.A., Galdames, F.J., Bowyer, K.W., Perez, C.A.: Face recognition under pose variation with local Gabor features enhanced by active shape and statistical models. Pattern Recogn. 48(11), 3371–3384 (2015)CrossRef Cament, L.A., Galdames, F.J., Bowyer, K.W., Perez, C.A.: Face recognition under pose variation with local Gabor features enhanced by active shape and statistical models. Pattern Recogn. 48(11), 3371–3384 (2015)CrossRef
5.
go back to reference Sandbach, G., Zafeiriou, S., Pantic, M., Yin, L.: Static and dynamic 3D facial expression recognition: a comprehensive survey. Image Vis. Comput. 30(10), 683–697 (2012)CrossRef Sandbach, G., Zafeiriou, S., Pantic, M., Yin, L.: Static and dynamic 3D facial expression recognition: a comprehensive survey. Image Vis. Comput. 30(10), 683–697 (2012)CrossRef
6.
go back to reference Malawski, F., Kwolek, B., Sako, S.: Using kinect for facial expression recognition under varying poses and illumination. In: International Conference on Active Media Technology, pp. 395–406, Springer International Publishing (2014) Malawski, F., Kwolek, B., Sako, S.: Using kinect for facial expression recognition under varying poses and illumination. In: International Conference on Active Media Technology, pp. 395–406, Springer International Publishing (2014)
7.
go back to reference Andò, B., Siciliano, P., Marletta, V., Monteriù, A.: Ambient Assisted Living. (2015) Andò, B., Siciliano, P., Marletta, V., Monteriù, A.: Ambient Assisted Living. (2015)
8.
go back to reference Chang, Y., Hu, C., Feris, R., Turk, M.: Manifold based analysis of facial expression. Image Vis. Comput. 24(6), 605–614 (2006)CrossRef Chang, Y., Hu, C., Feris, R., Turk, M.: Manifold based analysis of facial expression. Image Vis. Comput. 24(6), 605–614 (2006)CrossRef
9.
go back to reference Shbib, R., Zhou, S.: Facial expression analysis using active shape model. Int J Signal Process., Image Process. Pattern Recognit. 8(1), 9–22 (2015) Shbib, R., Zhou, S.: Facial expression analysis using active shape model. Int J Signal Process., Image Process. Pattern Recognit. 8(1), 9–22 (2015)
10.
go back to reference Cheon, Y., Kim, D.: Natural facial expression recognition using differential-AAM and manifold learning. Pattern Recognit. 42(7), 1340–1350 (2009)CrossRefMATH Cheon, Y., Kim, D.: Natural facial expression recognition using differential-AAM and manifold learning. Pattern Recognit. 42(7), 1340–1350 (2009)CrossRefMATH
11.
go back to reference Chen, Y., Hua, C., Bai, R.: Regression-based active appearance model initialization for facial feature tracking with missing frames. Pattern Recognit. Lett. 38, 113–119 (2014)CrossRef Chen, Y., Hua, C., Bai, R.: Regression-based active appearance model initialization for facial feature tracking with missing frames. Pattern Recognit. Lett. 38, 113–119 (2014)CrossRef
12.
go back to reference Soyel, H., Demirel, H.: Facial expression recognition based on discriminative scale invariant feature transform. Electron. Lett. 46(5), 343–345 (2010)CrossRef Soyel, H., Demirel, H.: Facial expression recognition based on discriminative scale invariant feature transform. Electron. Lett. 46(5), 343–345 (2010)CrossRef
13.
go back to reference Gu, W., Xiang, C., Venkatesh, Y.V., Huang, D., Lin, H.: Facial expression recognition using radial encoding of local Gabor features and classifier synthesis. Pattern Recognit. 45(1), 80–91 (2012)CrossRef Gu, W., Xiang, C., Venkatesh, Y.V., Huang, D., Lin, H.: Facial expression recognition using radial encoding of local Gabor features and classifier synthesis. Pattern Recognit. 45(1), 80–91 (2012)CrossRef
14.
go back to reference Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6): (2007) Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6): (2007)
15.
go back to reference Dahmane, M., Meunier, J.: Emotion recognition using dynamic grid-based HoG features. In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), pp. 884–888, IEEE (2011) Dahmane, M., Meunier, J.: Emotion recognition using dynamic grid-based HoG features. In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), pp. 884–888, IEEE (2011)
16.
go back to reference Jack, R.E., Garrod, O.G., Schyns, P.G.: Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time. Curr. Biol. 24(2), 187–192 (2014)CrossRef Jack, R.E., Garrod, O.G., Schyns, P.G.: Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time. Curr. Biol. 24(2), 187–192 (2014)CrossRef
17.
go back to reference Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124 (1971)CrossRef Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124 (1971)CrossRef
19.
go back to reference Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004)CrossRef Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004)CrossRef
20.
go back to reference Yu, X., Huang, J., Zhang, S., Yan, W., Metaxas, D.N.: Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1944–1951 (2013) Yu, X., Huang, J., Zhang, S., Yan, W., Metaxas, D.N.: Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1944–1951 (2013)
21.
go back to reference Dalal, N., & Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005 (CVPR 2005), vol. 1, pp. 886–893, IEEE (2005) Dalal, N., & Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005 (CVPR 2005), vol. 1, pp. 886–893, IEEE (2005)
22.
go back to reference Ekman, P., Friesen, W.V., Hager, J.C.: Facial Action Coding System (FACS). A Technique for the Measurement of Facial Action. Consulting, Palo Alto, 22 (1978) Ekman, P., Friesen, W.V., Hager, J.C.: Facial Action Coding System (FACS). A Technique for the Measurement of Facial Action. Consulting, Palo Alto, 22 (1978)
23.
go back to reference Knerr, S., Personnaz, L., Dreyfus, G.: Single-layer learning revisited: a stepwise procedure for building and training a neural network. In: Neurocomputing, pp. 41–50. Springer, Berlin (1990) Knerr, S., Personnaz, L., Dreyfus, G.: Single-layer learning revisited: a stepwise procedure for building and training a neural network. In: Neurocomputing, pp. 41–50. Springer, Berlin (1990)
24.
go back to reference Hsu, C.W., Chang, C.C., Lin, C.J.: A Practical Guide to Support Vector Classification. (2003) Hsu, C.W., Chang, C.C., Lin, C.J.: A Practical Guide to Support Vector Classification. (2003)
25.
go back to reference Aly, S., Trubanova, A., Abbott, L., White, S., Youssef, A.: VT-KFER: a Kinect-based RGBD+ time dataset for spontaneous and non-spontaneous facial expression recognition. In: 2015 International Conference on Biometrics (ICB), pp. 90–97. IEEE (2015) Aly, S., Trubanova, A., Abbott, L., White, S., Youssef, A.: VT-KFER: a Kinect-based RGBD+ time dataset for spontaneous and non-spontaneous facial expression recognition. In: 2015 International Conference on Biometrics (ICB), pp. 90–97. IEEE (2015)
Metadata
Title
RGB-D Sensor for Facial Expression Recognition in AAL Context
Authors
Andrea Caroppo
Alessandro Leone
Pietro Siciliano
Copyright Year
2018
DOI
https://doi.org/10.1007/978-3-319-66802-4_39