Xiaohua Huang
ABSTRACT
Local binary pattern from three orthogonal planes (LBP-TOP) has been widely used in emotion recognition in the wild. However, it suffers from illumination and pose changes. This paper mainly focuses on the robustness of LBP-TOP to unconstrained environment. Recent proposed method, spatiotemporal local monogenic binary pattern (STLMBP), was verified to work promisingly in different illumination conditions. Thus this paper proposes an improved spatiotemporal feature descriptor based on STLMBP. The improved descriptor uses not only magnitude and orientation, but also the phase information, which provide complementary information. In detail, the magnitude, orientation and phase images are obtained by using an effective monogenic filter, and multiple feature vectors are finally fused by multiple kernel learning. STLMBP and the proposed method are evaluated in the Acted Facial Expression in the Wild as part of the 2014 Emotion Recognition in the Wild Challenge. They achieve competitive results, with an accuracy gain of 6.35% and 7.65% above the challenge baseline (LBP-TOP) over video.
- T. Almaev and M. Valstar. Local Gabor binary patterns from three orthogonal planes for automatic facial expression recognition. In Humaine Association Conference on Affective Computing and Intelligent Interaction, pages 356--361, 2013. Google Scholar
Digital Library
- T. Almaev, A. Yüce, A. Ghitulescu, and M. Valstar. Distribution-based interative parirwise classification of emotions in the wild using LGBP-TOP. In ACM International Conference on Multimodal Interaction, pages 535--541, 2013. Google ScholarDigital Library
- C. Chang and C. Lin. Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(27):1--27, 2011. Google ScholarDigital Library
- J. Daugman. High confidence visual recognition of persons by a test of statistical independence. IEEE Transaction on Pattern Analysis and Machine Intelligence, 15(11):1148--1161, 1993. Google ScholarDigital Library
- A. Dhall, R. Goecke, J. Joshi, K. Sikka, and T. Gedeon. Emotion recognition in the wild challenge 2014: Baseline, data and protocol. In ACM International Conference on Multimodal Interaction, 2014. Google ScholarDigital Library
- A. Dhall, R. Goecke, J. Joshi, M. Wagner, and T. Gedeon. Emotion recognition in the wild challenge (EmotiW) challenge and workshop summary. In ACM International Conference on Multimodal Interaction, pages 371--372, 2013. Google ScholarDigital Library
- A. Dhall, R. Goecke, S. Lucey, and T. Gedeon. Collecting large, richly annotated facial-expression databases from movies. IEEE Multimedia, 19(3):34--41, 2012. Google ScholarDigital Library
- newblock Distribution-based interative parirwise classification of emotions in the wild using {LGBP-TOP}.Google Scholar
- P. Ekman and W. Friesen. Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, 1978. ACM International Conference on Multimodal Interaction}, pages 535--541, 2013.Google Scholar
- M. Felsberg and G. Sommer. The monogenic signal. IEEE Transactions on Signal Processings, 49(12):3136--3144, 2001. Google ScholarDigital Library
- D. Field. Relations between the statistics of natural images and the response properties of cortical cells. Journal of the Optical Society of America A, 4(12):2379--2394, 1987.Google ScholarCross Ref
- J. Hong and K. Song. Facial expression recognition under illumination variation. In Advanced Robotics and Its Social Impacts, pages 1--6, 2007.Google Scholar
- X. Huang, G. Zhao, W. Zheng, and M. Pietikäinen. Spatiotemporal local monogenic binary patterns for facial expression recognition. IEEE Signal Processing Letters, 19(5):243--246, May 2012.Google ScholarCross Ref
- S. Kahou, C. Pal, X. Bouthillier, P. Froumenty, C. Gülcehre, R. Memisevic, P. Vincent, A. Courville, and Y. Bengio. Combining modality specific deep neural networks for emotion recognition in video. In ACM International Conference on Multimodal Interaction, pages 543--550, 2013. Google ScholarDigital Library
- I. Kotsia, I. Buciu, and I. Pitas. An analysis of facial expression recognition under partial facial image occlusion. Image and Vision Computing, 26(7):1052--1067, 2008. Google ScholarDigital Library
- T. krishna, A. Rai, S. Bansal, S. Khandelwal, S. Gupta, and D. Goyal. Emotion recognition using facial and audio features. In International Conference on Language Resources and Evaluation, pages 557--564, 2013. Google ScholarDigital Library
- M. Liu, R. Wang, Z. Huang, S. Shan, and X. Chen. Partial least squares regression on grassmannian manifold for emotion recognition. In ACM International Conference on Multimodal Interaction, pages 525--530, 2013. Google ScholarDigital Library
- P. Lucey, J. Cohn, T. Kanade, J. Saragih, and Z. Ambadar. The extended Cohn-Kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression. In International Conference on Computer Vision and Pattern Recognition, pages 94{101, 2010.Google ScholarCross Ref
- M. Lyons, S. Akamatsu, M. Kamachi, and J. Gyoba. Coding facial expressions with Gabor wavelets. In International Conference on Automatic Face and Gesture Recognition, 1998. Google ScholarDigital Library
- T. Ojala, M. Pietikäinen, and T. Mäenpää. Multiresolution gray-scale and rotation invariant texture classification with local binary pattern. IEEE Transaction on Pattern Analysis and Machine Intelligence, 24(7):971--987, 2002. Google ScholarDigital Library
- F. Orabona and J. Luo. Ultra-fast optimization algorithm for sparse multiple kernel learning. In International Conference on Machine Learning, pages 1--8, 2011.Google Scholar
- K. Sikka, K. Dykstra, S. Sathyanarayana, G. Littlewort, and M. Bartlett. Multiple kernel learning for emotion recognition in the wild. In ACM International Conference on Multimodal Interaction, pages 517--523, 2013. Google ScholarDigital Library
- G. Tzimiropoulos, S. Zafeiriou, and M. Pantic. Subspace learning from image gradient orientations. IEEE Transaction on Pattern Analysis and Machine Intelligence, 34(12):2454--2466, 2012. Google ScholarDigital Library
- M. Valastar and M. Pantic. Induced disgust, happiness and surprise: an addition to the MMI facial expression database. In International Conference on Language Resources and Evaluation, pages 65--70, 2012.Google Scholar
- S. Xie, S. Shan, X. Chen, and J. Chen. Fusing local patterns of Gabor magnitude and phase for face recognition. IEEE Transactions on Image Processing, 19(5):1349--1361, January 2010. Google ScholarDigital Library
- M. Yang, L. Zhang, L. Zhang, and D. Zhang. Monogenic binary pattern (MBP): A novel feature extraction and representation model for face recognition. In International Conference on Pattern Recognition, pages 2683--2690, 2010. Google ScholarDigital Library
- Z. Zeng, M. Pantic, G. Roisman, and T. Huang. A survey of affect recognition methods: audio, visual and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1):39--58, 2009. Google ScholarDigital Library
- L. Zhang, L. Zhang, Z. Guo, and D. Zhang. Monogenic-LBP: A new approach for rotation invariant texture classification. In International Conference on Image Processing, pages 2677--2680, 2010.Google ScholarCross Ref
- W. Zhang, S. Shan, X. Chen, and W. Gao. Are Gabor phases really useless for face recognition. Pattern Analysis and Application, 12(3):301--307, 2009. Google ScholarDigital Library
- W. Zhang, S. Shan, X. Chen, and W. Gao. Local Gabor binary pattern based on kullback-leilber divergence for partially occluded face recognition. IEEE Signal Processing Letters, 14(11):875--878, 2010.Google ScholarCross Ref
- G. Zhao and M. Pietikäinen. Dynamic texture recognition using local binary pattern with application to facial expressions. IEEE Transaction on Pattern Analysis and Machine Intelligence, 29(6):915--928, 2007. Google ScholarDigital Library
- X. Zhu and D. Ramanan. Face detection, pose estimation and landmark localization in the wild. In Computer Vision and Pattern Recognition, pages 1--8, 2012. Google ScholarDigital Library
Index Terms
- Improved Spatiotemporal Local Monogenic Binary Pattern for Emotion Recognition in The Wild
Recommendations
3D face recognition using local binary patterns
It is well recognized that expressions can significantly change facial geometry that results in a severe problem for robust 3D face recognition. So it is crucial for many applications that how to extract expression-robust features to describe 3D faces. ...
Local binary patterns for multi-view facial expression recognition
Research into facial expression recognition has predominantly been applied to face images at frontal view only. Some attempts have been made to produce pose invariant facial expression classifiers. However, most of these attempts have only considered ...
Face recognition based on an improved center symmetric local binary pattern
This paper proposes a local texture feature descriptor which fuses the center pixel information into the Center-Symmetric Local Binary Pattern (CS-LBP) for the purpose of face recognition. Because of its tolerance to illumination changes, and ...
Comments