Mengyi Liu
ABSTRACT
In this paper, we propose a method for video-based human emotion recognition. For each video clip, all frames are represented as an image set, which can be modeled as a linear subspace to be embedded in Grassmannian manifold. After feature extraction, Class-specific One-to-Rest Partial Least Squares (PLS) is learned on video and audio data respectively to distinguish each class from the other confusing ones. Finally, an optimal fusion of classifiers learned from both modalities (video and audio) is conducted at decision level. Our method is evaluated on the Emotion Recognition In The Wild Challenge (EmotiW 2013). The experimental results on both validation set and blind test set are presented for comparison. The final accuracy achieved on test set outperforms the baseline by 26%.
- H. Cevikalp and B. Triggs. Face recognition based on image sets. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pages 2567--2573, 2010.Google Scholar
Cross Ref
- A. Dhall, R. Goecke, J. Joshi, M. Wagner, and T. Gedeon. Emotion recognition in the wild challenge 2013. In ACM International Conference on Multimodal Interaction (ICMI), pages 2496--2503. ACM, 2012. Google ScholarDigital Library
- A. Dhall, R. Goecke, S. Lucey, and T. Gedeon. Collecting large, richly annotated facial-expression databases from movies. IEEE MultiMedia, 19(3):0034, 2012. Google ScholarDigital Library
- F. Eyben, M. Wöllmer, and B. Schuller. Opensmile: the munich versatile and fast open-source audio feature extractor. In ACM International Conference on Multimedia, pages 1459--1462. ACM, 2010. Google ScholarDigital Library
- G. Guo and G. Mu. Simultaneous dimensionality reduction and human age estimation via kernel partial least squares regression. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 657--664, 2011. Google ScholarDigital Library
- J. Hamm and D. D. Lee. Grassmann discriminant analysis: a unifying view on subspace-based learning. In Proceedings of the 25th international conference on Machine learning, pages 376--383, 2008. Google ScholarDigital Library
- M. T. Harandi, C. Sanderson, S. Shirazi, and B. C. Lovell. Graph embedding discriminant analysis on grassmannian manifolds for improved image set matching. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 2705--2712, 2011. Google ScholarDigital Library
- T.-K. Kim, J. Kittler, and R. Cipolla. Discriminative learning and recognition of image set classes using canonical correlations. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 29(6):1005--1018, 2007. Google ScholarDigital Library
- S. Koelstra, M. Pantic, and I. Patras. A dynamic texture-based approach to recognition of facial actions and their temporal models. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 32(11):1940--1954, 2010. Google ScholarDigital Library
- M. Liu, S. Li, S. Shan, and X. Chen. Au-aware deep networks for facial expression recognition. In IEEE International Conference on Automatic Face and Gesture Recognition (FG), 2013.Google Scholar
- M. Pantic and L. J. M. Rothkrantz. Automatic analysis of facial expressions: The state of the art. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 22(12):1424--1445, 2000. Google ScholarDigital Library
- R. Rosipal and N. Krämer. Overview and recent advances in partial least squares. In Subspace, Latent Structure and Feature Selection, pages 34--51. Springer, 2006. Google ScholarDigital Library
- M. A. Turk and A. P. Pentland. Face recognition using eigenfaces. In Computer Vision and Pattern Recognition (CVPR), IEEE Conference on, pages 586--591, 1991.Google Scholar
- R. Wang, H. Guo, L. S. Davis, and Q. Dai. Covariance discriminative learning: A natural and efficient approach to image set classification. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 2496--2503, 2012. Google ScholarDigital Library
- R. Wang, S. Shan, X. Chen, and W. Gao. Manifold-manifold distance with application to face recognition based on image set. In Computer Vision and Pattern Recognition (CVPR), IEEE Conference on, pages 1--8, 2008.Google Scholar
- T. Wang and P. Shi. Kernel grassmannian distances and discriminant analysis for face recognition from image sets. Pattern Recognition Letters, 30(13):1161--1165, 2009. Google ScholarDigital Library
- H. Wold. Partial least squares. Encyclopedia of statistical sciences, 1985.Google Scholar
- P. Yang, Q. Liu, and D. N. Metaxas. Boosting coded dynamic features for facial action units and facial expression recognition. In Computer Vision and Pattern Recognition (CVPR), IEEE Conference on, pages 1--6, 2007.Google Scholar
- Y. Zhang and J. Qiang. Active and dynamic information fusion for facial expression understanding from image sequences. Pattern Analysis and Machine Intelligence, IEEE Transaction on, 27(5):699--714, 2005. Google ScholarDigital Library
- G. Zhao and M. Pietikainen. Dynamic texture recognition using local binary patterns with an application to facial expressions. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 29(6):915--928, 2007. Google ScholarDigital Library
Index Terms
- Partial least squares regression on grassmannian manifold for emotion recognition
Recommendations
Human gait recognition using localized Grassmann mean representatives with partial least squares regression
Gait recognition has become popular due to the rising demand for nonintrusive biometrics. At its nascent stage of development, gait recognition faces a number of challenges. The performance of a gait recognition system is sensitive towards factors like ...
Graph embedding discriminant analysis on Grassmannian manifolds for improved image set matching
CVPR '11: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern RecognitionA convenient way of dealing with image sets is to represent them as points on Grassmannian manifolds. While several recent studies explored the applicability of discriminant analysis on such manifolds, the conventional formalism of discriminant analysis ...
Comparison of Principal Component Regression and Partial Least Squares Regression by R
ICEICE '12: Proceedings of the 2012 Second International Conference on Electric Information and Control Engineering - Volume 04Principle component regression (PCR) and partial least squares regression (PLSR) are two methodologies commonly used to solve dimension reduction and the multi-co linearity problem. The goal of this paper is to analyze and predict dependent variable ...
Comments