Zero-Shot Estimation of Compound Emotional Expressions from Facial Images Using Only Basic Emotional Expression Features

Facial Expression Recognition (FER) technology can be applied to various systems targeting humans. However, conventional research is biased toward basic emotional expressions such as “Happiness” and “sadness,” which limits its application in real environments where there are many ambiguous and complex emotional expression. In particular, data collection and annotation are difficult, making database construction a bottleneck and a barrier to achieving highly accurate recognition. Based on the theory that complex emotional expressions are formed by combining basic emotional expressions, we propose a zero-shot method for estimating complex emotional expressions from basic emotional expression features. First, a continuous feature space reflecting the relationships among emotions is learned using facial images corresponding to six basic emotional expressions. Next, the middle point of the basic emotional expression is defined as the representative vector of the complex emotional expression in the space, and estimation is performed based on the similarity with the input image. This makes it possible to construct that feature space without using any data on complex emotional expressions, thereby overcoming the problem of insufficient data. The method outperforms supervised methods despite the fact that it does not use any supervised data on complex emotional expressions. Furthermore, despite using only facial expression data, the accuracy of the method is comparable to that of conventional multimodal methods.