Acoustic feature selection for automatic emotion recognition from speech

Abstract

Emotional expression and understanding are normal instincts of human beings, but automatical emotion recognition from speech without referring any language or linguistic information remains an unclosed problem. The limited size of existing emotional data samples, and the relative higher dimensionality have outstripped many dimensionality reduction and feature selection algorithms. This paper focuses on the data preprocessing techniques which aim to extract the most effective acoustic features to improve the performance of the emotion recognition. A novel algorithm is presented in this paper, which can be applied on a small sized data set with a high number of features. The presented algorithm integrates the advantages from a decision tree method and the random forest ensemble. Experiment results on a series of Chinese emotional speech data sets indicate that the presented algorithm can achieve improved results on emotional recognition, and outperform the commonly used Principle Component Analysis (PCA)/Multi-Dimensional Scaling (MDS) methods, and the more recently developed ISOMap dimensionality reduction method.

Introduction

Emotional recognition is a common instinct for human beings, which has been studied by researchers from different disciplines for more than 70 years (Fairbanks and Pronovost, 1939, Fairbanks and Pronovost, 1941). Fairbanks et al’s pioneering work on emotional speech (Fairbanks and Pronovost, 1939, Fairbanks and Pronovost, 1941) revealed the importance of vocal cues in the expression of emotion, and the powerful effects of vocal emotion expression on interpersonal interaction. Understanding the emotional state of the speaker during communication can help the listeners to catch more information than is represented by the content of the dialogue sentences, especially to detect the ‘real’ meaning of the speech hidden between words. The practical value of emotion recognition from speech is suggested by the rapidly growing number of areas to which it is being applied, such as humanoid robots, the car industry, calling centers, etc. (Cowie and Cornelius, 2003, Lee and Narayanan, 2004, Lee et al., 2004, Pantic and Rothkrantz, 2003, Schuller et al., 2005).

Although machine learning and data mining techniques have obtained flourishing applications (Mitchell, 1997), only a few works have utilized these powerful tools and achieved better performance in emotion recognition from speech. Here a serious encumbrance is the lack of available emotional speech data. There is only few public benchmark databases available for research purpose.

A sufficient number of training examples is the premise for most machine learning and data mining algorithms to work well. When there are only a few training examples, it is very possible to have the problem of overfitting, which means that a model can be trained with perfect performance on the training set, but can hardly generalize well on new examples. In practise, how many training examples will be adequate is task-dependent; for example, the task of learning the XOR function, four different training examples are sufficient, while for more complex tasks such as emotion recognition, thousands of training examples might still be insufficient. In general, if a data set can not fully cover the whole variable space then the data set is referred to as a small data set. In this sense, the data sets collected for emotion recognition are small because the typical size of the data set is less than 1000, while the number of features is close to 100. Such kind of data scarcity usually outstrips many machine learning and data mining algorithms (Vapnik, 1995).

There are two obvious ways to overcome the problem of data scarcity: one is to collect more data while the second is to design techniques that can deal with small data sets. Considering the fact that further data collection is manual, cost intensive and hard to achieve, it is more feasible and desirable for the second way. Based on this recognition, this paper presents a novel feature selection algorithm, ERFTrees, to extract effective features from small data sets. There are two facets of benefits by using this algorithm for emotion recognition: firstly, the irrelevant data can be removed and the dimensionality of the training data can be reduced; secondly, with a reduced data set, most existing machine learning algorithms which do not work well on small data set, can now produce better recognition accuracy. The empirical results on Chinese (Mandarin) emotional data sets indicate that the presented algorithm outperforms other linear and non-linear dimensionality reduction methods, including Principle Component Analysis (PCA), Multi-Dimensional Scaling (MDS), and ISOMap.

The rest of the paper is organized as follows: we introduce the background and the related work in Section 2. The algorithm, ERFTrees, is presented in Section 3. The experiment design and empirical results are presented in Section 4, and finally in Section 5, we conclude the paper with a perspective analysis of possible future work.