Graph regularized linear discriminant analysis and its generalization

Linear discriminant analysis (LDA) is a powerful dimensionality reduction technique, which has been widely used in many applications. Although, LDA is well-known for its discriminant capability, it clearly does not capture the geometric structure of the data. However, from the geometric perspective, the high-dimensional data resides on some low-dimensional manifolds in the sample space and these manifold structures are essential for data clustering and classification. In this paper, we propose a novel LDA algorithm named graph regularized linear discriminant analysis (GRLDA) to further improve the conventional LDA by incorporating such geometric information of data. GRLDA is achieved by penalizing the LDA with a Graph regularization, which is an affinity matrix encoding the geometric relationship of the data points. To take high-order geometric relationship among samples into consideration, we generalize GRLDA via using the hypergraph regularization instead of the graph regularization. We name this new version as hyper graph regularized linear discriminant analysis. Moreover, we exploit the null space of LDA via using an identity matrix to regularize the between-class scatter matrix. This strategy can further improve the discriminating power of LDA algorithms. Four popular face databases are used to evaluate our proposed LDA algorithms and the results of experiments demonstrate that they outperform the state-of-the-art dimensionality reduction algorithms.