Reduction of Dimensionality of Feature Vectors in Subject Classification of Text Documents

Within a paper we investigate the influence of dimensionality reduction of feature vector (PCA and random projection) on the results of subject classification of text documents in Polish. Two state of the art methods of text representation have been applied, i.e. bag of words consisting of the most frequent 1000 lemmatized nouns and the emerging fastText based on the word embedding dimensionality equal to 100. The methods have been evaluated on four corpora in Polish (two different data sets, divided into training and testing set in different proportions). Results show that PCA gives better accuracy in all analyzed cases. In case of fastText, a significant reduction of up to 10 times is possible without loss of quality regardless the quality of corpus. To analyse this phenomenon, we have performed a set of experiments in which we train fastText with different values of word embedding dimensionality (from 2 to recommended 100). Experiments show that even the dimension 3–10 (depending on the quality of data) allows to achieve very good accuracy.