Clustering Research Papers: A Qualitative Study of Concatenated Power Means Sentence Embeddings over Centroid Sentence Embeddings

Mathematical average of word embeddings is a common baseline for sentence embedding techniques which typically fall short of the performance of more complex models such as BERT and InferSent. There has been significant improvement in the field of sentence embeddings and especially towards the development of universal sentence encoder that can be used for transfer learning in a wide variety of downstream tasks. Academic paper retrieval systems are widely used in academic institutions to store and categorise scientific papers and find connections between them using citation links, but these methods do not account for the content of the papers. For unsupervised clustering of these papers, a new approach of sentence embeddings is proposed using concatenated power means sentence embeddings and centroid sentence embeddings. The sentence embeddings so created are clustered using K-means clustering algorithm. The results show a clear increase of 47.94% in cosine distance of nearest papers using concatenated power means sentence embeddings with respect to baseline centroid embeddings for the highest performing GloVe models proving that the computationally inexpensive P-Means clustering sentence embeddings can be used for unsupervised clustering of scientific research papers using their abstracts.