Improving Clinical Subjects Clustering by Learning and Optimizing Feature Weights

Data analytics methods in the clinical domain are challenging to put into practice. Unsupervised learning provides opportunity for giving the level of personalization in evidence based decision-making that can otherwise only be achieved through the use of prediction models, by helping doctors gaining insights from data. In this context, grouping of clinical subjects, in terms of biomedical information of patients, is an important task for patient cohort identification for comparative effectiveness studies and clinical decision-support applications. It allows the decision-making process to leverage not only on data but also on doctors’ domain knowledge. However, one of the issues that needs to be addressed for a focused and realist unsupervised clustering of clinical subjects, is the fact that in the majority of the cases patients datasets are heterogeneous, i.e. their data features belong to several different feature spaces, e.g. nominal, ordinal, interval or rational, with completely different variation ranges and statistical distributions, affecting clustering quality and performance. In order to use these data measurements properly in an unsupervised manner, their corresponding weights need to be modeled. In this paper, we present a method for learning feature weights on clinical data. We show that learning feature weights is necessary in order to generate meaningful separation of data in high dimensional space. The method is based on silhouette score and principal component analysis, demonstrating its performance on a clinical test dataset.