Complement information entropy for uncertainty measure in fuzzy rough set and its applications

Uncertainty measure is an important tool for analyzing imprecise and ambiguous data. Some information entropy models in rough set theory have been defined for various information systems. However, there are relatively few studies on evaluating uncertainty in fuzzy rough set. In this paper, we propose a new complement information entropy model in fuzzy rough set based on arbitrary fuzzy relation, which takes inner-class and outer-class information into consideration. The corresponding definitions of complement conditional entropy, complement joint entropy, complement mutual information and complement information granularity are also presented. The properties of these definitions are analyzed, which show complement information entropy shares some similar properties with Shannon’s entropy. Moreover, a generalized information entropy model is proposed by introducing probability distribution into fuzzy approximate space. This model can be used to measure uncertainty of data with the different sample distributions. Applications of the proposed entropy measures in feature importance evaluation and feature selection are studied with data set experiments. Experimental results show that the proposed method is effective and adaptable to different classifiers.