Investigating Ensemble Weight and the Certainty Distributions for Indicating Structural Diversity

In this paper an investigation of the distribution of the weights and the biases of the Multilayered Perceptron is conducted, in particular the variance of the weight vector (weights and biases) with the aim of indicating the existence of the structural diversity within the ensemble. This will indicate how well the weight vector samples are distributed from the mean and this will be used to serve as an indicator of the structural diversity of the classifiers within the ensemble. This is inspired by the fact that many measures of ensemble diversity are focused on the outcomes and not the classifier’s structure and hence may lose out in diversity measures that correlate well with ensemble performance. Three ensembles were compared, one non-diverse and the other two ensembles made diverse. The generalization across all the ensembles was approximately the same (74 % accuracy). This could be attributed to the data used. Certainty measures were also conducted and indicated that the non-diverse ensemble was biased, even though the performance across the ensembles was the same.