A fuzzy set theory based computational model to represent the quality of inter-rater agreement

A strategy that tries to avoid such problems is the centralization of the training, namely only a specific research group, usually the one that has developed the instrument, is recognized as legitimated to train the raters, and this is often formalized in terms of a specific program to be attended for formal certificating (e.g., the procedure for obtaining the certification required for using the Adult Attachment Interview). Yet this solution is only partial, and has more costs than benefits. First, the centralization of the training does not solve the problem of the application of the instrument to clinical material reflecting a different cultural context. Thus, rater B can be trained by rater A whose way of coding is taken as normative; and so B can get a satisfactory level of agreement. Yet such agreement concerns specific objects, the ones used in the training and it is not obvious that it can be generalized to Bs cultural or research contexts. Secondly, due to its cost, the centralization of the training is a mechanism that can treat only a very limited subset of raters and instruments. Above all, the centralization of the training, as with any form of monopoly, reduces the free circulation and exchange of knowledge that is the ground of any scientific community.

The use of the PCA for basic dimension characterization and the description of a clinical process is a procedure widely used in studies using such methods.

Briefly, we set up a specific algorithm based on the concepts of data information and entropy; in particular, this procedure allowed us to reconstruct an histogram related to a dataset with a specific fuzzy set (e.g., a triangular fuzzy set) capturing the whole information, as measured by the entropy formula, that was present in the empirical data. Strictly speaking, if we have a data vector represented by a specific histogram, we can express it by the best fuzzy set capturing the information present in the vector (using a specific formula, De Luca and Termini’s fuzzy weighted entropy), thus the fuzzy set obtained in this way best describes the empirical data vector. For the purposes of this paper, we do not report other details, which can be requested directly from the authors.

The fuzziness computed by the system is related to the definition of the overlap space between the fuzzy sets, and thus the greater is the overlap space, the greater is the possibility of having judgement scores with an high degree of fuzziness.