Informational analysis: a Shannon theoretic approach to measure the performance of a diagnostic test

Diagnostic test accuracy, based on sensitivity, specificity, positive/negative predictive values (dichotomous case), and on ROC analysis (continuous case), should be expressed with a single, coherent index. We propose to modelize the diagnostic test as a flow of information between the disease, that is, a hidden state of the patient, and the physicians. We assume that (1) sensitivity, specificity, and false-positive/false-negative rates are the probabilities of a binary asymmetric channel; (2) the diagnostic channel information is measured by mutual information. We introduce two summary measures of accuracy, namely the information ratio (IR) for the dichotomous case, and the global information ratio (GIR) for the continuous case. We apply our model to a study by Pisano et al. (N Engl J Med 353(17):1773–1783, 2005), who compared digital versus film mammography, in diagnosing breast cancer in a screening population of 42,760 women. In film mammography, the maximum IR (0.178) corresponds to the standard cutoff of sensitivity and specificity provided by the ROC analysis (GIR 0.200). Maximum IR and GIR for digital mammography are higher (0.201 and 0.229, respectively), but IR corresponds to a cutoff with higher sensitivity but lower specificity, thus suggesting that larger information provided by digital mammography carries the risk of more false-positive cases.