Assessment of meaningful change in routine outcome measurement (ROM) with a combination of a longitudinal and a ‘classify and count’ approach

To assess significant changes of health status in people receiving health care, distribution-based and anchor-based methods have been proposed. However, there is no real consensus on what method is the best for evaluating clinically meaningful change. To maximize the internal and external validity of outcome assessment, we propose combining two approaches as recommended by recent practical guidelines on this field. Specifically, we suggest applying longitudinal hierarchical linear models on subgroups of patients showing reliable change and reliable and clinically significant change. This combined approach improved the model’s ability (1) to quantify the magnitude of changes to be reliable and clinically meaningful and (2) to select significant predictors of changes. An empirical application on a prevalence sample of Italian outpatients attending four community mental health services was done. A cross-sectional model and three longitudinal models were applied on the entire study sample and reliable and clinically meaningful change subsamples to investigate the magnitude of change and the predictive effect on outcomes of clinical, socio-demographic and process variables on different patients’ subgroups. Differences were found suggesting that both the statistical method and the sample used to calculate individual changes affect the estimates. The main conclusion is that ignoring the longitudinal data structure or including patients with unreliable change at the follow-up might result in misleading inferences that can alter the real magnitude of changes and the contributions of predictors. The approach proposed provides robust feedback to clinicians on clinically significant change and can be recommended in outcome studies and research.