Top

Quality of Life Research

Published in:

28-07-2021 | Review

Triangulation of multiple meaningful change thresholds for patient-reported outcome scores

Authors: Andrew Trigg, Pip Griffiths

Published in: Quality of Life Research | Issue 10/2021

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Purpose

The notion of what constitutes meaningful differences or changes in patient-reported outcome scores is represented by meaningful change thresholds (MCTs). Applying multiple methods to estimate MCTs inevitably results in a range of estimates; however, a single estimate or small range is sought in practice to enable consistent interpretation of scores. While current recommendations for triangulation are appropriate in principle, the vital step of moving from all estimates to a value or small range lacks clarity and is subjective in nature. This article aims to review current triangulation approaches and provide more robust recommendations than what is currently available.

Methods

Current approaches to perform triangulation are described and discussed. Anchor-based estimates are focussed upon due to their recognition as the most valid and developed approach. Recommendations for triangulation are provided.

Results

A correlation-weighted average of MCT estimates is recommended to triangulate multiple MCT estimates derived from a single study into a single value, where increased weighting is given to stronger anchor measures. The choice of method to triangulate estimates from several published studies is highly dependent on the availability of information within the publications. MCTs designed for between-group differences, within-group changes, and within-individual changes should be considered separately.

Conclusion

The recommendations within this article provide a reliable and transparent approach to triangulation when a single value is sought, based on meta-analytic approaches. This approach is preferable to a simple mean of estimates where all are weighted equally, or through ‘eyeballing’ plotted estimates which is unreliable. We encourage researchers to adopt these methods, but to remain aware of the limitations within each method and further nuances in study design that result in heterogeneity. Sensitivity analyses with a range of plausible values are encouraged; however, the recommendations provide a suitable starting value for inferences. Unresolved issues in triangulation, requiring further exploration, are highlighted.

previous article Minimal important change (MIC): a conceptual clarification and systematic review of MIC estimates of PROMIS measures

next article Between-group minimally important change versus individual treatment responders

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

King, M. T. (2011). A point of minimal important difference (MID): A critique of terminology and methods. Expert Review of Pharmacoeconomics & Outcomes Research, 11(2), 171–184.CrossRef

Bell, M. L., Dhillon, H. M., Bray, V. J., & Vardy, J. L. (2018). Important differences and meaningful changes for the Functional Assessment of Cancer Therapy-Cognitive Function (FACT-Cog). Journal of Patient-Reported Outcomes, 2(1), 48. https://doi.org/10.1186/s41687-018-0071-4CrossRefPubMedCentral

King, M. T., Dueck, A. C., & Revicki, D. A. (2019). Can methods developed for interpreting group-level patient-reported outcome data be applied to individual patient management? Medical Care, 57(Suppl 5 1), S38–S45. https://doi.org/10.1097/mlr.0000000000001111CrossRefPubMedPubMedCentral

Cocks, K., King, M. T., Velikova, G., de Castro, G., Jr., Martyn St-James, M., Fayers, P. M., et al. (2012). Evidence-based guidelines for interpreting change scores for the European Organisation for the Research and Treatment of Cancer Quality of Life Questionnaire Core 30. European Journal of Cancer, 48(11), 1713–1721. https://doi.org/10.1016/j.ejca.2012.02.059CrossRefPubMed

Musoro, J. Z., Bottomley, A., Coens, C., Eggermont, A. M., King, M. T., Cocks, K., et al. (2018). Interpreting European Organisation for Research and Treatment for Cancer Quality of life Questionnaire core 30 scores as minimally importantly different for patients with malignant melanoma. European Journal of Cancer, 104, 169–181. https://doi.org/10.1016/j.ejca.2018.09.005CrossRefPubMed

Musoro, J. Z., Coens, C., Fiteni, F., Katarzyna, P., Cardoso, F., Russell, N. S., et al. (2019). Minimally important differences for interpreting EORTC QLQ-C30 scores in patients with advanced breast cancer. JNCI Cancer Spectrum, 3(3), pkz037. https://doi.org/10.1093/jncics/pkz037CrossRefPubMedPubMedCentral

United States Food and Drug Administration. (2009). Patient-reported outcome measures: Use in medical product development to support labeling claims. Guidance for Industry. https://www.fda.gov/media/77832/download. Accessed 26 July 2021.

Globe, G., Wiklund, I., Mattera, M., Zhang, H., & Revicki, D. A. (2019). Evaluating minimal important differences and responder definitions for the asthma symptom diary in patients with moderate to severe asthma. Journal of Patient-Reported Outcomes, 3(1), 22. https://doi.org/10.1186/s41687-019-0109-2CrossRefPubMedPubMedCentral

Coon, C. D., & Cappelleri, J. C. (2016). Interpreting change in scores on patient-reported outcome instruments. Therapeutic Innovation & Regulatory Science, 50(1), 22–29.CrossRef

10.

Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18(2), 5–11.CrossRef

11.

Revicki, D., Hays, R. D., Cella, D., & Sloan, J. (2008). Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. Journal of Clinical Epidemiology, 61(2), 102–109.CrossRefPubMed

12.

Staunton, H., Willgoss, T., Nelsen, L., Burbridge, C., Sully, K., Rofail, D., et al. (2019). An overview of using qualitative techniques to explore and define estimates of clinically important change on clinical outcome assessments. Journal of Patient-Reported Outcomes, 3(1), 1–10.CrossRef

13.

Harvill, L. M. (1991). Standard error of measurement. Educational Measurement: Issues and Practice, 10(2), 33–41. https://doi.org/10.1111/j.1745-3992.1991.tb00195.xCrossRef

14.

Lord, F. M., Novick, M. R., & Birnbaum, A. (1968). Statistical theories of mental test scores. Addison-Wesley.

15.

Hays, R. D., Farivar, S. S., & Liu, H. (2005). Approaches and recommendations for estimating minimally important differences for health-related quality of life measures. COPD, 2(1), 63–67. https://doi.org/10.1081/copd-200050663CrossRefPubMed

16.

Terwee, C. B., Roorda, L. D., Dekker, J., Bierma-Zeinstra, S. M., Peat, G., Jordan, K. P., et al. (2010). Mind the MIC: Large variation among populations and methods. Journal of Clinical Epidemiology, 63(5), 524–534.CrossRefPubMed

17.

Gerlinger, C., & Schmelter, T. (2011). Determining the non-inferiority margin for patient reported outcomes. Pharmaceutical Statistics, 10(5), 410–413. https://doi.org/10.1002/pst.507CrossRefPubMed

18.

de Vet, H. C. W., Ostelo, R. W. J. G., Terwee, C. B., van der Roer, N., Knol, D. L., Beckerman, H., et al. (2007). Minimally important change determined by a visual method integrating an anchor-based and a distribution-based approach. Quality of Life Research, 16(1), 131–142. https://doi.org/10.1007/s11136-006-9109-9CrossRefPubMed

19.

Hays, R. D., & Woolley, J. M. (2000). The concept of clinically meaningful difference in health-related quality-of-life research. How meaningful is it? PharmacoEconomics, 18(5), 419–423. https://doi.org/10.2165/00019053-200018050-00001CrossRefPubMed

20.

Santanello, N. C., Zhang, J., Seidenberg, B., Reiss, T. F., & Barber, B. L. (1999). What are minimal important changes for asthma measures in a clinical trial? European Respiratory Journal, 14(1), 23–27. https://doi.org/10.1034/j.1399-3003.1999.14a06.xCrossRefPubMed

21.

Wang, Y.-C., Hart, D. L., Stratford, P. W., & Mioduski, J. E. (2011). Baseline dependency of minimal clinically important improvement. Physical Therapy, 91(5), 675–688. https://doi.org/10.2522/ptj.20100229CrossRefPubMed

22.

Halme, A. S., Fritel, X., Benedetti, A., Eng, K., & Tannenbaum, C. (2015). Implications of the minimal clinically important difference for health-related quality-of-life outcomes: A comparison of sample size requirements for an incontinence treatment trial. Value in Health, 18(2), 292–298. https://doi.org/10.1016/j.jval.2014.11.004CrossRefPubMed

23.

European Medicines Agency. (2016). Guideline on multiplicity issues in clinical trials—draft. https://www.ema.europa.eu/en/documents/scientific-guideline/draft-guideline-multiplicity-issues-clinical-trials_en.pdf. Accessed 26 July 2021.

24.

The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. (1998). ICH Topic E9 Statistical Principles for Clinical Trials. https://www.ema.europa.eu/en/documents/scientific-guideline/ich-e-9-statistical-principles-clinical-trials-step-5_en.pdf. Accessed 26 July 2021.

25.

Schwind, J., Learman, K., O’Halloran, B., Showalter, C., & Cook, C. (2013). Different minimally important clinical difference (MCID) scores lead to different clinical prediction rules for the Oswestry disability index for the same sample of patients. Journal of Manual & Manipulative Therapy, 21(2), 71–78. https://doi.org/10.1179/2042618613Y.0000000028CrossRef

26.

Sloan, J. A., Cella, D., & Hays, R. D. (2005). Clinical significance of patient-reported questionnaire data: Another step toward consensus. Journal of Clinical Epidemiology, 58(12), 1217–1219. https://doi.org/10.1016/j.jclinepi.2005.07.009CrossRefPubMed

27.

Wyrwich, K. W., Metz, S. M., Kroenke, K., Tierney, W. M., Babu, A. N., & Wolinsky, F. D. (2007). Triangulating patient and clinician perspectives on clinically important differences in health-related quality of life among patients with heart disease. Health Services research, 42(6p1), 2257–2274.CrossRefPubMedPubMedCentral

28.

Myles, P. S., Myles, D. B., Galagher, W., Chew, C., MacDonald, N., & Dennis, A. (2016). Minimal clinically important difference for three quality of recovery scales. Anesthesiology, 125(1), 39–45. https://doi.org/10.1097/aln.0000000000001158CrossRefPubMed

29.

Ousmen, A., Touraine, C., Deliu, N., Cottone, F., Bonnetain, F., Efficace, F., et al. (2018). Distribution- and anchor-based methods to determine the minimally important difference on patient-reported outcome questionnaires in oncology: A structured review. Health and Quality of Life Outcomes, 16(1), 228. https://doi.org/10.1186/s12955-018-1055-zCrossRefPubMedPubMedCentral

30.

Griffiths, P., Williams, A., Brohan, E., & Cocks, K. (2019). Understanding the role of anchor correlations in the calculation of meaningful change thresholds for health-related quality of life research. Value in Health, 22, S826. https://doi.org/10.1016/j.jval.2019.09.2266CrossRef

31.

Devji, T., Carrasco-Labra, A., Qasim, A., Phillips, M., Johnston, B. C., Devasenapathy, N., et al. (2020). Evaluating the credibility of anchor based estimates of minimal important differences for patient reported outcomes: Instrument development and reliability study. BMJ, 369, m1714. https://doi.org/10.1136/bmj.m1714CrossRefPubMedPubMedCentral

32.

Harper, A., Trennery, C., Sully, K., & Trigg, A. (2018). Triangulating estimates of meaningful change or difference in patient-reported outcomes: Application of a correlation-based weighting procedure. Quality of Life Research, 27, S17.

33.

Sully, K., Trigg, A., Bonner, N., Moreno-Koehler, A., Trennery, C., Shah, N., et al. (2019). Estimation of minimally important differences and responder definitions for EORTC QLQ-MY20 scores in multiple myeloma patients. European Journal of Haematology, 103(5), 500–509.CrossRefPubMedPubMedCentral

34.

Gatz, D. F., & Smith, L. (1995). The standard error of a weighted mean concentration—I. Bootstrapping vs other methods. Atmospheric Environment, 29(11), 1185–1193. https://doi.org/10.1016/1352-2310(94)00210-CCrossRef

35.

Schisterman, E. F., & Perkins, N. (2007). Confidence intervals for the youden index and corresponding optimal cut-point. Communications in Statistics Simulation and Computation, 36(3), 549–563. https://doi.org/10.1080/03610910701212181CrossRef

36.

Fisher, R. A. (1934). Statistical methods for research workers (5th ed.). Springer.

37.

Jacobs, P., & Viechtbauer, W. (2017). Estimation of the biserial correlation and its sampling variance for use in meta-analysis. Research Synthesis Methods, 8(2), 161–180.CrossRefPubMed

38.

Myers, L., & Sirois, M. J. (2004). Spearman correlation coefficients, differences between. Encyclopedia of Statistical Sciences. https://doi.org/10.1002/0471667196.ess5050.pub2CrossRef

39.

Pustejovsky, J. E. (2014). Converting from d to r to z when the design uses extreme groups, dichotomization, or experimental control. Psychological Methods, 19(1), 92–112. https://doi.org/10.1037/a0033788CrossRefPubMed

40.

Lee, A. C., Driban, J. B., Price, L. L., Harvey, W. F., Rodday, A. M., & Wang, C. (2017). Responsiveness and minimally important differences for 4 patient-reported outcomes measurement information system short forms: Physical function, pain interference, depression, and anxiety in knee osteoarthritis. The Journal of Pain, 18(9), 1096–1110. https://doi.org/10.1016/j.jpain.2017.05.001CrossRefPubMedPubMedCentral

41.

Devji, T., Guyatt, G. H., Lytvyn, L., Brignardello-Petersen, R., Foroutan, F., Sadeghirad, B., et al. (2017). Application of minimal important differences in degenerative knee disease outcomes: A systematic review and case study to inform BMJ Rapid Recommendations. British Medical Journal Open, 7(5), e015587. https://doi.org/10.1136/bmjopen-2016-015587CrossRef

42.

Hao, Q., Devji, T., Zeraatkar, D., Wang, Y., Qasim, A., Siemieniuk, R. A. C., et al. (2019). Minimal important differences for improvement in shoulder condition patient-reported outcomes: A systematic review to inform a BMJ Rapid Recommendation. British Medical Journal Open, 9(2), e028777. https://doi.org/10.1136/bmjopen-2018-028777CrossRef

43.

Johnston, B. C., Ebrahim, S., Carrasco-Labra, A., Furukawa, T. A., Patrick, D. L., Crawford, M. W., et al. (2015). Minimally important difference estimates and methods: A protocol. British Medical Journal Open, 5(10), e007953. https://doi.org/10.1136/bmjopen-2015-007953CrossRef

44.

Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2009). Introduction to meta-analysis. Wiley.CrossRef

45.

Olsen, M. F., Bjerre, E., Hansen, M. D., Tendal, B., Hilden, J., & Hróbjartsson, A. (2018). Minimum clinically important differences in chronic pain vary considerably by baseline pain and methodological factors: Systematic review of empirical studies. Journal of Clinical Epidemiology, 101, 87–106. https://doi.org/10.1016/j.jclinepi.2018.05.007CrossRefPubMed

46.

Olsen, M. F., Bjerre, E., Hansen, M. D., Hilden, J., Landler, N. E., Tendal, B., et al. (2017). Pain relief that matters to patients: Systematic review of empirical studies assessing the minimum clinically important difference in acute pain. BMC Medicine, 15(1), 35. https://doi.org/10.1186/s12916-016-0775-3CrossRefPubMedPubMedCentral

47.

Ebrahim, S., Vercammen, K., Sivanand, A., Guyatt, G. H., Carrasco-Labra, A., Fernandes, R. M., et al. (2017). Minimally important differences in patient or proxy-reported outcome studies relevant to children: A systematic review. Pediatrics, 139(3), e20160833. https://doi.org/10.1542/peds.2016-0833CrossRefPubMed

48.

Chung, J. K., Kannappan, P. L., Ng, C. T., & Sahoo, P. K. (1989). Measures of distance between probability distributions. Journal of Mathematical Analysis and Applications, 138(1), 280–292. https://doi.org/10.1016/0022-247X(89)90335-1CrossRef

49.

Cocks, K., & Buchanan, J. (2015). Defining responders on the European Organization for Research and Treatment of Cancer (EORTC) Quality of Life Questionnaire (30-item core module)(QLQ-C30) subscales. Quality of Life Research, 24, 125.

50.

Molenaar, P. C. M. (2004). A manifesto on psychology as idiographic science: Bringing the person back into scientific psychology, this time forever. Measurement Interdisciplinary Research and Perspectives, 2(4), 201–218. https://doi.org/10.1207/s15366359mea0204_1CrossRef

51.

Vanier, A., Sébille, V., Blanchin, M., & Hardouin, J.-B. (2021). The minimal perceived change: A formal model of the responder definition according to the patient’s meaning of change for patient-reported outcome data analysis and interpretation. BMC Medical Research Methodology, 21(1), 128. https://doi.org/10.1186/s12874-021-01307-9CrossRefPubMedPubMedCentral

52.

Fisher, A. J., Medaglia, J. D., & Jeronimus, B. F. (2018). Lack of group-to-individual generalizability is a threat to human subjects research. Proceedings of the National Academy of Sciences USA, 115(27), E6106. https://doi.org/10.1073/pnas.1711978115CrossRef

53.

McLeod, L. D., Coon, C. D., Martin, S. A., Fehnel, S. E., & Hays, R. D. (2011). Interpreting patient-reported outcome results: US FDA guidance and emerging methods. Expert Review of Pharmacoeconomics & Outcomes Research, 11(2), 163–169. https://doi.org/10.1586/erp.11.12CrossRef

54.

Nixon, A., Doll, H., Kerr, C., Burge, R., & Naegeli, A. N. (2016). Interpreting change from patient reported outcome (PRO) endpoints: Patient global ratings of concept versus patient global ratings of change, a case study among osteoporosis patients. Health and Quality of Life Outcomes, 14(1), 25. https://doi.org/10.1186/s12955-016-0427-5CrossRefPubMedPubMedCentral

55.

Demidenko, E. (2007). Sample size determination for logistic regression revisited. Statistics in Medicine, 26(18), 3385–3397. https://doi.org/10.1002/sim.2771CrossRefPubMed

56.

Avery, K. N. L., Richards, H. S., Portal, A., Reed, T., Harding, R., Carter, R., et al. (2019). Developing a real-time electronic symptom monitoring system for patients after discharge following cancer-related surgery. BMC Cancer, 19(1), 463. https://doi.org/10.1186/s12885-019-5657-6CrossRefPubMedPubMedCentral

Title: Triangulation of multiple meaningful change thresholds for patient-reported outcome scores
Authors: Andrew Trigg
Pip Griffiths
Publication date: 28-07-2021
Publisher: Springer International Publishing
Published in: Quality of Life Research / Issue 10/2021
Print ISSN: 0962-9343
Electronic ISSN: 1573-2649
DOI: https://doi.org/10.1007/s11136-021-02957-4

Springer Professional

Abstract

Purpose

Methods

Results

Conclusion

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 10/2021

Estimating the minimum important difference in the DEMQOL instrument in people with dementia

Support for the higher-order factor structure of the WHODAS 2.0 self-report version in a Dutch outpatient psychiatric setting

Could the EQ-5D-3L predict all-cause mortality in older Chinese? Evidence from a 5-year longitudinal study in eastern China

Quality of life of COVID-19 critically ill survivors after ICU discharge: 90 days follow-up

Correction to: Health-related quality of life in adults with tetralogy of Fallot repair: a systematic review and meta-analysis

Mapping the Memorial Anxiety Scale for Prostate Cancer to the SF-6D