Skip to main content
Erschienen in: Quality of Life Research 1/2007

01.08.2007 | Original Paper

A comparison of three sets of criteria for determining the presence of differential item functioning using ordinal logistic regression

verfasst von: Paul K. Crane, Laura E. Gibbons, Katja Ocepek-Welikson, Karon Cook, David Cella, Kaavya Narasimhalu, Ron D. Hays, Jeanne A. Teresi

Erschienen in: Quality of Life Research | Sonderheft 1/2007

Einloggen

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Background

Several techniques have been developed to detect differential item functioning (DIF), including ordinal logistic regression (OLR). This study compared different criteria for determining whether items have DIF using OLR.

Objectives

To compare and contrast findings from three different sets of criteria for detecting DIF using OLR. General distress and physical functioning items were evaluated for DIF related to five covariates: age, marital status, gender, race, and Hispanic origin.

Research design

Cross-sectional study.

Subjects

1,714 patients with cancer or HIV/AIDS.

Measures

A total of 23 items addressing physical functioning and 15 items addressing general distress were selected from a pool of 154 items from four different health-related quality of life questionnaires.

Results

The three sets of criteria produced qualitatively and quantitatively different results. Criteria based on statistical significance alone detected DIF in almost all the items, while alternative criteria based on magnitude detected DIF in far fewer items. Accounting for DIF by using demographic-group specific item parameters had negligible effects on individual scores, except for race.

Conclusions

Specific criteria chosen to determine whether items have DIF have an impact on the findings. Criteria based entirely on statistical significance may detect small differences that are clinically negligible.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Hahn, E. A., Holzner, B., Kemmler, G., Sperner-Unterweger, B., Hudgens, S. A., & Cella, D. (2005). Cross-cultural evaluation of health status using item response theory: FACT-B comparisons between Austrian and U.S. patients with breast cancer. Evaluation & The Health Professions, 28, 233–259.CrossRef Hahn, E. A., Holzner, B., Kemmler, G., Sperner-Unterweger, B., Hudgens, S. A., & Cella, D. (2005). Cross-cultural evaluation of health status using item response theory: FACT-B comparisons between Austrian and U.S. patients with breast cancer. Evaluation & The Health Professions, 28, 233–259.CrossRef
2.
Zurück zum Zitat Eremenco, S. L., Cella, D., & Arnold, B. J. (2005). A comprehensive method for the translation and cross-cultural validation of health status questionnaires. Evaluation & The Health Professions, 28, 212–232.CrossRef Eremenco, S. L., Cella, D., & Arnold, B. J. (2005). A comprehensive method for the translation and cross-cultural validation of health status questionnaires. Evaluation & The Health Professions, 28, 212–232.CrossRef
3.
Zurück zum Zitat Martin, M., Blaisdell, B., Kwong, J. W., & Bjorner, J. B. (2004). The Short-Form Headache Impact Test (HIT-6) was psychometrically equivalent in nine languages. Journal of Clinical Epidemiology, 57, 1271–1278.PubMedCrossRef Martin, M., Blaisdell, B., Kwong, J. W., & Bjorner, J. B. (2004). The Short-Form Headache Impact Test (HIT-6) was psychometrically equivalent in nine languages. Journal of Clinical Epidemiology, 57, 1271–1278.PubMedCrossRef
4.
Zurück zum Zitat Roorda, L. D., Jones, C. A., Waltz, M., Lankhorst, G. J., Bouter, L. M., van der Eijken, J. W., Willems, W. J., Heyligers, I. C., Voaklander, D. C., Kelly, K. D., & Suarez-Almazor, M. E. (2004). Satisfactory cross cultural equivalence of the Dutch WOMAC in patients with hip osteoarthritis waiting for arthroplasty. Annals of the Rheumatic Diseases, 63, 36–42.PubMedCrossRef Roorda, L. D., Jones, C. A., Waltz, M., Lankhorst, G. J., Bouter, L. M., van der Eijken, J. W., Willems, W. J., Heyligers, I. C., Voaklander, D. C., Kelly, K. D., & Suarez-Almazor, M. E. (2004). Satisfactory cross cultural equivalence of the Dutch WOMAC in patients with hip osteoarthritis waiting for arthroplasty. Annals of the Rheumatic Diseases, 63, 36–42.PubMedCrossRef
5.
Zurück zum Zitat Ryall, N. H., Eyres, S. B., Neumann, V. C., Bhakta, B. B., & Tennant, A. (2003). Is the Rivermead Mobility Index appropriate to measure mobility in lower limb amputees? Disability and Rehabilitation, 25, 143–153.PubMedCrossRef Ryall, N. H., Eyres, S. B., Neumann, V. C., Bhakta, B. B., & Tennant, A. (2003). Is the Rivermead Mobility Index appropriate to measure mobility in lower limb amputees? Disability and Rehabilitation, 25, 143–153.PubMedCrossRef
6.
Zurück zum Zitat Angoff, W. H. (1993). Perspectives on differential item functioning methodology. In P. W. Holland & H. Wainer (Eds.), Differential item functioning. Hillsdale, NJ: Erlbaum. Angoff, W. H. (1993). Perspectives on differential item functioning methodology. In P. W. Holland & H. Wainer (Eds.), Differential item functioning. Hillsdale, NJ: Erlbaum.
7.
Zurück zum Zitat Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. Thousand Oaks: Sage. Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. Thousand Oaks: Sage.
8.
Zurück zum Zitat Millsap, R. E., & Everson, H. T. (1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17, 297–334.CrossRef Millsap, R. E., & Everson, H. T. (1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17, 297–334.CrossRef
9.
Zurück zum Zitat Holland, P. W., & Wainer, H. (Eds.) (1993). Differential item functioning. Hillsdale, NJ: Erlbaum. Holland, P. W., & Wainer, H. (Eds.) (1993). Differential item functioning. Hillsdale, NJ: Erlbaum.
10.
Zurück zum Zitat Crane, P. K., van Belle, G., & Larson, E. B. (2004). Test bias in a cognitive test: Differential item functioning in the CASI. Statistics in Medicine, 23, 241–256.PubMedCrossRef Crane, P. K., van Belle, G., & Larson, E. B. (2004). Test bias in a cognitive test: Differential item functioning in the CASI. Statistics in Medicine, 23, 241–256.PubMedCrossRef
11.
Zurück zum Zitat Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361–370.CrossRef Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361–370.CrossRef
12.
Zurück zum Zitat Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, ON: Directorate of Human Resources Research and Evaluation, Department of National Defense. Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, ON: Directorate of Human Resources Research and Evaluation, Department of National Defense.
13.
Zurück zum Zitat Gelin, M. N., & Zumbo, B. D. (2003). Differential item functioning results may change depending on how an item is scored: An illustration with Center for Epidemiologic Studies Depression scale. Educational & Psychological Measurement, 63, 65–74.CrossRef Gelin, M. N., & Zumbo, B. D. (2003). Differential item functioning results may change depending on how an item is scored: An illustration with Center for Epidemiologic Studies Depression scale. Educational & Psychological Measurement, 63, 65–74.CrossRef
14.
Zurück zum Zitat Crane, P. K., Gibbons, L. E., Jolley, L., & van Belle, G. (2006). Differential item functioning analysis with ordinal logistic regression techniques: DIFdetect and difwithpar. Medical Care, 44, S115–S123.PubMedCrossRef Crane, P. K., Gibbons, L. E., Jolley, L., & van Belle, G. (2006). Differential item functioning analysis with ordinal logistic regression techniques: DIFdetect and difwithpar. Medical Care, 44, S115–S123.PubMedCrossRef
15.
Zurück zum Zitat Ganz, P. A., Schag, C. A., Lee, J. J., & Sim, M. S. (1992). The CARES: A generic measure of health-related quality of life for patients with cancer. Quality of Life Research, 1, 19–29.PubMedCrossRef Ganz, P. A., Schag, C. A., Lee, J. J., & Sim, M. S. (1992). The CARES: A generic measure of health-related quality of life for patients with cancer. Quality of Life Research, 1, 19–29.PubMedCrossRef
16.
Zurück zum Zitat Schag, C. A., Ganz, P. A., & Heinrich, R. L. (1991). Cancer Rehabilitation Evaluation System-short form (CARES-SF). A cancer specific rehabilitation and quality of life instrument. Cancer, 68, 1406–1413.PubMedCrossRef Schag, C. A., Ganz, P. A., & Heinrich, R. L. (1991). Cancer Rehabilitation Evaluation System-short form (CARES-SF). A cancer specific rehabilitation and quality of life instrument. Cancer, 68, 1406–1413.PubMedCrossRef
17.
Zurück zum Zitat Aaronson, N. K., Ahmedzai, S., Bergman, B., Bullinger, M., Cull, A., Duez, N. J., Filiberti, A., Flechtner, H., Fleishman, S. B., & de Haes, J. C., et al. (1993). The European Organization for Research and Treatment of Cancer QLQ-C30: A quality-of-life instrument for use in international clinical trials in oncology. Journal of the National Cancer Institute, 85, 365–376.PubMedCrossRef Aaronson, N. K., Ahmedzai, S., Bergman, B., Bullinger, M., Cull, A., Duez, N. J., Filiberti, A., Flechtner, H., Fleishman, S. B., & de Haes, J. C., et al. (1993). The European Organization for Research and Treatment of Cancer QLQ-C30: A quality-of-life instrument for use in international clinical trials in oncology. Journal of the National Cancer Institute, 85, 365–376.PubMedCrossRef
18.
Zurück zum Zitat Cella, D. F., Tulsky, D. S., Gray, G., Sarafian, B., Linn, E., Bonomi, A., Silberman, M., Yellen, S. B., Winicour, P., & Brannon, J., et al. (1993). The Functional Assessment of Cancer Therapy scale: Development and validation of the general measure. Journal of Clinical Oncology, 11, 570–579.PubMed Cella, D. F., Tulsky, D. S., Gray, G., Sarafian, B., Linn, E., Bonomi, A., Silberman, M., Yellen, S. B., Winicour, P., & Brannon, J., et al. (1993). The Functional Assessment of Cancer Therapy scale: Development and validation of the general measure. Journal of Clinical Oncology, 11, 570–579.PubMed
19.
Zurück zum Zitat Cella, D. F., & Bonomi, A. E. (1995). Measuring quality of life: 1995 update. Oncology (Williston Park), 9, 47–60. Cella, D. F., & Bonomi, A. E. (1995). Measuring quality of life: 1995 update. Oncology (Williston Park), 9, 47–60.
20.
Zurück zum Zitat Hays, R. D., Sherbourne, C. D., & Mazel, R. M. (1993). The RAND 36-Item Health Survey 1.0. Health Economics, 2, 217–227.PubMedCrossRef Hays, R. D., Sherbourne, C. D., & Mazel, R. M. (1993). The RAND 36-Item Health Survey 1.0. Health Economics, 2, 217–227.PubMedCrossRef
21.
Zurück zum Zitat McHorney, C. A., Ware, J. E. Jr., & Raczek, A. E. (1993). The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Medical Care, 31, 247–263.PubMedCrossRef McHorney, C. A., Ware, J. E. Jr., & Raczek, A. E. (1993). The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Medical Care, 31, 247–263.PubMedCrossRef
22.
Zurück zum Zitat Ware, J. E. Jr., & Sherbourne, C. D. (1992). The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Medical Care, 30, 473–483.PubMedCrossRef Ware, J. E. Jr., & Sherbourne, C. D. (1992). The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Medical Care, 30, 473–483.PubMedCrossRef
23.
Zurück zum Zitat Hu, L.-T., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424–453.CrossRef Hu, L.-T., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424–453.CrossRef
24.
Zurück zum Zitat Hu, L.-T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.CrossRef Hu, L.-T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.CrossRef
25.
Zurück zum Zitat Muraki, E., & Bock, D. (2003). PARSCALE for Windows. Chicago: SSI. Version 4.1. Muraki, E., & Bock, D. (2003). PARSCALE for Windows. Chicago: SSI. Version 4.1.
26.
Zurück zum Zitat Samejima, F. (1997). Graded response model. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory. NY: Springer. Samejima, F. (1997). Graded response model. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory. NY: Springer.
27.
Zurück zum Zitat Samejima F. (1969) Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph No. 17. Samejima F. (1969) Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph No. 17.
28.
Zurück zum Zitat StataCorp. (2003). Stata statistical software: release 8.0. College Station, TX: StataCorp. StataCorp. (2003). Stata statistical software: release 8.0. College Station, TX: StataCorp.
29.
Zurück zum Zitat McCullagh P., & Nelder, J.A. (1989). Generalized linear models. London: Chapman and Hall. McCullagh P., & Nelder, J.A. (1989). Generalized linear models. London: Chapman and Hall.
30.
Zurück zum Zitat Maldonado, G., & Greenland, S. (1993). Simulation study of confounder-selection strategies. American Journal of Epidemiology, 138, 923–936.PubMed Maldonado, G., & Greenland, S. (1993). Simulation study of confounder-selection strategies. American Journal of Epidemiology, 138, 923–936.PubMed
31.
Zurück zum Zitat Crane, P. K., Hart, D. L., Gibbons, L. E., & Cook, K. F. (2006). A 37-item shoulder functional status item pool had negligible differential item functioning. Journal of Clinical Epidemiology, 59, 478–484.PubMedCrossRef Crane, P. K., Hart, D. L., Gibbons, L. E., & Cook, K. F. (2006). A 37-item shoulder functional status item pool had negligible differential item functioning. Journal of Clinical Epidemiology, 59, 478–484.PubMedCrossRef
32.
Zurück zum Zitat Cella, D., Hahn, E. A., & Dineen, K. (2002). Meaningful change in cancer-specific quality of life scores: Differences between improvement and worsening. Quality of Life Research, 11, 207–221.PubMedCrossRef Cella, D., Hahn, E. A., & Dineen, K. (2002). Meaningful change in cancer-specific quality of life scores: Differences between improvement and worsening. Quality of Life Research, 11, 207–221.PubMedCrossRef
33.
Zurück zum Zitat Eton, D. T., Cella, D., Yost, K. J., Yount, S. E., Peterman, A. H., Neuberg, D. S., Sledge, G. W., & Wood, W. C. (2004). A combination of distribution- and anchor-based approaches determined minimally important differences (MIDs) for four endpoints in a breast cancer scale. Journal of Clinical Epidemiology, 57, 898–910.PubMedCrossRef Eton, D. T., Cella, D., Yost, K. J., Yount, S. E., Peterman, A. H., Neuberg, D. S., Sledge, G. W., & Wood, W. C. (2004). A combination of distribution- and anchor-based approaches determined minimally important differences (MIDs) for four endpoints in a breast cancer scale. Journal of Clinical Epidemiology, 57, 898–910.PubMedCrossRef
34.
Zurück zum Zitat Crane, P. K., Gibbons, L. E., Narasimhalu, K., Lai, J. S., & Cella D. (2007). Rapid detection of differential item functioning in assessments of health-related quality of life: The Functional Assessment of Cancer Therapy. Quality of Life Research, 16, 101–114.PubMedCrossRef Crane, P. K., Gibbons, L. E., Narasimhalu, K., Lai, J. S., & Cella D. (2007). Rapid detection of differential item functioning in assessments of health-related quality of life: The Functional Assessment of Cancer Therapy. Quality of Life Research, 16, 101–114.PubMedCrossRef
35.
Zurück zum Zitat Long, J. S. (1997). Regression models for categorical and limited dependent variables. Advanced quantitative techniques in the social sciences. Thousand Oaks: Sage. Long, J. S. (1997). Regression models for categorical and limited dependent variables. Advanced quantitative techniques in the social sciences. Thousand Oaks: Sage.
36.
Zurück zum Zitat Shealy, R. T., & Stout, W. F. (1993). An item response theory model for test bias and differential test functioning. In P. W. Holland & H. Wainer (Eds.), Differential item functioning. Hillsdale, NJ: Erlbaum. Shealy, R. T., & Stout, W. F. (1993). An item response theory model for test bias and differential test functioning. In P. W. Holland & H. Wainer (Eds.), Differential item functioning. Hillsdale, NJ: Erlbaum.
Metadaten
Titel
A comparison of three sets of criteria for determining the presence of differential item functioning using ordinal logistic regression
verfasst von
Paul K. Crane
Laura E. Gibbons
Katja Ocepek-Welikson
Karon Cook
David Cella
Kaavya Narasimhalu
Ron D. Hays
Jeanne A. Teresi
Publikationsdatum
01.08.2007
Verlag
Springer Netherlands
Erschienen in
Quality of Life Research / Ausgabe Sonderheft 1/2007
Print ISSN: 0962-9343
Elektronische ISSN: 1573-2649
DOI
https://doi.org/10.1007/s11136-007-9185-5

Weitere Artikel der Sonderheft 1/2007

Quality of Life Research 1/2007 Zur Ausgabe