Skip to main content
Erschienen in: Quality & Quantity 4/2017

13.05.2016

Item response theory requires logically unjustifiable assumptions

verfasst von: Merton S. Krause

Erschienen in: Quality & Quantity | Ausgabe 4/2017

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

If items have different levels of difficulty (or sensitivity) relative to some psychological attribute, passing (or endorsing) any one cannot mean the same about a person as passing any other, so percent of items passed regardless of which these are cannot indicate a person’s level on any attribute. If persons have different levels on a psychological attribute, an item’s being passed by one person cannot mean the same about its difficulty level as being passed by any other person, so percent of persons passing it regardless of which persons these are cannot indicate the item’s difficulty level. Percent of items passed by a person and percent of persons passing an item are incommensurate quantities not expressible in terms of the same quality or dimension. Both such percents are dependent on what sample of items and of persons are used. A person’s attribute level is not demonstrably probabilistic, because truly independent replicate occasions of a person responding to an item are impossible. Passing an item depends on more than a person’s single attribute level, the item’s difficulty level, and random chance. On all these matters Item Response Theory relies on assumptions that are logically unjustifiable.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Batchelder, W.H., Narens, L.: A critical examination of the analysis of dichotomous data. Philos. Sci. 44, 113–135 (1977)CrossRef Batchelder, W.H., Narens, L.: A critical examination of the analysis of dichotomous data. Philos. Sci. 44, 113–135 (1977)CrossRef
Zurück zum Zitat Bejar, I.I.: Recent developments and prospects in item generation. In: Embretson, S.E. (ed.) Measuring Psychological Constructs: Advances in Model-Based Approaches, pp. 201–226. American Psychological Association, Washington (2010)CrossRef Bejar, I.I.: Recent developments and prospects in item generation. In: Embretson, S.E. (ed.) Measuring Psychological Constructs: Advances in Model-Based Approaches, pp. 201–226. American Psychological Association, Washington (2010)CrossRef
Zurück zum Zitat Berg, I.A. (ed.): Response set in Personality Assessment. Aldine, Chicago (1967) Berg, I.A. (ed.): Response set in Personality Assessment. Aldine, Chicago (1967)
Zurück zum Zitat Borg, G: To honor Stevens and improve his scaling methods. Proceedings of the twenty second annual meeting of the International Society for Psychophysics, St. Albans, England, vol. 22, pp. 31–36 (2006). (www.ispsychophysics.org. Accessed 29 April 16) Borg, G: To honor Stevens and improve his scaling methods. Proceedings of the twenty second annual meeting of the International Society for Psychophysics, St. Albans, England, vol. 22, pp. 31–36 (2006). (www.​ispsychophysics.​org. Accessed 29 April 16)
Zurück zum Zitat Borg, G., Borg, E.: A new generation of scaling methods: level-anchored ratio scaling. Psychologica 28, 15–45 (2001) Borg, G., Borg, E.: A new generation of scaling methods: level-anchored ratio scaling. Psychologica 28, 15–45 (2001)
Zurück zum Zitat Borg, I.: Some basic concepts of facet theory. In: Lingoes, J.C. (ed.) Geometric Representations of Relational Data: Readings in Multidimensional Scaling, pp. 65–102. Mathesis Press, Ann Arbor (1977) Borg, I.: Some basic concepts of facet theory. In: Lingoes, J.C. (ed.) Geometric Representations of Relational Data: Readings in Multidimensional Scaling, pp. 65–102. Mathesis Press, Ann Arbor (1977)
Zurück zum Zitat Bornstein, R.F.: Toward a process-focused model of test score validity: improving psychological assessment in science and practice. Psychol. Assess. 23, 532–544 (2011)CrossRef Bornstein, R.F.: Toward a process-focused model of test score validity: improving psychological assessment in science and practice. Psychol. Assess. 23, 532–544 (2011)CrossRef
Zurück zum Zitat Cohen, A.D.: The coming of age of research on test-taking strategies. Lang. Assess. Q. 3, 307–331 (2006)CrossRef Cohen, A.D.: The coming of age of research on test-taking strategies. Lang. Assess. Q. 3, 307–331 (2006)CrossRef
Zurück zum Zitat Danziger, K.: Constructing the Subject: Historical Origins of Psychological Research. Cambridge University Press, New York (1990)CrossRef Danziger, K.: Constructing the Subject: Historical Origins of Psychological Research. Cambridge University Press, New York (1990)CrossRef
Zurück zum Zitat De Ayala, R.J.: The Theory and Practice of Item Response Theory. Guilford, New York (2009) De Ayala, R.J.: The Theory and Practice of Item Response Theory. Guilford, New York (2009)
Zurück zum Zitat Dilchert, S., Ones, D.S., Viswesvaran, C., Deller, J.: Response distortion in personality measurement: born to deceive, yet capable of providing valid self-assessments? Psychol. Sci. 48, 209–225 (2006) Dilchert, S., Ones, D.S., Viswesvaran, C., Deller, J.: Response distortion in personality measurement: born to deceive, yet capable of providing valid self-assessments? Psychol. Sci. 48, 209–225 (2006)
Zurück zum Zitat Embretson, S.E. (ed.): Measuring Psychological Constructs: Advances in Model-Based Approaches. APA, Washington (2010) Embretson, S.E. (ed.): Measuring Psychological Constructs: Advances in Model-Based Approaches. APA, Washington (2010)
Zurück zum Zitat Embretson, S.E., Reise, S.P.: Item Response Theory for Psychologists. Erlbaum, Mahwah (2000) Embretson, S.E., Reise, S.P.: Item Response Theory for Psychologists. Erlbaum, Mahwah (2000)
Zurück zum Zitat Emons, W.H.M., Sijtsma, K., Meijer, R.R.: Global, local and graphical person-fit analysis using person response functions. Psychol. Methods 10, 101–119 (2005)CrossRef Emons, W.H.M., Sijtsma, K., Meijer, R.R.: Global, local and graphical person-fit analysis using person response functions. Psychol. Methods 10, 101–119 (2005)CrossRef
Zurück zum Zitat Feller, W.: An Introduction to Probability Theory and its Applications, vol. I, 3rd edn. Wiley, New York (1970) Feller, W.: An Introduction to Probability Theory and its Applications, vol. I, 3rd edn. Wiley, New York (1970)
Zurück zum Zitat Fiske, D.W.: Measuring the Concepts of Personality. Aldine, Chicago (1971) Fiske, D.W.: Measuring the Concepts of Personality. Aldine, Chicago (1971)
Zurück zum Zitat Goldstein, H., Wood, R.: Five decades of item-response modelling. Br. J. Math. Stat. Psychol. 42, 139–167 (1989)CrossRef Goldstein, H., Wood, R.: Five decades of item-response modelling. Br. J. Math. Stat. Psychol. 42, 139–167 (1989)CrossRef
Zurück zum Zitat Hoffmann, B.: The Tyranny of Testing. Collier, New York (1964) Hoffmann, B.: The Tyranny of Testing. Collier, New York (1964)
Zurück zum Zitat Holland, P.W.: On the sampling theory foundations of item response theory models. Psychometrika 55, 577–601 (1990)CrossRef Holland, P.W.: On the sampling theory foundations of item response theory models. Psychometrika 55, 577–601 (1990)CrossRef
Zurück zum Zitat Hulin, C.L., Drasgow, F., Parsons, C.K.: Item Response Theory: Application to Psychological Measurement. Dow Jones-Irwin, Homewood (1983) Hulin, C.L., Drasgow, F., Parsons, C.K.: Item Response Theory: Application to Psychological Measurement. Dow Jones-Irwin, Homewood (1983)
Zurück zum Zitat Irvine, S.H., Kyllonen, P.C. (eds.): Item Generation for Test Development. Routledge, New York (2010) Irvine, S.H., Kyllonen, P.C. (eds.): Item Generation for Test Development. Routledge, New York (2010)
Zurück zum Zitat Krause, M.S.: Measurement validity is fundamentally a matter of definition, not correlation. Rev. Gen. Psychol. 16, 391–400 (2012)CrossRef Krause, M.S.: Measurement validity is fundamentally a matter of definition, not correlation. Rev. Gen. Psychol. 16, 391–400 (2012)CrossRef
Zurück zum Zitat Krause, M.S.: The data analytic implications of human psychology’s dimensions being ordinally scaled. Rev. Gen. Psychol. 17, 318–325 (2013)CrossRef Krause, M.S.: The data analytic implications of human psychology’s dimensions being ordinally scaled. Rev. Gen. Psychol. 17, 318–325 (2013)CrossRef
Zurück zum Zitat Krause, M.S., Lutz, W., Bőhnke, J.R.: The role of sampling in clinical trial design. Psychother. Res. 21, 243–251 (2011)CrossRef Krause, M.S., Lutz, W., Bőhnke, J.R.: The role of sampling in clinical trial design. Psychother. Res. 21, 243–251 (2011)CrossRef
Zurück zum Zitat McCarthy, J., Hrabluik, C., Jelley, B.: Progression through the ranks: assessing employee reactions to high-stakes employment testing. Pers. Psychol. 62, 793–832 (2009)CrossRef McCarthy, J., Hrabluik, C., Jelley, B.: Progression through the ranks: assessing employee reactions to high-stakes employment testing. Pers. Psychol. 62, 793–832 (2009)CrossRef
Zurück zum Zitat Meijer, R.R.: The number of Guttman errors as a simple and powerful person-fit statistic. Appl. Psychol. Meas. 18(4), 311–314 (1994)CrossRef Meijer, R.R.: The number of Guttman errors as a simple and powerful person-fit statistic. Appl. Psychol. Meas. 18(4), 311–314 (1994)CrossRef
Zurück zum Zitat Meijer, R.R., Sijtsma, K.: Detection of aberrant item score patterns: a review of recent developments. Appl. Measur. Educ. 8, 261–272 (1995)CrossRef Meijer, R.R., Sijtsma, K.: Detection of aberrant item score patterns: a review of recent developments. Appl. Measur. Educ. 8, 261–272 (1995)CrossRef
Zurück zum Zitat Michell, J.: Item response models, pathological science, and the shape of error: reply to Borsboom and Mellenbergh. Theory Psychol. 14, 121–129 (2004)CrossRef Michell, J.: Item response models, pathological science, and the shape of error: reply to Borsboom and Mellenbergh. Theory Psychol. 14, 121–129 (2004)CrossRef
Zurück zum Zitat Mokken, R.J. (ed.): A Theory and Procedure of scale Analysis. Mouton, The Hague (1971) Mokken, R.J. (ed.): A Theory and Procedure of scale Analysis. Mouton, The Hague (1971)
Zurück zum Zitat Mokken, R.J.: Nonparametric models for dichotomous responses. In: van der Linden, W.J., Hambleton, R.K. (eds.) Handbook of Modern Item Response Theory, pp. 351–368. Springer, New York (1997)CrossRef Mokken, R.J.: Nonparametric models for dichotomous responses. In: van der Linden, W.J., Hambleton, R.K. (eds.) Handbook of Modern Item Response Theory, pp. 351–368. Springer, New York (1997)CrossRef
Zurück zum Zitat Mokken, R.J., Lewis, C.: A nonparametric approach to the analysis of dichotomous item responses. Appl. Psychol. Meas. 6, 417–430 (1982)CrossRef Mokken, R.J., Lewis, C.: A nonparametric approach to the analysis of dichotomous item responses. Appl. Psychol. Meas. 6, 417–430 (1982)CrossRef
Zurück zum Zitat Murphy, K.R., Cleveland, J.: Understanding Performance Appraisal: Social, Organizational, and Goal-Based Perspectives. Sage, Thousand Oaks (1995) Murphy, K.R., Cleveland, J.: Understanding Performance Appraisal: Social, Organizational, and Goal-Based Perspectives. Sage, Thousand Oaks (1995)
Zurück zum Zitat Paulhus, D.L.: Socially desirable responding: the evolution of a construct. In: Braun, H.I., Jackson, D.N., Wiley, D.E. (eds.) The Role of Constructs in Psychological and Educational Measurement, pp. 51–73. Routledge, New York (2001) Paulhus, D.L.: Socially desirable responding: the evolution of a construct. In: Braun, H.I., Jackson, D.N., Wiley, D.E. (eds.) The Role of Constructs in Psychological and Educational Measurement, pp. 51–73. Routledge, New York (2001)
Zurück zum Zitat Rasch, G.: On general laws and the meaning of measurement in psychology. In: Neyman, J. (ed.) Fourth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 321–333. University of California Press, Berkeley (1961) Rasch, G.: On general laws and the meaning of measurement in psychology. In: Neyman, J. (ed.) Fourth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 321–333. University of California Press, Berkeley (1961)
Zurück zum Zitat Reckase, M.D.: Multidimensional Item Response Theory. Springer, New York (2009)CrossRef Reckase, M.D.: Multidimensional Item Response Theory. Springer, New York (2009)CrossRef
Zurück zum Zitat Reise, S.P., Bonifay, W.E., Haviland, M.G.: Scoring and modeling psychological measures in the presence of multidimensionality. J. Pers. Assess. 95, 129–140 (2013)CrossRef Reise, S.P., Bonifay, W.E., Haviland, M.G.: Scoring and modeling psychological measures in the presence of multidimensionality. J. Pers. Assess. 95, 129–140 (2013)CrossRef
Zurück zum Zitat Reise, S.P., Cook, K.F., Moore, T.M.: Evaluating the impact of multidimensionality on unidimensional item response theory model parameters. In: Reise, R. (ed.) Measuring Psychological Constructs: Advances in Model-Based Approaches, pp. 13–40. American Psychological Association, Washington (2014) Reise, S.P., Cook, K.F., Moore, T.M.: Evaluating the impact of multidimensionality on unidimensional item response theory model parameters. In: Reise, R. (ed.) Measuring Psychological Constructs: Advances in Model-Based Approaches, pp. 13–40. American Psychological Association, Washington (2014)
Zurück zum Zitat Reise, S.P., Moore, T.M., Maydeu-Olivares, A.: Target rotations and assessing the impact of model violations on the parameters of unidimensional item response theory models. Educ. Psychol. Measur. 71, 684–711 (2011)CrossRef Reise, S.P., Moore, T.M., Maydeu-Olivares, A.: Target rotations and assessing the impact of model violations on the parameters of unidimensional item response theory models. Educ. Psychol. Measur. 71, 684–711 (2011)CrossRef
Zurück zum Zitat Reise, S.P., Revicki, D.A. (eds.): Handbook of Item Response Theory Modeling: Applications to Typical Performance Assessment. Routledge, New York (2014) Reise, S.P., Revicki, D.A. (eds.): Handbook of Item Response Theory Modeling: Applications to Typical Performance Assessment. Routledge, New York (2014)
Zurück zum Zitat Rogers, T.B.: The process of responding to personality items: some issues, a theory and some research. Multivar. Behav. Res. Monogr. 6(2), 1–66 (1971) Rogers, T.B.: The process of responding to personality items: some issues, a theory and some research. Multivar. Behav. Res. Monogr. 6(2), 1–66 (1971)
Zurück zum Zitat Roussos, L.A., Stout, W.: Differential item function analysis: detecting DIF items and testing DIF hypotheses. In: Kaplan, D. (ed.) The SAGE Handbook of Quantitative Methodology for the Social Sciences, pp. 107–115. SAGE, Thousand Oaks (2004) Roussos, L.A., Stout, W.: Differential item function analysis: detecting DIF items and testing DIF hypotheses. In: Kaplan, D. (ed.) The SAGE Handbook of Quantitative Methodology for the Social Sciences, pp. 107–115. SAGE, Thousand Oaks (2004)
Zurück zum Zitat Scruggs, T.E., Mastropieri, M.A.: Teaching Test-Taking Skills: Helping Students Show What They Know. Brookline Books, Cambridge (1995) Scruggs, T.E., Mastropieri, M.A.: Teaching Test-Taking Skills: Helping Students Show What They Know. Brookline Books, Cambridge (1995)
Zurück zum Zitat Shye, S.: Partial order scalogram analysis. In: Shye, S. (ed.) Theory Construction and data analysis in the behavioral sciences, pp. 265–278. Jossey-Bass, San Francisco (1978) Shye, S.: Partial order scalogram analysis. In: Shye, S. (ed.) Theory Construction and data analysis in the behavioral sciences, pp. 265–278. Jossey-Bass, San Francisco (1978)
Zurück zum Zitat Sijtsma, K.: Methodology review: nonparametric IRT approaches to the analysis of dichotomous item scores. Appl. Psychol. Meas. 22, 3–31 (1998)CrossRef Sijtsma, K.: Methodology review: nonparametric IRT approaches to the analysis of dichotomous item scores. Appl. Psychol. Meas. 22, 3–31 (1998)CrossRef
Zurück zum Zitat Sijtsma, K., Junker, B.W.: Item response theory: past performance, present developments, and future expectations. Behaviormetrika 33, 75–102 (2006)CrossRef Sijtsma, K., Junker, B.W.: Item response theory: past performance, present developments, and future expectations. Behaviormetrika 33, 75–102 (2006)CrossRef
Zurück zum Zitat Sijtsma, K., Meijer, R.R., Van der Ark, L.A.: Mokken scale analysis as time goes by: an update for scaling practitioners. Pers. Individ. Differ. 50, 31–37 (2011)CrossRef Sijtsma, K., Meijer, R.R., Van der Ark, L.A.: Mokken scale analysis as time goes by: an update for scaling practitioners. Pers. Individ. Differ. 50, 31–37 (2011)CrossRef
Zurück zum Zitat Stark, S., Chernyshenko, O.S., Chan, K.-Y., Lee, W.C., Drasgow, F.: Effects of the testing situation on item responding: cause for concern. J. Appl. Psychol. 86, 943–953 (2001)CrossRef Stark, S., Chernyshenko, O.S., Chan, K.-Y., Lee, W.C., Drasgow, F.: Effects of the testing situation on item responding: cause for concern. J. Appl. Psychol. 86, 943–953 (2001)CrossRef
Zurück zum Zitat Stark, S., Chernyshenko, O.S., Drasgow, F., Williams, B.A.: Examining assumptions about item responding in personality assessment: should ideal point methods be considered for scale development and scoring? J. Appl. Psychol. 91, 25–39 (2006)CrossRef Stark, S., Chernyshenko, O.S., Drasgow, F., Williams, B.A.: Examining assumptions about item responding in personality assessment: should ideal point methods be considered for scale development and scoring? J. Appl. Psychol. 91, 25–39 (2006)CrossRef
Zurück zum Zitat Thissen, D., Steinberg, L.: Using item response theory to disentangle constructs at different levels of generality. In: Embretson, S.E. (ed.) Measuring Psychological Constructs: Advances in Model-Based Approaches, pp. 123–144. American Psychological Association, Washington (2010)CrossRef Thissen, D., Steinberg, L.: Using item response theory to disentangle constructs at different levels of generality. In: Embretson, S.E. (ed.) Measuring Psychological Constructs: Advances in Model-Based Approaches, pp. 123–144. American Psychological Association, Washington (2010)CrossRef
Zurück zum Zitat Tourangeau, R., Rasinski, K.A.: Cognitive processes underlying context effects in attitude measurement. Psychol. Bull. 103, 299–314 (1988)CrossRef Tourangeau, R., Rasinski, K.A.: Cognitive processes underlying context effects in attitude measurement. Psychol. Bull. 103, 299–314 (1988)CrossRef
Zurück zum Zitat Tuerlinckx, F., De Boeck, P.: The effect of ignoring item interactions on the estimated discrimination parameters in Item Response Theory. Psychol. Methods 6, 181–195 (2001)CrossRef Tuerlinckx, F., De Boeck, P.: The effect of ignoring item interactions on the estimated discrimination parameters in Item Response Theory. Psychol. Methods 6, 181–195 (2001)CrossRef
Zurück zum Zitat Van Schuur, W.H.: Mokken scale analysis: between the Guttman scale and parametric item response theory. Polit. Anal. 11, 139–163 (2003)CrossRef Van Schuur, W.H.: Mokken scale analysis: between the Guttman scale and parametric item response theory. Polit. Anal. 11, 139–163 (2003)CrossRef
Zurück zum Zitat Woods, C.: Estimating the latent density in unidimensional IRT to permit non-normality. In: Reise, R. (ed.) Measuring Psychological Constructs: Advances in Model-Based Approaches, pp. 60–84. American Psychological Association, Washington (2014) Woods, C.: Estimating the latent density in unidimensional IRT to permit non-normality. In: Reise, R. (ed.) Measuring Psychological Constructs: Advances in Model-Based Approaches, pp. 60–84. American Psychological Association, Washington (2014)
Zurück zum Zitat Wright, B.D., Mok, M.M.: An overview of the family of Rasch measurement models. In: Smith, E.V., Smith, R.M. (eds.) Introduction to Rasch Measurement, pp. 1–24. JAM Press, Chicago (2004) Wright, B.D., Mok, M.M.: An overview of the family of Rasch measurement models. In: Smith, E.V., Smith, R.M. (eds.) Introduction to Rasch Measurement, pp. 1–24. JAM Press, Chicago (2004)
Zurück zum Zitat Wright, B.D., Stone, M.: Measurement essentials (2nd ed). Wilmington, Delaware: Wide Range Inc (1999). Also www.rasch.org/measess/me-all.pdf. Accessed 29 April 16: be sure to left-click “me-all.pdf” in the upper left corner) Wright, B.D., Stone, M.: Measurement essentials (2nd ed). Wilmington, Delaware: Wide Range Inc (1999). Also www.​rasch.​org/​measess/​me-all.​pdf. Accessed 29 April 16: be sure to left-click “me-all.pdf” in the upper left corner)
Zurück zum Zitat Zumbo, B.D.: Three generations of DIF analyses: considering where it has been, where it is now, and where it is going. Lang. Assess. Q. 4, 223–233 (2007)CrossRef Zumbo, B.D.: Three generations of DIF analyses: considering where it has been, where it is now, and where it is going. Lang. Assess. Q. 4, 223–233 (2007)CrossRef
Metadaten
Titel
Item response theory requires logically unjustifiable assumptions
verfasst von
Merton S. Krause
Publikationsdatum
13.05.2016
Verlag
Springer Netherlands
Erschienen in
Quality & Quantity / Ausgabe 4/2017
Print ISSN: 0033-5177
Elektronische ISSN: 1573-7845
DOI
https://doi.org/10.1007/s11135-016-0351-0

Weitere Artikel der Ausgabe 4/2017

Quality & Quantity 4/2017 Zur Ausgabe