nach oben

Journal of the Academy of Marketing Science

Erschienen in:

01.07.2014 | Methodological Paper

The utility of item response modeling in marketing research

verfasst von: Tenko Raykov, Roger J. Calantone

Erschienen in: Journal of the Academy of Marketing Science | Ausgabe 4/2014

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Item response modeling (IRM/IRT) has been known to marketing scholars for a number of years. However, with the exception of some notable and important applications in international (cross-cultural) marketing and consumer behavior, even a cursory reading of marketing journals reveals a general lack of interest in applying IRM, despite its ability to provide highly useful measurement-related information. To address and hopefully remedy the paucity of adoption, we offer an application-oriented discussion of the utility of IRM for marketing and related business research to enable researchers to utilize the strengths and realize the benefits of this methodology in their empirical work. After a short discussion of the history of IRM, we focus on its fundamentals within a modern statistical framework based on the generalized linear model and closely related non-linear factor analysis. We then engage major concepts of IRM, including item characteristic curve, local independence, and dimensionality, as well as parameter estimation and information functions. The popular one- and two-parameter logistic models are next discussed, as is the issue of model selection. Several polytomous item response models are subsequently dealt with, followed by a discussion of multidimensional IRM and data illustrations of item response models using widely available software. References to exemplar marketing applications are provided along the way, and a discussion of limitations of IRM concludes the article.

Nächster Artikel When do incentives work in channels of distribution?

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nur mit Berechtigung zugänglich

The threshold parameter is typically called “difficulty” parameter in achievement evaluation and educational research settings, a reference we will not use in this paper dealing with marketing and business research, where it is not generally applicable.

A different derivation of the model definition Eq. 12 was provided by Rasch (1960), which was developed within an ability evaluation context and based on the comparison of an examined person’s ability level to the “difficulty” of the item in question (see Footnote 1). An alternative and direct derivation of the 1PL-model, without reference to any subject-matter context can be found in Raykov (2014), which only uses logistic regression considerations.

This paper is based on the premise that removal of items from a multi-component measuring instrument in order to achieve constraint Eq. 13 may be carried out if this is not associated with loss in validity, since the Rasch model has clearly desirable properties. However, “specific objectivity” cannot be a replacement for validity, and hence it need not be pursued in cases when construct under-representation is possible if dropping particular items from a measuring instrument in an attempt to satisfy the Rasch model and specifically its unique feature of uniform item discrimination power, as reflected in Eq. 13.

A sufficient statistic is a function of the available data with the property that the data likelihood (probability of the data) can be calculated as soon as one knows the value of that statistic. In other words, a statistic is sufficient with respect to a particular parameter if the former extracts (holds) all the information about the parameter that is contained in the available dataset. ML estimators can be shown to be functions of sufficient statistics, hence the particular relevance of such statistics in ML-based estimation (for a formal definition of sufficiency see, e.g., Casella and Berger 2002).

An informal explanation of the idea of the EM algorithm can be provided by considering the trait parameters (i.e., the individual θ’s) as missing values—one per respondent in the current setting. If these values were observed, one could use directly ML to estimate the item parameters. Since the missing values are not observed, however, one could obtain first the expected likelihood with respect to an assumed distribution for the missing values (such as the normal), and then maximize that likelihood to obtain provisional item parameter estimates. One can then use the latter to obtain predictions of the missing values, and treating these predictions as known (i.e., as if they were “observed values”) furnish improved estimates of the item parameters, with them obtain improved predictions of the missing values, then of the item parameters, and so on until convergence (e.g., Thissen 1982). It can be shown that at each subsequent step, the likelihood increases. The EM algorithm can be slow in reaching convergence in some cases.

The general expressions for the elements of the matrix in Eq. 17, also for the 2PL-model, as functions of ability levels, item parameters, and model-implied probabilities of correct response are given for instance in Baker and Kim (2004).

For numerical indexes of essential unidimensionality (i.e., the degree of dominance of a major latent dimension over secondary ones) and how to point and interval estimate them in an empirical setting, see, e.g., Raykov and Pohl (2013a, 2013b). Those authors also offer some informal guidelines for interpretation of the indexes, and suggestions when a complex structure instrument may be considered essentially unidimensional for some research questions in social science research (see also Stout 1990).

Using the weighted least squares method of model fitting, with a mean and variance correction (Muthén and Muthén 2012), one obtains also the root mean square error of approximation (RMSEA) that is a popular goodness-of-fit index, which is in general less sensitive to sample size than the chi-square test statistic reported in this paragraph. The RMSEA index for the fitted model here is .0, with a 90%-confidence interval of (0, .043), indicating similarly a tenable model and thus plausibility of the unidimensionality hypothesis tested with it (e.g., Raykov and Marcoulides 2006).

This p-value associated with the LRT used, can be routinely calculated for instance using the following R command (e.g., Raykov and Marcoulides 2012):> 1-pchisq (21.597-21.136, 5–1) or more generally > 1-pchisq (“LRT reduced model” – “LRT full model”, “df full” – “df reduced”) where “>” stands for the R prompt and within inverted commas one needs to substitute the corresponding numerical values for the LRT statistics associated with the Rasch and 2PL-models, respectively, as well as their degrees of freedom - all obtained from the Mplus output files as discussed in the main text (see also Appendix, incl. Notes 1 and 2 to first command file).

The finally entertained 1PL-model in this example, being a NLFA model, is equivalent to a classical test theory-based model with equal loadings on the underlying common true score (e.g., Raykov and Marcoulides 2011). Thus, the last column of Table 3 shows that when classical test theory is correctly used, and in particular models based on it are fitted to data by accounting properly for the nature/scale of the items (viz. ordinal, and binary in this example), then the standard error associated with the individual trait parameter (i.e., the individual true score) estimates does depend on the true score/trait level, contrary to claims found particularly in the older IRT literature. (See Raykov and Marcoulides 2011 for other myths about classical test theory, as well as Raykov 2014.)

The IIFs and TIF are easily obtained also with the readily available software IRTPRO (Cai et al. 2011). To this end, upon loading the data select “Analysis” from the main toolbar, “Unidimensional IRT” from the drop-down menu, and “Items” in the middle of the opened window then. Upon highlighting and moving all analyzed items into the right blank window, pressing “Run” will fit the (default) 2PL-model. Chose then “Graph” under “Analysis” in the toolbar for the resulting output window, to obtain correspondingly the IIFs and TIF (check those in the small upper-left corner window opened then). For further details on using IRTPRO, refer to Cai et al. (2011).

Upon loading the data in IRTPRO, select “Analysis” from the main toolbar and “Unidimensional IRT.” Click on the “Items” tab in the opened central window, and highlight and move into the right blank window all items of interest. Pressing then “Run” will fit the GR model that is default in this software with polytomous items.

Agresti, A. (2012). Categorical data analysis. New York: Wiley.

Baker, F. B. (2001). The basics of item response theory. Washington: ERIC Clearinghouse on Assessment and Evaluation.

Baker, F. B., & Kim, S.-H. (2004). Item response theory. parameter estimation techniques. Boca Raton: Taylor & Francis.

Bartholomew, D. J., Knott, M., & Moustaki, I. (2011). Factor analysis and latent variable models. New York: Wiley.CrossRef

Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores. Reading, MA: Addison-Wesley.

Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scores in two or more nominal categories. Psychometrika, 37, 29–51.CrossRef

Bock, R. D. (1997a). A brief history of item response theory. Educational Measurement: Issues and Practice, 16, 21–33.CrossRef

Bock, R. D. (1997b). The nominal categories model. In W. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 32–49). New York: Springer.

Cai, L., Thissen, D., & duToit, S. H. C. (2011). IRTPRO user’s guide. Lincolnwood: Scientific Software International, Inc.

Casella, G., & Berger, J. O. (2002). Statistical inference. Monterey: Wadsworth.

Cox, E. P. (1980). The optimal number of response alternatives for a scale: a review. Journal of Marketing Research, 17, 407–422.CrossRef

De Jong, M. G., Steenkamp, J.-B. E. M., Fox, J.-P., & Baumgartner, H. (2008). Using item response theory to measure extreme response style in marketing research: a global investigation. Journal of Marketing Research, 45, 104–115.CrossRef

de Leeuw, J., & Verhelst, N. (1986). Maximum-likelihood estimation in generalized rasch models. Psychometrika, 11, 183–196.

Diamantopoulos, A., Sarstedt, M., Fuchs, C., Wilchynski, P., & Kaiser, S. (2012). Guidelines for choosing between multi-item and single-item scales for construct measurement: a predictive validity perspective. Journal of the Academy of Marketing Science, 40, 434–449.CrossRef

Ewing, M. T., Salzberger, T., & Sunkovic, R. R. (2005). An alternate approach to assessing cross-cultural measurement equivalence in advertising research. Journal of Advertising, 34, 17–36.CrossRef

Fischer, G. (1974). Einführung in die theorie psychologischer tests. Bern: Huber.

Goldstein, H. (2011). Multilevel statistical models. London: Arnold.

Green, P. E. (1984). Hybrid models for conjoint analysis: an expository review. Journal of Marketing Research, 21, 155–170.CrossRef

Kamakura, W. A., & Balasubramanian, S. (1989). Tailored interviewing: an application of item response theory for personality measurement. Journal of Personality Assessment, 53, 502–519.CrossRef

Lancaster, T. (2000). The incidental parameter problem since 1948. Journal of Econometrics, 95, 391–413.CrossRef

Lawley, D. N. (1943). On problems connected with item selection and test construction. Proceedings. Royal Society of Edinburgh, 61, 273–287.

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillside: Erlbaum.

Messick, S. (1995). Validation of inferences from person’s responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749.CrossRef

Muthén, L. K., & Muthén, B. O. (2012). Mplus user’s guide. Los Angeles: Muthén & Muthén.

Nelder, J. A., & Wedderburn, R. W. (1972). Generalized linear models. Journal of the Royal Statistical Society, Series A, 135, 370–384.CrossRef

Neyman, J., & Scott, E. L. (1948). Consistent estimation from partially consistent observations. Econometrica, 16, 1–32.CrossRef

Ostini, R., & Nering, M. L. (2006). Polytomous item response theory models. Thousand Oaks: Sage.

Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institute of Educational Research.

Raykov, T. (2007). Reliability if deleted, not “alpha if deleted”: evaluation of scale reliability following component deletion. British Journal of Mathematical and Statistical Psychology, 60, 201–216.CrossRef

Raykov, T. (2008). “Alpha if item deleted”: a note on loss of criterion validity in scale development if maximising coefficient alpha. British Journal of Mathematical and Statistical Psychology, 61, 275–285.CrossRef

Raykov, T. (2012). Scale development using structural equation modeling. In R. Hoyle (Ed.), Handbook of structural equation modeling (pp. 472–492). New York: Guilford Press.

Raykov, T. (2014). A course in item response modeling (Lecture Notes). Michigan State University, Measurement and Quantitative Methods. MI: East Lansing.

Raykov, T., & Marcoulides, G. A. (2006). A first course in structural equation modeling (second edition). Mahwah: Erlbaum.

Raykov, T., & Marcoulides, G. A. (2008). An introduction to applied multivariate analysis. New York: Taylor & Francis.

Raykov, T., & Marcoulides, G. A. (2011). Introduction to psychometric theory. New York: Taylor & Francis.

Raykov, T., & Marcoulides, G. A. (2012). Basic statistics: an introduction with R. New York: Rowman & Littlefield.

Raykov, T., Marcoulides, G. A. (2014). On examining the underlying normal variable assumption in latent variable models with categorical indicators. Structural Equation Modeling (in press).

Raykov, T., & Pohl, S. (2013a). On studying common factor variance in multiple component measuring instruments. Educational and Psychological Measurement, 73, 191–209.CrossRef

Raykov, T., & Pohl, S. (2013b). Essential unidimensionality in multiple component measuring instruments: a correlation decomposition approach. Educational and Psychological Measurement, 73, 581–600.CrossRef

Raykov, T., Rodenberg, C., Narayanan, A. (2015). On optimal shortenng of psychometric scales. Structural Equation Modeling (in press).

Reckase, M. D. (2009). Multidimensional item response theory. New York: Springer.CrossRef

Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph, No. 17.

Samejima, F. (1997). Graded response model. In W. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 85–100). New York: Springer.

Schultz, C., Salomo, S., & Talke, K. (2013). Measuring new product portfolio innovativeness: how differences in scale width and evaluator perspectives affect its relationship with performance. Journal of Product Innovation Management, 30, 93–109.CrossRef

Stout, W. F. (1990). A new item response theory modeling approach with applications to unidimensionality assessment and ability estimation. Psychometrika, 55, 293–325.CrossRef

Swaminathan, H., & Guifford, J. A. (1982). Bayesian estimation in the rasch model. Journal of Educational Statistics, 7, 175–192.CrossRef

Takane, Y., & de Leeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393–408.CrossRef

Thissen, D. (1982). Marginal maximum-likelihood estimation for the one-parameter logistic model. Psychometrika, 47, 175–186.CrossRef

Timm, N. H. (2002). Multivariate analysis. New York: Springer.

van der Linden, W., & Hambleton, R. K. (Eds.). (1997). Handbook of modern item response theory. New York: Springer.

Verhelst, N., & Molenaar, I. (1988). Logit-based parameter estimation in the rasch model. Statistica Neerlandica, 42, 273–295.CrossRef

Wechsler, D. (1958). The measurement and appraisal of adult intelligence. Baltimore: William & Wilkins.CrossRef

Titel: The utility of item response modeling in marketing research
verfasst von: Tenko Raykov
Roger J. Calantone
Publikationsdatum: 01.07.2014
Verlag: Springer US
Erschienen in: Journal of the Academy of Marketing Science / Ausgabe 4/2014
Print ISSN: 0092-0703
Elektronische ISSN: 1552-7824
DOI: https://doi.org/10.1007/s11747-014-0391-8

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 4/2014

Assessing the value of commonly used methods for measuring customer value: a multi-setting empirical study

The financial contribution of customer-oriented marketing capability

Unjustified prices: environmental drivers of managers’ propensity to overprice

As they sow, so shall they reap: customers’ influence on customer satisfaction at the customer interface

When do incentives work in channels of distribution?

Coping with information requests in marketing exchanges: an examination of pre-post affective control and behavioral coping