Skip to main content
SpringerLink
Log in

Test theory without an answer key

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

A general model is presented for homogeneous, dichotomous items when the answer key is not known a priori. The model is structurally related to the two-class latent structure model with the roles of respondents and items interchanged. For very small sets of respondents, iterative maximum likelihood estimates of the parameters can be obtained by existing methods. For other situations, new estimation methods are developed and assessed with Monte Carlo data. The answer key can be accurately reconstructed with relatively small sets of respondents. The model is useful when a researcher wants to study objectively the knowledge possessed by members of a culturally coherent group that the researcher is not a member of.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Batchelder, W. H., Maher, K. M., & Romney, A. K. (1986).Extensions of a model for test theory without an answer key. Manuscript submitted for publication. (Prepublication draft available upon request).

  • Batchelder, W. H., & Romney, A. K. (1986). The statistical analysis of a general Condorcet model for dichotomous choice situations. In B. Grofman & G. Owen (Eds.),Information pooling and group decision making (pp. 103–112). Greenwich, CT: JAI Press.

    Google Scholar 

  • Blackwell, H. R. (1963). Neural theories of simple visual discriminations.Journal of the Optical Society of America, 53, 129–160.

    Google Scholar 

  • Bliss, C. I. (1967).Statistics in biology (Vol. I). New York: McGraw-Hill.

    Google Scholar 

  • Carré B. (1979).Graphs and networks. Oxford, England: Oxford University Press.

    Google Scholar 

  • Clogg, Clifford C. (1981). New developments in latent structure analysis. In D. M. Jackson & E. F. Borgatta (Eds.),Factor analysis and measurement in sociological research. Beverly Hills, CA: Sage.

    Google Scholar 

  • Comrey, A. L. (1962). The minimum residual method of factor analysis.Psychological Reports, 11, 15–18.

    Google Scholar 

  • Goodman, L. A. (1974). Exploratory latent structure analysis using both identifiable and unidentifiable models.Biometrika, 61, 219–231.

    Google Scholar 

  • Green, D. M., & Swets, J. A. (1966).Signal detection theory and psychophysics. New York: Wiley.

    Google Scholar 

  • Grofman, B., Feld, S. L., & Owen, G. (1984). Group size and the performance of a composite group majority: Statistical truths and empirical results.Organizational Behavior and Human Performance, 33, 350–359.

    Google Scholar 

  • Guilford, J. P. (1954).Psychometric methods (2nd ed.). New York: McGraw-Hill.

    Google Scholar 

  • Hogg, R. V., & Craig, A. T. (1978).Introduction to mathematical statistics (4th ed.). New York: Macmillan.

    Google Scholar 

  • Lazarsfeld, P. F., & Henry, N. W. (1968).Latent structure analysis. New York: Houghton Mifflin.

    Google Scholar 

  • Loéve, M. (1963).Probability theory. Princeton: Van Nostrand.

    Google Scholar 

  • Lord, F. M. (1974). Individualized testing and item characteristic curve theory. In D. H. Krantz, R. C. Atkinson, R. D. Luce, & P. Suppes (Eds.),Contemporary developments in mathematical psychology (Vol. 2). San Francisco: Freeman.

    Google Scholar 

  • Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, Mass.: Addison-Wesley.

    Google Scholar 

  • Maher, K. M. (1987). A multiple choice model for aggregating group knowledge and estimating individual competencies. Unfinished doctoral dissertation, University of California, Irvine. (Available upon request.)

  • Nie, N. H., Hull, J. G., Jenkens, J. G., Steinbrenner, K., & Bent, D. M. (1975).Statistical package for the social sciences. New York: McGraw Hill.

    Google Scholar 

  • Nitzan, S., & Paroush, J. (1982). Optimal decision rules in uncertain dichotomous choice situations.International Economics Review, 23, 289–297.

    Google Scholar 

  • Rasch, G. (1960).Probabilistic models for some intelligence and attainment tests. Copenhagen: Danmarks Paedagogiske Institut.

    Google Scholar 

  • Romney, A. K., Weller, S. C., & Batchelder, W. H. (1986). Culture as consensus: A theory of culture and informant accuracy.American Anthropologist, 88, 313–338.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Social Sciences, University of California, 92717, Irvine, CA

    William H. Batchelder & A. Kimball Romney

Authors
  1. William H. Batchelder
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Kimball Romney
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

This research was supported by NSF Grant No. SES-8320173 to the authors. We gratefully acknowledge comments and suggestions from John Boyd, Tarow Indow, and Kathy Maher as well as the editor and several anonymous referees.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Batchelder, W.H., Romney, A.K. Test theory without an answer key. Psychometrika 53, 71–92 (1988). https://doi.org/10.1007/BF02294195

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02294195

Key words

Search

Navigation