Skip to main content

Advertisement

SpringerLink
Log in

Reconstructing Readability: Recent Developments and Recommendations in the Analysis of Text Difficulty

  • Review
  • Published:
Educational Psychology Review Aims and scope Submit manuscript

Abstract

Largely due to technological advances, methods for analyzing readability have increased significantly in recent years. While past researchers designed hundreds of formulas to estimate the difficulty of texts for readers, controversy has surrounded their use for decades, with criticism stemming largely from their application in creating new texts as well as their utilization of surface-level indicators as proxies for complex cognitive processes that take place when reading a text. This review focuses on examining developments in the field of readability during the past two decades with the goal of informing both current and future research and providing recommendations for present use. The fields of education, linguistics, cognitive science, psychology, discourse processing, and computer science have all made recent strides in developing new methods for predicting the difficulty of texts for various populations. However, there is a need for further development of these methods if they are to become widely available.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Anderson, J. R. (1983). A spreading activation theory of memory. Journal of Verbal Learning and Verbal Behavior, 22, 261–295.

    Article  Google Scholar 

  • Anderson, R., & Pichert, J. (1978). Recall of previously unrecallable information following a shift in perspective. Journal of Verbal Learning & Verbal Behavior, 17(1), 1–12. doi:10.1016/S0022-5371(78)90485-1.

    Article  Google Scholar 

  • Bailin, A., & Grafstein, A. (2001). The linguistic assumptions underlying readability formulae: A critique. Language & Communication, 21(3), 285–301.

    Article  Google Scholar 

  • Blackburn, B. (2000). Best practices for using Lexiles. Popular Measurement, 3(1), 22–24.

    Google Scholar 

  • Bormuth, J. (1966). Readability: A new approach. Reading Research Quarterly, 1, 79–132.

    Article  Google Scholar 

  • Bormuth, J.R. (1969). Development of readability analyses. Final Report, Project No. 7-0052, Contract No. 1, OEC-3-7-070052-0326. Washington, DC: U.S. Office of Education.

  • Bormuth, J. R. (1971). Development of standards of readability: Toward a rational criterion of passage performance. Final report, U.S. Office of Education, Project No. 9-0237. Chicago: University of Chicago.

  • Britton, B., & Gülgöz, S. (1991). Using Kintsch’s computational model to improve instructional text: Effects of repairing inference calls on recall and cognitive structures. Journal of Educational Psychology, 83(3), 329–345. doi:10.1037/0022-0663.83.3.329.

    Article  Google Scholar 

  • Britton, B., Gülgöz, S., Glynn, S. (1993). Impact of good and poor writing on learners: Research and theory. Learning from textbooks: Theory and practice (pp. 1–46). Hillsdale, NJ: Lawrence Erlbaum.

  • Carroll, J. B., Davies, P., & Richman, B. (Eds.). (1971). Word frequency book. New York: Houghton Mifflin.

    Google Scholar 

  • Caylor, J.S., Sticht, T.G., Fox, L.C., Ford, J.P. 1973. Methodologies for determining reading requirements of military occupational specialties: Technical report No. 73-5. Alexandria, VA: Human Resources Research Organization.

  • Chall, J. S., & Dale, E. (1995). Readability revisited: The new Dale–Chall readability formula. Cambridge: Brookline Books.

    Google Scholar 

  • Chall, J. S., Bissex, G. L., Conrad, S. S., & Harris-Sharples, S. (1996). Qualitative assessment of text difficulty: A practical guide for teachers and writers. Cambridge: Brookline Books.

    Google Scholar 

  • Coleman, E. (1962). Improving comprehensibility by shortening sentences. Journal of Applied Psychology, 46(2), 131–134. doi:10.1037/h0039740.

    Article  Google Scholar 

  • Collins-Thompson, K., & Callan, J. (2004). A language modeling approach to predicting reading difficulty. In S. Dumais, D. Marcu, & S. Roukos (Eds.), HLT-NAACL 2004: Main proceedings (pp. 193–200). Morristown: Association for Computational Linguistics.

    Google Scholar 

  • Collins-Thompson, K., & Callan, J. (2005). Predicting reading difficulty with statistical language models. Journal of the American Society for Information Science & Technology, 56(13), 1448–1462. doi:10.1002/asi.20243.

    Article  Google Scholar 

  • Crossley, S. A., Dufty, D. F., McCarthy, P. M., & McNamara, D. S. (2007). Toward a new readability: A mixed model approach. In D. S. McNamara & G. Trafton (Eds.), Proceedings of the 29th Annual Conference of the Cognitive Science Society (pp. 197–202). Austin: Cognitive Science Society.

    Google Scholar 

  • Dale, E., & Chall, J. S. (1948). A formula for predicting readability. Educational Research Bulletin, 27(1), 11–20–28.

    Google Scholar 

  • Dam, G., & Kaufmann, S. (2008). Computer assessment of interview data using latent semantic analysis. Behavior Research Methods, 40(1), 8–20. doi:10.3758/BRM.40.1.8.

    Article  Google Scholar 

  • Davison, A., & Kantor, R. (1982). On the failure of readability formulas to define readable texts: A case study from adaptations. Reading Research Quarterly, 17(2), 187–209.

    Article  Google Scholar 

  • DuBay, W.H. 2004. The principles of readability. Retrieved 30 August 2010 from http://www.impact-information.com/impactinfo/readability02.pdf.

  • Fellbaum, C. (Ed.). (1998). WordNet: An electronic lexical database. Cambridge: MIT.

    Google Scholar 

  • Feng, L. (2009). Automatic readability assessment for people with intellectual disabilities. ACM SIGACCESS Accessibility and Computing, 93, 84–91. doi:10.1145/1531930.1531940.

    Article  Google Scholar 

  • Foltz, P., Kintsch, W., & Landauer, T. (1998). The measurement of textual coherence with latent semantic analysis. Discourse Processes, 25(2–3), 285–307. doi:10.1080/01638539809545029.

    Article  Google Scholar 

  • Fountas, I. C., & Pinnell, G. S. (1999). Matching books to readers: Using leveled books in guided reading, K-3. Portsmouth: Heinemann.

    Google Scholar 

  • Fountas, I. C., & Pinnell, G. S. (2001). Guiding readers and writers: Grades 3–6. Portsmouth: Heinemann.

    Google Scholar 

  • Fry, E. (1990). A readability formula for short passages. Journal of Reading, 33(8), 594–97.

    Google Scholar 

  • Graesser, A. C., Gernsbacher, M. A., & Goldman, S. R. (1997). Cognition. In T. A. van Dijk & T. A. van Dijk (Eds.), Discourse as structure and process: Discourse studies: A multidisciplinary introduction, vol. 1 (pp. 292–319). Thousand Oaks: Sage.

    Google Scholar 

  • Graesser, A., McNamara, D., Louwerse, M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments & Computers, 36(2), 193–202.

    Article  Google Scholar 

  • Green, A., Ünaldi, A., & Weir, C. (2010). Empiricism versus connoisseurship: Establishing the appropriacy of texts in tests of academic reading. Language Testing, 27(2), 191–211. doi:10.1177/0265532209349471.

    Article  Google Scholar 

  • Heilman, M., Collins-Thompson, K., Callan, J., Eskenazi, M. (2007). Combining lexical and grammatical features to improve readability measures for first and second language texts. In Proceedings of the NAACL Human Language Technology Conference (pp. 460–467). Morristown, NJ: Association for Computational Linguistics.

  • Heilman, M., Collins-Thompson, K., Eskenazi, M. (2008). An analysis of statistical models and features for reading difficulty prediction. In EANL '08 Proceedings of the Third Workshop on Innovative Use of NLP for Building Educational Applications, June 19–19 (pp.71–79). Morristown, NJ: Association for Computational Linguistics.

  • Hiebert, E. (1999). Text matters in learning to read. Reading Teacher, 52(6), 552–66.

    Google Scholar 

  • Hiebert, E. H., Pearson, P. D. (2010). An examination of current text difficulty indices with early reading texts (Reading Research Report No. 10-01). Santa Cruz, CA: TextProject, Inc.

  • Just, M., & Carpenter, P. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87(4), 329–354. doi:10.1037/0033-295X.87.4.329.

    Article  Google Scholar 

  • Kalyuga, S., Chandler, P., Sweller, J. (2000). Incorporating learner experience into the design of multimedia instruction. Journal of Educational Psychology, 92, 126–136.2000-03003-01110.1037/0022-0663.92.1.126. doi:10.1037/0022-0663.92.1.126.

    Google Scholar 

  • Kalyuga, S., Chandler, P., & Sweller, J. (2001). Learner experience and efficiency of instructional guidance. Educational Psychology, 21, 5–23. doi:10.1080/0144341012468110.1080/014434101246812001-16707-001.10.1080/01443410124681.

    Article  Google Scholar 

  • Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving is superior to studying worked examples. Journal of Educational Psychology, 93, 579–588. doi:10.1037/0022-0663.93.3.57910.1037/0022-0663.93.3.5792001-18059-013.10.1037/0022-0663.93.3.579.

    Article  Google Scholar 

  • Kalyuga, S., Ayres, P., Chandler, P., Sweller, J. (2003). The expertise reversal effect. Educational Psychologist, 38, 23–31. doi:10.1207/S15326985EP3801. 10.1207/S15326985EP3801.

    Google Scholar 

  • Kim, H., Goryachev, S., Rosemblat, G., Browne, A., Keselman, A., & Zeng-Treitler, Q. (2007). Beyond surface characteristics: A new health text-specific readability measurement. American Medical Informatics (AMIA) Annual Symposium (pp. 418–422). Washington, DC.

  • Kintsch, W. (1988). The role of knowledge in discourse comprehension: A construction–integration model. Psychological Review, 95(2), 163–182. doi:10.1037/0033-295X.95.2.163.

    Article  Google Scholar 

  • Kintsch, W., & Keenan, J. (1973). Reading rate and retention as a function of the number of propositions in the base structure of sentences. Cognitive Psychology, 5(3), 257–274. doi:10.1016/0010-0285(73)90036-4.

    Article  Google Scholar 

  • Kintsch, W., & van Dijk, T. (1978). Toward a model of text comprehension and production. Psychological Review, 85(5), 363–394. doi:10.1037/0033-295X.85.5.363.

    Article  Google Scholar 

  • Kirsch, I. S., & Mosenthal, T. B. (1990). Exploring document literacy: Variables underlying the performance of young adults. Reading Research Quarterly, 25, 5–30.

    Article  Google Scholar 

  • Klare, G. R. (1963). The measurement of readability. Ames: Iowa State University Press.

    Google Scholar 

  • Klare, G. (1974). Assessing readability. Reading Research Quarterly, 10, 62–102.

    Article  Google Scholar 

  • Landauer, T., Foltz, P., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25(2–3), 259–284. doi:10.1080/01638539809545028.

    Article  Google Scholar 

  • Leahy, W., Chandler, P., Sweller, J. (2003). When auditory presentations should and should not be a component of multimedia instruction. Applied Cognitive Psychology, 17, 401–418.2003-00690-00410.1002/acp.877. doi:10.1002/acp.877.

  • Lin, S., Su, C., Lai, Y., Yang, L., & Hsieh, S. (2009). Assessing text readability using hierarchical lexical relations retrieved from WordNet. Computational Linguistics and Chinese Language Processing, 14(1), 45–84.

    Google Scholar 

  • Liu, X., Croft, W.B., Oh, P., Hart, D. (2004). Automatic recognition of reading levels from user queries. SIGIR '04 Proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval (pp. 548–549). New York, NY: ACM.

  • McClelland, J., & Rumelhart, D. (1981). An interactive activation model of context effects in letter perception: I. An account of basic findings. Psychological Review, 88(5), 375–407. doi:10.1037/0033-295X.88.5.375.

    Article  Google Scholar 

  • McNamara, D., & Kintsch, W. (1996). Learning from texts: Effects of prior knowledge and text coherence. Discourse Processes, 22(3), 247–288. doi:10.1080/01638539609544975.

    Article  Google Scholar 

  • McNamara, D., Kintsch, E., Songer, N., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14(1), 1–43. doi:10.1207/s1532690xci1401_1.

    Article  Google Scholar 

  • McNamara, D., Crossley, S., & McCarthy, P. (2010). Linguistic features of writing quality. Written Communication, 27(1), 57–86. doi:10.1177/0741088309351547.

    Article  Google Scholar 

  • McNamara, D., Louwerse, M., McCarthy, P., & Graesser, A. (2010). Coh-Metrix: Capturing linguistic features of cohesion. Discourse Processes, 47(4), 292–330. doi:10.1080/01638530902959943.

    Article  Google Scholar 

  • Meyer, B., Marsiske, M., & Willis, S. (1993). Text processing variables predict the readability of everyday documents read by older adults. Reading Research Quarterly, 28(3), 234–249. doi:10.2307/747996.

    Article  Google Scholar 

  • Miller, T. (2003). Essay assessment with latent semantic analysis. Journal of Educational Computing Research, 29(4), 495–512. doi:10.2190/W5AR-DYPW-40KX-FL99.

    Article  Google Scholar 

  • Miller, G. R., & Coleman, E. B. (1967). A set of thirty-six prose passages calibrated for complexity. Journal of Verbal Learning and Verbal Behavior, 6(6), 851–854.

    Article  Google Scholar 

  • Miller, J., & Kintsch, W. (1980). Readability and recall of short prose passages: A theoretical analysis. Journal of Experimental Psychology: Human Learning and Memory, 6(4), 335–354. doi:10.1037/0278-7393.6.4.335.

    Article  Google Scholar 

  • Millis, K., Magliano, J., Wiemer-Hastings, K., Todaro, S., McNamara, D. (2007). Assessing and improving comprehension with latent semantic analysis. Handbook of latent semantic analysis (pp. 207–225). Mahwah, NJ: Lawrence Erlbaum.

  • Milone, M. (2009). The development of ATOS: The renaissance readability formula. Wisconsin Rapids: Renaissance Learning.

    Google Scholar 

  • Miltsakaki, E., Troutt, A. (2007). Read-X: Automatic evaluation of reading difficulty of web text. Proceedings of E-Learn 2007, sponsored by the Association for the Advancement of Computing in Education. Quebec, Canada.

  • Miltsakaki, E., & Troutt, A. (2008). Real-time web text classification and analysis of reading difficulty. In J. Tetreault, J. Burstein, & R. De Felice (Eds.), EANL '08 Proceedings of the Third Workshop on Innovative Use of NLP for Building Educational Applications (pp. 89–97). Morristown: Association for Computational Linguistics.

    Chapter  Google Scholar 

  • Peterson, B. (1991). Selecting books for beginning readers: Children’s literature suitable for young readers. In D. E. DeFord, C. A. Lyons, & G. S. Pinnell (Eds.), Bridges to literacy: Learning from reading recovery (pp. 119–147). Portsmouth: Heinemann.

    Google Scholar 

  • Peterson, S., & Ostendorf, M. (2009). A machine learning approach to reading level assessment. Computer Speech and Language, 23(1), 89–106.

    Article  Google Scholar 

  • Rog, L., & Burton, W. (2002). Matching texts and readers: Leveling early reading materials for assessment and instruction. Reading Teacher, 55(4), 348–56.

    Google Scholar 

  • Rosch, E., Mervis, C. B., Gray, W., Johnson, D., & Boyes-Braem, P. (1976). Basic objects in natural categories. Cognitive Psychology, 8(3), 382–439.

    Article  Google Scholar 

  • Rumelhart, D., & McClelland, J. (1982). An interactive activation model of context effects in letter perception: II. The contextual enhancement effect and some tests and extensions of the model. Psychological Review, 89(1), 60–94. doi:10.1037/0033-295X.89.1.60.

    Article  Google Scholar 

  • School Renaissance Inst., Inc. (2000). The ATOS[TM] readability formula for books and how it compares to other formulas. Madison, WI: School Renaissance Inst., Inc. (ERIC Document Reproduction Service No. ED449468).

  • Schriver, K. A. (2000). Readability formulas in the new millennium: What’s the use? ACM Journal of Computer Documentation, 24(3), 105–106.

    Article  Google Scholar 

  • Schwarm, S.E., Ostendorf, M. 2005. Reading level assessment using support vector machines and statistical language models. Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pp. 523–530, Ann Arbor, MI.

  • Si, L., Callan, J. 2001. A statistical model for scientific readability. CIKM’01: Proceedings of the Tenth International Conference on Information and Knowledge Management, pp. 574–576.

  • Smith, R. (2000a). How the Lexile framework operates. Popular Measurement, 3(1), 18–19.

    Google Scholar 

  • Smith, R. (2000b). The Lexile community: From science to practice. Popular Measurement, 3(1), 20–21.

    Google Scholar 

  • Smith, D., Stenner, A.J., Horabin, I., Smith, M. (1989). The Lexile scale in theory and practice: Final report. Washington, DC: MetaMetrics (ERIC Document Reproduction Service No. ED307577).

  • Stenner, A.J. (1996). Measuring reading comprehension with the Lexile framework. Paper presented at the 4th North American Conference on Adolescent/Adult Literacy, Washington, DC.

  • Stenner, A.J. (1999). Instructional uses of the Lexile framework. Durham, NC: MetaMetrics, Inc. (ERIC Document Reproduction Service No. ED435976).

  • Stenner, A., Burdick, D. (1997). The objective measurement of reading comprehension: In response to technical questions raised by the California Department of Education Technical Study Group. Durham, NC: MetaMetrics, Inc. (ERIC Document Reproduction Service No. ED435978).

  • Stenner, A.J., Burdick, H., Sanford, E.E., Burdick, D.S. (2007). The Lexile framework for reading technical report. MetaMetrics, Inc.

  • vor der Brück, T., Hartrumpf, S., & Helbig, H. (2008). A readability checker with supervised learning using deep indicators. Informatica, 32(4), 429–435.

    Google Scholar 

  • Wolfe, M., Schreiner, M., Rehder, B., Laham, D., Foltz, P., Kintsch, W., et al. (1998). Learning from text: Matching readers and texts by latent semantic analysis. Discourse Processes, 25(2–3), 309–336. doi:10.1080/01638539809545030.

    Article  Google Scholar 

  • Wright, B., Stenner, A. (1998). Readability and reading ability. Paper presented to the Australian Council on Education Research (ACER) (ERIC Document Reproduction Service No. ED435979).

  • Wright, B., & Stenner, A. (2000). Lexile perspectives. Popular Measurement, 3(1), 16–17.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Educational Psychology, University of Georgia, Athens, GA, 30602, USA

    Rebekah George Benjamin

Authors
  1. Rebekah George Benjamin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rebekah George Benjamin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Benjamin, R.G. Reconstructing Readability: Recent Developments and Recommendations in the Analysis of Text Difficulty. Educ Psychol Rev 24, 63–88 (2012). https://doi.org/10.1007/s10648-011-9181-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10648-011-9181-8

Keywords

Search

Navigation