Skip to main content
Top

Hint

Swipe to navigate through the chapters of this book

2020 | OriginalPaper | Chapter

ReadNet: A Hierarchical Transformer Framework for Web Article Readability Analysis

Authors : Changping Meng, Muhao Chen, Jie Mao, Jennifer Neville

Published in: Advances in Information Retrieval

Publisher: Springer International Publishing

share
SHARE

Abstract

Analyzing the readability of articles has been an important sociolinguistic task. Addressing this task is necessary to the automatic recommendation of appropriate articles to readers with different comprehension abilities, and it further benefits education systems, web information systems, and digital libraries. Current methods for assessing readability employ empirical measures or statistical learning techniques that are limited by their ability to characterize complex patterns such as article structures and semantic meanings of sentences. In this paper, we propose a new and comprehensive framework which uses a hierarchical self-attention model to analyze document readability. In this model, measurements of sentence-level difficulty are captured along with the semantic meanings of each sentence. Additionally, the sentence-level features are incorporated to characterize the overall readability of an article with consideration of article structures. We evaluate our proposed approach on three widely-used benchmark datasets against several strong baseline approaches. Experimental results show that our proposed method achieves the state-of-the-art performance on estimating the readability for various web articles and literature.
Literature
1.
go back to reference Anderson, J.: Lix and Rix: variations on a little-known readability index. J. Read. 26(6), 490–496 (1983) Anderson, J.: Lix and Rix: variations on a little-known readability index. J. Read. 26(6), 490–496 (1983)
2.
go back to reference Brown, J., Eskenazi, M.: Student, text and curriculum modeling for reader-specific document retrieval. In: Proceedings of the IASTED International Conference on Human-Computer Interaction, Phoenix, AZ (2005) Brown, J., Eskenazi, M.: Student, text and curriculum modeling for reader-specific document retrieval. In: Proceedings of the IASTED International Conference on Human-Computer Interaction, Phoenix, AZ (2005)
3.
go back to reference Chall, J.S.: Readability: an appraisal of research and application, no. 34 (1958) Chall, J.S.: Readability: an appraisal of research and application, no. 34 (1958)
4.
go back to reference Chall, J.S., Dale, E.: Readability Revisited: The New Dale-Chall Readability Formula. Brookline Books (1995) Chall, J.S., Dale, E.: Readability Revisited: The New Dale-Chall Readability Formula. Brookline Books (1995)
5.
go back to reference Chen, M., et al.: Multifaceted protein-protein interaction prediction based on Siamese residual RCNN. Bioinformatics 35(14), i305–i314 (2019) Chen, M., et al.: Multifaceted protein-protein interaction prediction based on Siamese residual RCNN. Bioinformatics 35(14), i305–i314 (2019)
7.
go back to reference Chen, M., Meng, C., Huang, G., Zaniolo, C.: Learning to differentiate between main-articles and sub-articles in Wikipedia. In: Proceedings of the IEEE International Conference on Big Data (2019) Chen, M., Meng, C., Huang, G., Zaniolo, C.: Learning to differentiate between main-articles and sub-articles in Wikipedia. In: Proceedings of the IEEE International Conference on Big Data (2019)
8.
go back to reference Chen, M., Tian, Y., Chen, X., Xue, Z., Zaniolo, C.: On2Vec: embedding-based relation prediction for ontology population. In: Proceedings of the 2018 SIAM International Conference on Data Mining, pp. 315–323. SIAM (2018) Chen, M., Tian, Y., Chen, X., Xue, Z., Zaniolo, C.: On2Vec: embedding-based relation prediction for ontology population. In: Proceedings of the 2018 SIAM International Conference on Data Mining, pp. 315–323. SIAM (2018)
9.
go back to reference Coleman, M., Liau, T.L.: A computer readability formula designed for machine scoring. J. Appl. Psychol. 60(2), 283 (1975) Coleman, M., Liau, T.L.: A computer readability formula designed for machine scoring. J. Appl. Psychol. 60(2), 283 (1975)
10.
go back to reference Collins-Thompson, K., Callan, J.: A language modeling approach to predicting reading difficulty. In: Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, HLT-NAACL 2004 (2004) Collins-Thompson, K., Callan, J.: A language modeling approach to predicting reading difficulty. In: Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, HLT-NAACL 2004 (2004)
11.
go back to reference Collins-Thompson, K.: Computational assessment of text readability: a survey of current and future research. ITL-Int. J. Appl. Linguist. 165(2), 97–135 (2014) Collins-Thompson, K.: Computational assessment of text readability: a survey of current and future research. ITL-Int. J. Appl. Linguist. 165(2), 97–135 (2014)
12.
go back to reference Collins-Thompson, K., Callan, J.: Predicting reading difficulty with statistical language models. J. Am. Soc. Inform. Sci. Technol. 56(13), 1448–1462 (2005) Collins-Thompson, K., Callan, J.: Predicting reading difficulty with statistical language models. J. Am. Soc. Inform. Sci. Technol. 56(13), 1448–1462 (2005)
13.
go back to reference Coxhead, A.: A new academic word list. TESOL Q. 34(2), 213–238 (2000) Coxhead, A.: A new academic word list. TESOL Q. 34(2), 213–238 (2000)
14.
go back to reference Dale, E., Chall, J.S.: The concept of readability. Elem. Engl. 26(1), 19–26 (1949) Dale, E., Chall, J.S.: The concept of readability. Elem. Engl. 26(1), 19–26 (1949)
15.
go back to reference De Clercq, O., Hoste, V.: All mixed up? Finding the optimal feature set for general readability prediction and its application to English and Dutch. Comput. Linguist. 42(3), 457–490 (2016) MathSciNet De Clercq, O., Hoste, V.: All mixed up? Finding the optimal feature set for general readability prediction and its application to English and Dutch. Comput. Linguist. 42(3), 457–490 (2016) MathSciNet
16.
go back to reference Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12(Jul), 2121–2159 (2011) MathSciNetMATH Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12(Jul), 2121–2159 (2011) MathSciNetMATH
17.
go back to reference Feng, L., Elhadad, N., Huenerfauth, M.: Cognitively motivated features for readability assessment. In: Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics, pp. 229–237. Association for Computational Linguistics (2009) Feng, L., Elhadad, N., Huenerfauth, M.: Cognitively motivated features for readability assessment. In: Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics, pp. 229–237. Association for Computational Linguistics (2009)
18.
go back to reference François, T.L.: Combining a statistical language model with logistic regression to predict the lexical and syntactic difficulty of texts for FFL. In: Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, pp. 19–27. Association for Computational Linguistics (2009) François, T.L.: Combining a statistical language model with logistic regression to predict the lexical and syntactic difficulty of texts for FFL. In: Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, pp. 19–27. Association for Computational Linguistics (2009)
19.
go back to reference Fry, E.: A readability formula that saves time. J. Read. 11(7), 513–578 (1968) Fry, E.: A readability formula that saves time. J. Read. 11(7), 513–578 (1968)
20.
go back to reference Fry, E.B.: The varied uses of readability measurement today. J. Read. 30(4), 338–343 (1987) Fry, E.B.: The varied uses of readability measurement today. J. Read. 30(4), 338–343 (1987)
21.
go back to reference Gibson, E.: Linguistic complexity: locality of syntactic dependencies. Cognition 68(1), 1–76 (1998) Gibson, E.: Linguistic complexity: locality of syntactic dependencies. Cognition 68(1), 1–76 (1998)
23.
go back to reference Gunning, R.: The fog index after twenty years. J. Bus. Commun. 6(2), 3–13 (1969) Gunning, R.: The fog index after twenty years. J. Bus. Commun. 6(2), 3–13 (1969)
24.
go back to reference Heilman, M., Collins-Thompson, K., Eskenazi, M.: An analysis of statistical models and features for reading difficulty prediction. In: 3rd Workshop on Innovative Use of NLP for Building Educational Applications (2008) Heilman, M., Collins-Thompson, K., Eskenazi, M.: An analysis of statistical models and features for reading difficulty prediction. In: 3rd Workshop on Innovative Use of NLP for Building Educational Applications (2008)
25.
go back to reference Heilman, M., et al.: Combining lexical and grammatical features to improve readability measures for first and second language texts. In: Human Language Technologies (2007) Heilman, M., et al.: Combining lexical and grammatical features to improve readability measures for first and second language texts. In: Human Language Technologies (2007)
26.
go back to reference Kauchak, D.: Improving text simplification language modeling using un simplified text data. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 1537–1546 (2013) Kauchak, D.: Improving text simplification language modeling using un simplified text data. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 1537–1546 (2013)
27.
go back to reference Kim, Y.: Convolutional neural networks for sentence classification. In: Empirical Methods in Natural Language Processing (2014) Kim, Y.: Convolutional neural networks for sentence classification. In: Empirical Methods in Natural Language Processing (2014)
28.
go back to reference Kincaid, J.P., Fishburne Jr, R.P., Rogers, R.L., Chissom, B.S.: Derivation of new readability formulas for navy enlisted personnel (1975) Kincaid, J.P., Fishburne Jr, R.P., Rogers, R.L., Chissom, B.S.: Derivation of new readability formulas for navy enlisted personnel (1975)
29.
go back to reference Klare, G.R.: The measurement of readability: useful information for communicators. ACM J. Comput. Doc. (JCD) 24(3), 107–121 (2000) Klare, G.R.: The measurement of readability: useful information for communicators. ACM J. Comput. Doc. (JCD) 24(3), 107–121 (2000)
30.
31.
go back to reference Li, Z., Wei, Y., Zhang, Y., Yang, Q.: Hierarchical attention transfer network for cross-domain sentiment classification. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018) Li, Z., Wei, Y., Zhang, Y., Yang, Q.: Hierarchical attention transfer network for cross-domain sentiment classification. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)
32.
go back to reference Lin, R., Liu, S., Yang, M., Li, M., Zhou, M., Li, S.: Hierarchical recurrent neural network for document modeling. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 899–907 (2015) Lin, R., Liu, S., Yang, M., Li, M., Zhou, M., Li, S.: Hierarchical recurrent neural network for document modeling. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 899–907 (2015)
33.
go back to reference Louwerse, M.: An analytic and cognitive parametrization of coherence relations. Cogn. Linguist. 12(3), 291–316 (2001) Louwerse, M.: An analytic and cognitive parametrization of coherence relations. Cogn. Linguist. 12(3), 291–316 (2001)
34.
go back to reference Malvern, D., Richards, B.: Measures of lexical richness. In: The Encyclopedia of Applied Linguistics (2012) Malvern, D., Richards, B.: Measures of lexical richness. In: The Encyclopedia of Applied Linguistics (2012)
35.
go back to reference Mc Laughlin, G.H.: SMOG grading-a new readability formula. J. Read. 12(8), 639–646 (1969) Mc Laughlin, G.H.: SMOG grading-a new readability formula. J. Read. 12(8), 639–646 (1969)
36.
go back to reference McNamara, D.S., Graesser, A.C., McCarthy, P.M., Cai, Z.: Automated Evaluation of Text and Discourse with Coh-Metrix. Cambridge University Press, Cambridge (2014) McNamara, D.S., Graesser, A.C., McCarthy, P.M., Cai, Z.: Automated Evaluation of Text and Discourse with Coh-Metrix. Cambridge University Press, Cambridge (2014)
37.
go back to reference McNamara, D.S., Louwerse, M.M., McCarthy, P.M., Graesser, A.C.: Coh-Metrix: capturing linguistic features of cohesion. Discourse Process. 47(4), 292–330 (2010) McNamara, D.S., Louwerse, M.M., McCarthy, P.M., Graesser, A.C.: Coh-Metrix: capturing linguistic features of cohesion. Discourse Process. 47(4), 292–330 (2010)
38.
go back to reference Parikh, A.P., Täckström, O., Das, D., Uszkoreit, J.: A decomposable attention model for natural language inference. arXiv preprint arXiv:​1606.​01933 (2016) Parikh, A.P., Täckström, O., Das, D., Uszkoreit, J.: A decomposable attention model for natural language inference. arXiv preprint arXiv:​1606.​01933 (2016)
39.
go back to reference Pezeshkpour, P., Chen, L., Singh, S.: Embedding multimodal relational data for knowledge base completion. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3208–3218 (2018) Pezeshkpour, P., Chen, L., Singh, S.: Embedding multimodal relational data for knowledge base completion. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3208–3218 (2018)
40.
go back to reference Pilán, I., Volodina, E., Zesch, T.: Predicting proficiency levels in learner writings by transferring a linguistic complexity model from expert-written coursebooks. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 2101–2111 (2016) Pilán, I., Volodina, E., Zesch, T.: Predicting proficiency levels in learner writings by transferring a linguistic complexity model from expert-written coursebooks. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 2101–2111 (2016)
41.
go back to reference Pitler, E., Nenkova, A.: Revisiting readability: a unified framework for predicting text quality. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 186–195. Association for Computational Linguistics (2008) Pitler, E., Nenkova, A.: Revisiting readability: a unified framework for predicting text quality. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 186–195. Association for Computational Linguistics (2008)
42.
go back to reference Rennie, J.D., Srebro, N.: Loss functions for preference levels: regression with discrete ordered labels. In: Proceedings of the IJCAI Multidisciplinary Workshop on Advances in Preference Handling, pp. 180–186. Kluwer Norwell (2005) Rennie, J.D., Srebro, N.: Loss functions for preference levels: regression with discrete ordered labels. In: Proceedings of the IJCAI Multidisciplinary Workshop on Advances in Preference Handling, pp. 180–186. Kluwer Norwell (2005)
43.
go back to reference Schwarm, S.E., Ostendorf, M.: Reading level assessment using support vector machines and statistical language models. In: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pp. 523–530. Association for Computational Linguistics (2005) Schwarm, S.E., Ostendorf, M.: Reading level assessment using support vector machines and statistical language models. In: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pp. 523–530. Association for Computational Linguistics (2005)
44.
go back to reference Senter, R., Smith, E.A.: Automated readability index. Technical report, Cincinnati University, OH (1967) Senter, R., Smith, E.A.: Automated readability index. Technical report, Cincinnati University, OH (1967)
45.
go back to reference Si, L., Callan, J.: A statistical model for scientific readability. In: CIKM, vol. 1, pp. 574–576 (2001) Si, L., Callan, J.: A statistical model for scientific readability. In: CIKM, vol. 1, pp. 574–576 (2001)
46.
go back to reference Socher, R., et al.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631–1642 (2013) Socher, R., et al.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631–1642 (2013)
47.
go back to reference Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1556–1566 (2015) Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1556–1566 (2015)
48.
go back to reference Tang, D., Qin, B., Liu, T.: Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1422–1432 (2015) Tang, D., Qin, B., Liu, T.: Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1422–1432 (2015)
49.
go back to reference Vajjala, S., Meurers, D.: On improving the accuracy of readability classification using insights from second language acquisition. In: Proceedings of the Seventh Workshop on Building Educational Applications Using NLP, pp. 163–173. Association for Computational Linguistics (2012) Vajjala, S., Meurers, D.: On improving the accuracy of readability classification using insights from second language acquisition. In: Proceedings of the Seventh Workshop on Building Educational Applications Using NLP, pp. 163–173. Association for Computational Linguistics (2012)
50.
go back to reference Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017) Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
51.
go back to reference Xia, M., Kochmar, E., Briscoe, T.: Text readability assessment for second language learners. In: Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications, pp. 12–22 (2016) Xia, M., Kochmar, E., Briscoe, T.: Text readability assessment for second language learners. In: Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications, pp. 12–22 (2016)
52.
go back to reference Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1480–1489 (2016) Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1480–1489 (2016)
53.
go back to reference Zakaluk, B.L., Samuels, S.J.: Readability: Its Past, Present, and Future. ERIC (1988) Zakaluk, B.L., Samuels, S.J.: Readability: Its Past, Present, and Future. ERIC (1988)
Metadata
Title
ReadNet: A Hierarchical Transformer Framework for Web Article Readability Analysis
Authors
Changping Meng
Muhao Chen
Jie Mao
Jennifer Neville
Copyright Year
2020
DOI
https://doi.org/10.1007/978-3-030-45439-5_3