Top

Education and Information Technologies

Published in:

10-10-2020

Question generation model based on key-phrase, context-free grammar, and Bloom’s taxonomy

Authors: Bambang Dwi Wijanarko, Yaya Heryadi, Hapnes Toba, Widodo Budiharto

Published in: Education and Information Technologies | Issue 2/2021

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Automated question generation is a task to generate questions from structured or unstructured data. The increasing popularity of online learning in recent years has given momentum to automated question generation in education field for facilitating learning process, learning material retrieval, and computer-based testing. This paper report on the development of question generation framework based on key-phrase method for online learning with a constraint that the generated questions should comply with the learning outcomes and skills from Bloom’s Taxonomy. The proposed method was tested using learning materials of Software Engineering course for undergraduate level written in Bahasa Indonesia obtained from Bina Nusantara’s (Binus’s) Online Learning repository. Using one-semester lecture material, this study generated 92,608 essay-type questions from 6-level Bloom’s Taxonomy which were further sampled randomly to obtain 120 question samples for method evaluation. Performance evaluation using average Bilingual Evaluation Understudy (BLEU) involving five independent reviewers toward samples of these questions achieved 0.921 and 0.6 Cohen’s Kappa. The relevance of Bloom’s Taxonomy level of the generated questions was evaluated by means of classification model with 0.99 accuracy. The results indicate that not only are the generated questions well understood and agreed by the reviewers, they are also relevant to the expected Bloom’s Taxonomy level there for the questions can be delivered to students in the respected course delivery and evaluation.

previous article Identifying the differentiation practices of virtual school teachers

next article Evaluation of course outcome attainment of engineering course with traditional, blended and flipped classroom approaches

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Adamson, D., Bhartiya, D., Gujral, B., Kedia, R., Singh, A., & Rosé, C. P. (2013). Automatically generating discussion questions. In AIED (pp. 81–90). https://doi.org/10.1007/978-3-642-39112-5_9.

Al-Yahya, M. (2011). OntoQue: A question generation engine for educational assesment based on domain ontologies. In Proceedings of the 2011 11th IEEE International Conference on Advanced Learning Technologies, ICALT 2011 (pp. 393–395). IEEE. https://doi.org/10.1109/ICALT.2011.124.

Bloom, B.S. (1956). Taxonomy of educational objectives: The classification of educational goals.

Chao, Z., & Li, L. (2018). The combination of context information to enhance simple question answering. In Proceedings - 2018 5th international conference on behavioral, economic, and socio-cultural computing, BESC 2018 (pp. 109–114). https://doi.org/10.1109/BESC.2018.8697305.CrossRef

Diab, S., & Sartawi, B. (2017). Classification of questions and learning outcome statements (LOS) into Bloom’s taxonomy (BT) by similarity measurements towards extracting of learning outcome from learning material. International Journal of Managing Information Technology, 9(2), 01–12. https://doi.org/10.5121/ijmit.2017.9201.CrossRef

Divate, M., & Salgaonkar, A. (2017). Automatic question generation approaches and evaluation techniques. Current Science (00113891), 113(9).

Duan, N., Tang, D., Chen, P., & Zhou, M. (2017). Question generation for question answering. In EMNLP 2017 - Conference on Empirical Methods in Natural Language Processing, Proceedings (pp. 877–885). https://doi.org/10.18653/v1/d17-1090.

Elsahar, H., Gravier, C., & Laforest, F. F. (2018). Zero-shot question generation from knowledge graphs for unseen predicates and entity types. NAACL-HLT, abs/1802.0, 218–228. https://doi.org/10.18653/v1/n18-1020.

Emu, I. H., Ahmed, A. U., Islam, M., Al Mamun, S., & Uddin, A. (2017). An efficient approach for Keyphrase extraction from English document. International Journal of Intelligent Systems and Applications, 9(12), 59–66. https://doi.org/10.5815/ijisa.2017.12.06.CrossRef

Gan, W., & Yu, X. (2018). Automatic understanding and formalization of natural language geometry problems using syntax-semantics models. ICIC International, 14(1), 83–98.

Gleason, N. W. (2018). Higher education in the era of the fourth industrial revolution. https://doi.org/10.1007/978-981-13-0194-0.CrossRef

Graesser, A. C., Rus, V., Cai, Z., & Hu, X. (2009). Question answering and generation. Applied Natural Language Processing: Identification, Investigation and Resolution, 1, 1–16. https://doi.org/10.4018/978-1-60960-741-8.ch001.CrossRef

Han, H., Yang, F., Huang, J., & Zhou, B. (2018). CFXGBoost: Topic phrase extraction based on context features and XGBoost for knowledge base question answering. In ICNC-FSKD 2017 - 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (pp. 2397–2402). IEEE. https://doi.org/10.1109/FSKD.2017.8393148.

Heilman, M., & Smith, N. A. (2010). Good question! Statistical ranking for question generation. NAACL HLT 2010 - Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference, 609–617.

Heilman, M., Aleven, V., Cohen, W. W., Litman, D. J., & Smith, N. A. (2011). Automatic factual question generation from text. In Dissertation (p. 203). Retrieved from www.lti.cs.cmu.edu

Herranen, J., & Aksela, M. (2019). Student-question-based inquiry in science education. Studies in Science Education, 55(1), 1–36. https://doi.org/10.1080/03057267.2019.1658059.CrossRef

Hopcroft, J. E., Motwani, R., & Ullman, J. D. (2001). Introduction to automata theory, languages, and computation, 2nd edition. ACM SIGACT News, 32, 60–65. https://doi.org/10.1145/568438.568455.CrossRefMATH

Huang, Y.-T., Chen, M. C., & Sun, Y. S. (2018). Bringing personalized learning into computer-aided question generation. CoRR, abs/1808.0. Retrieved from http://arxiv.org/abs/1808.09735

Jouault, C., Seta, K., & Hayashi, Y. (2016). Content-dependent question generation using LOD for history learning in open learning space. New Generation Computing, 34(4), 367–394. https://doi.org/10.1007/s00354-016-0404-x.CrossRef

Kale, V. M., & Kiwelekar, A. W. (2013). An algorithm for question paper template generation in question paper generation system. In 2013 The International Conference on Technological Advances in Electrical, Electronics and Computer Engineering, TAEECE 2013 (pp. 256–261). https://doi.org/10.1109/TAEECE.2013.6557281.

Kumar, V., Ramakrishnan, G., & Li, Y.-F. (2018). A framework for automatic question generation from text using deep reinforcement learning. CoRR, abs/1808.0. Retrieved from http://arxiv.org/abs/1808.04961

Kurdi, G., Leo, J., Parsia, B., Sattler, U., & Al-Emari, S. (2019). A systematic review of automatic question generation for educational purposes. International Journal of Artificial Intelligence in Education., 30, 121–204. https://doi.org/10.1007/s40593-019-00186-y.CrossRef

Li, J., Gao, Y., Bing, L., King, I., & Lyu, M. R. (2019). Improving question generation with to the point context. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 3214–3224). Stroudsburg, PA, USA: Association for Computational Linguistics. https://doi.org/10.18653/v1/D19-1317.

Lindberg, D., & Sc, B. (2013). Automatic question generation from text for self-directed learning by.

Mandal, S., Mahata, S. K., & Das, D. (2018). Preparing Bengali-English Code-Mixed Corpus for Sentiment Analysis of Indian Languages. Retrieved from http://arxiv.org/abs/1803.04000

Meng, R., Zhao, S., Han, S., He, D., Brusilovsky, P., & Chi, Y. (2017). Deep Keyphrase generation. CoRR, abs/1704.0, 582–592. https://doi.org/10.18653/v1/P17-1054.

Mikolov, T., Chen, K., Corrado, G. S., Dean, J., Sutskever, I., Chen, K., … Dean, J. (2013). Distributed representations of words and phrases and their compositionality.

Moraes, S., Godbole, A., & Gharpure, P. (2018). Affinity analysis for context-free grammars. In IEEE International Conference on Power, Control, Signals and Instrumentation Engineering, ICPCSI 2017 (pp. 2471–2474). IEEE. https://doi.org/10.1109/ICPCSI.2017.8392161.

Nema, P., Mohankumar, A. K., Khapra, M. M., Srinivasan, B. V., & Ravindran, B. (2019). Let’s ask again: Refine network for automatic question generation. Retrieved from http://arxiv.org/abs/1909.05355

Quintana, R. M., & Tan, Y. (2019). Characterizing MOOC pedagogies: Exploring tools and methods for learning designers and researchers. Online Learning Journal, 23(4), 62–84. https://doi.org/10.24059/olj.v23i4.2084.

Rodrigues, H. P. (2017). Learning semantic patterns for question generation and question answering.

Rus, V., Graesser, A., & Cai, Z. (2008). Question generation: Example of a multi-year evaluation campaign. Workshop on the Question Generation Shared Task and Evaluation Challenge, (January). Retrieved from http://www.cs.memphis.edu/~vrus/questiongeneration/5-RusEtAl-QG08.pdf

Singh, S., Tiwari, S., Varshney, A., & Sharma, A. (2017). Unsupervised key-phrase extraction using noun phrases. International Journal of Computer Applications, 162(1), 1–5. https://doi.org/10.5120/ijca2017913171.CrossRef

Subramanian, S., Wang, T., Yuan, X., Zhang, S., Trischler, A., & Bengio, Y. (2018). Neural models for key phrase extraction and question generation. In QA@ACL (pp. 78–88). Association for Computational Linguistics. https://doi.org/10.18653/v1/w18-2609

Tong, P., Zhang, Q., & Yao, J. (2019). Leveraging domain context for question answering over knowledge graph. Data Science and Engineering, 4(4), 323–335. https://doi.org/10.1007/s41019-019-00109-w.CrossRef

Turney, P. D. (2003). Coherent Keyphrase extraction via web mining. In IJCAI International Joint Conference on Artificial Intelligence.

Willis, A., Davis, G., Ruan, S., Manoharan, L., Landay, J., & Brunskill, E. (2019). Key phrase extraction for generating educational question-answer pairs. In Proceedings of the 6th 2019 ACM conference on learning at scale, L@S 2019. https://doi.org/10.1145/3330430.3333636.

Xie, Z., Zeng, Z., Zhou, G., & Wang, W. (2017). Topic enhanced deep structured semantic models for knowledge base question answering. SCIENCE CHINA Information Sciences, 60(11). https://doi.org/10.1007/s11432-017-9136-x.

Yang, Z., Hu, J., Salakhutdinov, R., & Cohen, W. W. (2017). Semi-supervised QA with generative domain-adaptive nets. In ACL 2017 - 55th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers). https://doi.org/10.18653/v1/P17-1096.

Yao, X., Bouma, G., Zhang, Y., Piwek, P., & Boyer, K. E. (2012). Semantic-based question generation and implementation. Dialogue & Discourse, 3(2), 11–42. https://doi.org/10.5087/dad.2012.202.

Ye, H., & Wang, L. (2018). Semi-supervised learning for Neural Keyphrase generation. Retrieved from http://arxiv.org/abs/1808.06773

Title: Question generation model based on key-phrase, context-free grammar, and Bloom’s taxonomy
Authors: Bambang Dwi Wijanarko
Yaya Heryadi
Hapnes Toba
Widodo Budiharto
Publication date: 10-10-2020
Publisher: Springer US
Published in: Education and Information Technologies / Issue 2/2021
Print ISSN: 1360-2357
Electronic ISSN: 1573-7608
DOI: https://doi.org/10.1007/s10639-020-10356-4

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 2/2021

‘Dreaming in colour’: disabled higher education students’ perspectives on improving design practices that would enable them to benefit from their use of technologies

Acknowledgment to Reviewers

Dimensions of robotic education quality: teachers’ perspectives as teaching assistants in Thai elementary schools

Playfulness in the classroom: Gamification favor the learning of pharmacology

The effects of robotics training on students’ creativity and learning in physics

A literature review: efficacy of online learning courses for higher education institution using meta-analysis

Premium Partner