Skip to main content
Erschienen in: Education and Information Technologies 6/2018

19.06.2018

Short answer scoring system using automatic reference answer generation and geometric average normalized-longest common subsequence (GAN-LCS)

verfasst von: Feddy Setio Pribadi, Adhistya Erna Permanasari, Teguh Bharata Adji

Erschienen in: Education and Information Technologies | Ausgabe 6/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The Automatic Short Answer Scoring (ASAS) system is one of the tools that can be used to conduct assessment process on e-learning system. One of the methods applied in the ASAS system is a method for measuring similarities between the reference and student answers. There are two issues to be considered in the assessment process using this method. First, this method should be able to provide a variety of reference answers that can handle the diversity of student answers. Secondly, this method should be able to provide an accurate sentence similarity between the reference answers and student answers. Therefore, two methods are proposed to solve both problems. The first method is to generate a variety of reference answers automatically using Maximum Marginal Relevance (MMR) method, which obtains an accuracy of 91.95%. The second method is to measure accurately sentence similarity between student answers and reference answers that have significantly different length using GAN-LCS. The performance of the proposed method shows an improvement of the Root Mean Square Error (RMSE) value of 0.884 and a correlation value of 0.468.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Adhitia, R., Purwarianti, A., & Bandung, I. T. (2009). Automated essay grading system using SVM and LSA for essay answers in Indonesian. Journal of Information Systems, 5(1), 33–41.CrossRef Adhitia, R., Purwarianti, A., & Bandung, I. T. (2009). Automated essay grading system using SVM and LSA for essay answers in Indonesian. Journal of Information Systems, 5(1), 33–41.CrossRef
Zurück zum Zitat Bachman, L. F., et al. (2002). A reliable approach to automatic assessment of short answer free responses. Proceedings of the 19th International Conference on Computational Linguistics, 2, 1–4.CrossRef Bachman, L. F., et al. (2002). A reliable approach to automatic assessment of short answer free responses. Proceedings of the 19th International Conference on Computational Linguistics, 2, 1–4.CrossRef
Zurück zum Zitat Burrows, S., Gurevych, I., & Stein, B. (2015). The eras and trends of automatic short answer grading. International Journal of Artificial Intelligence in Education, 25(1), 60–117.CrossRef Burrows, S., Gurevych, I., & Stein, B. (2015). The eras and trends of automatic short answer grading. International Journal of Artificial Intelligence in Education, 25(1), 60–117.CrossRef
Zurück zum Zitat Carbonell, J., & Goldstein, J. (1998). The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR ‘98 (pp. 335–336).CrossRef Carbonell, J., & Goldstein, J. (1998). The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR ‘98 (pp. 335–336).CrossRef
Zurück zum Zitat Evans, J. D. (1996). Straightforward statistics for the behavioral sciences. Pacific Grove: Brooks/Cole Pub. Co. Evans, J. D. (1996). Straightforward statistics for the behavioral sciences. Pacific Grove: Brooks/Cole Pub. Co.
Zurück zum Zitat Gomaa, W. H., & Fahmy, A. A. (2012). Short answer grading using string similarity and corpus-based similarity. International Journal of Advanced Computer Science and Applications, 3(11), 115–121. Gomaa, W. H., & Fahmy, A. A. (2012). Short answer grading using string similarity and corpus-based similarity. International Journal of Advanced Computer Science and Applications, 3(11), 115–121.
Zurück zum Zitat Jayashankar, S., & Sridaran, R. (2017). Superlative model using word cloud for short answers evaluation in eLearning. Education and Information Technologies, 22(5), 2383–2402.CrossRef Jayashankar, S., & Sridaran, R. (2017). Superlative model using word cloud for short answers evaluation in eLearning. Education and Information Technologies, 22(5), 2383–2402.CrossRef
Zurück zum Zitat Klein, R., Kyrilov, A., & Tokman, M. (2011). Automated assessment of short free-text responses in computer science using latent semantic analysis. In Proceedings of the 16th annual joint conference on Innovation and technology in computer science education (pp. 158–162). Klein, R., Kyrilov, A., & Tokman, M. (2011). Automated assessment of short free-text responses in computer science using latent semantic analysis. In Proceedings of the 16th annual joint conference on Innovation and technology in computer science education (pp. 158–162).
Zurück zum Zitat Leacock, C., & Chodorow, M. (2003). C-rater: automated scoring of short-answer questions. Computers and the Humanities, 37(4), 389–405.CrossRef Leacock, C., & Chodorow, M. (2003). C-rater: automated scoring of short-answer questions. Computers and the Humanities, 37(4), 389–405.CrossRef
Zurück zum Zitat Mohler, M., & Mihalcea, R. (2009). Text-to-text semantic similarity for automatic short answer grading. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics on - EACL ‘09 (pp. 567–575). Mohler, M., & Mihalcea, R. (2009). Text-to-text semantic similarity for automatic short answer grading. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics on - EACL ‘09 (pp. 567–575).
Zurück zum Zitat Mohler, M., Bunescu, R., & Mihalcea, R. (2011). Learning to grade short answer questions using semantic similarity measures and dependency graph alignments. In ACL-HLT 2011 - Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (pp. 752–762). Mohler, M., Bunescu, R., & Mihalcea, R. (2011). Learning to grade short answer questions using semantic similarity measures and dependency graph alignments. In ACL-HLT 2011 - Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (pp. 752–762).
Zurück zum Zitat Noorbehbahani, F., & Kardan, A. A. (2011). The automatic assessment of free text answers using a modified BLEU algorithm. Computers & Education, 56(2), 337–345.CrossRef Noorbehbahani, F., & Kardan, A. A. (2011). The automatic assessment of free text answers using a modified BLEU algorithm. Computers & Education, 56(2), 337–345.CrossRef
Zurück zum Zitat Pérez, D., & Alfonseca, E. (2005). Application of the Bleu algorithm for recognising textual entailments. In Workshop Recognising Textual Entailment (pp. 1–4). Pérez, D., & Alfonseca, E. (2005). Application of the Bleu algorithm for recognising textual entailments. In Workshop Recognising Textual Entailment (pp. 1–4).
Zurück zum Zitat Rodrigues, F., & Araújo, L. (2012). Automatic assessment of short free text answers. In CSEDU 2012 - Proceedings of the 4th International Conference on Computer Supported Education (pp. 50–57). Rodrigues, F., & Araújo, L. (2012). Automatic assessment of short free text answers. In CSEDU 2012 - Proceedings of the 4th International Conference on Computer Supported Education (pp. 50–57).
Zurück zum Zitat Senthil Kumaran, V., & Sankar, A. (2015). Towards an automated system for short-answer assessment using ontology mapping. International Arab Journal of e-Technology, 4, 17–25. Senthil Kumaran, V., & Sankar, A. (2015). Towards an automated system for short-answer assessment using ontology mapping. International Arab Journal of e-Technology, 4, 17–25.
Zurück zum Zitat Shermis, M. D. (2015). Contrasting state-of-the-art in the machine scoring of short-form constructed responses. Educational Assessment, 20(1), 46–65.CrossRef Shermis, M. D. (2015). Contrasting state-of-the-art in the machine scoring of short-form constructed responses. Educational Assessment, 20(1), 46–65.CrossRef
Zurück zum Zitat Siddiqi, R., Harrison, C. J., & Siddiqi, R. (2010). Improving teaching and learning through automated short-answer marking. IEEE Transactions on Learning Technologies, 3(3), 237–249.CrossRef Siddiqi, R., Harrison, C. J., & Siddiqi, R. (2010). Improving teaching and learning through automated short-answer marking. IEEE Transactions on Learning Technologies, 3(3), 237–249.CrossRef
Zurück zum Zitat Spiliotopoulou-Papantoniou, V. (2007). Models of the universe: children’s experiences and evidence from the history of science. Science Education, 16, 801–833.CrossRef Spiliotopoulou-Papantoniou, V. (2007). Models of the universe: children’s experiences and evidence from the history of science. Science Education, 16, 801–833.CrossRef
Zurück zum Zitat Sukkarieh, J., & Blackmore, J. (2009). c-rater: Automatic content scoring for short constructed responses. In Proceedings of the Twenty-Second International FLAIRS Conference (pp. 290–295). Sukkarieh, J., & Blackmore, J. (2009). c-rater: Automatic content scoring for short constructed responses. In Proceedings of the Twenty-Second International FLAIRS Conference (pp. 290–295).
Zurück zum Zitat Sultan, M. A., Salazar, C., & Sumner, T. (2016). Fast and easy short answer grading with high accuracy. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 1070–1075). Sultan, M. A., Salazar, C., & Sumner, T. (2016). Fast and easy short answer grading with high accuracy. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 1070–1075).
Zurück zum Zitat Wolska, M., Horbach, A., & Palmer, A. (2014). Computer-assisted scoring of short responses: the efficiency of a clustering-based approach in a real-life task. Advances in Natural Language Processing, 298–310. Wolska, M., Horbach, A., & Palmer, A. (2014). Computer-assisted scoring of short responses: the efficiency of a clustering-based approach in a real-life task. Advances in Natural Language Processing, 298–310.
Zurück zum Zitat Xi, Y., & Liang, W. (2011). Automated computer-based CET4 essay scoring system. In Proceedings - PACCS 2011: 2011 3rd Pacific-Asia Conference on Circuits, Communications and System. Xi, Y., & Liang, W. (2011). Automated computer-based CET4 essay scoring system. In Proceedings - PACCS 2011: 2011 3rd Pacific-Asia Conference on Circuits, Communications and System.
Zurück zum Zitat Ziai, R., Ott, N., & Meurers, D. (2012). Short answer assessment: Establishing links between research strands. In Proceedings of the Seventh Workshop on Building Educational Applications Using NLP (pp. 190–200). Ziai, R., Ott, N., & Meurers, D. (2012). Short answer assessment: Establishing links between research strands. In Proceedings of the Seventh Workshop on Building Educational Applications Using NLP (pp. 190–200).
Metadaten
Titel
Short answer scoring system using automatic reference answer generation and geometric average normalized-longest common subsequence (GAN-LCS)
verfasst von
Feddy Setio Pribadi
Adhistya Erna Permanasari
Teguh Bharata Adji
Publikationsdatum
19.06.2018
Verlag
Springer US
Erschienen in
Education and Information Technologies / Ausgabe 6/2018
Print ISSN: 1360-2357
Elektronische ISSN: 1573-7608
DOI
https://doi.org/10.1007/s10639-018-9745-z

Weitere Artikel der Ausgabe 6/2018

Education and Information Technologies 6/2018 Zur Ausgabe