Skip to main content
Erschienen in: Education and Information Technologies 6/2020

04.05.2020

Adopting computer-assisted assessment in evaluation of handwritten answer books: An experimental study

verfasst von: Sujan Kumar Saha, Rushali Gupta

Erschienen in: Education and Information Technologies | Ausgabe 6/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The use of computers in educational assessment is a widely explored territory. Several studies have been performed to show the effectiveness of computer-assisted assessment (CAA) and it has been accepted in various education sectors. However, due to the lack of sufficient infrastructure and other issues, the paper-based examination is still being used for educational assessment in many countries including India. Existing CAA frameworks require the examination to be conducted on a digital platform. So, these do not apply to paper answer books. We propose a two-phase framework for automatic evaluation of handwritten answer books. The first phase converts the answers written on papers to a digital form using a neural network-based handwritten answer recognizer. The second phase evaluates the answer to generate a numerical score. For this evaluation, we use a model answer-based approach where various levels of similarities between the model answer and a student answer are computed. To assess the performance of the developed system, we apply it to the evaluation of class VII Social Science Geography answer books of an Indian school. The experimental result shows that the proposed approach is quite promising.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Alikaniotis, D., Yannakoudakis, H., & Rei, M. (2016). Automatic text scoring using neural networks. CoRR, abs/1606.04289. Alikaniotis, D., Yannakoudakis, H., & Rei, M. (2016). Automatic text scoring using neural networks. CoRR, abs/1606.04289.
Zurück zum Zitat Baker, E., & Mayer, R. E. (1999). Computer-based assessment of problem solving. Computers in Human Behavior, 15(3), 269–282. Baker, E., & Mayer, R. E. (1999). Computer-based assessment of problem solving. Computers in Human Behavior, 15(3), 269–282.
Zurück zum Zitat Burrows, S., Gurevych, I., & Stein, B. (2015). The eras and trends of automatic short answer grading. International Journal of Artificial Intelligence in Education, 25(1), 60–117. Burrows, S., Gurevych, I., & Stein, B. (2015). The eras and trends of automatic short answer grading. International Journal of Artificial Intelligence in Education, 25(1), 60–117.
Zurück zum Zitat Burstein, J., Kukich, K., Wolff, S., Lu, C., & Chodorow, M. (1998). Enriching automated scoring using discourse marking. Proceedings of the Workshop on Discourse Relations and Discourse Marking in ACL-1998. Burstein, J., Kukich, K., Wolff, S., Lu, C., & Chodorow, M. (1998). Enriching automated scoring using discourse marking. Proceedings of the Workshop on Discourse Relations and Discourse Marking in ACL-1998.
Zurück zum Zitat Callear, D. H., Jerrams-Smith, J., & Soh, V. (2001). CAA of short non-MCQ answers. In Proceedings of the 5th International CAA conference. Callear, D. H., Jerrams-Smith, J., & Soh, V. (2001). CAA of short non-MCQ answers. In Proceedings of the 5th International CAA conference.
Zurück zum Zitat Cartwright, G. F., & Derevensky, J. L. (1975). An attitudinal study of computer-assisted testing as a learning method. Psychology in the Schools, 13(3), 317–321. Cartwright, G. F., & Derevensky, J. L. (1975). An attitudinal study of computer-assisted testing as a learning method. Psychology in the Schools, 13(3), 317–321.
Zurück zum Zitat Chapelle, C. A., & Voss, E. (2016). 20 years of technology and language assessment in. Language Learning & Technology, 20(2), 116–128. Chapelle, C. A., & Voss, E. (2016). 20 years of technology and language assessment in. Language Learning & Technology, 20(2), 116–128.
Zurück zum Zitat Clariana, R., & Wallace, P. (2002). Paper–based versus computer–based assessment: Key factors associated with the test mode effect. British Journal of Educational Technology, 33(5), 593–602. Clariana, R., & Wallace, P. (2002). Paper–based versus computer–based assessment: Key factors associated with the test mode effect. British Journal of Educational Technology, 33(5), 593–602.
Zurück zum Zitat Conneau, A., Douwe, K., Holger, S., Loic, B., & Bordes, A. 2017. Supervised learning of universal sentence representations from natural language inference data. Volume~1. arXiv preprint arXiv:1705.02364. Conneau, A., Douwe, K., Holger, S., Loic, B., & Bordes, A. 2017. Supervised learning of universal sentence representations from natural language inference data. Volume~1. arXiv preprint arXiv:1705.02364.
Zurück zum Zitat Croft, A. C., Danson, M., Dawson, B. R., & Ward, J. P. (2001). Experiences of using computer assisted assessment in engineering mathematics. Computers & Education, 37(1), 53–66. Croft, A. C., Danson, M., Dawson, B. R., & Ward, J. P. (2001). Experiences of using computer assisted assessment in engineering mathematics. Computers & Education, 37(1), 53–66.
Zurück zum Zitat Dumais, S. T. (2005). Latent semantic analysis. Annual Review of Information Science and Technology, 38, 188–230. Dumais, S. T. (2005). Latent semantic analysis. Annual Review of Information Science and Technology, 38, 188–230.
Zurück zum Zitat Dzikovska, M. O., Nielsen, R. D., Brew, C., Leacock, C., Giampiccolo, D., Bentivogli, L., ... & Dang, H. T. (2013). Semeval-2013 task 7: The joint student response analysis and 8th recognizing textual entailment challenge. In proceedings of Seventh International Workshop on Semantic Evaluation (SemEval 2013). Dzikovska, M. O., Nielsen, R. D., Brew, C., Leacock, C., Giampiccolo, D., Bentivogli, L., ... & Dang, H. T. (2013). Semeval-2013 task 7: The joint student response analysis and 8th recognizing textual entailment challenge. In proceedings of Seventh International Workshop on Semantic Evaluation (SemEval 2013).
Zurück zum Zitat Fisteus, J. A., Pardo, A., & García, N. F. (2013). Grading multiple choice exams with low-cost and portable computer-vision techniques. Journal of Science Education and Technology, 22(4), 560–571. Fisteus, J. A., Pardo, A., & García, N. F. (2013). Grading multiple choice exams with low-cost and portable computer-vision techniques. Journal of Science Education and Technology, 22(4), 560–571.
Zurück zum Zitat Foltz, P. W., Laham, D., & Landauer T. K. (1999). Automated essay scoring: Applications to educational technology. Proceedings of ED-MEDIA Conference on Educational Multimedia, Hypermedia, and Telecommunications. Foltz, P. W., Laham, D., & Landauer T. K. (1999). Automated essay scoring: Applications to educational technology. Proceedings of ED-MEDIA Conference on Educational Multimedia, Hypermedia, and Telecommunications.
Zurück zum Zitat Khoshsima, H., Hosseini, M., & Toroujeni, S. M. H. (2017). Cross-mode comparability of Computer-Based Testing (CBT) versus Paper-Pencil Based Testing (PPT): An investigation of testing administration mode among Iranian intermediate EFL learners. English Language Teaching, 10(2), 23–32. Khoshsima, H., Hosseini, M., & Toroujeni, S. M. H. (2017). Cross-mode comparability of Computer-Based Testing (CBT) versus Paper-Pencil Based Testing (PPT): An investigation of testing administration mode among Iranian intermediate EFL learners. English Language Teaching, 10(2), 23–32.
Zurück zum Zitat Leacock, C., & Chodorow, M. (2003). C-rater: Automated scoring of short-answer questions. Computers and the Humanities, 37(4), 389–405. Leacock, C., & Chodorow, M. (2003). C-rater: Automated scoring of short-answer questions. Computers and the Humanities, 37(4), 389–405.
Zurück zum Zitat Levenshtein, V. I. (1966). Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady, 10:707. Levenshtein, V. I. (1966). Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady, 10:707.
Zurück zum Zitat Manmatha, R., & Srimal, N. (1999). Scale space technique for word segmentation in handwritten documents. In International Conference on Scale-Space Theories in Computer Vision, pp. 22-33. Manmatha, R., & Srimal, N. (1999). Scale space technique for word segmentation in handwritten documents. In International Conference on Scale-Space Theories in Computer Vision, pp. 22-33.
Zurück zum Zitat Marti, U., & Bunke, H. (2002). The IAM-database: An English sentence database for off-line handwriting recognition. Int. Journal on Document Analysis & Recognition, 5(2002), 39–46.MATH Marti, U., & Bunke, H. (2002). The IAM-database: An English sentence database for off-line handwriting recognition. Int. Journal on Document Analysis & Recognition, 5(2002), 39–46.MATH
Zurück zum Zitat Miller, G. A. (1995). WordNet: A lexical database for English. Communications of the ACM, 38(11), 39–41. Miller, G. A. (1995). WordNet: A lexical database for English. Communications of the ACM, 38(11), 39–41.
Zurück zum Zitat Mitchell, T., Russell, T., Broomhead, P., & Aldridge, N. (2002). Towards robust computerised marking of free-text responses. Proceedings of the Sixth International Computer Assisted Assessment Conference, Loughboroug University, UK. Mitchell, T., Russell, T., Broomhead, P., & Aldridge, N. (2002). Towards robust computerised marking of free-text responses. Proceedings of the Sixth International Computer Assisted Assessment Conference, Loughboroug University, UK.
Zurück zum Zitat Mohler, M., & Mihalcea, R. 2009. Text-to-text semantic similarity for automatic short answer grading. Proceedings of EACL-2009, pp. 567-575. Mohler, M., & Mihalcea, R. 2009. Text-to-text semantic similarity for automatic short answer grading. Proceedings of EACL-2009, pp. 567-575.
Zurück zum Zitat Mohler, M., Bunescu, R., & Mihalcea, R. (2011). Learning to grade short answer questions using semantic similarity measures and dependency graph alignments. In D.Lin (Ed.), Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, vol. 1, pp. 752–762. Mohler, M., Bunescu, R., & Mihalcea, R. (2011). Learning to grade short answer questions using semantic similarity measures and dependency graph alignments. In D.Lin (Ed.), Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, vol. 1, pp. 752–762.
Zurück zum Zitat Moschitti, A. (2006). Making tree kernels practical for natural language learning. In 11th conference of European Chapter of the Association for Computational Linguistics, pp. 113–120. Moschitti, A. (2006). Making tree kernels practical for natural language learning. In 11th conference of European Chapter of the Association for Computational Linguistics, pp. 113–120.
Zurück zum Zitat Nikou, S. A., & Economides, A. A. (2016). The impact of paper-based, computer-based and mobile-based self-assessment on students’ science motivation and achievement. Computers in Human Behavior, 55(2016), 1241–1248. Nikou, S. A., & Economides, A. A. (2016). The impact of paper-based, computer-based and mobile-based self-assessment on students’ science motivation and achievement. Computers in Human Behavior, 55(2016), 1241–1248.
Zurück zum Zitat Page, E. B. (1968). The use of the computer in analyzing student essays. International Review of Education, 14(3), 253–263. Page, E. B. (1968). The use of the computer in analyzing student essays. International Review of Education, 14(3), 253–263.
Zurück zum Zitat Paiva, R. C., Ferreira, M. S., Mendes, A. G., & Eusébio, A. M. J. (2015). Interactive and multimedia contents associated with a system for computer-aided assessment. Journal of Educational Computing Research, 52(2), 224–256. Paiva, R. C., Ferreira, M. S., Mendes, A. G., & Eusébio, A. M. J. (2015). Interactive and multimedia contents associated with a system for computer-aided assessment. Journal of Educational Computing Research, 52(2), 224–256.
Zurück zum Zitat Sangwin, C. (2015) Computer aided assessment of mathematics using STACK. In: Cho S. (eds) Selected regular lectures from the 12th International Congress on Mathematical Education. Sangwin, C. (2015) Computer aided assessment of mathematics using STACK. In: Cho S. (eds) Selected regular lectures from the 12th International Congress on Mathematical Education.
Zurück zum Zitat Sangwin, C., & Kocher, N. (2016). Automation of mathematics examinations. Computers & Education, 94, 215–227. Sangwin, C., & Kocher, N. (2016). Automation of mathematics examinations. Computers & Education, 94, 215–227.
Zurück zum Zitat Scheidl, H. (2018), Handwritten text recognition in historical documents. Diploma Thesis, Technische Universitat Wien. Scheidl, H. (2018), Handwritten text recognition in historical documents. Diploma Thesis, Technische Universitat Wien.
Zurück zum Zitat Shute, V. J., & Rahimi, S. (2017). Review of computer-based assessment for learning in elementary and secondary education. Journal of Computer Assisted Learning, 33(2017), 1–19. Shute, V. J., & Rahimi, S. (2017). Review of computer-based assessment for learning in elementary and secondary education. Journal of Computer Assisted Learning, 33(2017), 1–19.
Zurück zum Zitat Sim, G., Holifield, P., & Brown, M. (2004). Implementation of computer assisted assessment: Lessons from the literature. Research in Learning Technology, 12(3), 215–229. Sim, G., Holifield, P., & Brown, M. (2004). Implementation of computer assisted assessment: Lessons from the literature. Research in Learning Technology, 12(3), 215–229.
Zurück zum Zitat Smith, A. M. (1981) Optical Mark Reading - making it easy for users, In Proceedings of the 9th annual ACM SIGUCCS Conference on User services, pp: 257–263. Smith, A. M. (1981) Optical Mark Reading - making it easy for users, In Proceedings of the 9th annual ACM SIGUCCS Conference on User services, pp: 257–263.
Zurück zum Zitat Tandalla, L. (2012). Scoring short answer essays. ASAP ‘12 SAS Methodology Paper. Tandalla, L. (2012). Scoring short answer essays. ASAP ‘12 SAS Methodology Paper.
Metadaten
Titel
Adopting computer-assisted assessment in evaluation of handwritten answer books: An experimental study
verfasst von
Sujan Kumar Saha
Rushali Gupta
Publikationsdatum
04.05.2020
Verlag
Springer US
Erschienen in
Education and Information Technologies / Ausgabe 6/2020
Print ISSN: 1360-2357
Elektronische ISSN: 1573-7608
DOI
https://doi.org/10.1007/s10639-020-10192-6

Weitere Artikel der Ausgabe 6/2020

Education and Information Technologies 6/2020 Zur Ausgabe