Skip to main content

2019 | OriginalPaper | Buchkapitel

An Annotation Protocol for Collecting User-Generated Counter-Arguments Using Crowdsourcing

verfasst von : Paul Reisert, Gisela Vallejo, Naoya Inoue, Iryna Gurevych, Kentaro Inui

Erschienen in: Artificial Intelligence in Education

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Constructive feedback is important for improving critical thinking skills. However, little work has been done to automatically generate such feedback for an argument. In this work, we experiment with an annotation protocol for collecting user-generated counter-arguments via crowdsourcing. We conduct two parallel crowdsourcing experiments, where workers are instructed to produce (i) a counter-argument, and (ii) a counter-argument after identifying a fallacy. Our analysis indicates that we can collect counter-arguments that are useful as constructive feedback, especially when workers are first asked to identify a fallacy type.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat El Khoiri, N., Widiati, U.: Logical fallacies in Indonesian EFL learners’ argumentative writing: students’ perspectives. Dinamika Ilmu 17(1), 71–81 (2017)CrossRef El Khoiri, N., Widiati, U.: Logical fallacies in Indonesian EFL learners’ argumentative writing: students’ perspectives. Dinamika Ilmu 17(1), 71–81 (2017)CrossRef
2.
Zurück zum Zitat Ghosh, D., Khanam, A., Han, Y., Muresan, S.: Coarse-grained argumentation features for scoring persuasive essays. In: Proceedings of the 54th Annual Meeting of ACL (Volume 2: Short Papers), pp. 549–554 (2016) Ghosh, D., Khanam, A., Han, Y., Muresan, S.: Coarse-grained argumentation features for scoring persuasive essays. In: Proceedings of the 54th Annual Meeting of ACL (Volume 2: Short Papers), pp. 549–554 (2016)
3.
Zurück zum Zitat Habernal, I., Pauli, P., Gurevych, I.: Adapting serious game for fallacious argumentation to German: pitfalls, insights, and best practices. In: Proceedings of the Eleventh International Conference on LREC, pp. 3329–3335 (2018) Habernal, I., Pauli, P., Gurevych, I.: Adapting serious game for fallacious argumentation to German: pitfalls, insights, and best practices. In: Proceedings of the Eleventh International Conference on LREC, pp. 3329–3335 (2018)
4.
Zurück zum Zitat Habernal, I., Wachsmuth, H., Gurevych, I., Stein, B.: The argument reasoning comprehension task: identification and reconstruction of implicit warrants. In: Proceedings of the 2018 Conference of NAACL: HLT, Volume 1 (Long Papers), pp. 1930–1940. Association for Computational Linguistics (2018) Habernal, I., Wachsmuth, H., Gurevych, I., Stein, B.: The argument reasoning comprehension task: identification and reconstruction of implicit warrants. In: Proceedings of the 2018 Conference of NAACL: HLT, Volume 1 (Long Papers), pp. 1930–1940. Association for Computational Linguistics (2018)
5.
Zurück zum Zitat Hua, X., Wang, L.: Neural argument generation augmented with externally retrieved evidence. In: Proceedings of the 56th Annual Meeting of ACL (Volume 1: Long Papers), pp. 219–230 (2018) Hua, X., Wang, L.: Neural argument generation augmented with externally retrieved evidence. In: Proceedings of the 56th Annual Meeting of ACL (Volume 1: Long Papers), pp. 219–230 (2018)
6.
Zurück zum Zitat Indah, R.N., Kusuma, A.W.: Fallacies in English department students’ claims: a rhetorical analysis of critical thinking. Jurnal Pendidikan Humaniora 3(4), 295–304 (2015) Indah, R.N., Kusuma, A.W.: Fallacies in English department students’ claims: a rhetorical analysis of critical thinking. Jurnal Pendidikan Humaniora 3(4), 295–304 (2015)
7.
Zurück zum Zitat Lucas, C., Gibson, A., Buckingham Shum, S.: Utilization of a novel online reflective learning tool for immediate formative feedback to assist pharmacy students’ reflective writing skills. Am. J. Pharm. Educ. (2018) Lucas, C., Gibson, A., Buckingham Shum, S.: Utilization of a novel online reflective learning tool for immediate formative feedback to assist pharmacy students’ reflective writing skills. Am. J. Pharm. Educ. (2018)
8.
Zurück zum Zitat Nguyen, H.V., Litman, D.J.: Argument mining for improving the automated scoring of persuasive essays. In: The Thirty-Second AAAI Conference on Artificial Intelligence, pp. 5892–5899 (2018) Nguyen, H.V., Litman, D.J.: Argument mining for improving the automated scoring of persuasive essays. In: The Thirty-Second AAAI Conference on Artificial Intelligence, pp. 5892–5899 (2018)
9.
Zurück zum Zitat Oktavia, W., Yasin, A., et al.: An analysis of students’ argumentative elements and fallacies in students’ discussion essays. Engl. Lang. Teach. 2(3) (2014) Oktavia, W., Yasin, A., et al.: An analysis of students’ argumentative elements and fallacies in students’ discussion essays. Engl. Lang. Teach. 2(3) (2014)
10.
Zurück zum Zitat Persing, I., Davis, A., Ng, V.: Modeling organization in student essays. In: Proceedings of the 2010 Conference on EMNLP, pp. 229–239. Association for Computational Linguistics (2010) Persing, I., Davis, A., Ng, V.: Modeling organization in student essays. In: Proceedings of the 2010 Conference on EMNLP, pp. 229–239. Association for Computational Linguistics (2010)
11.
Zurück zum Zitat Persing, I., Ng, V.: Modeling thesis clarity in student essays. In: Proceedings of the 51st Annual Meeting of ACL (Volume 1: Long Papers), vol. 1, pp. 260–269 (2013) Persing, I., Ng, V.: Modeling thesis clarity in student essays. In: Proceedings of the 51st Annual Meeting of ACL (Volume 1: Long Papers), vol. 1, pp. 260–269 (2013)
12.
Zurück zum Zitat Persing, I., Ng, V.: Modeling stance in student essays. In: Proceedings of the 54th Annual Meeting of ACL (Volume 1: Long Papers), vol. 1, pp. 2174–2184 (2016) Persing, I., Ng, V.: Modeling stance in student essays. In: Proceedings of the 54th Annual Meeting of ACL (Volume 1: Long Papers), vol. 1, pp. 2174–2184 (2016)
13.
Zurück zum Zitat Wachsmuth, H., Al-Khatib, K., Stein, B.: Using argument mining to assess the argumentation quality of essays. In: Proceedings of the 26th International Conference on COLING, pp. 1680–1692 (2016) Wachsmuth, H., Al-Khatib, K., Stein, B.: Using argument mining to assess the argumentation quality of essays. In: Proceedings of the 26th International Conference on COLING, pp. 1680–1692 (2016)
14.
Zurück zum Zitat Wachsmuth, H., Syed, S., Stein, B.: Retrieval of the best counterargument without prior topic knowledge. In: Proceedings of the 56th Annual Meeting of ACL (Volume 1: Long Papers), vol. 1, pp. 241–251 (2018) Wachsmuth, H., Syed, S., Stein, B.: Retrieval of the best counterargument without prior topic knowledge. In: Proceedings of the 56th Annual Meeting of ACL (Volume 1: Long Papers), vol. 1, pp. 241–251 (2018)
Metadaten
Titel
An Annotation Protocol for Collecting User-Generated Counter-Arguments Using Crowdsourcing
verfasst von
Paul Reisert
Gisela Vallejo
Naoya Inoue
Iryna Gurevych
Kentaro Inui
Copyright-Jahr
2019
DOI
https://doi.org/10.1007/978-3-030-23207-8_43

Premium Partner