nach oben

Erschienen in:

2020 | OriginalPaper | Buchkapitel

Innovative Deep Neural Network Fusion for Pairwise Translation Evaluation

verfasst von : Despoina Mouratidis, Katia Lida Kermanidis, Vilelmini Sosoni

Erschienen in: Artificial Intelligence Applications and Innovations

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

A language independent deep learning (DL) architecture for machine translation (MT) evaluation is presented. This DL architecture aims at the best choice between two MT (S1, S2) outputs, based on the reference translation (Sr) and the annotation score. The outputs were generated from a statistical machine translation (SMT) system and a neural machine translation (NMT) system. The model applied in two language pairs: English - Greek (EN-EL) and English - Italian (EN-IT). In this paper, a variety of experiments with different parameter configurations is presented. Moreover, linguistic features, embeddings representation and natural language processing (NLP) metrics (BLEU, METEOR, TER, WER) were tested. The best score was achieved when the proposed model used source segments (SSE) information and the NLP metrics set. Classification accuracy has increased up to 5% (compared to previous related work) and reached quite satisfactory results for the Kendall τ score.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Deepbots: A Webots-Based Deep Reinforcement Learning Framework for Robotics

Nächstes Kapitel Introducing an Edge-Native Deep Learning Platform for Exergames

Abadi, M., et al.: Tensorflow: a system for large-scale machine learning. In: 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 2016), USA, pp. 265–283. USENIX Association (2016)

Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: Proceedings of 3th International Conference on Learning Representations, San Diego, pp. 1–15. ICLR (2015)

Barrón-Cedeño, A., Màrquez Villodre, L., Henríquez Quintana, C.A., Formiga Fanals, L., Romero Merino, E., May, J.: Identifying useful human correction feedback from an on-line machine translation service. In: Proceedings of 23rd International Joint Conference on Artificial Intelligence, Beijing, pp. 2057–2063. AAAI Press (2013)

Denkowski, M., Lavie, A.: Meteor universal: language specific translation evaluation for any target language. In: Proceedings of the 9th Workshop on Statistical Machine Translation, Baltimore, Maryland, USA, pp. 376–380. ACL (2014)

Duh, K.: Ranking vs. regression in machine translation evaluation. In: Proceedings of the 3rd Workshop on Statistical Machine Translation, Columbus, Ohio, pp. 191–194. ACL (2008)

GitHub. https://github.com/gcunhase/NLPMetrics. Accessed 20 Feb 2020

Guzmán, F., Joty, S., Màrquez, L., Nakov, P.: Pairwise neural machine translation evaluation. arXiv preprint arXiv:1912.03135. In: Proceedings of the 53^rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China, pp. 805–814. ACL (2015)

Jaitly, N., Sussillo, D., Le, Q.V., Vinyals, O., Sutskever, I., Bengio, S.: A neural transducer. Cornell University Library. arXiv preprint arXiv:1511.04868 (2015)

Kendall, M.: A new measure of rank correlation. Biometrika 30(1/2), 81–93 (1938)CrossRef

10.

Keras: Deep learning library for theano and tensorflow. https://keras.io/k7.8. Accessed 20 Feb 2020

11.

Koehn, P., et al.: Moses: open source toolkit for statistical machine translation. In: Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, Prague, pp. 177–180. ACL (2007)

12.

Kordoni, V., et al.: TraMOOC (translation for massive open online courses): providing reliable MT for MOOCs. In: Proceedings of the 19th Annual Conference of the European Association for Machine Translation (EAMT), Riga, pp. 376–400. European Association for Machine Translation (EAMT) (2016)

13.

Loper, E., Bird, S.: NLTK: the natural language toolkit. In: Proceedings of the ACL 2002 Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics, USA, pp. 63–70. ACL (2002)

14.

Luong, M.T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, pp. 1412–1421. ACL (2015)

15.

Mouratidis, D., Kermanidis, K.L.: Automatic selection of parallel data for machine translation. In: Iliadis, L., Maglogiannis, I., Plagianakos, V. (eds.) AIAI 2018. IAICT, vol. 520, pp. 146–156. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-92016-0_14CrossRef

16.

Mouratidis, D., Kermanidis, K.L.: Comparing a hand-crafted to an automatically generated feature set for deep learning: pairwise translation evaluation. In: 2nd Workshop on Human-Informed Translation and Interpreting Technology, Varna, Bulgaria, pp. 66–74. HiT-IT (2019)

17.

Mouratidis, D., Kermanidis, K.L.: Ensemble and deep learning for language-independent automatic selection of parallel data. Algorithms 12(1), 12–26 (2019)CrossRef

18.

Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, Philadelphia, pp. 311–318. Association for Computational Linguistics (2002)

19.

Pedregosa, F., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetMATH

20.

Peris, Á., Cebrián, L., Casacuberta, F.: Online learning for neural machine translation post-editing. Cornell University Library. arXiv preprint 1, pp. 1–12. arXiv:1706.03196 (2017)

21.

Pouliquen, B., Steinberger, R., Ignat, C.: Automatic identification of document translations in large multilingual document collections. In: Proceedings of the International Conference Recent Advances in Natural Language Processing (RANLP), Borovets, pp. 401–408. Recent Advances in Natural Language Processing (RANLP) (2003)

22.

Sennrich, R., et al.: Nematus: a toolkit for neural machine translation. In: Proceedings of the EACL 2017 Software Demonstrations, Valencia, pp. 65–68. ACL (2017)

23.

Singhal, S., Jena, M.: A study on WEKA tool for data preprocessing, classification and clustering. Int. J. Innov. Technol. Explor. Eng. (IJITEE) 2(6), 250–253 (2013)

24.

Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J.: A study of translation edit rate with targeted human annotation. In: Proceedings of the 7th Conference of the Association for Machine Translation in the Americas, Cambridge, pp. 223–231. The Association for Machine Translation in the Americas (2006)

25.

Su, K.Y., Wu, M.W., Chang, J.S.: A new quantitative quality measure for machine translation systems. In: Proceedings of the 14th Conference on Computational Linguistics, Nantes, France, vol. 2, pp. 433–439. Association for Computational Linguistics (1992)

26.

Vaswani, A., et al.: Attention is all you need. In: 31st Conference on Neural Information Processing Systems, Long Beach, CA, USA, pp. 5998–6008. NIPS (2017)

27.

Sosoni, V., et al.: Translation crowdsourcing: creating a multilingual corpus of online educational content. In: Proceedings of the 11th International Conference on Language Resources and Evaluation, Japan, pp. 479–483. European Language Resources Association (2018)

28.

Wu, Y., et al.: Google’s neural machine translation system: bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016)

Titel: Innovative Deep Neural Network Fusion for Pairwise Translation Evaluation
verfasst von: Despoina Mouratidis
Katia Lida Kermanidis
Vilelmini Sosoni
Verlag: Springer International Publishing
Buch: Artificial Intelligence Applications and Innovations
Print ISBN: 978-3-030-49185-7

Electronic ISBN: 978-3-030-49186-4

Copyright-Jahr: 2020
DOI: https://doi.org/10.1007/978-3-030-49186-4_7

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner