Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Coreference Resolution: Are the Eliminated Spans Totally Worthless? Read chapter Read first chapter

2021 | OriginalPaper | Chapter

Learning to Select Relevant Knowledge for Neural Machine Translation

Authors : Jian Yang, Juncheng Wan, Shuming Ma, Haoyang Huang, Dongdong Zhang, Yong Yu, Zhoujun Li, Furu Wei

Published in: Natural Language Processing and Chinese Computing

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Most memory-based methods use encoded retrieved pairs as the translation memory (TM) to provide external guidance, but there still exist some noisy words in the retrieved pairs. In this paper, we propose a simple and effective end-to-end model to select useful sentence words from the encoded memory and incorporate them into the NMT model. Our model uses a novel memory selection mechanism to avoid the noise from similar sentences and provide external guidance simultaneously. To verify the positive influence of selected retrieved words, we evaluate our model on the single-domain dataset namely JRC-Acquis and multi-domain dataset comprised of existing benchmarks including WMT, IWSLT, JRC-Acquis, and OpenSubtitles. Experimental results demonstrate our method can improve the translation quality under different scenarios.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter CTRD: A Chinese Theme-Rheme Discourse Dataset
next chapter Contrastive Learning for Machine Translation Quality Estimation
Footnotes
Literature
1.
Akoury, N., Krishna, K., Iyyer, M.: Syntactically supervised transformers for faster neural machine translation. In: ACL 2019, pp. 1269–1281 (2019)
2.
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR 2015 (2015)
3.
Bapna, A., Firat, O.: Non-parametric adaptation for neural machine translation. In: NAACL 2019, pp. 1921–1931 (2019)
4.
5.
Bulté, B., Tezcan, A.: Neural fuzzy repair: integrating fuzzy matches into neural machine translation. In: ACL 2019, pp. 1800–1809 (2019)
6.
Cao, Q., Xiong, D.: Encoding gated translation memory into neural machine translation. In: EMNLP 2018, pp. 3042–3047 (2018)
7.
Dahlmann, L., Matusov, E., Petrushkov, P., Khadivi, S.: Neural machine translation leveraging phrase-based models in a hybrid search. In: EMNLP 2017, pp. 1411–1420 (2017)
8.
Farajian, M.A., Turchi, M., Negri, M., Federico, M.: Multi-domain neural machine translation through unsupervised adaptation. In: WMT 2017, pp. 127–137 (2017)
9.
Firat, O., Cho, K., Bengio, Y.: Multi-way, multilingual neural machine translation with a shared attention mechanism. In: NAACL 2016, pp. 866–875 (2016)
10.
Gehring, J., Auli, M., Grangier, D., Yarats, D., Dauphin, Y.N.: Convolutional sequence to sequence learning. In: ICML 2017, pp. 1243–1252 (2017)
11.
Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. CoRR arXiv:​1312.​6211 (2013)
12.
Gu, J., Lu, Z., Li, H., Li, V.O.K.: Incorporating copying mechanism in sequence-to-sequence learning. In: ACL 2016 (2016)
13.
Gu, J., Wang, Y., Cho, K., Li, V.O.K.: Search engine guided neural machine translation. In: AAAI 2018, pp. 5133–5140 (2018)
14.
Hewavitharana, S., Vogel, S., Waibel, A.: Augmenting a statistical translation system with a translation memory. In: EAMT 2005, vol. 5 (2005)
15.
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR 2015 (2015)
16.
Koehn, P., et al.: Moses: open source toolkit for statistical machine translation. In: ACL 2007, pp. 177–180 (2007)
17.
Koehn, P., Och, F.J., Marcu, D.: Statistical phrase-based translation. In: NAACL 2003 (2003)
18.
Koehn, P., Senellart, J.: Convergence of translation memory and statistical machine translation. In: AMTA 2010, pp. 21–31 (2010)
19.
Koehn, P., Senellart, J.: Fast approximate string matching with suffix arrays and A* parsing. In: AMTA 2010 (2010)
20.
Li, J., Xiong, D., Tu, Z., Zhu, M., Zhang, M., Zhou, G.: Modeling source syntax for neural machine translation. In: ACL 2017, pp. 688–697 (2017)
21.
Luong, M.T., Manning, C.D.: Stanford neural machine translation systems for spoken language domains. In: IWSLT 2015, pp. 76–79 (2015)
22.
Maruf, S., Martins, A.F.T., Haffari, G.: Selective attention for context-aware neural machine translation. In: NAACL 2019, pp. 3092–3102 (2019)
23.
Ortega, J.E., Sánchez-Martınez, F., Forcada, M.L.: Fuzzy-match repair using black-box machine translation systems: what can be expected. In: AMTA 2016, vol. 1, pp. 27–39 (2016)
24.
Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. In: ACL 2016 (2016)
25.
Simard, M., Isabelle, P.: Phrase-based machine translation in a computer-assisted translation environment. In: MT Summit XII, pp. 120–127 (2009)
26.
Simard, M., Langlais, P.: Sub-sentential exploitation of translation memories. In: Machine Translation Summit, vol. 8, pp. 335–339 (2001)
27.
Vaswani, A., et al.: Attention is all you need. In: NIPS 2017, pp. 5998–6008 (2017)
28.
Wang, K., Zong, C., Su, K.: Integrating translation memory into phrase-based machine translation during decoding. In: ACL 2013, pp. 11–21 (2013)
29.
Wuebker, J., Green, S., DeNero, J.: Hierarchical incremental adaptation for statistical machine translation. In: EMNLP 2015, pp. 1059–1065 (2015)
30.
Xia, Y., et al.: Deliberation networks: sequence generation beyond one-pass decoding. In: NIPS 2017, pp. 1784–1794 (2017)
31.
Xu, J., Crego, J.M., Senellart, J.: Boosting neural machine translation with similar translations. In: ACL 2020, pp. 1580–1590 (2020)
32.
Yang, B., Li, J., Wong, D.F., Chao, L.S., Wang, X., Tu, Z.: Context-aware self-attention networks. In: AAAI 2019, pp. 387–394 (2019)
33.
Zhang, J., Utiyama, M., Sumita, E., Neubig, G., Nakamura, S.: Improving neural machine translation through phrase-based forced decoding. In: IJCNLP 2017, pp. 152–162 (2017)
34.
Zhang, J., Utiyama, M., Sumita, E., Neubig, G., Nakamura, S.: Guiding neural machine translation with retrieved translation pieces. In: NAACL 2018, pp. 1325–1335 (2018)
35.
Zhou, L., Hu, W., Zhang, J., Zong, C.: Neural system combination for machine translation. In: ACL 2017, pp. 378–384 (2017)
36.
Zhu, C., Yu, H., Cheng, S., Luo, W.: Language-aware interlingua for multilingual neural machine translation. In: ACL 2020, pp. 1650–1655 (2020)
Metadata
Title
Learning to Select Relevant Knowledge for Neural Machine Translation
Authors
Jian Yang
Juncheng Wan
Shuming Ma
Haoyang Huang
Dongdong Zhang
Yong Yu
Zhoujun Li
Furu Wei
Copyright Year
2021
Book
Natural Language Processing and Chinese Computing

Print ISBN: 978-3-030-88479-6

Electronic ISBN: 978-3-030-88480-2

Copyright Year: 2021

DOI
https://doi.org/10.1007/978-3-030-88480-2_7

Premium Partner

    Image Credits