Top

Published in:

2021 | OriginalPaper | Chapter

SMT Versus NMT: An Experiment with Punjabi–English

Authors : Kamal Deep, Ajit Kumar, Vishal Goyal

Published in: Recent Innovations in Computing

Publisher: Springer Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

In this paper, a comparison of the Statistical machine translation (SMT) and neural machine translation (NMT) for Punjabi to English in the fixed domain of health and tourism is provided. We have tried to answer does NMT perform equivalent well or better with respect to the SMT system? We have developed the three base models viz., SMT-based model using the Moses toolkit, followed by long short-term memory (LSTM) model and bidirectional LSTM model using the OpenNMT toolkit. All three models used the Punjabi–English parallel corpus of TDIL health and tourism. Finally, the quality of translation is validated using the automatic parameter like the BLEU score and the TER score. We observed that bidirectional LSTM performs better than simple LSTM an SMT system.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter A Comparative Study of NoSQL Databases

next chapter Performance Enhancement of MIMO Configurations in FSO System Under Different Weather Conditions

Garje, G.V., Bansode, A., Gandhi, S., Kulkarni, A.: Marathi to English sentence translator for simple assertive and interrogative sentences. Int. J. Comput. Appl. 138(5), 42–45 (2016). https://doi.org/10.5120/ijca2016908837CrossRef

Singh, P.K., Kar, A.K., Singh, Y., Kolekar, M.H., Tanwar, S.: Proceedings of ICRIC 2019. Recent Innovations in Computing, 2020. Lecture Notes in Electrical Engineering, vol. 597, pp. 3–920. Springer, Cham, Switzerland

Singh, U., Goyal, V., Singh, G.: Urdu to Punjabi machine translation: an incremental training approach. Int. J. Adv. Comput. Sci. Appl. 7(4), 227–238 (2016). https://doi.org/10.14569/ijacsa.2016.070428CrossRef

Chaudhary, J.R., Patel, A.C.: Machine translation using deep learning: a survey. Int. J. Sci. Res. Sci. Eng. Technol. 4(2), 145–150 (2018)

Mahata, S.K., Mandal, S., Das, D., Bandyopadhyay, S.: SMT vs NMT: a comparison over Hindi & Bengali simple sentences. In: International Conference on Natural Language Processing, Dec 2018, pp. 175–182. [Online]. Available: https://arxiv.org/abs/1812.04898

Bombay, T.: Statistical machine translation with rule based re-ordering of source sentences. In: International Conference on Natural Language Processing, Aug 2008

Singh, P.K., Pawłowski, W., Tanwar, S., Kumar, N., Rodrigues, J.J.P.C.: Proceedings of First International Conference on Computing, Communications, and Cyber-Security (IC4S 2019), 2020. Lecture Notes in Networks and Systems, vol. 121, pp. 3–917. Springer, Cham, Switzerland

Visweswariah, K., Rajkumar, R., Gandhe, A., Ramanathan, A., Navratil, J.: A word reordering model for improved machine translation. In: EMNLP-2011, pp. 486–496. [Online]. Available: https://aclweb.org/anthology-new/D/D11/D11-1045.pdf

Anand, A., Bhattacharyya, P.: Literature survey: neural machine translation (2014)

10.

He, W., He, Z., Wu, H., Wang, H.: Improved neural machine translation with SMT features. In: 30th AAAI Conference on Artificial Intelligence, AAAI 2016, no. 10, pp. 151–157 (2016)

11.

Luong, M.-T., Sutskever, I., Le, Q.V., Vinyals, O., Zaremba, W.: Addressing the rare word problem in neural machine translation (2014). https://doi.org/10.3115/v1/P15-1002

12.

TDIL CORPUS Homepage. https://www.tdil-dc.in/index.php?lang=enwww.tdil-dc.in. Last accessed 21 Nov 2018

13.

Bhalla, D., Joshi, N., Mathur, I.: Rule based transliteration scheme for English to Punjabi. Int. J. Nat. Lang. Comput. 2(2), 67–73 (2013). https://doi.org/10.5121/ijnlc.2013.2207CrossRef

14.

Azath, M., Kiros, T.: Statistical machine translator for English to Tigrigna translation. Int. J. Sci. Technol. Res. 9(1), 2095–2099 (2020)

15.

Östling, R., Scherrer, Y., Tiedemann, J., Tang, G., Nieminen, T.: The Helsinki neural machine translation system, vol. 2, pp. 338–347 (2018). https://doi.org/10.18653/v1/w17-4733

16.

Cho, K., van Merrienboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: encoder–decoder approaches. In: Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, pp. 103–111 (2014). [Online]. Available: https://arxiv.org/pdf/1409.1259v2.pdf%5Cn; https://arxiv.org/abs/1409.1259

17.

Wu, Y., et al.: Google’s neural machine translation system: bridging the gap between human and machine translation, pp. 1–23 (2016). [Online]. Available: https://arxiv.org/abs/1609.08144

18.

Zaremba, W., Sutskever, I., Vinyals, O.: Recurrent neural network regularization, no. 2013, pp. 1–8 (2014)

19.

Dušek, O., Novikova, J., Rieser, V.: Evaluating the state-of-the-art of end-to-end natural language generation: the E2E NLG challenge. Comput. Speech Lang. 59, 123–156 (2020). https://doi.org/10.1016/j.csl.2019.06.009CrossRef

20.

OPENMT Homepage. https://github.com/OpenNMT/OpenNMT-py. Last accessed 21 Nov 2018

Title: SMT Versus NMT: An Experiment with Punjabi–English
Authors: Kamal Deep
Ajit Kumar
Vishal Goyal
Publisher: Springer Singapore
Book: Recent Innovations in Computing
Print ISBN: 978-981-15-8296-7

Electronic ISBN: 978-981-15-8297-4

Copyright Year: 2021
DOI: https://doi.org/10.1007/978-981-15-8297-4_6

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner