Top

previous chapter A Comparative Study of NoSQL Databases

next chapter Performance Enhancement of MIMO Configurations ...

Hint

Swipe to navigate through the chapters of this book

2021 | OriginalPaper | Chapter

SMT Versus NMT: An Experiment with Punjabi–English

Read first chapter

Authors: Kamal Deep, Ajit Kumar, Vishal Goyal

Publisher: Springer Singapore

Published in: Recent Innovations in Computing

» Get access to the full-text

Abstract

In this paper, a comparison of the Statistical machine translation (SMT) and neural machine translation (NMT) for Punjabi to English in the fixed domain of health and tourism is provided. We have tried to answer does NMT perform equivalent well or better with respect to the SMT system? We have developed the three base models viz., SMT-based model using the Moses toolkit, followed by long short-term memory (LSTM) model and bidirectional LSTM model using the OpenNMT toolkit. All three models used the Punjabi–English parallel corpus of TDIL health and tourism. Finally, the quality of translation is validated using the automatic parameter like the BLEU score and the TER score. We observed that bidirectional LSTM performs better than simple LSTM an SMT system.

Please log in to get access to this content

To get access to this content you need the following product:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 69.000 Bücher
über 500 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Umwelt
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Testen Sie jetzt 30 Tage kostenlos.

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 50.000 Bücher
über 380 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Umwelt
Maschinenbau + Werkstoffe

Testen Sie jetzt 30 Tage kostenlos.

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 58.000 Bücher
über 300 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Testen Sie jetzt 30 Tage kostenlos.

inform now

Garje, G.V., Bansode, A., Gandhi, S., Kulkarni, A.: Marathi to English sentence translator for simple assertive and interrogative sentences. Int. J. Comput. Appl. 138(5), 42–45 (2016). https://doi.org/10.5120/ijca2016908837 CrossRef

Singh, P.K., Kar, A.K., Singh, Y., Kolekar, M.H., Tanwar, S.: Proceedings of ICRIC 2019. Recent Innovations in Computing, 2020. Lecture Notes in Electrical Engineering, vol. 597, pp. 3–920. Springer, Cham, Switzerland

Singh, U., Goyal, V., Singh, G.: Urdu to Punjabi machine translation: an incremental training approach. Int. J. Adv. Comput. Sci. Appl. 7(4), 227–238 (2016). https://doi.org/10.14569/ijacsa.2016.070428 CrossRef

Chaudhary, J.R., Patel, A.C.: Machine translation using deep learning: a survey. Int. J. Sci. Res. Sci. Eng. Technol. 4(2), 145–150 (2018)

Mahata, S.K., Mandal, S., Das, D., Bandyopadhyay, S.: SMT vs NMT: a comparison over Hindi & Bengali simple sentences. In: International Conference on Natural Language Processing, Dec 2018, pp. 175–182. [Online]. Available: https://arxiv.org/abs/1812.04898

Bombay, T.: Statistical machine translation with rule based re-ordering of source sentences. In: International Conference on Natural Language Processing, Aug 2008

Singh, P.K., Pawłowski, W., Tanwar, S., Kumar, N., Rodrigues, J.J.P.C.: Proceedings of First International Conference on Computing, Communications, and Cyber-Security (IC4S 2019), 2020. Lecture Notes in Networks and Systems, vol. 121, pp. 3–917. Springer, Cham, Switzerland

Visweswariah, K., Rajkumar, R., Gandhe, A., Ramanathan, A., Navratil, J.: A word reordering model for improved machine translation. In: EMNLP-2011, pp. 486–496. [Online]. Available: https://aclweb.org/anthology-new/D/D11/D11-1045.pdf

Anand, A., Bhattacharyya, P.: Literature survey: neural machine translation (2014)

10.

He, W., He, Z., Wu, H., Wang, H.: Improved neural machine translation with SMT features. In: 30th AAAI Conference on Artificial Intelligence, AAAI 2016, no. 10, pp. 151–157 (2016)

11.

Luong, M.-T., Sutskever, I., Le, Q.V., Vinyals, O., Zaremba, W.: Addressing the rare word problem in neural machine translation (2014). https://doi.org/10.3115/v1/P15-1002

12.

TDIL CORPUS Homepage. https://www.tdil-dc.in/index.php?lang=enwww.tdil-dc.in. Last accessed 21 Nov 2018

13.

Bhalla, D., Joshi, N., Mathur, I.: Rule based transliteration scheme for English to Punjabi. Int. J. Nat. Lang. Comput. 2(2), 67–73 (2013). https://doi.org/10.5121/ijnlc.2013.2207 CrossRef

14.

Azath, M., Kiros, T.: Statistical machine translator for English to Tigrigna translation. Int. J. Sci. Technol. Res. 9(1), 2095–2099 (2020)

15.

Östling, R., Scherrer, Y., Tiedemann, J., Tang, G., Nieminen, T.: The Helsinki neural machine translation system, vol. 2, pp. 338–347 (2018). https://doi.org/10.18653/v1/w17-4733

16.

Cho, K., van Merrienboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: encoder–decoder approaches. In: Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, pp. 103–111 (2014). [Online]. Available: https://arxiv.org/pdf/1409.1259v2.pdf%5Cn; https://arxiv.org/abs/1409.1259

17.

Wu, Y., et al.: Google’s neural machine translation system: bridging the gap between human and machine translation, pp. 1–23 (2016). [Online]. Available: https://arxiv.org/abs/1609.08144

18.

Zaremba, W., Sutskever, I., Vinyals, O.: Recurrent neural network regularization, no. 2013, pp. 1–8 (2014)

19.

Dušek, O., Novikova, J., Rieser, V.: Evaluating the state-of-the-art of end-to-end natural language generation: the E2E NLG challenge. Comput. Speech Lang. 59, 123–156 (2020). https://doi.org/10.1016/j.csl.2019.06.009 CrossRef

20.

OPENMT Homepage. https://github.com/OpenNMT/OpenNMT-py. Last accessed 21 Nov 2018

Title

SMT Versus NMT: An Experiment with Punjabi–English

Book

Recent Innovations in Computing

Print ISBN: 978-981-15-8296-7

Electronic ISBN: 978-981-15-8297-4

https://doi.org/10.1007/978-981-15-8297-4

DOI

https://doi.org/10.1007/978-981-15-8297-4_6

Authors:

Kamal Deep
Ajit Kumar
Vishal Goyal

Publisher

Springer Singapore

Sequence number

Premium Partner

Image Credits