Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Rule Induction of Automotive Historic Styles Using Decision Tree Classifier Read chapter Read first chapter

2020 | OriginalPaper | Chapter

Deep Learning for Multilingual POS Tagging

Authors : Alymzhan Toleu, Gulmira Tolegen, Rustam Mussabayev

Published in: Advances in Computational Collective Intelligence

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Various neural networks for sequence labeling tasks have been studied extensively in recent years. The main research focus on neural networks for the task are range from the feed-forward neural network to the long short term memory (LSTM) network with CRF layer. This paper summarizes the existing neural architectures and develop the most representative four neural networks for part-of-speech tagging and apply them on several typologically different languages. Experimental results show that the LSTM type of networks outperforms the feed-forward network in most cases and the character-level networks can learn the lexical features from characters within words, which makes the model achieve better results than no-character ones.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Rule Induction of Automotive Historic Styles Using Decision Tree Classifier
next chapter Study of Machine Learning Techniques on Accident Data
Footnotes
1
For all those LSTM type of models, we did not use the CRF layer, since LSTM can captures sentence-level information.
 
Literature
1.
Baba Ali, B., Wójcik, W., Orken, M., Turdalyuly, M., Mekebayev, N.: Speech recognizer-based non-uniform spectral compression for robust MFCC feature extraction. Przegl. Elektrotechniczny 94, 90–93 (2018)
2.
Bengio, Y., Ducharme, R., Vincent, P., Janvin, C.: A neural probabilistic language model. J. Mach. Learn. Res. 3, 1137–1155 (2003)MATH
3.
Chiu, J.P., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. Trans. Assoc. Comput. Linguist. 4, 357–370 (2016)CrossRef
4.
Cohen, T., Geiger, M., Köhler, J., Welling, M.: Spherical CNNS. ArXiv abs/1801.10130 (2018)
5.
Collobert, R., Weston, J.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning (ICML 2008), pp. 160–167. ACM, New York, NY, USA (2008)
6.
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)MATH
7.
8.
9.
Elsayed, G.F., et al.: Adversarial examples that fool both computer vision and time-limited humans. In: Proceedings of the 32Nd International Conference on Neural Information Processing Systems (NIPS 2018), pp. 3914–3924. Curran Associates Inc., USA (2018)
10.
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRef
11.
Horsmann, T., Zesch, T.: Do LSTMs really work so well for PoS tagging? – a replication study. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark (Sep 2017)
12.
13.
Kalimoldayev, M., Mamyrbayev, O., Kydyrbekova, A., Mekebayev, N.: Voice verification and identification using I-vector representation. Int. J. Math.Phys. 10(1), 66–74 (2019)CrossRef
14.
15.
Ling, W., et al.: Finding function in form: compositional character models for open vocabulary word representation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1520–1530. Association for Computational Linguistics, Lisbon, Portugal (Sep 2015)
16.
Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Vol. 1: Long Papers), pp. 1064–1074. Association for Computational Linguistics, Berlin, Germany (Aug 2016)
17.
Mamyrbayev, O., Turdalyuly, M., Mekebayev, N., Alimhan, K., Kydyrbekova, A., Turdalykyzy, T.: Automatic recognition of Kazakh speech using deep neural networks. In: Nguyen, N.T., Gaol, F.L., Hong, T.-P., Trawiński, B. (eds.) ACIIDS 2019. LNCS (LNAI), vol. 11432, pp. 465–474. Springer, Cham (2019). https://​doi.​org/​10.​1007/​978-3-030-14802-7_​40CrossRef
18.
Mamyrbayev, O., et al.: Continuous speech recognition of Kazakh language. ITM Web of Conferences 24, 01012 (2019)
19.
Mikolov, T.: Statistical Language Models Based on Neural Networks. Ph.D. Thesis, Brno University of Technology (2012)
20.
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS 2013), Vol. 2, pp. 3111–3119, Curran Associates Inc., USA (2013)
21.
Nivre, J., et al.: Universal dependencies v1: a multilingual treebank collection. In: Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016). European Language Resources Association (ELRA), Portorož, Slovenia (May 2016)
22.
Paul, D.B., Baker, J.M.: The design for the wall street journal-based CSR corpus. In: Speech and Natural Language: Proceedings of a Workshop Held at Harriman, New York, February 23–26, 1992 (1992)
23.
Pei, W., Ge, T., Chang, B.: Max-margin tensor neural network for Chinese word segmentation. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, (Vol. 1: Long Papers), pp. 293–303. Association for Computational Linguistics, Baltimore, Maryland (Jun 2014)
24.
Tolegen, G., Toleu, A., Mamyrbayev, O., Mussabayev, R.: Neural named entity recognition for Kazakh. In: Proceedings of the 20th International Conference on Computational Linguistics and Intelligent Text Processing. CICLing, Springer Lecture Notes in Computer Science (2019)
25.
Tolegen, G., Toleu, A., Zheng, X.: Named entity recognition for Kazakh using conditional random fields. In: Proceedings of the 4-th International Conference on Computer Processing of Turkic Languages TurkLang 2016, pp. 118–127. Izvestija KGTU im.I.Razzakova (2016)
26.
Toleu, A., Tolegen, G., Makazhanov, A.: Character-aware neural morphological disambiguation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, (Vol. 2: Short Papers), pp. 666–671. Association for Computational Linguistics, Vancouver, Canada (Jul 2017). 10.18653/v1/P17-2105
27.
Toleu, A., Tolegen, G., Makazhanov, A.: Character-based deep learning models for token and sentence segmentation. In: Proceedings of the 5th International Conference on Turkic Languages Processing (TurkLang 2017). Kazan, Tatarstan, Russian Federation (October 2017)
28.
Wang, M., Manning, C.D.: Effect of non-linear deep architecture in sequence labeling. In: IJCNLP (2013)
29.
Zhou, J., Xu, W.: End-to-end learning of semantic role labeling using recurrent neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Vol. 1: Long Papers), pp. 1127–1137. Association for Computational Linguistics, Beijing, China (Jul 2015)
Metadata
Title
Deep Learning for Multilingual POS Tagging
Authors
Alymzhan Toleu
Gulmira Tolegen
Rustam Mussabayev
Copyright Year
2020
Book
Advances in Computational Collective Intelligence

Print ISBN: 978-3-030-63118-5

Electronic ISBN: 978-3-030-63119-2

Copyright Year: 2020

DOI
https://doi.org/10.1007/978-3-030-63119-2_2

Premium Partner

    Image Credits