Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Stochastic DCA for Sparse Multiclass Logistic Regression Read chapter Read first chapter

2018 | OriginalPaper | Chapter

Bidirectional Deep Learning of Context Representation for Joint Word Segmentation and POS Tagging

Authors : Prachya Boonkwan, Thepchai Supnithi

Published in: Advanced Computational Methods for Knowledge Engineering

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Word segmentation and POS tagging are crucial steps for natural language processing. Though deep learning facilitates learning a joint model without feature engineering, it still suffers from unreliable word embedding when words are rare or unknown. We introduce two-level backoff models to which morphological information and character-level contexts are integrated. Experimental results on Thai and Chinese show that our backoff models improve the accuracy of both tasks and excels in OOV recovery.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter An Early-Biologisation Process to Improve the Acceptance of Biomimetics in Organizations
next chapter A Model for a Computing Cluster with Two Asynchronous Servers
Literature
1.
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C.: A neural probabilistic language model. JMLR 3, 1137–1155 (2003)MATH
2.
Boonkwan, P., Supnithi, T., Pailai, J., Kongkachandra, R.: Gradient-descent error correction of POS tagging. In: Proceedings of SNLP (2013)
3.
Boriboon, M., Kriengket, K., Chootrakool, P., Phaholphinyo, S., Purodakananda, S., Thanakulwarapas, T., Kosawat, K.: BEST corpus development and analysis. In: Proceedings of the 2009 International Conference on Asian Language Processing, pp. 322–327 (2009)
4.
Chen, K.L., Hsieh, Y.M.: Chinese treebanks and grammar extraction. In: Proceedings of IJCNLP, pp. 560–565 (2004)
5.
Chung, J., Gülçehre, Ç., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. In: NIPS 2014: Deep Learning and Representation Learning Workshop (2014)
6.
Collins, M.: Discriminative training methods for hidden Markov models: theory and experiments with perceptron algorithms. In: Proceedings of EMNLP (2002)
7.
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.P.: Natural language processing (almost) from scratch. JMLR 12, 2493–2537 (2011)MATH
8.
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of AIStats, vol. 9, pp. 249–256 (2010)
9.
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRef
10.
Kaji, N., Kitsuregawa, M.: Accurate word segmentation and POS tagging for japanese microblogs: corpus annotation and joint modeling with lexical normalization. In: Proceedings of EMNLP, pp. 99–109 (2014)
11.
Kongyoung, S., Rugchatjaroen, A., Kosawat, K.: TLex+: a hybrid method using conditional random fields and dictionaries for Thai word segmentation. In: Proceedings of KICSS (2015)
12.
Kruengkrai, C., Uchimoto, K., Kazama, J., Torisawa, K., Isahara, H., Jaruskulchai, C.: A word and character-cluster hybrid model for Thai word segmentation. In: Proceedings of InterBEST 2009: Thai Word Segmentation Workshop, pp. 24–29 (2009)
13.
Kruengkrai, C., Uchimoto, K., Kazama, J., Wang, Y., Torisawa, K., Isahara, H.: An error-driven word-character hybrid model for joint Chinese word segmentation and POS tagging. In: Proceedings of the Joint Conference of the 47th ACL and the 4th IJCNLP of the AFNLP, vol. 1, pp. 513–521 (2009)
14.
Lyu, C., Zhang, Y., Ji, D.: Joint word segmentation, POS-tagging, and syntactic chunking. In: Proceedings of AAAI, pp. 3007–3014 (2016)
15.
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of NIPS (2013)
16.
Murata, M., Ma, Q., Isahara, H.: Part of speech tagging in Thai language using support vector machine. In: Proceedings of NLPRS: The 2nd Workshop on Natural Language Processing and Neural Networks (2001)
17.
Peng, N., Dredze, M.: Improving named entity recognition for Chinese social media with word segmentation representation learning. In: Proceedings of ACL, pp. 149–155 (2016)
18.
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of EMNLP, pp. 1532–1543 (2014)
19.
Qian, T., Zhang, Y., Zhang, M., Ren, Y., Ji, D.: A transition-based model for joint segmentation, POS-tagging, and normalization. In: Proceedings of EMNLP, pp. 1837–1846 (2015)
20.
Qian, X., Liu, Y.: Joint Chinese word segmentation, POS tagging, and parsing. In: Proceedings of the 2012 Joint Conference on EMNLP and CoNLL, pp. 501–511 (2012)
21.
Ratliff, N., Bagnell, J.A., Zinkevich, M.: (Online) Subgradient methods for structured prediction. In: Proceedings of AIStats (2007)
22.
Shi, Y., Wang, M.: A dual-layer CRFs based joint decoding method for cascaded segmentation and labeling tasks. In: Proceedings of the IJCAI, pp. 1707–1712 (2007)
23.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. JMLR 15, 1929–1958 (2014)MathSciNetMATH
24.
Zeng, X., Wong, D.F., Chao, L.S., Trancoso, I.: Graph-based semi-supervised model for joint Chinese word segmentation and part-of-speech tagging. In: Proceedings of ACL, pp. 770–779 (2013)
25.
Zhang, Y., Clark, S.: A fast decoder for joint word segmentation and POS-tagging using a single discriminative model. In: Proceedings of EMNLP, pp. 843–852 (2010)
26.
Zheng, X., Chen, H., Xu, T.: Deep learning for Chinese word segmentation and POS tagging. In: Proceedings of EMNLP, pp. 647–657 (2013)
Metadata
Title
Bidirectional Deep Learning of Context Representation for Joint Word Segmentation and POS Tagging
Authors
Prachya Boonkwan
Thepchai Supnithi
Copyright Year
2018
Book
Advanced Computational Methods for Knowledge Engineering

Print ISBN: 978-3-319-61910-1

Electronic ISBN: 978-3-319-61911-8

Copyright Year: 2018

DOI
https://doi.org/10.1007/978-3-319-61911-8_17

Premium Partner

    Image Credits