nach oben

Erschienen in:

2017 | OriginalPaper | Buchkapitel

LSTM Recurrent Neural Networks for Short Text and Sentiment Classification

verfasst von : Jakub Nowak, Ahmet Taspinar, Rafał Scherer

Erschienen in: Artificial Intelligence and Soft Computing

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Recurrent neural networks are increasingly used to classify text data, displacing feed-forward networks. This article is a demonstration of how to classify text using Long Term Term Memory (LSTM) network and their modifications, i.e. Bidirectional LSTM network and Gated Recurrent Unit. We present the superiority of this method over other algorithms for text classification on the example of three sets: Spambase Data Set, Farm Advertisement and Amazon book reviews. The results of the first two datasets were compared with AdaBoost ensemble of feedforward neural networks. In the case of the last database, the result is compared to the bag-of-words algorithm. In this article, we focus on classifying two groups in the first two collections, since we are only interested in whether something is classified into a SPAM or an eligible message. In the last dataset, we distinguish three classes.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Swarm Intelligence in Solving Stochastic Capacitated Vehicle Routing Problem

Nächstes Kapitel Categorization of Multilingual Scientific Documents by a Compound Classification System

Auli, M., Galley, M., Quirk, C., Zweig, G.: Joint language and translation modeling with recurrent neural networks. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Washington, USA, pp. 1044–1054, October 2013 Association for Computational Linguistics (2013)

Bas, E.: The training of multiplicative neuron model based artificial neural networks with differential evolution algorithm for forecasting. J. Artif. Intell. Soft Comput. Res. 6(1), 5–11 (2016)CrossRef

Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (1994)CrossRef

Bertini Junior, J.R., Nicoletti, M.D.C.: Enhancing constructive neural network performance using functionally expanded input data. J. Artif. Intell. Soft Comput. Res. 6(2), 119–131 (2016)CrossRef

Britz, D.: Recurrent neural network tutorial, part 4 - implementing a GRU/LSTM RNN with python and theano. http://www.wildml.com/. Accessed 27 Oct 2015

Chen, C., Xia, L.: Recurrent neural network and long short-term memory. http://ace.cs.ohiou.edu/~razvan/courses/dl6890/presentations/lichen-lijie.pdf

Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation (2014). arXiv preprint arXiv:1406.1078

Gers, F.A., Schmidhuber, E.: LSTM recurrent networks learn simple context-free and context-sensitive languages. IEEE Trans. Neural Netw. 12(6), 1333–1340 (2001)CrossRef

Gers, F.A., Schmidhuber, J., Cummins, F.: Learning to forget: continual prediction with LSTM. Neural Comput. 12(10), 2451–2471 (2000)CrossRef

10.

Gers, F.A., Schraudolph, N.N., Schmidhuber, J.: Learning precise timing with LSTM recurrent networks. J. Mach. Learn. Res. 3, 115–143 (2002)MathSciNetMATH

11.

Graves, A.: Supervised sequence labelling. In: Supervised Sequence Labelling with Recurrent Neural Networks, pp. 5–13. Springer (2012)

12.

Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural Netw. 18(5), 602–610 (2005)CrossRef

13.

Hochreiter, S.: The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int. J. Uncertainty Fuzz. Knowl. Based Syst. 6(02), 107–116 (1998)CrossRefMATH

14.

Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRef

15.

Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRFmodels for sequence tagging (2015). arXiv preprint arXiv:1508.01991

16.

Murata, M., Ito, S., Tokuhisa, M., Ma, Q.: Order estimation of Jaanese paragraphs by supervised machine learning and various textual features. J. Artif. Intell. Soft Comput. Res. 5(4), 247–255 (2015)CrossRef

17.

Olah, C.: Understanding LSTM networks, August 2015. http://colah.github.io/posts/2015-08-Understanding-LSTMs

18.

Patgiri, C., Sarma, M., Sarma, K.K.: A class of neuro-computational methods for assamese fricative classification. J. Artif. Intell. Soft Comput. Res. 5(1), 59–70 (2015)CrossRef

19.

Shuang Bi, W.Z.: Cs294-1 final project algorithms comparison deep learning neural network — adaboost — random forest. http://bid.berkeley.edu/cs294-1-spring13/images/0/0d/ProjectReport(Shuang_and_Wenchang).pdf. Accessed 15 May 2013

20.

Taspinar, A.: Sentiment analysis with bag-of-words. http://ataspinar.com/2016/01/21/sentiment-analysis-with-bag-of-words/. Accessed 21 Jan 2016

21.

Trask, A.: Anyone can learn to code an LSTM-RNN in python. https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/. Accessed 15 Nov 2015

22.

Trofimovich, J.: Comparison of neural network architectures for sentiment analysis of Russian tweets, 1–4 June 2016

23.

Williams, R.J., Zipser, D.: Gradient-based learning algorithms for recurrent networks and their computational complexity. In: Backpropagation: Theory, Architectures, and Applications, vol. 1, pp. 433–486 (1995)

Titel: LSTM Recurrent Neural Networks for Short Text and Sentiment Classification
verfasst von: Jakub Nowak
Ahmet Taspinar
Rafał Scherer
Verlag: Springer International Publishing
Buch: Artificial Intelligence and Soft Computing
Print ISBN: 978-3-319-59059-2

Electronic ISBN: 978-3-319-59060-8

Copyright-Jahr: 2017
DOI: https://doi.org/10.1007/978-3-319-59060-8_50

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner