Skip to main content
Top
Published in: Neural Processing Letters 3/2019

10-06-2019

A Robust Sentiment Analysis Method Based on Sequential Combination of Convolutional and Recursive Neural Networks

Authors: Hossein Sadr, Mir Mohsen Pedram, Mohammad Teshnehlab

Published in: Neural Processing Letters | Issue 3/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

With explosive development of the World Wide Web, an enormous amount of text information containing users’ feeling, emotions and opinions has been generated and is increasingly employed by individuals and companies for making decisions. Whereas unstructured form of data must be analyzed to extract and summarize the opinions in them, sentiment analysis has changed to a significant research area in the field of Natural Language Processing. In this regard, deep learning methods have attracted a lot of attentions in recent years and various deep learning models have been proven as effective network architectures for the task of sentiment analysis. However, each of them has its potentials and weak points. To eliminate their drawbacks and make optimal use of their benefits, convolutional and recursive neural network are merged into a new robust model in this paper. The proposed model employs recursive neural network due to its tree structure as a substitute of pooling layer in the convolutional network with the aim of capturing long-term dependencies and reducing the loss of local information. The proposed model is validated on Stanford Sentiment Treebank by conducting a series of experiments and empirical results revealed that our model outperforms basic convolutional and recursive neural networks while requires fewer parameters.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Cambria E, Poria S, Gelbukh A, Thelwall M (2017) Sentiment analysis is a big suitcase. IEEE Intell Syst 32(6):74–80CrossRef Cambria E, Poria S, Gelbukh A, Thelwall M (2017) Sentiment analysis is a big suitcase. IEEE Intell Syst 32(6):74–80CrossRef
2.
go back to reference Vyas V, Uma V (2019) Approaches to sentiment analysis on product reviews. In: Sentiment analysis and knowledge discovery in contemporary business. IGI Global, pp 15–30 Vyas V, Uma V (2019) Approaches to sentiment analysis on product reviews. In: Sentiment analysis and knowledge discovery in contemporary business. IGI Global, pp 15–30
3.
go back to reference Ravi K, Ravi V (2015) A survey on opinion mining and sentiment analysis: tasks, approaches and applications. Knowl-Based Syst 89:14–46CrossRef Ravi K, Ravi V (2015) A survey on opinion mining and sentiment analysis: tasks, approaches and applications. Knowl-Based Syst 89:14–46CrossRef
4.
go back to reference Souma W, Vodenska I, Aoyama H (2019) Enhanced news sentiment analysis using deep learning methods. J Comput Soc Sci 2:1–14CrossRef Souma W, Vodenska I, Aoyama H (2019) Enhanced news sentiment analysis using deep learning methods. J Comput Soc Sci 2:1–14CrossRef
5.
go back to reference Ain QT, Ali M, Riaz A, Noureen A, Kamran M, Hayat B, Rehman A (2017) Sentiment analysis using deep learning techniques: a review. Int J Adv Comput Sci Appl 8(6):424 Ain QT, Ali M, Riaz A, Noureen A, Kamran M, Hayat B, Rehman A (2017) Sentiment analysis using deep learning techniques: a review. Int J Adv Comput Sci Appl 8(6):424
6.
go back to reference Sadr H, Nazari Solimandarabi M (2019) Presentation of an efficient automatic short answer grading model based on combination of pseudo relevance feedback and semantic relatedness measures. J Adv Comput Res 10(2):1–10 Sadr H, Nazari Solimandarabi M (2019) Presentation of an efficient automatic short answer grading model based on combination of pseudo relevance feedback and semantic relatedness measures. J Adv Comput Res 10(2):1–10
7.
go back to reference Jadidinejad A, Sadr H (2015) Improving weak queries using local cluster analysis as a preliminary framework. Indian J Sci Technol 8:15CrossRef Jadidinejad A, Sadr H (2015) Improving weak queries using local cluster analysis as a preliminary framework. Indian J Sci Technol 8:15CrossRef
8.
go back to reference Abirami A, Gayathri V (2017) A survey on sentiment analysis methods and approach. In: 8th International conference on advanced computing (ICoAC), IEEE, pp 72–76 Abirami A, Gayathri V (2017) A survey on sentiment analysis methods and approach. In: 8th International conference on advanced computing (ICoAC), IEEE, pp 72–76
9.
go back to reference Medhat W, Hassan A, Korashy H (2014) Sentiment analysis algorithms and applications: a survey. Ain Shams Eng J 5(4):1093–1113CrossRef Medhat W, Hassan A, Korashy H (2014) Sentiment analysis algorithms and applications: a survey. Ain Shams Eng J 5(4):1093–1113CrossRef
10.
go back to reference Soleimandarabi MN, Mirroshandel SA (2015) A novel approach for computing semantic relatedness of geographic terms. Indian J Sci Technol 8:27CrossRef Soleimandarabi MN, Mirroshandel SA (2015) A novel approach for computing semantic relatedness of geographic terms. Indian J Sci Technol 8:27CrossRef
11.
go back to reference Soleimandarabi MN, Mirroshandel SA, Sadr H (2015) A Survey of semantic relatedness measures. Int J Comput Sci Network Solut 3(2):243–247 Soleimandarabi MN, Mirroshandel SA, Sadr H (2015) A Survey of semantic relatedness measures. Int J Comput Sci Network Solut 3(2):243–247
12.
go back to reference Young T, Hazarika D, Poria S, Cambria E (2018) Recent trends in deep learning based natural language processing. IEEE Comput Intell Mag 13(3):55–75CrossRef Young T, Hazarika D, Poria S, Cambria E (2018) Recent trends in deep learning based natural language processing. IEEE Comput Intell Mag 13(3):55–75CrossRef
13.
go back to reference Mohammad SM (2017) Challenges in sentiment analysis. In: A practical guide to sentiment analysis. Springer, pp 61–83 Mohammad SM (2017) Challenges in sentiment analysis. In: A practical guide to sentiment analysis. Springer, pp 61–83
14.
go back to reference Ouyang X, Zhou P, Li CH, Liu L (2015) Sentiment analysis using convolutional neural network. In: 2015 IEEE International conference on computer and information technology; ubiquitous computing and communications; dependable, autonomic and secure computing; pervasive intelligence and computing. IEEE, pp 2359–2364 Ouyang X, Zhou P, Li CH, Liu L (2015) Sentiment analysis using convolutional neural network. In: 2015 IEEE International conference on computer and information technology; ubiquitous computing and communications; dependable, autonomic and secure computing; pervasive intelligence and computing. IEEE, pp 2359–2364
15.
go back to reference Hassan A, Mahmood A (2017) Deep learning approach for sentiment analysis of short texts. In: 3rd International conference on control, automation and robotics (ICCAR), IEEE, pp 705–710 Hassan A, Mahmood A (2017) Deep learning approach for sentiment analysis of short texts. In: 3rd International conference on control, automation and robotics (ICCAR), IEEE, pp 705–710
16.
go back to reference Van VD, Thai T, Nghiem M-Q (2017) Combining convolution and recursive neural networks for sentiment analysis. In: Proceedings of the 8th international symposium on information and communication technology, 2017. ACM, pp 151–158 Van VD, Thai T, Nghiem M-Q (2017) Combining convolution and recursive neural networks for sentiment analysis. In: Proceedings of the 8th international symposium on information and communication technology, 2017. ACM, pp 151–158
17.
go back to reference Chen D, Zhang Y (2018) Robust zeroing neural-dynamics and its time-varying disturbances suppression model applied to mobile robot manipulators. IEEE Trans Neural Netw Learn Syst 29(9):4385–4397CrossRef Chen D, Zhang Y (2018) Robust zeroing neural-dynamics and its time-varying disturbances suppression model applied to mobile robot manipulators. IEEE Trans Neural Netw Learn Syst 29(9):4385–4397CrossRef
18.
go back to reference Chen K, Yi C (2016) Robustness analysis of a hybrid of recursive neural dynamics for online matrix inversion. Appl Math Comput 273:969–975MathSciNetMATH Chen K, Yi C (2016) Robustness analysis of a hybrid of recursive neural dynamics for online matrix inversion. Appl Math Comput 273:969–975MathSciNetMATH
19.
go back to reference Jin L, Li S, Hu B (2018) RNN models for dynamic matrix inversion: a control-theoretical perspective. IEEE Trans Ind Inf 14(1):189–199CrossRef Jin L, Li S, Hu B (2018) RNN models for dynamic matrix inversion: a control-theoretical perspective. IEEE Trans Ind Inf 14(1):189–199CrossRef
20.
go back to reference You Q, Cao L, Jin H, Luo J (2016) Robust visual-textual sentiment analysis: When attention meets tree-structured recursive neural networks. In: Proceedings of the 24th ACM international conference on multimedia, ACM, pp 1008–1017 You Q, Cao L, Jin H, Luo J (2016) Robust visual-textual sentiment analysis: When attention meets tree-structured recursive neural networks. In: Proceedings of the 24th ACM international conference on multimedia, ACM, pp 1008–1017
21.
go back to reference Le Q, Mikolov T (2014) Distributed representations of sentences and documents. In: International conference on machine learning, pp 1188–1196 Le Q, Mikolov T (2014) Distributed representations of sentences and documents. In: International conference on machine learning, pp 1188–1196
22.
go back to reference Maas AL, Daly RE, Pham PT, Huang D, Ng AY, Potts C (2011) Learning word vectors for sentiment analysis. In: Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies-volume 1, Association for Computational Linguistics, pp 142–150 Maas AL, Daly RE, Pham PT, Huang D, Ng AY, Potts C (2011) Learning word vectors for sentiment analysis. In: Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies-volume 1, Association for Computational Linguistics, pp 142–150
23.
go back to reference Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 3111–3119 Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 3111–3119
24.
go back to reference Islam J, Zhang Y (2016) Visual sentiment analysis for social images using transfer learning approach. In: 2016 IEEE international conferences on big data and cloud computing (BDCloud), social computing and networking (SocialCom), sustainable computing and communications (SustainCom)(BDCloud-SocialCom-SustainCom), IEEE, pp 124–130 Islam J, Zhang Y (2016) Visual sentiment analysis for social images using transfer learning approach. In: 2016 IEEE international conferences on big data and cloud computing (BDCloud), social computing and networking (SocialCom), sustainable computing and communications (SustainCom)(BDCloud-SocialCom-SustainCom), IEEE, pp 124–130
26.
go back to reference Zhang X, Zhao J, LeCun Y (2015) Character-level convolutional networks for text classification. In: Advances in neural information processing systems, pp 649-657 Zhang X, Zhao J, LeCun Y (2015) Character-level convolutional networks for text classification. In: Advances in neural information processing systems, pp 649-657
27.
go back to reference Socher R, Lin CC, Manning C, Ng AY (2011) Parsing natural scenes and natural language with recursive neural networks. In: Proceedings of the 28th international conference on machine learning (ICML-11), pp 129–136 Socher R, Lin CC, Manning C, Ng AY (2011) Parsing natural scenes and natural language with recursive neural networks. In: Proceedings of the 28th international conference on machine learning (ICML-11), pp 129–136
28.
go back to reference Socher R, Huval B, Manning CD, Ng AY (2012) Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, association for computational linguistics, pp 1201–1211 Socher R, Huval B, Manning CD, Ng AY (2012) Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, association for computational linguistics, pp 1201–1211
29.
go back to reference Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng A, Potts C (2013) Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 conference on empirical methods in natural language processing, pp 1631–1642 Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng A, Potts C (2013) Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 conference on empirical methods in natural language processing, pp 1631–1642
30.
go back to reference Wang X, Jiang W, Luo Z (2016) Combination of convolutional and recurrent neural network for sentiment analysis of short texts. In: Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers, pp 2428–2437 Wang X, Jiang W, Luo Z (2016) Combination of convolutional and recurrent neural network for sentiment analysis of short texts. In: Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers, pp 2428–2437
31.
go back to reference Huang Q, Chen R, Zheng X, Dong Z (2017) Deep sentiment representation based on CNN and LSTM. In: 2017 International conference on green informatics (ICGI), IEEE, pp 30–33 Huang Q, Chen R, Zheng X, Dong Z (2017) Deep sentiment representation based on CNN and LSTM. In: 2017 International conference on green informatics (ICGI), IEEE, pp 30–33
32.
go back to reference Timmaraju A, Khanna V (2015) Sentiment analysis on movie reviews using recursive and recurrent neural network architectures. Semantic Scholar Timmaraju A, Khanna V (2015) Sentiment analysis on movie reviews using recursive and recurrent neural network architectures. Semantic Scholar
33.
go back to reference Jin L, Li S, Hu B, Liu M, Yu J (2019) A noise-suppressing neural algorithm for solving the time-varying system of linear equations: a control-based approach. IEEE Trans Ind Inf 15(1):236–246CrossRef Jin L, Li S, Hu B, Liu M, Yu J (2019) A noise-suppressing neural algorithm for solving the time-varying system of linear equations: a control-based approach. IEEE Trans Ind Inf 15(1):236–246CrossRef
34.
go back to reference Chen D, Li S, Wu Q (2019) Rejecting chaotic disturbances using a super-exponential-zeroing neurodynamic approach for synchronization of chaotic sensor systems. Sensors 19(1):74CrossRef Chen D, Li S, Wu Q (2019) Rejecting chaotic disturbances using a super-exponential-zeroing neurodynamic approach for synchronization of chaotic sensor systems. Sensors 19(1):74CrossRef
35.
go back to reference Chen D, Zhang Y, Li S (2018) Tracking control of robot manipulators with unknown models: a Jacobian-matrix-adaption method. IEEE Trans Ind Inf 14(7):3044–3053CrossRef Chen D, Zhang Y, Li S (2018) Tracking control of robot manipulators with unknown models: a Jacobian-matrix-adaption method. IEEE Trans Ind Inf 14(7):3044–3053CrossRef
36.
go back to reference Kalchbrenner N, Grefenstette E, Blunsom P (2014) A convolutional neural network for modelling sentences. arXiv:14042188 Kalchbrenner N, Grefenstette E, Blunsom P (2014) A convolutional neural network for modelling sentences. arXiv:​14042188
37.
go back to reference Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P (2011) Natural language processing (almost) from scratch. J Mach Learn Res 12:2493–2537MATH Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P (2011) Natural language processing (almost) from scratch. J Mach Learn Res 12:2493–2537MATH
38.
go back to reference Uličný M, Lundström J, Byttner S (2016) Robustness of deep convolutional neural networks for image recognition. In: International symposium on intelligent computing systems. Springer, pp 16–30 Uličný M, Lundström J, Byttner S (2016) Robustness of deep convolutional neural networks for image recognition. In: International symposium on intelligent computing systems. Springer, pp 16–30
39.
go back to reference Park S, Kwak N (2016) Analysis on the dropout effect in convolutional neural networks. In: Asian conference on computer vision. Springer, pp 189–204 Park S, Kwak N (2016) Analysis on the dropout effect in convolutional neural networks. In: Asian conference on computer vision. Springer, pp 189–204
40.
go back to reference Du C, Huang L (2017) Sentiment classification via recurrent convolutional neural networks. DEStech Trans Comput Scie Eng (cii) Du C, Huang L (2017) Sentiment classification via recurrent convolutional neural networks. DEStech Trans Comput Scie Eng (cii)
42.
go back to reference Tai KS, Socher R, Manning CD (2015) Improved semantic representations from tree-structured long short-term memory networks. arXiv:150300075 Tai KS, Socher R, Manning CD (2015) Improved semantic representations from tree-structured long short-term memory networks. arXiv:​150300075
43.
44.
go back to reference Wang Y, Huang M, Zhao L (2016) Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 606-615 Wang Y, Huang M, Zhao L (2016) Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 606-615
45.
go back to reference Sharif M, Abadi HK (2018) Recursive nested neural network for sentiment analysis. Stanford University Report Sharif M, Abadi HK (2018) Recursive nested neural network for sentiment analysis. Stanford University Report
Metadata
Title
A Robust Sentiment Analysis Method Based on Sequential Combination of Convolutional and Recursive Neural Networks
Authors
Hossein Sadr
Mir Mohsen Pedram
Mohammad Teshnehlab
Publication date
10-06-2019
Publisher
Springer US
Published in
Neural Processing Letters / Issue 3/2019
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-019-10049-1

Other articles of this Issue 3/2019

Neural Processing Letters 3/2019 Go to the issue