Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
World Wide Web
Published in: World Wide Web 4/2023

17-01-2023

SGPT: Semantic graphs based pre-training for aspect-based sentiment analysis

Authors: Qian Yong, Chen Chen, Zhongqing Wang, Rong Xiao, Haihong Tang

Published in: World Wide Web | Issue 4/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Previous studies show effective of pre-trained language models for sentiment analysis. However, most of these studies ignore the importance of sentimental information for pre-trained models. Therefore, we fully investigate the sentimental information for pre-trained models and enhance pre-trained language models with semantic graphs for sentiment analysis. In particular, we introduce Semantic Graphs based Pre-training(SGPT) using semantic graphs to obtain synonym knowledge for aspect-sentiment pairs and similar aspect/sentiment terms. We then optimize the pre-trained language model with the semantic graphs. Empirical studies on several downstream tasks show that proposed model outperforms strong pre-trained baselines. The results also show the effectiveness of proposed semantic graphs for pre-trained model.
previous article Personalized federated learning with model interpolation among client clusters and its application in smart home
next article Cross-platform product matching based on entity alignment of knowledge graph with raea model

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
Lai, G., Xie, Q., Liu, H., Yang, Y., Hovy, E.: RACE: Large-scale ReAding Comprehension Dataset From Examinations (2017)
2.
Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: SQuAD: 100,000+ Questions for Machine Comprehension of Text (2016)
3.
Zhang, L., Wang, S., Liu, B.: Deep learning for sentiment analysis : a survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 1253 (2018)
4.
Zeng, Z., Zhou, W., Liu, X., Song, Y.: A variational approach to weakly supervised document-level multi-aspect sentiment classification. In: Conference of the North (2019)
5.
6.
7.
8.
9.
Tian, H., Gao, C., Xiao, X., Liu, H., He, B., Wu, H., Wang, H., Wu, F.: Skep: Sentiment knowledge enhanced pre-training for sentiment analysis. arXiv:2005.​05635 (2020)
10.
Ke, P., Ji, H., Liu, S., Zhu, X., Huang, M.: Sentilare: Linguistic knowledge enhanced language representation for sentiment analysis. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 6975–6988 (2020)
11.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: Roberta: A robustly optimized bert pretraining approach. arXiv:1907.​11692 (2019)
12.
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.​04805 (2018)
13.
Zhang, X., Zhao, J.J., LeCun, Y.: Character-level convolutional networks for text classification. CoRR arXiv:abs/​1509.​01626 (2015)
14.
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C.D., Ng, A.Y., Potts, C.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631–1642 (2013)
15.
Maria, P., Dimitris, G., John, P., Harris, P., Ion, A., Suresh, M.: Semeval-2014 task 4: As- pect based sentiment analysis. In: Proceedings of the 8th International Workshop on Semantic Evaluation, pp. 27–35 (2014)
16.
Wiebe, J., Wilson, T., Cardie, C.: Annotating expressions of opinions and emotions in language. Lang. Resour. Eval. 39(2), 165–210 (2005)CrossRef
17.
Wilson, T.A.: Fine-Grained Subjectivity and Sentiment Analysis: Recognizing the Intensity, Polarity, and Attitudes of Private States. PhD thesis, University of Pittsburgh (2008)
18.
Marasovic, A., Frank, A.: SRL4ORL: Improving opinion role labelling using multi-task learning with semantic role labeling. CoRR arXiv:abs/​1711.​00768 (2017)
19.
Taboada, M., Brooke, J., Tofiloski, M., Voll, K.D., Stede, M.: Lexicon-based methods for sentiment analysis. Comput. Linguist. 37(2), 267–307 (2011)CrossRef
20.
Shin, B., Lee, T., Choi, J.D.: Lexicon integrated cnn models with attention for sentiment analysis. Alexandra Balahur Dobrescu, 149–158 (2017)
21.
Vo, D.-T., Zhang, Y.: Target-Dependent Twitter Sentiment Classification with Rich Automatic Features. In: Twenty-Fourth International Joint Conference on Artificial Intelligence (2015)
22.
Li, X., Lam, W.: Deep multi-task learning for aspect term extraction with memory interaction. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2886–2892 (2017)
23.
Gui, L., Hu, J., He, Y., Xu, R., Lu, Q., Du, J.: A question answering approach to emotion cause extraction. arXiv:1708.​05482 (2017)
24.
Fan, C., Yan, H., Du, J., Gui, L., Bing, L., Yang, M., Xu, R., Mao, R.: A knowledge regularized hierarchical approach for emotion cause analysis. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 5618–5628 (2019)
25.
Turney, P.D.: Thumbs up or thumbs down? semantic orientation applied to unsupervised classification of reviews. arXiv:cs/​0212032 (2002)
26.
27.
Bakshi, R.K., Kaur, N., Kaur, R., Kaur, G.: Opinion mining and sentiment analysis. In: 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom), pp 452–455. IEEE (2016)
28.
Agarwal, A., Xie, B., Vovsha, I., Rambow, O., Passonneau, R.J.: Sentiment analysis of twitter data. In: Proceedings of the Workshop on Language in Social Media (LSM 2011), pp. 30–38 (2011)
29.
Shin, B., Lee, T., Choi, J.D.: Lexicon integrated cnn models with attention for sentiment analysis. arXiv:1610.​06272 (2016)
30.
Qian, Q., Huang, M., Lei, J., Zhu, X.: Linguistically regularized lstms for sentiment classification. arXiv:1611.​03949 (2016)
31.
Zeng, Z., Zhou, W., Liu, X., Song, Y.: A variational approach to weakly supervised document-level multi-aspect sentiment classification. arXiv:1904.​05055 (2019)
32.
Yang, Z., Salakhutdinov, R., Cohen, W.W.: Transfer learning for sequence tagging with hierarchical recurrent networks. arXiv:1703.​06345 (2017)
33.
Ding, Y., Yu, J., Jiang, J.: Recurrent neural networks with auxiliary labels for cross-domain opinion target extraction. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)
34.
Wang, X., Gao, T., Zhu, Z., Liu, Z., Li, J., Tang, J.: Kepler: A unified model for knowledge embedding and pre-trained language representation. arXiv:1911.​06136 (2019)
35.
Peters, M.E., Neumann, M., Logan IV, R.L., Schwartz, R., Joshi, V., Singh, S., Smith, N.A.: Knowledge enhanced contextual word representations. arXiv:1909.​04164 (2019)
36.
Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., Liu, Q.: Ernie: Enhanced language representation with informative entities. arXiv:1905.​07129(2019)
37.
Liu, W., Zhou, P., Zhao, Z., Wang, Z., Ju, Q., Deng, H., Wang, P.: K-bert: Enabling language representation with knowledge graph. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 2901–2908 (2020)
38.
Wang, R., Tang, D., Duan, N., Wei, Z., Huang, X., Cao, C., Jiang, D., Zhou, M., et al.: K-adapter: Infusing knowledge into pre-trained models with adapters. arXiv:2002.​01808 (2020)
Metadata
Title
SGPT: Semantic graphs based pre-training for aspect-based sentiment analysis
Authors
Qian Yong
Chen Chen
Zhongqing Wang
Rong Xiao
Haihong Tang
Publication date
17-01-2023
Publisher
Springer US
Published in
World Wide Web / Issue 4/2023
Print ISSN: 1386-145X
Electronic ISSN: 1573-1413
DOI
https://doi.org/10.1007/s11280-022-01123-1

Other articles of this Issue 4/2023

World Wide Web 4/2023 Go to the issue

OriginalPaper

Modeling the social influence of COVID-19 via personalized propagation with deep learning

OriginalPaper

How the four-nodes motifs work in heterogeneous node representation?

OriginalPaper

Spatiotemporal contrastive modeling for video moment retrieval

OriginalPaper

Graph attention network based detection of causality for textual emotion-cause pair

OriginalPaper

HGNN-ETA: Heterogeneous graph neural network enriched with text attribute

OriginalPaper

Durable queries over non-synchronized temporal data

Premium Partner

    Image Credits