nach oben

Erschienen in:

2021 | OriginalPaper | Buchkapitel

Integrating N-Gram Features into Pre-trained Model: A Novel Ensemble Model for Multi-target Stance Detection

verfasst von : Pengyuan Chen, Kai Ye, Xiaohui Cui

Erschienen in: Artificial Neural Networks and Machine Learning – ICANN 2021

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Multi-target stance detection in tweets aims to detect the stance of given texts towards a specific target entity. Most existing models on stance detection consider word embedding as input, however, recent developments pointed out that it would be beneficial to incorporate feature-based information appropriately. Motivated by the strong performance of the pre-trained models in many Natural Language Processing field, and n-gram features that have been proved to be effective in prior competition, we present a novel combination module to obtain both advantages. This paper has proposed a pre-trained model integrated with n-gram features module (PMINFM) to better utilize multi-scale feature representation information and semantic features. Then connect it to a Bidirectional Long Short-Term Memory networks with target-specific attention mechanism. The experimental results show that our proposed model outperforms other baseline models in the SemEval-2016 stance detection dataset and achieves state-of-the-art performance.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel STGATP: A Spatio-Temporal Graph Attention Network for Long-Term Traffic Prediction

Nächstes Kapitel Hierarchical Ensemble for Multi-view Clustering

Aldayel, A., Magdy, W.: Your stance is exposed! analysing possible factors for stance detection on social media. Proc. ACM Hum.-Comput. Interact. 3(CSCW), 1–20 (2019)

Alghanmi, I., Espinosa-Anke, L., Schockaert, S.: Combining BERT with static word embeddings for categorizing social media (2020)

Anand, P., Walker, M., Abbott, R., Tree, J.E.F., Bowmani, R., Minor, M.: Cats rule and dogs drool!: classifying stance in online debate. In: Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis (WASSA 2011), pp. 1–9 (2011)

Augenstein, I., Rocktäschel, T., Vlachos, A., Bontcheva, K.: Stance detection with bidirectional conditional encoding. arXiv preprint arXiv:1606.05464 (2016)

Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)

Baly, R., et al.: A characterization study of Arabic twitter data with a benchmarking for state-of-the-art opinion mining models. In: Proceedings of the Third Arabic Natural Language Processing Workshop, pp. 110–118 (2017)

Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017)CrossRef

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

Du, J., Xu, R., He, Y., Gui, L.: Stance classification with target-specific neural attention networks. In: International Joint Conferences on Artificial Intelligence (2017)

10.

Dulhanty, C., Deglint, J.L., Daya, I.B., Wong, A.: Taking a stance on fake news: towards automatic disinformation assessment via deep bidirectional transformer language models for stance detection. arXiv preprint arXiv:1911.11951 (2019)

11.

Li, Y., Caragea, C.: Multi-task stance detection with sentiment and stance lexicons. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 6300–6306 (2019)

12.

Liu, Y., et al.: Roberta: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)

13.

Mohammad, S., Kiritchenko, S., Sobhani, P., Zhu, X., Cherry, C.: SemEval-2016 task 6: detecting stance in tweets. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 31–41 (2016)

14.

Mohammad, S.M., Sobhani, P., Kiritchenko, S.: Stance and sentiment in tweets. ACM Trans. Internet Technol. (TOIT) 17(3), 1–23 (2017)CrossRef

15.

Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetMATH

16.

Prakash, A., Madabushi, H.T.: Incorporating count-based features into pre-trained models for improved stance detection. arXiv preprint arXiv:2010.09078 (2020)

17.

Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving language understanding by generative pre-training (2018)

18.

Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45(11), 2673–2681 (1997)CrossRef

19.

Somasundaran, S., Wiebe, J.: Recognizing stances in ideological on-line debates. In: Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, pp. 116–124 (2010)

20.

Sun, Q., Wang, Z., Zhu, Q., Zhou, G.: Stance detection with hierarchical attention network. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 2399–2409 (2018)

21.

Vlad, G.A., Tanase, M.A., Onose, C., Cercel, D.C.: Sentence-level propaganda detection in news articles with transfer learning and BERT-BiLSTM-capsule model. In: Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda, pp. 148–154 (2019)

22.

Walker, M., Anand, P., Abbott, R., Grant, R.: Stance classification using dialogic properties of persuasion. In: Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 592–596 (2012)

23.

Wang, L., Cardie, C.: Improving agreement and disagreement identification in online discussions with a socially-tuned sentiment lexicon. arXiv preprint arXiv:1606.05706 (2016)

24.

Wei, P., Mao, W., Zeng, D.: A target-guided neural memory model for stance detection in twitter. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2018)

25.

Wei, W., Zhang, X., Liu, X., Chen, W., Wang, T.: pkudblab at SemEval-2016 task 6: a specific convolutional neural network system for effective stance detection. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 384–388 (2016)

26.

Yang, R., Xie, W., Liu, C., Yu, D.: BLCU\_NLP at SemEval-2019 task 7: an inference chain-based GPT model for rumour evaluation. In: Proceedings of the 13th International Workshop on Semantic Evaluation, pp. 1090–1096 (2019)

27.

Yang, Y., Wu, B., Zhao, K., Guo, W.: Tweet stance detection: a two-stage DC-BiLSTM model based on semantic attention. In: 2020 IEEE Fifth International Conference on Data Science in Cyberspace (DSC), pp. 22–29. IEEE (2020)

28.

Zhang, C., Yamana, H.: WUY at SemEval-2020 task 7: combining BERT and Naïve Bayes-SVM for humor assessment in edited news headlines. In: Proceedings of the Fourteenth Workshop on Semantic Evaluation, pp. 1071–1076 (2020)

29.

Zhou, Y., Cristea, A.I., Shi, L.: Connecting targets to tweets: semantic attention-based model for target-specific stance detection. In: Bouguettaya, A., et al. (eds.) WISE 2017, Part I. LNCS, vol. 10569, pp. 18–32. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68783-4_2 CrossRef

Titel: Integrating N-Gram Features into Pre-trained Model: A Novel Ensemble Model for Multi-target Stance Detection
verfasst von: Pengyuan Chen
Kai Ye
Xiaohui Cui
Verlag: Springer International Publishing
Buch: Artificial Neural Networks and Machine Learning – ICANN 2021
Print ISBN: 978-3-030-86364-7

Electronic ISBN: 978-3-030-86365-4

Copyright-Jahr: 2021
DOI: https://doi.org/10.1007/978-3-030-86365-4_22

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner