nach oben

Neural Processing Letters

Erschienen in:

22.06.2022

A Word-Concept Heterogeneous Graph Convolutional Network for Short Text Classification

verfasst von: Shigang Yang, Yongguo Liu, Yun Zhang, Jiajing Zhu

Erschienen in: Neural Processing Letters | Ausgabe 1/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Text classification is an important task in natural language processing. However, most of the existing models focus on long texts, and their performance in short texts is not satisfied due to the problem of data sparsity. To solve this problem, recent studies have introduced the concepts of words to enrich the representation of short texts. However, these methods ignore the interactive information between words and concepts and lead introduced concepts to be noises unsuitable for semantic understanding. In this paper, we propose a new model called word-concept heterogeneous graph convolution network (WC-HGCN) to introduce interactive information between words and concepts for short text classification. WC-HGCN develops words and relevant concepts and adopts graph convolution networks to learn the representation with interactive information. Furthermore, we design an innovative learning strategy, which can make full use of the introduced concept information. Experimental results on seven real short text datasets show that our model outperforms latest baseline methods.

Vorheriger Artikel A Perceptually Important Points Approach Based on Imputation Clustering with Weighted Distance Techniques for Big Data Reduction in Internet of Things Cloud

Nächster Artikel A Method of Sharing Sentence Vectors for Opinion Triplet Extraction

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

https://concept.research.microsoft.com/Home/API.

https://drive.google.com/file/d/1Li2QyQm8dCVT81jBLmbT0tQm6HNVzTyA.

https://www.cs.york.ac.uk/semeval-2013/task2/.

http://nlp.stanford.edu/data/glove.6B.zip.

Alsmadi IM, Gan KH (2019) Review of short-text classification. Int J Web Inf Syst 15(2):155–182. https://doi.org/10.1108/IJWIS-12-2017-0083CrossRef

Arevian G (2007) Recurrent neural networks for robust real-world text classification. In: 2007 IEEE / WIC/ACM international conference on web intelligence, WI 2007, 2–5 November 2007, Silicon Valley, CA, USA, Main Conference Proceedings, IEEE Computer Society, pp 326–329. https://doi.org/10.1109/WI.2007.126

Batal I, Hauskrecht M (2009) Boosting KNN text classification accuracy by using supervised term weighting schemes. In: Cheung DW, Song I, Chu WW, Hu X, Lin JJ (eds) Proceedings of the 18th ACM conference on information and knowledge management, CIKM 2009, Hong Kong, China, November 2–6, 2009, ACM, pp 2041–2044. https://doi.org/10.1145/1645953.1646296

Chen J, Hu Y, Liu J, Xiao Y, Jiang H (2019) Deep short text classification with knowledge powered attention. In: The thirty-third aaai conference on artificial intelligence, AAAI 2019, the thirty-first innovative applications of artificial intelligence conference, IAAI 2019, The Ninth AAAI symposium on educational advances in artificial intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27–February 1, 2019, AAAI Press, pp 6252–6259. https://doi.org/10.1609/aaai.v33i01.33016252

Dilrukshi I, de Zoysa K (2014) A feature selection method for twitter news classification. Int J Mach Learn Comput 4(4):365CrossRef

Ding W, Yu S, Wang Q, Yu J, Guo Q (2008) A novel naive bayesian text classifier. In: Yu F, Luo Q (eds) International symposium on information processing, ISIP 2008/international pacific workshop on web mining, and web-based application, WMWA 2008, Moscow, Russia, 23–25 May 2008, IEEE Computer Society, pp 78–82.https://doi.org/10.1109/ISIP.2008.54

Du C, Huang L (2018) Text classification research with attention-based recurrent neural networks. Int J Comput Commun Control 13(1):50–61. https://doi.org/10.15837/ijccc.2018.1.3142CrossRef

Han E, Karypis G, Kumar V (2001) Text categorization using weight adjusted k-nearest neighbor classification. In: Cheung DW, Williams GJ, Li Q (eds) Knowledge discovery and data mining—PAKDD 2001, 5th Pacific-Asia Conference, Hong Kong, China, April 16–18, 2001, Proceedings, Springer, Lecture Notes in Computer Science, vol 2035, pp 53–65.https://doi.org/10.1007/3-540-45357-1_9

Hindi KME, Aljulaidan RR, AlSalman H (2020) Lazy fine-tuning algorithms for naïve bayesian text classification. Appl Soft Comput 96:106652.https://doi.org/10.1016/j.asoc.2020.106652

10.

Hu L, Yang T, Shi C, Ji H, Li X (2019) Heterogeneous graph attention networks for semi-supervised short text classification. In: Inui K, Jiang J, Ng V, Wan X (eds) Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3–7, 2019, Association for Computational Linguistics, pp 4820–4829.https://doi.org/10.18653/v1/D19-1488

11.

Huang L, Ma D, Li S, Zhang X, Wang H (2019) Text level graph neural network for text classification. In: Inui K, Jiang J, Ng V, Wan X (eds) Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3–7, 2019, Association for Computational Linguistics, pp 3442–3448.https://doi.org/10.18653/v1/D19-1345

12.

Islam MZ, Liu J, Li J, Liu L, Kang W (2019) A semantics aware random forest for text classification. In: Zhu W, Tao D, Cheng X, Cui P, Rundensteiner EA, Carmel D, He Q, Yu JX (eds) Proceedings of the 28th ACM international conference on information and knowledge management, CIKM 2019, Beijing, China, November 3–7, 2019, ACM, pp 1061–1070. https://doi.org/10.1145/3357384.3357891

13.

Joulin A, Grave E, Bojanowski P, Mikolov T (2017) Bag of tricks for efficient text classification. In: Lapata M, Blunsom P, Koller A (eds) Proceedings of the 15th conference of the european chapter of the association for computational linguistics, EACL 2017, Valencia, Spain, April 3–7, 2017, Volume 2: Short Papers, Association for Computational Linguistics, pp 427–431.https://doi.org/10.18653/v1/e17-2068

14.

Keerthi SS (2005) Generalized LARS as an effective feature selection tool for text classification with svms. In: Raedt LD, Wrobel S (eds) Machine learning, proceedings of the twenty-second international conference (ICML 2005), Bonn, Germany, August 7–11, 2005, ACM, ACM International Conference Proceeding Series, vol 119, pp 417–424.https://doi.org/10.1145/1102351.1102404

15.

Kim Y (2014) Convolutional neural networks for sentence classification. In: Moschitti A, Pang B, Daelemans W (eds) Proceedings of the 2014 conference on empirical methods in natural language processing, EMNLP 2014, October 25–29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, ACL, pp 1746–1751.https://doi.org/10.3115/v1/d14-1181

16.

Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: Bengio Y, LeCun Y (eds) 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings.http://arxiv.org/abs/1412.6980

17.

Li C, Ouyang J, Li X (2019) Classifying extremely short texts by exploiting semantic centroids in word mover’s distance space. In: Liu L, White RW, Mantrach A, Silvestri F, McAuley JJ, Baeza-Yates R, Zia L (eds) The world wide web conference, WWW 2019, San Francisco, CA, USA, May 13–17, 2019, ACM, pp 939–949.https://doi.org/10.1145/3308558.3313397

18.

Li Y, Liu B (2020) A new vector representation of short texts for classification. Int Arab J Inf Technol 17(2):241–249. https://doi.org/10.34028/iajit/17/2/12CrossRef

19.

Li Y, Tarlow D, Brockschmidt M, Zemel RS (2016) Gated graph sequence neural networks. In: Bengio Y, LeCun Y (eds) 4th international conference on learning representations, ICLR 2016, San Juan, Puerto Rico, May 2–4, 2016, Conference Track Proceedings.http://arxiv.org/abs/1511.05493

20.

Liu P, Qiu X, Huang X (2016) Recurrent neural network for text classification with multi-task learning. In: Kambhampati S (ed) Proceedings of the twenty-fifth international joint conference on artificial intelligence, IJCAI 2016, New York, NY, USA, 9–15 July 2016, IJCAI/AAAI Press, pp 2873–2879.http://www.ijcai.org/Abstract/16/408

21.

Liu X, You X, Zhang X, Wu J, Lv P (2020) Tensor graph convolutional networks for text classification. In: The thirty-fourth AAAI conference on artificial intelligence, AAAI 2020, the thirty-second innovative applications of artificial intelligence conference, IAAI 2020, the tenth AAAI symposium on educational advances in artificial intelligence, EAAI 2020, New York, NY, USA, February 7–12, 2020, AAAI Press, pp 8409–8416.https://aaai.org/ojs/index.php/AAAI/article/view/6359

22.

Niu G, Xu H, He B, Xiao X, Wu H, Gao S (2019) Enhancing local feature extraction with global representation for neural text classification. In: Inui K, Jiang J, Ng V, Wan X (eds) Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3–7, 2019, Association for Computational Linguistics, pp 496–506.https://doi.org/10.18653/v1/D19-1047

23.

Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Moschitti A, Pang B, Daelemans W (eds) Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, ACL, pp 1532–1543.https://doi.org/10.3115/v1/d14-1162

24.

Salles T, Gonçalves MA, Rodrigues V, da Rocha LC (2018) Improving random forests by neighborhood projection for effective text classification. Inf Syst 77:1–21CrossRef

25.

Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2009) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80. https://doi.org/10.1109/TNN.2008.2005605CrossRef

26.

Shanahan JG, Roma N (2003) Improving SVM text classification performance through threshold adjustment. In: Lavrac N, Gamberger D, Todorovski L, Blockeel H (eds) Machine learning: ECML 2003, 14th European conference on machine learning, Cavtat-Dubrovnik, Croatia, September 22–26, 2003, Proceedings, Springer, Lecture Notes in Computer Science, vol 2837, pp 361–372.https://doi.org/10.1007/978-3-540-39857-8_33

27.

Song G, Ye Y, Du X, Huang X, Bie S (2014) Short text classification: a survey. J Multim 9(5):635–643. https://doi.org/10.4304/jmm.9.5.635-643CrossRef

28.

Tang J, Qu M, Mei Q (2015) PTE: predictive text embedding through large-scale heterogeneous text networks. In: Cao L, Zhang C, Joachims T, Webb GI, Margineantu DD, Williams G (eds) Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, Sydney, NSW, Australia, August 10–13, 2015, ACM, pp 1165–1174.https://doi.org/10.1145/2783258.2783307

29.

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Guyon I, von Luxburg U, Bengio S, Wallach HM, Fergus R, Vishwanathan SVN, Garnett R (eds) Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, December 4–9, 2017, Long Beach, CA, USA, pp 5998–6008.http://papers.nips.cc/paper/7181-attention-is-all-you-need

30.

Wang S, Li D, Song X, Wei Y, Li H (2011) A feature selection method based on improved fisher’s discriminant ratio for text sentiment classification. Expert Syst Appl 38(7):8696–8702. https://doi.org/10.1016/j.eswa.2011.01.077CrossRef

31.

Xu B, Huang JZ, Williams GJ, Li MJ, Ye Y (2012) Hybrid random forests: advantages of mixed trees in classifying text data. In: Tan P, Chawla S, Ho CK, Bailey J (eds) Advances in knowledge discovery and data mining—16th Pacific-Asia conference, PAKDD 2012, Kuala Lumpur, Malaysia, May 29-June 1, 2012, Proceedings, Part I, Springer, Lecture Notes in Computer Science, vol 7301, pp 147–158.https://doi.org/10.1007/978-3-642-30217-6_13

32.

Xu J, Cai Y, Wu X, Lei X, Huang Q, Leung H, Li Q (2020) Incorporating context-relevant concepts into convolutional neural networks for short text classification. Neurocomputing 386:42–53. https://doi.org/10.1016/j.neucom.2019.08.080CrossRef

33.

Xu S (2018) Bayesian naïve bayes classifiers to text classification. J Inf Sci 44(1):48–59. https://doi.org/10.1177/0165551516677946MathSciNetCrossRef

34.

Yang Y, Wang H, Zhu J, Wu Y, Jiang K, Guo W, Shi W (2020) Dataless short text classification based on biterm topic model and word embeddings. In: Bessiere C (ed) Proceedings of the twenty-ninth international joint conference on artificial intelligence, IJCAI 2020, ijcai.org, pp 3969–3975.https://doi.org/10.24963/ijcai.2020/549

35.

Yao L, Mao C, Luo Y (2019) Graph convolutional networks for text classification. In: The thirty-third AAAI conference on artificial intelligence, AAAI 2019, the thirty-first innovative applications of artificial intelligence conference, IAAI 2019, the Ninth AAAI symposium on educational advances in artificial intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27—February 1, 2019, AAAI Press, pp 7370–7377.https://doi.org/10.1609/aaai.v33i01.33017370

36.

Zeng J, Li J, Song Y, Gao C, Lyu MR, King I (2018) Topic memory networks for short text classification. In: Riloff E, Chiang D, Hockenmaier J, Tsujii J (eds) Proceedings of the 2018 conference on empirical methods in natural language processing, Brussels, Belgium, October 31—November 4, 2018, Association for Computational Linguistics, pp 3120–3131.https://doi.org/10.18653/v1/d18-1351

37.

Zhang H, Ni W, Zhao M, Lin Z (2019) Cluster-gated convolutional neural network for short text classification. In: Bansal M, Villavicencio A (eds) Proceedings of the 23rd Conference on Computational Natural Language Learning, CoNLL 2019, Hong Kong, China, November 3-4, 2019, Association for Computational Linguistics, pp 1002–1011.https://doi.org/10.18653/v1/K19-1094

38.

Zhang Y, Yu X, Cui Z, Wu S, Wen Z, Wang L (2020) Every document owns its structure: Inductive text classification via graph neural networks. In: Jurafsky D, Chai J, Schluter N, Tetreault JR (eds) Proceedings of the 58th annual meeting of the association for computational linguistics, ACL 2020, Online, July 5–10, 2020, Association for Computational Linguistics, pp 334–339. https://doi.org/10.18653/v1/2020.acl-main.31

Titel: A Word-Concept Heterogeneous Graph Convolutional Network for Short Text Classification
verfasst von: Shigang Yang
Yongguo Liu
Yun Zhang
Jiajing Zhu
Publikationsdatum: 22.06.2022
Verlag: Springer US
Erschienen in: Neural Processing Letters / Ausgabe 1/2023
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI: https://doi.org/10.1007/s11063-022-10906-6

Neuer Inhalt

Bildnachweise

VDI-Icon, Profil Icon, inhalt2, Springer Professional Modul/© Springer Fachmedien Wiesbaden GmbH, Zukunftswerkstatt Sales Excellence_ieS/© Springer Fachmedien Wiesbaden GmbH, Search Icon, Banner Hanser, Strompreise/© vejaa / stock.adobe.com, Bunte Männchen, die Kunden darstelle, werden von einem riesigen Magneten angezogen. /© Oleksiy Mark, Dr. Daniel Schneider/© Fraunhofer IESE, Zeitschrift Wissensmanagement Cover, PatentFit-Logo/© Springer Fachmedien Wiesbaden GmbH, Zukunftswerkstatt Sales Excellence 2024/© AndreyPopov / Getty Images / iStock, 2023_Antrieb/© supervisuell, ATZ-Webinar: Prototypenfreie Entwicklung durch Offline- und Driver-in-the-Loop-HiL-Tests /© (c) VI-grade

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 1/2023

AI-Based Self-Learning System in Distributed Structural Health Monitoring and Control

A Hybrid VAE Based Network Embedding Method for Biomedical Relation Mining

Retraction Note: Cerebrum Tumor Segmentation of High Resolution Magnetic Resonance Images Using 2D-Convolutional Network with Skull Stripping

An Adaboost Support Vector Machine Based Harris Hawks Optimization Algorithm for Intelligent Quotient Estimation from MRI Images

ChaInNet: Deep Chain Instance Segmentation Network for Panoptic Segmentation

Weighted Pseudo Almost-Automorphic Solutions of Quaternion-Valued RNNs With Mixed Delays

Neuer Inhalt

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.