Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
International Journal of Machine Learning and Cybernetics
Published in: International Journal of Machine Learning and Cybernetics 5/2023

18-11-2022 | Original Article

Multi-label sequence generating model via label semantic attention mechanism

Authors: Xiuling Zhang, Xiaofei Tan, Zhaoci Luo, Jun Zhao

Published in: International Journal of Machine Learning and Cybernetics | Issue 5/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In recent years, a new attempt has been made to capture label co-occurrence by applying the sequence-to-sequence (Seq2Seq) model to multi-label text classification (MLTC). However, existing approaches frequently ignore the semantic information contained in the labels themselves. Besides, the Seq2Seq model is susceptible to the negative impact of label sequence order. Furthermore, it has been demonstrated that the traditional attention mechanism underperforms in MLTC. Therefore, we propose a novel Seq2Seq model with a different label semantic attention mechanism (S2S-LSAM), which generates fused information containing label and text information through the interaction of label semantics and text features in the label semantic attention mechanism. With the fused information, our model can select the text features that are most relevant to the labels more effectively. A combination of the cross-entropy loss function and the policy gradient-based loss function is employed to reduce the label sequence order effect. The experiments show that our model outperforms the baseline models.
previous article Multi-level regularization-based unsupervised multi-view feature selection with adaptive graph learning
next article Generative face inpainting hashing for occluded face retrieval

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
Show more products
Footnotes
Literature
1.
Zhang ML (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26:1819–1837CrossRef
2.
Du J, Chen Q, Peng Y, Xiang Y, Tao C, Lu Z (2019) Ml-net: multi-label classification of biomedical texts with deep neural networks. J Am Med Inform Assoc 26(11):1279–1285CrossRef
3.
Katakis I, Tsoumakas G, Vlahavas I (2008) Multilabel text classification for automated tag suggestion. In: Proceedings of the ECML/PKDD, vol. 18, p. 5 . Citeseer
4.
Cambria E, Olsher D, Rajagopal D (2014) Senticnet 3: a common and common-sense knowledge base for cognition-driven sentiment analysis. In: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, pp. 1515–1521
5.
MJ B (2014) Large scale multi-label text classification with semantic word vectors. In: Technical Report, pp. 1–8
6.
Chalkidis I, Fergadiotis E, Malakasiotis P, Androutsopoulos I (2019) Large-scale multi-label text classification on eu legislation. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 6314–6322
7.
Nam J, KHFJ Mencía EL (2017) Maximizing subset accuracy with recurrent neural networks in multi-label classification. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 5419–5429. Curran Associates Inc., Red Hook, NY, USA
8.
Yang PC, Lwea Sun X (2018) Sgm: sequence generation model for multi-label classification. In: In Proceedings of the 27th International Conference on Computational Linguistics, pp. 3915–3926
9.
Yang P, Luo F, Ma S, Lin J, Sun X (2019) A deep reinforced sequence-to-set model for multi-label classification. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5252–5258
10.
Lin J, Su Q, Yang P, Ma S, Sun X (2018) Semantic-unit-based dilated convolution for multi-label text classification. arXiv preprint arXiv:​1808.​08561
11.
Wang D, Cui P, Zhu W (2016) Structural deep network embedding. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1225–1234
12.
Rennie SJ, Marcheret E, Mroueh Y, Ross J, Goel V (2016) Self-critical sequence training for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern RECOGNITION, pp. 7008–7024
13.
Yu L, Zhang W, Wang J, Yu Y (2017) Seqgan: Sequence generative adversarial nets with policy gradient. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31
14.
Zhang M-L, Zhou Z-H (2006) Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338–1351CrossRef
15.
Nam J, Kim J (2014) Large-scale multi-label text classification-revisiting neural networks. Joint European Conference on machine learning and knowledge discovery in databases. Springer, Berlin, pp 437–452CrossRef
16.
Y K (2014) Convolutional neural networks for sentence classification. In: EMNLP 2014-2014 Conference on Empirical Methods in Natural Language Processing, pp. 437–452 . Springer
17.
Kurata G, Xiang B, Zhou B (2016) Improved neural network-based multi-label classification with better initialization leveraging label co-occurrence. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 521–526
18.
Zhang X, Zhang Q-W, Yan Z, Liu R, Cao Y (2021) Enhancing label correlation feedback in multi-label text classification via multi-task learning. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 1190–1200
19.
Maltoudoglou L, Paisios A, Lenc L, Martínek J, Král P, Papadopoulos H (2022) Well-calibrated confidence measures for multi-label text classification with a large number of labels. Pattern Recogn 122:108271CrossRef
20.
Xiao L, Zhang X, Jing L, Huang C, Song M (2021) Does head label help for long-tailed multi-label text classification. Proc AAAI Conf Artif Intell 35:14103–14111
21.
Chen G, Ye D, Xing Z, Chen J, Cambria E (2017) Ensemble application of convolutional and recurrent neural networks for multi-label text categorization. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 2377–2383 . IEEE
22.
Pappas N, Henderson J (2019) Gile: a generalized input-label embedding for text classification. Trans Assoc Comput Linguist 7:139–155CrossRef
23.
Wang G, Li C, Wang W, Zhang Y, Shen D, Zhang X, Henao R, Carin L (2018) Joint embedding of words and labels for text classification. arXiv preprint arXiv:​1805.​04174
24.
Du C, Chen Z, Feng F, Zhu L, Gan T, Nie L (2019) Explicit interaction model towards text classification. Proc AAAI Conf Artif Intell 33:6359–6366
25.
Zhang W, Yan J, Wang X, Zha H (2018) Deep extreme multi-label learning. In: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, pp. 100–107
26.
Huang X, Chen B, Xiao L, Yu J, Jing L (2021) Label-aware document representation via hybrid attention for extreme multi-label text classification. Neural Process Lett 2:1–17
27.
Zhang X, Xu J, Soh C, Chen L (2022) La-hcn: Label-based attention for hierarchical multi-label text classification neural network. Expert Syst Appl 187:115922CrossRef
28.
Peng H, Li J, Wang S, Wang L, Gong Q, Yang R, Li B, Philip SY, He L (2019) Hierarchical taxonomy-aware and attentional graph capsule rcnns for large-scale multi-label text classification. IEEE Trans Knowl Data Eng 33(6):2505–2519CrossRef
29.
Wiseman S, Rush AM (2016) Sequence-to-sequence learning as beam-search optimization. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1296–1306
30.
Sutton RS, McAllester D, Singh S, Mansour Y (1999) Policy gradient methods for reinforcement learning with function approximation. Advances in neural information processing systems 12
31.
Lewis DD, Yang Y, Russell-Rose T, Li F (2004) Rcv1: a new benchmark collection for text categorization research. J Mach Learn Res 5:361–397
32.
33.
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958MathSciNetMATH
34.
Pascanu R, Mikolov T, Bengio Y (2013) On the difficulty of training recurrent neural networks. In: International Conference on Machine Learning, pp. 1310–1318 . PMLR
35.
Schapire RE, Singer Y (1999) Improved boosting algorithms using confidence-rated predictions. Mach Learn 37(3):297–336CrossRefMATH
36.
Manning C, Raghavan P, Schütze H (2010) Introduction to information retrieval. Nat Lang Eng 16(1):100–103MATH
37.
Boutell MR (2004) Learning multi-label scene classification. Pattern Recogn 37(9):1757–1771CrossRef
38.
Tsoumakas GKI (2006) Multi-label classification: an overview. Int J Data Warehouse Min 3(3):1–13
39.
Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:​1409.​0473
40.
Liu H, Yuan C, Wang X (2020) Label-wise document pre-training for multi-label text classification. In: CCF International Conference on Natural Language Processing and Chinese Computing, pp. 641–653 . Springer
41.
Pal A, Selvakumar M, Sankarasubbu M (2020) Multi-label text classification using attention-based graph neural network. arXiv preprint arXiv:​2003.​11644
42.
Wang R, Ridley R, Qu W, Dai X (2021) A novel reasoning mechanism for multi-label text classification. Inf Process Manag 58(2):102441CrossRef
43.
Chen Z, Ren J (2021) Multi-label text classification with latent word-wise label information. Appl Intell 51(2):966–979CrossRef
Metadata
Title
Multi-label sequence generating model via label semantic attention mechanism
Authors
Xiuling Zhang
Xiaofei Tan
Zhaoci Luo
Jun Zhao
Publication date
18-11-2022
Publisher
Springer Berlin Heidelberg
Published in
International Journal of Machine Learning and Cybernetics / Issue 5/2023
Print ISSN: 1868-8071
Electronic ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-022-01722-4

Other articles of this Issue 5/2023

International Journal of Machine Learning and Cybernetics 5/2023 Go to the issue

Original Article

A novel consensus model with probabilistic linguistic preference relation for the utilization mode selection of renewable energy sources

Original Article

Fusing collaborative transformation with temporally aware target interaction networks for sequential recommendation

Original Article

Contrastive embedding-based feature generation for generalized zero-shot learning

Original Article

KAMTFENet: a fall detection algorithm based on keypoint attention module and temporal feature extraction

Original Article

Maximum density minimum redundancy based hypergraph regularized support vector regression

Original Article

RADCU-Net: residual attention and dual-supervision cascaded U-Net for retinal blood vessel segmentation