Skip to main content
Erschienen in: International Journal of Machine Learning and Cybernetics 11/2023

01.07.2023 | Original Article

Multi-label feature selection via joint label enhancement and pairwise label correlations

verfasst von: Jinghua Liu, Songwei Yang, Yaojin Lin, Chenxi Wang, Cheng Wang, Jixiang Du

Erschienen in: International Journal of Machine Learning and Cybernetics | Ausgabe 11/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Multi-label feature selection(MFS) has gained in importance, and it is today confronted with the current need to process multi-semantic high-dimensional data. Recent studies usually figure out the MFS problems either simply assume that all associated labels are equally important for each instance; or that the labels are independent of each other. In many real-world applications, however, both cases may occur that the significance of each relevant label is generally different and label correlations are ubiquitous. Based on this observation, we propose a new algorithm, called FSEP, to perform MFS by considering label significance and pairwise label correlations. In FSEP, we first construct a label enhancement method that is able to obtain label distribution and further earn the information of label significance. Then, FSEP explores the influence mechanism of label correlations to features by using neighborhood mutual information and incorporates this influence into the process of feature evaluation. After that, a novel multi-label feature selection strategy, namely, Max-Relevance, Max-Contribution, and Min-Redundancy, is proposed, which achieves a favorable trade-off among feature relevance, the contribution of label correlations to features, and feature redundancy, simultaneously. Extensive experiments on both public and real-world datasets show that the proposed method achieves encouraging results compared with state-of-the-art MFS algorithms.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Weitere Produktempfehlungen anzeigen
Literatur
1.
Zurück zum Zitat Al-Salemi B, Ayob M, Kendall G, Noah SAM (2019) Multi-label arabic text categorization: A benchmark and baseline comparison of multi-label learning algorithms. Information Processing & Management 56(1):212–227CrossRef Al-Salemi B, Ayob M, Kendall G, Noah SAM (2019) Multi-label arabic text categorization: A benchmark and baseline comparison of multi-label learning algorithms. Information Processing & Management 56(1):212–227CrossRef
2.
Zurück zum Zitat Zhang M-L, Zhou Z-H (2013) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837CrossRef Zhang M-L, Zhou Z-H (2013) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837CrossRef
3.
Zurück zum Zitat Trohidis K, Tsoumakas G, Kalliris G, Vlahavas IP et al (2008) Multi-label classification of music into emotions. International Society for Music Information Retrieval Conference (ISMIR) 8:325–330 Trohidis K, Tsoumakas G, Kalliris G, Vlahavas IP et al (2008) Multi-label classification of music into emotions. International Society for Music Information Retrieval Conference (ISMIR) 8:325–330
4.
Zurück zum Zitat Qian W, Dong P, Wang Y, Dai S, Huang J (2022) Local rough set-based feature selection for label distribution learning with incomplete labels, International Journal of Machine Learning and Cybernetics 1-20 Qian W, Dong P, Wang Y, Dai S, Huang J (2022) Local rough set-based feature selection for label distribution learning with incomplete labels, International Journal of Machine Learning and Cybernetics 1-20
5.
Zurück zum Zitat Liu J, Lin Y, Wu S, Wang C (2018) Online multi-label group feature selection. Knowl-Based Syst 143:42–57CrossRef Liu J, Lin Y, Wu S, Wang C (2018) Online multi-label group feature selection. Knowl-Based Syst 143:42–57CrossRef
6.
Zurück zum Zitat Spolaôr N, Cherman EA, Monard MC, Lee HD (2013) Relieff for multi-label feature selection, in: 2013 Brazilian Conference on Intelligent Systems, IEEE, pp. 6–11 Spolaôr N, Cherman EA, Monard MC, Lee HD (2013) Relieff for multi-label feature selection, in: 2013 Brazilian Conference on Intelligent Systems, IEEE, pp. 6–11
7.
Zurück zum Zitat Wang J, Lin Y, Li Y, Wang Y, Xu M, Chen J (2022) Multi-label causal feature selection based on neighborhood mutual information, International Journal of Machine Learning and Cybernetics 1-14 Wang J, Lin Y, Li Y, Wang Y, Xu M, Chen J (2022) Multi-label causal feature selection based on neighborhood mutual information, International Journal of Machine Learning and Cybernetics 1-14
9.
Zurück zum Zitat Hu L, Gao L, Li Y, Zhang P, Gao W (2022) Feature-specific mutual information variation for multi-label feature selection. Inf Sci 593:449–471CrossRef Hu L, Gao L, Li Y, Zhang P, Gao W (2022) Feature-specific mutual information variation for multi-label feature selection. Inf Sci 593:449–471CrossRef
10.
Zurück zum Zitat Liu J, Lin Y, Ding W, Zhang H, Du J (2023) Fuzzy mutual information-based multi-label feature selection with label dependency and streaming labels. IEEE Trans Fuzzy Syst 31:77–89CrossRef Liu J, Lin Y, Ding W, Zhang H, Du J (2023) Fuzzy mutual information-based multi-label feature selection with label dependency and streaming labels. IEEE Trans Fuzzy Syst 31:77–89CrossRef
11.
Zurück zum Zitat Lee J, Kim D-W (2015) Memetic feature selection algorithm for multi-label classification. Inf Sci 293:80–96CrossRef Lee J, Kim D-W (2015) Memetic feature selection algorithm for multi-label classification. Inf Sci 293:80–96CrossRef
12.
Zurück zum Zitat Lin Y, Hu Q, Liu J, Duan J (2015) Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing 168:92–103CrossRef Lin Y, Hu Q, Liu J, Duan J (2015) Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing 168:92–103CrossRef
13.
Zurück zum Zitat Liu J, Li Y, Weng W, Zhang J, Chen B, Wu S (2020) Feature selection for multi-label learning with streaming label. Neurocomputing 387:268–278CrossRef Liu J, Li Y, Weng W, Zhang J, Chen B, Wu S (2020) Feature selection for multi-label learning with streaming label. Neurocomputing 387:268–278CrossRef
14.
Zurück zum Zitat Geng X (2016) Label distribution learning. IEEE Trans Knowl Data Eng 28(7):1734–1748CrossRef Geng X (2016) Label distribution learning. IEEE Trans Knowl Data Eng 28(7):1734–1748CrossRef
15.
Zurück zum Zitat Xu N, Liu Y-P, Geng X (2019) Label enhancement for label distribution learning. IEEE Trans Knowl Data Eng 33(4):1632–1643CrossRef Xu N, Liu Y-P, Geng X (2019) Label enhancement for label distribution learning. IEEE Trans Knowl Data Eng 33(4):1632–1643CrossRef
17.
Zurück zum Zitat Xu N, Shu J, Liu Y-P, Geng X (2020) Variational label enhancement, in: International Conference on Machine Learning, PMLR, 10597–10606 Xu N, Shu J, Liu Y-P, Geng X (2020) Variational label enhancement, in: International Conference on Machine Learning, PMLR, 10597–10606
18.
Zurück zum Zitat Li W, Chen J, Gao P, Huang Z (2022) Label enhancement with label-specific feature learning, International Journal of Machine Learning and Cybernetics 1–11 Li W, Chen J, Gao P, Huang Z (2022) Label enhancement with label-specific feature learning, International Journal of Machine Learning and Cybernetics 1–11
20.
Zurück zum Zitat Zhu Y, Kwok JT, Zhou Z-H (2017) Multi-label learning with global and local label correlation. IEEE Trans Knowl Data Eng 30(6):1081–1094CrossRef Zhu Y, Kwok JT, Zhou Z-H (2017) Multi-label learning with global and local label correlation. IEEE Trans Knowl Data Eng 30(6):1081–1094CrossRef
21.
Zurück zum Zitat Che X, Chen D, Mi J (2021) Feature distribution-based label correlation in multi-label classification. Int J Mach Learn Cybern 12(6):1705–1719CrossRef Che X, Chen D, Mi J (2021) Feature distribution-based label correlation in multi-label classification. Int J Mach Learn Cybern 12(6):1705–1719CrossRef
22.
Zurück zum Zitat Zhang J, Lin Y, Jiang M, Li S, Tang Y, Tan KC (2020) Multi-label feature selection via global relevance and redundancy optimization. In: International Joint Conference on Artificial Intelligence (IJCAI), pp 2512–2518 Zhang J, Lin Y, Jiang M, Li S, Tang Y, Tan KC (2020) Multi-label feature selection via global relevance and redundancy optimization. In: International Joint Conference on Artificial Intelligence (IJCAI), pp 2512–2518
24.
Zurück zum Zitat Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recogn 37(9):1757–1771CrossRef Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recogn 37(9):1757–1771CrossRef
25.
Zurück zum Zitat Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333–359MathSciNetCrossRef Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333–359MathSciNetCrossRef
26.
Zurück zum Zitat Huang J, Li G, Huang Q, Wu X (2016) Learning label-specific features and class-dependent labels for multi-label classification. IEEE Trans Knowl Data Eng 28(12):3309–3323CrossRef Huang J, Li G, Huang Q, Wu X (2016) Learning label-specific features and class-dependent labels for multi-label classification. IEEE Trans Knowl Data Eng 28(12):3309–3323CrossRef
27.
Zurück zum Zitat Zhao W, Kong S, Bai J, Fink D, Gomes C (2021) Hot-vae: Learning high-order label correlation for multi-label classification via attention-based variational autoencoders. Proceedings of the AAAI Conference on Artificial Intelligence 35:15016–15024CrossRef Zhao W, Kong S, Bai J, Fink D, Gomes C (2021) Hot-vae: Learning high-order label correlation for multi-label classification via attention-based variational autoencoders. Proceedings of the AAAI Conference on Artificial Intelligence 35:15016–15024CrossRef
28.
Zurück zum Zitat Elisseeff A, Weston J (2001) A kernel method for multi-labeled classification. Adv Neural Inf Process Syst 14:1–7 Elisseeff A, Weston J (2001) A kernel method for multi-labeled classification. Adv Neural Inf Process Syst 14:1–7
29.
Zurück zum Zitat Li Y, Song Y, Luo J (2017) Improving pairwise ranking for multi-label image classification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3617–3625 Li Y, Song Y, Luo J (2017) Improving pairwise ranking for multi-label image classification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3617–3625
30.
Zurück zum Zitat Xie M-K, Huang S-J (2021) Multi-label learning with pairwise relevance ordering. Adv Neural Inf Process Syst 34:23545–23556 Xie M-K, Huang S-J (2021) Multi-label learning with pairwise relevance ordering. Adv Neural Inf Process Syst 34:23545–23556
31.
Zurück zum Zitat Brinker K, Hüllermeier E (2005) Calibrated label-ranking, in: Proceedings of the NIPS-2005 Workshop on Learning to Rank, Citeseer, pp. 1–6 Brinker K, Hüllermeier E (2005) Calibrated label-ranking, in: Proceedings of the NIPS-2005 Workshop on Learning to Rank, Citeseer, pp. 1–6
32.
Zurück zum Zitat Weng W, Lin Y, Wu S, Li Y, Kang Y (2018) Multi-label learning based on label-specific features and local pairwise label correlation. Neurocomputing 273:385–394CrossRef Weng W, Lin Y, Wu S, Li Y, Kang Y (2018) Multi-label learning based on label-specific features and local pairwise label correlation. Neurocomputing 273:385–394CrossRef
33.
Zurück zum Zitat Huang J, Li G, Wang S, Xue Z, Huang Q (2017) Multi-label classification by exploiting local positive and negative pairwise label correlation. Neurocomputing 257:164–174CrossRef Huang J, Li G, Wang S, Xue Z, Huang Q (2017) Multi-label classification by exploiting local positive and negative pairwise label correlation. Neurocomputing 257:164–174CrossRef
34.
Zurück zum Zitat Huang R, Kang L (2021) Local positive and negative label correlation analysis with label awareness for multi-label classification. Int J Mach Learn Cybern 12(9):2659–2672CrossRef Huang R, Kang L (2021) Local positive and negative label correlation analysis with label awareness for multi-label classification. Int J Mach Learn Cybern 12(9):2659–2672CrossRef
35.
Zurück zum Zitat Wu G, Tian Y, Liu D (2018) Cost-sensitive multi-label learning with positive and negative label pairwise correlations. Neural Netw 108:411–423CrossRef Wu G, Tian Y, Liu D (2018) Cost-sensitive multi-label learning with positive and negative label pairwise correlations. Neural Netw 108:411–423CrossRef
36.
Zurück zum Zitat Zhang M-L, Peña JM, Robles V (2009) Feature selection for multi-label naive bayes classification. Inf Sci 179(19):3218–3229CrossRefMATH Zhang M-L, Peña JM, Robles V (2009) Feature selection for multi-label naive bayes classification. Inf Sci 179(19):3218–3229CrossRefMATH
37.
Zurück zum Zitat Zhang J, Luo Z, Li C, Zhou C, Li S (2019) Manifold regularized discriminative feature selection for multi-label learning. Pattern Recogn 95:136–150CrossRef Zhang J, Luo Z, Li C, Zhou C, Li S (2019) Manifold regularized discriminative feature selection for multi-label learning. Pattern Recogn 95:136–150CrossRef
38.
Zurück zum Zitat Gu Q, Li Z, Han J (2011) Correlated multi-label feature selection, in: Proceedings of the 20th ACM International Conference on Information and Knowledge Management, 1087–1096 Gu Q, Li Z, Han J (2011) Correlated multi-label feature selection, in: Proceedings of the 20th ACM International Conference on Information and Knowledge Management, 1087–1096
40.
Zurück zum Zitat Che X, Chen D, Mi J (2020) A novel approach for learning label correlation with application to feature selection of multi-label data. Inf Sci 512:795–812MathSciNetCrossRefMATH Che X, Chen D, Mi J (2020) A novel approach for learning label correlation with application to feature selection of multi-label data. Inf Sci 512:795–812MathSciNetCrossRefMATH
41.
Zurück zum Zitat Li Y, Hu J, Gao W (2022) Robust multi-label feature selection with shared label enhancement. Knowl Inf Syst 64:3343–3372CrossRef Li Y, Hu J, Gao W (2022) Robust multi-label feature selection with shared label enhancement. Knowl Inf Syst 64:3343–3372CrossRef
43.
Zurück zum Zitat Long X, Qian W, Wang Y, Shu W (2021) Cost-sensitive feature selection on multi-label data via neighborhood granularity and label enhancement. Appl Intell 51(4):2210–2232CrossRef Long X, Qian W, Wang Y, Shu W (2021) Cost-sensitive feature selection on multi-label data via neighborhood granularity and label enhancement. Appl Intell 51(4):2210–2232CrossRef
44.
Zurück zum Zitat Xiong C, Qian W, Wang Y, Huang J (2021) Feature selection based on label distribution and fuzzy mutual information. Inf Sci 574:297–319MathSciNetCrossRef Xiong C, Qian W, Wang Y, Huang J (2021) Feature selection based on label distribution and fuzzy mutual information. Inf Sci 574:297–319MathSciNetCrossRef
45.
Zurück zum Zitat Hu Q, Zhang L, Zhang D, Pan W, An S, Pedrycz W (2011) Measuring relevance between discrete and continuous features based on neighborhood mutual information. Expert Syst Appl 38(9):10737–10750CrossRef Hu Q, Zhang L, Zhang D, Pan W, An S, Pedrycz W (2011) Measuring relevance between discrete and continuous features based on neighborhood mutual information. Expert Syst Appl 38(9):10737–10750CrossRef
46.
Zurück zum Zitat Lin Y, Hu Q, Liu J, Chen J, Duan J (2016) Multi-label feature selection based on neighborhood mutual information. Appl Soft Comput 38:244–256CrossRef Lin Y, Hu Q, Liu J, Chen J, Duan J (2016) Multi-label feature selection based on neighborhood mutual information. Appl Soft Comput 38:244–256CrossRef
47.
Zurück zum Zitat Zhang Y, Zhou Z-H (2010) Multilabel dimensionality reduction via dependence maximization. ACM Transactions on Knowledge Discovery from Data (TKDD) 4(3):1–21CrossRef Zhang Y, Zhou Z-H (2010) Multilabel dimensionality reduction via dependence maximization. ACM Transactions on Knowledge Discovery from Data (TKDD) 4(3):1–21CrossRef
48.
Zurück zum Zitat Lee J, Lim H, Lim D-W (2012) Approximating mutual information for multi-label feature selection. Electron Lett 48(15):929–930CrossRef Lee J, Lim H, Lim D-W (2012) Approximating mutual information for multi-label feature selection. Electron Lett 48(15):929–930CrossRef
49.
Zurück zum Zitat Lin Y, Li Y, Wang C, Chen J (2018) Attribute reduction for multi-label learning with fuzzy rough set. Knowl-Based Syst 152:51–61CrossRef Lin Y, Li Y, Wang C, Chen J (2018) Attribute reduction for multi-label learning with fuzzy rough set. Knowl-Based Syst 152:51–61CrossRef
50.
Zurück zum Zitat Qian W, Xiong Y, Yang J, Shu W (2022) Feature selection for label distribution learning via feature similarity and label correlation. Inf Sci 582:38–59MathSciNetCrossRef Qian W, Xiong Y, Yang J, Shu W (2022) Feature selection for label distribution learning via feature similarity and label correlation. Inf Sci 582:38–59MathSciNetCrossRef
51.
Zurück zum Zitat Liu J, Lin Y, Du J, Zhang H, Chen Z, Zhang J (2022) Asfs: A novel streaming feature selection for multi-label data based on neighborhood rough set, Applied Intelligence 1–18 Liu J, Lin Y, Du J, Zhang H, Chen Z, Zhang J (2022) Asfs: A novel streaming feature selection for multi-label data based on neighborhood rough set, Applied Intelligence 1–18
52.
Zurück zum Zitat Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11(1):86–92MathSciNetCrossRefMATH Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11(1):86–92MathSciNetCrossRefMATH
54.
Zurück zum Zitat Liu J, Lin Y, Ding W, Zhang H, Wang C, Du J (2023) Multi-label feature selection based on label distribution and neighborhood rough set. Neurocomputing 524:142–157CrossRef Liu J, Lin Y, Ding W, Zhang H, Wang C, Du J (2023) Multi-label feature selection based on label distribution and neighborhood rough set. Neurocomputing 524:142–157CrossRef
55.
Zurück zum Zitat Xu N, Liu Y, Geng X (2021) Label enhancement for label distribution learning. IEEE Trans Knowl Data Eng 33(4):1632–1643CrossRef Xu N, Liu Y, Geng X (2021) Label enhancement for label distribution learning. IEEE Trans Knowl Data Eng 33(4):1632–1643CrossRef
Metadaten
Titel
Multi-label feature selection via joint label enhancement and pairwise label correlations
verfasst von
Jinghua Liu
Songwei Yang
Yaojin Lin
Chenxi Wang
Cheng Wang
Jixiang Du
Publikationsdatum
01.07.2023
Verlag
Springer Berlin Heidelberg
Erschienen in
International Journal of Machine Learning and Cybernetics / Ausgabe 11/2023
Print ISSN: 1868-8071
Elektronische ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-023-01874-x

Weitere Artikel der Ausgabe 11/2023

International Journal of Machine Learning and Cybernetics 11/2023 Zur Ausgabe

Neuer Inhalt