Skip to main content
Top
Published in: Knowledge and Information Systems 3/2022

09-01-2022 | Regular Paper

Ensemble of classifier chains and decision templates for multi-label classification

Authors: Victor Freitas Rocha, Flávio Miguel Varejão, Marcelo Eduardo Vieira Segatto

Published in: Knowledge and Information Systems | Issue 3/2022

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Multi-label classification is the task of inferring the set of unseen instances using the knowledge obtained through the analysis of a set of training examples with known label sets. In this paper, a multi-label classifier fusion ensemble approach named decision templates for ensemble of classifier chains is presented, which is derived from the decision templates method. The proposed method estimates two decision templates per class, one representing the presence of the class and the other representing its absence, based on the same examples used for training the set of classifiers. For each unseen instance, a new decision profile is created and the similarity between the decision templates and the decision profile determines the resulting label set. The method is incorporated into a traditional multi-label classifier algorithm: the ensemble of classifier chains. Empirical evidence indicates that the use of the proposed decision templates adaptation can improve the performance over the traditionally used combining schemes, especially for domains with a large number of instances available, improving the performance of an already high-performing multi-label learning method.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333–359MathSciNetCrossRef Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333–359MathSciNetCrossRef
2.
go back to reference Kuncheva LI, Bezdek JC, Duin RP (2001) Decision templates for multiple classifier fusion: an experimental comparison. Pattern Recogn 34(2):299–314CrossRef Kuncheva LI, Bezdek JC, Duin RP (2001) Decision templates for multiple classifier fusion: an experimental comparison. Pattern Recogn 34(2):299–314CrossRef
3.
go back to reference Moyano JM, Gibaja EL, Cios KJ, Ventura S (2018) Review of ensembles of multi-label classifiers: models, experimental study and prospects. Inform Fusion 44:33–45CrossRef Moyano JM, Gibaja EL, Cios KJ, Ventura S (2018) Review of ensembles of multi-label classifiers: models, experimental study and prospects. Inform Fusion 44:33–45CrossRef
4.
go back to reference Chen ZM, Wei XS, Wang P, Guo Y (2019) Multi-label image recognition with graph convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5177–5186 Chen ZM, Wei XS, Wang P, Guo Y (2019) Multi-label image recognition with graph convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5177–5186
5.
go back to reference Chou KC (2019) Advances in predicting subcellular localization of multi-label proteins and its implication for developing multi-target drugs. Curr Med Chem 26(26):4918–4943CrossRef Chou KC (2019) Advances in predicting subcellular localization of multi-label proteins and its implication for developing multi-target drugs. Curr Med Chem 26(26):4918–4943CrossRef
8.
go back to reference Opitz D, Maclin R (1999) Popular ensemble methods: an empirical study. J Artif Intell Res 11:169–198CrossRef Opitz D, Maclin R (1999) Popular ensemble methods: an empirical study. J Artif Intell Res 11:169–198CrossRef
9.
go back to reference Tama BA, Comuzzi M, Rhee KH (2019) TSE-IDS: a two-stage classifier ensemble for intelligent anomaly-based intrusion detection system. IEEE Access 7:94497–94507CrossRef Tama BA, Comuzzi M, Rhee KH (2019) TSE-IDS: a two-stage classifier ensemble for intelligent anomaly-based intrusion detection system. IEEE Access 7:94497–94507CrossRef
10.
go back to reference Large J, Lines J, Bagnall A (2019) A probabilistic classifier ensemble weighting scheme based on cross-validated accuracy estimates. Data Min Knowl Disc 33(6):1674–1709MathSciNetCrossRef Large J, Lines J, Bagnall A (2019) A probabilistic classifier ensemble weighting scheme based on cross-validated accuracy estimates. Data Min Knowl Disc 33(6):1674–1709MathSciNetCrossRef
11.
go back to reference Guo X, Liu F, Ju Y, Wang Z, Wang C (2016) Human protein subcellular localization with integrated source and multi-label ensemble classifier. Sci Rep 6:28087CrossRef Guo X, Liu F, Ju Y, Wang Z, Wang C (2016) Human protein subcellular localization with integrated source and multi-label ensemble classifier. Sci Rep 6:28087CrossRef
12.
go back to reference Moyano JM, Gibaja EL, Cios KJ, Ventura S (2019) An evolutionary approach to build ensembles of multi-label classifiers. Inform Fusion 50:168–180CrossRef Moyano JM, Gibaja EL, Cios KJ, Ventura S (2019) An evolutionary approach to build ensembles of multi-label classifiers. Inform Fusion 50:168–180CrossRef
13.
go back to reference Woods K, Kegelmeyer WP, Bowyer K (1997) Combination of multiple classifiers using local accuracy estimates. IEEE Trans Pattern Anal Mach Intell 19(4):405–410CrossRef Woods K, Kegelmeyer WP, Bowyer K (1997) Combination of multiple classifiers using local accuracy estimates. IEEE Trans Pattern Anal Mach Intell 19(4):405–410CrossRef
14.
go back to reference Kuncheva LI (2002) Switching between selection and fusion in combining classifiers: an experiment. IEEE Trans Syst Man Cybern Part B 32(2):146–156CrossRef Kuncheva LI (2002) Switching between selection and fusion in combining classifiers: an experiment. IEEE Trans Syst Man Cybern Part B 32(2):146–156CrossRef
15.
go back to reference Ho T (2000) Complexity of classification problems and comparative advantages of combined classifiers. Multiple classifier systems, vol 1857 of lecture notes in computer science, pp 97–106 Ho T (2000) Complexity of classification problems and comparative advantages of combined classifiers. Multiple classifier systems, vol 1857 of lecture notes in computer science, pp 97–106
16.
go back to reference Jacobs RA, Jordan MI, Nowlan SJ, Hinton GE (1991) Adaptive mixtures of local experts. Neural Comput 3(1):79–87CrossRef Jacobs RA, Jordan MI, Nowlan SJ, Hinton GE (1991) Adaptive mixtures of local experts. Neural Comput 3(1):79–87CrossRef
17.
go back to reference Alpaydin E, Jordan MI (1996) Local linear perceptrons for classification. IEEE Trans Neural Netw 7(3):788–794CrossRef Alpaydin E, Jordan MI (1996) Local linear perceptrons for classification. IEEE Trans Neural Netw 7(3):788–794CrossRef
18.
go back to reference Giacinto G, Roli F (2001) An approach to the automatic design of multiple classifier systems. Pattern Recogn Lett 22(1):25–33CrossRef Giacinto G, Roli F (2001) An approach to the automatic design of multiple classifier systems. Pattern Recogn Lett 22(1):25–33CrossRef
19.
go back to reference Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140MATH Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140MATH
20.
go back to reference Schapire RE (1990) The strength of weak learnability. Mach Learn 5(2):197–227 Schapire RE (1990) The strength of weak learnability. Mach Learn 5(2):197–227
21.
go back to reference Freund Y, Schapire R, Abe N (1999) A short introduction to boosting. J Jpn Soc Artif Intell 14(771–780):1612 Freund Y, Schapire R, Abe N (1999) A short introduction to boosting. J Jpn Soc Artif Intell 14(771–780):1612
22.
go back to reference Shipp CA, Kuncheva LI (2002) Relationships between combination methods and measures of diversity in combining classifiers. Inform Fusion 3(2):135–148CrossRef Shipp CA, Kuncheva LI (2002) Relationships between combination methods and measures of diversity in combining classifiers. Inform Fusion 3(2):135–148CrossRef
23.
go back to reference Rogova G (1994) Combining the results of several neural network classifiers. Neural Netw 7(5):777–781CrossRef Rogova G (1994) Combining the results of several neural network classifiers. Neural Netw 7(5):777–781CrossRef
24.
go back to reference Gharroudi O, Elghazel H, Aussem A (2015) Ensemble multi-label classification: a comparative study on threshold selection and voting methods. In: 2015 IEEE 27th international conference on tools with artificial intelligence (ICTAI), pp 377–384 Gharroudi O, Elghazel H, Aussem A (2015) Ensemble multi-label classification: a comparative study on threshold selection and voting methods. In: 2015 IEEE 27th international conference on tools with artificial intelligence (ICTAI), pp 377–384
25.
go back to reference Petković M, Džeroski S, Kocev D (2020) Multi-label feature ranking with ensemble methods. Mach Learn 109:2141–2159MathSciNetCrossRef Petković M, Džeroski S, Kocev D (2020) Multi-label feature ranking with ensemble methods. Mach Learn 109:2141–2159MathSciNetCrossRef
26.
go back to reference Antonucci A, Corani G, Mauá DD, Gabaglio S (2013) An ensemble of bayesian networks for multilabel classification. In: Twenty-third international joint conference on artificial intelligence, pp 1220–1225 Antonucci A, Corani G, Mauá DD, Gabaglio S (2013) An ensemble of bayesian networks for multilabel classification. In: Twenty-third international joint conference on artificial intelligence, pp 1220–1225
27.
go back to reference Tsoumakas G, Spyromitros-Xioufis E, Vilcek J, Vlahavas I (2011) Mulan: a java library for multi-label learning. J Mach Learn Res 12(Jul):2411–2414 Tsoumakas G, Spyromitros-Xioufis E, Vilcek J, Vlahavas I (2011) Mulan: a java library for multi-label learning. J Mach Learn Res 12(Jul):2411–2414
28.
go back to reference Rauber TW, da Silva Loca AL, de Assis Boldt F, Rodrigues AL, Varejão FM (2020) An experimental methodology to evaluate machine learning methods for fault diagnosis based on vibration signals. Exp Syst Appl 114022 Rauber TW, da Silva Loca AL, de Assis Boldt F, Rodrigues AL, Varejão FM (2020) An experimental methodology to evaluate machine learning methods for fault diagnosis based on vibration signals. Exp Syst Appl 114022
29.
go back to reference Zhang ML, Zhou ZH (2013) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837CrossRef Zhang ML, Zhou ZH (2013) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837CrossRef
30.
go back to reference Schapire RE, Singer Y (2000) BoosTexter: a boosting-based system for text categorization. Mach Learn 39(2–3):135–168CrossRef Schapire RE, Singer Y (2000) BoosTexter: a boosting-based system for text categorization. Mach Learn 39(2–3):135–168CrossRef
31.
go back to reference Cheng W, Hüllermeier E, Dembczynski KJ (2010) Bayes optimal multilabel classification via probabilistic classifier chains. In: Proceedings of the 27th international conference on machine learning (ICML-10). Haifa, Israel, pp 279–286 Cheng W, Hüllermeier E, Dembczynski KJ (2010) Bayes optimal multilabel classification via probabilistic classifier chains. In: Proceedings of the 27th international conference on machine learning (ICML-10). Haifa, Israel, pp 279–286
32.
go back to reference Pereira RB, Plastino A, Zadrozny B, Merschmann LHC (2018) Correlation analysis of performance measures for multi-label classification. Inf Process Manage 54(3):359–369CrossRef Pereira RB, Plastino A, Zadrozny B, Merschmann LHC (2018) Correlation analysis of performance measures for multi-label classification. Inf Process Manage 54(3):359–369CrossRef
33.
go back to reference Read J, Pfahringer B, Holmes G (2008) Multi-label classification using ensembles of pruned sets. In: Eighth IEEE international conference on data mining, 2008. ICDM’08. IEEE, pp 995–1000 Read J, Pfahringer B, Holmes G (2008) Multi-label classification using ensembles of pruned sets. In: Eighth IEEE international conference on data mining, 2008. ICDM’08. IEEE, pp 995–1000
34.
go back to reference Tenenboim-Chekina L, Rokach L, Shapira B (2010) Identification of label dependencies for multi-label classification. In: Working notes of the second international workshop on learning from multi-label data, pp 53–60 Tenenboim-Chekina L, Rokach L, Shapira B (2010) Identification of label dependencies for multi-label classification. In: Working notes of the second international workshop on learning from multi-label data, pp 53–60
35.
go back to reference Tsoumakas G, Vlahavas I (2007) Random k-labelsets: an ensemble method for multilabel classification. Mach Learn ECML 2007:406–417 Tsoumakas G, Vlahavas I (2007) Random k-labelsets: an ensemble method for multilabel classification. Mach Learn ECML 2007:406–417
Metadata
Title
Ensemble of classifier chains and decision templates for multi-label classification
Authors
Victor Freitas Rocha
Flávio Miguel Varejão
Marcelo Eduardo Vieira Segatto
Publication date
09-01-2022
Publisher
Springer London
Published in
Knowledge and Information Systems / Issue 3/2022
Print ISSN: 0219-1377
Electronic ISSN: 0219-3116
DOI
https://doi.org/10.1007/s10115-021-01647-4

Other articles of this Issue 3/2022

Knowledge and Information Systems 3/2022 Go to the issue

Premium Partner