Skip to main content
Erschienen in: International Journal of Machine Learning and Cybernetics 5/2022

01.12.2021 | Original Article

DRCW-FRkNN-OVO: distance-based related competence weighting based on fixed radius k nearest neighbour for one-vs-one scheme

verfasst von: Zhong-Liang Zhang, Xing-Gang Luo, Qing Zhou

Erschienen in: International Journal of Machine Learning and Cybernetics | Ausgabe 5/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The one-versus-one (OVO) binarization decomposition scheme is considered as one of the most effective techniques to deal with multi-class classification problems. Its inherent mechanism is to use the “divide-and-conquer” strategy to decompose the multi-class classification problem into as many pairs of easier-to-solve binary sub-problems as possible. One common issue in the OVO scheme is that of non-competent classifiers. In this study, we proposed a novel OVO scheme strategy, named DRCW-FRkNN-OVO, to reduce the negative effect of non-competent classifiers. Specifically, we focused on the definition of region of competence, which plays a crucial role in managing the non-competent classifiers. To overcome the issue of skew and sparse distribution during the management of non-competent classifiers, we developed a relative competence weighting combination method via the fixed radius nearest neighbour search to find the local region within each class for the query sample. Our proposed DRCW-FRkNN-OVO is tested on 30 real-world multi-class datasets compared with several well-known related works. Experimental results supported by thorough statistical analysis confirmed the effectiveness and robustness of our proposed method.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Weitere Produktempfehlungen anzeigen
Literatur
1.
Zurück zum Zitat Aha DW, Kibler D, Albert MK (1991) Instance-based learning algorithms. Mach Learn 6(1):37–66 Aha DW, Kibler D, Albert MK (1991) Instance-based learning algorithms. Mach Learn 6(1):37–66
2.
Zurück zum Zitat AL-Sharuee MT, Liu F, Pratama M (2018) Sentiment analysis: an automatic contextual analysis and ensemble clustering approach and comparison. Data Knowl Eng 115:194–213CrossRef AL-Sharuee MT, Liu F, Pratama M (2018) Sentiment analysis: an automatic contextual analysis and ensemble clustering approach and comparison. Data Knowl Eng 115:194–213CrossRef
3.
Zurück zum Zitat Alcalá-Fdez J, Sanchez L, Garcia S, del Jesus MJ, Ventura S, Garrell JM, Otero J, Romero C, Bacardit J, Rivas VM et al (2009) KEEL: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput 13(3):307–318CrossRef Alcalá-Fdez J, Sanchez L, Garcia S, del Jesus MJ, Ventura S, Garrell JM, Otero J, Romero C, Bacardit J, Rivas VM et al (2009) KEEL: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput 13(3):307–318CrossRef
4.
Zurück zum Zitat Allwein EL, Schapire RE, Singer Y (2000) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141MathSciNetMATH Allwein EL, Schapire RE, Singer Y (2000) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141MathSciNetMATH
5.
Zurück zum Zitat Alqudah AM (2020) AOCT-NET: a convolutional network automated classification of multiclass retinal diseases using spectral-domain optical coherence tomography images. Med Biol Eng Comput 58(1):41–53MathSciNetCrossRef Alqudah AM (2020) AOCT-NET: a convolutional network automated classification of multiclass retinal diseases using spectral-domain optical coherence tomography images. Med Biol Eng Comput 58(1):41–53MathSciNetCrossRef
6.
Zurück zum Zitat Bouazizi M, Ohtsuki T (2019) Multi-class sentiment analysis on twitter: classification performance and challenges. Big Data Min Anal 2(3):181–194CrossRef Bouazizi M, Ohtsuki T (2019) Multi-class sentiment analysis on twitter: classification performance and challenges. Big Data Min Anal 2(3):181–194CrossRef
7.
Zurück zum Zitat Breiman L, Friedman J, Stone CJ, Olshen RA (1984) Classification and regression trees. CRC Press, Boca RatonMATH Breiman L, Friedman J, Stone CJ, Olshen RA (1984) Classification and regression trees. CRC Press, Boca RatonMATH
8.
Zurück zum Zitat Cerf L, Gay D, Selmaoui-Folcher N, Crémilleux B, Boulicaut JF (2013) Parameter-free classification in multi-class imbalanced data sets. Data Knowl Eng 87:109–129CrossRef Cerf L, Gay D, Selmaoui-Folcher N, Crémilleux B, Boulicaut JF (2013) Parameter-free classification in multi-class imbalanced data sets. Data Knowl Eng 87:109–129CrossRef
9.
Zurück zum Zitat Chang CC, Lin CJ (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):1–27CrossRef Chang CC, Lin CJ (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):1–27CrossRef
10.
Zurück zum Zitat Clark P, Boswell R (1991) Rule induction with CN2: some recent improvements. European working session on learning. Springer, Berlin, pp 151–163 Clark P, Boswell R (1991) Rule induction with CN2: some recent improvements. European working session on learning. Springer, Berlin, pp 151–163
11.
Zurück zum Zitat Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37–46CrossRef Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37–46CrossRef
12.
Zurück zum Zitat Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetMATH Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetMATH
13.
Zurück zum Zitat Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18CrossRef Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18CrossRef
14.
Zurück zum Zitat Dietterich TG, Bakiri G (1994) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263–286CrossRef Dietterich TG, Bakiri G (1994) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263–286CrossRef
15.
Zurück zum Zitat Don DR, Iacob IE (2020) DCSVM: fast multi-class classifcation using support vector machines. Int J Mach Learn Cybern 11:433–447CrossRef Don DR, Iacob IE (2020) DCSVM: fast multi-class classifcation using support vector machines. Int J Mach Learn Cybern 11:433–447CrossRef
16.
Zurück zum Zitat Fernandes ER, de Carvalho AC (2019) Evolutionary inversion of class distribution in overlapping areas for multi-class imbalanced learning. Inf Sci 494:141–154CrossRef Fernandes ER, de Carvalho AC (2019) Evolutionary inversion of class distribution in overlapping areas for multi-class imbalanced learning. Inf Sci 494:141–154CrossRef
17.
Zurück zum Zitat Fernández A, Calderón M, Barrenechea E, Bustince H, Herrera F (2010) Solving multi-class problems with linguistic fuzzy rule based classification systems based on pairwise learning and preference relations. Fuzzy Sets Syst 161(23):3064–3080MathSciNetCrossRef Fernández A, Calderón M, Barrenechea E, Bustince H, Herrera F (2010) Solving multi-class problems with linguistic fuzzy rule based classification systems based on pairwise learning and preference relations. Fuzzy Sets Syst 161(23):3064–3080MathSciNetCrossRef
18.
Zurück zum Zitat Friedman J (1996) Another approach to polychotomous classification. Department of Statistics, Stanford University, Tech. rep Friedman J (1996) Another approach to polychotomous classification. Department of Statistics, Stanford University, Tech. rep
19.
Zurück zum Zitat Galar M, Fernández A, Barrenechea E, Bustince H, Herrera F (2011) An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes. Pattern Recogn 44(8):1761–1776CrossRef Galar M, Fernández A, Barrenechea E, Bustince H, Herrera F (2011) An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes. Pattern Recogn 44(8):1761–1776CrossRef
20.
Zurück zum Zitat Galar M, Fernández A, Barrenechea E, Bustince H, Herrera F (2013) Dynamic classifier selection for one-vs-one strategy: avoiding non-competent classifiers. Pattern Recogn 46(12):3412–3424CrossRef Galar M, Fernández A, Barrenechea E, Bustince H, Herrera F (2013) Dynamic classifier selection for one-vs-one strategy: avoiding non-competent classifiers. Pattern Recogn 46(12):3412–3424CrossRef
21.
Zurück zum Zitat Galar M, Fernández A, Barrenechea E, Herrera F (2015) DRCW-OVO: distance-based relative competence weighting combination for one-vs-one strategy in multi-class problems. Pattern Recogn 48(1):28–42CrossRef Galar M, Fernández A, Barrenechea E, Herrera F (2015) DRCW-OVO: distance-based relative competence weighting combination for one-vs-one strategy in multi-class problems. Pattern Recogn 48(1):28–42CrossRef
22.
Zurück zum Zitat García S, Fernández A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf Sci 180(10):2044–2064CrossRef García S, Fernández A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf Sci 180(10):2044–2064CrossRef
23.
Zurück zum Zitat García S, Zhang ZL, Altalhi A, Alshomrani S, Herrera F (2018) Dynamic ensemble selection for multi-class imbalanced datasets. Inf Sci 445:22–37MathSciNetCrossRef García S, Zhang ZL, Altalhi A, Alshomrani S, Herrera F (2018) Dynamic ensemble selection for multi-class imbalanced datasets. Inf Sci 445:22–37MathSciNetCrossRef
24.
Zurück zum Zitat Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The WEKA data mining software: an update. ACM SIGKDD Explor Newsl 11(1):10–18CrossRef Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The WEKA data mining software: an update. ACM SIGKDD Explor Newsl 11(1):10–18CrossRef
25.
Zurück zum Zitat Hühn JC, Hüllermeier E (2008) FR3: a fuzzy rule learner for inducing reliable classifiers. IEEE Trans Fuzzy Syst 17(1):138–149CrossRef Hühn JC, Hüllermeier E (2008) FR3: a fuzzy rule learner for inducing reliable classifiers. IEEE Trans Fuzzy Syst 17(1):138–149CrossRef
26.
Zurück zum Zitat Hüllermeier E, Brinker K (2008) Learning valued preference structures for solving classification problems. Fuzzy Sets Syst. 159(18):2337–2352MathSciNetCrossRef Hüllermeier E, Brinker K (2008) Learning valued preference structures for solving classification problems. Fuzzy Sets Syst. 159(18):2337–2352MathSciNetCrossRef
27.
Zurück zum Zitat Hüllermeier E, Vanderlooy S (2010) Combining predictions in pairwise classification: an optimal adaptive voting strategy and its relation to weighted voting. Pattern Recogn 43(1):128–142CrossRef Hüllermeier E, Vanderlooy S (2010) Combining predictions in pairwise classification: an optimal adaptive voting strategy and its relation to weighted voting. Pattern Recogn 43(1):128–142CrossRef
28.
Zurück zum Zitat Islam MM, Kim JM (2019) Reliable multiple combined fault diagnosis of bearings using heterogeneous feature models and multiclass support vector machines. Reliab Eng Syst Saf 184:55–66CrossRef Islam MM, Kim JM (2019) Reliable multiple combined fault diagnosis of bearings using heterogeneous feature models and multiclass support vector machines. Reliab Eng Syst Saf 184:55–66CrossRef
29.
Zurück zum Zitat Karagoz GN, Yazici A, Dokeroglu T, Cosar A (2021) A new framework of multi-objective evolutionary algorithms for feature selection and multi-label classification of video data. Int J Mach Learn Cybern 12(1):53–71CrossRef Karagoz GN, Yazici A, Dokeroglu T, Cosar A (2021) A new framework of multi-objective evolutionary algorithms for feature selection and multi-label classification of video data. Int J Mach Learn Cybern 12(1):53–71CrossRef
30.
Zurück zum Zitat Knerr S, Personnaz L, Dreyfus G (1990) Single-layer learning revisited: a stepwise procedure for building and training a neural network. Neurocomputing: Algorithms. Archit Appl 68:41–50 Knerr S, Personnaz L, Dreyfus G (1990) Single-layer learning revisited: a stepwise procedure for building and training a neural network. Neurocomputing: Algorithms. Archit Appl 68:41–50
31.
Zurück zum Zitat Liu Y, Bi JW, Fan ZP (2017) Multi-class sentiment classification: the experimental comparisons of feature selection and machine learning algorithms. Expert Syst Appl 80:323–339CrossRef Liu Y, Bi JW, Fan ZP (2017) Multi-class sentiment classification: the experimental comparisons of feature selection and machine learning algorithms. Expert Syst Appl 80:323–339CrossRef
32.
Zurück zum Zitat Nayak DR, Dash R, Majhi B (2020) Automated diagnosis of multi-class brain abnormalities using mri images: a deep convolutional neural network based method. Pattern Recogn Lett 138:385–391CrossRef Nayak DR, Dash R, Majhi B (2020) Automated diagnosis of multi-class brain abnormalities using mri images: a deep convolutional neural network based method. Pattern Recogn Lett 138:385–391CrossRef
33.
34.
Zurück zum Zitat Pang Y, He Q, Jiang G, Xie P (2020) Spatio-temporal fusion neural network for multi-class fault diagnosis of wind turbines based on scada data. Renew Energy 161:510–524CrossRef Pang Y, He Q, Jiang G, Xie P (2020) Spatio-temporal fusion neural network for multi-class fault diagnosis of wind turbines based on scada data. Renew Energy 161:510–524CrossRef
35.
Zurück zum Zitat Platt JC (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv Large Margin Classif 10(3):61–74 Platt JC (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv Large Margin Classif 10(3):61–74
36.
Zurück zum Zitat Ramzan F, Khan MUG, Rehmat A, Iqbal S, Saba T, Rehman A, Mehmood Z (2020) A deep learning approach for automated diagnosis and multi-class classification of Alzheimer’s disease stages using resting-state fmri and residual neural networks. J Med Syst 44(2):37CrossRef Ramzan F, Khan MUG, Rehmat A, Iqbal S, Saba T, Rehman A, Mehmood Z (2020) A deep learning approach for automated diagnosis and multi-class classification of Alzheimer’s disease stages using resting-state fmri and residual neural networks. J Med Syst 44(2):37CrossRef
37.
Zurück zum Zitat Vapnik V (1998) Statistical learning theory. Wiley, New YorkMATH Vapnik V (1998) Statistical learning theory. Wiley, New YorkMATH
38.
Zurück zum Zitat Villacampa-Calvo C, Zaldivar B, Garrido-Merchán EC, Hernández-Lobato D (2020) Multi-class gaussian process classification with noisy inputs Villacampa-Calvo C, Zaldivar B, Garrido-Merchán EC, Hernández-Lobato D (2020) Multi-class gaussian process classification with noisy inputs
39.
Zurück zum Zitat Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83CrossRef Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83CrossRef
40.
Zurück zum Zitat Wilson DR, Martinez TR (1997) Improved heterogeneous distance functions. J Artif Intell Res 6(1):1–34MathSciNetMATH Wilson DR, Martinez TR (1997) Improved heterogeneous distance functions. J Artif Intell Res 6(1):1–34MathSciNetMATH
41.
Zurück zum Zitat Wu S, Xia X, Liu T, Han B, Gong M, Wang N, Liu H, Niu G (2020) Multi-class classification from noisy-similarity-labeled data. arXiv preprint arXiv:200206508 Wu S, Xia X, Liu T, Han B, Gong M, Wang N, Liu H, Niu G (2020) Multi-class classification from noisy-similarity-labeled data. arXiv preprint arXiv:​200206508
42.
Zurück zum Zitat Xu F, Pan Z, Xia R (2020) E-commerce product review sentiment classification based on a na\(\ddot{i}\)ve bayes continuous learning framework. Inf Process Manag 57(5):102221CrossRef Xu F, Pan Z, Xia R (2020) E-commerce product review sentiment classification based on a na\(\ddot{i}\)ve bayes continuous learning framework. Inf Process Manag 57(5):102221CrossRef
43.
Zurück zum Zitat Zhang X, Ding S, Sun T (2016) Multi-class LSTMSVM based on optimal directed acyclic graph and shuffled frog leaping algorithm. Int J Mach Learn Cybern 7:241–251CrossRef Zhang X, Ding S, Sun T (2016) Multi-class LSTMSVM based on optimal directed acyclic graph and shuffled frog leaping algorithm. Int J Mach Learn Cybern 7:241–251CrossRef
Metadaten
Titel
DRCW-FRkNN-OVO: distance-based related competence weighting based on fixed radius k nearest neighbour for one-vs-one scheme
verfasst von
Zhong-Liang Zhang
Xing-Gang Luo
Qing Zhou
Publikationsdatum
01.12.2021
Verlag
Springer Berlin Heidelberg
Erschienen in
International Journal of Machine Learning and Cybernetics / Ausgabe 5/2022
Print ISSN: 1868-8071
Elektronische ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-021-01458-7

Weitere Artikel der Ausgabe 5/2022

International Journal of Machine Learning and Cybernetics 5/2022 Zur Ausgabe