Skip to main content
Top
Published in: International Journal of Machine Learning and Cybernetics 4/2021

03-01-2021 | Original Article

Class-weighted neural network for monotonic imbalanced classification

Published in: International Journal of Machine Learning and Cybernetics | Issue 4/2021

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In real life scenarios, classification problems with the characters of monotonicity constraints and imbalanced class distribution widely exist. However, at present, the research on this kind of problem is still rare. Traditional algorithms designed only for monotonic classification and imbalanced classification are not available for monotonic imbalanced classification. So far, there is only one approach specially designed for monotonic imbalanced classification problems, which is based on the resampling technique. In this paper, from the algorithmic point of view, we propose a weighted single-hidden-layer feedforward neural network (WMCS-SLFN) based on multi-objective genetic algorithm, where both the monotonicity constraints and the imbalanced distribution are considered. Additionally, in order to improve the generalization capability of WMCS-SLFN, we put forward a selective ensemble strategy for WMCS-SLFN based on the 0–1 knapsack problem, which can generate an ensemble of WMCS-SLFN with the optimal prediction accuracy under the monotonicity constraints. Contrast experiments conducted on eight monotonic imbalanced datasets verify the effectiveness of our proposed methods, and moreover, the experimental results analyzed by Wilcoxon statistical test highlight the advantage of our work significantly.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Show more products
Literature
1.
go back to reference Cano JR, Gutierrez PA, Krawczyk B et al (2019) Monotonic classification: an overview on algorithms, performance measures and data sets. Neurocomputing 341:168–182CrossRef Cano JR, Gutierrez PA, Krawczyk B et al (2019) Monotonic classification: an overview on algorithms, performance measures and data sets. Neurocomputing 341:168–182CrossRef
2.
go back to reference Cano JR, Luengo J, Garcia S (2019) Label noise filtering techniques to improve monotonic classification. Neurocomputing 353:83–95CrossRef Cano JR, Luengo J, Garcia S (2019) Label noise filtering techniques to improve monotonic classification. Neurocomputing 353:83–95CrossRef
3.
go back to reference Garcia S, Derrac J, Cano J et al (2012) Prototype selection for nearest neighbor classification: taxonomy and empirical study. IEEE Trans Pattern Anal Mach Intell 34(3):417–435CrossRef Garcia S, Derrac J, Cano J et al (2012) Prototype selection for nearest neighbor classification: taxonomy and empirical study. IEEE Trans Pattern Anal Mach Intell 34(3):417–435CrossRef
4.
go back to reference Hu Q, Pan W, Zhang L et al (2012) Feature selection for monotonic classification. IEEE Trans Fuzzy Syst 20(1):69–81CrossRef Hu Q, Pan W, Zhang L et al (2012) Feature selection for monotonic classification. IEEE Trans Fuzzy Syst 20(1):69–81CrossRef
5.
go back to reference Cano JR, Garcia S (2017) Training set selection for monotonic ordinal classification. Data Knowl Eng 112:94–105CrossRef Cano JR, Garcia S (2017) Training set selection for monotonic ordinal classification. Data Knowl Eng 112:94–105CrossRef
6.
go back to reference Feelders A, Kolkman T (2016) Exploiting monotonicity constraints to reduce label noise: an experimental evaluation. In: International joint conference on neural networks (IJCNN) IEEE Feelders A, Kolkman T (2016) Exploiting monotonicity constraints to reduce label noise: an experimental evaluation. In: International joint conference on neural networks (IJCNN) IEEE
7.
go back to reference Wang JT, Qian YH, Li FJ et al (2020) Fusing fuzzy monotonic decision trees. IEEE Trans Fuzzy Syst 28:887–900CrossRef Wang JT, Qian YH, Li FJ et al (2020) Fusing fuzzy monotonic decision trees. IEEE Trans Fuzzy Syst 28:887–900CrossRef
8.
go back to reference Daniels H, Velikova M (2010) Monotone and partially monotone neural networks. IEEE Trans Neural Netw 21(6):906–917CrossRef Daniels H, Velikova M (2010) Monotone and partially monotone neural networks. IEEE Trans Neural Netw 21(6):906–917CrossRef
9.
go back to reference Zhu H, Tsang ECC, Wang XZ et al (2017) Monotonic classification extreme learning machine. Neurocomputing 225:205–213CrossRef Zhu H, Tsang ECC, Wang XZ et al (2017) Monotonic classification extreme learning machine. Neurocomputing 225:205–213CrossRef
10.
go back to reference Li ST, Chen CC (2015) A regularized monotonic fuzzy support vector machine model for data mining with prior knowledge”. IEEE Trans Fuzzy Syst 23(5):1713–1727CrossRef Li ST, Chen CC (2015) A regularized monotonic fuzzy support vector machine model for data mining with prior knowledge”. IEEE Trans Fuzzy Syst 23(5):1713–1727CrossRef
11.
go back to reference Duivesteijn W, Feelders A (2008) Nearest neighbour classification with monotonicity constraints. In: Joint European conference on machine learning and knowledge discovery in databases Springer, Berlin, Heidelberg Duivesteijn W, Feelders A (2008) Nearest neighbour classification with monotonicity constraints. In: Joint European conference on machine learning and knowledge discovery in databases Springer, Berlin, Heidelberg
12.
go back to reference He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284CrossRef He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284CrossRef
13.
go back to reference Guo H, Li Y, Shang J et al (2016) Learning from class-imbalanced data: review of methods and applications. Expert Syst Appl 73:220–239 Guo H, Li Y, Shang J et al (2016) Learning from class-imbalanced data: review of methods and applications. Expert Syst Appl 73:220–239
14.
go back to reference Zhai J, Zhang S, Zhang M et al (2018) Fuzzy integral-based ELM ensemble for imbalanced big data classification. Soft Comput 22(11):3519–3531CrossRef Zhai J, Zhang S, Zhang M et al (2018) Fuzzy integral-based ELM ensemble for imbalanced big data classification. Soft Comput 22(11):3519–3531CrossRef
15.
go back to reference Tahir MA, Kittler J, Mikolajczyk K et al (2009) A multiple expert approach to the class imbalance problem using inverse random under sampling. International workshop on multiple classifier systems. Springer, Berlin Tahir MA, Kittler J, Mikolajczyk K et al (2009) A multiple expert approach to the class imbalance problem using inverse random under sampling. International workshop on multiple classifier systems. Springer, Berlin
16.
go back to reference Chawla NV, Bowyer KW, Hall LO et al (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16(1):321–357CrossRef Chawla NV, Bowyer KW, Hall LO et al (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16(1):321–357CrossRef
17.
go back to reference Li QQ, Liu XY (2014) EasyEnsemble. M for multiclass imbalance problem Moshi Shibie yu Rengong Zhineng. Pattern Recogn Artif Intell 27(2):187-192 Li QQ, Liu XY (2014) EasyEnsemble. M for multiclass imbalance problem Moshi Shibie yu Rengong Zhineng. Pattern Recogn Artif Intell 27(2):187-192
18.
go back to reference Liu XY, Wu JX, Zhou ZH (2006) A cascade-based classification method for class-imbalanced data. J Nanjing Univ (Natural sciences) Liu XY, Wu JX, Zhou ZH (2006) A cascade-based classification method for class-imbalanced data. J Nanjing Univ (Natural sciences)
19.
go back to reference Han H, Wang WY, Mao BH (2005) Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. In: Proceedings of the international conference on advances in intelligent computing—Volume Part I:2005 Han H, Wang WY, Mao BH (2005) Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. In: Proceedings of the international conference on advances in intelligent computing—Volume Part I:2005
20.
go back to reference Lopez V, Rio SD, Benitez JM et al (2015) Cost-sensitive linguistic fuzzy rule based classification systems under the MapReduce framework for imbalanced big data. Fuzzy Sets Syst 258:5–38MathSciNetCrossRef Lopez V, Rio SD, Benitez JM et al (2015) Cost-sensitive linguistic fuzzy rule based classification systems under the MapReduce framework for imbalanced big data. Fuzzy Sets Syst 258:5–38MathSciNetCrossRef
21.
go back to reference Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1/3):489–501CrossRef Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1/3):489–501CrossRef
22.
go back to reference Zong WW, Huang GB, Chen Y (2013) Weighted extreme learning machine for imbalance learning. Neurocomputing 101:229–242CrossRef Zong WW, Huang GB, Chen Y (2013) Weighted extreme learning machine for imbalance learning. Neurocomputing 101:229–242CrossRef
23.
go back to reference Zhang L, Zhang D (2017) Evolutionary cost-sensitive extreme learning machine. IEEE Trans Neural Netw Learn Syst 28(12):3045–3060MathSciNetCrossRef Zhang L, Zhang D (2017) Evolutionary cost-sensitive extreme learning machine. IEEE Trans Neural Netw Learn Syst 28(12):3045–3060MathSciNetCrossRef
24.
go back to reference Gonzalez S, Garcia S, Li ST et al (2018) Chain based sampling for monotonic imbalanced classification. Inf Sci 474:187–204CrossRef Gonzalez S, Garcia S, Li ST et al (2018) Chain based sampling for monotonic imbalanced classification. Inf Sci 474:187–204CrossRef
25.
go back to reference Gao Y, Shi L, Yao P (2000) Study on multi-objective genetic algorithm. In: World congress on intelligent control and automation IEEE Gao Y, Shi L, Yao P (2000) Study on multi-objective genetic algorithm. In: World congress on intelligent control and automation IEEE
26.
go back to reference Deb K, Agrawal S, Pratap A et al (2000) A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. In: International conference on parallel problem solving from Nature Springer Deb K, Agrawal S, Pratap A et al (2000) A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. In: International conference on parallel problem solving from Nature Springer
27.
29.
go back to reference Perrone MP, Cooper LN (1993) When networks disagree: ensemble methods for hybrid neural networks. How we learn; how we remember: toward an understanding of brain and neural systems. In: Selected papers of Leon N Cooper Perrone MP, Cooper LN (1993) When networks disagree: ensemble methods for hybrid neural networks. How we learn; how we remember: toward an understanding of brain and neural systems. In: Selected papers of Leon N Cooper
30.
go back to reference Lievens S, Baets BD, Cao-Van K (2008) A probabilistic framework for the design of instance-based supervised ranking algorithms in an ordinal setting. Ann Oper Res 163:115–142MathSciNetCrossRef Lievens S, Baets BD, Cao-Van K (2008) A probabilistic framework for the design of instance-based supervised ranking algorithms in an ordinal setting. Ann Oper Res 163:115–142MathSciNetCrossRef
31.
go back to reference Ben-David A (2010) Automatic generation of symbolic multiattribute ordinal knowledge-based DSSs: methodology and applications. Decis Sci 23(6):1357–1372CrossRef Ben-David A (2010) Automatic generation of symbolic multiattribute ordinal knowledge-based DSSs: methodology and applications. Decis Sci 23(6):1357–1372CrossRef
32.
go back to reference Ben-David A (1995) Monotonicity maintenance in information-theoretic machine learning algorithms. Mach Learn 19(1):29–43 Ben-David A (1995) Monotonicity maintenance in information-theoretic machine learning algorithms. Mach Learn 19(1):29–43
33.
go back to reference Lang B (2005) Monotonic multi-layer perceptron networks as universal approximators. In: International conference on artificial neural networks, Springer, pp 31–37 Lang B (2005) Monotonic multi-layer perceptron networks as universal approximators. In: International conference on artificial neural networks, Springer, pp 31–37
34.
go back to reference Triguero I, Gonzalez S, Moyano JM et al (2017) KEEL 3.0: an open source software for multi-stage analysis in data mining. Int J Comput Intell Syst 10(1):1238–1249CrossRef Triguero I, Gonzalez S, Moyano JM et al (2017) KEEL 3.0: an open source software for multi-stage analysis in data mining. Int J Comput Intell Syst 10(1):1238–1249CrossRef
35.
go back to reference Wilcoxon F (1945) Individual comparisons by ranking methods. Biometr Bull 1(6):80–83CrossRef Wilcoxon F (1945) Individual comparisons by ranking methods. Biometr Bull 1(6):80–83CrossRef
Metadata
Title
Class-weighted neural network for monotonic imbalanced classification
Publication date
03-01-2021
Published in
International Journal of Machine Learning and Cybernetics / Issue 4/2021
Print ISSN: 1868-8071
Electronic ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-020-01228-x

Other articles of this Issue 4/2021

International Journal of Machine Learning and Cybernetics 4/2021 Go to the issue