Skip to main content
Top
Published in: Pattern Analysis and Applications 2/2017

29-07-2015 | Theoretical Advances

MoNGEL: monotonic nested generalized exemplar learning

Authors: Javier García, Habib M. Fardoun, Daniyal M. Alghazzawi, José-Ramón Cano, Salvador García

Published in: Pattern Analysis and Applications | Issue 2/2017

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In supervised prediction problems, the response attribute depends on certain explanatory attributes. Some real problems require the response attribute to represent ordinal values that should increase with some of the explaining attributes. They are called classification problems with monotonicity constraints. In this paper, we aim at formalizing the approach to nested generalized exemplar learning with monotonicity constraints, proposing the monotonic nested generalized exemplar learning (MoNGEL) method. It accomplishes learning by storing objects in \({\mathbb {R}}^n\), hybridizing instance-based learning and rule learning into a combined model. An experimental analysis is carried out over a wide range of monotonic data sets. The results obtained have been verified by non-parametric statistical tests and show that MoNGEL outperforms well-known techniques for monotonic classification, such as ordinal learning model, ordinal stochastic dominance learner and k-nearest neighbor, considering accuracy, mean absolute error and simplicity of constructed models.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Aha DW (ed) (1997) Lazy learning. Springer, New York Aha DW (ed) (1997) Lazy learning. Springer, New York
2.
go back to reference Aha DW, Kibler D, Albert MK (1991) Instance-based learning algorithms. Mach Learn 6(1):37–66 Aha DW, Kibler D, Albert MK (1991) Instance-based learning algorithms. Mach Learn 6(1):37–66
3.
go back to reference Alcala-Fdez J, Fernández A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2011) KEEL data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J Mult-Valued Log Soft Comput 17(2–3):255–287 Alcala-Fdez J, Fernández A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2011) KEEL data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J Mult-Valued Log Soft Comput 17(2–3):255–287
5.
go back to reference Ben-David A (1992) Automatic generation of symbolic multiattribute ordinal knowledge-based dsss: methodology and applications. Decis Sci 23:1357–1372CrossRef Ben-David A (1992) Automatic generation of symbolic multiattribute ordinal knowledge-based dsss: methodology and applications. Decis Sci 23:1357–1372CrossRef
6.
go back to reference Ben-David A (1995) Monotonicity maintenance in information-theoretic machine learning algorithms. Mach Learn 19(1):29–43 Ben-David A (1995) Monotonicity maintenance in information-theoretic machine learning algorithms. Mach Learn 19(1):29–43
7.
go back to reference Ben-David A, Sterling L, Pao YH (1989) Learning, classification of monotonic ordinal concepts. Comput Intel 5:45–49CrossRef Ben-David A, Sterling L, Pao YH (1989) Learning, classification of monotonic ordinal concepts. Comput Intel 5:45–49CrossRef
8.
go back to reference Ben-David A, Sterling L, Tran T (2009) Adding monotonicity to learning algorithms may impair their accuracy. Expert Syst Appl 36(3):6627–6634CrossRef Ben-David A, Sterling L, Tran T (2009) Adding monotonicity to learning algorithms may impair their accuracy. Expert Syst Appl 36(3):6627–6634CrossRef
9.
go back to reference Cao-Van K (2003) Supervised ranking, from semantics to algorithms. Ph.D. dissertation, Ghent University, Ghent Cao-Van K (2003) Supervised ranking, from semantics to algorithms. Ph.D. dissertation, Ghent University, Ghent
10.
go back to reference Cao-Van K, Baets BD (2003) Growing decision trees in an ordinal setting. Int J Intel Syst 18(7):733–750CrossRefMATH Cao-Van K, Baets BD (2003) Growing decision trees in an ordinal setting. Int J Intel Syst 18(7):733–750CrossRefMATH
11.
go back to reference Chen CC, Li ST (2014) Credit rating with a monotonicity-constrained support vector machine model. Expert Syst Appl 41(16):7235–7247CrossRef Chen CC, Li ST (2014) Credit rating with a monotonicity-constrained support vector machine model. Expert Syst Appl 41(16):7235–7247CrossRef
12.
go back to reference Daniels H, Velikova M (2010) Monotone and partially monotone neural networks. IEEE Trans Neural Netw 21(6):906–917CrossRef Daniels H, Velikova M (2010) Monotone and partially monotone neural networks. IEEE Trans Neural Netw 21(6):906–917CrossRef
13.
go back to reference Dembczyński K, Kotłowski W, Słowiński R (2009) Learning rule ensembles for ordinal classification with monotonicity constraints. Fundam Inform 94(2):163–178MathSciNetMATH Dembczyński K, Kotłowski W, Słowiński R (2009) Learning rule ensembles for ordinal classification with monotonicity constraints. Fundam Inform 94(2):163–178MathSciNetMATH
14.
go back to reference Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetMATH Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetMATH
15.
go back to reference Derrac J, García S, Herrera F (2014) Fuzzy nearest neighbor algorithms: taxonomy, experimental analysis and prospects. Inf Sci 260:98–119CrossRef Derrac J, García S, Herrera F (2014) Fuzzy nearest neighbor algorithms: taxonomy, experimental analysis and prospects. Inf Sci 260:98–119CrossRef
16.
go back to reference Domingos P (1996) Unifying instance-based and rule-based induction. Mach Learn 24(2):141–168MathSciNet Domingos P (1996) Unifying instance-based and rule-based induction. Mach Learn 24(2):141–168MathSciNet
17.
go back to reference Duivesteijn W, Feelders A (2008) Nearest neighbour classification with monotonicity constraints. In: ECML/PKDD (1), pp 301–316 Duivesteijn W, Feelders A (2008) Nearest neighbour classification with monotonicity constraints. In: ECML/PKDD (1), pp 301–316
18.
go back to reference Escalante HJ, Marin-Castro M, Morales-Reyes A, Graff M, Rosales-Pérez A, y Gómez MM, Reyes CA, González JA (2015) MOPG: a multi-objective evolutionary algorithm for prototype generation. Pattern Anal Appl. doi:10.1007/s10044-015-0454-6 (in press) Escalante HJ, Marin-Castro M, Morales-Reyes A, Graff M, Rosales-Pérez A, y Gómez MM, Reyes CA, González JA (2015) MOPG: a multi-objective evolutionary algorithm for prototype generation. Pattern Anal Appl. doi:10.​1007/​s10044-015-0454-6 (in press)
19.
go back to reference Feelders AJ, Pardoel M (2003) Pruning for monotone classification trees. In: IDA. Lecture notes in computer science, vol, 2810. Springer, New York, pp 1–12 Feelders AJ, Pardoel M (2003) Pruning for monotone classification trees. In: IDA. Lecture notes in computer science, vol, 2810. Springer, New York, pp 1–12
20.
go back to reference Fernández-Navarro F, Riccardi A, Carloni S (2014) Ordinal neural networks without iterative tuning. IEEE Trans Neural Netw Learn Syst 25(11):2075–2085CrossRef Fernández-Navarro F, Riccardi A, Carloni S (2014) Ordinal neural networks without iterative tuning. IEEE Trans Neural Netw Learn Syst 25(11):2075–2085CrossRef
21.
22.
go back to reference García S, Herrera F (2008) An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. J Mach Learn Res 9:2677–2694MATH García S, Herrera F (2008) An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. J Mach Learn Res 9:2677–2694MATH
23.
go back to reference García S, Fernández A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf Sci 180(10):2044–2064CrossRef García S, Fernández A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf Sci 180(10):2044–2064CrossRef
24.
go back to reference García S, Derrac J, Luengo J, Carmona CJ, Herrera F (2011) Evolutionary selection of hyperrectangles in nested generalized exemplar learning. Appl Soft Comput 11(3):3032–3045CrossRef García S, Derrac J, Luengo J, Carmona CJ, Herrera F (2011) Evolutionary selection of hyperrectangles in nested generalized exemplar learning. Appl Soft Comput 11(3):3032–3045CrossRef
25.
go back to reference García S, Derrac J, Triguero I, Carmona CJ, Herrera F (2012) Evolutionary-based selection of generalized instances for imbalanced classification. Knowl-Based Syst 25(1):3–12CrossRef García S, Derrac J, Triguero I, Carmona CJ, Herrera F (2012) Evolutionary-based selection of generalized instances for imbalanced classification. Knowl-Based Syst 25(1):3–12CrossRef
26.
go back to reference García S, Luengo J, Herrera F (2015) Data preprocessing in data mining. Springer, New York García S, Luengo J, Herrera F (2015) Data preprocessing in data mining. Springer, New York
27.
go back to reference Gaudette L, Japkowicz N (2009) Evaluation methods for ordinal classification. In: Canadian conference on AI. Lecture notes in computer science, vol 5549, pp 207–210 Gaudette L, Japkowicz N (2009) Evaluation methods for ordinal classification. In: Canadian conference on AI. Lecture notes in computer science, vol 5549, pp 207–210
28.
go back to reference Han J, Kamber M (2011) Data mining: concepts and techniques. Morgan Kaufmann Publishers Inc., San Francisco Han J, Kamber M (2011) Data mining: concepts and techniques. Morgan Kaufmann Publishers Inc., San Francisco
29.
go back to reference Hu Q, Che X, Zhang L, Zhang D, Guo M, Yu D (2012) Rank entropy-based decision trees for monotonic classification. IEEE Trans Knowl Data Eng 24(11):2052–2064CrossRef Hu Q, Che X, Zhang L, Zhang D, Guo M, Yu D (2012) Rank entropy-based decision trees for monotonic classification. IEEE Trans Knowl Data Eng 24(11):2052–2064CrossRef
30.
go back to reference Japkowicz N, Shah M (eds) (2011) Evaluating learning algorithms: a classification perspective. Cambridge University Press, CambridgeMATH Japkowicz N, Shah M (eds) (2011) Evaluating learning algorithms: a classification perspective. Cambridge University Press, CambridgeMATH
31.
go back to reference Kotłowski W, Słowiński R (2009) Rule learning with monotonicity constraints. In: ICML, vol 382 Kotłowski W, Słowiński R (2009) Rule learning with monotonicity constraints. In: ICML, vol 382
32.
go back to reference Kotłowski W, Słowiński R (2013) On nonparametric ordinal classification with monotonicity constraints. IEEE Trans Knowl Data Eng 25(11):2576–2589CrossRef Kotłowski W, Słowiński R (2013) On nonparametric ordinal classification with monotonicity constraints. IEEE Trans Knowl Data Eng 25(11):2576–2589CrossRef
33.
go back to reference Lievens S, Baets BD, Cao-Van K (2008) A probabilistic framework for the design of instance-based supervised ranking algorithms in an ordinal setting. Ann Oper Res 163(1):115–142MathSciNetCrossRefMATH Lievens S, Baets BD, Cao-Van K (2008) A probabilistic framework for the design of instance-based supervised ranking algorithms in an ordinal setting. Ann Oper Res 163(1):115–142MathSciNetCrossRefMATH
34.
go back to reference Liu T, Moore AW, Gray A (2006) New algorithms for efficient high-dimensional nonparametric classification. J Mach Learn Res 7:1135–1158MathSciNetMATH Liu T, Moore AW, Gray A (2006) New algorithms for efficient high-dimensional nonparametric classification. J Mach Learn Res 7:1135–1158MathSciNetMATH
35.
go back to reference López V, Fernández A, García S, Palade V, Herrera F (2013) An insight into classification with imbalanced data: empirical results and current trends on using data intrinsic characteristics. Inf Sci 250:113–141CrossRef López V, Fernández A, García S, Palade V, Herrera F (2013) An insight into classification with imbalanced data: empirical results and current trends on using data intrinsic characteristics. Inf Sci 250:113–141CrossRef
36.
go back to reference Mariolis IG, Dermatas E (2013) Automatic classification of seam pucker images based on ordinal quality grades. Pattern Anal Appl 16(3):447–457MathSciNetCrossRef Mariolis IG, Dermatas E (2013) Automatic classification of seam pucker images based on ordinal quality grades. Pattern Anal Appl 16(3):447–457MathSciNetCrossRef
37.
go back to reference Potharst R, Feelders AJ (2002) Classification trees for problems with monotonicity constraints. SIGKDD Explor 4(1):1–10CrossRef Potharst R, Feelders AJ (2002) Classification trees for problems with monotonicity constraints. SIGKDD Explor 4(1):1–10CrossRef
38.
go back to reference Potharst R, Ben-David A, van Wezel MC (2009) Two algorithms for generating structured and unstructured monotone ordinal datasets. Eng Appl Artif Intel 22(4–5):491–496CrossRef Potharst R, Ben-David A, van Wezel MC (2009) Two algorithms for generating structured and unstructured monotone ordinal datasets. Eng Appl Artif Intel 22(4–5):491–496CrossRef
39.
go back to reference Salzberg S (1991) A nearest hyperrectangle learning method. Mach Learn 6(3):251–276 Salzberg S (1991) A nearest hyperrectangle learning method. Mach Learn 6(3):251–276
40.
go back to reference Triguero I, Peralta D, Bacardit J, García S, Herrera F (2015) MRPR: a mapreduce solution for prototype reduction in big data classification. Neurocomputing 150:331–345CrossRef Triguero I, Peralta D, Bacardit J, García S, Herrera F (2015) MRPR: a mapreduce solution for prototype reduction in big data classification. Neurocomputing 150:331–345CrossRef
41.
go back to reference Wettschereck D, Dietterich TG (1995) An experimental comparison of the nearest-neighbor and nearest-hyperrectangle algorithms. Mach Learn 19(1):5–27 Wettschereck D, Dietterich TG (1995) An experimental comparison of the nearest-neighbor and nearest-hyperrectangle algorithms. Mach Learn 19(1):5–27
42.
go back to reference Wolpert DH (1996) The lack of a priori distinctions between learning algorithms. Neural Comput 8(7):1341–1390CrossRef Wolpert DH (1996) The lack of a priori distinctions between learning algorithms. Neural Comput 8(7):1341–1390CrossRef
Metadata
Title
MoNGEL: monotonic nested generalized exemplar learning
Authors
Javier García
Habib M. Fardoun
Daniyal M. Alghazzawi
José-Ramón Cano
Salvador García
Publication date
29-07-2015
Publisher
Springer London
Published in
Pattern Analysis and Applications / Issue 2/2017
Print ISSN: 1433-7541
Electronic ISSN: 1433-755X
DOI
https://doi.org/10.1007/s10044-015-0506-y

Other articles of this Issue 2/2017

Pattern Analysis and Applications 2/2017 Go to the issue

Premium Partner