Skip to main content
Erschienen in: Soft Computing 24/2017

30.07.2016 | Methodologies and Application

Cautious classification with nested dichotomies and imprecise probabilities

verfasst von: Gen Yang, Sébastien Destercke, Marie-Hélène Masson

Erschienen in: Soft Computing | Ausgabe 24/2017

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In some applications of machine learning and information retrieval (e.g. medical diagnosis, image recognition, pre-classification...), it can be preferable to provide less informative but more reliable predictions. This can be done by making partial predictions in the form of class subsets when the available information is insufficient to provide a reliable unique class. Imprecise probabilistic approaches offer nice tools to learn models from which such cautious predictions can be produced. However, the learning and inference processes of such models are computationally harder than their precise counterparts. In this paper, we introduce and study a particular binary decomposition strategy, nested dichotomies, that offer computational advantages in both the learning (due to the binarization process) and the inference (due to the decomposition strategy) processes. We show with experiments that these computational advantages do not lower the performances of the classifiers, and can even improve them when the class space has some structure.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Literatur
Zurück zum Zitat Abellán J, Masegosa A (2012) Imprecise classification with credal decision trees. Int J Uncertain Fuzziness Knowl Based Syst 20(05):763–787CrossRefMathSciNet Abellán J, Masegosa A (2012) Imprecise classification with credal decision trees. Int J Uncertain Fuzziness Knowl Based Syst 20(05):763–787CrossRefMathSciNet
Zurück zum Zitat Allwein E, Schapire R, Singer Y, Kaelbling P (2000) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141MATHMathSciNet Allwein E, Schapire R, Singer Y, Kaelbling P (2000) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141MATHMathSciNet
Zurück zum Zitat Alonso J, Del Coz J, Díez J, Luaces O, Bahamonde A (2008) Learning to predict one or more ranks in ordinal regression tasks. In: Conference on machine learning and knowledge discovery in databases. Springer, pp 39–54 Alonso J, Del Coz J, Díez J, Luaces O, Bahamonde A (2008) Learning to predict one or more ranks in ordinal regression tasks. In: Conference on machine learning and knowledge discovery in databases. Springer, pp 39–54
Zurück zum Zitat Bengio S, Weston J, Grangier D (2010) Label embedding trees for large multi-class tasks. NIPS 23:3 Bengio S, Weston J, Grangier D (2010) Label embedding trees for large multi-class tasks. NIPS 23:3
Zurück zum Zitat Bernard J-M (2005) An introduction to the imprecise dirichlet model for multinomial data. Intl J Approx Reason 39(2–3):123–150CrossRefMATHMathSciNet Bernard J-M (2005) An introduction to the imprecise dirichlet model for multinomial data. Intl J Approx Reason 39(2–3):123–150CrossRefMATHMathSciNet
Zurück zum Zitat Cesa-Bianchi N, Freund Y, Haussler D, Helmbold DP, Schapire RE, Warmuth MK (1997) How to use expert advice. JACM 44(3):427–485CrossRefMATHMathSciNet Cesa-Bianchi N, Freund Y, Haussler D, Helmbold DP, Schapire RE, Warmuth MK (1997) How to use expert advice. JACM 44(3):427–485CrossRefMATHMathSciNet
Zurück zum Zitat Chow C (1970) An optimum recognition error and reject tradeoff. IEEE Trans Inf Theory 16(1):41–46CrossRefMATH Chow C (1970) An optimum recognition error and reject tradeoff. IEEE Trans Inf Theory 16(1):41–46CrossRefMATH
Zurück zum Zitat Corani G, Antonucci A, De Rosa R (2012) Compression-based AODE classifiers. In: European conference on artificial intelligence, pp 264–269 Corani G, Antonucci A, De Rosa R (2012) Compression-based AODE classifiers. In: European conference on artificial intelligence, pp 264–269
Zurück zum Zitat Corani G, Mignatti A (2015) Credal model averaging for classification: representing prior ignorance and expert opinions. Intl J Approx Reason 56:264–277CrossRefMATHMathSciNet Corani G, Mignatti A (2015) Credal model averaging for classification: representing prior ignorance and expert opinions. Intl J Approx Reason 56:264–277CrossRefMATHMathSciNet
Zurück zum Zitat Corani G, Zaffalon M (2008) Credal model averaging: an extension of bayesian model averaging to imprecise probabilities. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, pp 257–271 Corani G, Zaffalon M (2008) Credal model averaging: an extension of bayesian model averaging to imprecise probabilities. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, pp 257–271
Zurück zum Zitat De Cooman G, Hermans F (2008) Imprecise probability trees: bridging two theories of imprecise probability. Artif Intell 172:1400–1427CrossRefMATHMathSciNet De Cooman G, Hermans F (2008) Imprecise probability trees: bridging two theories of imprecise probability. Artif Intell 172:1400–1427CrossRefMATHMathSciNet
Zurück zum Zitat del Coz J, Bahamonde A (2009) Learning nondeterministic classifiers. J Mach Learn Res 10:2273–2293MATHMathSciNet del Coz J, Bahamonde A (2009) Learning nondeterministic classifiers. J Mach Learn Res 10:2273–2293MATHMathSciNet
Zurück zum Zitat Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MATHMathSciNet Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MATHMathSciNet
Zurück zum Zitat Destercke S, Quost B (2011) Combining binary classifiers with imprecise probabilities. In: Proceedings of the 2011 international conference on Integrated uncertainty in knowledge modelling and decision making, IUKM’11. Springer, Berlin, pp 219–230 Destercke S, Quost B (2011) Combining binary classifiers with imprecise probabilities. In: Proceedings of the 2011 international conference on Integrated uncertainty in knowledge modelling and decision making, IUKM’11. Springer, Berlin, pp 219–230
Zurück zum Zitat Dietterich T, Bakiri G (1995) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263–286MATH Dietterich T, Bakiri G (1995) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263–286MATH
Zurück zum Zitat Elkan C (2001) The foundations of cost-sensitive learning. Int Jt Conf Artif Intell 17:973–978 Elkan C (2001) The foundations of cost-sensitive learning. Int Jt Conf Artif Intell 17:973–978
Zurück zum Zitat Fayyad U, Irani K (1993) Multi-interval discretization of continuous-valued attributes for classification learning. In: IJCAI, pp 1022–1029 Fayyad U, Irani K (1993) Multi-interval discretization of continuous-valued attributes for classification learning. In: IJCAI, pp 1022–1029
Zurück zum Zitat Fox J (1997) Applied regression analysis, linear models, and related methods. Sage, Beverly Hills Fox J (1997) Applied regression analysis, linear models, and related methods. Sage, Beverly Hills
Zurück zum Zitat Frank E, Hall M (2001) A simple approach to ordinal classification. In Proceedings of the 12th European conference on machine learning. Springer, pp 145–156 Frank E, Hall M (2001) A simple approach to ordinal classification. In Proceedings of the 12th European conference on machine learning. Springer, pp 145–156
Zurück zum Zitat Frank E, Kramer S (2004) Ensembles of nested dichotomies for multi-class problems. In: ICML 2004, p 39 Frank E, Kramer S (2004) Ensembles of nested dichotomies for multi-class problems. In: ICML 2004, p 39
Zurück zum Zitat Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200):675–701CrossRefMATH Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200):675–701CrossRefMATH
Zurück zum Zitat Grunbaum B, Perles MA, Shephard GC (1967) Convex polytopes. Springer, BerlinMATH Grunbaum B, Perles MA, Shephard GC (1967) Convex polytopes. Springer, BerlinMATH
Zurück zum Zitat Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning. Springer, BerlinCrossRefMATH Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning. Springer, BerlinCrossRefMATH
Zurück zum Zitat Levi I (1983) The enterprise of knowledge: an essay on knowledge, credal probability, and chance. MIT Press, Cambridge Levi I (1983) The enterprise of knowledge: an essay on knowledge, credal probability, and chance. MIT Press, Cambridge
Zurück zum Zitat Lorena AC, De Carvalho A (2010) Building binary-tree-based multiclass classifiers using separability measures. Neurocomputing 73(16–18):2837–2845CrossRef Lorena AC, De Carvalho A (2010) Building binary-tree-based multiclass classifiers using separability measures. Neurocomputing 73(16–18):2837–2845CrossRef
Zurück zum Zitat Mantas C, Abellan J (2014) Credal-c4.5: decision tree based on imprecise probabilities to classify noisy data. Expert Syst Appl 41(10):4625–4637CrossRef Mantas C, Abellan J (2014) Credal-c4.5: decision tree based on imprecise probabilities to classify noisy data. Expert Syst Appl 41(10):4625–4637CrossRef
Zurück zum Zitat Masnadi-Shirazi H, Vasconcelos N (2010) Risk minimization, probability elicitation, and cost-sensitive SVMs. In: International conference machine learning, pp 759–766 Masnadi-Shirazi H, Vasconcelos N (2010) Risk minimization, probability elicitation, and cost-sensitive SVMs. In: International conference machine learning, pp 759–766
Zurück zum Zitat Nemenyi P (1963) Distribution-free multiple comparisons. Ph.D. thesis, Princeton University Nemenyi P (1963) Distribution-free multiple comparisons. Ph.D. thesis, Princeton University
Zurück zum Zitat Rokach L (2006) Decomposition methodology for classification tasks: a meta decomposer framework. Pattern Anal Appl 9(2–3):257–271CrossRefMathSciNet Rokach L (2006) Decomposition methodology for classification tasks: a meta decomposer framework. Pattern Anal Appl 9(2–3):257–271CrossRefMathSciNet
Zurück zum Zitat Rokach L (2010) Ensemble-based classifiers. Artif Intell Rev 33(1–2):1–39 Rokach L (2010) Ensemble-based classifiers. Artif Intell Rev 33(1–2):1–39
Zurück zum Zitat Walley P (1991) Statistical reasoning with imprecise probabilities. Chapman and Hall, LondonCrossRefMATH Walley P (1991) Statistical reasoning with imprecise probabilities. Chapman and Hall, LondonCrossRefMATH
Zurück zum Zitat Walley P (1996) Inferences from multinomial data: learning about a bag of marbles. J Roy Stat Soc Ser B Methodol 58(1):3–57 Walley P (1996) Inferences from multinomial data: learning about a bag of marbles. J Roy Stat Soc Ser B Methodol 58(1):3–57
Zurück zum Zitat Wu T, Lin C, Weng R (2004) Probability estimates for multi-class classification by pairwise coupling. J Mach Learn Res 5:975–1005MATHMathSciNet Wu T, Lin C, Weng R (2004) Probability estimates for multi-class classification by pairwise coupling. J Mach Learn Res 5:975–1005MATHMathSciNet
Zurück zum Zitat Xu P, Davoine F, Zha H, Denoeux T (2015) Evidential calibration of binary svm classifiers. Int J Approx Reason 72:55–70 Xu P, Davoine F, Zha H, Denoeux T (2015) Evidential calibration of binary svm classifiers. Int J Approx Reason 72:55–70
Zurück zum Zitat Yang G, Destercke S, Masson M-H (2014) Nested dichotomies with probability sets for multi-class classification. In: European conference on artificial intelligence Yang G, Destercke S, Masson M-H (2014) Nested dichotomies with probability sets for multi-class classification. In: European conference on artificial intelligence
Zurück zum Zitat Zaffalon M, Corani G, Maua D (2012) Evaluating credal classifiers by utility-discounted predictive accuracy. Int J Approx Reason 53(8):1282–1301CrossRefMATHMathSciNet Zaffalon M, Corani G, Maua D (2012) Evaluating credal classifiers by utility-discounted predictive accuracy. Int J Approx Reason 53(8):1282–1301CrossRefMATHMathSciNet
Metadaten
Titel
Cautious classification with nested dichotomies and imprecise probabilities
verfasst von
Gen Yang
Sébastien Destercke
Marie-Hélène Masson
Publikationsdatum
30.07.2016
Verlag
Springer Berlin Heidelberg
Erschienen in
Soft Computing / Ausgabe 24/2017
Print ISSN: 1432-7643
Elektronische ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-016-2287-7

Weitere Artikel der Ausgabe 24/2017

Soft Computing 24/2017 Zur Ausgabe

Premium Partner