Skip to main content
Erschienen in: Advances in Data Analysis and Classification 3/2019

31.07.2018 | Regular Article

On support vector machines under a multiple-cost scenario

verfasst von: Sandra Benítez-Peña, Rafael Blanquero, Emilio Carrizosa, Pepa Ramírez-Cobo

Erschienen in: Advances in Data Analysis and Classification | Ausgabe 3/2019

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Support vector machine (SVM) is a powerful tool in binary classification, known to attain excellent misclassification rates. On the other hand, many realworld classification problems, such as those found in medical diagnosis, churn or fraud prediction, involve misclassification costs which may be different in the different classes. However, it may be hard for the user to provide precise values for such misclassification costs, whereas it may be much easier to identify acceptable misclassification rates values. In this paper we propose a novel SVM model in which misclassification costs are considered by incorporating performance constraints in the problem formulation. Specifically, our aim is to seek the hyperplane with maximal margin yielding misclassification rates below given threshold values. Such maximal margin hyperplane is obtained by solving a quadratic convex problem with linear constraints and integer variables. The reported numerical experience shows that our model gives the user control on the misclassification rates in one class (possibly at the expense of an increase in misclassification rates for the other class) and is feasible in terms of running times.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Literatur
Zurück zum Zitat Alcalá-Fdez J, Sanchez L, Garcia S, del Jesus MJ, Ventura S, Garrell JM, Otero J, Romero C, Bacardit J, Rivas VM et al (2009) Keel: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput 13(3):307–318CrossRef Alcalá-Fdez J, Sanchez L, Garcia S, del Jesus MJ, Ventura S, Garrell JM, Otero J, Romero C, Bacardit J, Rivas VM et al (2009) Keel: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput 13(3):307–318CrossRef
Zurück zum Zitat Allwein EL, Schapire RE, Singer Y (2001) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141MathSciNetMATH Allwein EL, Schapire RE, Singer Y (2001) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141MathSciNetMATH
Zurück zum Zitat Bertsimas D, Weismantel R (2005) Optimization over integers. Dynamic Ideas, Belmont Bertsimas D, Weismantel R (2005) Optimization over integers. Dynamic Ideas, Belmont
Zurück zum Zitat Bewick V, Cheek L, Ball J (2004) Statistics review 13: receiver operating characteristic curves. Crit Care 8(6):508–512CrossRef Bewick V, Cheek L, Ball J (2004) Statistics review 13: receiver operating characteristic curves. Crit Care 8(6):508–512CrossRef
Zurück zum Zitat Bonami P, Biegler LT, Conn AR, Cornujols G, Grossmann IE, Laird CD, Lee J, Lodi A, Margot F, Sawaya N, Wchter A (2008) An algorithmic framework for convex mixed integer nonlinear programs. Discrete Optim 5(2):186–204 (in Memory of George B. Dantzig)MathSciNetMATHCrossRef Bonami P, Biegler LT, Conn AR, Cornujols G, Grossmann IE, Laird CD, Lee J, Lodi A, Margot F, Sawaya N, Wchter A (2008) An algorithmic framework for convex mixed integer nonlinear programs. Discrete Optim 5(2):186–204 (in Memory of George B. Dantzig)MathSciNetMATHCrossRef
Zurück zum Zitat Bradford JP, Kunz C, Kohavi R, Brunk C, Brodley CE (1998) Pruning decision trees with misclassification costs. In: Proceedings of the 10th European conference on machine learning, ECML’98. Springer, pp. 131–136 Bradford JP, Kunz C, Kohavi R, Brunk C, Brodley CE (1998) Pruning decision trees with misclassification costs. In: Proceedings of the 10th European conference on machine learning, ECML’98. Springer, pp. 131–136
Zurück zum Zitat Burer S, Letchford AN (2012) Non-convex mixed-integer nonlinear programming: a survey. Surv Oper Res Manag Sci 17(2):97–106MathSciNet Burer S, Letchford AN (2012) Non-convex mixed-integer nonlinear programming: a survey. Surv Oper Res Manag Sci 17(2):97–106MathSciNet
Zurück zum Zitat Burges CJ (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2(2):121–167CrossRef Burges CJ (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2(2):121–167CrossRef
Zurück zum Zitat Camm JD, Raturi AS, Tsubakitani S (1990) Cutting big M down to size. Interfaces 20(5):61–66CrossRef Camm JD, Raturi AS, Tsubakitani S (1990) Cutting big M down to size. Interfaces 20(5):61–66CrossRef
Zurück zum Zitat Carrizosa E, Martin-Barragan B, Romero Morales D (2008) Multi-group support vector machines with measurement costs: a biobjective approach. Discrete Appl Math 156(6):950–966MathSciNetMATHCrossRef Carrizosa E, Martin-Barragan B, Romero Morales D (2008) Multi-group support vector machines with measurement costs: a biobjective approach. Discrete Appl Math 156(6):950–966MathSciNetMATHCrossRef
Zurück zum Zitat Carrizosa E, Romero Morales D (2013) Supervised classification and mathematical optimization. Comput Oper Res 40(1):150–165MathSciNetMATHCrossRef Carrizosa E, Romero Morales D (2013) Supervised classification and mathematical optimization. Comput Oper Res 40(1):150–165MathSciNetMATHCrossRef
Zurück zum Zitat Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297MATH Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297MATH
Zurück zum Zitat Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, New YorkMATHCrossRef Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, New YorkMATHCrossRef
Zurück zum Zitat Datta S, Das S (2015) Near-Bayesian support vector machines for imbalanced data classification with equal or unequal misclassification costs. Neural Netw 70:39–52MATHCrossRef Datta S, Das S (2015) Near-Bayesian support vector machines for imbalanced data classification with equal or unequal misclassification costs. Neural Netw 70:39–52MATHCrossRef
Zurück zum Zitat Freitas A, Costa-Pereira A, Brazdil P (2007) Cost-sensitive decision trees applied to medical data. In: Data warehousing and knowledge discovery: 9th international conference, DaWaK 2007, Regensburg Germany, September 3–7, 2007. Proceedings. Springer, Berlin, pp 303–312 Freitas A, Costa-Pereira A, Brazdil P (2007) Cost-sensitive decision trees applied to medical data. In: Data warehousing and knowledge discovery: 9th international conference, DaWaK 2007, Regensburg Germany, September 3–7, 2007. Proceedings. Springer, Berlin, pp 303–312
Zurück zum Zitat Guo J (2010) Simultaneous variable selection and class fusion for high-dimensional linear discriminant analysis. Biostatistics 11(4):599–608MathSciNetCrossRef Guo J (2010) Simultaneous variable selection and class fusion for high-dimensional linear discriminant analysis. Biostatistics 11(4):599–608MathSciNetCrossRef
Zurück zum Zitat Hastie T, Tibshirani R, Friedman J (2001) The elements of statistical learning. Springer series in statistics. Springer, New YorkMATHCrossRef Hastie T, Tibshirani R, Friedman J (2001) The elements of statistical learning. Springer series in statistics. Springer, New YorkMATHCrossRef
Zurück zum Zitat He H, Ma Y (2013) Imbalanced learning: foundations, algorithms, and applications. Wiley, HobokenMATHCrossRef He H, Ma Y (2013) Imbalanced learning: foundations, algorithms, and applications. Wiley, HobokenMATHCrossRef
Zurück zum Zitat Hsu CW, Chang CC, Lin CJ et al (2003) A practical guide to support vector classification. Tech. rep., Department of Computer Science, National Taiwan University Hsu CW, Chang CC, Lin CJ et al (2003) A practical guide to support vector classification. Tech. rep., Department of Computer Science, National Taiwan University
Zurück zum Zitat Kohavi R et al (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: IJCAI, vol 14. Stanford, CA, pp 1137–1145 Kohavi R et al (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: IJCAI, vol 14. Stanford, CA, pp 1137–1145
Zurück zum Zitat Lin Y, Lee Y, Wahba G (2002) Support vector machines for classification in nonstandard situations. Mach Learn 46(1–3):191–202MATHCrossRef Lin Y, Lee Y, Wahba G (2002) Support vector machines for classification in nonstandard situations. Mach Learn 46(1–3):191–202MATHCrossRef
Zurück zum Zitat Lyu T, Lock EF, Eberly LE (2017) Discriminating sample groups with multi-way data. Biostatistics 18:434–450MathSciNet Lyu T, Lock EF, Eberly LE (2017) Discriminating sample groups with multi-way data. Biostatistics 18:434–450MathSciNet
Zurück zum Zitat Maldonado S, Prez J, Bravo C (2017) Cost-based feature selection for support vector machines: an application in credit scoring. Eur J Oper Res 261(2):656–665MathSciNetMATHCrossRef Maldonado S, Prez J, Bravo C (2017) Cost-based feature selection for support vector machines: an application in credit scoring. Eur J Oper Res 261(2):656–665MathSciNetMATHCrossRef
Zurück zum Zitat Mansouri K, Ringsted T, Ballabio D, Todeschini R, Consonni V (2013) Quantitative structureactivity relationship models for ready biodegradability of chemicals. J Chem Inf Model 53(4):867–878CrossRef Mansouri K, Ringsted T, Ballabio D, Todeschini R, Consonni V (2013) Quantitative structureactivity relationship models for ready biodegradability of chemicals. J Chem Inf Model 53(4):867–878CrossRef
Zurück zum Zitat Mercer J (1909) Functions of positive and negative type, and their connection with the theory of integral equations. Philos Trans R Soc Lond Ser A 209:415–446MATHCrossRef Mercer J (1909) Functions of positive and negative type, and their connection with the theory of integral equations. Philos Trans R Soc Lond Ser A 209:415–446MATHCrossRef
Zurück zum Zitat Prati RC, Batista GE, Silva DF (2015) Class imbalance revisited: a new experimental setup to assess the performance of treatment methods. Knowl Inf Syst 45(1):247–270CrossRef Prati RC, Batista GE, Silva DF (2015) Class imbalance revisited: a new experimental setup to assess the performance of treatment methods. Knowl Inf Syst 45(1):247–270CrossRef
Zurück zum Zitat Sánchez BN, Wu M, Song PXK, Wang W (2016) Study design in high-dimensional classification analysis. Biostatistics 17(4):722MathSciNetCrossRef Sánchez BN, Wu M, Song PXK, Wang W (2016) Study design in high-dimensional classification analysis. Biostatistics 17(4):722MathSciNetCrossRef
Zurück zum Zitat Tang Y, Zhang YQ, Chawla NV, Krasser S (2009) SVMs modeling for highly imbalanced classification. IEEE Trans Syst Man Cybern B (Cybern) 39(1):281–288CrossRef Tang Y, Zhang YQ, Chawla NV, Krasser S (2009) SVMs modeling for highly imbalanced classification. IEEE Trans Syst Man Cybern B (Cybern) 39(1):281–288CrossRef
Zurück zum Zitat Van Rossum G, Drake FL (2011) An Introduction to Python. Network Theory Ltd, United Kingdom Van Rossum G, Drake FL (2011) An Introduction to Python. Network Theory Ltd, United Kingdom
Zurück zum Zitat Vapnik VN (1998) Statistical learning theory, vol 1, 1st edn. Wiley, New YorkMATH Vapnik VN (1998) Statistical learning theory, vol 1, 1st edn. Wiley, New YorkMATH
Zurück zum Zitat Yao Y, Lee Y (2014) Another look at linear programming for feature selection via methods of regularization. Stat Comput 24(5):885–905MathSciNetMATHCrossRef Yao Y, Lee Y (2014) Another look at linear programming for feature selection via methods of regularization. Stat Comput 24(5):885–905MathSciNetMATHCrossRef
Metadaten
Titel
On support vector machines under a multiple-cost scenario
verfasst von
Sandra Benítez-Peña
Rafael Blanquero
Emilio Carrizosa
Pepa Ramírez-Cobo
Publikationsdatum
31.07.2018
Verlag
Springer Berlin Heidelberg
Erschienen in
Advances in Data Analysis and Classification / Ausgabe 3/2019
Print ISSN: 1862-5347
Elektronische ISSN: 1862-5355
DOI
https://doi.org/10.1007/s11634-018-0330-5

Weitere Artikel der Ausgabe 3/2019

Advances in Data Analysis and Classification 3/2019 Zur Ausgabe

Premium Partner