Skip to main content
Erschienen in: Soft Computing 20/2018

09.08.2017 | Methodologies and Application

A proximal quadratic surface support vector machine for semi-supervised binary classification

verfasst von: Xin Yan, Yanqin Bai, Shu-Cherng Fang, Jian Luo

Erschienen in: Soft Computing | Ausgabe 20/2018

Einloggen

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Semi-supervised support vector machine is a popular method in the research area of machine learning. Considering a large amount of unlabeled data points in real-life world, the semi-supervised support machine has the ability of good generalization for dealing with nonlinear classification problems. In this paper, a proximal quadratic surface support vector machine model is proposed for semi-supervised binary classification. The main advantage of our new model is that the proximal quadratic surfaces are constructed directly for nonlinear classification instead of using the kernel function, which avoids the tasks of choosing kernels and tuning their parameters. We reformulate this proposed model as an unconstrained mixed-integer quadratic programming problem. Semi-definite relaxation is then adopted, and a primal alternating direction method is further proposed for fast computation. We test the proposed method on some artificial and public benchmark data sets. Preliminary results indicate that our method outperforms some well-known methods for semi-supervised classification in terms of the efficiency and classifying accuracy.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Astorino A, Fuduli A (2007) Nonsmooth optimization techniques for semisupervised classification. IEEE Trans Pattern Anal 29:2135–2142CrossRef Astorino A, Fuduli A (2007) Nonsmooth optimization techniques for semisupervised classification. IEEE Trans Pattern Anal 29:2135–2142CrossRef
Zurück zum Zitat Astorino A, Fuduli A (2015b) Support vector machine polyhedral separability in semisupervised learning. J Optim Theory Appl 164:1039–1050MathSciNetCrossRef Astorino A, Fuduli A (2015b) Support vector machine polyhedral separability in semisupervised learning. J Optim Theory Appl 164:1039–1050MathSciNetCrossRef
Zurück zum Zitat Bai Y, Yan X (2016) Conic relaxation for semi-supervised support vector machines. J Optim Theory Appl 169:299–313MathSciNetCrossRef Bai Y, Yan X (2016) Conic relaxation for semi-supervised support vector machines. J Optim Theory Appl 169:299–313MathSciNetCrossRef
Zurück zum Zitat Bai Y, Chen Y, Niu B (2012) SDP relaxation for semi-supervised support vector machine. Pac J Optim 8(1):3–14MathSciNetMATH Bai Y, Chen Y, Niu B (2012) SDP relaxation for semi-supervised support vector machine. Pac J Optim 8(1):3–14MathSciNetMATH
Zurück zum Zitat Bai Y, Niu B, Chen Y (2013) New SDP models for protein homology detection with semi-supervised SVM. Optimization 62(4):561–572MathSciNetCrossRef Bai Y, Niu B, Chen Y (2013) New SDP models for protein homology detection with semi-supervised SVM. Optimization 62(4):561–572MathSciNetCrossRef
Zurück zum Zitat Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434MathSciNetMATH Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434MathSciNetMATH
Zurück zum Zitat Bennett K, Demiriz A (1999) Semi-supervised support vector machines. In: Kearns MJ, Solla SA, Cohn DA (eds) Advances in neural information processing systems 11. MIT Press, Cambridge, pp 368–374 Bennett K, Demiriz A (1999) Semi-supervised support vector machines. In: Kearns MJ, Solla SA, Cohn DA (eds) Advances in neural information processing systems 11. MIT Press, Cambridge, pp 368–374
Zurück zum Zitat Cao L, Tay FEH (2003) Support vector machine with adaptive parameters in financial time series forecasting. IEEE Trans Neural Netw Learn Syst 14(6):1506–1518CrossRef Cao L, Tay FEH (2003) Support vector machine with adaptive parameters in financial time series forecasting. IEEE Trans Neural Netw Learn Syst 14(6):1506–1518CrossRef
Zurück zum Zitat Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, pp 57–64 Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, pp 57–64
Zurück zum Zitat Chapelle O, Schölkopf B, Zien A (2006) Semi-supervised learning. MIT Press, CambridgeCrossRef Chapelle O, Schölkopf B, Zien A (2006) Semi-supervised learning. MIT Press, CambridgeCrossRef
Zurück zum Zitat Chapelle O, Sindhwani V, Keerthi SS (2008) Optimization techniques for semi-supervised support vector machines. J Mach Learn Res 9:203–233MATH Chapelle O, Sindhwani V, Keerthi SS (2008) Optimization techniques for semi-supervised support vector machines. J Mach Learn Res 9:203–233MATH
Zurück zum Zitat Collobert R, Sinz F, Weston J, Bottou L (2006) Large scale transductive SVMs. J Mach Learn Res 7:1687–1712MathSciNetMATH Collobert R, Sinz F, Weston J, Bottou L (2006) Large scale transductive SVMs. J Mach Learn Res 7:1687–1712MathSciNetMATH
Zurück zum Zitat De Bie T, Cristianini N (2004) Convex methods for transduction. In: Thrun S, Saul LK, Schölkopf B (eds) Advances in neural information processing systems 16. MIT Press, Cambridge, pp 73–80 De Bie T, Cristianini N (2004) Convex methods for transduction. In: Thrun S, Saul LK, Schölkopf B (eds) Advances in neural information processing systems 16. MIT Press, Cambridge, pp 73–80
Zurück zum Zitat Deng N, Tian Y, Zhang C (2012) Support vector machines-optimization based theory, algorithms and extensions. CRC Press, Boca RatonCrossRef Deng N, Tian Y, Zhang C (2012) Support vector machines-optimization based theory, algorithms and extensions. CRC Press, Boca RatonCrossRef
Zurück zum Zitat Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceedings of 7th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 77–86 Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceedings of 7th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 77–86
Zurück zum Zitat He B, Xu M, Yuan X (2011) Solving large-scale least squares semidefinite programming by alternating direction methods. SIAM J Matrix Anal Appl 32(1):136–152MathSciNetCrossRef He B, Xu M, Yuan X (2011) Solving large-scale least squares semidefinite programming by alternating direction methods. SIAM J Matrix Anal Appl 32(1):136–152MathSciNetCrossRef
Zurück zum Zitat Jayadeva RK, Suresh C (2007) Fuzzy multi-category proximal support vector classification via generalized eigenvalues. Soft Comput 11(7):679–685CrossRef Jayadeva RK, Suresh C (2007) Fuzzy multi-category proximal support vector classification via generalized eigenvalues. Soft Comput 11(7):679–685CrossRef
Zurück zum Zitat Joachims T (1999) Transductive inference for text classification using support vector machines. In: Proceedings of 16th international conference on machine learning, pp 200–209 Joachims T (1999) Transductive inference for text classification using support vector machines. In: Proceedings of 16th international conference on machine learning, pp 200–209
Zurück zum Zitat Luo J (2014) Quadratic surface support vector machines with applications. Ph.D. thesis, North Carolina State University Luo J (2014) Quadratic surface support vector machines with applications. Ph.D. thesis, North Carolina State University
Zurück zum Zitat Melacci S, Belkin M (2011) Laplacian support vector machines trained in the primal. J Mach Learn Res 12:1149–1184MathSciNetMATH Melacci S, Belkin M (2011) Laplacian support vector machines trained in the primal. J Mach Learn Res 12:1149–1184MathSciNetMATH
Zurück zum Zitat Osuna E, Freund R, Girosi F (1997) Training support vector machines: an application to face detection. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 130–136 Osuna E, Freund R, Girosi F (1997) Training support vector machines: an application to face detection. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 130–136
Zurück zum Zitat Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53CrossRef Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53CrossRef
Zurück zum Zitat Sindhwani V, Keerthi SS, Chapelle O (2006) Deterministic annealing for semi-supervised kernel machines. In: Proceedings of 23rd international conference on machine learning, pp 841–848 Sindhwani V, Keerthi SS, Chapelle O (2006) Deterministic annealing for semi-supervised kernel machines. In: Proceedings of 23rd international conference on machine learning, pp 841–848
Zurück zum Zitat Sun J, Zhang S (2010) A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs. Eur J Oper Res 207(3):1210–1220MathSciNetCrossRef Sun J, Zhang S (2010) A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs. Eur J Oper Res 207(3):1210–1220MathSciNetCrossRef
Zurück zum Zitat Sun L, Jing L, Xia XD (2006) A new proximal support vector machine for semi-supervised classification. In: Wang J, Yi Z, Zurada JM, Lu B, Yin H (eds) Advances in neural networks-ISNN 2006, Lecture notes in computer science, vol 3971. Springer, Heidelburg, pp 1076–1082CrossRef Sun L, Jing L, Xia XD (2006) A new proximal support vector machine for semi-supervised classification. In: Wang J, Yi Z, Zurada JM, Lu B, Yin H (eds) Advances in neural networks-ISNN 2006, Lecture notes in computer science, vol 3971. Springer, Heidelburg, pp 1076–1082CrossRef
Zurück zum Zitat Sun D, Toh KC, Yang L (2015) A convergent 3-block semiproximal alternating direction method of multipliers for conic programming with 4-type constraints. SIAM J Optim 25(2):882–915MathSciNetCrossRef Sun D, Toh KC, Yang L (2015) A convergent 3-block semiproximal alternating direction method of multipliers for conic programming with 4-type constraints. SIAM J Optim 25(2):882–915MathSciNetCrossRef
Zurück zum Zitat Tian Y, Luo J (2017) A new branch-and-bound approach to semi-supervised support vector machine. Soft Comput 21(1):245–254CrossRef Tian Y, Luo J (2017) A new branch-and-bound approach to semi-supervised support vector machine. Soft Comput 21(1):245–254CrossRef
Zurück zum Zitat Valizadegan H, Jin R (2006) Generalized maximum margin clustering and unsupervised kernel learning. In: Schölkopf B, Platt J, Hofmann T (eds) Advances in neural information processing systems 19. MIT Press, Cambridge, pp 1417–1424 Valizadegan H, Jin R (2006) Generalized maximum margin clustering and unsupervised kernel learning. In: Schölkopf B, Platt J, Hofmann T (eds) Advances in neural information processing systems 19. MIT Press, Cambridge, pp 1417–1424
Zurück zum Zitat Wen Z, Goldfarb D, Yin W (2010) Alternating direction augmented Lagrangian methods for semidefinite programming. Math Prog Comp 2(3–4):203–230MathSciNetCrossRef Wen Z, Goldfarb D, Yin W (2010) Alternating direction augmented Lagrangian methods for semidefinite programming. Math Prog Comp 2(3–4):203–230MathSciNetCrossRef
Zurück zum Zitat Xu L, Schuurmans D (2005) Unsupervised and semi-supervised multi-class support vector machines. In: Proceedings of 20th National conference on artificial intelligence, pp 904–910 Xu L, Schuurmans D (2005) Unsupervised and semi-supervised multi-class support vector machines. In: Proceedings of 20th National conference on artificial intelligence, pp 904–910
Zurück zum Zitat Xu Z, Jin R, Zhu J, King I, Lyu M (2008) Efficient convex relaxation for transductive support vector machine. In: Platt JC, Koller D, Singer Y, Roweis S (eds) Advances in neural information processing systems 20. MIT Press, Cambridge, pp 1641–1648 Xu Z, Jin R, Zhu J, King I, Lyu M (2008) Efficient convex relaxation for transductive support vector machine. In: Platt JC, Koller D, Singer Y, Roweis S (eds) Advances in neural information processing systems 20. MIT Press, Cambridge, pp 1641–1648
Zurück zum Zitat Yan X, Bai Y, Fang SC, Luo J (2016) A kernel-free quadratic surface support vector machine for semi-supervised learning. J Oper Res Soc 67(7):1001–1011CrossRef Yan X, Bai Y, Fang SC, Luo J (2016) A kernel-free quadratic surface support vector machine for semi-supervised learning. J Oper Res Soc 67(7):1001–1011CrossRef
Zurück zum Zitat Zhao B, Wang F, Zhang C (2008) CutS3VM: a fast semi-supervised SVM algorithm. In: Proceedings of 4th ACM SIGKDD international conference on knowledge discovery and data mining, pp 830–838 Zhao B, Wang F, Zhang C (2008) CutS3VM: a fast semi-supervised SVM algorithm. In: Proceedings of 4th ACM SIGKDD international conference on knowledge discovery and data mining, pp 830–838
Metadaten
Titel
A proximal quadratic surface support vector machine for semi-supervised binary classification
verfasst von
Xin Yan
Yanqin Bai
Shu-Cherng Fang
Jian Luo
Publikationsdatum
09.08.2017
Verlag
Springer Berlin Heidelberg
Erschienen in
Soft Computing / Ausgabe 20/2018
Print ISSN: 1432-7643
Elektronische ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-017-2751-z

Weitere Artikel der Ausgabe 20/2018

Soft Computing 20/2018 Zur Ausgabe