Skip to main content
Top

19-08-2024

Kernel-free Reduced Quadratic Surface Support Vector Machine with 0-1 Loss Function and L\(_p\)-norm Regularization

Authors: Mingyang Wu, Zhixia Yang

Published in: Annals of Data Science

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

This paper presents a novel nonlinear binary classification method, namely the kernel-free reduced quadratic surface support vector machine with 0-1 loss function and L\(_{p}\)-norm regularization (L\(_p\)-RQSSVM\(_{0/1}\)). It uses kernel-free trick aimed at finding a reduced quadratic surface to separate samples, without considering the cross terms in quadratic form. This saves computational costs and provides better interpretability than methods using kernel functions. In addition, adding the 0-1 loss function and L\(_p\)-norm regularization to construct our L\(_p\)-RQSSVM\(_{0/1}\) enables sample sparsity and feature sparsity. The support vector (SV) of L\(_p\)-RQSSVM\(_{0/1}\) is defined, and it is derived that all SVs fall on the support hypersurfaces. Moreover, the optimality condition is explored theoretically, and a new iterative algorithm based on the alternating direction method of multipliers (ADMM) framework is used to solve our L\(_p\)-RQSSVM\(_{0/1}\) on the selected working set. The computational complexity and convergence of the algorithm are discussed. Furthermore, numerical experiments demonstrate that our L\(_p\)-RQSSVM\(_{0/1}\) achieves better classification accuracy, less SVs, and higher computational efficiency than other methods on most datasets. It also has feature sparsity under certain conditions.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Literature
1.
go back to reference Shi Y (2022) Advances in big data analytics: theory, algorithm and practice. Springer, SingaporeCrossRef Shi Y (2022) Advances in big data analytics: theory, algorithm and practice. Springer, SingaporeCrossRef
2.
go back to reference Olson DL, Shi Y (2007) Introduction to business data mining. McGraw-Hill/Irwin, New York Olson DL, Shi Y (2007) Introduction to business data mining. McGraw-Hill/Irwin, New York
3.
go back to reference Shi Y, Tian YJ, Kou G, Peng Y, Li JP (2011) Optimization based data mining: theory and applications. Springer, BerlinCrossRef Shi Y, Tian YJ, Kou G, Peng Y, Li JP (2011) Optimization based data mining: theory and applications. Springer, BerlinCrossRef
4.
go back to reference Tien JM (2017) Internet of things, real-time decision making, and artificial intelligence. Ann Data Sci 4(2):149–178CrossRef Tien JM (2017) Internet of things, real-time decision making, and artificial intelligence. Ann Data Sci 4(2):149–178CrossRef
5.
go back to reference Jin W, Zhang JQ, Zhang X (2011) Face recognition method based on support vector machine and particle swarm optimization. Expert Sys Appl 38(4):4390–4393CrossRef Jin W, Zhang JQ, Zhang X (2011) Face recognition method based on support vector machine and particle swarm optimization. Expert Sys Appl 38(4):4390–4393CrossRef
6.
go back to reference Chaabane SB, Hijji M, Harrabi R, Seddik H (2022) Face recognition based on statistical features and SVM classifier. Multimedia Tools Appl 81(6):8767–8784CrossRef Chaabane SB, Hijji M, Harrabi R, Seddik H (2022) Face recognition based on statistical features and SVM classifier. Multimedia Tools Appl 81(6):8767–8784CrossRef
7.
go back to reference Hoang ND, Nguyen QL, Tien Bui D (2018) Image processing-based classification of asphalt pavement cracks using support vector machine optimized by artificial bee colony. J Comput Civ Eng 32(5):04018037CrossRef Hoang ND, Nguyen QL, Tien Bui D (2018) Image processing-based classification of asphalt pavement cracks using support vector machine optimized by artificial bee colony. J Comput Civ Eng 32(5):04018037CrossRef
8.
go back to reference Kumari VA, Chitra R (2013) Classification of diabetes disease using support vector machine. Int J Eng Res Appl 3(2):1797–1801 Kumari VA, Chitra R (2013) Classification of diabetes disease using support vector machine. Int J Eng Res Appl 3(2):1797–1801
9.
go back to reference Sethy PK, Barpanda NK, Rath AK, Behera SK (2020) Deep feature based rice leaf disease identification using support vector machine. Comput Electron Agric 175:105527CrossRef Sethy PK, Barpanda NK, Rath AK, Behera SK (2020) Deep feature based rice leaf disease identification using support vector machine. Comput Electron Agric 175:105527CrossRef
10.
go back to reference Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297CrossRef Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297CrossRef
11.
go back to reference Dagher I (2008) Quadratic kernel-free non-linear support vector machine. J Global Optim 41(1):15–30CrossRef Dagher I (2008) Quadratic kernel-free non-linear support vector machine. J Global Optim 41(1):15–30CrossRef
12.
go back to reference Luo J, Fang SC, Deng ZB, Guo XL (2016) Soft quadratic surface support vector machine for binary classification. Asia Pac J Oper Res 33(6):1650046CrossRef Luo J, Fang SC, Deng ZB, Guo XL (2016) Soft quadratic surface support vector machine for binary classification. Asia Pac J Oper Res 33(6):1650046CrossRef
13.
go back to reference Bai YQ, Han X, Chen T, Yu H (2015) Quadratic kernel-free least squares support vector machine for target diseases classification. J Combin Optim 30(4):850–870CrossRef Bai YQ, Han X, Chen T, Yu H (2015) Quadratic kernel-free least squares support vector machine for target diseases classification. J Combin Optim 30(4):850–870CrossRef
14.
go back to reference Tian Y, Yong ZY, Luo J (2018) A new approach for reject inference in credit scoring using kernel-free fuzzy quadratic surface support vector machines. Appl Soft Comput 73:96–105CrossRef Tian Y, Yong ZY, Luo J (2018) A new approach for reject inference in credit scoring using kernel-free fuzzy quadratic surface support vector machines. Appl Soft Comput 73:96–105CrossRef
15.
go back to reference Yan X, Bai YQ, Fang SC, Luo J (2016) A kernel-free quadratic surface support vector machine for semi-supervised learning. J Oper Res Soc 67(7):1001–1011CrossRef Yan X, Bai YQ, Fang SC, Luo J (2016) A kernel-free quadratic surface support vector machine for semi-supervised learning. J Oper Res Soc 67(7):1001–1011CrossRef
16.
go back to reference Gao QQ, Bai YQ, Zhan YR (2019) Quadratic kernel-free least square twin support vector machine for binary classification problems. J Oper Res Soc China 7(4):539–559CrossRef Gao QQ, Bai YQ, Zhan YR (2019) Quadratic kernel-free least square twin support vector machine for binary classification problems. J Oper Res Soc China 7(4):539–559CrossRef
17.
go back to reference Luo J, Yan X, Tian Y (2020) Unsupervised quadratic surface support vector machine with application to credit risk assessment. Eur J Oper Res 280(3):1008–1017CrossRef Luo J, Yan X, Tian Y (2020) Unsupervised quadratic surface support vector machine with application to credit risk assessment. Eur J Oper Res 280(3):1008–1017CrossRef
18.
go back to reference Gao ZM, Fang SC, Luo J, Medhin N (2021) A kernel-free double well potential support vector machine with applications. Eur J Oper Res 290(1):248–262CrossRef Gao ZM, Fang SC, Luo J, Medhin N (2021) A kernel-free double well potential support vector machine with applications. Eur J Oper Res 290(1):248–262CrossRef
19.
go back to reference Zhou JY, Tian Y, Luo J, Zhai QR (2022) Novel non-kernel quadratic surface support vector machines based on optimal margin distribution. Soft Comput 26(18):9215–9227CrossRef Zhou JY, Tian Y, Luo J, Zhai QR (2022) Novel non-kernel quadratic surface support vector machines based on optimal margin distribution. Soft Comput 26(18):9215–9227CrossRef
20.
go back to reference Chen RD, Yang ZX, Ye JY et al (2023) Kernel-free nonlinear support vector machines for multiview binary classification problems. Int J Intell Syst 2023:1–19 Chen RD, Yang ZX, Ye JY et al (2023) Kernel-free nonlinear support vector machines for multiview binary classification problems. Int J Intell Syst 2023:1–19
21.
go back to reference Ye JY, Yang ZX, Ma MP, Wang YL, Yang XM (2022) \(\epsilon \)-kernel-free soft quadratic surface support vector regression. Inf Sci 594:177–199CrossRef Ye JY, Yang ZX, Ma MP, Wang YL, Yang XM (2022) \(\epsilon \)-kernel-free soft quadratic surface support vector regression. Inf Sci 594:177–199CrossRef
22.
go back to reference Zheng JL, Tian Y, Luo J, Hong T (2023) A novel hybrid method based on kernel-free support vector regression for stock indices and price forecasting. J Oper Res Soc 74(3):690–702CrossRef Zheng JL, Tian Y, Luo J, Hong T (2023) A novel hybrid method based on kernel-free support vector regression for stock indices and price forecasting. J Oper Res Soc 74(3):690–702CrossRef
23.
go back to reference Luo J, Tian Y, Yan X (2017) Clustering via fuzzy one-class quadratic surface support vector machine. Soft Comput 21(19):5859–5865CrossRef Luo J, Tian Y, Yan X (2017) Clustering via fuzzy one-class quadratic surface support vector machine. Soft Comput 21(19):5859–5865CrossRef
24.
go back to reference Gao ZM, Wang YW, Huang M, Luo J, Tang SS (2022) A kernel-free fuzzy reduced quadratic surface \(\nu \)-support vector machine with applications. Appl Soft Comput 127:109390CrossRef Gao ZM, Wang YW, Huang M, Luo J, Tang SS (2022) A kernel-free fuzzy reduced quadratic surface \(\nu \)-support vector machine with applications. Appl Soft Comput 127:109390CrossRef
25.
go back to reference Fu SJ, Yu XT, Tian YJ (2022) Cost sensitive \(\nu \)-support vector machine with linex loss. Inf Process Manage 59(2):102809CrossRef Fu SJ, Yu XT, Tian YJ (2022) Cost sensitive \(\nu \)-support vector machine with linex loss. Inf Process Manage 59(2):102809CrossRef
26.
go back to reference Fu SJ, Tian YJ, Tang L (2023) Robust regression under the general framework of bounded loss functions. Eur J Oper Res 310(3):1325–1339CrossRef Fu SJ, Tian YJ, Tang L (2023) Robust regression under the general framework of bounded loss functions. Eur J Oper Res 310(3):1325–1339CrossRef
27.
go back to reference Wang HJ, Shao YH, Zhou SL, Zhang C, Xiu NH (2022) Support vector machine classifier via L\(_{0/1}\) soft-margin loss. IEEE Trans Pattern Anal Mach Intell 44(10):7253–7265CrossRef Wang HJ, Shao YH, Zhou SL, Zhang C, Xiu NH (2022) Support vector machine classifier via L\(_{0/1}\) soft-margin loss. IEEE Trans Pattern Anal Mach Intell 44(10):7253–7265CrossRef
28.
go back to reference Liu J, Huang LW, Shao YH, Chen WJ, Li CN (2024) A nonlinear kernel SVM classifier via L\(_{0/1}\) soft-margin loss with classification performance. J Comput Appl Math 437:115471CrossRef Liu J, Huang LW, Shao YH, Chen WJ, Li CN (2024) A nonlinear kernel SVM classifier via L\(_{0/1}\) soft-margin loss with classification performance. J Comput Appl Math 437:115471CrossRef
29.
go back to reference Mousavi A, Gao ZM, Han LS, Lim A (2022) Quadratic surface support vector machine with L\(_{1}\) norm regularization. J Ind Manage Optim 18(3):1835–1861CrossRef Mousavi A, Gao ZM, Han LS, Lim A (2022) Quadratic surface support vector machine with L\(_{1}\) norm regularization. J Ind Manage Optim 18(3):1835–1861CrossRef
30.
go back to reference Moosaei H, Mousavi A, Hladík M, Gao ZM (2023) Sparse L\(_{1}\)-norm quadratic surface support vector machine with universum data. Soft Comput 27(9):5567–5586CrossRef Moosaei H, Mousavi A, Hladík M, Gao ZM (2023) Sparse L\(_{1}\)-norm quadratic surface support vector machine with universum data. Soft Comput 27(9):5567–5586CrossRef
31.
go back to reference Shao YH, Li CN, Liu MZ, Wang Z, Deng NY (2018) Sparse L\(_{q}\)-norm least squares support vector machine with feature selection. Pattern Recogn 78:167–181CrossRef Shao YH, Li CN, Liu MZ, Wang Z, Deng NY (2018) Sparse L\(_{q}\)-norm least squares support vector machine with feature selection. Pattern Recogn 78:167–181CrossRef
32.
go back to reference Li CN, Ren PW, Shao YH, Ye YF, Guo YR (2020) Generalized elastic net L\(_{p}\)-norm nonparallel support vector machine. Eng Appl Artif Intell 88:103397CrossRef Li CN, Ren PW, Shao YH, Ye YF, Guo YR (2020) Generalized elastic net L\(_{p}\)-norm nonparallel support vector machine. Eng Appl Artif Intell 88:103397CrossRef
33.
go back to reference Li YH, Hu L, Gao WF (2023) Multi-label feature selection via robust flexible sparse regularization. Pattern Recogn 134:109074CrossRef Li YH, Hu L, Gao WF (2023) Multi-label feature selection via robust flexible sparse regularization. Pattern Recogn 134:109074CrossRef
34.
go back to reference Tian YJ, Zhang YQ (2022) A comprehensive survey on regularization strategies in machine learning. Inf Fusion 80:146–166CrossRef Tian YJ, Zhang YQ (2022) A comprehensive survey on regularization strategies in machine learning. Inf Fusion 80:146–166CrossRef
35.
go back to reference Mordukhovich BS, Nam NM (2014) An easy path to convex analysis and applications. Morgan and Claypool Publishers, KentfieldCrossRef Mordukhovich BS, Nam NM (2014) An easy path to convex analysis and applications. Morgan and Claypool Publishers, KentfieldCrossRef
Metadata
Title
Kernel-free Reduced Quadratic Surface Support Vector Machine with 0-1 Loss Function and L-norm Regularization
Authors
Mingyang Wu
Zhixia Yang
Publication date
19-08-2024
Publisher
Springer Berlin Heidelberg
Published in
Annals of Data Science
Print ISSN: 2198-5804
Electronic ISSN: 2198-5812
DOI
https://doi.org/10.1007/s40745-024-00573-w

Premium Partner