Top

Journal of Classification

23-08-2024

Multi-task Support Vector Machine Classifier with Generalized Huber Loss

Authors: Qi Liu, Wenxin Zhu, Zhengming Dai, Zhihong Ma

Published in: Journal of Classification

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Compared to single-task learning (STL), multi-task learning (MTL) achieves a better generalization by exploiting domain-specific information implicit in the training signals of several related tasks. The adaptation of MTL to support vector machines (SVMs) is a rather successful example. Inspired by the recently published generalized Huber loss SVM (GHSVM) and regularized multi-task learning (RMTL), we propose a novel generalized Huber loss multi-task support vector machine including linear and non-linear cases for binary classification, named as MTL-GHSVM. The new method extends the GHSVM from single-task to multi-task learning, and the application of Huber loss to MTL-SVM is innovative to the best of our knowledge. The proposed method has two main advantages: on the one hand, compared with SVMs with hinge loss and GHSVM, our MTL-GHSVM using the differentiable generalized Huber loss has better generalization performance; on the other hand, it adopts functional iteration to find the optimal solution, and does not need to solve a quadratic programming problem (QPP), which can significantly reduce the computational cost. Numerical experiments have been conducted on fifteen real datasets, and the results demonstrate the effectiveness of the proposed multi-task classification algorithm compared with the state-of-the-art algorithms.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

http://people.ee.duke.edu/ lcarin/LandmineData.zip.

http://www.ics.uci.edu/ mlearn/MLRepository.html.

Allenby, G. M., & Rossi, P. E. (1998). Marketing models of consumer heterogeneity. Journal of Econometrics, 89(1–2), 57–78.CrossRef

Argyriou, A., Evgeniou, T., & Pontil, M. (2008). Convex multi-task feature learning. Machine learning, 73, 243–272.

Arora, N., Allenby, G. M., & Ginter, J. L. (1998). A hierarchical bayes model of primary and secondary demand. Marketing Science, 17(1), 29–44.CrossRef

Bai, L., Ong, Y.-S., He, T., & Gupta, A. (2020). Multi-task gradient descent for multi-task learning. Memetic Computing, 12, 355–369.CrossRef

Balasundaram, S., & Meena, Y. (2019). Robust support vector regression in primal with asymmetric huber loss. Neural Processing Letters, 49, 1399–1431.CrossRef

Balasundaram, S., & Prasad, S. C. (2020). Robust twin support vector regression based on huber loss function. Neural Computing and Applications, 32(15), 11285–11309.CrossRef

Baytas, I. M., Yan, M., Jain, A. K., & Zhou, J. (2016). Asynchronous multi-task learning. In 2016 IEEE 16th International Conference on Data Mining (ICDM) (pp. 11–20).

Bertsekas, D. P. (1999). Nonlinear programming: 2nd edition.

Borah, P., & Gupta, D. (2020). Functional iterative approaches for solving support vector classification problems based on generalized huber loss. Neural Computing and Applications, 32(13), 9245–9265.CrossRef

Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20, 273–297.CrossRef

Demšar, J. (2006). Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research, 7, 1–30.MathSciNet

Dheeru, D. & Taniskidou, E. K. (2017). Uci machine learning repository. http: archive. ics. uci. edu. ml.

Evgeniou, T. & Pontil, M. (2004). Regularized multi–task learning. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 109–117).

Evgeniou, T., Pontil, M., & Toubia, O. (2007). A convex optimization approach to modeling consumer heterogeneity in conjoint estimation. Marketing Science, 26(6), 805–818.CrossRef

García, S., Fernández, A., Luengo, J., & Herrera, F. (2010). Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power. Information Sciences, 180(10), 2044–2064.CrossRef

Golub, G. H. & Van Loan, C. F. (1996). Matrix computations (3rd ed.). Matrix computations.

Grant, M. & Boyd, S. (2014). Cvx: Matlab software for disciplined convex programming, version 2.1. http://cvxr.com/cvx

Gupta, D., Hazarika, B. B., & Berlin, M. (2020). Robust regularized extreme learning machine with asymmetric huber loss function. Neural Computing and Applications, 32(16), 12971–12998.CrossRef

Gupta, U., & Gupta, D. (2021). On regularization based twin support vector regression with huber loss. Neural Processing Letters, 53(1), 459–515.CrossRef

Heskes, T. (2000). Empirical bayes for learning to learn. In Proceedings of the Seventeenth International Conference on Machine Learning (pp. 367–374).

Huang, X., Shi, L., & Suykens, J. A. (2013). Support vector machine classifier with pinball loss. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(5), 984–997.CrossRef

Jebara, T. (2004). Multi-task feature and kernel selection for SVMs. In Proceedings of the twenty-first international conference on Machine learning (pp. 55).

Khan, I., Roth, P. M., Bais, A., & Bischof, H. (2013). Semi-supervised image classification with huberized laplacian support vector machines. In 2013 IEEE 9th International Conference on Emerging Technologies (ICET) (pp. 1–6).

Kong, W., Li, H., Zhang, X., & Zhao, G. (2020). A multi-context representation approach with multi-task learning for object counting. Knowledge-Based Systems, 197, 105927.CrossRef

Leiva-Murillo, J. M., Gómez-Chova, L., & Camps-Valls, G. (2012). Multitask remote sensing data classification. IEEE Transactions on Geoscience and Remote Sensing, 51(1), 151–161.CrossRef

Li, Y., Tian, X., Song, M., & Tao, D. (2015). Multi-task proximal support vector machine. Pattern Recognition, 48(10), 3249–3257.CrossRef

Liang, L. & Cherkassky, V. (2008). Connection between svm+ and multi-task learning. In 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence) (pp. 2048–2054).

Lu, L., Lin, Q., Pei, H., & Zhong, P. (2018). The als-svm based multi-task learning classifiers. Applied Intelligence, 48, 2393–2407.CrossRef

Mangasarian, O. L., & Ren, J. (1994). New improved error bounds for the linear complementarity problem. Mathematical Programming, 66(1), 241–255.MathSciNetCrossRef

Mangasarian, O. L. & Wild, E. W. (2001). Proximal support vector machine classifiers. Proceedings KDD-2001: knowledge discovery and data mining, (pp. 77–86).

Mei, B., & Xu, Y. (2019). Multi-task least squares twin support vector machine for classification. Neurocomputing, 338, 26–33.CrossRef

Ripley, B. D. (2007). Pattern recognition and neural networks. Cambridge University Press.

Samui, P., & Kim, D. (2013). Least square support vector machine and multivariate adaptive regression spline for modeling lateral load capacity of piles. Neural Computing and Applications, 23, 1123–1127.CrossRef

Shafiabady, N., Lee, L. H., Rajkumar, R., Kallimani, V. P., Akram, N. A., & Isa, D. (2016). Using unsupervised clustering approach to train the support vector machine for text classification. Neurocomputing, 211, 4–10.CrossRef

Singh, D., Khan, M. A., Bansal, A., & Bansal, N. (2015). An application of svm in character recognition with chain code. In 2015 Communication, Control and Intelligent Systems (CCIS) (pp. 167–171).

Song, Y., & Zhu, W. (2016). Multi-task support vector machine for data classification. International Journal of Signal Processing, Image Processing and Pattern Recognition, 9(7), 341–350.CrossRef

Standley, T., Zamir, A., Chen, D., Guibas, L., Malik, J., & Savarese, S. (2020). Which tasks should be learned together in multi-task learning? In International conference on machine learning (pp. 9120–9132).

Suykens, J. A., & Vandewalle, J. (1999). Least squares support vector machine classifiers. Neural Processing Letters, 9, 293–300.CrossRef

Suykens, J. A., & Vandewalle, J. (2000). Recurrent least squares support vector machines. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 47(7), 1109–1114.CrossRef

Thung, K.-H., & Wee, C.-Y. (2018). A brief review on multi-task learning. Multimedia Tools and Applications, 77(22), 29705–29725.CrossRef

Vapnik, V. (2006). Empirical inference science afterword of 2006. Springer.

Vapnik, V., & Chapelle, O. (2000). Bounds on error expectation for support vector machines. Neural Computation, 12(9), 2013–2036.CrossRef

Varghese, N. V., & Mahmoud, Q. H. (2020). A survey of multi-task deep reinforcement learning. Electronics, 9(9), 1363.CrossRef

Wang, L., Zhu, J., & Zou, H. (2008). Hybrid huberized support vector machines for microarray classification and gene selection. Bioinformatics, 24(3), 412–419.CrossRef

Wang, Y., Hu, S., & Wu, S. (2019). Object tracking based on huber loss function. The Visual Computer, 35, 1641–1654.CrossRef

Xiao, Y., Wen, J., & Liu, B. (2021). A new multi-task learning method with universum data. Applied Intelligence, 51, 3421–3434.CrossRef

Xie, F., Pang, X., & Xu, Y. (2021). Pinball loss-based multi-task twin support vector machine and its safe acceleration method. Neural Computing and Applications, 33(22), 15523–15539.CrossRef

Xu, Q., Zhou, H., Wang, Y., & Huang, J. (2009). Fuzzy support vector machine for classification of eeg signals using wavelet-based features. Medical Engineering & Physics, 31(7), 858–865.CrossRef

Xu, S., An, X., Qiao, X., & Zhu, L. (2014). Multi-task least-squares support vector machines. Multimedia Tools and Applications, 71, 699–715.CrossRef

Xu, Y., Akrotirianakis, I., & Chakraborty, A. (2016). Proximal gradient method for huberized support vector machine. Pattern Analysis and Applications, 19, 989–1005.MathSciNetCrossRef

Xue, Y. & Beauseroy, P. (2016). Multi-task learning for one-class svm with additional new features. In 2016 23rd International Conference on Pattern Recognition (ICPR) (pp. 1571–1576).

Yang, H., King, I., & Lyu, M. R. (2010). Multi-task learning for one-class classification. In The 2010 International Joint Conference on Neural Networks (IJCNN) (pp. 1–8).

Yang, W., Dong, Y., Du, Q., Qiang, Y., Wu, K., Zhao, J., Yang, X., & Zia, M. B. (2021). Integrate domain knowledge in training multi-task cascade deep learning model for benign-malignant thyroid nodule classification on ultrasound images. Engineering Applications of Artificial Intelligence, 98, 104064.CrossRef

Yang, Y., & Zou, H. (2013). An efficient algorithm for computing the hhsvm and its generalizations. Journal of Computational and Graphical Statistics, 22(2), 396–415.MathSciNetCrossRef

Zhai, S., & Jiang, T. (2015). A new sense-through-foliage target recognition method based on hybrid differential evolution and self-adaptive particle swarm optimization-based support vector machine. Neurocomputing, 149, 573–584.CrossRef

Zhang, K., Liu, Z., & Zheng, L. (2019). Short-term prediction of passenger demand in multi-zone level: Temporal convolutional neural network with multi-task learning. IEEE Transactions on Intelligent Transportation Systems, 21(4), 1480–1490.CrossRef

Zhang, X., Chen, W., Wang, B., & Chen, X. (2015). Intelligent fault diagnosis of rotating machinery using support vector machine with ant colony algorithm for synchronous feature selection and parameter optimization. Neurocomputing, 167, 260–279.CrossRef

Zhang, Y., & Yang, Q. (2018). An overview of multi-task learning. National Science Review, 5(1), 30–43.CrossRef

Zhang, Y., Yu, J., Dong, X., & Zhong, P. (2021). Multi-task support vector machine with pinball loss. Engineering Applications of Artificial Intelligence, 106, 104458.CrossRef

Zhu, W., Song, Y., & Xiao, Y. (2020). Support vector machine classifier with huberized pinball loss. Engineering Applications of Artificial Intelligence, 91, 103635.CrossRef

Title: Multi-task Support Vector Machine Classifier with Generalized Huber Loss
Authors: Qi Liu
Wenxin Zhu
Zhengming Dai
Zhihong Ma
Publication date: 23-08-2024
Publisher: Springer US
Published in: Journal of Classification
Print ISSN: 0176-4268
Electronic ISSN: 1432-1343
DOI: https://doi.org/10.1007/s00357-024-09488-w

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"