Top

International Journal of Machine Learning and Cybernetics

Published in:

18-12-2018 | Original Article

A nonlinear kernel support matrix machine for matrix learning

Author: Yunfei Ye

Published in: International Journal of Machine Learning and Cybernetics | Issue 10/2019

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

In many problems of supervised tensor learning, real world data such as face images or MRI scans are naturally represented as matrices, which are also called as second order tensors. Most existing classifiers based on tensor representation, such as support tensor machine and kernelized support tensor machine need to solve iteratively which occupy much time and may suffer from local minima. In this paper, we present a kernel support matrix machine which performs a matrix-form inner product with maximum margin classifier. Specifically, the matrix inner product is introduced to leverage the inherent structural information within matrix data. Further, matrix kernel functions are applied to detect the nonlinear relationships. We analyze a unifying optimization problem for which we propose an asymptotically convergent algorithm. Theoretical analysis for the generalization bounds is derived based on Rademacher complexity with respect to a probability distribution. We demonstrate the merits of the proposed method by exhaustive experiments on both simulation study and a number of real-word datasets from a variety of application domains.

previous article Network embedding based on deep extreme learning machine

next article Prediction model of hot metal temperature for blast furnace based on improved multi-layer extreme learning machine

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

ATZelectronics worldwide

ATZlectronics worldwide is up-to-speed on new trends and developments in automotive electronics on a scientific level with a high depth of information.

Order your 30-days-trial for free and without any commitment.

inform now

ATZelektronik

Die Fachzeitschrift ATZelektronik bietet für Entwickler und Entscheider in der Automobil- und Zulieferindustrie qualitativ hochwertige und fundierte Informationen aus dem gesamten Spektrum der Pkw- und Nutzfahrzeug-Elektronik.

Lassen Sie sich jetzt unverbindlich 2 kostenlose Ausgabe zusenden.

inform now

Available only for authorised users

http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html.

https://www.sheffield.ac.uk/eee/research/iel/research/face.

http://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php.

http://www.cs.toronto.edu/~roweis/data.html.

Cai D, He X, Han J (2006) Learning with tensor representation. Technical report, Computer Science Department, UIUC, UIUCDCS-R-2006-2716. http://hdl.handle.net/2142/11195

Chen Y, Wang K, Zhong P (2016) One-class support tensor machine. Knowl Based Syst 96:14–28CrossRef

Chu C, Kim SK, Lin YA, Yu Y, Bradski G, Ng AY, Olukotun K (2007) Map-reduce for machine learning on multicore. In: Proceedings of the 2006 conference on advances in neural information processing systems, vol 19. MIT Press, Cambridge, pp 281–288

Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297MATH

Erfani SM, Baktashmotlagh M, Rajasegarar S, Nguyen V, Leckie C, Bailey J, Ramamohanarao K (2016) R1stm: One-class support tensor machine with randomised kernel. In: Proceedings of the 2016 SIAM international conference on data mining. SIAM, pp 198–206. https://doi.org/10.1137/1.9781611974348.23

Evgeniou T, Micchelli CA, Pontil M (2005) Learning multiple tasks with kernel methods. J Mach Learn Res 6:615–637MathSciNetMATH

Gao X, Fan L, Xu H (2018) Multiple rank multi-linear kernel support vector machine for matrix data classification. Int J Mach Learn Cybernet 9(2):251–261CrossRef

Hao Z, He L, Chen B, Yang X (2013) A linear support higher-order tensor machine for classification. IEEE Trans Image Process 22(7):2911–2920CrossRef

He L, Kong X, Yu PS, Yang X, Ragin AB, Hao Z (2014) Dusk: a dual structure-preserving kernel for supervised tensor learning with applications to neuroimages. In: Proceedings of the 2014 SIAM international conference on data mining. SIAM, pp 127–135. https://doi.org/10.1137/1.9781611973440.15

10.

He L, Lu CT, Ma G, Wang S, Shen L, Philip SY, Ragin AB (2017) Kernelized support tensor machines. In: Proceedings of the 34th international conference on machine learning, vol 70, pp 1442–1451

11.

Horn RA (1990) The hadamard product. In: Proceedings of symposia in applied mathematics, vol 40. American Mathematical Society, Providence, pp 87–169

12.

Joachims T (1999) Transductive inference for text classification using support vector machines. In: Proceedings of the 16th international conference on machine learning, vol 99, pp 200–209

13.

Kadri H, Duflos E, Preux P, Rakotomamonjy A, Audiffren J (2016) Operator-valued kernels for learning from functional response data. J Mach Learn Res 17(1):613–666MathSciNetMATH

14.

Khemchandani R, Chandra S et al (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910CrossRef

15.

Le Gall F (2014) Powers of tensors and fast matrix multiplication. In: Proceedings of the 39th international symposium on symbolic and algebraic computation. ACM, pp 296–303. https://doi.org/10.1145/2608628.2608664

16.

Luo L, Xie Y, Zhang Z, Li WJ (2015) Support matrix machines. In: Proceedings of the 32nd international conference on machine learning, vol 37, pp 938–947

17.

Micchelli CA, Pontil MA (2005) On learning vector-valued functions. Neural Comput 17(1):177–204MathSciNetCrossRef

18.

Nene SA, Nayar SK, Murase H (1996) Columbia object image library (coil-20). Technical report, Columbia University, CUCS-005-96. http://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php

19.

Platt JC (1999) Fast training of support vector machines using sequential minimal optimization. In: Schölkopf B, Burges CJC, Smola AJ (eds) Advances in kernel methods—support vector learning. MIT Press, Cambridge, pp 185–208

20.

Reisert M, Burkhardt H (2007) Learning equivariant functions with matrix valued kernels. J Mach Learn Res 8:385–408MathSciNetMATH

21.

Samaria FS, Harter AC (1994) Parameterisation of a stochastic model for human face identification. In: Proceedings of 1994 IEEE workshop on applications of computer vision. IEEE, pp 138–142. https://doi.org/10.1109/ACV.1994.341300

22.

Schölkopf B, Sung KK, Burges CJ, Girosi F, Niyogi P, Poggio T, Vapnik V (1997) Comparing support vector machines with gaussian kernels to radial basis function classifiers. IEEE Trans Signal Process 45(11):2758–2765CrossRef

23.

Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245CrossRef

24.

Schölkopf B, Platt JC, Shawe-Taylor J, Smola AJ, Williamson RC (2001) Estimating the support of a high-dimensional distribution. Neural Comput 13(7):1443–1471CrossRef

25.

Shalev-Shwartz S, Ben-David S (2014) Understanding machine learning: from theory to algorithms. Cambridge University Press, New YorkCrossRef

26.

Signoretto M, De Lathauwer L, Suykens JA (2011) A kernel-based framework to tensorial data analysis. Neural Netw 24(8):861–874CrossRef

27.

Stitson MO, Gammerman A, Vapnik V, Vovk V, Watkins C, Weston J (1997) Support vector regression with anova decomposition kernels. In: Schölkopf B, Burges CJC, Smola AJ (eds) Advances in kernel methods—support vector learning. MIT Press, Cambridge, pp 285–292

28.

Suykens JA, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300CrossRef

29.

Tao D, Li X, Hu W, Maybank S, Wu X (2007) Supervised tensor learning. Knowl Inf Syst 13(1):1–42CrossRef

30.

Vapnik V (1995) The nature of statistical learning theory. Springer, New YorkCrossRef

31.

Weston J, Gammerman A, Stitson M, Vapnik V, Vovk V, Watkins C (1997) Density estimation using support vector machines. In: Schölkopf B, Burges CJC, Smola AJ (eds) Advances in kernel methods—support vector learning. MIT Press, Cambridge, pp 293–306

32.

Wong WK, Lai Z, Xu Y, Wen J, Ho CP (2015) Joint tensor feature analysis for visual object recognition. IEEE Trans Cybern 45(11):2425–2436CrossRef

33.

Yang Y, Liu X (1999) A re-examination of text categorization methods. In: Proceedings of the 22nd annual international ACM SIGIR conference on research and development in information retrieval. ACM, pp 42–49. https://doi.org/10.1145/312624.312647

34.

Ye Y (2017) The Matrix Hilbert space and its application to matrix learning. arXiv:1706.08110

35.

Zhou H, Li L (2014) Regularized matrix regression. J R Stat Soc B 76(2):463–483MathSciNetCrossRef

Title: A nonlinear kernel support matrix machine for matrix learning
Author: Yunfei Ye
Publication date: 18-12-2018
Publisher: Springer Berlin Heidelberg
Published in: International Journal of Machine Learning and Cybernetics / Issue 10/2019
Print ISSN: 1868-8071
Electronic ISSN: 1868-808X
DOI: https://doi.org/10.1007/s13042-018-0896-4

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

ATZelectronics worldwide

ATZelektronik

Other articles of this Issue 10/2019

Simultaneous clustering and feature selection via nonparametric Pitman–Yor process mixture models

Network embedding based on deep extreme learning machine

Social web video clustering based on multi-view clustering via nonnegative matrix factorization

Adaptive sliding-mode observer for second order discrete-time MIMO nonlinear systems based on recurrent neural-networks

Person detector for different overhead views using machine learning

Prediction model of hot metal temperature for blast furnace based on improved multi-layer extreme learning machine