Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Data Analytics and Optimisation for Assessing a Ride Sharing System Read chapter Read first chapter

2015 | OriginalPaper | Chapter

Implicitly Constrained Semi-supervised Least Squares Classification

Authors : Jesse H. Krijthe, Marco Loog

Published in: Advances in Intelligent Data Analysis XIV

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

We introduce a novel semi-supervised version of the least squares classifier. This implicitly constrained least squares (ICLS) classifier minimizes the squared loss on the labeled data among the set of parameters implied by all possible labelings of the unlabeled data. Unlike other discriminative semi-supervised methods, our approach does not introduce explicit additional assumptions into the objective function, but leverages implicit assumptions already present in the choice of the supervised least squares classifier. We show this approach can be formulated as a quadratic programming problem and its solution can be found using a simple gradient descent procedure. We prove that, in a certain way, our method never leads to performance worse than the supervised classifier. Experimental results corroborate this theoretical result in the multidimensional case on benchmark datasets, also in terms of the error rate.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Probabilistic Active Learning in Datastreams
next chapter Diagonal Co-clustering Algorithm for Document-Word Partitioning
Literature
1.
2.
Bennett, K.P., Demiriz, A.: Semi-supervised support vector machines. Adv. Neural Inf. Process. Syst. 11, 368–374 (1998)
3.
Bottou, L.: Large-scale machine learning with stochastic gradient descent. In: Lechevallier, Y., Saporta, G. (eds.) COMPSTAT 2010, pp. 177–186. Springer, Heidelberg (2010)CrossRef
4.
Byrd, R.H., Lu, P., Nocedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16(5), 1190–1208 (1995)MathSciNetCrossRefMATH
5.
Chapelle, O., Schölkopf, B., Zien, A.: Semi-Supervised Learning. MIT press, Cambridge (2006)CrossRef
6.
Cozman, F., Cohen, I.: Risks of semi-supervised learning. In: Chapelle, O., Schölkopf, B., Zien, A. (eds.) Semi-Supervised Learning, Chap. 4, pp. 56–72. MIT press (2006)
7.
Cozman, F.G., Cohen, I., Cirelo, M.C.: Semi-supervised learning of mixture models. In: Proceedings of the Twentieth International Conference on Machine Learning (2003)
8.
Hastie, T., Tibshirani, R., Friedman, J.H.: The Elements of Statistical Learning. Spinger, New York (2001)CrossRefMATH
9.
Krijthe, J.H., Loog, M.: Implicitly constrained semi-supervised linear discriminant analysis. In: International Conference on Pattern Recognition, pp. 3762–3767, Stockholm (2014)
10.
Li, Y.F., Zhou, Z.H.: Towards making unlabeled data never hurt. IEEE Trans. Pattern Anal. Mach. Intell. 37(1), 175–188 (2015)CrossRef
11.
Loog, M., Jensen, A.: Semi-supervised nearest mean classification through a constrained log-likelihood. IEEE Trans. Neural Networks Learn. Syst. 26(5), 995–1006 (2015)MathSciNetCrossRef
12.
Loog, M.: Semi-supervised linear discriminant analysis through moment-constraint parameter estimation. Pattern Recognit. Lett. 37, 24–31 (2014)CrossRef
13.
McLachlan, G.J.: Iterative reclassification procedure for constructing an asymptotically optimal rule of allocation in discriminant analysis. J. Am. Stat. Assoc. 70(350), 365–369 (1975)MathSciNetCrossRefMATH
14.
Nigam, K., McCallum, A.K., Thrun, S., Mitchell, T.: Text classification from labeled and unlabeled documents using EM. Mach. Learn. 34, 1–34 (2000)MATH
15.
Opper, M., Kinzel, W.: Statistical mechanics of generalization. In: Domany, E., Hemmen, J.L., Schulten, K. (eds.) Models of Neural Networks III, pp. 151–209. Springer, New York (1996)CrossRef
16.
Poggio, T., Smale, S.: The mathematics of learning: dealing with data. Not. AMS 50, 537–544 (2003)MathSciNetMATH
17.
Raudys, S., Duin, R.P.: Expected classification error of the fisher linear classifier with pseudo-inverse covariance matrix. Pattern Recogn. Lett. 19(5–6), 385–392 (1998)CrossRefMATH
18.
Rifkin, R., Yeo, G., Poggio, T.: Regularized least-squares classification. Nato Sci. Ser. Sub Ser. III Comput. Syst. Sci. 190, 131–154 (2003)
19.
Seeger, M.: Learning with labeled and unlabeled data. Technical report (2001)
20.
Singh, A., Nowak, R.D., Zhu, X.: Unlabeled data: now it helps, now it doesnt. In: Advances in Neural Information Processing Systems, pp. 1513–1520 (2008)
21.
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soci. Ser. B 58(1), 267–288 (1996)MathSciNetMATH
22.
Widrow, B., Hoff, M.E.: Adaptive switching circuits. IRE WESCON Convention Rec. 4, 96–104 (1960)
23.
Zhu, X., Goldberg, A.B.: Introduction to Semi-Supervised Learning, vol. 3. Morgan & Claypool, San Rafael (2009) MATH
Metadata
Title
Implicitly Constrained Semi-supervised Least Squares Classification
Authors
Jesse H. Krijthe
Marco Loog
Copyright Year
2015
Book
Advances in Intelligent Data Analysis XIV

Print ISBN: 978-3-319-24464-8

Electronic ISBN: 978-3-319-24465-5

Copyright Year: 2015

DOI
https://doi.org/10.1007/978-3-319-24465-5_14

Premium Partner

    Image Credits