Skip to main content
Top

2016 | OriginalPaper | Chapter

A Theoretical Analysis of Semi-supervised Learning

Authors : Takashi Fujii, Hidetaka Ito, Seiji Miyoshi

Published in: Neural Information Processing

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

We analyze the dynamical behaviors of semi-supervised learning in the framework of on-line learning by using the statistical-mechanical method. A student uses several correlated input vectors in each update. The student is given a desired output for only one input vector out of these correlated input vectors. In this model, we derive simultaneous differential equations with deterministic forms that describe the dynamical behaviors of order parameters using the self-averaging property in the thermodynamic limit. We treat the Hebbian and Perceptron learning rules. As a result, it is shown that using unlabeled data is effective in the early stages for both of the two learning rules. In addition, we show that the two learning rules have qualitatively different dynamical behaviors. Furthermore, we propose a new algorithm that improves the generalization performance by switching the number of input vectors used in an update as the time step proceeds.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Saad, D. (ed.): On-Line Learning in Neural Networks. Cambridge University Press, Cambridge (1998)MATH Saad, D. (ed.): On-Line Learning in Neural Networks. Cambridge University Press, Cambridge (1998)MATH
2.
go back to reference Urakami, M., Miyoshi, S., Okada, O.: Statistical mechanics of on-line learning when a moving teacher goes around an unlearnable true teacher. J. Phys. Soc. Jpn. 76, 044003 (2007)CrossRef Urakami, M., Miyoshi, S., Okada, O.: Statistical mechanics of on-line learning when a moving teacher goes around an unlearnable true teacher. J. Phys. Soc. Jpn. 76, 044003 (2007)CrossRef
3.
go back to reference Hirama, T., Hukushima, K.: On-line learning of an unlearnable true teacher through mobile ensemble teachers. J. Phys. Soc. Jpn. 77, 094801 (2008)CrossRef Hirama, T., Hukushima, K.: On-line learning of an unlearnable true teacher through mobile ensemble teachers. J. Phys. Soc. Jpn. 77, 094801 (2008)CrossRef
4.
go back to reference Chapelle, O., Scholkopf, B., Zien, A.: Semi-Supervised Learning. MIT Press, Cambridge (2006)CrossRef Chapelle, O., Scholkopf, B., Zien, A.: Semi-Supervised Learning. MIT Press, Cambridge (2006)CrossRef
5.
go back to reference Zhu, X., Goldberg, A.B.: Introduction to Semi-Supervised Learning: Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, city of publication (2009) Zhu, X., Goldberg, A.B.: Introduction to Semi-Supervised Learning: Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, city of publication (2009)
6.
go back to reference Engel, A., Broeck, C.: Statistical Mechanics of Learning. Cambridge University Press, Cambridge (2001)CrossRefMATH Engel, A., Broeck, C.: Statistical Mechanics of Learning. Cambridge University Press, Cambridge (2001)CrossRefMATH
7.
go back to reference Nakao, K., Narukawa, Y., Miyoshi, S.: Statistical mechanics of on-line learning using correlated examples. IEICE Trans. Inf. Syst. E94–D(10), 1941–1944 (2011)CrossRef Nakao, K., Narukawa, Y., Miyoshi, S.: Statistical mechanics of on-line learning using correlated examples. IEICE Trans. Inf. Syst. E94–D(10), 1941–1944 (2011)CrossRef
8.
go back to reference Nishimori, H.: Statistical Physics of Spin Glasses and Information Processing: An Introduction. Oxford University Press, Oxford (2001)CrossRefMATH Nishimori, H.: Statistical Physics of Spin Glasses and Information Processing: An Introduction. Oxford University Press, Oxford (2001)CrossRefMATH
Metadata
Title
A Theoretical Analysis of Semi-supervised Learning
Authors
Takashi Fujii
Hidetaka Ito
Seiji Miyoshi
Copyright Year
2016
DOI
https://doi.org/10.1007/978-3-319-46672-9_4

Premium Partner