Skip to main content
Erschienen in: Neural Processing Letters 1/2015

01.08.2015

Scalable Semi-Supervised Classification via Neumann Series

verfasst von: Chen Gong, Keren Fu, Lei Zhou, Jie Yang, Xiangjian He

Erschienen in: Neural Processing Letters | Ausgabe 1/2015

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Traditional graph-based semi-supervised learning (GBSSL) algorithms usually scale badly due to the expensive computational burden. The main bottleneck is that they need to compute the inversion of a huge matrix. In order to alleviate this problem, this paper proposes Neumann series approximation (NSA) to explicitly approximate the inversion process required by conventional GBSSL methodologies, which makes them computationally tractable for relatively large datasets. It is proved that the deviation between the approximation and direct inversion is bounded. Using real-world datasets related to handwritten digit recognition, speech recognition and text classification, the experimental results reveal that NSA accelerates the speed significantly without decreasing too much precision. We also empirically show that NSA outperforms other scalable approaches such as Nyström method, Takahashi equation, Lanczos process based SVD and AnchorGraph regularization, in terms of both efficiency and accuracy.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Belkin M et al (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434MathSciNetMATH Belkin M et al (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434MathSciNetMATH
2.
Zurück zum Zitat Campbell Y, Davis T (1995) Computing the sparse inverse subset: an inverse multifrontal approach. Technical report TR-95-021, University of Florida, Gainesville, FL Campbell Y, Davis T (1995) Computing the sparse inverse subset: an inverse multifrontal approach. Technical report TR-95-021, University of Florida, Gainesville, FL
3.
Zurück zum Zitat Delalleau O, Bengio Y et al (2005) Efficient non-parametric function induction in semi-supervised learning. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, p 12–19 Delalleau O, Bengio Y et al (2005) Efficient non-parametric function induction in semi-supervised learning. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, p 12–19
4.
Zurück zum Zitat Fergus R, Kandola J et al (2009) Semi-supervised learning in gigantic image collections. In: Proceedings of the advances in neural information processing systems, Vancouver Fergus R, Kandola J et al (2009) Semi-supervised learning in gigantic image collections. In: Proceedings of the advances in neural information processing systems, Vancouver
5.
Zurück zum Zitat Fowlkes C et al (2004) Spectral grouping using the nyström method. Pattern Anal Mach Intell IEEE Trans 26(2):214–225CrossRefMATH Fowlkes C et al (2004) Spectral grouping using the nyström method. Pattern Anal Mach Intell IEEE Trans 26(2):214–225CrossRefMATH
6.
Zurück zum Zitat Garcke J, Griebel M (2005) Semi-supervised learning with sparse grids. In: Proceedings of the international conference on machine learning, Bonn Garcke J, Griebel M (2005) Semi-supervised learning with sparse grids. In: Proceedings of the international conference on machine learning, Bonn
7.
Zurück zum Zitat Golub G, Loan V (1996) Matrix computations. Johns Hopkins University Press, Baltimore Golub G, Loan V (1996) Matrix computations. Johns Hopkins University Press, Baltimore
8.
Zurück zum Zitat Kawano S et al (2012) Semi-supervised logistic discrimination via graph-based regularization. Neural Process Lett 36(3):203–216CrossRef Kawano S et al (2012) Semi-supervised logistic discrimination via graph-based regularization. Neural Process Lett 36(3):203–216CrossRef
9.
Zurück zum Zitat Larsen R (1998) Lanczos bidiagonalization with partial reorthogonalization. DAIMI Report Series 27(537):1–101 Larsen R (1998) Lanczos bidiagonalization with partial reorthogonalization. DAIMI Report Series 27(537):1–101
10.
Zurück zum Zitat Liu W, He J et al (2010) Large graph construction for scalable semi-supervised learning. In: Proceedings of the international conference on machine learning, Haifa, pp 679–686 Liu W, He J et al (2010) Large graph construction for scalable semi-supervised learning. In: Proceedings of the international conference on machine learning, Haifa, pp 679–686
11.
Zurück zum Zitat Salton G, McGill M (1986) Introduction to modern information retrieval, McGraw-Hill, New York Salton G, McGill M (1986) Introduction to modern information retrieval, McGraw-Hill, New York
12.
Zurück zum Zitat Shang F et al (2012) Integrating spectral kernel learning and constraints in semi-supervised classification. Neural Process Lett 36(2):101–115CrossRef Shang F et al (2012) Integrating spectral kernel learning and constraints in semi-supervised classification. Neural Process Lett 36(2):101–115CrossRef
13.
Zurück zum Zitat Sinha K, Belkin M (2009) Semi-supervised learning using sparse eigenfunction bases. Adv Neural Info Process Sys 22:1687–1695 Sinha K, Belkin M (2009) Semi-supervised learning using sparse eigenfunction bases. Adv Neural Info Process Sys 22:1687–1695
14.
15.
Zurück zum Zitat Subramanya A, Bilmes J (2011) Semi-supervised learning with measure propagation. J Mach Learn Res 12:3311–3370MathSciNet Subramanya A, Bilmes J (2011) Semi-supervised learning with measure propagation. J Mach Learn Res 12:3311–3370MathSciNet
16.
Zurück zum Zitat Talwalkar A, Kumar S et al (2008) Large-scale manifold learning. In: Proceedings of the IEEE conference on the computer vision and pattern recognition (CVPR) Talwalkar A, Kumar S et al (2008) Large-scale manifold learning. In: Proceedings of the IEEE conference on the computer vision and pattern recognition (CVPR)
17.
Zurück zum Zitat Tsang I, Kwok J (2007) Large-scale sparsified manifold regularization. In: Proceedings of the advances in neural information processing systems, Vancouver Tsang I, Kwok J (2007) Large-scale sparsified manifold regularization. In: Proceedings of the advances in neural information processing systems, Vancouver
18.
Zurück zum Zitat Valls G et al (2007) Semi-supervised graph-based hyperspectral image classification. Geosci Remote Sens IEEE Trans 45(10):3044–3054CrossRef Valls G et al (2007) Semi-supervised graph-based hyperspectral image classification. Geosci Remote Sens IEEE Trans 45(10):3044–3054CrossRef
19.
Zurück zum Zitat Wang J et al (2009) Linear neighborhood propagation and its applications. Pattern Anal Mach Intell IEEE Trans 31(9):1600–1615CrossRef Wang J et al (2009) Linear neighborhood propagation and its applications. Pattern Anal Mach Intell IEEE Trans 31(9):1600–1615CrossRef
20.
Zurück zum Zitat Zhang K, Kwok J, et al (2009) Prototype vector machine for large scale semi-supervised learning. In: Proceedings of the international conference on machine learning, pp 1233–1240 Zhang K, Kwok J, et al (2009) Prototype vector machine for large scale semi-supervised learning. In: Proceedings of the international conference on machine learning, pp 1233–1240
21.
Zurück zum Zitat Zhou D, Bousquet O (2003) Learning with local and global consistency. In: Proceedings of the advances in neural information processing systems, Vancouver, pp 321–328 Zhou D, Bousquet O (2003) Learning with local and global consistency. In: Proceedings of the advances in neural information processing systems, Vancouver, pp 321–328
22.
Zurück zum Zitat Zhu X, Ghahramani Z et al (2003) Semi-supervised learning using Gaussian fields and harmonic functions. In: Proceedings of the international conference on machine learning, Washington, DC, pp 912–919 Zhu X, Ghahramani Z et al (2003) Semi-supervised learning using Gaussian fields and harmonic functions. In: Proceedings of the international conference on machine learning, Washington, DC, pp 912–919
23.
Zurück zum Zitat Zhu X, Goldberg B (2009) Introduction to semi-supervised learning. Morgan & Claypool Publishers, San RafaelMATH Zhu X, Goldberg B (2009) Introduction to semi-supervised learning. Morgan & Claypool Publishers, San RafaelMATH
24.
Zurück zum Zitat Zhu X (2005) Lafferty: Harmonic mixtures: combining mixture models and graph-based methods for inductive and scalable semi-supervised learning. In: Proceedings of the international conference on machine learning, Bonn, pp 1052–1059 Zhu X (2005) Lafferty: Harmonic mixtures: combining mixture models and graph-based methods for inductive and scalable semi-supervised learning. In: Proceedings of the international conference on machine learning, Bonn, pp 1052–1059
Metadaten
Titel
Scalable Semi-Supervised Classification via Neumann Series
verfasst von
Chen Gong
Keren Fu
Lei Zhou
Jie Yang
Xiangjian He
Publikationsdatum
01.08.2015
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 1/2015
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-014-9351-z

Weitere Artikel der Ausgabe 1/2015

Neural Processing Letters 1/2015 Zur Ausgabe

Neuer Inhalt