Skip to main content
Top
Published in: International Journal of Machine Learning and Cybernetics 3/2017

09-11-2015 | Original Article

Graph based semi-supervised learning via label fitting

Authors: Weiya Ren, Guohui Li

Published in: International Journal of Machine Learning and Cybernetics | Issue 3/2017

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The global smoothness and the local label fitting are two key issues for estimating the function on the graph in graph based semi-supervised learning (GSSL). The unsupervised normalized cut method can provide a more reasonable criterion for learning the global smoothness of the data than classic GSSL methods. However, the semi-supervised norm of the normalized cut, which is a NP-hard problem, has not been studied well. In this paper, a new GSSL framework is proposed by extending normalized cut to its semi-supervised norm. The NP-hard semi-supervised normalized cut problem is innovatively solved by effective algorithms. In addition, we can design more reasonable local label fitting terms than conventional GSSL methods. Other graph cut methods are also investigated to extend the proposed semi-supervised learning algorithms. Furthermore, we incorporate the nonnegative matrix factorization with the proposed learning algorithms to solve the out-of-sample problem in semi-supervised learning. Solutions obtained by the proposed algorithms are sparse, nonnegative and congruent with unit matrix. Experiment results on several real benchmark datasets indicate that the proposed algorithms achieve good results compared with state-of-art methods.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Show more products
Literature
1.
go back to reference Chawla NV, Karakoulas GI (2005) Learning from labeled and unlabeled data: an empirical study across techniques and domains. J Artif Intell Res (JAIR) 23:331–366MATH Chawla NV, Karakoulas GI (2005) Learning from labeled and unlabeled data: an empirical study across techniques and domains. J Artif Intell Res (JAIR) 23:331–366MATH
2.
go back to reference Chapelle O, Schölkopf B, Zien A (eds) (2006) Semi-supervised learning. MIT Press, Cambridge, MACrossRef Chapelle O, Schölkopf B, Zien A (eds) (2006) Semi-supervised learning. MIT Press, Cambridge, MACrossRef
3.
go back to reference Zhu X, Goldberg AB (2009) Introduction to semi-supervised learning. Synth Lect Artif Intell Mach Learn 3(1):1–130CrossRefMATH Zhu X, Goldberg AB (2009) Introduction to semi-supervised learning. Synth Lect Artif Intell Mach Learn 3(1):1–130CrossRefMATH
4.
go back to reference Nie F, Xu D, Tsang IWH, Zhang C (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. Image Process IEEE Trans 19(7):1921–1932MathSciNetCrossRef Nie F, Xu D, Tsang IWH, Zhang C (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. Image Process IEEE Trans 19(7):1921–1932MathSciNetCrossRef
5.
go back to reference Gao Q, Huang Y, Gao X, Shen W, Zhang H (2015) A novel semi-supervised learning for face recognition. Neurocomputing 152:69–76CrossRef Gao Q, Huang Y, Gao X, Shen W, Zhang H (2015) A novel semi-supervised learning for face recognition. Neurocomputing 152:69–76CrossRef
6.
go back to reference Cai D, He X, Han J (2007) Semi-supervised discriminant analysis. In: Computer Vision, 2007. ICCV 2007. IEEE 11th international conference on IEEE, pp 1–7 Cai D, He X, Han J (2007) Semi-supervised discriminant analysis. In: Computer Vision, 2007. ICCV 2007. IEEE 11th international conference on IEEE, pp 1–7
7.
go back to reference Zhou D, Bousquet O, Lal TN, Weston J, Schölkopf B (2004) Learning with local and global consistency. Adv Neural Inf Process Syst 16(16):321–328 Zhou D, Bousquet O, Lal TN, Weston J, Schölkopf B (2004) Learning with local and global consistency. Adv Neural Inf Process Syst 16(16):321–328
8.
go back to reference Wang J, Jebara T, Chang SF (2013) Semi-supervised learning using greedy max-cut. J Mach Learn Res 14(1):771–800MathSciNetMATH Wang J, Jebara T, Chang SF (2013) Semi-supervised learning using greedy max-cut. J Mach Learn Res 14(1):771–800MathSciNetMATH
9.
go back to reference Zhu X, Ghahramani Z, Lafferty J (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: ICML, vol 3, pp 912–919 Zhu X, Ghahramani Z, Lafferty J (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: ICML, vol 3, pp 912–919
10.
go back to reference Tang J, Hua XS, Qi GJ, Wang M, Mei T, Wu X (2007) Structure-sensitive manifold ranking for video concept detection. In: Proceedings of the 15th international conference on Multimedia, ACM, pp 852–861 Tang J, Hua XS, Qi GJ, Wang M, Mei T, Wu X (2007) Structure-sensitive manifold ranking for video concept detection. In: Proceedings of the 15th international conference on Multimedia, ACM, pp 852–861
11.
go back to reference Tang J, Hua XS, Qi GJ, Song Y, Wu X (2008) Video annotation based on kernel linear neighborhood propagation. Multimed IEEE Trans 10(4):620–628CrossRef Tang J, Hua XS, Qi GJ, Song Y, Wu X (2008) Video annotation based on kernel linear neighborhood propagation. Multimed IEEE Trans 10(4):620–628CrossRef
12.
go back to reference Wang M, Mei T, Yuan X, Song Y, Dai LR (2007) Video annotation by graph-based learning with neighborhood similarity. In: Proceedings of the 15th international conference on multimedia, ACM, pp 325–328 Wang M, Mei T, Yuan X, Song Y, Dai LR (2007) Video annotation by graph-based learning with neighborhood similarity. In: Proceedings of the 15th international conference on multimedia, ACM, pp 325–328
13.
go back to reference Zhao M, Chow TW, Zhang Z, Li B (2015) Automatic image annotation via compact graph based semi-supervised learning. Knowl-Based Syst 76:148–165CrossRef Zhao M, Chow TW, Zhang Z, Li B (2015) Automatic image annotation via compact graph based semi-supervised learning. Knowl-Based Syst 76:148–165CrossRef
14.
go back to reference Huang L, Wang Y, Liu X, Lang B (2013) Efficient semi-supervised annotation with proxy-based local consistency propagation. In: Multimedia and Expo (ICME), 2013 IEEE international conference on, IEEE, pp 1–6 Huang L, Wang Y, Liu X, Lang B (2013) Efficient semi-supervised annotation with proxy-based local consistency propagation. In: Multimedia and Expo (ICME), 2013 IEEE international conference on, IEEE, pp 1–6
15.
go back to reference Liu S, Yan S, Zhang T, Xu C, Liu J, Lu H (2012) Weakly supervised graph propagation towards collective image parsing. Multimed IEEE Trans 14(2):361–373CrossRef Liu S, Yan S, Zhang T, Xu C, Liu J, Lu H (2012) Weakly supervised graph propagation towards collective image parsing. Multimed IEEE Trans 14(2):361–373CrossRef
16.
go back to reference Zhao M, Chan RH, Chow TW, Tang P (2014) Compact graph based semi-supervised learning for medical diagnosis in Alzheimer’s disease. Signal Process Lett IEEE 21(10):1192–1196CrossRef Zhao M, Chan RH, Chow TW, Tang P (2014) Compact graph based semi-supervised learning for medical diagnosis in Alzheimer’s disease. Signal Process Lett IEEE 21(10):1192–1196CrossRef
17.
go back to reference Wang F, Zhang C (2008) Label propagation through linear neighborhoods. Knowl Data Eng IEEE Trans 20(1):55–67CrossRef Wang F, Zhang C (2008) Label propagation through linear neighborhoods. Knowl Data Eng IEEE Trans 20(1):55–67CrossRef
18.
go back to reference Zhuang L, Gao H, Lin Z, Ma Y, Zhang X, Yu N (2012) Non-negative low rank and sparse graph for semi-supervised learning. In: Computer vision and pattern recognition (CVPR), 2012 IEEE conference on, IEEE, pp 2328–2335 Zhuang L, Gao H, Lin Z, Ma Y, Zhang X, Yu N (2012) Non-negative low rank and sparse graph for semi-supervised learning. In: Computer vision and pattern recognition (CVPR), 2012 IEEE conference on, IEEE, pp 2328–2335
19.
go back to reference Ni B, Yan S, Kassim A (2012) Learning a propagable graph for semisupervised learning: classification and regression. Knowl Data Eng IEEE Trans 24(1):114–126CrossRef Ni B, Yan S, Kassim A (2012) Learning a propagable graph for semisupervised learning: classification and regression. Knowl Data Eng IEEE Trans 24(1):114–126CrossRef
20.
go back to reference Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434MathSciNetMATH Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434MathSciNetMATH
21.
go back to reference Shi J, Malik J (2000) Normalized cuts and image segmentation. Pattern Anal Mach Intell IEEE Trans 22(8):888–905CrossRef Shi J, Malik J (2000) Normalized cuts and image segmentation. Pattern Anal Mach Intell IEEE Trans 22(8):888–905CrossRef
22.
go back to reference Hagen L, Kahng AB (1992) New spectral methods for ratio cut partitioning and clustering. Comput Aid Des Integr Circuit Syst IEEE Trans 11(9):1074–1085CrossRef Hagen L, Kahng AB (1992) New spectral methods for ratio cut partitioning and clustering. Comput Aid Des Integr Circuit Syst IEEE Trans 11(9):1074–1085CrossRef
23.
go back to reference Sarkar S, Soundararajan P (2000) Supervised learning of large perceptual organization: graph spectral partitioning and learning automata. Pattern Anal Mach Intell IEEE Trans 22(5):504–525CrossRef Sarkar S, Soundararajan P (2000) Supervised learning of large perceptual organization: graph spectral partitioning and learning automata. Pattern Anal Mach Intell IEEE Trans 22(5):504–525CrossRef
24.
go back to reference Wu Z, Leahy R (1993) An optimal graph theoretic approach to data clustering: theory and its application to image segmentation. Pattern Anal Mach Intell IEEE Trans 15(11):1101–1113CrossRef Wu Z, Leahy R (1993) An optimal graph theoretic approach to data clustering: theory and its application to image segmentation. Pattern Anal Mach Intell IEEE Trans 15(11):1101–1113CrossRef
25.
go back to reference Ding CH, He X, Zha H, Gu M, Simon HD (2001) A min–max cut algorithm for graph partitioning and data clustering. In: Data mining, 2001. ICDM 2001, proceedings IEEE international conference on, IEEE, pp 107–114 Ding CH, He X, Zha H, Gu M, Simon HD (2001) A min–max cut algorithm for graph partitioning and data clustering. In: Data mining, 2001. ICDM 2001, proceedings IEEE international conference on, IEEE, pp 107–114
26.
go back to reference Xu L, Li W, Schuurmans D (2009) Fast normalized cut with linear constraints. In: Computer vision and pattern recognition, IEEE conference on, IEEE, pp 2866–2873 Xu L, Li W, Schuurmans D (2009) Fast normalized cut with linear constraints. In: Computer vision and pattern recognition, IEEE conference on, IEEE, pp 2866–2873
27.
go back to reference Hu H, Feng J, Yu C, Zhou J (2013) Multi-class constrained normalized cut with hard, soft, unary and pairwise priors and its applications to object segmentation. Image Process IEEE Trans 22(11):4328–4340MathSciNetCrossRef Hu H, Feng J, Yu C, Zhou J (2013) Multi-class constrained normalized cut with hard, soft, unary and pairwise priors and its applications to object segmentation. Image Process IEEE Trans 22(11):4328–4340MathSciNetCrossRef
28.
go back to reference Yang YT, Fishbain B, Hochbaum DS, Norman EB, Swanberg E (2013) The supervised normalized cut method for detecting, classifying, and identifying special nuclear materials. INFORMS J Comput 26(1):45–58CrossRef Yang YT, Fishbain B, Hochbaum DS, Norman EB, Swanberg E (2013) The supervised normalized cut method for detecting, classifying, and identifying special nuclear materials. INFORMS J Comput 26(1):45–58CrossRef
29.
go back to reference Kulis B, Basu S, Dhillon I, Mooney R (2009) Semi-supervised graph clustering: a kernel approach. Mach Learn 74(1):1–22CrossRef Kulis B, Basu S, Dhillon I, Mooney R (2009) Semi-supervised graph clustering: a kernel approach. Mach Learn 74(1):1–22CrossRef
30.
go back to reference Ren W, Li G, Tu D, Jia L (2014) Nonnegative matrix factorization with regularizations. Emerg Select Topn Circuit Syst IEEE J 4(1):153–164CrossRef Ren W, Li G, Tu D, Jia L (2014) Nonnegative matrix factorization with regularizations. Emerg Select Topn Circuit Syst IEEE J 4(1):153–164CrossRef
32.
go back to reference Ding CH, He X, Simon HD (2005) On the equivalence of nonnegative matrix factorization and spectral clustering. In: SDM, vol 5, pp 606–610 Ding CH, He X, Simon HD (2005) On the equivalence of nonnegative matrix factorization and spectral clustering. In: SDM, vol 5, pp 606–610
34.
35.
go back to reference James W, Stein C (1961) Estimation with quadratic loss. In: Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, vol 1 pp 361–379 James W, Stein C (1961) Estimation with quadratic loss. In: Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, vol 1 pp 361–379
36.
go back to reference Kjeldsen TH (2000) A contextualized historical analysis of the Kuhn–Tucker Theorem in nonlinear programming: the impact of World War II. Hist Math 27(4):331–361MathSciNetCrossRefMATH Kjeldsen TH (2000) A contextualized historical analysis of the Kuhn–Tucker Theorem in nonlinear programming: the impact of World War II. Hist Math 27(4):331–361MathSciNetCrossRefMATH
37.
go back to reference Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, vol 1, pp 57–64 Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, vol 1, pp 57–64
38.
go back to reference Niyogi X (2004) Locality preserving projections. In: Neural information processing systems, vol 16, p 153. MIT Niyogi X (2004) Locality preserving projections. In: Neural information processing systems, vol 16, p 153. MIT
39.
go back to reference Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2013) Robust recovery of subspace structures by low-rank representation. Pattern Anal Mach Intell IEEE Trans 35(1):171–184CrossRef Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2013) Robust recovery of subspace structures by low-rank representation. Pattern Anal Mach Intell IEEE Trans 35(1):171–184CrossRef
40.
go back to reference Hoyer PO (2004) Non-negative matrix factorization with sparseness constraints. J Mach Learn Res 5:1457–1469MathSciNetMATH Hoyer PO (2004) Non-negative matrix factorization with sparseness constraints. J Mach Learn Res 5:1457–1469MathSciNetMATH
41.
go back to reference Fujiwara Y, Irie G (2014) Efficient label propagation. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 784–792 Fujiwara Y, Irie G (2014) Efficient label propagation. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 784–792
42.
go back to reference Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791CrossRef Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791CrossRef
43.
go back to reference Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. In: Advances in neural information processing systems, pp 556–562 Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. In: Advances in neural information processing systems, pp 556–562
44.
go back to reference Cai D, He X, Wu X, Han J (2008) Non-negative matrix factorization on manifold. In: Data mining, 2008. ICDM’08, 8th IEEE international conference on (pp 63–72), IEEE Cai D, He X, Wu X, Han J (2008) Non-negative matrix factorization on manifold. In: Data mining, 2008. ICDM’08, 8th IEEE international conference on (pp 63–72), IEEE
45.
go back to reference Cai D, He X, Han J, Huang TS (2011) Graph regularized nonnegative matrix factorization for data representation. Pattern Anal Mach Intell IEEE Trans 33(8):1548–1560CrossRef Cai D, He X, Han J, Huang TS (2011) Graph regularized nonnegative matrix factorization for data representation. Pattern Anal Mach Intell IEEE Trans 33(8):1548–1560CrossRef
47.
go back to reference Wang XZ, Xing HJ, Li Y, Hua Q, Dong CR, Pedrycz W (2015) A study on relationship between generalization abilities and fuzziness of base classifiers in ensemble learning. IEEE Trans Fuzzy Syst 23(5):1638–1654CrossRef Wang XZ, Xing HJ, Li Y, Hua Q, Dong CR, Pedrycz W (2015) A study on relationship between generalization abilities and fuzziness of base classifiers in ensemble learning. IEEE Trans Fuzzy Syst 23(5):1638–1654CrossRef
48.
go back to reference Wang XZ, Aamir Raza Ashfaq R, Fu AM (2015) Fuzziness based sample categorization for classifier performance improvement. J Intell Fuzzy Syst 29(3):1185–1196MathSciNetCrossRef Wang XZ, Aamir Raza Ashfaq R, Fu AM (2015) Fuzziness based sample categorization for classifier performance improvement. J Intell Fuzzy Syst 29(3):1185–1196MathSciNetCrossRef
49.
go back to reference Wang XZ, Dong LC, Yan JH (2012) Maximum ambiguity-based sample selection in fuzzy decision tree induction. Knowl Data Eng IEEE Trans 24(8):1491–1505CrossRef Wang XZ, Dong LC, Yan JH (2012) Maximum ambiguity-based sample selection in fuzzy decision tree induction. Knowl Data Eng IEEE Trans 24(8):1491–1505CrossRef
51.
go back to reference Hartley RVL (1949) Transmission of information. Bell Syst Tech J 7:535–563CrossRef Hartley RVL (1949) Transmission of information. Bell Syst Tech J 7:535–563CrossRef
Metadata
Title
Graph based semi-supervised learning via label fitting
Authors
Weiya Ren
Guohui Li
Publication date
09-11-2015
Publisher
Springer Berlin Heidelberg
Published in
International Journal of Machine Learning and Cybernetics / Issue 3/2017
Print ISSN: 1868-8071
Electronic ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-015-0458-y

Other articles of this Issue 3/2017

International Journal of Machine Learning and Cybernetics 3/2017 Go to the issue