Guang Dai
ABSTRACT
Existing semi-supervised learning methods are mostly based on either the cluster assumption or the manifold assumption. In this paper, we propose an integrated regularization framework for semi-supervised kernel machines by incorporating both the cluster assumption and the manifold assumption. Moreover, it supports kernel learning in the form of kernel selection. The optimization problem involves joint optimization over all the labeled and unlabeled data points, a convex set of basic kernels, and a discrete space of unknown labels for the unlabeled data. When the manifold assumption is incorporated, graph Laplacian kernels are used as the basic kernels for learning an optimal convex combination of graph Laplacian kernels. Comparison with related methods on the USPS data set shows very promising results.
- Argyriou, A., Herbster, M., & Pontil, M. (2006). Combining graph Laplacians for semi--supervised learning. Advances in Neural Information Processing Systems 18. MIT Press, Cambridge, MA, USA.Google Scholar
- Argyriou, A., Micchelli, C. A., & Pontil, M. (2005). Learning convex combinations of continuously parameterized basic kernels. Proceedings of the Eighteenth Annual Conference on Learning Theory (pp. 338--352). Google ScholarDigital Library
- Belkin, M., & Niyogi, P. (2004). Semi-supervised learning on Riemannian manifolds. Machine Learning, 56, 209--239. Google ScholarDigital Library
- Belkin, M., Niyogi, P., & Sindhwani, V. (2006). Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 7, 2399--2434. Google ScholarDigital Library
- Chapelle, O., Chi, M., & Zien, A. (2006). A continuation method for semi-supervised SVMs. Proceedings of the Twenty-Third International Conference on Machine Learning. Google ScholarDigital Library
- Chapelle, O., Vapnik, V., Bousquet, O., & Mukherjee, S. (2002). Choosing multiple parameters for support vector machines. Machine Learning, 46, 131--159. Google ScholarDigital Library
- Chapelle, O., & Zien, A. (2005). Semi-supervised classification by low density separation. Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics (pp. 57--64).Google Scholar
- Joachims, T. (1999). Transductive inference for text classification using support vector machines. Proceedings of the Sixteenth International Conference on Machine Learning (pp. 41--48) Google ScholarDigital Library
- Lanckriet, G. R. G., Cristianini, N., Ghaoui, L. E., Bartlett, P. L., & Jordan, M. I. (2004). Image categorization by learning and reasoning with regions. Journal of Machine Learning Research, 5, 27--72. Google ScholarDigital Library
- Micchelli, C., & Pontil, M. (2005). Learning the kernel function via regularization. Journal of Machine Learning Research, 6, 1099--1125. Google ScholarDigital Library
- Rockafellar, R. (1996). Convex analysis. Princeton Landmarks in Mathematics and Physics. Princeton University Press; Reprint edition.Google Scholar
- Sindhwani, V., Keerthi, S., & Chapelle, O. (2006). Deterministic annealing for semi-supervised kernel machines. Proceedings of the Twenty-Third International Conference on Machine Learning. Google ScholarDigital Library
- Sindhwani, V., Niyogi, P., & Belkin, M. (2005). Beyond the point cloud: from transductive to semi-supervised learning. Proceedings of the Twenty-Second International Conference on Machine Learning. Google ScholarDigital Library
- Zhang, T., & Ando, R. (2005). Graph based semi-supervised learning and spectral kernel design (Technical Report RC23713). IBM T.J. Watson Research Center.Google Scholar
- Zhou, D., Bousquet, O., Lal, T., Weston, J., & Schöölkopf, B. (2004). Learning with local and global consistency. Advances in Neural Information Processing Systems 16 (pp. 321--328). MIT Press, Cambridge, MA, USA.Google Scholar
- Zhou, D., Huang, J., & Scholkopf, B. (2005). Learning from labeled and unlabeled data on a directed graph. Proceedings of the Twenty-Second International Conference on Machine Learning (pp. 1041--1048). Google ScholarDigital Library
- Zhu, X., Ghahramani, Z., & Lafferty, J. (2003). Semisupervised learning using Gaussian fields and harmonic functions. Proceedings of the Twentieth International Conference on Machine Learning (pp. 912--919).Google Scholar
- Zhu, X., Kandola, J., Ghahramani, Z., & Lafferty, J. (2005). Nonparametric transforms of graph kernels for semi-supervised learning. Advances in Neural Information Processing Systems 17.Google Scholar
- Kernel selection forl semi-supervised kernel machines
Recommendations
Semi-supervised Kernel Based Progressive SVM
ICGEC '10: Proceedings of the 2010 Fourth International Conference on Genetic and Evolutionary ComputingMost existing semi-supervised methods implemented either the cluster assumption or the manifold assumption. The performance will degrade if the assumption was not proper for the data. A method was proposed by combining both the cluster assumption and ...
Data-dependent kernel sparsity preserving projection and its application for semi-supervised classification
Dimensionality reduction methods (DR) have been commonly used as a principled way to understand the high-dimensional data. In this paper, a novel semi-supervised nonlinear method called semi-supervised data-dependent kernel sparsity preserving ...
Semi-Supervised Kernel Regression
ICDM '06: Proceedings of the Sixth International Conference on Data MiningInsufficiency of training data is a major obstacle in machine learning and data mining applications. Many different semi-supervised learning algorithms have been proposed to tackle this difficulty by leveraging a large amount of unlabeled data. However, ...
Comments