Jason Weston
ABSTRACT
In this paper we study a new framework introduced by Vapnik (1998) and Vapnik (2006) that is an alternative capacity concept to the large margin approach. In the particular case of binary classification, we are given a set of labeled examples, and a collection of "non-examples" that do not belong to either class of interest. This collection, called the Universum, allows one to encode prior knowledge by representing meaningful concepts in the same domain as the problem at hand. We describe an algorithm to leverage the Universum by maximizing the number of observed contradictions, and show experimentally that this approach delivers accuracy improvements over using labeled data alone.
- Baird, H. (1990). Document image defect models. Proceedings, IAPR Workshop on Syntactic and Structural Pattern Recognition (pp. 38--46). Murray Hill, NJ.Google Scholar
- Bernardo, J. M., & Smith, A. F. M. (1994). Bayesian theory. John Wiley and Sons.Google ScholarCross Ref
- Boser, B. E., Guyon, I. M., & Vapnik, V. (1992). A training algorithm for optimal margin classifiers. Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory (pp. 144--152). Pittsburgh, PA: ACM Press. Google ScholarDigital Library
- Grandvalet, Y., Canu, S., & Boucheron, S. (1997). Noise injection: Theoretical prospects. Neural Computation, 9, 1093--1108. Google ScholarDigital Library
- Leen, T. K. (1995). From data distributions to regularization in invariant learning. Advances in Neural information processing systems 7. Cambridge MA: MIT Press. Google ScholarDigital Library
- Lewis, D. D., Yang, Y., Rose, T., & Li, F. (2004). Rcv1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5, 361--397. Google ScholarDigital Library
- Mangasarian, O. L. (1965). Linear and nonlinear separation of patterns by linear programming. Operations Research, 13, 444--452.Google ScholarDigital Library
- Niyogi, P., Girosi, F., & Poggio, T. (1998). Incorporating prior information in machine learning by creating virtual examples. Proceedings of the IEEE, 86, 2196--2209.Google ScholarCross Ref
- Schölkopf, B., Burges, C., & Vapnik, V. (1996). Incorporating invariances in support vector learning machines. Artificial Neural Networks --- ICANN'96 (pp. 47--52). Berlin: Springer Lecture Notes in Computer Science, Vol. 1112. Google ScholarDigital Library
- Shawe-Taylor, J., Bartlett, P. L., Williamson, R. C., & Anthony, M. (1998). Structural risk minimization over data-dependent hierarchies. 44, 1926--1940.Google Scholar
- Vapnik, V. (2006). Estimation of dependences based on empirical data. Berlin: Springer Verlag. 2nd edition. Google ScholarDigital Library
- Vapnik, V. N. (1998). Statistical learning theory. New York: Wiley. Google ScholarDigital Library
- Zhong, P., & Fukushima, M. (2006). A new multi-class support vector algorithm. Optimization Methods and Software, 21, 359--372.Google ScholarCross Ref
Index Terms
- Inference with the Universum
Recommendations
Self-Universum support vector machine
In this paper, for an improved twin support vector machine (TWSVM), we give it a theoretical explanation based on the concept of Universum and then name it Self-Universum support vector machine (SUSVM). For the binary classification problem, SUSVM takes ...
Twin support vector machine with Universum data
The Universum, which is defined as the sample not belonging to either class of the classification problem of interest, has been proved to be helpful in supervised learning. In this work, we designed a new Twin Support Vector Machine with Universum (...
Inverse Free Universum Twin Support Vector Machine
Learning and Intelligent OptimizationAbstractUniversum twin support vector machine (-TSVM) is an efficient method for binary classification problems . In this paper, we improve the -TSVM algorithm and propose an improved Universum twin bounded support vector machine (named as IUTBSVM) . ...
Comments