Data-Dependent Bounds for Multi-category Classification Based on Convex Losses

Algorithms for solving multi-category classification problems using output coding have become very popular in recent years. Following initial attempts with discrete coding matrices, recent work has attempted to alleviate some of their shortcomings by considering real-valued ‘coding’ matrices. We consider an approach to multi-category classification, based on minimizing a convex upper bound on the 0-1 loss. We show that this approach is closely related to output coding, and derive data-dependent bounds on the performance. These bounds can be optimized in order to obtain effective coding matrices, which guarantee small generalization error. Moreover, our results apply directly to kernel based approaches.