Learning from dependent observations

Abstract

In most papers establishing consistency for learning algorithms it is assumed that the observations used for training are realizations of an i.i.d. process. In this paper we go far beyond this classical framework by showing that support vector machines (SVMs) only require that the data-generating process satisfies a certain law of large numbers. We then consider the learnability of SVMs for $\alpha$-mixing (not necessarily stationary) processes for both classification and regression, where for the latter we explicitly allow unbounded noise.