Support vector neurofuzzy models

The class of models considered so far are generalised linear models that construct nonlinear models by linear combinations of nonlinear basis functions such as B-splines, or Gaussian radial basis in the input or observed variables x. The power of these models is their ability to incorporate prior knowledge by structurally designing the network through choice of the type, number and position of the basis functions. This form of structural regularisation is the basis of many of the construction algorithms introduced in this book. All of these generalised linear model are examples of parametric models in which weights or parameters are identified by using linear optimisation techniques (see Chapter 3). A further class of models can be defined which do not explicitly depend on a set of parameters, the so-called nonparametric models, are applicable to sparse data sets in relatively high dimensional spaces. In nonparametric models, the parameters are not predetermined, but are determined by the training data so that the model capacity reflects complexity contained in the data. Of particular importance in this context is the class of nonparametric models whose output is a linear combination of functions of the observations x, where the linear weighting functions are determined by the characteristics of the kernel functions. The approximation of the approximant is taken over a functional including some measure of the data fit and a functional penalising certain characteristics of the approximant, this ensures a well posed solution.