Regression with individual weigths to prevent overfitting

The Moving Least Squares (MLS) method has been established in the context of regression analysis for the mathematical description of a non-linear relationship. Compared to the Ordinary Least Squares (LS) method, it takes known points in the region of a desired point into greater consideration (weighted). Therefore this approach allows to model non-linear functions by local linear approximations. Especially with data sets of varying quality in the sampled independent variables or with a very different correlation with the dependent variable, MLS does not always deliver optimal models. In statistics this can be assigned to the problem of overfitting. Overfitting means that the model learns a too complex representation of the data and fail to generalize. In practice, this problem can be solved by different approaches of regularization. In this paper, a regularization is presented by adapting a variable-dependent coefficient to the weighting function of the Moving Least Squares method (CMLS). Besides the mathematical implementation, a methology to determine the coefficients and further hyperparameters is given. The significant quantitative and qualitative improvements are demonstrated through two examples. The first example comprises a data set of a simulated, finite-element overload clutch as used in pneumatic commercial-vehicle brakes. The second data set describes a periodically analytic function. For both data sets, the new CMLS approach exhibits significantly better results than MLS (Sect. 4).