Theory and Applications of Recent Robust Methods

Rousseeuw introduced the Minimum Volume Ellipsoid (MVE) estimates of covariance matrix and multivariate location. These estimates, which are broadly used, are affine equivariant and have high breakdown point. Croux et al. (2002) derived the maximum bias curve for point mass contaminations of a non equivariant version of the MVE. In this paper we obtain a similar result for the equivariant version of the MVE estimates.

This paper studies the distribution of Belgian consumer price changes and its interaction with aggregate inflation over the period June 1976-September 2000. Given the fat-tailed nature of this distribution, both classical and robust measures of location, scale, skewness and tail weight are presented. The chronic right skewness of the distribution, revealed by the robust measures, is cointegrated with aggregate inflation, suggesting that it is largely dependent on the inflationary process itself and would disappear at zero inflation.

We show how quantile estimation combined with robust methods can be used in quantitative investment management. A portfolio manager uses a quantitative model to select securities. The objective is to outperform a benchmark portfolio, subject to risk constraints. Traditional stock selection models express expected returns as a function of factors where all parts of the return distribution are affected similarly. This is subsumed by the quantile approach in which a stock’s entire return distribution is a conditional function of factors. Robust methods insure that our estimates do not depend on a small subset of the data. Regression quantile estimates then detect the potentially different impact of factors at the center and tails of the return distribution. This is illustrated in assessing a model’s forecasting accuracy while controlling for return differences between economic sectors. We are thereby able to detect forecasting properties that would have been missed by a conventional analysis of the data.

In this article, we introduce an algorithm to compute the regression quantile functions. This algorithm combines three algorithms — the simplex, interior point, and smoothing algorithm. The simplex and interior point algorithms come from the background of linear programming (Portnoy and Koenker, 1997). While the simplex method can handle small to middle sized data sets, the interior point method can handle large to huge data sets efficiently. The smoothing algorithm is specially designed for the L ₁ or quantile regression type of problems (Chen, 2002), and it outperforms the other two algorithms for fat data sets. Combining these three algorithms produces an algorithm, which is adaptive in the sense that it can intelligently detect the input data sets and select one of the three algorithms to efficiently compute the regression quantile functions.

The support vector machine proposed by Vapnik belongs to a class of modern statistical learning methods based on convex risk minimization. Other special cases are AdaBoost, kernel logistic regression and least squares. The support vector machine has the advantage that it usually leads to a reduction of complexity, because only the support vectors and not all observations contribute to the prediction of a new response. This paper addresses robustness properties of the support vector machine for pattern recognition in finite samples. Sensitivity curves in the sense of J. W. Tukey are used to investigate the possible impact of a single data point.

The Nadaraya-Watson regression estimator is known to be highly sensitive to the presence of outliers in the sample. A possible robustification consists in using local L-estimates of regression. Whereas the local L-estimation is traditionally done using the empirical conditional distribution function, Tamine et al. (2003) have recently proposed to use a smoothed conditional distribution function instead. This work studies computational aspects and small-sample properties of the smoothed L-estimation approach. The smoothed nonparametric L-estimator is applied to the estimation of the so-called implied volatilities, which describe the conditional variance of high-frequency financial time series (such as exchange rates or stock prices) inferred from the prices of related financial derivatives.

Robust estimation of covariance matrices when some of the data at hand are missing is an important problem. It has been studied by Little and Smith (1987) and more recently by Cheng and Victoria-Feser (2002). The latter propose the use of high breakdown estimators and so-called hybrid algorithms (see, e.g., Woodruff and Rocke, 1994). In particular, the minimum volume ellipsoid of Rousseeuw (1984) is adapted to the case of missing data. To compute it, they use (a modified version of) the forward search algorithm (see e.g. Atkinson, 1994). In this paper, we propose to use instead a modification of the C-step algorithm proposed by Rousseeuw and Van Driessen (1999) which is actually a lot faster. We also adapt the orthogonalized Gnanadesikan-Kettenring (OGK) estimator proposed by Maronna and Zamar (2002) to the case of missing data and use it as a starting point for an adapted S-estimator. Moreover, we conduct a simulation study to compare different robust estimators in terms of their efficiency and breakdown.

The d-fullness technique of Vandev (1993) for the finite sample breakdown point study of the Weighted Trimmed Likelihood Estimator is extended. The proposed generalized d-fullness technique is illustrated over the generalized logistic regression model.

The purpose of this work is to investigate on the use of the Integrated Square Error, or L ₂ distance, as a practical tool for parameters estimation of mixtures of two normal bivariates in presence of outliers, situations in which maximum likelihood estimators are usually unstable. Theory is outlined, closed expressions for the L ₂ minimizing estimate criterion for bivariate gaussian mixtures are given. In order to evaluate robustness of maximum likelihood and L ₂ minimizing estimate criteria we compare results arising from a Monte Carlo simulation for some mixtures of gaussian bivariates in occurrence of different outliers positioning and consistency, matching some typical situations that frequently arise in industrial and chemical fields.

Principal Component Regression (PCR) and Partial Least Squares Regression (PLSR) are the two most popular regression techniques in chemo-metrics. They both fit a linear relationship between two sets of variables. The responses are usually low-dimensional whereas the regressors are very numerous compared to the number of observations. In this paper we compare two recent robust PCR and PLSR methods and their classical versions in terms of efficiency, goodness-of-fit, predictive power and robustness.

An expression is given for the exact probability density function of the parameter values that maximize the likelihood of a statistical model, where the data generating model is allowed to differ from the estimation model. The density can be used to study the robustness of estimation of alternative hypothetical models. It is described for curved exponential families, then specifically for gamma distribution models and for nonlinear regression models. An example is given in the context of alternative models for data from the biochemical ELISA test method. Finally an indication is given of how a robustness index can be calculated to assess the effects of estimation of a common parameter vector under a wrong model.

The aim of this paper is to look at the behavior of the total probability of misclassification of robust linear and quadratic discriminant analysis. The effect of outliers on the discriminant rules is studied by comparing their total probabilities of misclassification in presence of outliers.

The authors recently constructed several nonparametric tests of one-sided hypotheses on the value of the Pareto-type tail index in the family of distributions with nondegenerate right tails. Inverting the tests in the Hodges-Lehmann manner (Hodges and Lehmann, 1963), we obtain strongly consistent estimators of m. The simulation study demonstrates surprisingly good approximations of m, namely by two of the three proposed estimators.

Classical multivariate analysis is based on the assumption that the data come from a multivariate normal distribution. The tests of multinormality have therefore received very much attention. Several tests for assessing multinormality, among them Mardia’s popular multivariate skewness and kurtosis statistics, are based on standardized third and fourth moments. In Mardia’s construction of the affine invariant test statistics, the data vectors are first standardized using the sample mean vector and the sample covariance matrix. In this paper we investigate whether, in the test construction, it is advantageous to replace the regular sample mean vector and sample covariance matrix by their affine equivariant robust competitors. Limiting distributions of the standardized third and fourth moments and the resulting test statistics are derived under the null hypothesis and are shown to be strongly dependent on the choice of the location vector and scatter matrix estimate. Finally, the effects of the modification on the limiting and finite-sample efficiencies are illustrated by simple examples in the case of testing for the bivariate normality. In the cases studied, the modifications seem to increase the power of the tests.

The problem of robustness is considered for sequential hypotheses testing on the parameters of Markov chains. The exact expressions for the conditional error probabilities, and for the conditional expected sequence lengths are obtained. Robustness analysis under “contamination” is performed. Numerical results are given to illustrate the theory.

The use of the Box-Cox family of transformations is a popular approach to make data behave according to a linear regression model. The regression coefficients, as well as the parameter λ defining the transformation, are generally estimated by maximum likelihood, assuming homoscedastic normal errors. These estimates are nonrobust; in addition, consistency to the true parameters holds only if the assumptions of normality and homoscedasticity are satisfied. We present here a new class of estimates, for the case of simple regression, which are robust and consistent even if the assumptions of normality and homoscedasticity do not hold.

The paper deals with the least weighted squares estimator which is robust and generalizes classical least trimmed squares. We provide conditions under which this estimator is consistent and we prove consistency for general regression.

In modelling situations, we often have a choice. We can add to model complexity to describe some unusual observations or groups of observations or we can remove those observations and fit the rest with a different, perhaps simpler, model. Or, we might put weights on observations and/or variables and use all of the data or all of the model complexity or some combination. To decide what is unusual generally requires some sort of model, but that very model may be what is causing some observations to be seen as unusual or, in fact, masking unusual observations.When confronted both by unnecessary model complexity and by unusual observations, a model selection technique must be able to identify the correct model structure as well as unusual observations. This paper proposes some new algorithms that are designed to address these problems and compares them with currently available approaches such as robust C _p and robust cross-validated selection.

This paper analyzes the number of samples required for an approximate Monte-Carlo least median of squares (LMS) line estimator. We provide a general computational framework, followed by detailed derivations for several point distributions and subsequent numerical results for the required number of samples.

Regression is the study of the conditional distribution of the response y given the predictors x. In a 1D regression, y is independent of x given a single linear combination βTx of the predictors. Special cases of 1D regression include multiple linear regression, binary regression and generalized linear models. If a good estimate b of some non-zero multiple cβ of β can be constructed, then the 1D regression can be visualized with a scatterplot of bTx versus y. A resistant method for estimating cβ is presented along with applications.

Given two groups of variables redundancy analysis searches for linear combinations of variables in one group that maximize the variance of the other group that is explained by each one of the linear combination. The method is important as an alternative to canonical correlation analysis, and can be seen as an alternative to multivariate regression when there are collinearity problems in the dependent set of variables. Principal component analysis is itself a special case of redundancy analysis.In this work we propose a new robust method to estimate the redundancy analysis parameters based on alternating regressions. These estimators are compared with the classical estimator as well as other robust estimators based on robust covariance matrices. The behavior of the proposed estimators is also investigated under large contamination by the analysis of the empirical breakdown point.

Location of a radio transmitter may be estimated using the triangulation principle and angular measurements from at least two sensor arrays. For example, two base-stations equipped with antenna arrays observe the waveforms emitted by a transmitter such as a mobile phone. Based on the estimates of angle-of-arrival (AoA) of waveforms impinging the antenna array at base-stations, an estimate of the location of the transmitter can be formed using elemental geometry. In this paper, we discuss the ML-estimation of the location of the transmitter by modelling the distribution of the AoAs by two well-known angular distributions, namely the von Mises and the wrapped Cauchy distribution. These distributions are well justified by the physical measurements on radio wave propagation. Since the received signals at the base-stations are corrupted by noise which may be impulsive in nature, robust estimation of the signal parameters is an important issue.

A very common practice in literature is that of building scale estimators by means of a location estimator. The most common scale estimator used is the standard deviation, which is obtained by using the mean; its use is not the most suitable one due to its weakness under the presence of outliers. We can find other scale estimators, based on location estimators, which are more resistant under the presence of outliers, as for instance, the so called Mad, which uses for its construction the median.Since the mean and the median can be considered extreme elements of a location estimators family known as trimmed means, in this paper we propose a scale estimator family called αβ_Trimmed. For its definition, we will use the above mentioned trimmed means family and different parameters α and β, which are called trimming levels. We will demonstrate the good robustness behavior of the elements of such a family. It will be proved that they are affine equivariant, and (depending on the trimming levels) have a high exact fit point, a high breakdown point and a bounded sensitivity curve.

This paper focuses on the robust Efficient Method of Moments (EMM) estimation of a general parametric stationary process and proposes a broad framework for constructing robust EMM statistics in this context. We characterize the local robustness properties of EMM estimators for time series by computing the corresponding influence functions and propose two versions of a robust EMM (REMM) estimator with bounded influence function. We then show by Monte Carlo simulation that our REMM estimators are very successful in controlling for the asymptotic bias under model misspecification while maintaining a high efficiency under the ideal structural model.

The computational geometry community has long recognized that there are many important and challenging problems that lie at the interface of geometry and statistics (e.g., Shamos, 1976; Bentley and Shamos, 1977). The relatively new notion of data depth for non-parametric multivariate data analysis is inherently geometric in nature, and therefore provides a fertile ground for expanded collaboration between the two communities. New developments and increased emphasis in the area of multivariate analysis heighten the need for new and efficient computational tools and for an enhanced partnership between statisticians and computational geometers.Over a decade ago point-line duality and combinatorial and computational results on arrangements of lines contributed to the development of an efficient algorithm for two-dimensional computation of the LMS regression line (Souvaine and Steele, 1987; Edelsbrunner and Souvaine, 1990). The same principles and refinements of them are being used today for more efficient computation of data depth measures. These principles will be reviewed and their application to statistical problems such as the LMS regression line and the computation of the half-space depth contours will be presented. In addition, results of collaborations between computational geometers and statisticians on data-depth measures (such as half-space depth and simplicial depth) will be surveyed.

The covariance matrix is a key component of many multivariate robust procedures, whether or not the data are assumed to be Gaussian. We examine the idea of robustly fitting a mixture of multivariate Gaussian densities in the situation when the number of components estimated is intentionally too few. Using a minimum distance criterion, we show how useful results may be obtained in practice. Application areas are numerous, and examples will be provided.

Much of computer vision and image analysis involves the extraction of “meaningful” information from images using concepts akin to regression and model fitting. Applications include: robot vision, automated surveillance (civil and military) and inspection, biomedical image analysis, video coding, human-machine interface, visualization, historical film restoration etc. However, problems in computer vision often have characteristics that are distinct from those usually addressed by the statistical community. These include pseudo-outliers: in a given image, there are usually several populations of data. Some parts may correspond to one object in a scene and other parts will correspond to other, rather unrelated, objects. When attempting to fit a model to this data, one must consider all populations as outliers to other populations — the term pseudo-outlier has been coined for this situation. Thus it will rarely happen that a given population achieves the critical size of 50% of the total population and, therefore, techniques that have been touted for their high breakdown point (e.g., Least Median of Squares) are no longer reliable candidates, being limited to a 50% breakdown point.Computer vision researchers have developed their own techniques that perform in a robust fashion. These include RANSAC, ALKS, RESC and MUSE. In this paper new robust procedures are introduced and applied to two problems in computer vision: range image fitting and segmentation, and image motion estimation. The performance is shown, empirically, to be superior to existing techniques and effective even when as little as 5-10% of the data actually belongs to any one structure.

A simulation study had been carried out to compare the Type I error and power of S₁, a statistic recommended by Babu et al. (1999) for testing the equality of location parameters for skewed distributions. Othman et al. (in press) showed that this statistic is robust to the underlying populations and is also powerful. In our work, we modified this statistic by replacing the standard errors of the sample medians with four alternative robust scale estimators; the median absolute deviation (MAD) and three of the scale estimators proposed by Rousseeuw and Croux (1993); Q _n , S _n , and T _n These estimators were chosen based on their high breakdown value and bounded influence function, and in addition, they are simple and easy to compute. Even though MAD is more appropriate for symmetric distributions (Rousseeuw and Croux, 1993), due to its popularity and for the purpose of comparison, we decided to include it in our study. The comparison of these methods was based on their Type I error and the power for J = 4 groups in an unbalanced design having heterogeneous variances. Data from the Chi-square distribution with 3 degrees of freedom were considered. Since the null distribution of S₁ is intractable, and its asymptotic null distribution may not be of much use for practical sample sizes, bootstrap methods were used to give a better approximation. The S₁ statistic combined with each of the scale estimators was shown to have good control of Type I errors.

New rank scores test statistics are proposed for testing whether two random vectors are independent. The tests are asymptotically distribution-free for elliptically symmetric marginal distributions. Recently, Gieser and Randies (1997), Taskinen et al. (2003a) and Taskinen et al. (2003b) introduced and discussed different multivariate extensions of the quadrant test, Kendall’s tau and Spearman’s rho statistics. In this paper, standardized multivariate spatial signs and the (univariate) ranks of the Mahalanobis-type distances of the observations from the origin are combined to construct rank scores tests of independence. The limiting distributions of the test statistics are derived under the null hypothesis as well as under contiguous sequences of alternatives. Three different choices of the score functions, namely the sign scores, the Wilcoxon scores and the van der Waerden scores, are discussed in greater detail. The small sample and limiting efficiencies of the test procedures are compared and the robustness properties are illustrated by an example. It is remarkable that, in the multinormal case, the limiting Pitman efficiency of the van der Waerden scores test equals to that of the classical parametric Wilks’ test.

We reintroduced the idea of an n-fold compound option as a generalization of Geske’s (2-fold) compound option in the same framework of constant interest rates. For the valuation of long-term financial agreements (life insurance products) this assumption is not always realistic so that the stochastic modelling of the interest rates might be a better approach.According to Miltersen et al. (1997), we will use the requirement of simple interest rates over a fixed finite period to be log-normal distributed. With these assumptions, closed-form solutions are determined for the n-fold compound call options written on zero-coupon bonds.A numerical illustration of the application of robust methods to interest rates is discussed.

In this paper we construct estimators for the parameters of the multivariate sinh-1-normal distribution. We discuss some properties of these estimators, such as consistency, B-robustness and asymptotic normality.

The objectives of a robust statistical analysis and of an extreme value analysis apparently are contradictory. Where the extreme data are downweighted in robust statistics, these observations receive most attention in an extreme value approach. The most prominent extreme value methods however are constructed on maximum likelihood estimates based on specific parametric models which are fitted to exceedances over large thresholds. So within an extreme value framework some robust algorithms replacing the maximum likelihood part of this methodology can be of use leading to estimates which are less sensitive to few particular observations. This study is motivated by a soil database quality management project, where in the background of Pareto-type tails, automatic identification of suspicious data is needed.

When performing mechanical vibration measurements, it is often assumed that the disturbing measurement noise behaves Gaussian. In reality, this is not always the case, and therefore classical least-squares procedures can give poor results when processing the measurements. In this paper, several robust statistical procedures will be used in four selected problems in vibration measurement and analysis. It will be shown that the procedures improve the results compared to classical least-squares methods. Because fast processing is required, a tradeoff between robustness of the methods and computation speed is made. In particular the following steps in vibration engineering were robustified: (1) positional calibration, (2) measurement post-processing, (3) system identification and (4) data classification for damage detection.

The robust estimation of regression parameters is formulated in terms of mixed integer-quadratic programming problem. The main contribution of this technique is that it improves the estimator efficiency by down-weighting only bad influential points, either y-outliers or x-outliers. We follow the minimax strategy where the objective function of our mathematical programming formulation is mainly a Huber loss function, and bad influential outliers pulled towards the regression line with low cost. This penalized pulling cost is a function of Mallows type weights, and in the modified data a GM estimator (Schweppe type) could be defined. The main advantage of the proposed technique is that data points are not down-weighted, unless they have increased substantially the square residuals. Previously published mixed integer formulations withdraw data points, the most influential even if they are not bad influential points. GM estimators are compared to our proposal via simulated experiments, the robust estimator obtained by quadratic programming is reasonable.