Bayesian inference for -stable distributions: A random walk MCMC approach
Introduction
The -stable family of distributions stems from a more general version of the central limit theorem which replaces the assumption of the finiteness of the variance with a much less restrictive one concerning the regular behavior of the tails (Gnedenko and Kolmogorov, 1954); the Gaussian distribution is thus a particular case of -stable distribution. This family of distributions has a very interesting pattern of shapes, allowing for asymmetry and thick tails, that makes them suitable for the modelling of several phenomena, ranging from engineering (noise of degraded audio sources) to finance (asset returns); moreover, it is closed under linear combinations.
The family is identified by means of the characteristic functionwhich depends on four parameters: , measuring the tail thickness (thicker tails for smaller values of the parameter), determining the degree and sign of asymmetry, (scale) and (location). The shorthand notation I will employ for the distribution is .
While the characteristic function (1) has a quite manageable expression and can straightforwardly produce several interesting analytic results (Zolotarev, 1986), it has unfortunately a major drawback in what concerns estimation and inferential purposes: it is not continuous with respect to the parameters, having a pole at .
An alternative way to write the characteristic function that overcomes this problem, due to Zolotarev (1986), is the following:In this case, the distribution will be denoted as . The formulation of the characteristic function is, in this case, quite more cumbersome, and the analytic properties have a less intuitive meaning. At any rate, this formulation is much more useful in what concerns statistical purposes. The only parameter that needs to be “translated” according to the following relationship is :On the basis of the above equations, a distribution corresponds to a , provided that .
Another parameterization which is sometimes used is the following (Zolotarev, 1986):Also in this case, however, the function is not continuous with respect to and presents a pole at . Another unpleasant feature of this way of writing the characteristic function is that the meaning of the asymmetry parameter changes according to the value of : when a negative indicates negative skewness, whereas for it produces positive skewness. For what concerns the “translation” of this parameterization into (1), one has, for : andwhile and remain unchanged.
Unfortunately, (1), (2) and (4) cannot be inverted to yield a closed-form density function except for a very few cases: , corresponding to the normal distribution, and , yielding the Cauchy distribution, and for the Lévy distribution.
This difficulty, coupled with the fact that moments of order greater than do not exist whenever , has made impossible the use of standard estimation methods such as maximum likelihood and the method of moments. Researchers have thus proposed alternative estimation schemes, mainly based on quantiles (McCulloch, 1986). Albeit consistent, however they are not efficient, and most of all they are designed only for the estimation of the distribution parameters: employing them for the estimation of more complex models requires therefore a two-step estimation approach. With the availability of powerful computing machines, it has become possible to exploit computationally-intensive estimation methods for the estimation of -stable distributions parameters, such as maximum likelihood based on the FFT of the characteristic function (Mittnik et al., 1999) or numerical quadrature of the characteristic function (Nolan, 1997). Those methods, however, present some inconveniences: their accuracy is quite poor for small values of because of the spikedness of the density function; furthermore, when the parameters are near their boundary, the distributions of the estimators become degenerate making frequentist inferential procedures unreliable.
The Bayesian approach has suffered from the same difficulties as the frequentist one, as the absence of a closed-form density prevented from evaluating the likelihood function and thus constructing posterior inferential schemes. Also in this case, however, the availability of fast computing machines has made possible the use of MCMC methods. In particular, Buckle (1995) has shown that, conditionally on an auxiliary variable, it is possible to express the density function in a closed form. With this result, he proposes a Gibbs sampling scheme for the -stable distribution parameters.
This approach, however, presents some difficulties. It is unfortunately not straightforward to produce random numbers from this auxiliary variable and one must resort to rejection sampling. Since one needs a random sample from the auxiliary variable of the same size as the observation vector for each iteration of the chain, it follows that this approach can be particularly slow, especially when large sample sizes are involved. Furthermore, since the parameterization involved is (4) and has poles at and , one needs to a priori constrain and on a portion of their support.
Section snippets
A random walk Metropolis sampler
I shall here introduce a novel approach for the construction of the posterior density of the (possibly) asymmetric -stable law parameters that avoids the use of the auxiliary vector.
To put it on more formal grounds, one looks for a computable expression for the likelihood function in order to be able to produce samples from the posterior distribution of the parameters according to Bayes’ theorem, namely As I have previously noted, an approximate version of
A practical example
As an illustration on how the proposed method works on real-world data, I have carried out the estimation of a sample of audio noise drawn from a set of recordings of songs taken by Robert Lachmann in Palestine in early twentieth century by means of a mobile recording studio (Lachmann, 1929); as one might guess, the audio medium is very degraded and the noise is extremely heavy tailed. The audio sample consists of 44487 observations and is the same used in Lombardi and Godsill (2006), to which
Concluding remarks
In this paper, I have presented a random walk Metropolis MCMC scheme for the parameters of -stable distributions. Although it is based on an approximated version of the likelihood, this approach was shown to perform remarkably well, being as twice as fast than the Gibbs sampler proposed by Buckle (1995). This contribution is a very first step in what appears to be a very promising direction; future research will aim at extending this approach to regression and time series models.
In this paper,
Acknowledgements
I would like to thank Steve Brooks, Fabio Corradi and Federico M. Stefanini for their useful comments and especially my co-supervisor Fabrizia Mealli for her insightful suggestions and discussions. A preliminary version of this paper was presented at the SIS 2004 scientific meeting in Bari and at the S.Co. 2005 Conference in Bressanone.
References (13)
- et al.
Maximum likelihood estimation of stable paretian models
Math. Comput. Model.
(1999) Monte Carlo inference in econometric models with symmetric stable distributions
J. Econometrics
(1999)On some expansions of stable distribution functions
Ark. Math.
(1952)Bayesian inference for stable distributions
J. Amer. Statist. Assoc.
(1995)On the asymptotic normality of the maximum-likelihood estimate when sampling from a stable distribution
Ann. Statist.
(1973)- et al.
Limit Distributions for Sums of Independent Random Variables
(1954)
Cited by (59)
Bayesian inference for amplitude distribution with application to radar clutter
2024, Digital Signal Processing: A Review JournalModeling and Bayesian inference for processes characterized by abrupt variations
2023, Journal of Process ControlMultivariate α-stable distributions: VAR(1) processes, measures of dependence and their estimations
2023, Journal of Multivariate AnalysisStable Randomized Generalized Autoregressive Conditional Heteroskedastic Models
2020, Econometrics and StatisticsCitation Excerpt :However, both these approximations have quite poor accuracy for small values of α because of the spikedness of the density function. Another method that is benefited by the increased computational power is the Bayesian approach: simulation-based MCMC samplers have been proposed by Buckle (1995); Qiou and Ravishanker (1998) and Lombardi (2007). It is known, from statistical literature, that financial data, audio signals, among others, are heavy-tailed.
A nonlinear population Monte Carlo scheme for the Bayesian estimation of parameters of α-stable distributions
2016, Computational Statistics and Data AnalysisCitation Excerpt :Thus, it is fairly complicated, if not impossible, to adjust those parameters for a general case, particularly when the distribution of interest is heavy-tailed, as already noted in Peters et al. (2012). For this reason, we do not present simulation results for the algorithm of Lombardi (2007) in this paper. We have not considered the Gibbs sampling method proposed in Buckle (1995) either since it turns out even more computationally demanding than the random walk MCMC algorithm, as stated in Lombardi (2007).
Performance degradation assessment for mechanical system based on semi-analytical solution of self-similar stable distribution process
2023, Structural Health Monitoring