A Random-Discretization Based Monte Carlo Sampling Method and its Applications

Abstract

Recently, several Monte Carlo methods, for example, Markov Chain Monte Carlo (MCMC), importance sampling and data-augmentation, have been developed for numerical sampling and integration in statistical inference, especially in Bayesian analysis. As dimension increases, problems of sampling and integration can become very difficult. In this manuscript, a simple numerical sampling based method is systematically developed, which is based on the concept of random discretization of the density function with respect to Lebesgue measure. This method requires the knowledge of the density function (up to a normalizing constant) only. In Bayesian context, this eliminates the “conjugate restriction” in choosing prior distributions, since functional forms of full conditionals of posterior distributions are not needed. Furthermore, this method is non-iterative, dimension-free, easy to implement and fast in computing time. Some benchmark examples in this area are used to check the efficiency and accuracy of the method. Numerical results demonstrate that this method performs well for all these examples, including an example of evaluating the small probability values of a high dimensional multivariate normal distribution. As a byproduct, this method also provides an easy way of computing maximum likelihood estimates and modes of posterior distributions.