Resampling Methods for Dependent Data

The bootstrap is a computer-intensive method that provides answers to a large class of statistical inference problems without stringent structural assumptions on the underlying random process generating the data. Since its introduction by Efron (1979), the bootstrap has found its application to a number of statistical problems, including many standard ones, where it has outperformed the existing methodology as well as to many complex problems where conventional approaches failed to provide satisfactory answers. However, it is not a panacea for every problem of statistical inference, nor does it apply equally effectively to every type of random process in its simplest form. In this monograph, we shall consider certain classes of dependent processes and point out situations where different types of bootstrap methods can be applied effectively, and also look at situations where these methods run into problems and point out possible remedies, if there is one known.

In this chapter, we describe various commonly used bootstrap methods that have been proposed in the literature. Section 2.2 begins with a brief description of Efron’s (1979) bootstrap method based on simple random sampling of the data, which forms the basis for almost all other bootstrap methods. In Section 2.3, we describe the famous example of Singh (1981), which points out the limitation of this resampling scheme for dependent variables. In Section 2.4, we present bootstrap methods for time-series models driven by iid variables, such as the autoregression model. In Sections 2.5, 2.6, and 2.7, we describe various block bootstrap methods. A description of the subsampling method is given in Section 2.8. Bootstrap methods based on the discrete Fourier transform of the data are described in Section 2.9, while those based on the method of sieves are presented in Section 2.10.

In this chapter, we study the first-order properties of the MBB, the NBB, the CBB, and the SB for the sample mean. Note that for the first three block bootstrap methods, the block length is nonrandom. In Section 3.2, we establish consistency of these block bootstrap methods for variance and distribution function estimations for the sample mean. The SB method uses a random block length and hence, requires a somewhat different treatment. We study consistency properties of the SB method for the sample mean in Section 3.3.

In this chapter, we establish consistency of different block bootstrap methods for some general classes of estimators and consider some specific examples illustrating the theoretical results. Section 4.2 establishes consistency of estimators that may be represented as smooth functions of sample means. Section 4.3 deals with (generalized) M-estimators, including the maximum likelihood estimators of parameters, which are defined through estimating equations. Some special considerations are required while defining the bootstrap versions of such estimators. We describe the relevant issues in detail in Section 4.3. Section 4.4 gives results on the bootstrapped empirical process, and establishes consistency of bootstrap estimators for certain differentiable statistical functionals. Section 4.5 contains three numerical examples, illustrating the theoretical results of Sections 4.2–4.4.

In this chapter, we compare the performance of the MBB, the NBB, the CBB, and the SB methods considered in Chapters 3 and 4. In Section 5.2, we present a simulated data example and illustrate the behavior of the block bootstrap methods under some simple time series models. Although the example treats the simple case of the sample mean, it provides a representative picture of the properties of the four methods in more general problems. In the subsequent sections, the empirical findings of Section 5.2 are substantiated through theoretical results that provide a comparison of the methods in terms of the (asymptotic) MSEs of the bootstrap estimators. In Section 5.3, we describe the framework for the theoretical comparison. In Section 5.4, we obtain expansions for the MSEs of the relevant bootstrap estimators as a function of the block size (expected block size, for the SB). These expansions provide the basis for the theoretical comparison of the sampling properties of the bootstrap methods. In Section 5.5, the main theoretical findings are presented. Here, we compare the bootstrap methods using the leading terms in the expansions of the MSEs derived in the previous section. In Section 5.5, we also derive theoretical optimal (expected) block lengths for each of the block bootstrap estimators and compare the methods at the corresponding optimal block lengths. Some conclusions and implications of the theoretical and finite sample simulation results are discussed in Section 5.6. Proofs of two key results from Section 5.4 are separated out into Section 5.7.

In this chapter, we consider second-order properties of block bootstrap estimators for estimating the sampling distribution of a statistic of interest. The basic tool for studying second-order properties of block bootstrap distribution function estimators is based on the theory of Edgeworth expansions.

As we have seen in the earlier chapters, performance of block bootstrap methods critically depends on the block size. In this chapter, we describe the theoretical optimal block lengths for the estimation of various level-2 parameters and discuss the problem of choosing the optimal block sizes empirically. For definiteness, we restrict attention to the MBB method. Analogs of the block size estimation methods presented here can be defined for other block bootstrap methods. In Section 7.2, we describe the forms of the MSE-optimal block lengths for estimating the variance and the distribution function. In Section 7.3, we present a data-based method for choosing the optimal block length based on the subsampling method. This is based on the work of Hall, Horowitz and Jing (1995). A second method based on the Jackknife-After-Bootstrap (JAB) method is presented in Section 7.4. Numerical results on finite sample performance of these optimal block length selection rules are also given in the respective sections.

In this chapter, we consider bootstrap methods for some popular time series models, such as the autoregressive processes, that are driven by iid random variables through a structural equation. As indicated in Chapter 2, for such models, it is often possible to adapt the basic ideas behind bootstrapping a linear regression model with iid error variables (cf. Freedman (1981)). In Section 8.2, we consider stationary autoregressive processes of a general order and describe a version of the autoregressive bootstrap (ARB) method. Like Efron’s (1979) IID resampling scheme, the ARB also resamples a single value at a time. We describe theoretical and empirical properties of the ARB for the stationary case in Section 8.2. In Section 8.3, we consider the explosive autoregressive processes. In the explosive case, the initial variables defining the model have nontrivial effects on the limit distributions of the least squares estimators of the autoregression (AR) parameters. As a result, the validity of the ARB critically depends on the initial values. In Section 8.3, we describe the relevant issues and provide conditions for the validity of the ARB method in the explosive case.

The models considered so far in this book dealt with the case where the data can be modeled as realizations of a weakly dependent process. In this chapter, we consider a class of random processes that exhibit long-range dependence. The condition of long-range dependence in the data may be described in more than one way (cf. Beran (1994), Hall (1997)). For this book, an operational definition of long-range dependence for a second-order stationary process is that the sum of the (lag) autocovariances of process diverges. In particular, this implies that the variance of the sample mean based on a sample of size n from a long-range dependent process decays at a rate slower than O(n ⁻¹) as n → ∞. As a result, the scaling factor for the centered sample mean under long-range dependence is of smaller order than the usual scaling factor n ^1/2 used in the independent or weakly dependent cases. Furthermore, the limit distribution of the normalized sample mean can be nonnormal. In Section 10.2, we describe the basic framework and review some relevant properties of the sample mean under long-range dependence. In Section 10.3, we investigate properties of the MBB approximation. Here the MBB provides a valid approximation if and only if the limit law of the normalized sample mean is normal. In Section 10.4, we consider properties of the subsampling method under long-range dependence. We show that unlike the MBB, the subsampling method provides valid approximations to the distributions of normalized and studentized versions of the sample mean for both normal and nonnormal limit cases. In Section 10.5, we report the results from a small simulation study on finite sample performance of the subsampling method.

In this chapter, we consider two topics, viz., bootstrapping heavy-tailed time series data and bootstrapping the extremes (i.e., the maxima and the minima) of stationary processes. We call a random variable heavy-tailed if its variance is infinite. For iid random variables with such heavy tails, it is well known (cf. Feller (1971b), Chapter 17) that under some regularity conditions on the tails of the underlying distribution, the normalized sample mean converges to a stable distribution. Similar results are also known for the sample mean under weak dependence. In Section 11.2, we introduce some relevant definitions and review some known results in this area. In Sections 11.3 and 11.4, we present some results on the performance of the MBB for heavy-tailed data under dependence. Like the iid case, here the MBB works if the resample size is of a smaller order than the original sample size. Consistency properties of the MBB are presented in Section 11.3, while its invalidity for a resample size equal to the sample size is considered in Section 11.4.

In this chapter, we describe bootstrap methods for spatial processes observed at finitely many locations in a sampling region in ℝ ^d . Depending on the spatial sampling mechanism that generates the locations of these data-sites, one gets quite different behaviors of estimators and test statistics. As a result, formulation of resampling methods and their properties depend on the underlying spatial sampling mechanism. In Section 12.2, we describe some common frameworks that are often used for studying asymptotic properties of estimators based on spatial data. In Section 12.3, we consider the case where the sampling sites (also referred to as data-sites in this book) lie on the integer grid and describe a block bootstrap method that may be thought of as a direct extension of the MBB method to spatial data. Here, some care is needed to handle sampling regions that are not rectangular. We establish consistency of the bootstrap method and give some numerical examples to illustrate the use of the method. Section 12.4 gives a special application of the block resampling methods. Here, we make use of the resampling methods to formulate an asymptotically efficient least squares method of estimating spatial covariance parameters, and discuss its advantages over the existing estimation methods. In Section 12.5, we consider irregularly spaced spatial data, generated by a stochastic sampling design. Here, we present a block bootstrap method and show that it provides a valid approximation under nonuniform concentration of sampling sites even in presence of infill sampling. It may be noted that infill sam-pling leads to conditions of long-range dependence in the data, and thus, the block bootstrap method presented here provides a valid approximation under this form of long-range dependence. Resampling methods for spatial prediction are presented in Section 12.6.