Asymptotic Theory of Statistical Inference for Time Series

Much of statistical analysis is concerned with models in which the observations are assumed to vary independently. However, a great deal of data in economics, engineering, and the natural sciences occur in the form of time series where observations are dependent and where the nature of this dependence is of interest in itself. A model which describes the probability structure of a series of observations X _t , t = 1,…, n, is called a stochastic process. An X_t might be the value of a stock price at time point t, the water level in a lake at time point t, and so on. The primary purpose of this book is to provide statistical inference for stochastic processes, which is based on the probability theory for them. In this chapter some of the elements of stochastic processes will be reviewed. Because the statistical analysis for stochastic processes largely relies on the asymptotic theory, we also explain some useful limit theorems and central limit theorems. We have placed some fundamental results of mathematics, probability, and statistics in the Appendix.

In classical time series analysis the asymptotic estimation and testing theory was developed for linear processes, which include the AR, MA, and ARMA models. However, in the last twenty years a lot of more complicated stochastic process models have been introduced, such as, nonlinear time series models, diffusion processes, point processes, and nonergodic processes. This chapter is devoted to providing a modern asymptotic estimation and testing theory for those various stochastic process models. The approach is mainly based on the LAN results given in the previous chapter. More concretely, in Section 3.1 we discuss the asymptotic estimation and testing theory for non-Gaussian vector linear processes in view of LAN. The results are very general and grasp a lot of other works dealing with AR, MA, and ARMA models as special cases. Section 3.2 reviews some elements of nonlinear time series models and the asymptotic estimation theory based on the conditional least squares estimator and maximum likelihood estimator (MLE). We address the problem of statistical model selection in general fashion. Also the asymptotic theory for nonergodic models is mentioned. Recently much attention has been paid to continuous time processes (especially diffusion processes), which appear in finance. Hence, in Section 3.3 we describe the foundation of stochastic integrals and diffusion processes. Then the LAN-based asymptotic theory of estimation for them is studied.

We have already seen that various typical estimators and tests have the same limiting behavior. Thus it is required to illuminate their distinction. In this chapter, using higher order approximations (Edgeworth expansions) of the distribution of estimators and tests we discuss their higher order asymptotic optimality.

In Section 2.2 we already mentioned stationary processes whose autocovariance functions converge to zero with power law decay. Because this rate of convergence is slower than that of the usual AR, MA, and ARMA processes, we call them long-memory processes (or processes with long-range dependence). The phenomenon of long-range dependence was known long before suitable statistical models were introduced. Hurst (1951) studied the records of water flows through the Nile and through other rivers, the price of wheat, and meteorological series such as rainfall, temperature, and so on. His empirical conclusion was that the range (to be defined in Section 5.1) of the records shows long-range dependence. Motivated by Hurst’s results, Mandelbrot and Van Ness (1968) introduced fractional Brownian motions and fractional noises (to be defined in Section 5.1), and related works (Mandelbrot and Wallis (1968, 1969a, b)) claimed that Hurst’s findings could be modeled by them. Since then, a lot of probabilistic and statistical methods have been brought in long-memory processes (see Beran (1994a) and Robinson (1994a)). Interestingly, the illuminated results are often different from those for ordinary short-memory processes. Also the applications have been extended from hydrology to a variety of fields such as economics, engineering, enviromental sciences, and physics (see Beran (1994a)). Thus long-memory has become a central component of time series analysis. This chapter is devoted to presenting a concise and modern review of statistical analysis for long-memory processes.

Many important quantities in stationary time series are often expressed as functionals of spectral density. For a linear functional, a natural idea of constructing an estimator is to replace an unknown spectral density by the periodogram based on the data. The functional of interest is, however, not always linear with respect to the spectral density. For the nonlinear case replacing the unknown spectral density by the periodogram causes asymptotic bias since the periodogram divided by the spectral density is asymptotically exponential. It is well known that the periodogram is an asymptotically unbiased but inconsistent estimator of the spectral density. By virtue of the fact of asymptotic independence of the periodograms, it is possible to construct a consistent estimator of the spectal density by an approach that uses a smoothed periodogram (nonparametric kernel spectral estimator). Although the rate of convergence of the nonparametric kernel spectral estimator is smaller than the usual order n ^1/2, where n is a sample size, the integration of the nonparametric kernel spectral estimator over [-π, π] leads to the recovery of the ordinary n ^1/2 asymptotics.

The extension of classical discriminant analysis techniques in multivariate analysis to time series data is a problem of practical interest. In Section 7.1 we give a basic formulation of discriminant analysis. We begin with the standard methods from classical multivariate analysis and then introduce the frequency domain approach in time series analysis (Section 7.2). Even if the Gaussianity of the process is not assumed, we can construct the Gaussian likelihood or its spectral version, which is called the Whittle asymptotic likelihood. In Section 7.3 we discuss the non-Gaussian robustness of the discriminant rule based on the Whittle asymptotic likelihood. By the non-Gaussian robustness we mean that two kinds of misclassification probability, P(2|1) and P(1|2), are asymptotically independent of the nonGaussianity of the sequence {U(t)= (U₁(t),..., U_m(t))’}of the i.i.d. errors, which appear in rnvector linear processes. Generally, the asymptotic distribution of the resulting discriminant statistic depends on the fourth order cumulants kabcd = Cum{U _a (1),U _b (1), U _c (1), U _d (1)}, a, b, c,d = 1, ..., m. We usually consider the local analysis in such a way that, as the sample size n tends to infinity, we move the hypothetical spectral density under a category II₂ closer to the one under a category II₁. The Whittle asymptotic log-likelihood can be regarded as a spectral measure. It is natural to define a more general spectral distance, including two important measures like the Kullback—Leibler divergence and the Chernoff information divergence which are, in Section 7.6, derived as spectral approximations. Based on such a general spectral distance, two approaches for the discriminant analysis are possible. One approach is to use a nonparametric kernel spectral density estimator constructed from a realization that we want to classify. Section 7.4 extends the results of Section 7.3 to nonparametric discriminant statistics. The other approach treated in Section 7.5 is based on a fitted parametric spectrum (for example, an AR spectrum). We also discuss the higher orde asymptotics of parametric discriminant statistics for Gaussian stationary processes. Section 7.7 gives a brief discuddion on the discriminant analysis for other stochastic processes such as diffusion processes and nonlinear processes.

A large number of results in Chapters 1–7 have been obtained via the central limit theorem (CLT) and the Edgeworth expansion of suitably normalized quantities. The latter is the basic tool for higher order asymptotic theory as in Chapter 4. On the other hand, there are other important tools, namely large deviation theorem and saddlepoint approximation. Section 8.1 is devoted to a derivation of a variant of the large deviation theorem, which turns out to be available for several statistical problems in Gaussian stationary processes, especially for quadratic functionals. Further we also give another example of the Ornstein-Uhlenbeck (O-U) diffusion process. In Section 8.2 we review general concepts of Bahadur’s asymptotic efficiency of estimators and tests based on the large deviation approach, with special emphasis on the spectral analysis of Gaussian stationary processes. Section 8.3 describes some developments for the O-U diffusion process.