ARMA Model Identification

Consider the autoregressive moving-average (ARMA) model of orders p and q, where \(\phi \left( B \right) = - {\phi _0} - {\phi _1}B - \cdots - {\phi _p}{B^p},\theta \left( B \right) = - {\theta _0} - {\theta _1}B - \cdots - {\theta _q}{B^q},{\theta _0} = {\theta _0} = - 1,{\phi _p} \ne 0,{\theta _q} \ne 0\), B is the backshift operator, and {v _t } is a sequence of independent and identically distributed random variables with means 0 and variances σ² (> 0). The sequence {v _t } is called either a white noise process or an innovation process. In some time series books, the white noise process is defined as a sequence of uncorrelated random variables instead of that of independent random variables. In practical time series analysis, there is not as much difference between the two definitions. We assume that the model is stationary and invertible, i.e., the equations ϕ(z) = 0 and θ(z) = 0 have all the roots outside the unit circle. We assume that the two equations have no common root. This assumption is sometimes called coprimal. The stationarity and the invertibility conditions have been discussed by several authors. Interested readers may consult the references in Section 1.6. In statistical literature, the white noise process is frequently assumed to be Gaussian, i.e., normally distributed. There are some references about non-Gaussian ARMA processes in Section 1.6. In this book, we assume that the coefficients ϕ₁…, ϕ _p , ϕ₁…, ϕ _q , and the white noise variance σ² are constants, i.e., they do not depend on time.

It is known (see, e.g., T. W. Anderson [1971, pp. 463–495]) that, for large T, the bias of the sample ACRF is .

Since the early 1970s, some estimation-type identification procedures have been proposed. They are to choose the orders k and i minimizing , where σ _k,i ² is an estimate of the white noise variance obtained by fitting the ARMA(k, i) model to the observations. Because σ _k,i ² decreases as the orders increase, it cannot be a good criterion to choose the orders minimizing it. If the orders increase, the bias of the estimated model will decrease while the variance increases. Therefore, we should compromise between them. For this purpose we add the penalty term, (k + i)C(T)/T, into the model selection criterion The penalty function identification methods are regarded as objective.

A penalty function identification of ARMA processes is to choose the orders minimizing among k = 0,…, K and i = 0,…, I. Here σ _k,i ² is an estimate of the innovation variance obtained by fitting the ARMA(k, i) model to the observations and K and I are determined a priori as upper bounds of the orders. Because there are (K+1)×(I+1) possible ARMA models to be estimated, it is computationally onerous to apply ML estimation methods. Even though many algorithms have been presented to obtain the exact ML estimates as mentioned in Chapter 1, there are still many problems in applying them to all possible ARMA models. Especially if the MA part exists, then the ML estimates are not always on the stationary and invertible region. They are sensitive to the quality of starting values for the algorithms.