Skip to main content
main-content

Über dieses Buch

Smoothness Priors Analysis of Time Series addresses some of the problems of modeling stationary and nonstationary time series primarily from a Bayesian stochastic regression "smoothness priors" state space point of view. Prior distributions on model coefficients are parametrized by hyperparameters. Maximizing the likelihood of a small number of hyperparameters permits the robust modeling of a time series with relatively complex structure and a very large number of implicitly inferred parameters. The critical statistical ideas in smoothness priors are the likelihood of the Bayesian model and the use of likelihood as a measure of the goodness of fit of the model. The emphasis is on a general state space approach in which the recursive conditional distributions for prediction, filtering, and smoothing are realized using a variety of nonstandard methods including numerical integration, a Gaussian mixture distribution-two filter smoothing formula, and a Monte Carlo "particle-path tracing" method in which the distributions are approximated by many realizations. The methods are applicable for modeling time series with complex structures.

Inhaltsverzeichnis

Frontmatter

1. Introduction

Abstract
Some of the problems of modeling scalar and multivariate, linear and nonlinear, Gaussian and non-Gaussian stationary and nonstationary time series are addressed here primarily from a smoothness priors state space point of view. Smoothness priors is a stochastic regression treatment of stationary and nonstationary time series. Least squares and standard (Kalman filter) state space methods are used in linear Gaussian time series modeling. A general state space approach is introduced and exploited to model not necessarily Gaussian-not necessarily linear time series with abrupt dis-continuities and outliers.
Genshiro Kitagawa, Will Gersch

2. Modeling Concepts and Methods

Abstract
In this chapter the ideas that we identify as the basis of our approach to time series analysis are outlined. Several topics in parameter estimation and model selection are treated. Akaike’s AIC for parametric model selection is treated first. That treatment includes a discussion of the Kullback-Leibler information, and a theoretical development of the AIC. Also included are treatments of the Householder transformation based least squares estimation, the maximum likelihood method of parameter estimation, (including a method of minimizing a function of several variables), a fairly general discussion of state space modeling including Kalman filter for standard linear Gaussian state space modeling, and general state space modeling.
Genshiro Kitagawa, Will Gersch

3. The Smoothness Priors Concept

Abstract
The “history” of smoothness priors essentially starts with a problem addressed in Whittaker (1923). It was followed by Shiller (1973), and Akaike (1980a) in which the framework initiated by Shiller was continued Akaike (1980a) was a quasi-Bayesian Gaussian disturbances linear regression, least squares computations, model framework. Stochastic difference equation constraints were placed on the prior distributions of the model parameters. The critical computation was that of the likelihood of hyperparameters of those distributions. Our own work and a considerable amount of other work was motivated by Akaike (1980a). Here we identify some of that work and some relationship of that work to other research as well as developments and extensions. The least squares computational framework of smoothness priors is also presented here.
Genshiro Kitagawa, Will Gersch

4. Scalar Least Squares Modeling

Abstract
In this chapter we review some of the applications of smoothness priors modeling of time series that can be done by least squares, or equivalently as linear Gaussian modeling. Smoothness priors trend estimation for scalar time series is treated in Section 4.1. There, the smoothness priors constraint is expressed as a k-th order random walk with a normally distributed zero-mean, unknown variance perturbation. The (normalized) variance is a hyperparameter of the prior distribution. This constraint is a time domain constraint on the priors. The concept of frequency domain priors is introduced and exploited in Sections 4.2 and 4.3. There, the fitting of a smoothness priors long AR model for the spectrum estimation of a scalar stationary time series and a smoothness priors model for transfer function estimation between two simultaneously observed time series, are respectively shown. In addition in Section 4.2, the superiority of the smoothness priors long AR model as compared to ordinary AIC AR model order determined spectral analysis, is demonstrated by a Monte Carlo computation of entropy.
Genshiro Kitagawa, Will Gersch

5. Linear Gaussian State Space Modeling

Abstract
Linear Gaussian state space modeling is treated in this chapter. The prediction, filtering and smoothing formulas in the standard Kalman filter are shown. Model identification or, computation of the likelihood of the model is also treated. Some of the well known state space models that are used in this book as well as state space modeling of missing observations and a state space model for unequally spaced time series are shown. The final section is a discussion of the information square root filter/smoother, that we use in linear Gaussian state space seasonal decomposition modeling in Chapter 9. Not necessarily linear - not necessarily Gaussian state space modeling is treated in Chapter 6. A variety of illustrative examples of linear state space modeling is shown in Chapter 7.
Genshiro Kitagawa, Will Gersch

6. General State Space Modeling

Abstract
Non-Gaussian state space modeling of time series and the more general, general state space modeling are treated in this chapter. They were introduced because of the need to model time series with abrupt discontinuities, and time series with outliers and to model time series whose state and or observation processes were nonlinear. The general state space model and its recursive formulas for prediction, filtering and smoothing are treated in Section 6.2 after an introduction in Section 6.1. Three alternative computational approaches for realizing the general state space modeling, numerical integration method, a Gaussian mixture-two filter formula method and the most recently developed Monte Carlo “particle-path tracing” method are discussed respectively in Sections 6.3, 6.4 and 6.5. In addition, starting from the general state space model, an alternative derivation of the Kalman filter is shown in Section 6.6. Applications of the non-Gaussian methods appear in Chapters 8 through 16. In those chapters where appropriate, the non-Gaussian and general state space methods performance are contrasted with the linear Gaussian state space analysis methods.
Genshiro Kitagawa, Will Gersch

7. Applications of Linear Gaussian State Space Modeling

Abstract
Some applications of the state space models that were described in Chapter 5 are presented in this chapter. In particular, the modeling of the famous Canadian lynx data by an AR state space model, the modeling of irregularly spaced data and an example of the decomposition of an observed time series into a signal, background noise and observation noise are shown.
Genshiro Kitagawa, Will Gersch

8. Modeling Trends

Abstract
In this chapter we consider the modeling of nonstationary mean time series, or trends, by state space methods. Initially we consider a Gaussian disturbances state space smoothness priors modeling that is appropriate when the trend is a smooth function. An example of the analysis of simulated data illustrates the method of analysis. Secondly we model simulated time series data with local trend and globally stochastic components. A generalization of the latter procedure is shown in the analysis of a collection of 22 years of daily maximum temperatures recorded in Tokyo. Evidence is shown to justify modeling this data with a common trend plus an individual annual AR processes model. The analysis of the Tiao and Tsay (1985, 1989) flour price data, another example of the analysis of multiple time series with common trend and individual AR process is also shown. On the basis of these examples, the common trend individual AR process modeling is offered as a candidate parsimonious model of multiple nonstationary mean time series.
Genshiro Kitagawa, Will Gersch

9. Seasonal Adjustment

Abstract
Time series with seasonal components is an important generalization of nonstationary mean time series. Such time series occur for example in meteorological, oceanographic and econometric studies. In this section we consider the modeling of nonstationary mean time series with trend and seasonal and perhaps other components, (such as globally stochastic AR and trading day components), by state space methods. As in the modeling of the simpler nonstationary mean trend alone time series, first we consider the modeling of time series with smooth trend and smooth seasonal components. Such time series can be well modeled by Gaussian process noise and Gaussian observation noise using the Kalman filter/smoother algorithms. As in the trend alone modeling, when we consider that there might be outliers or abrupt changes in either the trend or seasonal components, it will be necessary to model such time series by the more general non-Gaussian state space modeling methods. In the more general non-Gaussian state space trend plus seasonal modeling, state estimation is achieved here in two different way, using the Gaussian sum and Monte Carlo filter methods.
Genshiro Kitagawa, Will Gersch

10. Estimation of Time Varying Variance

Abstract
There are many time series whose structure involves a substantial change of variance. The decomposition of the seismic time series problem in Chapter 7 is such an example. In other practical data situations, the relatively fast wiggles of a nonstationary covariance time series appears to be modulated by a relatively slowly changing envelope function. For example, seismic measurements during an earthquake exhibit this behavior. That envelope function can be interpreted as a change of scale associated with the instantaneous innovations variance of the state space model of the time series. In this chapter the changing variance structure of the Urakawa-Oki, Hokkaido Japan March 21 1983, earthquake data, (code name MYN2F, Takanami 1991) is estimated by both Gaussian and non-Gaussian state space models. Additional applications include the change of variance modeling in the estimation of the log-periodogram of a time series and the estimation of the instantaneous variance of the collection of 21 years of daily maximum temperatures in Tokyo.
Genshiro Kitagawa, Will Gersch

11. Modeling Scalar Nonstationary Covariance Time Series

Abstract
In this chapter scalar nonstationary covariance time series are modeled using time varying coefficient autoregressive models, as in equation 11.1. In such a model for N observations, if the order of the AR model is m there will be N × m AR coefficient parameters and as many asNinnovations variance parameters. Fitting a time varying AR, (TVAR), model with smoothness priors constraints permits those parameters to be estimated implicitly in terms of only a small number of explicitly estimated hyperparameters. Several different mechanisms for implementing the smoothness priors constraints are possible. Our own approach to this important topic has evolved over several years, and two different methods for fitting the time varying AR coefficient model with smoothness priors constraints are shown here. (Each of the methods has implications for the modeling of multivariate nonstationary covariance data. That topic is treated in Chapter 12.) Our primary application for the scalar nonstationary covariance modeling is the evolution with time of the power spectrum. The estimated TVAR model yields what we refer to as an “instantaneous power spectral density”.
Genshiro Kitagawa, Will Gersch

12. Modeling Multivariate Nonstationary Covariance Time Series

Abstract
The modeling of multivariate nonstationary covariance time series is relevant to many econometric, human electroencephalogram, oceanographic, meteorological and seismic data situations. (See for example Brillinger and Hatanaka 1969, 1970.) In this chapter multivariate nonstationary covariance time series are modeled using time varying multivariate autoregressive or equivalently, time varying vector autoregressive, (TVVAR), models. This model is a natural extension of the scalar TVAR model shown in Chapter 11. Analogous to Chapter 11, two different modeling methods are presented. In one method, smoothness priors constraints are placed directly on the evolution of the individual AR parameters of the TVVAR model. In the other method, smoothness priors constraints are placed on the evolution of the PARCORs of AR models. The key idea which permits TVVAR modeling is a “one channel at-atime” paradigm. This idea is a consequence of an instantaneous response-orthogonal innovations representation of multivariate AR models. That permits the multivariate time series to be modeled one scalar AR model at-a-time.
Genshiro Kitagawa, Will Gersch

13. Modeling Inhomogeneous Discrete Processes

Abstract
In this chapter we illustrate the application of non-Gaussian modeling method to the analysis of inhomogeneous discrete random processes data. Two data analytic examples, Tokyo rainfall data, a nonstationary binary process and the analysis of a simulated nonstationary Poisson process data set are shown. The latter simulation is based on research of the count of X-rays from the star Cygnus Xl data. Kashiwagi and Yanagimoto (1992), is an application of the non-Gaussian state space methodology in the analysis of nonstationary Poisson process analysis of medical data.
Genshiro Kitagawa, Will Gersch

14. Quasi-Periodic Process Modeling

Abstract
The early development of time series has been related to a quest for an understanding of cyclical phenomena. For example, Shuster’s periodogram (1898, 1906), and Yule’s (1927) introduction of autoregressive models, were devoted to the analysis of cyclical sunspot numbers and Whittle’s (1954) analysis of the water level in a rock channel on the Wellington coast of New Zealand is also related to a cyclical phenomenon. In fact many time series exhibit cyclical behavior in the sense there appears to be an approximate repetition of a pattern with a not very well defined period or amplitude. Rather both the period and amplitude appear to change gradually. Typically ecological data, air pollution data and several other physical phenomena exhibit such behavior. Two of the most familiar examples of time series which exhibit such behavior and which have been extensively analyzed are the Canadian lynx data, (for example see Campbell and Walker 1977, Tong 1977, Bhansali 1979 and Priestly 1981), and the Wolfer sunspot series, (Morris 1977, Tong 1983). Such series are frequently modeled by AR, ARMA or AR plus sinusoidal models. However, none of these modeling methods are very satisfactory for the prediction of more than one lead time, (Tong 1982). As pointed out in Akaike (1977b) in his discussion of several analyses of the Canadian lynx data, those analyses were unconvincing and that the critical issue in modeling time series is the selection of a proper model.
Genshiro Kitagawa, Will Gersch

15. Nonlinear Smoothing

Abstract
A nonlinear state space approach to the smoothing of time series is developed. The time series is expressed in state space model form where the system model or the observation model contains nonlinear functions of the state vector. Recursive formulas for prediction, filtering and smoothing for the nonlinear state space model are shown. The performance of the method is illustrated by the analyses of both a one dimensional and a two dimensional example that have been previously considered in the literature. These examples require numerical approximations of the relevant densities and numerical computations for the nonlinear transformations of variables, the convolution of two densities, Bayes formula, and normalization. Results for the one dimensional example are compared and contrasted with those obtained by the extended Kalman filter method, by a second moment approximation method and also by the Monte Carlo filter method. An empirical study of the numerical accuracy is also shown. A more complete treatment of this material is in Kitagawa (1991).
Genshiro Kitagawa, Will Gersch

16. Other Applications

Abstract
In this chapter three additional applications of smoothness priors time series modeling are addressed which for a variety of reasons, were not included in other chapters. The first application is a study of the modeling of a very large data set, (500,000 observations), with missing data and outliers in a complex stochastic trend and regression on covariates modeling (Kitagawa and Matsumoto 1996). The objective of the analysis is to decompose the data into its component parts. The second application is a Markov state classification problem in which each observed state corresponds to a different time series process and the states are switched at random times. An illustrative analysis is done on simulated data. The third application involves an extension of the smoothness priors long AR model for spectral estimation in scalar stationary time series, (discussed in Chapter 4), to the multivariate case.
Genshiro Kitagawa, Will Gersch

Backmatter

Weitere Informationen