Selected Papers of Hirotugu Akaike

We have many occasions to analyse the pattern of occurrences of serial random events, for example, automobile flows, dropping ends of cocoon filaments. Many works have been done for the case when the lengthes of gaps independently follow one and the same negative exponential distribution. In this case, as is well known, the number of occurrences in one time interval is entirely independent of that in another time interval which does not overlap the former, and follows the Poisson distribution. Nevertheless, we sometimes observe random events concerning which the numbers of occurrences in some disjoint time intervals are not entirely independent, that is, they show wave-like movement. In some cases, this is due to the fact that the distribution of gaps are not necessarily of negative exponential type. In this paper we shall treat a discrete parameter processes for the purpose to analyse practically the processes of the type just stated.

In this paper we define a type of transformation of probability distribution and analyze the limiting behavior of the result of successive applications of the transformation to some initial probability distribution. By using the results of this analysis we can get a fairly general insight into the so-called optimum-gradient method in numerical analysis. We can prove the conjecture which was stated by Forsythe and Motzkin [7] and was used as the logical basis of an acceleration procedure for the optimum gradient method [4][5][6]. It was stated by Forsythe [4] that this conjecture seems to be hard to prove as the related transformation is rather complicated. But our present proof is rather simple. Further, we can see the relation between the condition-number of the related matrix and the convergence rate of the optimum gradient method. By using the relation which according to [5] is first proved by Kantrovich [8], we can say that when the matrix is ill-conditioned the convergence rate tends near to its worst possible value. Using the same data as those treated by Forsythes in paper [5], we give some numerical examples.

Nowadays it is very common to use time-sampled data to analyse or control an object fluctuating continuously in time. This seems to be motivated by the recent developments of digital methods serving for these purposes. In relation to such procedures there are many papers which treat the noise or error due to quantization ([4], [5]). The effect of time-sampling on the spectral properties is also well-known as folding or aliasing for the case where the fluctuation of object is represented by a stationary stochastic process and the timings are performed without error. As to the cases where timing-errors are present we have seen yet little quantitative description of their effects on the spectral properties of the time-sampled data [3]. In the present paper we treat this problem for the case where the timing is independent of the original process and the intervals between sampling-time points form a stationary process. After the general discussion of this case we treat two special types of time-sampling in more details. The first corresponds to the case where, though it is intended to sample the record at the time points which are the integral multiples of a constant time Δt the deviations of the sampling-time points from the preassigned ones are present and form a purely random process, i.e., they are random variables which are mutually independent and follow one and the same probability distribution.

In the recent papers in which the results of the spectral analyses of roughnesses of runways or roadways are reported [1, 2, 3,4] the power spectral densities of approximately of the form f-2 (f: frequency) are often treated. This fact directed the present author to the investigation of the limiting process which will provide the f-2 form under fairly general assumptions. In this paper a very simple model is given which explains a way how the f-2 form is obtained asymptotically. Our fundamental model is that the stochastic process, which might be considered to represent the roughness of the runway, is obtained by alternative repetitions of roughening and smoothing. We can easily get the limiting form of the spectrum for this model. Further, by taking into account the physical meaning of roughening and smoothing we can formulate the conditions under which this general result assures that the f-2 form will eventually take place.

At present, the spectral method is used very commonly for the analysis.of an electrical or mechanical system. The spectral method is used not only for the estimation of the individual spectral density functions of the input and output of the system but also for the estimation of the frequency characteristics of the system.

A basic linear model of stationary stochastic processes is proposed for the analysis of linear feedback systems. The model suggests a simple computational procedure which gives estimates of the response characteristics of the system and the spectra of the noise source. These estimates are obtained through the estimate of the linear predictor of the process, which is obtained by the ordinary least squares method.The necessary assumption for the validity of the estimation procedure is so general that the procedure can be applied to the analysis of wide variety of practical systems with feedback.The content of the present paper forms an answer to the problem discussed by the author in a former paper [1].

This is a preliminary report on a newly developed simple and practical procedure of statistical identification of predictors by using autoregressive models. The use of autoregressive representation of a stationary time series (or the innovations approach) in the analysis of time series has recently been attracting attentions of many research workers and it is expected that this time domain approach will give answers to many problems, such as the identification of noisy feedback systems, which could not be solved by the direct application of frequency domain approach [1], [2], [3], [9].

In a recent paper by the present author [1] a simple practical procedure of predictor identification has been proposed. It is the purpose of this paper to provide a theoretical and empirical basis of the procedure.

The use of a multidimensional extension of the minimum final prediction error (FPE) criterion which was originally developed for the decision of the order of one-dimensional autoregressive process [1] is discussed from the standpoint of controller design. It is shown by numerical examples that the criterion will also be useful for the decision of inclusion or exclusion of a variable into the model. Practical utility of the procedure was verified in the real controller design process of cement rotary kilns.

A fully computerized cement rotary kiln process control was tested in a real production line and the results are presented in this paper. The controller design was based on the understanding of the process behavior obtained by careful statistical analyses, and it was realized by using a very efficient statistical identification procedure and the orthodox optimal controller design by the statespace method. All phases of analysis, design and adjustment during the practical application are discussed in detail. Technical impact of the success of the control on the overall kiln installation is also discussed. The computational procedure for the identification is described in an Appendix.

The use of a multivariate autoregressive model for the implementation of a new practical optimal control of a supercritical thermal power plant is discussed. The control is realized by identifying the system characteristics of the plant under the conventional PID control by the autoregressive model fitting and then implementing the digital control to correct the defect of the analog control. The procedure of identification and the controller implementation is described in detail by using the experimental results of a real plant. The results clearly demonstrate the advantage of the new controller over the conventional PID controller. The experience of the commercial operation of the plant Confirms that the new controller is extremely robust against the gradual change of the plant characteristics, and this shows the practical utility of the identification procedure on which the design of the controller is based.

In this paper it is shown that the classical maximum likelihood principle can be considered to be a method of asymptotic realization of an optimum estimate with respect to a very general information theoretic criterion. This observation shows an extension of the principle to provide answers to many practical problems of statistical model fitting.

The history of the development of statistical hypothesis testing in time series analysis is reviewed briefly and it is pointed out that the hypothesis testing procedure is not adequately defined as the procedure for statistical model identification. The classical maximum likelihood estimation procedure is reviewed and a new estimate minimum information theoretical criterion (AIC) estimate (MAICE) which is designed for the purpose of statistical identification is introduced. When there are several competing models the MAICE is defined by the model and the maximum likelihood estimates of the parameters which give the minimum of AIC defined by AIC = (−2)log- (maximum likelihood) + 2(number of independently adjusted parameters within the model). MAICE provides a versatile procedure for statistical model identification which is free from the ambiguities inherent in the application of conventional hypothesis testing procedure. The practical utility of MAICE in time series analysis is demonstrated with some numerical examples.

The problem of identifiability of a multivariate autoregressive moving average process is considered and a complete solution is obtained by using the Markovian representation of the process. The maximum likelihood procedure for the fitting of the Markovian representation is discussed. A practical procedure for finding an initial guess of the representation is introduced and its feasibility is demonstrated with numerical examples.

A recursive procedure for the computation of one-step ahead predictions for a finite span of time series data by a Gaussian autoregressive moving average model can be realized by using the Markovian representation of the model. The covariance matrix of the stationary state variable of the Markovian representation is required to implement a computational procedure of the predictions. A simple computational procedure of the covariance matrix which does not need an iterative method is obtained by using a canonical representation of the autoregressive moving average process. The recursive computation of the predictions realized by using this procedure provides a computationary efficient method of exact likelihood evaluation of a Gaussian autoregressive moving average model.

The purpose of the present paper is to propose a simple but practically useful procedure for the analysis of multidimensional contingency tables of survey data. By the procedure we can determine the predictor on which a specific variable has the strongest dependence and also the optimal combination of predictors. The procedure is very simply realized by the search for the minimum of the statistic AIC within a set of models proposed in this paper. The practical utility of the procedure is demonstrated by the results of some successful applications to the analysis of the survey data of the Japanese national character. The difference between the present procedure and the conventional test procedure is briefly discussed.

A flexible family of parametric models for intensity processes is introduced to represent a causal relationship between a point process and another stochastic process. Algorithms for the maximum likelihood computation and the procedure of model selection are discussed.

By using a simple example a minimax type optimality of the minimum AIC procedure for the selection of models is demonstrated.

In developing an estimate of the distribution of a future observation it becomes natural and necessary to consider a distribution over the space of parameters. This justifies the use of Bayes procedures in statistical inference. An objective procedure of evaluation of the prior distribution in a Bayesian model is developed and the classical ignorance prior distribution is newly interpreted as the locally impartial prior distribution.

The conventional approach to parametric model fitting of time series is realized through the comparison of various competing models by some ad hoc criterion. Since each of the models is usually specified by the parameters determined by the information from the data, the extension of the classical concept of likelihood to this situation is not obvious. By asking the log likelihood of a model to be an unbiased estimate of the expected log likelihood of the model, a reasonable definition of the likelihood is obtained and this allows us to develop a systematic approach to parametric time series modelling. Practical utility of this approach is demonstrated by numerical examples.

In this paper the likelihood function is considered to be the primary source of the objectivity of a Bayesian method. The necessity of using the expected behavior of the likelihood function for the choice of the prior distribution is emphasized. Numerical examples, including seasonal adjustment of time series, are given to illustrate the practical utility of the common-sense approach to Bayesian statistics proposed in this paper.

The basic ideas underlying the construction of a newly introduced seasonal adjustment procedure by a Bayesian modeling are discussed in detail. Particular emphasis is placed on the use of the concept of the likelihood of a Bayesian model for model selection. The performance of the procedure is illustrated by a numerical example.

A quasi Bayesian procedure is developed for the detection of outliers. A particular Gaussian distribution with ordered means is assumed as the basic model of the data distribution. By introducing a definition of the likelihood of a model whose parameters are determined by the method of maximum likelihood, the posterior probability of the model is obtained for a particular choice of the prior probability distribution. Numerical examples are given to illustrate the practical utility of the procedure.

By using the direct and inverse binomial experiments it is shown that there is a situation where Birnbaum’s basic axiom of mathematical equivalence and the likelihood principle is a tautology. This observation disqualifies Birnbaum’s proof of the likelihood principle based on the axioms of mathematical equivalence and conditionality. The implication of this disproof of Birnbaum’s argument for Bayesian statistics is briefly discussed.

A new Bayesian method for tidal analysis is proposed. In contrast with the conventional filtering approach, this method is based on a time domain model which includes the response of the Earth to theoretical tidal input and other associated meteorological variables. It also includes the term which represents the drift of the record.The basic assumption of this new procedure is the smoothness of the drift, and this requirement is represented in the form of probability of the Bayesian model. The parameters of the model are given as the mean of the posterior distribution defined by the data distribution and the prior distribution of the parameters.The method allows an objective decision on the choice of the lag of the response functions and of the grouping of the tidal waves as a problem of statistical model selection. The problem of missing observation data and unexpected steps of the drift can be easily handled.Practical applicability of this model to the analysis of earth tide records is demonstrated by using synthetic and actual earth tides data.

The information criterion AIC was introduced to extend the method of maximum likelihood to the multimodel situation. It was obtained by relating the successful experience of the order determination of an autoregressive model to the determination of the number of factors in the maximum likelihood factor analysis. The use of the AIC criterion in the factor analysis is particularly interesting when it is viewed as the choice of a Bayesian model. This observation shows that the area of application of AIC can be much wider than the conventional i.i.d. type models on which the original derivation of the criterion was based. The observation of the Bayesian structure of the factor analysis model leads us to the handling of the problem of improper solution by introducing a natural prior distribution of factor loadings.

The emergence of the magic number 2 in recent statistical literature is explained by adopting the predictive point of view of statistics with entropy as the basic criterion of the goodness of a fitted model. The historical development of the concept of entropy is reviewed, and its relation to statistics is explained by examples. The importance of the entropy maximization principle as a basis of the unification of conventional and Bayesian statistics is discussed.

The development of statistical models is realized through the accumulation of successful experiences of the analysis of real data. As is discussed in Akaike (1992) statistical modeling activity contains highly subjective or personal aspect which Polanyi (1962) related to the scientific talent of the researcher. However, scientific activity is never isolated from the society and Turing (1969) explicitly characterized the search for new techniques as the ‘ cultural search ’. This paper is intended to provide the background information of some of the experiences of the author on time series modeling. It is hoped that the description of the interaction between the author and his environment will provide some suggestion for those who intend to organize effective statistical modeling activity in the future.

Informational approach represents a new trend in the development of statistical science. This paper is intended for the discussion of the informational outlook in relation to the development of statistics or statistical science.