Dynamic Nonlinear Econometric Models

In the last twenty-five years considerable progress has been made in the theory of inference in nonlinear econometric models. A review of the literature up to the beginning of the 1980s is given in Burguete, Gallant and Souza (1982) and Amemiya (1983). For an account of related contributions in the statistics literature see, e.g., Humak (1983). The theory reviewed in these references assumes that the model is essentially static in nature and that the data generating process exhibits a certain degree of temporal homogeneity, e.g., some form of stationarity. Developments in recent years have focused on the extension of the theory to dynamic models, and in particular to situations where the data generating process can exhibit not only temporal dependence but also certain forms of temporal heterogeneity. As in the static case, this analysis also allows for model misspecification.

We start with a brief review of the basic structure of the classical estimation problem, which can be described as follows: The researcher observes a set of data assumed to be generated by a stochastic process. The probability law of this process is determined by a model. This “true” model is assumed to belong to a class of models where each model is indexed by a parameter. This parameter may either characterize the probability law of the stochastic process completely or only partially (e.g., it may only characterize the first and second moments). Apart from knowing that the true model belongs to the given model class, the value of the true parameter is not known. The parameter may be an element of a finite or infinite dimensional space. The estimation problem is then to infer the value of the true parameter (or of certain components of interest) on the basis of the observed data. Specific estimators are often derived from general principles such as the maximum likelihood principle or the method of moments. Given a particular estimator it is then of interest to analyze its performance.

In this chapter we describe the structure of the consistency proof for Mestimators in nonlinear econometric models as it has evolved from Jennrich (1969) and Malinvaud (1970). We shall refer to this proof as the classical consistency proof. The basic ideas date back to Doob (1934), Wald (1949) and LeCam (1953). The consistency proofs in the articles on asymptotic inference in nonlinear econometric models listed in Chapter 1 all share this common structure.

In this chapter we comment further on the classical consistency proof, discuss possible extensions and point out limitations of the theory.¹ In the following let \({R_n}\left( {\omega ,\beta } \right) = {Q_n}\left( {{z_1},...,{z_n},{{\hat \tau }_n},\beta } \right)\) be some objective function, let \({\hat \beta _n}\) be a corresponding M-estimator satisfying (2.1), and let \({\bar R_n}\left( \beta \right) = {\bar Q_n}\left( {{{\bar \tau }_n},\beta } \right)\).

As documented in Chapters 3 and 4 a basic ingredient for typical consistency proofs is that the difference between the objective function and its nonstochastic counterpart converges to zero uniformly over the parameter space (or, if the approach in Section 4.3 is followed, at least over a suitably chosen subset). In many cases uniform convergence of the objective function will follow from a uniform law of large numbers (ULLN), either directly or via Lemmata 3.2 and 3.3.

In this chapter we provide formalizations of the notion that a stochastic process has a “fading memory”. Some of these formalizations employ concepts of approximation of one process by another process. The aim of these formalizations is to define classes of processes that — while still satisfying limit theorems (LLNs and CLTs) — are broad and cover, in particular, processes that are generated from a dynamic system.¹ In Section 6.1 we start with a discussion of the limitations of the concept of α-mixing [Ø-mixing], followed by the definition of L _{p -}approximability of a stochastic process in Section 6.2. This approximation concept was introduced in Pötscher and Prucha (1991a). It encompasses the approximation concept of stochastic stability and near epoch dependence, and helps to clarify the relationship between these concepts. In Section 6.3 we then discuss LLNs for L _p -approximable and near epoch dependent processes. (The discussion of CLTs is deferred to Chapter 10.) Frequently we are interested in limit theorems for a function of an L _{p -}approximable [near epoch dependent] process. E.g., when proving consistency via the use of a ULLN, we need to establish local LLNs, i.e., LLNs for the “bracketing” functions \(q_t^*\left( {{z_t},\theta ;\eta } \right)and{q_{{t_*}}}\left( {{z_t},\theta ;\eta } \right)\). If the underlying process (z _t ) is L _p approximable [near epoch dependent] this can be accomplished by making use of results that show under which circumstances functions preserve the L _P approximability [near epoch dependence] property. Preservation results of this type are the subject of Section 6.4. In considering dynamic systems it is important to know when the process generated by the system will satisfy the L _P approximability [near epoch dependence] property. Hence, in Section 6.4 we also provide sufficient conditions for dynamic systems under which the output process is L _p approximable [near epoch dependent]. Since limit theorems for Lp approximable [near epoch dependent] processes are available (cf. Section 6.3 and Chapter 10), such results are fundamental for the derivation of limit theorems for (functions of) processes that are generated by dynamic systems. Several of these results, and in particular those that cover higher order systems, are new and have not been available in the literature previously. Finally, in Section 6.5 we utilize the results developed in this chapter to give sets of sufficient conditions which ensure that q _t (z _t , θ) satisfies a local LLN, i.e., we provide sufficient conditions for Assumption 5.2 in Chapter 5.

In the previous chapters we have discussed the basic structure of the classical consistency proof for M-estimators and have established basic modules that can be employed for consistency proofs in dynamic nonlinear models. Various catalogues of assumptions that imply the consistency of M-estimators in dynamic nonlinear models can be obtained by combining respective modules. In the following we illustrate this by specifying two alternative catalogues of assumptions for the consistency of a general class of M-estimators, which includes least mean distance and generalized method of moments estimators. For a further illustrative application of the respective modules see Chapter 14, which contains a derivation of the asymptotic properties of the (quasi) NFIML estimator of a dynamic implicit nonlinear simultaneous equation system.

In this chapter we describe the basic structure underlying the derivation of the asymptotic distribution of M-estimators in nonlinear econometric models. As remarked in Chapter 1, the basic methods used in this derivation date back to Doob (1934), Cramér (1946), LeCam (1953), Huber (1967) and Jennrich (1969), to mention a few; for a more extensive bibliography see Norden (1972, 1973) and the references in Chapter 1. The asymptotic normality proofs in the articles on nonlinear econometric models listed in Chapter 1 all share this common structure. The basic idea is to express the estimator as a linear function of the score vector by means of a Taylor series expansion and then to derive the asymptotic distribution of the estimator from the asymptotic distribution of the score vector.

The standard approach for deriving the asymptotic distribution of M-estimators outlined in the previous chapter relies on the assumption that the objective function Q _n is twice continuously differentiable w.r.t. both the parameter of interest β and the nuisance parameter τ. (Or, if the estimator \({\hat \beta _n}\) is derived as an approximate solution of a set of estimating equations F _n = 0, it is maintained that F _n is continuously differentiable.) In a number of applications this smoothness assumption is too stringent. E.g., if the objective function corresponds to the least absolute deviation estimator or Huber’s M-estimator this assumption is violated. Also in a semiparametric context, where τ represents an infinite dimensional nuisance parameter that varies in a metric space T which is not a subset of Euclidean space, the notion of differentiability w.r.t. τ may not be available, although Q _n may be smooth as a function of β for every given value of τ. Such situations can often be handled by a refinement of the argument underlying Lemma 8.1. The basic idea is again to show that the (normalized) estimator \({\hat \beta _n}\) is asymptotically equivalent to a linear transformation of the score vector evaluated at the true parameter, and then to invoke a CLT for the score vector. Of course, in the absence of the smoothness assumptions of Chapter 8, establishing such a linear transformation is now more delicate (and, if Q _n is not differentiable at all, special care has to be given to defining the notion of a score vector properly). The linearization is frequently attempted by showing that the objective function can — in a certain sense — be replaced by its asymptotic counterpart \({\bar Q_n}\) and by exploiting the usually greater degree of smoothness of the latter function in the linearization argument. The fact that \({\bar Q_n}\) is frequently a smooth function, even when Q _n is not, originates from the fact that \({\bar Q_n}\) is frequently equal to EQ _n or lim _{n →∞} EQ _n and taking expectations is a smoothing operation. This approach was pioneered by Daniels (1961) and Huber (1967). For a modern exposition see Pollard (1985). The following discussion will be informal.

A key ingredient for the asymptotic normality proof, as outlined in Chapter 8, is that the normalized score vector can be expressed as a linear function of random variables ζ _n which converge in distribution, cf. Assumption 8.1(g). In this chapter we present central limit theorems (CLTs) which can be used to imply this distributional convergence of ζ _n in the important case where ζ _n , can be expressed as a normalized sum of random variables. We give two alternative CLTs.

Based on the discussion in Chapters 8 and 10 it is now possible to provide various sets of sufficient conditions for the asymptotic normality of M-estimators in dynamic nonlinear models. In Sections 11.1 and 11.2 we establish the asymptotic normality of least mean distance estimators and of generalized method of moments estimators under exemplary catalogues of assumptions. In Section 11.3 we relate these results to those available in the econometrics literature and provide further remarks.

Inspection of the asymptotic normality results for least mean distance and generalized method of moments estimators given in, e.g., Theorems 11.2(a) and 11.5(a) shows that in both cases a matrix of the form \(C_n^{ - 1}{D_n}{D'_n}C_n^{ - 1'}\) acts as an asymptotic variance covariance matrix of \({n^{1/2}}\left( {{{\hat \beta }_n} - {{\hat \beta }_n}} \right)\), where C _n and D _n are given in those theorems. For purposes of inference we need estimators of C _n and D _n . Inspection of the matrices C _n reveals that these matrices are essentially composed of terms of the form \({n^{ - 1}}\sum\nolimits_{t = 1}^n {E{w_{t,n}}} \), where \({w_{t,n}} = {w_t}\left( {{{\bar \tau }_n},{{\bar \beta }_n}} \right)\) equals \({\nabla _{\beta \beta qt}}\left( {{z_t},{{\bar \tau }_n},{{\bar \beta }_n}} \right)\) in the case of least mean distance estimators or equals \({\nabla _{\beta qt}}\left( {{z_t},{{\bar \tau }_n},{{\bar \beta }_n}} \right)\) in the case of generalized method of moments estimators. The matrices D _n — apart from containing similar terms — also contain an expression of the form \({n^{ - 1}}E\left[ {\left( {\sum\limits_{t = 1}^n {{V_{t,n}}} } \right)\left( {\sum\limits_{t = 1}^n {{{v'}_{i,n}}} } \right)} \right],\) where equals \(\nabla \beta 'qt\left( {{z_t},{{\overline \tau }_n},{{\overline \beta }_n}} \right)\) in the case of least mean distance estimators or equals \({q_t}\left( {{z_t},{{\bar \tau }_n},{{\bar \beta }_n}} \right)\) in the case of generalized method of moments estimators. The expressions of the form \({n^{ - 1}}\sum\nolimits_{t = 1}^n {E{w_{t,n}}} \) will typically be estimated by \({n^{ - 1}}\sum\nolimits_{t = 1}^n {{{\hat w}_{t,n}}} \) where \({\hat w_{t,n}} = {w_t}\left( {{{\hat \tau }_n},{{\hat \beta }_n}} \right)\). Consistency of such estimators can be derived from ULLNs and from consistency of \(\left( {{{\hat \tau }_n},{{\hat \beta }_n}} \right)\) in a rather straightforward manner via Lemma 3.2.¹ The estimation of \({\Psi _n} = {n^{ - 1}}\sum\limits_{t = 1}^n {E{v_{t,n}}{{v'}_{t,n}}j = \sum\limits_{j = 1}^{n - 1} {{n^{ - 1}}} \sum\limits_{t = 1}^{n - j} {\left[ {E{v_{t,n}}{{v'}_{t + j,n}} + E{v_{t = j,n}}v't,n} \right]} } \) reduces to a similar problem in the important special case where (v_t,_n) is a martingale difference array (or, more generally is uncorrelated and has mean zero), since then the expression for Ψ _n reduces to \({n^{ - 1}}\sum\nolimits_{t = 1}^n {E{v_{t,n}}{{v'}_{t,n}}} \). As discussed above (v _t,n ) will have a martingale difference structure in certain correctly specified cases. In the general case, however, (v _t,n ) will typically be autocorrelated (with autocorrelation of unknown form) and hence the estimation of Ψ _n is more involved.

In this chapter we provide consistency results for the estimation of the variance covariance matrices of least mean distance and generalized method of moments estimators as given in parts (a) of Theorems 11.2 and 11.5. Consistency results for the estimation of the variance covariance matrices as given in parts (b) of those theorems can be obtained analogously.

This book provides an asymptotic theory for M-estimators in the context of dynamic nonlinear models. To accommodate processes generated by dynamic nonlinear models the theory has to allow for temporal dependence and (possibly also) for temporal heterogeneity in the data generating process. This is achieved by employing weak dependence concepts like L _p -approximability or near epoch dependence, which are flexible enough to cover processes generated by dynamic nonlinear models, yet are strong enough to permit the derivation of laws of large numbers and central limit theorems for such processes.