Testing for zero inflation and overdispersion in INAR(1) models

Abstract

The marginal distribution of count data processes rarely follows a simple Poisson model in practice. Instead, one commonly observes deviations such as overdispersion or zero inflation. To express the extend of such deviations from a Poisson model, one can compute an appropriately defined dispersion index or zero index. In this article, we develop several tests based on such indexes, including joint tests being based on an index combination. The asymptotic distribution of the resulting test statistics under the null hypothesis of a Poisson INAR(1) model is derived, and the finite-sample performance of the resulting tests is analyzed. Real data examples illustrate the application of these tests in practice.

Appendices

Appendix 1: The ZIB-ZIP-INAR(1) model for zero-inflation

The basic INAR(1) model from Definition 1 has been generalized in several ways by replacing the binomial thinning operator by another type of thinning operation, see the surveys by Weiß (2008) and Scotto et al. (2015). To inflate the number of zeros being observed for a Poisson INAR(1) process, we combine the idea of using zero-inflated innovations (Jazi et al. 2012) with the idea of using an appropriately modified type of thinning operation: the zero-inflated binomial thinning (ZIB thinning) operator. It is defined by a conditional ZIB-distribution (instead of a conditional binomial distribution) via

$$\begin{aligned} \textstyle \alpha \circ _{\omega } N\sim \mathrm{ZIB}\big (N,\frac{\alpha }{1-\omega },\omega \big ), \quad \text {where}\; 0\le \omega <1-\alpha . \end{aligned}$$

(17)

The latter guarantees that $\frac{\alpha }{1-\omega }<1$. Note that $\omega =0$ just corresponds to the usual binomial thinning operator, while negative values for $\omega$ (and hence some kind of “zero-deflated binomial thinning”) are impossible as long as N is allowed to take arbitrarily large values from $\mathbb N_0$. The parametrization in (17) is chosen such that ${\mathbb {E}}[\alpha \circ _{\omega } N\,|\,N]$ still equals $\alpha \cdot N$, while $\mathrm{Var}[\alpha \circ _{\omega } N\,|\,N]=\alpha (1-\frac{\alpha }{1-\omega })\,N + \alpha ^2\,\frac{\omega }{1-\omega }\,N^2$.

It should be noted that ZIB-thinning is just that instance of random coefficient thinning (Zheng et al. 2007), where the thinning parameter equals 0 with probability $\omega$, and $\frac{\alpha }{1-\omega }$ with probability $1-\omega$. The same operator (and, hence, also the same model as below) was recently also considered by Nastić et al. (2017), although not motivated by the issue of zero inflation but by obtaining some kind of “dependent binomial thinning”. Therefore, these authors refer to the operator ‘$\circ _{\omega }$’ as the “alternative generalized binomial thinning operator”, but we shall prefer the name “ZIB thinning” here.

The modified type of INAR(1) model to be considered for our power analyses in Sect. 4, referred to as the ZIB-ZIP-INAR(1) model, is now defined by the recursion

$$\begin{aligned} \begin{array}{ll} &{}X_t\ =\ \alpha \circ _{\omega } X_{t-1} + \varepsilon _t \quad \text {with i.i.d. } \varepsilon _t\sim \mathrm{ZIP}\big (\frac{\mu (1-\alpha )}{1-\omega },\omega \big ), \\ &{} \text {i.e., with } \mu _{\varepsilon }=\mu (1-\alpha ),\quad \sigma _{\varepsilon }^2=\mu _{\varepsilon }\,\big (1+\mu _{\varepsilon }\, \frac{\omega }{1-\omega }\big ). \end{array} \end{aligned}$$

(18)

This differs from Nastić et al. (2017), who mainly focus on the case of a geometric marginal distribution. It immediately follows, also see (Zheng et al. 2007; Nastić et al. 2017), that ${\mathbb {E}}[X_t]=\mu$ and $\rho (k)=\alpha ^k$, like for the usual INAR(1) model, while

$$\begin{aligned} \sigma ^2\ =\ \frac{\mu \,\alpha \,(1-\frac{\alpha }{1-\omega }) + \sigma _{\varepsilon }^2 + \mu ^2\,\alpha ^2\,\frac{\omega }{1-\omega }}{1-\frac{\alpha ^2}{1-\omega }}. \end{aligned}$$

The transition probabilities are computed as

$$\begin{aligned} p_{i|j}\ =\ \sum _{m=0}^{\min {\{i,j\}}} \underbrace{P(\alpha \circ _{\omega } X_{t-1}=m\ |\ X_{t-1}=j)}_{\mathrm{ZIB}\big (j,\frac{\alpha }{1-\omega },\omega \big )}\cdot \underbrace{P(\varepsilon _t=i-m)}_{\mathrm{ZIP}\big (\frac{\mu (1-\alpha )}{1-\omega },\omega \big )}. \end{aligned}$$

Further marginal properties like the zero probability $p_0$ can be computed numerically by utilizing the Markov property: choosing M sufficiently large and defining $\tilde{{\mathbf {P}}}:=(p_{i|j})_{i,j=0,\ldots ,M}$, the marginal probabilities $(p_0,\ldots ,p_M)^{\top }$ are approximated by the solution of the eigenvalue problem $\tilde{{\mathbf {P}}}\,\tilde{{\varvec{p}}}=\tilde{{\varvec{p}}}$.

Appendix 2: Proofs

1.1 Appendix 2.1: Proof of Theorem 3.1.1

For the computation of the joint moments the cases $k=0$ and $k>0$ have to be considered. As we will see, the joint moment formula for $k>0$ also holds for $k=0$, when calculating $\sigma _{11}, \sigma _{12}, \sigma _{13}$.

For $\sigma _{11}$, we compute

$$\begin{aligned} {\mathbb {E}}[Z_{t,1} \cdot Z_{t-k,1}]&\ =\ {\mathbb {E}}[ \mathbb {1}_{\{X_t=0\}}\cdot \mathbb {1}_{\{X_{t-k}=0\}}] - p_0^2\ =\ P(X_t = X_{t-k} = 0) - p_0^2 \\&\ =\ p_{0|0}^{(k)} \cdot p_0 - p_0^2\ =\ p_0\big (e^{-\mu (1-\alpha ^k)}-p_0\big )\ =\ e^{-2\mu }(e^{\mu \alpha ^k}-1), \\ {\mathbb {E}}[Z_{t,1}^2]&\ =\ P(X_t=0) - p_0^2\ =\ p_0(1-p_0)\ =\ e^{-\mu }(1-e^{-\mu }). \end{aligned}$$

So it follows that

$$\begin{aligned} \sigma _{11}= & {} e^{-\mu }(1-e^{-\mu }) + 2e^{-2\mu } \sum _{k=1}^\infty \big (e^{\mu \alpha ^k}-1\big ) \\= & {} e^{-\mu }(1-e^{-\mu }) + 2e^{-2\mu } \sum _{k=1}^\infty \sum _{j=1}^\infty \frac{\mu ^j\alpha ^{kj}}{j!}\\= & {} e^{-\mu }(1-e^{-\mu }) + 2e^{-2\mu } \sum _{j=1}^\infty \frac{\mu ^j}{j!} \sum _{k=1}^\infty (\alpha ^j)^k\\= & {} e^{-\mu }(1-e^{-\mu }) + 2e^{-2\mu } \sum _{j=1}^\infty \frac{\mu ^j}{j!} \frac{\alpha ^j}{1-\alpha ^j}. \end{aligned}$$

For $\sigma _{12}$, it follows that

$$\begin{aligned} {\mathbb {E}}[Z_{t,2} \cdot Z_{t-k,1}]&\ =\ {\mathbb {E}}[X_t \cdot \mathbb {1}_{\{X_{t-k}=0\}}] - p_0\mu \ =\ p_0 \cdot {\mathbb {E}}[X_t|X_{t-k}=0] - p_0\mu \\&\ =\ p_0 \big (\alpha ^k \cdot 0 + \mu (1-\alpha ^k)\big ) - p_0\mu \ =\ -p_0\mu \alpha ^k = -\mu e^{-\mu }\alpha ^k, \\ {\mathbb {E}}[Z_{t,1} \cdot Z_{t,2}]&\ =\ -\mu e^{-\mu }. \end{aligned}$$

Hence,

$$\begin{aligned} \sigma _{12}\ =\ -\mu e^{-\mu } - 2\mu e^{-\mu } \sum _{k=1}^\infty \alpha ^k \ =\ -\mu e^{-\mu } - 2\mu e^{-\mu }\frac{\alpha }{1-\alpha } \ =\ -\mu e^{-\mu }\,\frac{1+\alpha }{1-\alpha }. \end{aligned}$$

Note that if the process would not be t.r., we would have to compute ${\mathbb {E}}[Z_{t,1} \cdot Z_{t-k,2}]$ separately.

Before computing $\sigma _{13}$, we derive

$$\begin{aligned} {\mathbb {E}}[X_t^2|X_{t-k}=x]&= \mathrm{Var}[X_t|X_{t-k}=x]+{\mathbb {E}}^2[X_t|X_{t-k}=x] \\&\mathop {=}\limits ^{(7)} x\alpha ^k(1-\alpha ^k)+\mu (1-\alpha ^k)+x^2\alpha ^{2k}+2\mu x\alpha ^k(1-\alpha ^k)\\&\quad +\mu ^2(1-\alpha ^k)^2 \\&\ =\ x^2\alpha ^{2k}+(1+2\mu )x\alpha ^k(1-\alpha ^k)+\mu (1-\alpha ^k)(1+\mu (1-\alpha ^k)). \end{aligned}$$

So it follows that

$$\begin{aligned} {\mathbb {E}}[Z_{t,3} \cdot Z_{t-k,1}]&\ =\ {\mathbb {E}}\left[ X_t^2 \cdot \mathbb {1}_{\{X_{t-k}=0\}}\right] - p_0\mu (1+\mu )\\&\ =\ p_0 \cdot {\mathbb {E}}\left[ X_t^2|X_{t-k}=0\right] - p_0\mu (1+\mu ) \\&\ =\ p_0\mu \underbrace{(1-\alpha ^k)\big (1+\mu (1-\alpha ^k)\big )}_{=\ 1+\mu -\alpha ^k(1+2\mu )+\mu \alpha ^{2k}} - p_0\mu (1+\mu ) \\&\ =\ -e^{-\mu }\mu (1+2\mu )\alpha ^k + e^{-\mu }\mu ^2\alpha ^{2k}, \\ {\mathbb {E}}[Z_{t,1} \cdot Z_{t,3}]&\ =\ {\mathbb {E}}\left[ \underbrace{\mathbb {1}_{\{X_t=0\}}\cdot X_t^2}_{\equiv 0}\right] - p_0\mu (1+\mu )\\&\ =\ - p_0\mu (1+\mu ) = - e^{-\mu }\mu (1+\mu ). \end{aligned}$$

$$\begin{aligned} \sigma _{13}&\ =\ -e^{-\mu }\mu (1+\mu )-2e^{-\mu }\mu (1+2\mu ) \sum _{k=1}^\infty \alpha ^k+2e^{-\mu }\mu ^2 \sum _{k=1}^\infty \alpha ^{2k}\\&\ =\ -e^{-\mu }\mu (1+2\mu )+e^{-\mu }\mu ^2 - 2e^{-\mu }\mu (1+2\mu )\frac{\alpha }{1-\alpha } + 2e^{-\mu }\mu ^2\frac{\alpha ^2}{1-\alpha ^2}\\&\ =\ -\mu e^{-\mu }(1+2\mu ) \frac{1+\alpha }{1-\alpha }+\mu ^2 e^{-\mu } \frac{1+\alpha ^2}{1-\alpha ^2}. \end{aligned}$$

The remaining entries of ${\varvec{\Sigma }}$ have already been calculated in Lemma A.5.1 in Schweer and Weiß (2014), so the proof of Theorem 3.1.1 is complete.

1.2 Appendix 2.2: Proof of Theorem 3.2.1

We apply the Delta method to Theorem 3.1.1 and the function ${\varvec{g}}:\mathbb {R}^3\rightarrow \mathbb {R}^2$ with components

$$\begin{aligned} g_1(x_1,x_2,x_3)\ :=\ f_{\mathrm{z}}(x_1,x_2),\quad g_2(x_1,x_2,x_3)\ :=\ f_{d }(x_2,x_3), \end{aligned}$$

(19)

where the dispersion function $f_{d }$ satisfies $f_{d }(\mu , \mu (1+\mu ))=1$ for all $\mu$. The Jacobian of ${\varvec{g}}$, evaluated at $(e^{-\mu }, \mu , \mu (1+\mu ))$, then has the form

$$\begin{aligned} {\mathbf {D}}\ =\ \begin{pmatrix} d_{11} &{}\quad d_{12} &{}\quad 0\\ 0 &{}\quad d_{22} &{}\quad d_{23}\\ \end{pmatrix}. \end{aligned}$$

Hence, we obtain

$$\begin{aligned} {\mathbf {D}}{\varvec{\Sigma }}{\mathbf {D}}^{\top }\ =\ \begin{pmatrix} d_{11} &{}\quad d_{12} &{}\quad 0\\ 0 &{}\quad d_{22} &{}\quad d_{23}\\ \end{pmatrix} \begin{pmatrix} \sigma _{11} &{}\quad \sigma _{12} &{} \quad \sigma _{13} \\ \sigma _{12} &{}\quad \sigma _{22} &{}\quad \sigma _{23} \\ \sigma _{13} &{}\quad \sigma _{23} &{}\quad \sigma _{33} \\ \end{pmatrix} \begin{pmatrix} d_{11} &{}\quad 0\\ d_{12} &{}\quad d_{22} \\ 0 &{}\quad d_{23}\\ \end{pmatrix} \ =:\ \begin{pmatrix} A &{}\quad B \\ B &{}\quad C \\ \end{pmatrix}, \end{aligned}$$

where

$$\begin{aligned} A&\ =\ d_{11}^2\, \sigma _{11} +2 d_{11} d_{12}\, \sigma _{12} +d_{12}^2\, \sigma _{22}, \\ B&\ =\ d_{11} d_{22}\, \sigma _{12} +d_{11} d_{23}\, \sigma _{13} + d_{12} d_{22}\, \sigma _{22} +d_{12} d_{23}\, \sigma _{23}, \\ C&\ =\ d_{22}^2\, \sigma _{22} +2 d_{22} d_{23}\, \sigma _{23} +d_{23}^2\, \sigma _{33}. \end{aligned}$$

Considering that all $\sigma _{ij}$ from Theorem 3.1.1 except $\sigma _{11}$ have a similar structure, we can further simplify

$$\begin{aligned} A&\ =\ d_{11}^2\, \sigma _{11} +d_{12}\,(d_{12} -2 e^{-\mu }\, d_{11})\,\mu \,\frac{1+\alpha }{1-\alpha },\\ B&\ =\ d_{11} d_{23}\, \mu ^2 e^{-\mu }\, \frac{1+\alpha ^2}{1-\alpha ^2} \\&\qquad +\ \Big (d_{12} d_{22} - e^{-\mu }\, d_{11} d_{22} + (1+2\mu )\, d_{12} d_{23} - (1+2\mu )\, e^{-\mu }\, d_{11} d_{23}\Big )\,\mu \,\frac{1+\alpha }{1-\alpha } \\&\ =\ d_{11} d_{23}\, \mu ^2 e^{-\mu }\, \frac{1+\alpha ^2}{1-\alpha ^2} \ +\ (d_{12} - e^{-\mu }\, d_{11})\,\big (d_{22} + (1+2\mu )\, d_{23}\big )\,\mu \,\frac{1+\alpha }{1-\alpha }, \\ C&\ =\ 2d_{23}^2\, \mu ^2\, \frac{1+\alpha ^2}{1-\alpha ^2} \ +\ \Big (d_{22}^2 +2 (1+2\mu )\, d_{22} d_{23} + (1+2\mu )^2\, d_{23}^2\Big )\, \mu \, \frac{1+\alpha }{1-\alpha } \\&\ =\ 2d_{23}^2\, \mu ^2\, \frac{1+\alpha ^2}{1-\alpha ^2} \ +\ \big (d_{22} + (1+2\mu )\, d_{23}\big )^2\, \mu \, \frac{1+\alpha }{1-\alpha }. \end{aligned}$$

Finally, consider the particular case of the dispersion index (10), i.e., the case where $f_{d }(x_2,x_3) := \frac{x_3}{x_2}-x_2$. Then

$$\begin{aligned} \tfrac{\partial }{\partial x_2}\,f_{d }=-\frac{x_3}{x_2^2}-1, \qquad \tfrac{\partial }{\partial x_3}\,f_{d }=\frac{1}{x_2}. \end{aligned}$$

Hence, for the above Jacobian of ${\varvec{g}}$, evaluated at $(e^{-\mu }, \mu , \mu (1+\mu ))$, we obtain

$$\begin{aligned} d_{22}\ =\ -\frac{1}{\mu }-2,\qquad d_{23}\ =\ \frac{1}{\mu }. \end{aligned}$$

As a result, $d_{22} + (1+2\mu )\, d_{23} = 0$ such that the expressions B, C for the covariance matrix further simplify to

$$\begin{aligned} B\ =\ d_{11} d_{23}\, \mu ^2 e^{-\mu }\, \frac{1+\alpha ^2}{1-\alpha ^2}, \qquad C\ =\ 2d_{23}^2\, \mu ^2\, \frac{1+\alpha ^2}{1-\alpha ^2}. \end{aligned}$$

Inserting $d_{23}\ =\ 1/\mu$ completes the proof of Theorem 3.2.1. Note that the expression for C was already shown in Schweer and Weiß (2014).

1.3 Appendix 2.3: Proof of Proposition 1

The asymptotic behavior of the particular combination $\hat{I}_{\mathrm{PV}}$ or $\hat{I}_{\mathrm{vdB}}$ and $\hat{I}_{\mathrm{d}}$ is a direct consequence of Theorem 3.2.1. For the case $\hat{I}_{\mathrm{PV}}$, to obtain the expressions for $d_{11}, d_{12}$, we have to use the function $f_{\mathrm{z}}$ in Theorem 3.2.1 given by $f_{\mathrm{z}}(x_1,x_2) := 1+\frac{\ln {x_1}}{x_2}$, with partial derivatives

$$\begin{aligned} \tfrac{\partial }{\partial x_1}\,f_{\mathrm{z}}\ =\ \frac{1}{x_1x_2},\qquad \tfrac{\partial }{\partial x_2}\,f_{\mathrm{z}}\ =\ -\frac{\ln {x_1}}{x_2^2}. \end{aligned}$$

Evaluating in $(e^{-\mu }, \mu )$, we obtain

$$\begin{aligned} d_{11}\ =\ \frac{1}{\mu e^{-\mu }},\qquad d_{12}\ =\ \frac{1}{\mu }. \end{aligned}$$

Similarly, for the case $\hat{I}_{\mathrm{vdB}}$ we have to use, $f_{\mathrm{z}}(x_1,x_2)\ :=\ x_1e^{x_2}-1,$ with partial derivatives

$$\begin{aligned} \tfrac{\partial }{\partial x_1}\,f_{\mathrm{z}}\ =\ e^{x_2},\qquad \tfrac{\partial }{\partial x_2}\,f_{\mathrm{z}}\ =\ x_1e^{x_2}. \end{aligned}$$

Again, evaluating in $(e^{-\mu }, \mu )$, we obtain

$$\begin{aligned} d_{11}\ =\ e^{\mu },\qquad d_{12}\ =\ 1. \end{aligned}$$

Inserting $d_{11}$ and $d_{12}$ into Theorem 3.2.1, the results hold immediately.

1.4 Appendix 2.4: Proof of Proposition 2

The expectation of $\hat{I}_{\mathrm{z}}$ is determined by applying the second-order Taylor expansion to $\hat{I}_{\mathrm{z}}$. We obtain, ${\mathbb {E}}[\hat{I}_{\mathrm{z}}-I_{\mathrm{z}}]\approx {\mathbb {E}}[\frac{1}{2}{\varvec{Y}}_T^{'} {\mathbf {H}}_{f_{\mathrm{z}}}{\varvec{Y}}_T]$, with ${\varvec{Y}}_T=\frac{1}{\sqrt{T}}\sum _{t=1}^T{\varvec{Z}}_T$ satisfying ${\mathbb {E}}[{\varvec{Y}}_T]={\varvec{0}}$, and therefore

$$\begin{aligned} {\mathbb {E}}\left[ \tfrac{1}{2}{\varvec{Y}}_T^{'}{\mathbf {H}}_{f_{\mathrm{z}}}{\varvec{Y}}_T\right] \ =\ \tfrac{1}{2}\, {\mathbb {E}}\left[ h_{11}Y_1^2+2h_{12}Y_1Y_2+h_{22}Y_2^2\right] \\ \ =\ \tfrac{1}{2T}\,\left( h_{11}\sigma _{11}+2h_{12}\sigma _{12} +h_{22}\sigma _{22}\right) . \end{aligned}$$

The results for $\hat{I}_{\mathrm{PV}}$ and $\hat{I}_{\mathrm{vdB}}$ follow by direct calculations, taking into account that for $I_{\mathrm{PV}}$ the Hessian matrix of $f_{\mathrm{z}}$ is

$$\begin{aligned} {\mathbf {H}}_{f_{\mathrm{z}}}(e^{-\mu },\mu )\ =\ \begin{pmatrix} -\frac{1}{\mu e^{-2\mu }} &{}\quad -\frac{1}{\mu ^2e^{-\mu }} \\ -\frac{1}{\mu ^2e^{-\mu }} &{}\quad -\frac{2}{\mu ^2} \\ \end{pmatrix}, \end{aligned}$$

and for $I_{\mathrm{vdB}}$ the Hessian matrix of $f_{\mathrm{z}}$ is

$$\begin{aligned} {\mathbf {H}}_{f_{\mathrm{z}}}\big (e^{-\mu }, \mu \big )\ =\ \begin{pmatrix} 0 &{}\quad e^{\mu } \\ e^{\mu } &{}\quad 1 \\ \end{pmatrix}. \end{aligned}$$