The Statistical Analysis of Categorical Data

This book is about categorical data, i.e. data which can only take a finite or countable number of values.

In this chapter a short review is given of some basic elements of statistical theory which are necessary prerequisites for the theory and methods developed in subsequent chapters.

The majority of interesting models for categorical data are log-linear models. A family of log-linear models is often referred to as an exponential family.

A two-way contingency is a number of observed counts set up in a matrix with I rows and J columns Data are thus given as a matrix The statistical model for such data depends on the way the data are collected. A great variety of tables can, however, be treated by three closely connected statistical models. Let the random variables corresponding to the contingency table be X₁₁,…,X_IJ. Then in the first model the X_ij’sare assumed to be independent with i.e. X_ij is Poisson distributed with parameter λ_ij. The likelihood function for this model is The log-likelihood is accordingly given by The model is thus a IJ-dimensional log-linear model with canonical parameters lnλ₁₁,...,1nλ_IJ and sufficient statistics T_ij=X_ij, i=1,...,I, j=1,...,J.

Consider a three-way contingency table {X_ijk, i=1,...I, j=1,...,J, k=1,...,K}. As model for such data, it may be assumed that the x’s are observed values of random variables X_ijk, i=1,...,I, j=1,...,J, k=1,...,K with a multinomial distribution

In chapter 5, log-linear models for three-dimensional tables were treated in great details. Hence we shall not for higher order tables go into details with the parameterizations of the models or with the exact expressions for test quantities and their distributions. Besides for higher order tables the mathematical expressions quickly becomes large and cumbersome to write down.

An observed contingency table is incomplete if it contains zeros in certain cells. Such zeros are of two types, random zeros and structural zeros. A cell has a random zero, if the observed value in the cell is zero, but the expected value is positive. A cell has a structural zero if the expected number is zero, i.e. if it is known a priori that the cell will contain a zero. Random or structural zeros does not impaire the log-linear structure of a given model. It means, however, that certain log-linear parameters can not be estimated.

In chapters 4, 5 and 6 the categorical variables appeared in the model in a symmetrical way. In many situations, for example in examples 6.1 and 6.2 in chapter 6, one of the variable is of special interest. For the survival data in example 6.1, survival is the variable of special interest, and the problem is to study if the other three variables have influenced the chance of survival. Variable B in example 6.1 may, therefore, be called a response variable and variables A, C and D explanatory variables. This terminology is the same as the one used in regression analysis, and when survival is regarded as a response variable the data in example 6.1 can in fact be analysed by a regression model. In example 6.2 the position on the truck of the collision can be regarded as a response variable. We are here primarily interested in the effect of explanatory variable A, i.e. the introduction of the safety measure in November 1971, but have to take into account that the other explanatory variables, i.e. whether the truck was parked or not and what the light conditions were, may be of importance for the location of the collision. When the response variable is binary and the explanatory variables are categorical, the appropriate regression model is known as the logit model. More precisely the assumptions for a logit model are:

In chapter 8 the connection to log-linear models for contingency tables was stressed. The direct connection to regression analysis for continuous response variables will now be brought more clearly into focus. Assume as before that the response variable is binary and that it is observed together with p explanatory variables. For n cases the data will then consist of n vectors of jointly observed values of the response variable and the explanatory variables.

If the statistical analysis of a contingency table is based on one of the log-linear models in chapters 5, 6 and 7, a number of natural models are easily overlooked. Many useful models can thus be expressed as structures in the log-linear interaction parameters. In this chapter a number of such models are discussed. Many of these models can be viewed as attempts to describe the non-zero interactions by a simple structure if the analysis of the data by a log linear model has failed to give a satisfactory fit to the model. If e.g. the independence hypothesis for a two-way table has been rejected, a residual analysis will often reveal a certain structure in the two-factor interactions.

The structure of a log-linear model can be described on an association diagram by the lines connecting the points. Especially for higher order contingency tables the structure on an assocition diagram can be very complicated, implicating a complicated interpretation of the model. Adding to the interpretation problems for a multi-dimensional contingency table is the fact, that the decision to exclude or include a given interaction in the model can be based on conflicting significance levels depending on the order in which the statistical tests are carried out. These decisions are thus based on the intuition and experience of the data analyst rather than on objective criteria. Hence a good deal of arbitrariness is often involved, when a model is selected to describe the data. We recall for example from several of the examples in the previous chapters that the log-linear model often gave an adequate description of the data judged by a direct test of the model against the saturated model, while among the sequence of successive tests leading to the model, there were cases of significant levels.

For most of the models in this book it is necessary to use computer programs to execute the statistical computations.