Finite mixtures of multivariate skew t-distributions: some recent and new results

Abstract

Finite mixtures of multivariate skew t (MST) distributions have proven to be useful in modelling heterogeneous data with asymmetric and heavy tail behaviour. Recently, they have been exploited as an effective tool for modelling flow cytometric data. A number of algorithms for the computation of the maximum likelihood (ML) estimates for the model parameters of mixtures of MST distributions have been put forward in recent years. These implementations use various characterizations of the MST distribution, which are similar but not identical. While exact implementation of the expectation-maximization (EM) algorithm can be achieved for ‘restricted’ characterizations of the component skew t-distributions, Monte Carlo (MC) methods have been used to fit the ‘unrestricted’ models. In this paper, we review several recent fitting algorithms for finite mixtures of multivariate skew t-distributions, at the same time clarifying some of the connections between the various existing proposals. In particular, recent results have shown that the EM algorithm can be implemented exactly for faster computation of ML estimates for mixtures with unrestricted MST components. The gain in computational time is effected by noting that the semi-infinite integrals on the E-step of the EM algorithm can be put in the form of moments of the truncated multivariate non-central t-distribution, similar to the restricted case, which subsequently can be expressed in terms of the non-truncated form of the central t-distribution function for which fast algorithms are available. We present comparisons to illustrate the relative performance of the restricted and unrestricted models, and demonstrate the usefulness of the recently proposed methodology for the unrestricted MST mixture, by some applications to three real datasets.

Appendices

Appendix A: The truncated multivariate t-distribution

In this appendix, we briefly describe the truncated multivariate t-distribution and provide some formulas for computing its moments (Lee and McLachlan 2011). These expressions are crucial for the swift evaluation of the conditional expectations on the E-step of the FM-uMST model discussed in Sect. 5. We follow the approach of Lee and McLachlan (2011). A alternative description is given by Ho et al. (2012a), which provides equivalent expressions for the doubly truncated case.

Let X be a p-dimensional random variable having a multivariate t-distribution with location vector μ, scale matrix Σ, and ν degrees of freedom. Truncating x to the hyperplane region $\mathbb{A} = \{\boldsymbol{x} \geq\boldsymbol{a}, \boldsymbol{a} \in\mathbb{R}^{p}\}$, where x≥a means each element x _i =(x) _i is greater than or equal to a _i =(a) _i for i=1,…,p, results in a left-truncated t-distribution whose density is given by

$$ f_{\mathbb{A}}(\boldsymbol{x}; \boldsymbol{\mu}, \boldsymbol{\varSigma }, \nu) = T_{p,\nu}^{-1} (\boldsymbol{a};\boldsymbol{\mu},\boldsymbol { \varSigma} ) t_{p,\nu} (\boldsymbol{x};\boldsymbol{\mu}, \boldsymbol{ \varSigma } ),\quad \boldsymbol{x} \in\mathbb{A}. $$

(69)

For a random vector X with density (69), we write $\boldsymbol{X} \sim tt_{p,\nu} (\boldsymbol{\mu}, \boldsymbol{\varSigma}; \mathbb{A})$. For our purposes, we will be concerned with the first two moments of X, specifically E(X) and E(XX ^T). Explicit formulas for the truncated central t-distribution in the univariate case $tt_{1,\nu}(0, \sigma^{2}; \mathbb {A})$ were provided by O’Hagan (1973), who expressed the moments in terms of the non-truncated t-distribution. The multivariate case was studied in O’Hagan (1976), but still considering the central case only. Here we describe a generalization of the results in O’Hagan (1976) to the multivariate non-central case and express them in a form suitable for undertaking the E-step in the direct application of the EM algorithm to the fitting of mixtures of MST distributions.

Before presenting the expressions, it will be convenient to introduce some notation. Let x be a vector, then

x _i :

denotes the ith element,

x _ij :

is a two-dimensional vector with elements x _i and x _j ,

x _−i :

represents the (p−1)-dimensional vector with the ith element removed, and

x _−ij :

represents the (p−2)-dimensional vector with the ith and jth elements removed.

For a matrix X, let

x _ij :

denote the ijth element,

X _ij :

defines the 2×2 matrix consisting of the elements x _ii , x _ij , x _ji and x _jj ,

X _−i :

be created by removing the ith row and column from X,

X _−ij :

be the (p−2) square matrix resulting from the removal of the ith and jth row and column from X, and

X _(ij) :

be the ith and jth column of X with the elements of X _ij removed, yielding a (p−2)×2 matrix.

We now proceed to the expressions for the first two moments of X.

One can show that the first moment of (69) is

$$ E (\boldsymbol{X} ) = \boldsymbol{\mu}+ \boldsymbol {\epsilon}, $$

(70)

where ϵ=c ⁻¹ Σξ and c=T _p,ν (μ−a;0,Σ), and ξ is a p×1 vector with elements

for i=1,…,p, and where

The second moment is given by

(71)

where H is a p×p matrix with off-diagonal elements

and diagonal elements,

It is worth noting that evaluation of the expressions (70) and (71) rely on algorithms for computing the multivariate central t-distribution function for which highly efficient procedures are readily available in many statistical packages. For example, an implementation of Genz’s algorithm (Genz and Bretz 2002; Kotz and Nadarajah 2004) is provided by the mvtnorm package available from the R website.

Appendix B: E-step for uMST

Derivations of $e_{1,j}^{(k)}$, $\boldsymbol{e}_{3,j}^{(k)}$ and $\boldsymbol{e}_{4,j}^{(k)}$ are detailed as follows.

2.1 B.1 Calculation of $e_{1,j}^{(k)}$

Concerning the calculation of the expectation $e_{1,j}^{(k)}$, the conditional density of W _j given y _j , is given by

(72)

where

and 0 is the zero vector of appropriate dimension.

The conditional expectation $E_{\boldsymbol{\theta}^{(k)}}\{\log(W_{j}) \mid\boldsymbol {y}_{j}\}$ can be reduced to

(73)

where

$$\boldsymbol{y}_{2j}^{(k)} = \boldsymbol{q}_j^{(k)} \sqrt{\frac{\nu ^{(k)}+p+2}{\nu^{(k)}+d^{(k)}(\boldsymbol{y}_j)}}, $$

and where the last term S is given by

(74)

and $S_{1,j}^{(k)}$ is an integral given by

(75)

Combining (73) and (74), $e_{1,j}^{(k)}$ can be reduced to

(76)

We note that the term $S_{1,j}^{(k)}$ will be very small in practice since it would be zero if we adopted an OSL EM algorithm. In which case, there would be no need to calculate the multiple integral $S_{1,j}^{(k)}$ in (74). Hence then, $e_{1,j}^{(k)}$ can be reduced to

(77)

2.2 B.2 Calculation of $\boldsymbol{e}_{3,j}^{(k)}$ and $\boldsymbol{e}_{4,j}^{(k)}$

To obtain $\boldsymbol{e}_{3,j}^{(k)}$ and $\boldsymbol{e}_{4,j}^{(k)}$, first note that the joint density of y _j , u _j , and w _j is given by

(78)

Using Bayes’ rule, the conditional density of u _j and w _j given y _j can be written as

(79)

From (79), standard conditional expectation calculations yield

(80)

where X _j is a p-dimensional t-variate truncated to the positive hyperplane ℝ⁺, which is conditionally distributed as

$$ \boldsymbol{X}_j \mid\boldsymbol{y}_j \sim tt_{p,\nu^{(k)}+p+2} \biggl(\boldsymbol{q}_j^{(k)}, \biggl( \frac{\nu^{(k)}+d^{(k)}(\boldsymbol{y}_j)}{\nu^{(k)}+p+2} \biggr) \boldsymbol{\varLambda}^{(k)}; \mathbb{R}^+ \biggr). $$

(81)

Analogously, $\boldsymbol{e}_{4,j}^{(k)}$ can be reduced to

$$ \boldsymbol{e}_{4,j}^{(k)} = e_{2,j}^{(k)} E \bigl(\boldsymbol{X}_j \boldsymbol{X}_j^T \mid \boldsymbol{y}_j\bigr). $$

(82)

The truncated moments E(X _j ∣y _j ) and $E(\boldsymbol{X}_{j} \boldsymbol{X}_{j}^{T} \mid\boldsymbol{y}_{j})$ can be swiftly evaluated using the expressions (70) and (71) in Sect. 3.2.

Appendix C: E-step for FM-uMST

The four conditional expectations $e_{1,hj}^{(k)}$, $e_{2,hj}^{(k)}$, $\boldsymbol{e}_{3,hj}^{(k)}$, and $\boldsymbol{e}_{4,hj}^{(k)}$ involved in the E-step are given by

(83)

(84)

(85)

(86)

where $S_{1,hj}^{(k)}$ is a scalar defined by

(87)

and X _hj is a truncated p-dimensional t-variate given by

$$\boldsymbol{X}_{hj} \mid\boldsymbol{y}_j \sim tt_{p,\nu_h+p+2} \biggl(\boldsymbol{q}_{hj}^{(k)}, \biggl( \frac{\nu_h^{(k)}+d_h^{(k)}(\boldsymbol{y}_j)}{ \nu_h^{(k)}+p+2} \biggr)\boldsymbol{\varLambda}_h^{(k)}, \mathbb{R}^+ \biggr). $$

The first two moments of X _hj can be implicitly expressed in terms of the parameters $\boldsymbol{q}_{hj}^{(k)}$, $d_{h}^{(k)}(\boldsymbol{y}_{j})$, $\boldsymbol{\varLambda}_{h}^{(k)}$, $\nu_{h}^{(k)}$ using results (70) and (71). It is worth emphasizing that computation of $\boldsymbol{e}_{3hj}^{(k)}$ and $\boldsymbol{e}_{4hj}^{(k)}$ depends on algorithms for evaluating the multivariate t-distribution function, for which fast procedures are available.