Kurtosis removal for data pre-processing

A fourth cumulant of a random vector with finite fourth moments is the fourth derivative, evaluated at the origin, of the cumulant generating function of the random vector itself, that is the logarithm of its characteristic function. All fourth cumulants of a multivariate normal distribution equal zero, and we refer to distributions with the same property as to mesokurtic distributions. Also, we refer to transformations leading to null fourth cumulants as to mesokurtic transformations. In particular, when the mesokurtic transformation is a linear function of the original variables, we refer to it as to a mesokurtic projection. With a little abuse of language, and for ease of description, we write that mesokurtic transformations remove kurtosis. Mesokurtic transformations include as special cases transformations to normality, also known as gaussianizing transformations. They have been extensively studied in scientific fields other than Statistics, as for example Physics (Yu et al. 2016). Mesokurtic transformations are of interest whenever the performance of the chosen statistical method either increases or decreases with the absolute values of fourth-order cumulants.

Researchers from social and life sciences are often concerned with the lack of normality of their data, since many multivariate statistical techniques are not robust when the sampled distribution is wrongly assumed to be normal. Sometimes the performance of the statistical technique depends on the fourth cumulants of the sampled distribution, as exemplified by the following cases.

Inference on covariance matrices is notoriously very sensitive to fourth cumulants. Mardia (1974) showed that the performance of a likelihood test based on the erroneous assumption of normality crucially depends on a measure of multivariate kurtosis which is a simple function of fourth cumulants. Schott (2002) used fourth cumulants to derive a robust procedure for testing the equality of the population covariance and a given matrix. Yanagihara et al. (2005) extended Mardia’s results to several likelihood tests on covariance matrices based on the normality assumption, and showed that their performances depend on their fourth cumulants.

Variogram estimation is of paramount importance in spatial statistics but spatial data are often nonnormal, also due to preferential sampling (Loperfido and Guttorp 2008). Genton et al. (2001) modelled nonnormality with the multivariate skew-normal distribution and derived analytical formulae for covariances of variogram estimators, and showed that they depend on the fourth cumulants of the same distribution. Kim (2005) extended their results to mixtures of skew-normal distributions. Rezvandehy and Deutsch (2018) addressed preferential sampling with fourth cumulants.

Relevance of fourth cumulants is not limited to covariance testing and variogram estimation. Yanagihara (2007) stressed the importance of fourth cumulants in the multivariate linear model. Arevalillo and Navarro (2012) investigated the effect of fourth cumulants on the Fisher discriminant function (i.e. the projection which best separates the means of two multivariate distributions), and found it to be nonnegligible. The large-sample approximation of the multivariate sample mean by the skew-normal distribution improves when the fourth cumulants of the two distributions are close to each other (Christiansen and Loperfido 2014).

Multivariate normality is usually pursued by means of the componentwise, nonlinear, univariate transformations proposed by either Box and Cox (1964) or Tukey (1977). Tsay et al. (2017) thoroughly review these and other transformations to normality, while proposing a new one which is particularly apt at dealing with platykurtic distributions. Unfortunately, nonlinearly transformed variables might not be easily interpretable nor jointly normal, as pointed out by Lin and Lin (2010), among others. Moreover, the same methods are inappropriate when the joint distribution is not normal, but univariate marginals are, as it happens for the distributions described in Arnold et al. (2001). Loperfido (2014) and Loperfido (2019) used a simple argument to show how nonliner transformations to normality are inappropriate when using Hotelling’s one-sample test.

Mesokurtic projections do not suffer from these limitations. We illustrate this point with the bivariate distribution \(2\phi \left( x\right) \phi \left( y\right) \varPhi \left( \lambda xy\right) \) introduced by Arnold et al. (2001), where \(\phi \left( \cdot \right) \) and \(\varPhi \left( \cdot \right) \) denote the probability and the cumulative density functions of a standard normal distribution, while \(\lambda \) is a real value. Adcock (2021) thoroughly investigated its properties and proposed some generalizations. The default approach to the gaussianization of the random vector \(\left( X,Y\right) \) would be raising both its components to an appropriate power. However, there do not exist two real values \(\alpha \) and \(\beta \) such that the joint distribution of \(\left( X^{\alpha },Y^{\beta }\right) \) is bivariate normal. On the other hand, there are two normal, and therefore mesokurtic, projections of \(\left( X,Y\right) \), that is X and Y.

The performance of kurtosis-based projection pursuit tends to increase with the absolute values of fourth cumulants. Projection pursuit is a multivariate statistical technique aimed at finding interesting data projections, with interestingness quantified by the projection index: the data projection with the highest value of the projection index is regarded as the most interesting. The normal distribution is commonly regarded as the least interesting (Huber 1985), so that the projection index often measures a nonnormality feature such as skewness, kurtosis or multimodality. In particular, kurtosis-based projection pursuit uses the absolute value of the fourth standardized cumulant as a projection index, consistently with the projection pursuit criteria stated in Huber (1985). Kurtosis-based projection pursuit therefore looks for interesting projections by either maximizing or minimizing kurtosis, that is the fourth standardized moment. Kurtosis-based projection pursuit appears in cluster analysis (Peña and Prieto 2000, 2001b), outlier detection (Galeano et al. 2006; Peña and Prieto 2001a), normality testing (Malkovich and Afifi 1973), independent component analysis (Girolami and Fyfe 1996), invariant coordinate selection (Alashwali and Kent 2016), chemometrics (Hou and Wentzell 2014), finance (Loperfido 2020b).

Projection pursuit is commonly regarded as more problematic when applied to datasets with more variables than units (see, for example, Hui and Lindsay 2010; Lindsay and Yao 2012). Bickel et al. (2018) thoroughly investigated the asymptotic properties of projection pursuit for several ratios of the number of variables to the number of units, finding serious shortcomings of the method when the number of variables is much greater than the number of units. Pires and Branco (2019) showed that two-dimensional projections of datasets with more variables than units could approximate any given set of bivariate data with the same number of units. Lee and Cook (2010) illustrated the limitation of projection pursuit for classification with a real dataset containing 3571 variables recorded from 72 cases. However, the performance of kurtosis-based projection pursuit might rapidly deteriorate as the number of variables increases while the number of units remains fixed, even if the former remains much smaller than the latter (Loperfido 2020a).

The problems posed by high-dimensional data might be eased by means of sparse projection pursuit, that is projection pursuit performed on a small subset of variables, either original or projected (Bickel et al. 2018). Jones and Sibson (1987) proposed a linear transformation of the data into two mutually disjoint subsets of variables, one deemed uninteresting and the other deemed interesting. The former should be discarded while the latter should be analysed by means of projection pursuit. Following this approach, Blough (1989), Loperfido (2017), Franceschini and Loperfido (2019) removed skewness from the data by means of linear transformations, within the framework of skewness-based projection pursuit. Hui and Lindsay (2010) and Lindsay and Yao (2012) linearly transformed the data into two mutually orthogonal subsets of interesting and uninteresting projections. Ray (2010) used a simulated dataset with many more variables than units to illustrate the merits of this approach. In kurtosis-based projection pursuit, where the least interesting distributions are those with null fourth order cumulants, data are linearly transformed into two mutually orthogonal sets of variables, of which only one is mesokurtic. Then the mesokurtic set of variables is removed and kurtosis optimization is carried out on the remaining variables.

This paper uses several kurtosis matrices to obtain mesokurtic projections. It states both sufficient and necessary conditions for the existence of mesokurtic projections. The approach is algebraic in nature, since it relies on matrix concepts, including null linear spaces, spectra and generalized eigenvalues. As a major advantage, sampling properties of mesokurtic projections might be easily derived from the theory of random matrices and their spectra. The remainder of the paper is organized as follows. Section 2 uses kurtosis matrices to investigate the existence and the properties of mesokurtic projections. Section 3 contains some simulation studies related to model-based clustering. Section 4 applies the proposed method to a subset of the AIS dataset containing 23 units and 11 variables, which is a nonnegligible variables-to-units ratio. Section 5 uses the RANDU dataset to show that mesokurtic projections might be a viable alternative to nonlinear transformations to normality. Section 6 contains some concluding remarks, mentions further applications of mesokurtic projections, discusses their limitations and suggests some extensions of the proposed method to datasets with more variables than units. The proofs of the theorems are in the Appendix.

This section describes a method for obtaining projections with null or negligible fourth cumulants, that is mesokurtic and nearly mesokurtic projections. Firstly, it uses bivariate random vectors to illustrate situations where mesokurtic projections do not exist. Secondly, provides necessary and sufficient conditions for the existence of mesokurtic projections of random vectors in any dimension. Thirdly, the section briefly discusses the sampling properties of mesokurtic projections.

The ijhk-th cumulant of a d-dimensional random vector \(x=\left( X_{1},\ldots ,X_{d}\right) ^{T}\) satisfying \(E\left( X_{i}^{4}\right) <+\infty \) for \(i=1\), ..., d iswhere \(\iota =\sqrt{-1}\), \(t\in \mathbb {R}^{d}\) and \(E\left[ \exp \left( \iota t^{T}x\right) \right] \) is the cumulant generating function. In the univariate case, the only fourth cumulant iswhere \(\mu \) and \(\sigma ^{2}\) are the mean and the variance of a random variable X satisfying \(E\left( X^{4}\right) <\infty \).

The bivariate case provides a good insight into nonexistence of mesokurtic projections. Let U and W be two standardized, independent random variables whose fourth cumulants \(\kappa _{4}\left( U\right) \) and \(\kappa _{4}\left( W\right) \) are both positive. Elementary properties of cumulants imply that the fourth cumulant of the projection \(hU+kW\) is positive, too:As a direct consequence, there does not exist a mesokurtic projection of \( \left( U,W\right) \).

Nonexistence of mesokurtic projections of bivariate and standardized random vectors becomes more difficult to ascertain when the independence assumption is removed. Let \(\left( X,Y\right) \) be a bivariate and standardized random vector:Also, let \(\beta _{i}=E\left( X^{i}Y^{4-i}\right) \), for\(\;i=0,1,2,3,4\). The fourth cumulant of the linear combination \(aX+Y\) is a fourth-order polynomial in a:A mesokurtic linear function of X and Y exists if and only if the polynomial has real roots.

The problem of ascertain the existence of mesokurtic projections becomes even more complicated when considering standardized random vectors with more than two components. We address the problem by recalling that fourth cumulants are simple functions of the fourth and second central moments. The ijhk-th moment of x is the expectation \(\mu _{ijhk}=E\left( X_{i}X_{j}X_{h}X_{k}\right) \), for \(i,j,h,k=1,\ldots ,d\). The ijhk-th central moment of x is the ijhk-th moment of \(x-\mu \), where \(\mu =\left( \mu _{1},\ldots ,\mu _{d}\right) ^{T}\) is the mean of x:The ijhk-cumulant can then be represented aswhere \(\sigma _{ab}\) is the covariance between the a-th and the b-th components of x, with a, b=1,..., d. For example, the fourth cumulants of \(\left( X_{1},X_{2}\right) ^{T}\), expressed as functions of its second and fourth central moments, areWe first consider the fourth moment (matrix) of a \(d-\)dimensional random vector \(x=\left( X_{1},\ldots ,X_{d}\right) ^{T}\) satisfying \(E\left( X_{i}^{4}\right) <+\infty \), for \(i=1,\ldots ,d\), that is the \(d^{2}\times d^{2}\) matrix \(M_{4,x}=E\left( x\otimes x^{T}\otimes x\otimes x^{T}\right) \), where “\(\otimes \)” denotes the Kronecker product. It conveniently arranges all the fourth-order moments \(\mu _{ijhk}=E\left( X_{i}X_{j}X_{h}X_{k}\right) \) of x (where \(i,j,h,k=1,\ldots ,d\)) and admits the block matrix representation \( M_{4,x}=\left\{ B_{pq}\right\} \), where \(B_{pq}=E\left( X_{p}X_{q}xx^{T}\right) \), for \(p,q=1,\ldots ,d\). For example, the fourth moment of \(\left( X_{1},X_{2}\right) ^{T}\) isIf the variance \(\varSigma \) of x is positive definite, its fourth standardized moment \(M_{4,z}\) is the fourth moment of \(z=\varSigma ^{-1/2}\left( x-\mu \right) \), where \(\varSigma ^{-1/2}\) is the symmetric, positive definite square root of the concentration matrix \(\varSigma ^{-1}\). Most measures of multivariate kurtosis are either scalar-type or matrix-type functions of \(M_{4,z}\) (Loperfido 2017, 2020a). In the univariate case \(M_{4,z}\) coincides with Pearson’s measure of kurtosis:The eigenstructure of \(M_{4,z}\) has several interesting features: eigenvectors of \(M_{4,z}\) associated with its positive eigenvalues are vectorized, symmetric matrices and the eigenvector of \(M_{4,z}\) associated with its largest eigenvalue (that is the dominant eigenvector of \(M_{4,z}\)) is a semidefinite matrix (Loperfido 2017). The following theorem gives a necessary condition for the existence of mesokurtic projections based on the eigenstructure of the fourth standardized moment matrix. It is reminescent of the distinction between platykurtic and leptokurtic random variables, whose fourth standardized moments are smaller and greater than three, respectively.

Proposition 1 suggests the following procedure for computing projections with negligible fourth cumulants. First, compute the nonnull eigenvalues of the fourth standardized moment matrix. If they are all greater (smaller) than three the projections with fourth cumulants closer to zero are those which minimize (maximize) kurtosis. In these circumstances, kurtosis alleviation coincides with kurtosis optimization, that is kurtosis-based projection pursuit. The computational issues of the latter might be addressed by the method proposed in Franceschini and Loperfido (2018), which are implemented in the R package Kurt (Franceschini and Loperfido 2020).

On the other hand, if some nonnull eigenvalues of the fourth standardized moment are smaller than three while other eigenvalues are greater than three, it is worth looking for mesokurtic projections. An intuitively appealing approach has been proposed by Peña and Prieto (2001a) and Peña and Prieto (2001b) within the framework of outlier detection. First, find the projection with maximal kurtosis. Second, project the data onto its orthogonal subspace. Then repeat these steps until all remaining projections are mesokurtic or nearly so. This approach, which may be referred to as to iteractive projection pursuit approach, proved to be useful when nonnormality is due to the presence of outliers, but may not detect mesokurtic projections when data are generated from different models, as it happens in independent component analysis.

Independent component analysis (ICA) is a multivariate statistical technique aimed at recovering independent, unobserved signals by appropriate data projections. The basic ICA model is \(x=b+As\), where x is a d-dimensional random vector, b is a d-dimensional real vector, A is a \(d\times d\) invertible real matrix and the components of s are mutually independent, standardized random variables of which at most one is normal (Miettinen et al. 2015).

Iteractive projection pursuit does not detect mesokurtic projections when the fourth cumulants of the signals are nonnull and take both signs. We illustrate the point with the bivariate ICA modelwhere the first, second, fourth cumulants of \(S_{1}\) and \(S_{1}\) areThe projection of x with maximal kurtosis coincides, up to location and scale changes, with the first signal, which is leptokurtic:The projection of x orthogonal to \(2X_{2}-3X_{1}\) coincides, up to location and scale changes, with the second signal, which is platykurtic:Neither projection is mesokurtic, but the model implies the existence of the mesokurtic projection \(X_{1}+X_{2}\).

A sufficient condition for the existence of mesokurtic projections might be established using the cokurtosis matrix (see, for example, Jondeau and Rockinger 2006), that is the \(d\times d^{3}\) matrixFor example, the cokurtosis of \(\left( X_{1},X_{2}\right) ^{T}\) isThe standardized cokurtosis of x is just the cokurtosis of \(z=\varSigma ^{-1/2}\left( x-\mu \right) \). The fourth standardized matrix \( M_{4,z}=\left\{ M_{pq}=E\left( Z_{p}Z_{q}zz^{T}\right) \right\} \) and the standardized cokurtosis \(cok\left( z\right) \) might be regarded as block matrices where the same blocks are arranged in different ways. The latter aligns side by side the blocks of the fourth standardized moment in such a way that the blocks with the smallest first indices appear first, followed by those having the smallest second indices: \(cok\left( x\right) =\left( M_{11},M_{12},\ldots ,M_{dd}\right) \) where the block \(M_{ij}\) appears before the block \(M_{aj}\) if \(i<a\), and the block \(M_{ij}\) appears before the block \(M_{ib}\) if \(j<b\).

Both \(M_{4,z}\) and \(cok\left( z\right) \) may contain up to \(d\left( d+1\right) \left( d+2\right) \left( d+3\right) /24\) distinct elements. Since this number grows very quickly with the vector’s dimension, it is convenient to summarize them with smaller matrices. Cardoso (1989) proposed \( K_{z}=E\left( z^{T}zzz^{T}\right) \) as a kurtosis matrix. Mòri et al. (1993) independently proposed and discussed its variant \(K_{z}-\left( d+2\right) I_{d}\). Statistical applications of both kurtosis matrices include independent component analysis (Cardoso 1989), generalized principal components (Caussinus and Ruiz-Gazen 2009), invariant coordinate selection (Alashwali and Kent 2016 and cluster analysis (Peña et al. 2010). Loperfido 2017) highlighted the connection between \(K_{z}\) and tensor contraction. The kurtosis matrix \(K_{z}\) is a function of the fourth moment matrix of a standardized random vector and is commonly regarded as a good compromise between detail and synthesis, when studying the kurtosis structure of the random vector itself. The kurtosis matrix \(K_{z}\) does not account for expectations of products of four different components of z: elements in \(K_{z}\) do not depend on \(E\left( Z_{i}Z_{j}Z_{h}Z_{k}\right) \), when i, j, h, k differ from each other. For this reason we refer to \(K_{z}\) as the partial kurtosis matrix. In the bivariate case it isThe following theorem establishes a sufficient condition for the existence of mesokurtic projections. It describes a matrix depending on both standardized cokurtosis and partial kurtosis which recovers mesokurtic projections, when it is not of full rank.

In practice, kurtosis matrices need to be estimated. To this end, we use a \( n\times d\) data matrix X and the corresponding sample covariance matrix S , which is assumed to be of full rank. This assumption implies than there are more units than variables, the opposite case falling outside the scope of the present paper. Possible extensions of the proposed method to datasets with more variables than units are discussed in the Conclusions section.

First we obtain the standardized data matrix \(Z=H_{n}XS^{-1/2}\), where \( H_{n} \) is the \(n\times n\) centring matrix and \(S^{-1/2}\) is the symmetric square root of \(S^{-1}\). The fourth sample standardized moment, the sample standardized cokurtosis and the sample partial kurtosis arewhere \(z_{i}^{T}\) is the \(i-\)th row of Z. Under random sampling, the sequencesalmost surely converge to the matrices \(M_{4,z}\), \(cok\left( z\right) \) and \( K_{z}\), as long as \(E\left( X_{i}^{4}\right) <\infty \), for \(i=1\), \(\ldots \), d . Since eigenvalues are continuous functions of their matrices (Ortega 1987 , page 41) eigenvalues of matrices in the three sequences converge almost surely to eigenvalues of the corresponding matrices. Convergence of eigenvectors and convergence in law require some additional assumptions, which are thoroughly described in Rublik (2001). Propositions 1, 2 connect mesokurtic projections to the eigenvalues of symmetric, positive semidefinite matrices, thus easing the task of making inference on the former by using the well-established theory regarding the latter.

Consistently with Proposition 2, mesokurtic projections might be obtained as follows, if some nonnull eigenvalues of the fourth sample standardized moment being smaller than three while other eigenvalues are greater than three. First, compute the eigenvectors corresponding to the smallest eigenvalues of the matrixFor the sake of clarity we assume that all eigenvalues of \(Q_{n}\) are distinct, as it often occurs in practice, according to our personal experience. The matrix \(ZB_{n}\) estimates the mesokurtic projections that can be obtained from the sampled distributions, where the columns of the matrix \(B_{n}\) are the eigenvectors associated with the k smallest eigenvalues of \(Q_{n}\), with \(k<d\). Alternatively, we could look for random projections onto the subspace generated by the same eigenvectors. Random projections proved to be very useful for finding interesting projections (Peña and Prieto 2007), but they might also be useful for detecting uninteresting ones, since most data projections are approximatively normal, under mild assumptions (Diaconis and Freedman 1984). The number of projections might be chosen on the ground of subject-matter considerations or after testing for nullity the smallest eigenvalues of \(Q_{n}\).

The above mentioned approach aimed at finding mesokurtic projections relies on little restrictive assumptions, being Propositions 1, 2 nonparametric in nature. However, a practitioner would greatly appreciate more precise indications when looking for mesokurtic projections. We address the problem with another kurtosis matrix and within the framework of closed skew-normal distributions. Fourth cumulants are often arranged in the exkurtosis, that is the \(d\times d^{3}\) block matrixand the blocks with smaller first indices appear first, followed by those having smaller second indices: the block \(K_{ij}\) appears before the block \( K_{qj}\) if \(i<q\), and the block \(K_{ij}\) appears before the block \(K_{ip}\) if \(j<p\). The name “exkurtosis” is a shorthand for “excess kurtosis matrix”, since the best known measures of multivariate excess kurtosis are functions of the exkurtosis. Loperfido (2020c) investigates some properties of the exkurtosis matrix. In particular, we havewhere b is a d-dimensional real vector, y is a d-dimensional random vector with finite fourth-order moments and independent of x, A is a \( k\times d\) real matrix.

The closed skew-normal distribution introduced by Gonzalez-Farias et al. (2003) provides another useful tool for modelling the sample bias. Its name reminds that it is closed with respect to conditioning, affine transformations and convolutions. The random vector x has a closed skew-normal distribution of parameters \(\xi ,\varOmega ,\varPsi ,\eta ,\varDelta \), and we write \(x\sim CSN\left( \xi ,\varOmega ,\varPsi ,\eta ,\varDelta \right) ,\) if its density function isand \(\varPhi \left( z;\mu ,\varSigma \right) \) is the cdf of \(N_{p}\left( \mu ,\varSigma \right) \) evaluated at \(z\in \mathbb {R}^{p}\), while \(\varOmega \) and \( \varDelta \) are symmetric, positive definite matrices. The following proposition gives a sufficient condition for the existence of a linear projection of a closed skew-normal random vector which is normal (and hence mesokurtic).

Proposition 3 and known inferential results on the closed skew-normal distribution pave the way toward a criterion aimend at finding normal (and therefore mesokurtic) projections. First, estimate the parameters of the closed skew-normal distribution either by maximum likelihood or by alternative methods (Flecher et al. 2009; He et al. 2019). The latter methods should be preferred when the likelihood function is deemed too difficult to compute, since it involves the evaluation of multivariate integrals.

Second, estimate the exkurtosis matrix \(exk\left( x\right) \) by its sample counterpart \(\widehat{E}\) and compute the normalized eigenvectors associated with the \(d-\widehat{h}\) smallest eigenvalues of \(\widehat{E}\widehat{E}^{T}\) , where the estimate \(\widehat{h}\) of h is just the size of the symmetric matrix \(\widehat{\varDelta }\), i.e. the estimate of \(\varDelta \). Finally, estimate the matrix B with the matrix \(\widehat{B}\) whose i-th row is the normalized eigenvector associated with the above mentioned matrix. Convergence of \(\widehat{B}\) to a matrix whose columns belong to the null space of \(exk\left( x\right) ^{T}\) follows from the mild assumptions stated in Tyler (1981) for eigenvectors of symmetric random matrices.

The scope of the simulation studies in this section is twofold. Firstly, simulations are used to assess the relevance of the problems posed by projection pursuit when the number of variables approaches the number of units. Secondly, simulations are used to assess the number of mesokurtic projections detected by the method described in the previous section, when sampling from finite normal mixtures.

We addressed the first question by simulating 1000 samples of size \(n=25\) and \(n=50\) fromwhere \(0_{d}\), \(1_{d}\) and \(I_{d}\) are the \(d-\)dimensional null vector, the \( d-\)dimensional vector of ones and the \(d\times d\) identity matrix, \(\pi _{1}=0.1\), 0.2, 0.3, \(\mu =1\), 2, 3, 4, \(d=3\), 6, 12, 24 if \( n=25\) and \(d=6\), 12, 24, 48 if \(n=50\). The ratio of the number of variables to the number of units increases from 12 to \(96\%\), with greater values of \(\mu \) denoting better separations between the mixture’s components and greater values of \(\pi _{1}\) denoting smaller skewness and kurtosis. The projection maximizing kurtosis is also the one which best separates the two normal components and is proportional to \(x^{T}1_{d}\) (see, for example, Loperfido 2020b). For each simulated sample, we computed the squared correlation between the projection maximizing sample kurtosis and the projection onto the direction of \(1_{d}\): higher values of the squared correlation indicate a better performance of kurtosis-based projection pursuit in detecting the cluster structure, that is in separating the mixture components.

The simulation’s results in Table 1 clearly indicate that kurtosis-based projection pursuit is highly sensitive to the ratio of the number of variables to the number of units: higher ratios lead to worse performances. Kurtosis-based projection pursuit becomes virtually useless when the number of variables is just slightly smaller than the number of units. Another simulations study (not reported here) suggests that under this circumstance the projection maximizing sample kurtosis and the projection of the data onto the direction of \(1_{d}\) are uncorrelated. Quite surprisingly, the problem gets more serious when the number of units increases. On the other hand, the performance of kurtosis-based projection pursuit improves when the components are better separated and nonnormality increases.

A natural question to ask is whether mesokurtic projections exist for widely used families of multivariate distributions, such as normal mixtures. Let the distribution of the \(d-\)dimensional random vector x be the mixture, with weights \(\pi _{1}\),..., \(\pi _{k}\) of the normal distributions \( N_{d}\left( \mu _{1},\varSigma \right) \), \(\ldots \), \(N_{d}\left( \mu _{k},\varSigma \right) \), with \(k<d\). In the general case, the fourth cumulants of x are nonnull. Consider now a \(k\times d\) matrix B satisfying \(B\mu _{i}=0_{k}\) for \(i=1,\) ..., k. It follows that Ax is normally distributed and hence mesokurtic. Does the method described in the previous section detect all these mesokurtic projections? The question becomes more difficult to answer when the covariances of the normal components are not assumed to coincide. As an example, consider the mixture of two normal distributions with different means and proportional covariance matrices: \(\pi _{1}N_{d}\left( \mu _{1},\varSigma \right) +\pi _{2}N_{d}\left( \mu _{2},\alpha \varSigma \right) \) , with \(\pi _{1}+\pi _{2}=1\), \(\mu _{1}\ne \mu _{2}\), \(\pi _{1},\pi _{2}>0\) , \(\alpha \ne 1\). There is no projection whose distribution is normal. Does this mean that there are not mesokurtic projections?

We use simulations to address these problems. Multivariate samples of size 100 were simulated from mixtures of two multivariate normal distributions. Other simulation studies, not reported here, were based on samples of sizes 200 and 300 and led to conclusions similar to the below-mentioned ones.

We first simulated 1000 samples from the normal, homoscedastic mixture \( \pi _{1}N_{d}\left( 0_{d},I_{d}\right) +\left( 1-\pi _{1}\right) N_{d}\left( 5\cdot 1_{d},I_{d}\right) \), where \(\pi _{1}\) is either 0.1 or 0.5, for \( d=8\), 12, 16, 20. The sampled distribution is skewed and leptokurtic (symmetric and platykurtic) when \(\pi _{1}=0.1\) (\(\pi _{1}=0.5\)), and admits \(d-1\) jointly normal projections onto the linear subspace orthogonal to \( 1_{d}\). All samples exhibit either moderate or large units-to-variables ratios, thus motivating mesokurtic projections, either for variable selection or for parametric inference.

Then we simulated 1000 samples from the normal, heteroscedastic mixturewhere \(\alpha \) is either 0.5 or 2, for \(d=8\), 12, 16, 20. It models the presence of multivariate outliers, by assuming that the sampled distribution is a two-component mixture, with one mixture weight much smaller than the other. The components with the largest and smallest mixture weights model the bulk of the data and the outliers, respectively (Loperfido 2020b). The outliers generated from the model are said to be concentrated or dispersed depending on whether the parameter \(\alpha \) equals 0.5 or 2 .

We measure kurtosis by Koziol’s statistic (Koziol 1989), that is the squared Euclidean norm of the fourth standardized moment:where m is the sample mean, S is the sample variance (which is assumed to be nonsingular) and \(x_{h}^{T}\) is the h-th row of the \(n\times d\) data matrix X, for \(h=1\), \(\ldots \), d. We favored Koziol’s kurtosis over the more popular Mardia’s kurtosis because the former, unlike the latter, depends on all fourth-order standardized moments. Other properties of Koziol’s kurtosis, including its advantages over Mardia’s kurtosis, are discussed in Loperfido (2020a).

The Koziol’s kurtosis of either a random vector or a sampled dataset increases with its dimension: the Koziol’s kurtosis of a d-dimensional normal random vector is \(3d\left( d+2\right) \). In order to compare the nonnormality of datasets with different numbers of variables we use the relative difference between their Koziol’s kurtoses and the Koziol’s kurtoses of normal random vectors with the same dimensions. For example, if the dataset at hand contains three variables and its Koziol’s kurtosis is 54, its relative difference to the Koziol’s kurtosis of a three-dimensional normal random vector is \((54-45)/45=0.2\).

We applied the proposed method to each sample to obtain \(d-1\), d/2, \(\sqrt{ d}\) projections. Ideally, a method purported to obtain mesokurtic projections should detect all the \(d-1\) jointly normal projections. When this does not happen, it is desirable to have some guidance about the achievable number of jointly normal linear projections, as for example d/2 or \(\sqrt{d}\). For both the original and the projected data we computed the absolute relative difference between the observed Koziol measure of kurtosis and its expectation under the normality assumption. We would like this ratio to be much lower for the projected data than for the original ones. For each sample we also computed the \(p-\)value of Koziol kurtosis, using its asymptotically normal approximation. We would like it to be, on average and for the projected data, well above the commonly used threshold value of 0.05.

Tables 2 and 3 contain the results of the two simulation studies. Columns A and B report the integer part of the average absolute relative difference, multiplied by 100, between the Koziol measure of kurtosis and the value which it is expected to attain in the normal case, for the original and the projected variables, respectively. Column C reports the integer part of the average p values, multiplied by 100, corresponding to the Koziol measure of multivariate kurtosis computed for the projected variables. The subtables denoted with \(d-1\), d/2 and \(\sqrt{d }\) report the simulation’s results corresponding to a number of projections equal to the number of variables minus one, half the number of variables and the integer part of the square root of the number of variables, respectively.

The subtables of Table 2 with headers “Leptokurtic” and “Platykurtic” report the results for samples simulated from mixtures where the weight of the standard normal component are 0.1 and 0.5, respectively. The results of the first simulation, related to homoscedastic normal mixtures, may be summarized as follows. The ratio statistic is always much smaller for the projected data than for the original ones. It increases (decreases) with the number of variables when \(d-1\) (\(\sqrt{d}\)) mesokurtic projections are seeked. When d/2 mesokurtic projections are seeked, the ratio statistic first decreases and then increases. Normality of the projections, as measured by Koziol’s kurtosis, is better achieved when their number is small compared to the number of variables, and the latter is small, either.

The subtables of Table 3 with headers “Concentrated” and “Dispersed” report the results for samples simulated from mixtures where the outliers generated from the model are more concentrated and more dispersed, respectively. The proposed method appears to perform better in the second simulation study. The differences between the ratio statistics for the original and the projected variables are always greater in the second simulation study. The performances of the proposed method are worst and best when \(d-1\) and \(\sqrt{d}\) mesokurtic projections are seeked, and when the number of original variables is small. The performances of mesokurtic transformations appear to be similar in the presence of concentrated and dispersed outliers. Similar remarks hold when considering the p values of Koziol’s kurtosis tests for the projected variables.

We conclude that the proposed method succeeds in either removing or alleviating kurtosis. It is particularly successful when the number of projections is moderate with respect to the number of variables, and the latter is moderate, too. As a major drawback, it may not detect all mesokurtic projections, especially in high dimensions.

In this section we illustrate the use of mesokurtic projections in projection pursuit, as described in Sect. 2, with the data collected from the Australian Institute of Sport by Telford and Cunningham (1991), also known as AIS dataset. It contains eleven blood and body measurements from 202 athletes of both genders competing in different sport events: red blood cell count, white blood cell count, hematocrit, hemoglobin concentration, plasma ferritins, body mass index, sum of skin folds, body fat, lean body mass, height and weight. Here, we focus on the 23 female athletes playing netball, so that the number of units is about twice the number of variables.

Nine variables in the dataset are mildly platykurtic, while the remaining two variables are mildly leptokurtic: their kurtoses range from 2.079 to 4.681, with just two of them being greater than 3. We assess the joint normality of the eleven variables by means of Koziol’s kurtosis tests for multivariate normality, whose p value is virtually zero. The maximum number of jointly mesokurtic projection detected by the method described in Sect. 2 is three (the p value of Koziol’s kurtosis test for multivariate normality is greater than 0.17), which is much smaller than the number of original variables. Further dimension reduction might be achieved by looking for two sets of mutually orthogonal projections, one nearly mesokurtic and the other containing the information on the interesting structure.

Mild nonnormality might be due to the presence of outliers, often detectable by means of visual inspection. The boxplots in Fig. 1 suggest the presence of several potential outliers, represented by the asterisks above and below the whiskers. The third, ninth and eleventh variables correspond to the third, ninth and eleventh boxplots starting from the left of Fig. 1. They hint that the potential outliers might be the first, the second and the fifteenth units. On the other hand, the Healy’s plot in Fig. 1 clearly suggests that the twenty-first unit is the most likely to be an outlier, since it has the highest Mahalanobis distance from the mean and it is represented by the point farthest away from the bisector line. We conclude that visual inspection of the original variables gives contradictory results about the presence of outliers (Fig. 2).

We addressed outlier detection by means of kurtosis maximization, as suggested by several authors (Peña and Prieto 2001a; Hou and Wentzell 2014; Loperfido 2020b). The projection with maximal kurtosis is represented by the last boxplot from the left in Fig. 3, which clearly shows the presence of a single outlier, that is the twenty-first observation. Then we performed kurtosis maximization on the projections orthogonal to those with small absolute fourth cumulants, as computed with the method described in Sect. 2. The first, second, third, fourth and fifth boxplots from the left of Fig. 3 represent the projections with maximal kurtoses orthogonal to 10, 9, 8, 7 and 6 nearly mesokurtic projections. All boxplots clearly suggest that the twenty-first unit is the most likely to be an outlier and all but the first two boxplots on the left suggest that it is the only potential outlier. All projections maximizing kurtosis are strongly correlated with each other: the correlations between them are never smaller than 0.93. This statistical analysis is exploratory in nature and more sophisticated, inferential methods are needed to assess the presence of outliers. This section is just aimed at showing how mesokurtic projections might be helpful for dimension reduction while retaining the relevant data structure to be uncovered by kurtosis-based projection pursuit.

In simulation studies, a normal random sample is often obtained by applying the normal quantile transformation to data generated from a uniform distribution. The method might lead to unsatisfactory results, even if visual and formal procedures support the hypothesis of the randomly generated data being uniform. We illustrate the problem with the RANDU data and show how mesokurtic projections might be helpful in alleviating it. The same dataset provides a good case of mesokurtic projections virtually identical to those with extreme kurtosis.

The RANDU dataset contains 400 cases and 3 variables. Each datum was generated by the RANDU multiplicative congruential scheme \( x_{n+1}=(2^{16}+3)x_{n}\) (mod \(2^{31}\)). RANDU data satisfy the constraint \( x_{n}-6x_{n+1}+x_{n+2}\equiv 0\) (mod \(2^{31}\)), so that all triplets \(\left( x_{n},x_{n+1},x_{n+2}\right) \) lie on 15 parallel lines through the unit cube. This structure might be detected by means of appropriate rotations. Despite that, each variable in the dataset appear to be generated from a uniform distribution in the interval [0, 1]. For example, their means, standard deviations, skewnesses and kurtoses (reported in Table 4) are very close to their expected values 0.5, 0.2889, 0 and 1.8, respectively. These features made the RANDU dataset a perfect case for projection pursuit (see, for example, Huber 1985).

Since each variable appears to be uniformly distributed, it is natural to trasform them to normality by applying the normal quantile transformation. Unfortunately, this method fails to achieve normality, as it is apparent from their univariate kurtoses: 3.626, 3.449, 3.795. The p values of the kurtosis-based tests for normality of the three transformed variables are smaller than 0.05. The Box-Cox transformation gives even worse results. It leads to variables which are significantly platykurtic: their kurtoses are 2.056, 2.026, 2.117. The p values of both the skewness and the kurtosis tests for normality of the three transformed variables are smaller than 0.05.

Finally, we applied the proposed method to obtain two data projections. The first one is virtually normal (its skewness and kurtosis are \(-0.0359\) and 2.9815), while the second one is only slightly so (its skewness and kurtosis are \(-0.1784\) and 2.5485). The p values of the kurtosis tests for testing the normality of the first and second projections are 0.94 and 0.0654, respectively. The two projections are not jointly normal: the \(p-\) value of the Koziol test for multivariate normality is 0.0067. The histograms (Figs. 4 and 5) and the scatterplot of the two projections (Fig. 6) are consistent with these findings.

Interestingly, the first projection coincides, up to location and scale changes, with the projection achieving maximal kurtosis. Hence the RANDU dataset provides a good example of kurtosis-based projection pursuit as a tool for recovering normality, rather than detecting nonnormal features.

The empirical findings in this section might be summarized as follows. We applied the proposed method to a well-known, three-dimensional, platykurtic dataset, obtaining a normal univariate projection and a slightly nonnormal bivariate projection. We therefore succeeded in achieving univariate normality and in alleviating bivariate nonnormality. Commonly used, componentwise, nonlinear methods failed in both recovering either univariate or bivariate normality. We conclude that the RANDU dataset supports the use of mesokurtic projections when a randomly generated normal sample is seeked.

The paper describes a method for obtaining data projection with null or negligible fourth cumulants. It uses several kurtosis matrices to establish either sufficient or necessary conditions for the existence of mesokurtic projections. It also relates mesokurtic projections to spectra of kurtosis matrices, thus easing inferential tasks. Mesokurtic projections address the problems arising in the application of kurtosis-based projection pursuit to datasets where the number of variables is only slightly smaller than the number of units. The proposed method has been implemented in the R package Kurt (Franceschini and Loperfido 2020).

Data preprocessing is a default practice in projection pursuit (see, for example, Jones and Sibson (1987)). Laa and Cook (2020) proposed to remove skewness from the data when preprocessing them, which can be done by means of linear projections (Loperfido 2014; Franceschini and Loperfido 2019). We recommend the use of mesokurtic projections after linear projections to symmetry when preprocessing the data. Centering, sphering, symmetrizing and mesokurtic projections conveniently address the first, second, third and fourth sample cumulants.

Mesokurtic projections might also be used in independent component analysis, a multivariate statistical method aimed at recovering independent random variables from the observed data which are their one-to-one affine transformations. The model is identifiable if at most one independent variable is normal (Miettinen et al. 2015). The nonnormal independent variables are commonly assumed to be leptokurtic, so that fourth cumulants may be used to recover them (Girolami and Fyfe 1996). The same assumption implies that the model is identifiable if there is at most one linear combination of the observed variables which is mesokurtic. Hence model identification is related to the rank of the matrix which characterizes mesokurtic projections.

Mesokurtic projections might also preprocess the data before using a statistical method whose performance heavily depends on the sampled distribution having null or at least negligible cumulants. Examples include inference on covariance matrices (Yanagihara et al. 2005), linear discriminant analysis (Arevalillo and Navarro 2012), multivariate linear regression (Yanagihara 2007) and variogram estimation (Rezvandehy and Deutsch 2018). Mesokurtic projections might provide a viable alternative to other statistical methods which are commonly used to address these problems: flexible modelling, nonparametric inference and nonlinear transformations.

Despite its theoretical properties and practical usefulness, the proposed method suffers from several drawbacks. In the first place, mesokurtic projections do not exist for some distributions, as for example scale mixtures of multivariate normal distributions. In the second place, the proposed method may not detect all mesokurtic projections, especially in high dimensional cases. In the third place, mesokurtic projections are not appropriate when the sampled distribution is nonnormal, but all its fourth cumulants equal zero. In the fourth place, mesokurtic projections imply some information loss, since the original data are projected onto a lower dimensional space.

A natural question to ask is whether the proposed method might be extended to datasets with more variables than units. Both Propositions 1, 2 rely on the standardized random variable \(z=\varSigma ^{-1/2}\left( x-\mu \right) \), which is defined only if the number of units is greater than the number of variables. In the opposite case, a common solution is the replacement of the covariance matrix \(\varSigma \) with the positive definite convex linear combination \(\varGamma =\lambda \varSigma +\left( 1-\lambda \right) \varOmega \) of \(\varSigma \) itself with a full rank matrix \( \varOmega \) of the same size, where \(0<\lambda <1\) (see, for example, Hastie et al. 2008). This approach already appeared in the literature on projection pursuit (Lee and Cook 2010; Hui and Lindsay 2010). We are currently investigating the generalizations of Propositions 1,  2 to situations where \(z=\varSigma ^{-1/2}\left( x-\mu \right) \) is replaced by \(y=\varGamma ^{-1/2}\left( x-\mu \right) \), where \(\varGamma ^{-1/2}\) is the positive definite symmetric square root of the inverse \(\varGamma ^{-1}\) of \(\varGamma \).