Test Statistics Null Distribution

A key feature of our proposed multiple testing procedures (MTP) is the test statistics null distribution (rather than a data generating null distribution) used to obtain rejection regions (i.e., cut-offs) for the test statistics, confidence regions for the parameters of interest, and adjusted p-values. Indeed, whether testing single or multiple hypotheses, one needs the (joint) distribution of the test statistics in order to derive a procedure that probabilistically controls Type I errors. In practice, however, the true distribution of the test statistics is unknown and replaced by a null distribution. The choice of a proper null distribution is crucial in order to ensure that (finite sample or asymptotic) control of the Type I error rate under the assumed null distribution does indeed provide the desired control under the true distribution. This issue is particularly relevant for large-scale testing problems, such as those encountered in biomedical and genomic research (Chapters 9–12), which concern high-dimensional multivariate distributions, with complex and unknown dependence structures among variables.