Skip to main content
Top

2020 | OriginalPaper | Chapter

Multiple Testing Procedures: Monotonicity and Some of Its Implications

Author : Alexander Y. Gordon

Published in: Statistical Modeling for Biological Systems

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

We review some results concerning the levels at which multiple testing procedures (MTPs) control certain type I error rates under a general and unknown dependence structure of the p-values on which the MTP is based. The type I error rates we deal with are (1) the classical family-wise error rate (FWER); (2) its immediate generalization: the probability of k or more false rejections (the generalized FWER); (3) the per-family error rate—the expected number of false rejections (PFER). The procedures considered are those satisfying the condition of monotonicity: reduction in some (or all) of the p-values used as input for the MTP can only increase the number of rejected hypotheses. It turns out that this natural condition, either by itself or combined with a property of being a step-down or step-up MTP (where the terms “step-down” and “step-up” are understood in their most general sense), has powerful consequences. Those include optimality results, inequalities, and identities involving different numerical characteristics of a procedure, and computational formulas.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 57, 289–300.MathSciNetMATH Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 57, 289–300.MathSciNetMATH
2.
go back to reference Benjamini, Y., & Liu, W. (1999). A distribution-free multiple test procedure that controls the false discovery rate. Technical Report. RP-SOR-99-3, Department of Statistics and Operations Research, Tel Aviv University. Benjamini, Y., & Liu, W. (1999). A distribution-free multiple test procedure that controls the false discovery rate. Technical Report. RP-SOR-99-3, Department of Statistics and Operations Research, Tel Aviv University.
3.
go back to reference Dudoit, S., Shaffer, J. P., & Boldrich, J. C. (2003). Multiple hypothesis testing in microarray experiments. Statistical Science, 18, 71–103.MathSciNetCrossRef Dudoit, S., Shaffer, J. P., & Boldrich, J. C. (2003). Multiple hypothesis testing in microarray experiments. Statistical Science, 18, 71–103.MathSciNetCrossRef
4.
go back to reference Dudoit, S., & van der Laan, M. J. (2008). Multiple testing procedures with applications to genomics. New York, NY: Springer.CrossRef Dudoit, S., & van der Laan, M. J. (2008). Multiple testing procedures with applications to genomics. New York, NY: Springer.CrossRef
5.
go back to reference Finner, H., & Roters, M. (2002). Multiple hypothesis testing and expected number of type I errors. The Annals of Statistics, 30, 220–238.MathSciNetCrossRef Finner, H., & Roters, M. (2002). Multiple hypothesis testing and expected number of type I errors. The Annals of Statistics, 30, 220–238.MathSciNetCrossRef
6.
go back to reference Gordon, A. Y. (2007a). Explicit formulas for generalized family-wise error rates and unimprovable step-down multiple testing procedures. Journal of Statistical Planning and Inference, 137, 3497–3512.MathSciNetCrossRef Gordon, A. Y. (2007a). Explicit formulas for generalized family-wise error rates and unimprovable step-down multiple testing procedures. Journal of Statistical Planning and Inference, 137, 3497–3512.MathSciNetCrossRef
7.
go back to reference Gordon, A. Y. (2007b). Family-wise error rate of a step-down procedure. Random Operators and Stochastic Equations, 15, 399–408.MathSciNetCrossRef Gordon, A. Y. (2007b). Family-wise error rate of a step-down procedure. Random Operators and Stochastic Equations, 15, 399–408.MathSciNetCrossRef
8.
go back to reference Gordon, A. Y. (2007c). Unimprovability of the Bonferroni procedure in the class of general step-up multiple testing procedures. Statistics & Probability Letters, 77, 117–122.MathSciNetCrossRef Gordon, A. Y. (2007c). Unimprovability of the Bonferroni procedure in the class of general step-up multiple testing procedures. Statistics & Probability Letters, 77, 117–122.MathSciNetCrossRef
9.
go back to reference Gordon, A. Y. (2009). Inequalities between generalized familywise error rates of a multiple testing procedure. Statistics & Probability Letters, 79, 1996–2004.MathSciNetCrossRef Gordon, A. Y. (2009). Inequalities between generalized familywise error rates of a multiple testing procedure. Statistics & Probability Letters, 79, 1996–2004.MathSciNetCrossRef
10.
go back to reference Gordon, A. Y. (2011). A new optimality property of the Holm step-down procedure. Statistical Methodology, 8, 129–135.MathSciNetCrossRef Gordon, A. Y. (2011). A new optimality property of the Holm step-down procedure. Statistical Methodology, 8, 129–135.MathSciNetCrossRef
11.
go back to reference Gordon, A. Y. (2012). A sharp upper bound for the expected number of false rejections. Statistics & Probability Letters, 82, 1507–1514.MathSciNetCrossRef Gordon, A. Y. (2012). A sharp upper bound for the expected number of false rejections. Statistics & Probability Letters, 82, 1507–1514.MathSciNetCrossRef
12.
go back to reference Gordon, A. Y., & Salzman, P. (2008). Optimality of the Holm procedure among general step-down multiple testing procedures. Statistics & Probability Letters, 78, 1878–1884.MathSciNetCrossRef Gordon, A. Y., & Salzman, P. (2008). Optimality of the Holm procedure among general step-down multiple testing procedures. Statistics & Probability Letters, 78, 1878–1884.MathSciNetCrossRef
13.
14.
go back to reference Hochberg, Y., & Tamhane, A. C. (1987). Multiple comparison procedures. New York, NY: Wiley.CrossRef Hochberg, Y., & Tamhane, A. C. (1987). Multiple comparison procedures. New York, NY: Wiley.CrossRef
15.
go back to reference Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65–70.MathSciNetMATH Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65–70.MathSciNetMATH
16.
go back to reference Hommel, G., & Hoffman, T. (1988). Controlled uncertainty. In P. Bauer, G. Hommel, & E. Sonnemann (Eds.), Multiple hypothesis testing (pp. 154–161). Heidelberg: Springer.CrossRef Hommel, G., & Hoffman, T. (1988). Controlled uncertainty. In P. Bauer, G. Hommel, & E. Sonnemann (Eds.), Multiple hypothesis testing (pp. 154–161). Heidelberg: Springer.CrossRef
17.
go back to reference Korn, E. L., Troendle, J. F., McShane, L. M., & Simon, R. (2004). Controlling the number of false discoveries: application to high-dimensional genomic data. Journal of Statistical Planning and Inference, 124, 379–398.MathSciNetCrossRef Korn, E. L., Troendle, J. F., McShane, L. M., & Simon, R. (2004). Controlling the number of false discoveries: application to high-dimensional genomic data. Journal of Statistical Planning and Inference, 124, 379–398.MathSciNetCrossRef
18.
go back to reference Lehmann, E. L., & Romano, J. P. (2005a). Generalizations of the familywise error rate. The Annals of Statistics, 33, 1138–1154.MathSciNetCrossRef Lehmann, E. L., & Romano, J. P. (2005a). Generalizations of the familywise error rate. The Annals of Statistics, 33, 1138–1154.MathSciNetCrossRef
19.
go back to reference Lehmann, E. L., & Romano, J. P. (2005b). Testing statistical hypotheses (3rd ed.). New York, NY: Springer.MATH Lehmann, E. L., & Romano, J. P. (2005b). Testing statistical hypotheses (3rd ed.). New York, NY: Springer.MATH
20.
go back to reference Liu, W. (1996). Multiple tests of a non-hierarchical family of hypotheses. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 58, 455–461.MathSciNetMATH Liu, W. (1996). Multiple tests of a non-hierarchical family of hypotheses. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 58, 455–461.MathSciNetMATH
21.
go back to reference Sarkar, S. K. (2002). Some results on false discovery rate in stepwise multiple testing procedures. The Annals of Statistics, 30, 239–257.MathSciNetCrossRef Sarkar, S. K. (2002). Some results on false discovery rate in stepwise multiple testing procedures. The Annals of Statistics, 30, 239–257.MathSciNetCrossRef
22.
go back to reference Shaffer, J. (1995). Multiple hypothesis testing: A review. Annual Review of Psychology, 46, 561–584.CrossRef Shaffer, J. (1995). Multiple hypothesis testing: A review. Annual Review of Psychology, 46, 561–584.CrossRef
24.
go back to reference Tamhane, A. C., Liu, W., & Dunnett, C. W. (1998). A generalized step-up-down multiple test procedure. The Canadian Journal of Statistics, 26, 353–363.MathSciNetCrossRef Tamhane, A. C., Liu, W., & Dunnett, C. W. (1998). A generalized step-up-down multiple test procedure. The Canadian Journal of Statistics, 26, 353–363.MathSciNetCrossRef
25.
go back to reference Tukey, J. W. (1953). The problem of multiple comparison. Unpublished manuscript. In The collected works of John W. Tukey VIII. Multiple comparisons: 1948–1983 (pp. 1–300). New York, NY: Chapman and Hall. Tukey, J. W. (1953). The problem of multiple comparison. Unpublished manuscript. In The collected works of John W. Tukey VIII. Multiple comparisons: 1948–1983 (pp. 1–300). New York, NY: Chapman and Hall.
26.
go back to reference van der Laan, M. J., Dudoit, S., & Pollard, K. S. (2004). Augmentation procedures for control of the generalized family-wise error rate and tail probabilities for the proportion of false positives. Statistical Applications in Genetics and Molecular Biology, 3, Article 15. van der Laan, M. J., Dudoit, S., & Pollard, K. S. (2004). Augmentation procedures for control of the generalized family-wise error rate and tail probabilities for the proportion of false positives. Statistical Applications in Genetics and Molecular Biology, 3, Article 15.
27.
go back to reference Victor, N., (1982). Exploratory data analysis and clinical research. Methods of Information in Medicine, 21, 53–54.CrossRef Victor, N., (1982). Exploratory data analysis and clinical research. Methods of Information in Medicine, 21, 53–54.CrossRef
28.
go back to reference Yang, H. Y., & Speed, T. (2003). Design and analysis of comparative microarray experiments. In T. Speed (ed.), Statistical analysis of gene expression microarray data (pp. 35–92). Boca Raton, FL: Chapman and Hall. Yang, H. Y., & Speed, T. (2003). Design and analysis of comparative microarray experiments. In T. Speed (ed.), Statistical analysis of gene expression microarray data (pp. 35–92). Boca Raton, FL: Chapman and Hall.
Metadata
Title
Multiple Testing Procedures: Monotonicity and Some of Its Implications
Author
Alexander Y. Gordon
Copyright Year
2020
DOI
https://doi.org/10.1007/978-3-030-34675-1_5

Premium Partner