Multiple test functions and adjusted \(p\)-values for test statistics with discrete distributions. (English) Zbl 1326.62043
Summary: The randomized \(p\)-value, (nonrandomized) mid-\(p\)-value and abstract randomized \(p\)-value have all been recommended for testing a null hypothesis whenever the test statistic has a discrete distribution. This paper provides a unifying framework for these approaches and extends it to the multiple testing setting. In particular, multiplicity adjusted versions of the aforementioned \(p\)-values and multiple test functions are developed. It is demonstrated that, whenever the usual nonrandomized and randomized decisions to reject or retain the null hypothesis may differ, the (adjusted) abstract randomized \(p\)-value and test function should be reported, especially when the number of tests is large. It is shown that the proposed approach dominates the traditional randomized and nonrandomized approaches in terms of bias and variability. Tools for plotting adjusted abstract randomized \(p\)-values and for computing multiple test functions are developed. Examples are used to illustrate the method and to motivate a new type of multiplicity adjusted mid-\(p\)-value.
MSC:
| 62F03 | Parametric hypothesis testing |
| 62J15 | Paired and multiple comparisons; multiple testing |
| 62L10 | Sequential statistical analysis |
Keywords:
abstract randomized \(p\)-value; adjusted \(p\)-value; decision function; false discovery rate; fuzzy \(p\)-value; mid-\(p\)-value; randomized \(p\)-value; sequential hypothesis testing; test functionReferences:
| [1] | Agresti, A., A survey of exact inference for contingency tables, Statist. Sci., 7, 131-153 (1992) · Zbl 0955.62587 |
| [2] | Agresti, A.; Gottard, A., Nonconservative exact small-sample inference for discrete data, Comput. Statist. Data Anal., 51, 6447-6458 (2007) · Zbl 1445.62051 |
| [3] | Agresti, A.; Min, Y., On small-sample confidence intervals for parameters in discrete distributions, Biometrics, 57, 963-971 (2001) · Zbl 1209.62041 |
| [4] | Barnard, G. A., On the alleged gains in power from lower \(P\)-values, Statist. Med., 8, 1469-1477 (1989) |
| [5] | Benjamini, Y.; Hochberg, Y., Controlling the false discovery rate: a practical and powerful approach to multiple testing, J. R. Stat. Soc. Ser. B Stat. Methodol., 57, 289-300 (1995) · Zbl 0809.62014 |
| [6] | Benjamini, Y.; Yekutieli, D., The control of the false discovery rate in multiple testing under dependency, Ann. Statist., 29, 1165-1188 (2001) · Zbl 1041.62061 |
| [7] | Blanchard, G.; Roquain, E., Adaptive FDR control under independence and dependence, J. Mach. Learn. Res., 10, 2837-2871 (2009) · Zbl 1235.62093 |
| [8] | Blyth, C. R.; Staudte, R. G., Estimating statistical hypotheses, Statist. Probab. Lett., 23, 45-52 (1995) · Zbl 0819.62018 |
| [9] | Cox, D. R.; Hinkley, D. V., Theoretical Statistics (1974), Chapman and Hall: Chapman and Hall London · Zbl 0334.62003 |
| [11] | Dickhaus, T., Randomized p-values for multiple testing of composite null hypotheses, J. Statist. Plann. Inference, 143, 1968-1979 (2013) · Zbl 1279.62150 |
| [12] | Dudoit, S.; van der Laan, M. J., Multiple testing procedures with applications to genomics, (Springer Series in Statistics (2008), Springer: Springer New York) · Zbl 1261.62014 |
| [13] | Efron, B., Correlation and large-scale simultaneous significance testing, J. Amer. Statist. Assoc., 102, 93-103 (2007) · Zbl 1284.62340 |
| [14] | Efron, B., Large-scale inference, (Institute of Mathematical Statistics (IMS) Monographs, vol. 1 (2010), Cambridge University Press: Cambridge University Press Cambridge), Empirical Bayes methods for estimation, testing, and prediction · Zbl 1277.62016 |
| [15] | Fan, J.; Han, X.; Gu, W., Estimating false discovery proportion under arbitrary covariance dependence, J. Amer. Statist. Assoc., 107, 1019-1035 (2012) · Zbl 1395.62219 |
| [16] | Farcomeni, A., A review of modern multile hypothesis testing, with particular attention to the false discovery proportion, Stat. Methods Med. Res., 17, 347-388 (2008) · Zbl 1156.62048 |
| [17] | Geyer, C. J.; Meeden, G. D., Fuzzy and randomized confidence intervals and \(P\)-values, Statist. Sci., 20, 358-387 (2005), With comments and a rejoinder by the authors · Zbl 1130.62319 |
| [18] | Gilbert, P. B., A modified false discovery rate multiple-comparisons procedure for discrete data, applied to human immunodeficiency virus genetics, J. R. Stat. Soc. Ser. C. Appl. Stat., 54, 143-158 (2005) · Zbl 1490.62358 |
| [19] | Gutman, R.; Hochberg, Y., Improved multiple test procedures for discrete distributions: new ideas and analytical review, J. Statist. Plann. Inference, 137, 2380-2393 (2007) · Zbl 1120.62054 |
| [20] | Habiger, J. D., A method for modifying multiple testing procedures, J. Statist. Plann. Inference, 142, 2227-2231 (2012) · Zbl 1237.62091 |
| [21] | Habiger, J.; Peña, E., Randomized p-values and nonparametric procedures in multiple testing, J. Nonparametr. Stat., 23, 583-604 (2011) · Zbl 1228.62088 |
| [22] | Holm, S., A simple sequentially rejective multiple test procedure, Scand. J. Stat., 6, 65-70 (1979) · Zbl 0402.62058 |
| [23] | Kulinskaya, E.; Lewin, A., On fuzzy familywise error rate and false discovery rate procedures for discrete distributions, Biometrika, 96, 201-211 (2009) · Zbl 1163.62055 |
| [24] | Lancaster, H. O., Significance tests in discrete distributions, J. Amer. Statist. Assoc., 56, 223-234 (1961) · Zbl 0104.13201 |
| [25] | Lehmann, E. L.; Romano, J. P.; Shaffer, J. P., On optimality of stepdown and stepup multiple test procedures, Ann. Statist., 33, 1084-1108 (2005) · Zbl 1073.62063 |
| [26] | Neyman, J.; Pearson, E., On the problem of the most efficient tests of statistical hypotheses, Phil. Trans. R. Soc. A, 231, 289-337 (1933) · JFM 59.1163.02 |
| [27] | Pearson, E. S., On questions raised by the combination of tests based on discontinuous distributions, Biometrika, 37, 383-398 (1950) · Zbl 0040.07503 |
| [28] | Peña, E.; Habiger, J.; Wu, W., Power-enhanced multiple decision functions controlling family-wise error and false discovery rates, Ann. Statist., 39, 556-583 (2011) · Zbl 1274.62143 |
| [29] | Pratt, J., Length of Confidence Intervals, J. Amer. Statist. Assoc., 56, 549-567 (1961) · Zbl 0099.14002 |
| [30] | Robert, C. P.; Casella, G., Monte Carlo statistical methods, (Springer Texts in Statistics (2004), Springer-Verlag: Springer-Verlag New York) · Zbl 1096.62003 |
| [31] | Roth, A. J., Multiple comparison procedures for discrete test statistics, J. Statist. Plann. Inference, 82, 101-117 (1999) · Zbl 1079.62527 |
| [32] | Sarkar, S. K., Some results on false discovery rate in stepwise multiple testing procedures, Ann. Statist., 30, 239-257 (2002) · Zbl 1101.62349 |
| [33] | Sarkar, S. K., Stepup procedures controlling generalized FWER and generalized FDR, Ann. Statist., 35, 2405-2420 (2007) · Zbl 1129.62066 |
| [34] | Storey, J. D., A direct approach to false discovery rates, J. R. Stat. Soc. Ser B Stat. Methodol., 64, 479-498 (2002) · Zbl 1090.62073 |
| [35] | Storey, J. D.; Taylor, J. E.; Siegmund, D., Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach, J. R. Stat. Soc. Ser B Stat. Methodol., 66, 187-205 (2004) · Zbl 1061.62110 |
| [36] | Tamhane, A. C.; Liu, W.; Dunnett, C. W., A generalized step-up-down multiple test procedure, Canad. J. Statist., 26, 353-363 (1998) · Zbl 0914.62013 |
| [37] | Tarone, R. E., A modified bonferroni method for discrete data, Biometrics, 46, 515-522 (1990) · Zbl 0715.62140 |
| [38] | Timmons, J. A.; Wennmalm, K.; Larsson, O.; Walden, T. B.; Lassmann, T.; Petrovic, N.; Hamilton, D. L.; Gimeno, R. E.; Wahlestedt, C.; Baar, K.; Nedergaard, J.; Cannon, B., Myogenic gene expresion signature establishes that brown and white adipocytes originate from distinct cell lineages, Proc. Natl. Acad. Sci. USA, 104 (11), 4401-4406 (2007) |
| [39] | Tocher, K. D., Extension of the Neyman-Pearson theory of tests to discontinuous variates, Biometrika, 37, 130-144 (1950) · Zbl 0040.07502 |
| [40] | Westfall, P. H.; Wolfinger, R. D., Multiple tests with discrete distributions, Amer. Statist., 51, 3-8 (1997) |
| [41] | Westfall, P. H.; Young, S., Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment, (Wiley Series in Probability and Statistics (1993)) |
| [42] | Wilcoxon, F., Probability tables for individual comparisons by ranking methods, Biometrics, 3, 119-122 (1947) |
| [43] | Wright, S. P., Adjusted P-values for simultaneous inference, Biometrics, 48, 1005-1013 (1992) |
| [44] | Yang, M. C., The equivalence of the mid \(p\)-value and the expected \(p\)-value for testing equality of two balanced binomial proportions, J. Statist. Plann. Inference, 126, 273-280 (2004) · Zbl 1075.62017 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
