Stepwise multiple test procedures and control of directional errors. (English) Zbl 0978.62057
Summary: One of the most difficult problems occurring with stepwise multiple test procedures for a set of two-sided hypotheses is the control of directional errors if rejection of a hypothesis is accomplished with a directional decision. We generalize a result for so-called step-down procedures derived by J.P. Shaffer [ibid. 8, 1342-1347 (1980; Zbl 0484.62089)] to a large class of stepwise or closed multiple test procedures. In a unifying way we obtain results for a large class of order statistics procedures including step-down as well as step-up procedures, but also a procedure of G. Hommel [Biometrika 75, No. 2, 383-386 (1988; Zbl 0639.62025)] based on critical values derived by R.J. Simes [ibid. 73, 751-754 (1986; Zbl 0613.62067)].
Our method of proof is also applicable in situations where directional decisions are mainly based on conditionally independent \(t\)-statistics. A closed \(F\)-test procedure applicable in regression models with orthogonal design, the modified \(S\)-method of Scheffé applicable in the Analysis of Variance and Fisher’s LSD-test for the comparison of three means will be considered in more detail.
Our method of proof is also applicable in situations where directional decisions are mainly based on conditionally independent \(t\)-statistics. A closed \(F\)-test procedure applicable in regression models with orthogonal design, the modified \(S\)-method of Scheffé applicable in the Analysis of Variance and Fisher’s LSD-test for the comparison of three means will be considered in more detail.
MSC:
| 62J15 | Paired and multiple comparisons; multiple testing |
| 62F03 | Parametric hypothesis testing |
| 62F07 | Statistical ranking and selection procedures |
Keywords:
totally positive of order 3; closure principle; directional error; unimodality; familywise error rate; multiple comparisons; multiple hypotheses testing; multiple level of significance; type III error; variation diminishing property; stepwise multiple test procedures; step-down procedures; closed multiple test procedures; step-up procedures; F-testReferences:
| [1] | Bahadur, R. R. (1952). A property of the t-statistic. Sankhy\?a Ser. A 12 79-88. · Zbl 0049.10005 |
| [2] | Bauer, P., Hackl, P., Hommel, G. and Sonnemann, E. (1986). Multiple testing of pairs of onesided hypotheses. Metrika 33 121-127. · Zbl 0591.62071 · doi:10.1007/BF01894737 |
| [3] | Begun, J. M. and Gabriel, K. R. (1981). Closure of the Newman-Keuls multiple comparisons procedure. J. Amer. Statist. Assoc. 76 241-245. · doi:10.2307/2287817 |
| [4] | Brown, L. D., Johnstone, I. M. and MacGibbon, K. B. (1981). Variation diminishing transformations: a direct approach to total positivity and its statistical applications. J. Amer. Statist. Assoc. 76 824-832. JSTOR: · Zbl 0481.62021 · doi:10.2307/2287577 |
| [5] | Dalal, S. R. and Mallows, C. L. (1992). Buying with exact confidence. Ann. Appl. Probab. 2 752-765. Finner, H. (1988a). Abgeschlossene multiple Spannweitentests. In Multiple Hypothesenpr üfungen (P. Bauer, G. Hommel and E. Sonnemann, eds.) 10-32. Springer, Berlin. Finner, H. (1988b). Multiple tests und Fehler III. Art. In Multiple Hypothesenpr üfungen (P. Bauer, G. Hommel and E. Sonnemann, eds.) 144-153. Springer, Berlin. · Zbl 0753.62051 · doi:10.1214/aoap/1177005658 |
| [6] | Finner, H. (1990). On the modified S-method and directional errors. Comm. Statist. Theory Methods 19 41-53. Finner, H. (1994a). Two-sided tests and one-sided confidence bounds. Ann. Statist. 22 1502-1516. Finner, H. (1994b). Testing multiple hypotheses: general theory, specific problems, and relationships to other multiple decision procedures. Habilitationsschrift, Fachbereich IV Mathematik, Univ. Trier. · Zbl 0900.62098 · doi:10.1080/03610929008830186 |
| [7] | Fisher, R. A. (1935). The Design of Experiments. Oliver and Boyd, Edinburgh. · Zbl 0011.03205 |
| [8] | Games, P. A. (1973). Type IV errors revisited. Psychol. Bull. 80 304-307. |
| [9] | Hayter, A. J. and Hsu, J. C. (1994). On the relationship between stepwise decision procedures and confidence sets. J. Amer. Statist. Assoc. 89 128-136. · Zbl 0800.62182 · doi:10.2307/2291208 |
| [10] | Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75 800-802. Holm, S. (1979a). A simple sequentially rejective multiple test procedure. Scand. J. Statist. 6 65-70. Holm, S. (1979b). A stagewise directional test based on t statistics. Statistical Research Report 1979-3, Inst. Mathematics, Chalmers Univ. Technology, Gothenburg. JSTOR: · Zbl 0661.62067 · doi:10.1093/biomet/75.4.800 |
| [11] | Holm, S. (1981). A stagewise directional test for the normal regression situation. In Procedings of the Sixth Conference on Probability Theory (B. Bereanu, S. Grigorescu, M. Josifescu and T. Postelnicu, eds.) 103-106. National Institute of Metrology, Bucharest. · Zbl 0469.62058 |
| [12] | Hommel, G. (1988). A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika 75 383-386. · Zbl 0639.62025 · doi:10.1093/biomet/75.2.383 |
| [13] | Kaiser, H. F. (1960). Directional statistical decisions. Psychol. Rev. 67 160-167. |
| [14] | Karlin, S. (1968). Total Positivity. Stanford Univ. Press. · Zbl 0219.47030 |
| [15] | Keselman, H. J. and Murray, R. (1974). Tukey tests for pairwise contrasts following the analysis of variance: is there a type IV error? Psychol. Bull. 9 608-609. |
| [16] | Kimball, A. W. (1957). Errors of the third kind in statistical consulting. J. Amer. Statist. Assoc. 52 133-142. |
| [17] | Lehmann, E. L. (1950). Some principles of the theory of testing hypotheses. Ann. Math. Statist. 21 1-26. Lehmann, E. L. (1957a). A theory of some multiple decision problems I. Ann. Math. Statist. 28 1-25. Lehmann, E. L. (1957b). A theory of some multiple decision problems II. Ann. Math. Statist. 28 547-572. · Zbl 0036.09501 · doi:10.1214/aoms/1177729884 |
| [18] | Lehmann, E. L. and Shaffer, J. P. (1979). Optimum significance levels for multistage comparison procedures. Ann. Statist. 7 27-45. · Zbl 0401.62057 · doi:10.1214/aos/1176344553 |
| [19] | Levin, J. R. and Marasculio, L. A. (1972). Type IV errors and interactions. Psychol. Bull. 78 368-374. |
| [20] | Liebermann, B. (1971). Contemporary Problems in Statistics. Oxford Univ. Press. |
| [21] | Liu, W. (1996). Control of directional errors with step-up multiple tests. Statist. Probab. Lett. 31 239-242. · Zbl 0909.62079 · doi:10.1016/S0167-7152(96)00036-3 |
| [22] | Marasculio, L. A. and Levin, J. L. (1970). Appropriate post hoc comparisons for interaction and nested hypotheses in analysis of variance designs: the elimination of type IV errors. Amer. Educational Research J. 7 397-421. |
| [23] | Marcus, R., Peritz, E. and Gabriel, K. R. (1976). On closed testing procedures with special reference to ordered analysis of variance. Biometrika 63 655-660. JSTOR: · Zbl 0353.62037 · doi:10.1093/biomet/63.3.655 |
| [24] | Mosteller, F. (1948). A k-sample slippage test for an extreme population. Ann. Math. Statist. 19 58-65. · Zbl 0031.37102 · doi:10.1214/aoms/1177730290 |
| [25] | Rom, D. M. (1990). A sequentially rejective test procedure based on a modified Bonferroni inequality. Biometrika 77 663-665. JSTOR: · doi:10.1093/biomet/77.3.663 |
| [26] | Scheffé, H. (1970). Multiple testing versus multiple estimation. Improper confidence sets. Estimation of directions and ratios. Ann. Math. Statist. 41 1-29. · Zbl 0193.16201 · doi:10.1214/aoms/1177697184 |
| [27] | Scheffé, H. (1977). A note on a reformulation of the S-method of multiple comparison. J. Amer. Statist. Assoc. 72 143-146. JSTOR: · Zbl 0364.62074 · doi:10.2307/2286922 |
| [28] | Shaffer, J. P. (1980). Control of directional errors with stagewise multiple test procedures. Ann. Statist. 8 1342-1347. · Zbl 0484.62089 · doi:10.1214/aos/1176345205 |
| [29] | Simes, R. J. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika 73 751-754. JSTOR: · Zbl 0613.62067 · doi:10.1093/biomet/73.3.751 |
| [30] | Stefansson, G., Kim, W.-C. and Hsu, J. C. (1988). On confidence sets in multiple comparisons. In Statistical Decision Theory and Related Topics 4 (S. S. Gupta and J. O. Berger, eds.) 89-104. Academic Press, New York. · Zbl 0685.62034 |
| [31] | Tukey, J. W. (1953). The problem of multiple comparisons. Unpublished manuscript. |
| [32] | Tukey, J. W. (1994). The Collected Works of John W. Tukey. Volume VIII: Multiple Comparisons: 1948-1983 (H. I. Brown, B. Kaplan, K. M. Sheehan and M.-H. Wang, eds.). Chapman & Hall, New York. |
| [33] | Welsch, R. E. (1977). Stepwise multiple comparison procedures. J. Amer. Statist. Assoc. 72 566- 575. JSTOR: · Zbl 0369.62081 · doi:10.2307/2286218 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
