×

Formalized data snooping based on generalized error rates. (English) Zbl 1284.62789

Summary: It is common in econometric applications that several hypothesis tests are carried out simultaneously. The problem then becomes how to decide which hypotheses to reject, accounting for the multitude of tests. The classical approach is to control the familywise error rate (FWE), which is the probability of one or more false rejections. But when the number of hypotheses under consideration is large, control of the FWE can become too demanding. As a result, the number of false hypotheses rejected may be small or even zero. This suggests replacing control of the FWE by a more liberal measure. To this end, we review a number of recent proposals from the statistical literature. We briefly discuss how these procedures apply to the general problem of model selection. A simulation study and two empirical applications illustrate the methods.

MSC:

62P20 Applications of statistics to economics
62F03 Parametric hypothesis testing
62H15 Hypothesis testing in multivariate analysis

Software:

PcGets
Full Text: DOI

References:

[1] DOI: 10.1198/016214504000000539 · Zbl 1117.62416 · doi:10.1198/016214504000000539
[2] Holm, Scandinavian Journal of Statistics 6 pp 65– (1979)
[3] DOI: 10.1214/009053606000000461 · Zbl 1246.62172 · doi:10.1214/009053606000000461
[4] DOI: 10.1214/aos/1013699998 · Zbl 1041.62061 · doi:10.1214/aos/1013699998
[5] Benjamini, Journal of Educational and Behavioral Statistics 25 pp 60– (2000) · doi:10.3102/10769986025001060
[6] DOI: 10.1214/aos/1176346253 · Zbl 0521.62072 · doi:10.1214/aos/1176346253
[7] Benjamini, Journal of the Royal Statistical Society Series B 57 pp 289– (1995)
[8] DOI: 10.1080/02331888808802068 · Zbl 0644.62024 · doi:10.1080/02331888808802068
[9] DOI: 10.2307/2951574 · Zbl 0778.62103 · doi:10.2307/2951574
[10] DOI: 10.2469/faj.v58.n4.2453 · doi:10.2469/faj.v58.n4.2453
[11] Abramovich, Annals of Statistics 22 pp 351– (2005)
[12] DOI: 10.1214/009053605000000084 · Zbl 1072.62060 · doi:10.1214/009053605000000084
[13] DOI: 10.1016/0167-9473(96)00003-5 · doi:10.1016/0167-9473(96)00003-5
[14] DOI: 10.1016/S0165-1889(00)00058-0 · Zbl 0978.91073 · doi:10.1016/S0165-1889(00)00058-0
[15] DOI: 10.1046/j.0305-9049.2003.00086.x · doi:10.1046/j.0305-9049.2003.00086.x
[16] DOI: 10.1016/S0304-4076(01)00077-X · Zbl 1040.62095 · doi:10.1016/S0304-4076(01)00077-X
[17] DOI: 10.1198/073500105000000063 · doi:10.1198/073500105000000063
[18] DOI: 10.1111/0022-1082.00163 · doi:10.1111/0022-1082.00163
[19] DOI: 10.1214/aos/1069362303 · Zbl 0906.62040 · doi:10.1214/aos/1069362303
[20] DOI: 10.1111/j.1467-9868.2004.00439.x · Zbl 1061.62110 · doi:10.1111/j.1467-9868.2004.00439.x
[21] DOI: 10.1214/009053604000000283 · Zbl 1092.62065 · doi:10.1214/009053604000000283
[22] DOI: 10.1214/aos/1074290335 · Zbl 1042.62026 · doi:10.1214/aos/1074290335
[23] DOI: 10.1214/aos/1176344552 · Zbl 0406.62024 · doi:10.1214/aos/1176344552
[24] DOI: 10.1111/1467-9868.00346 · Zbl 1090.62073 · doi:10.1111/1467-9868.00346
[25] DOI: 10.1023/A:1003483128844 · Zbl 0898.62091 · doi:10.1023/A:1003483128844
[26] DOI: 10.1198/016214504000001097 · Zbl 1117.62423 · doi:10.1198/016214504000001097
[27] DOI: 10.1214/ss/1056397487 · Zbl 1048.62099 · doi:10.1214/ss/1056397487
[28] DOI: 10.1214/009053606000001622 · Zbl 1127.62063 · doi:10.1214/009053606000001622
[29] DOI: 10.1080/10485250600687812 · Zbl 1099.62102 · doi:10.1080/10485250600687812
[30] DOI: 10.1111/j.1468-0262.2005.00615.x · Zbl 1153.62310 · doi:10.1111/j.1468-0262.2005.00615.x
[31] DOI: 10.1016/S0378-3758(03)00211-8 · Zbl 1074.62070 · doi:10.1016/S0378-3758(03)00211-8
[32] DOI: 10.1198/016214505000001140 · Zbl 1119.62322 · doi:10.1198/016214505000001140
[33] DOI: 10.1023/A:1007631014630 · Zbl 0954.68083 · doi:10.1023/A:1007631014630
[34] DOI: 10.1111/1468-0262.00152 · Zbl 1008.62116 · doi:10.1111/1468-0262.00152
[35] Buena, Journal of Statistical Planning and Inference 29 pp 1165– (2006)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.