Placebo inference on treatment effects when the number of clusters is small. (English) Zbl 1456.62286
Summary: I introduce a general, Fisher-style randomization testing framework to conduct nearly exact inference about the lack of effect of a binary treatment in the presence of very few, large clusters when the treatment effect is identified across clusters. The proposed randomization test formalizes and extends the intuitive notion of generating null distributions by assigning placebo treatments to untreated clusters. I show that under simple and easily verifiable conditions, the placebo test leads to asymptotically valid inference in a very large class of empirically relevant models. Examples discussed explicitly are (i) least squares regression with cluster-level treatment, (ii) difference-in-differences estimation, and (iii) binary choice models with cluster-level treatment. A simulation study and an empirical example are provided. The proposed inference procedure is easy to implement and performs well with as few as three treated and three untreated clusters.
MSC:
| 62P20 | Applications of statistics to economics |
| 62M10 | Time series, auto-correlation, regression, etc. in statistics (GARCH) |
| 62G10 | Nonparametric hypothesis testing |
| 62G20 | Asymptotic properties of nonparametric inference |
References:
| [1] | Bell, R. M.; McCaffrey, D. F., Bias reduction in standard errors for linear regression with multi-stage samples, Surv. Methodol., 28, 169-181 (2002) |
| [2] | Bertrand, M.; Duflo, E.; Mullainathan, S., How much should we trust differences-in-differences estimates?, Q. J. Econ., 119, 249-275 (2004) · Zbl 1053.62132 |
| [3] | Bester, C. A.; Conley, T. G.; Hansen, C. B., Inference with dependent data using cluster covariance estimators, J. Econometrics, 165, 137-151 (2011) · Zbl 1441.62611 |
| [4] | Cameron, A. C.; Gelbach, J. B.; Miller, D. L., Bootstrap-based improvements for inference with clustered errors, Rev. Econom. Stat., 90, 414-427 (2008) |
| [5] | Cameron, A. C.; Miller, D. L., A practitoner’s guide to cluster robust inference, J. Hum. Resour., 50, 317-372 (2015) |
| [6] | Canay, I.; Romano, J. P.; Shaikh, A. M., Randomization tests under an approximate symmetry assumption, Econometrica, 85, 1013-1030 (2017) · Zbl 1410.62103 |
| [7] | Carter, A. V.; Schnepel, K. T.; Steigerwald, D. G., Asymptotic behavior of a \(t\) test robust to cluster heterogeneity, Rev. Econom. Stat., 99, 678-709 (2017) |
| [8] | Dal Bó, P.; Fréchette, G. R., The evolution of cooperation in infinitely repeated games: Experimental evidence, Amer. Econ. Rev., 101, 411-429 (2011) |
| [9] | Dedecker, J.; Doukhan, P.; Lang, G.; León, J. R.; Louhichi, S.; Prieur, C., Weak Dependence: With Examples and Applications (2007), Springer · Zbl 1165.62001 |
| [10] | Donald, S. G.; Lang, K., Inference with difference-in-differences and other panel data, Rev. Econom. Stat., 89, 221-233 (2007) |
| [11] | Durrett, R., Probability: Theory and Examples (2010), Cambridge University Press · Zbl 1202.60001 |
| [12] | El Machkouri, M.; Volný, D.; Wu, W. B., A central limit theorem for stationary random fields, Stochastic Process. Appl., 123, 1-14 (2013) · Zbl 1308.60025 |
| [13] | Fisher, R. A., “The coefficient of racial likeness” and the future of craniometry, J. R. Anthropol. Inst. G. B. Irel., 66, 57-63 (1935) |
| [14] | Hansen, C. B., Asymptotic properties of a robust variance matrix estimator for panel data when \(t\) is large, J. Econometrics, 141, 597-620 (2007) · Zbl 1418.62461 |
| [15] | Hoeffding, W., The large-sample power of tests based on permutations of observations, Ann. Math. Stat., 23, 169-192 (1952) · Zbl 0046.36403 |
| [16] | Horowitz, J. L., A smoothed maximum score estimator for the binary response model, Econometrica, 60, 505-531 (1992) · Zbl 0761.62166 |
| [17] | Ibragimov, R.; Müller, U., \(T\)-statistic based correlation and heterogeneity robust inference, J. Bus. Econom. Statist., 28, 453-468 (2010) · Zbl 1202.62034 |
| [18] | Ibragimov, R.; Müller, U., Inference with few heterogenous clusters, Rev. Econom. Stat., 98, 83-96 (2016) |
| [19] | Imbens, G. W.; Kolesár, M., Robust standard errors in small samples: Some practical advice, Rev. Econom. Stat., 98, 701-712 (2017) |
| [20] | Janssen, A., Studentized permutation tests for non-i.i.d. hypotheses and the generalized Behrens-Fisher problem, Statist. Probab. Lett., 36, 9-21 (1997) · Zbl 1064.62526 |
| [21] | Janssen, A., Resampling student’s \(t\)-type statistics, Ann. Inst. Statist. Math., 57, 507-529 (2005) · Zbl 1095.62050 |
| [22] | Kiefer, N. M.; Vogelsang, T. J., Heteroskedasticity autocorrelation robust testing using bandwidth equal to sample size, Econometric Theory, 18, 1350-1366 (2002) · Zbl 1033.62081 |
| [23] | Kiefer, N. M.; Vogelsang, T. J., A new asymptotic theory for heteroskedasticity-autocorrelation robust tests, Econometric Theory, 21, 1130-1164 (2005) · Zbl 1082.62040 |
| [24] | Kloek, T., OLS estimation in a model where a microvariable is explained by aggregates and contemporaneous disturbances are equicorrelated, Econometrica, 49, 205-207 (1981) · Zbl 0451.62080 |
| [25] | Lehmann, E. L.; Stein, C. M., On the theory of some non-parametric hypotheses, Ann. Math. Stat., 20, 28-45 (1949) · Zbl 0033.07901 |
| [26] | MacKinnon, J. G.; Webb, M. D., Wild bootstrap inference for wildly different cluster sizes, J. Appl. Econometrics, 32, 233-254 (2016) |
| [27] | MacKinnon, J.G., Webb, M.D., 2017. The Subcluster Wild Bootstrap for Few (Treated) Clusters, Queen’s Economics Department Working Paper No. 1364.; MacKinnon, J.G., Webb, M.D., 2017. The Subcluster Wild Bootstrap for Few (Treated) Clusters, Queen’s Economics Department Working Paper No. 1364. |
| [28] | Moulton, B. R., An illustration of a pitfall in estimating the effects of aggregate variables on micro units, Rev. Econ. Stat., 72, 334-338 (1990) |
| [29] | Neuhaus, G., Conditional rank test for the two-sample problem under random censorship, Ann. Statist., 21, 1760-1779 (1993) · Zbl 0793.62027 |
| [30] | Romano, J. P., On the Behavior of Randomization Tests Without a Group Invariance AssumptionTechnical Report No. 334 (1989), Department of Statistics, Stanford University |
| [31] | Shao, X.; Wu, W. B., Asymptotic spectral theory for nonlinear time series, Ann. Statist., 35, 1773-1801 (2007) · Zbl 1147.62076 |
| [32] | van der Vaart, A. W., Asymptotic Statistics (1998), Cambridge University Press · Zbl 0910.62001 |
| [33] | Webb, M.D., 2014. Reworking Wild Bootstrap Based Inference for Clustered Errors, Queen’s Economics Department Working Paper No. 1315.; Webb, M.D., 2014. Reworking Wild Bootstrap Based Inference for Clustered Errors, Queen’s Economics Department Working Paper No. 1315. |
| [34] | Wu, W. B., Nonlinear system theory: Another look at dependence, Proc. Natl. Acad. Sci., 102, 14150-14154 (2005) · Zbl 1135.62075 |
| [35] | Xiong, S.; Li, G., Some results on the convergence of conditional distributions, Statist. Probab. Lett., 78, 3249-3253 (2008) · Zbl 1489.60005 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
