Cluster-robust inference: a guide to empirical practice. (English) Zbl 07648714
Summary: Methods for cluster-robust inference are routinely used in economics and many other disciplines. However, it is only recently that theoretical foundations for the use of these methods in many empirically relevant situations have been developed. In this paper, we use these theoretical results to provide a guide to empirical practice. We do not attempt to present a comprehensive survey of the (very large) literature. Instead, we bridge theory and practice by providing a thorough guide on what to do and why, based on recently available econometric theory and simulation evidence. To practice what we preach, we include an empirical analysis of the effects of the minimum wage on labor supply of teenagers using individual data.
MSC:
| 62-XX | Statistics |
| 91-XX | Game theory, economics, finance, and other social and behavioral sciences |
Keywords:
clustered data; cluster jackknife; cluster-robust variance estimator (CRVE); robust inference; wild cluster bootstrapSoftware:
boottestReferences:
| [1] | Abadie, A., Using synthetic controls: Feasibility, data requirements, and methodological aspects, J. Econ. Lit., 59, 391-425 (2021) |
| [2] | Abadie, A.; Athey, S.; Imbens, G. W.; Wooldridge, J., When Should you Adjust Standard Errors for Clustering?Working Paper 24003 (2017), National Bureau of Economic Research |
| [3] | Abadie, A.; Athey, S.; Imbens, G. W.; Wooldridge, J., Sampling-based vs. design-based uncertainty in regression analysis, Econometrica, 88, 265-296 (2020) · Zbl 1467.62119 |
| [4] | Andrews, I.; Stock, J. H.; Sun, L., Weak instruments in instrumental variables regression: Theory and practice, Annu. Rev. Econ., 11, 727-753 (2019) |
| [5] | Angrist, J. D.; Pischke, J.-S., Mostly Harmless Econometrics: An Empiricist’s Companion (2008), Princeton University Press · Zbl 1159.62090 |
| [6] | Anselin, L., Spatial Econometrics: Methods and Models (1988), Kluwer: Kluwer New York |
| [7] | Arellano, M., Computing robust standard errors for within groups estimators, Oxf. Bull. Econ. Stat., 49, 431-434 (1987) |
| [8] | Bell, R. M.; McCaffrey, D. F., Bias reduction in standard errors for linear regression with multi-stage samples, Surv. Methodol., 28, 169-181 (2002) |
| [9] | Belsley, D. A.; Kuh, E.; Welsch, R. E., Regression Diagnostics (1980), Wiley: Wiley New York · Zbl 0479.62056 |
| [10] | Bertrand, M.; Duflo, E.; Mullainathan, S., How much should we trust differences-in-differences estimates?, Q. J. Econ., 119, 249-275 (2004) · Zbl 1053.62132 |
| [11] | Bester, C. A.; Conley, T. G.; Hansen, C. B., Inference with dependent data using cluster covariance estimators, J. Econometrics, 165, 137-151 (2011) · Zbl 1441.62611 |
| [12] | Brewer, M.; Crossley, T. F.; Joyce, R., Inference with difference-in-differences revisited, J. Econom. Methods, 7, 1-16 (2018) · Zbl 1420.62477 |
| [13] | Cai, Y., A Modified Randomization Test for the Level of Clustering (2021), Northwestern University, ArXiv e-prints 2105.01008 |
| [14] | Cai, Y.; Canay, I. A.; Kim, D.; Shaikh, A. M., On the Implementation of Approximate Randomization Tests in Linear Models with a Small Number of Clusters (2021), Northwestern University, ArXiv e-prints 2102.09058v2 |
| [15] | Callaway, B.; Sant’Anna, P. H., Difference-in-differences with multiple time periods, J. Econometrics, 225, 200-230 (2021) · Zbl 07414289 |
| [16] | Cameron, A. C.; Gelbach, J. B.; Miller, D. L., Bootstrap-based improvements for inference with clustered errors, Rev. Econ. Stat., 90, 414-427 (2008) |
| [17] | Cameron, A. C.; Gelbach, J. B.; Miller, D. L., Robust inference with multiway clustering, J. Bus. Econom. Statist., 29, 238-249 (2011) · Zbl 1217.62087 |
| [18] | Cameron, A. C.; Miller, D. L., A practitioner’s guide to cluster-robust inference, J. Hum. Resour., 50, 317-372 (2015) |
| [19] | Canay, I. A.; Romano, J. P.; Shaikh, A. M., Randomization tests under an approximate symmetry assumption, Econometrica, 85, 1013-1030 (2017) · Zbl 1410.62103 |
| [20] | Canay, I. A.; Santos, A.; Shaikh, A., The wild bootstrap with a ‘small’ number of ‘large’ clusters, Rev. Econ. Stat., 103, 346-363 (2021) |
| [21] | Card, D.; Krueger, A., Minimum wages and employment: A case study of the fast-food industry in New Jersey and Pennsylvania, Amer. Econ. Rev., 84, 772-793 (1994) |
| [22] | Carter, A. V.; Schnepel, K. T.; Steigerwald, D. G., Asymptotic behavior of a \(t\) test robust to cluster heterogeneity, Rev. Econ. Stat., 99, 698-709 (2017) |
| [23] | Chatterjee, S.; Hadi, A. S., Influential observations, high-leverage points, and outliers in linear regression, Statist. Sci., 1, 379-416 (1986) · Zbl 0633.62059 |
| [24] | Chiang, H. D.; Kato, K.; Ma, Y.; Sasaki, Y., Multiway cluster robust double/debiased machine learning, J. Bus. Econom. Statist. (2022), in press |
| [25] | Chiang, H. D.; Kato, K.; Sasaki, Y., Inference for high-dimensional exchangeable arrays, J. Amer. Statist. Assoc. (2022), in press |
| [26] | Conley, T. G.; Gonçalves, S.; Hansen, C. B., Inference with dependent data in accounting and finance applications, J. Account. Res., 56, 1139-1203 (2018) |
| [27] | Conley, T. G.; Taber, C. R., Inference with “difference in differences” with a small number of policy changes, Rev. Econ. Stat., 93, 113-125 (2011) |
| [28] | Cook, R. D.; Weisberg, S., Characterizations of an empirical influence function for detecting influential cases in regression, Technometrics, 22, 495-508 (1980) · Zbl 0453.62051 |
| [29] | Corrado, L.; Fingleton, B., Where is the economics in spatial econometrics?, J. Reg. Sci., 52, 210-239 (2012) |
| [30] | Davezies, L.; D’Haultfœuille, X.; Guyonvarch, Y., Empirical process results for exchangeable arrays, Ann. Statist., 49, 845-862 (2021) · Zbl 1480.60074 |
| [31] | Davidson, R.; Flachaire, E., The wild bootstrap, tamed at last, J. Econometrics, 146, 162-169 (2008) · Zbl 1418.62183 |
| [32] | Davidson, R.; MacKinnon, J. G., Bootstrap tests: How many bootstraps?, Econometric Rev., 19, 55-68 (2000) · Zbl 0949.62030 |
| [33] | Djogbenou, A. A.; MacKinnon, J. G.; Nielsen, M.Ø., Asymptotic theory and wild bootstrap inference with clustered errors, J. Econometrics, 212, 393-412 (2019) · Zbl 1452.62902 |
| [34] | Efron, B., Nonparametric estimates of standard error: The jackknife, the bootstrap and other methods, Biometrika, 68, 589-599 (1981) · Zbl 0487.62031 |
| [35] | Esarey, J.; Menger, A., Practical and effective approaches to dealing with clustered data, Political Sci. Res. Methods, 7, 541-559 (2019) |
| [36] | Ferman, B.; Pinto, C., Inference in differences-in-differences with few treated groups and heteroskedasticity, Rev. Econ. Stat., 101, 452-467 (2019) |
| [37] | Fisher, R. A., The Design of Experiments (1935), Oliver and Boyd: Oliver and Boyd Edinburgh |
| [38] | Gelfand, A. E.; Diggle, P.; Fuentes, M.; Guttorp, P., Handbook of Spatial Statistics (2010), Chapman and Hall/CRC Press: Chapman and Hall/CRC Press Boca Raton · Zbl 1188.62284 |
| [39] | Hagemann, A., Permutation Inference with a Finite Number of Heterogeneous Clusters (2019), University of Michigan, ArXiv e-prints 1907.01049 |
| [40] | Hagemann, A., Placebo inference on treatment effects when the number of clusters is small, J. Econometrics, 213, 190-209 (2019) · Zbl 1456.62286 |
| [41] | Hall, P., The Bootstrap and Edgeworth Expansion (1992), Springer-Verlag: Springer-Verlag New York · Zbl 0829.62021 |
| [42] | Hansen, B. E., The grid bootstrap and the autoregressive model, Rev. Econ. Stat., 81, 594-607 (1999) |
| [43] | Hansen, C. B., Asymptotic properties of a robust variance matrix estimator for panel data when \(T\) is large, J. Econometrics, 141, 597-620 (2007) · Zbl 1418.62461 |
| [44] | Hansen, B. E.; Lee, S., Asymptotic theory for clustered samples, J. Econometrics, 210, 268-290 (2019) · Zbl 1452.62915 |
| [45] | Hu, A.; Spamann, H., Inference with Cluster Imbalance: The Case of State Corporate LawsDiscussion Paper (2020), Harvard Law School |
| [46] | Hwang, J., Simple and trustworthy cluster-robust GMM inference, J. Econometrics, 222, 993-1023 (2021) · Zbl 1471.62531 |
| [47] | Ibragimov, R.; Müller, U. K., \(t\)-Statistic based correlation and heterogeneity robust inference, J. Bus. Econom. Statist., 28, 453-468 (2010) · Zbl 1202.62034 |
| [48] | Ibragimov, R.; Müller, U. K., Inference with few heterogeneous clusters, Rev. Econ. Stat., 98, 83-96 (2016) |
| [49] | Imbens, G. W.; Kolesár, M., Robust standard errors in small samples: Some practical advice, Rev. Econ. Stat., 98, 701-712 (2016) |
| [50] | Imbens, G. W.; Rubin, D. B., Causal Inference in Statistics, Social, and Biomedical Sciences (2015), Cambridge University Press: Cambridge University Press New York · Zbl 1355.62002 |
| [51] | Jardim, E.; Long, M. C.; Plotnick, R.; van Inwegen, E.; Vigdor, J.; Wething, H., Minimum Wage Increases, Wages, and Low-Wage Employment: Evidence from SeattleWorking Paper 23532 (2017), National Bureau of Economic Research |
| [52] | Leeb, H.; Pötscher, B. M., Model selection and inference: Facts and fiction, Econom. Theory, 21, 21-59 (2005) · Zbl 1085.62004 |
| [53] | Lehmann, E. L.; Romano, J. P., Testing Statistical Hypotheses (2005), Springer: Springer New York · Zbl 1076.62018 |
| [54] | MacKinnon, J. G., Wild cluster bootstrap confidence intervals, Actual. Écon., 91, 11-33 (2015) |
| [55] | MacKinnon, J. G., Inference with large clustered datasets, Actual. Écon., 92, 649-665 (2016) |
| [56] | MacKinnon, J. G., How cluster-robust inference is changing applied econometrics, Can. J. Econ., 52, 851-881 (2019) |
| [57] | MacKinnon, J. G., Fast cluster bootstrap methods for linear regression models, Econom. Stat., 21 (2022), in press |
| [58] | MacKinnon, J. G.; Nielsen, M.Ø.; Webb, M. D., Testing for the Appropriate Level of Clustering in Linear Regression ModelsQED Working Paper 1428 (2020), Queen’s University |
| [59] | MacKinnon, J. G.; Nielsen, M.Ø.; Webb, M. D., Wild bootstrap and asymptotic inference with multiway clustering, J. Bus. Econom. Statist., 39, 505-519 (2021) |
| [60] | MacKinnon, J. G.; Nielsen, M.Ø.; Webb, M. D., Fast and Reliable Jackknife and Bootstrap Methods for Cluster-Robust InferenceQED Working Paper 1485 (2022), Queen’s University |
| [61] | MacKinnon, J. G.; Nielsen, M.Ø.; Webb, M. D., Leverage, Influence, and the Jackknife in Clustered Regression Models: Reliable Inference using summclustQED Working Paper 1483 (2022), Queen’s University |
| [62] | MacKinnon, J. G.; Webb, M. D., Pitfalls when estimating treatment effects using clustered data, Political Methodol., 24, 2, 20-31 (2017) |
| [63] | MacKinnon, J. G.; Webb, M. D., Wild bootstrap inference for wildly different cluster sizes, J. Appl. Econometrics, 32, 233-254 (2017) |
| [64] | MacKinnon, J. G.; Webb, M. D., The wild bootstrap for few (treated) clusters, Econom. J., 21, 114-135 (2018) · Zbl 07547388 |
| [65] | MacKinnon, J. G.; Webb, M. D., Clustering methods for statistical inference, (Zimmermann, K. F., Handbook of Labor, Human Resources and Population Economics (2020), Springer: Springer Cham, Switzerland) |
| [66] | MacKinnon, J. G.; Webb, M. D., Randomization inference for difference-in-differences with few treated clusters, J. Econometrics, 218, 435-450 (2020) · Zbl 1464.62226 |
| [67] | MacKinnon, J. G.; White, H., Some heteroskedasticity consistent covariance matrix estimators with improved finite sample properties, J. Econometrics, 29, 305-325 (1985) |
| [68] | Manning, A., The elusive employment effect of the minimum wage, J. Econ. Perspect., 35, 3-26 (2021) |
| [69] | Menzel, K., Bootstrap with cluster-dependence in two or more dimensions, Econometrica, 89, 2143-2188 (2021) · Zbl 07684915 |
| [70] | Miglioretti, D. L.; Heagerty, P. J., Marginal modeling of nonnested multilevel data using standard software, Am. J. Epidemiol., 165, 453-463 (2006) |
| [71] | Mincer, J., Unemployment effects of minimum wages, J. Polit. Econ., 84, S87-S104 (1976) |
| [72] | Moulton, B. R., Random group effects and the precision of regression estimates, J. Econometrics, 32, 385-397 (1986) · Zbl 0619.62107 |
| [73] | Neumark, D., State minimum wage data set through Sept. 2019 (2019), URL: http://www.economics.uci.edu/ dneumark/datasets.html |
| [74] | Neumark, D.; Shirley, P., Myth or Measurement: What does the New Minimum Wage Research Say about Minimum Wages and Job Loss in the United States?Working Paper 28388 (2021), National Bureau of Economic Research |
| [75] | Neumark, D.; Wascher, W., Minimum wage effects on employment and school enrollment, J. Bus. Econom. Statist., 13, 199-206 (1995) |
| [76] | Neumark, D., Wascher, W., 2007. Minimum Wages, the Earned Income Tax Credit, and Employment: Evidence from the Post-Welfare Reform Era. IZA Discussion Papers 2610. |
| [77] | Niccodemi, G.; Alessie, R.; Angelini, V.; Mierau, J.; Wansbeek, T., Refining Clustered Standard Errors with Few ClustersWorking Paper 2020002-EEF (2020), University of Groningen |
| [78] | Pustejovsky, J. E.; Tipton, E., Small sample methods for cluster-robust variance estimation and hypothesis testing in fixed effects models, J. Bus. Econom. Statist., 36, 672-683 (2018) |
| [79] | Racine, J. S.; MacKinnon, J. G., Simulation-based tests that can use any number of simulations, Commun. Stat.-Simul. Comput., 36, 357-365 (2007) · Zbl 1118.62046 |
| [80] | Roodman, D.; MacKinnon, J. G.; Nielsen, M.Ø.; Webb, M. D., Fast and wild: Bootstrap inference in Stata using boottest, Stata J., 19, 4-60 (2019) |
| [81] | Ruggles, S., Flood, S., Goeken, R., Grover, J., Meyer, E., Pacas, J., Sobek, M., 2020. IPUMS USA: Version 10.0 [dataset]. Minneapolis, MN. |
| [82] | Thompson, S. B., Simple formulas for standard errors that cluster by both firm and time, J. Financ. Econ., 99, 1-10 (2011) |
| [83] | Webb, M. D., Reworking Wild Bootstrap Based Inference for Clustered ErrorsQED Working Paper 1315 (2014), Queen’s University |
| [84] | White, H., A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity, Econometrica, 48, 817-838 (1980) · Zbl 0459.62051 |
| [85] | White, H., Asymptotic Theory for Econometricians (1984), Academic Press: Academic Press San Diego |
| [86] | Wolfson, P.; Belman, D., 15 Years of research on US employment and the minimum wage, Labour, 33, 488-506 (2019) |
| [87] | Young, A., Improved, Nearly Exact, Statistical Inference with Robust and Clustered Covariance Matrices using Effective Degrees of Freedom CorrectionsWorking Paper (2016), London School of Economics |
| [88] | Young, A., Channeling Fisher: Randomization tests and the statistical insignificance of seemingly significant experimental results, Q. J. Econ., 134, 557-598 (2019) · Zbl 1507.62410 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
