Assessing the causal effect of binary interventions from observational panel data with few treated units. (English) Zbl 1429.62695
Summary: Researchers are often challenged with assessing the impact of an intervention on an outcome of interest in situations where the intervention is nonrandomised, the intervention is only applied to one or few units, the intervention is binary, and outcome measurements are available at multiple time points. In this paper, we review existing methods for causal inference in these situations. We detail the assumptions underlying each method, emphasize connections between the different approaches and provide guidelines regarding their practical implementation. Several open problems are identified thus highlighting the need for future research.
MSC:
| 62P20 | Applications of statistics to economics |
| 62M10 | Time series, auto-correlation, regression, etc. in statistics (GARCH) |
| 62G05 | Nonparametric estimation |
Keywords:
causal impact; causal inference; difference-in-differences; intervention evaluation; latent factor models; panel data; synthetic controlsReferences:
| [1] | Abadie, A. (2005). Semiparametric difference-in-differences estimators. Rev. Econ. Stud. 72 1-19. · Zbl 1112.62132 · doi:10.1111/0034-6527.00321 |
| [2] | Abadie, A., Diamond, A. and Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. J. Amer. Statist. Assoc. 105 493-505. |
| [3] | Abadie, A., Diamond, A. and Hainmueller, J. (2011). Synth: An R package for synthetic control methods in comparative case studies. Journal of Statistical Software 42. |
| [4] | Abadie, A., Diamond, A. and Hainmueller, J. (2015). Comparative politics and the synthetic control method. Amer. J. Polit. Sci. 59 495-510. |
| [5] | Abadie, A. and Gardeazabal, J. (2003). The economic costs of conflict: A case study of the Basque country. Am. Econ. Rev. 93 113-132. |
| [6] | Acemoglu, D., Johnson, S., Kermani, A., Kwak, J. and Mitton, T. (2016). The value of connections in turbulent times: Evidence from the United States. J. Financ. Econ. 121 368-391. |
| [7] | Ahn, S. C., Lee, Y. H. and Schmidt, P. (2013). Panel data models with multiple time-varying individual effects. J. Econometrics 174 1-14. · Zbl 1277.62202 · doi:10.1016/j.jeconom.2012.12.002 |
| [8] | Amjad, M., Shah, D. and Shen, D. (2018). Robust synthetic control. J. Mach. Learn. Res. 19 Paper No. 22, 51. · Zbl 1445.62113 |
| [9] | Andrews, D. W. K. (2003). End-of-sample instability tests. Econometrica 71 1661-1694. · Zbl 1154.62412 · doi:10.1111/1468-0262.00466 |
| [10] | Angrist, J. D. andPischke, J.-S. (2009). Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton Univ. Press, Princeton, NJ. · Zbl 1159.62090 |
| [11] | Antonakis, J., Bendahan, S., Jacquart, P. and Lalive, R. (2010). On making causal claims: A review and recommendations. Leadersh. Q. 21 1086-1120. |
| [12] | Ashenfelter, O. (1978). Estimating the effect of training programs on earnings. Rev. Econ. Stat. 60 47-57. |
| [13] | Ashenfelter, O. and Card, D. (1985). Using the longitudinal structure of earnings to estimate the effect of training programs. Rev. Econ. Stat. 67 648-660. |
| [14] | Atanasov, V. and Black, B. (2016). Shock-based causal inference in corporate finance and accounting research. Critical Finance Review 5 207-304. |
| [15] | Athey, S. and Imbens, G. W. (2006). Identification and inference in nonlinear difference-in-differences models. Econometrica 74 431-497. · Zbl 1145.62316 · doi:10.1111/j.1468-0262.2006.00668.x |
| [16] | Athey, S., Bayati, M., Doudchenko, N., Imbens, G. and Khosravi, K. (2017). Matrix completion methods for causal panel data models. Preprint. Available at arXiv:1710.10251. · Zbl 1506.15030 |
| [17] | Austin, P. C. (2011). An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivar. Behav. Res. 46 399-424. |
| [18] | Aytuğ, H., Kütük, M. M., Oduncu, A. and Togan, S. (2017). Twenty years of the EU-Turkey customs union: A synthetic control method analysis. J. Common Mark. Stud. 55 419-431. |
| [19] | Bai, J. (2009). Panel data models with interactive fixed effects. Econometrica 77 1229-1279. · Zbl 1183.62196 · doi:10.3982/ECTA6135 |
| [20] | Bai, J. and Ng, S. (2002). Determining the number of factors in approximate factor models. Econometrica 70 191-221. · Zbl 1103.91399 · doi:10.1111/1468-0262.00273 |
| [21] | Ben-Michael, E., Feller, A. and Rothstein, J. (2018). The augmented synthetic control method. Preprint. Available at arXiv:1811.04170. · Zbl 1506.62484 |
| [22] | Bertrand, M., Duflo, E. and Mullainathan, S. (2004). How much should we trust differences-in-differences estimates? Q. J. Econ. 119 249-275. · Zbl 1053.62132 · doi:10.1162/003355304772839588 |
| [23] | Billmeier, A. and Nannicini, T. (2013). Assessing economic liberalization episodes: A synthetic control approach. Rev. Econ. Stat. 95 983-1001. |
| [24] | Blundell, R., Dias, M. C., Meghir, C. and Van Reenen, J. (2004). Evaluating the employment impact of a mandatory job search program. J. Eur. Econ. Assoc. 2 569-606. |
| [25] | Branas, C. C., Cheney, R. A., MacDonald, J. M., Tam, V. W., Jackson, T. D. and Ten Have, T. R. (2011). A difference-in-differences analysis of health, safety, and greening vacant urban space. Am. J. Epidemiol. 174 1296-1306. |
| [26] | Brodersen, K. H., Gallusser, F., Koehler, J., Remy, N. and Scott, S. L. (2015). Inferring causal impact using Bayesian structural time-series models. Ann. Appl. Stat. 9 247-274. · Zbl 1454.62473 · doi:10.1214/14-AOAS788 |
| [27] | Bruhn, C. A., Hetterich, S., Schuck-Paim, C., Kürüm, E., Taylor, R. J., Lustig, R., Shapiro, E. D., Warren, J. L., Simonsen, L. et al. (2017). Estimating the population-level impact of vaccines using synthetic controls. Proc. Natl. Acad. Sci. USA 114 1524-1529. |
| [28] | Card, D. (1990). The impact of the Mariel boatlift on the Miami labor market. Ind. Labor Relat. Rev. 43 245-257. |
| [29] | Card, D. and Krueger, A. B. (1994). Minimum wages and employment: A case study of the fast-food industry in New Jersey and Pennsylvania. Am. Econ. Rev. 84 772-793. |
| [30] | Carvalho, C., Masini, R. and Medeiros, M. C. (2018). ArCo: An artificial counterfactual approach for high-dimensional panel time-series data. J. Econometrics 207 352-380. · Zbl 1452.62891 · doi:10.1016/j.jeconom.2018.07.005 |
| [31] | Cavallo, E., Galiani, S., Noy, I. and Pantano, J. (2013). Catastrophic natural disasters and economic growth. Rev. Econ. Stat. 95 1549-1561. |
| [32] | Chan, M. K. and Kwok, S. (2016). Policy evaluation with interactive fixed effects. Preprint. Available at https://ideas.repec.org/p/syd/wpaper/2016-11.html. |
| [33] | Chernozhukov, V., Wüthrich, K. and Zhu, Y. (2017). An exact and robust conformal inference method for counterfactual and synthetic controls. Preprint. Available at arXiv:1712.09089. · Zbl 1506.62246 |
| [34] | de Vocht, F. (2016). Inferring the 1985-2014 impact of mobile phone use on selected brain cancer subtypes using Bayesian structural time series and synthetic controls. Environ. Int. 97 100-107. |
| [35] | de Vocht, F., Tilling, K., Pliakas, T., Angus, C., Egan, M., Brennan, A., Campbell, R. and Hickman, M. (2017). The intervention effect of local alcohol licensing policies on hospital admission and crime: A natural experiment using a novel Bayesian synthetic time-series method. J. Epidemiol. Community Health 71 912-918. |
| [36] | Donald, S. G. and Lang, K. (2007). Inference with difference-in-differences and other panel data. Rev. Econ. Stat. 89 221-233. |
| [37] | Doudchenko, N. and Imbens, G. W. (2016). Balancing, regression, difference-in-differences and synthetic control methods: A synthesis. Preprint. Available at arXiv:1610.07748. |
| [38] | Dube, A. and Zipperer, B. (2015). Pooling multiple case studies using synthetic controls: An application to minimum wage policies. |
| [39] | Ferman, B. and Pinto, C. (2016). Revisiting the synthetic control estimator. |
| [40] | Ferman, B., Pinto, C. and Possebom, V. (2016). Cherry picking with synthetic controls. |
| [41] | Filzmoser, P., Maronna, R. and Werner, M. (2008). Outlier identification in high dimensions. Comput. Statist. Data Anal. 52 1694-1711. · Zbl 1452.62370 · doi:10.1016/j.csda.2007.05.018 |
| [42] | Firpo, S. and Possebom, V. (2018). Synthetic control method: Inference, sensitivity analysis and confidence sets. J. Causal Inference 6. |
| [43] | Fujiki, H. and Hsiao, C. (2015). Disentangling the effects of multiple treatments—measuring the net economic impact of the 1995 great Hanshin-Awaji earthquake. J. Econometrics 186 66-73. · Zbl 1331.62468 · doi:10.1016/j.jeconom.2014.10.010 |
| [44] | Galiani, S., Gertler, P. and Schargrodsky, E. (2005). Water for life: The impact of the privatization of water services on child mortality. J. Polit. Econ. 113 83-120. |
| [45] | Gardeazabal, J. and Vega-Bayo, A. (2017). An empirical comparison between the synthetic control method and Hsiao et al.’s panel data approach to program evaluation. J. Appl. Econometrics 32 983-1002. |
| [46] | Glass, T. A., Goodman, S. N., Hernán, M. A. and Samet, J. M. (2013). Causal inference in public health. Annu. Rev. Public Health 34 61-75. |
| [47] | Gobillon, L. and Magnac, T. (2016). Regional policy evaluation: Interactive fixed effects and synthetic controls. Rev. Econ. Stat. 98 535-551. |
| [48] | González, R. and Hosoda, E. B. (2016). Environmental impact of aircraft emissions and aviation fuel tax in Japan. J. Air Transp. Manag. 57 234-240. |
| [49] | Hahn, J. and Shi, R. (2017). Synthetic control and inference. Econometrics 5 52. |
| [50] | Hansen, C. and Liao, Y. (2019). The factor-lasso and \(k\)-step bootstrap approach for inference in high-dimensional economic applications. Econometric Theory 35 465-509. · Zbl 1419.62509 · doi:10.1017/S0266466618000245 |
| [51] | Hazlett, C. and Xu, Y. (2018). Trajectory balancing: A general reweighting approach to causal inference with time-series cross-sectional data. |
| [52] | Holland, P. W. (1986). Statistics and causal inference. J. Amer. Statist. Assoc. 81 945-970. · Zbl 0607.62001 · doi:10.1080/01621459.1986.10478354 |
| [53] | Hsiao, C., Ching, H. S. and Wan, S. K. (2012). A panel data approach for program evaluation: Measuring the benefits of political and economic integration of Hong Kong with mainland China. J. Appl. Econometrics 27 705-740. |
| [54] | Imai, K., Kim, I. S. and Wang, E. (2018). Matching methods for causal inference with time-series cross-section data. |
| [55] | Imbens, G. W. and Wooldridge, J. M. (2009). Recent developments in the econometrics of program evaluation. J. Econ. Lit. 47 5-86. |
| [56] | Jones, A. M. and Rice, N. (2011). Econometric evaluation of health policies. In The Oxford Handbook of Health Economics (S. Glied and P. C. Smith, eds.). Oxford Handbooks 890-923. Oxford Univ. Press, Oxford. |
| [57] | Keele, L. (2015). The statistics of causal inference: A view from political methodology. Polit. Anal. 23 313-335. |
| [58] | Keele, L. and Minozzi, W. (2013). How much is Minnesota like Wisconsin? Assumptions and counterfactuals in causal inference with observational data. Polit. Anal. 21 193-216. |
| [59] | Kim, D. and Oka, T. (2014). Divorce law reforms and divorce rates in the USA: An interactive fixed-effects approach. J. Appl. Econometrics 29 231-245. |
| [60] | King, M., Essick, C., Bearman, P. and Ross, J. S. (2013). Medical school gift restriction policies and physician prescribing of newly marketed psychotropic medications: Difference-in-differences analysis. BMJ 346 f264. |
| [61] | Kinn, D. (2018). Synthetic control methods and big data. Preprint. Available at arXiv:1803.00096. |
| [62] | Kreif, N., Grieve, R., Hangartner, D., Turner, A. J., Nikolova, S. and Sutton, M. (2016). Examination of the synthetic control method for evaluating health policies with multiple treated units. Health Econ. 25 1514-1528. |
| [63] | Li, K. (2018). Inference for factor model based average treatment effects. |
| [64] | Li, K. T. and Bell, D. R. (2017). Estimation of average treatment effects with panel data: Asymptotic theory and implementation. J. Econometrics 197 65-75. · Zbl 1443.62488 · doi:10.1016/j.jeconom.2016.01.011 |
| [65] | Lopes, H. F., Salazar, E. and Gamerman, D. (2008). Spatial dynamic factor analysis. Bayesian Anal. 3 759-792. · Zbl 1330.62356 · doi:10.1214/08-BA329 |
| [66] | Lopez Bernal, J., Cummins, S. and Gasparrini, A. (2016). Interrupted time series regression for the evaluation of public health interventions: A tutorial. Int. J. Epidemiol. 46 348-355. |
| [67] | Morgan, S. L. and Winship, C. (2007). Counterfactuals and Causal Inference: Methods and Principles for Social Research. Analytical Methods for Social Research. Cambridge Univ. Press, Cambridge. |
| [68] | O’Neill, S., Kreif, N., Grieve, R., Sutton, M. and Sekhon, J. S. (2016). Estimating causal effects: Considering three alternatives to difference-in-differences estimation. Health Serv. Outcomes Res. Methodol. 16 1-21. |
| [69] | R Core Team (2016). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. |
| [70] | Robbins, M. W., Saunders, J. and Kilmer, B. (2017). A framework for synthetic control methods with high-dimensional, micro-level data: Evaluating a neighborhood-specific crime intervention. J. Amer. Statist. Assoc. 112 109-126. |
| [71] | Rosenbaum, P. R. (2002). Observational Studies, 2nd ed. Springer Series in Statistics. Springer, New York. · Zbl 0985.62091 |
| [72] | Rosenbaum, P. R. and Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika 70 41-55. · Zbl 0522.62091 · doi:10.1093/biomet/70.1.41 |
| [73] | Rothman, K. J. and Greenland, S. (2005). Causation and causal inference in epidemiology. Am. J. Publ. Health 95 S144-S150. |
| [74] | Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66 688. |
| [75] | Rubin, D. B. (1990). Formal mode of statistical inference for causal effects. J. Statist. Plann. Inference 25 279-292. |
| [76] | Rubin, D. B. and Waterman, R. P. (2006). Estimating the causal effects of marketing interventions using propensity score methodology. Statist. Sci. 21 206-222. · Zbl 1426.62325 · doi:10.1214/088342306000000259 |
| [77] | Ryan, A. M., Krinsky, S., Kontopantelis, E. and Doran, T. (2016). Long-term evidence for the effect of pay-for-performance in primary care on mortality in the UK: A population study. Lancet 388 268-274. |
| [78] | Ryan, A. M., Kontopantelis, E., Linden, A. and Burgess, J. F. Jr. (2018). Now trending: Coping with non-parallel trends in difference-in-differences analysis. Stat. Methods Med. Res. DOI:10.1177/0962280218814570. |
| [79] | Sanso-Navarro, M., Sanz-Gracia, F. and Vera-Cabello, M. (2018). The demographic impact of terrorism: Evidence from municipalities in the Basque Country and Navarre. Regional Studies 0 1-11. |
| [80] | Saunders, J., Lundberg, R., Braga, A. A., Ridgeway, G. and Miles, J. (2015). A synthetic control approach to evaluating place-based crime interventions. J. Quant. Criminol. 31 413-434. |
| [81] | Schmitt, E., Tull, C. and Atwater, P. (2018). Extending Bayesian structural time-series estimates of causal impact to many-household conservation initiatives. Ann. Appl. Stat. 12 2517-2539. · Zbl 1412.62205 · doi:10.1214/18-AOAS1166 |
| [82] | Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statist. Sci. 25 1-21. · Zbl 1328.62007 · doi:10.1214/09-STS313 |
| [83] | Varian, H. R. (2016). Causal inference in economics and marketing. Proc. Natl. Acad. Sci. USA 113 7310-7315. |
| [84] | Vega-Bayo, A. (2015). An R package for the panel approach method for program evaluation. R J. 7. |
| [85] | Viboud, C., Boëlle, P.-Y., Carrat, F., Valleron, A.-J. and Flahault, A. (2003). Prediction of the spread of influenza epidemics by the method of analogues. Am. J. Epidemiol. 158 996-1006. |
| [86] | Vizzotti, C., Juarez, M. V., Bergel, E., Romanin, V., Califano, G., Sagradini, S., Rancaño, C., Aquino, A., Libster, R. et al. (2016). Impact of a maternal immunization program against pertussis in a developing country. Vaccine 34 6223-6228. |
| [87] | Wing, C., Simon, K. and Bello-Gomez, R. A. (2018). Designing difference in difference studies: Best practices for public health policy research. Annu. Rev. Public Health 39 453-469. |
| [88] | Wooldridge, J. M. (2013). Introductory Econometrics: A Modern Approach. Cengage Learning, Mason. |
| [89] | Xu, Y. (2017). Generalized synthetic control method: Causal inference with interactive fixed effects models. Polit. Anal. 25 57-76. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
