Synthetic learner: model-free inference on treatments over time. (English) Zbl 07693690
Summary: Understanding the effect of a particular treatment or a policy pertains to many areas of interest, ranging from political economics, marketing to healthcare. In this paper, we develop a non-parametric algorithm for detecting the effects of treatment over time in the context of Synthetic Controls. The method builds on counterfactual predictions from many algorithms without necessarily assuming that the algorithms correctly capture the model. We introduce an inferential procedure to detect treatment effects and show that the testing procedure controls size asymptotically for stationary, beta mixing processes without imposing any restriction on the set of base algorithms under consideration. We discuss consistency guarantees for average treatment effect estimates and derive regret bounds for the proposed methodology. The class of algorithms may include Random Forest, Lasso, or any other machine-learning estimator. Numerical studies and an application illustrate the advantages of the method.
MSC:
| 62-XX | Statistics |
| 91-XX | Game theory, economics, finance, and other social and behavioral sciences |
References:
| [1] | Abadie, A.; Diamond, A.; Hainmueller, J., Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program, J. Amer. Statist. Assoc., 105, 490, 493-505 (2010) |
| [2] | Abadie, A.; Gardeazabal, J., The economic costs of conflict: A case study of the Basque country, Amer. Econ. Rev., 93, 1, 113-132 (2003) |
| [3] | Abbring, J. H.; Heckman, J. J., Econometric evaluation of social programs, part III: Distributional treatment effects, dynamic treatment effects, dynamic discrete choice, and general equilibrium policy evaluation, Handb. Econometr., 6, 5145-5303 (2007) |
| [4] | Amjad, M.; Shah, D.; Shen, D., Robust synthetic control, J. Mach. Learn. Res., 19, 1, 802-852 (2018) · Zbl 1445.62113 |
| [5] | Anderson, M.; Dobkin, C.; Gross, T., The effect of health insurance coverage on the use of medical services, Am. Econ. J.: Econ. Policy, 4, 1, 1-27 (2012) |
| [6] | Argys, L. M.; Friedson, A. I.; Pitts, M. M.; Tello-Trillo, D. S., Losing public health insurance: TennCare reform and personal financial distress, J. Publ. Econ., 187, Article 104202 pp. (2020) |
| [7] | Arkhangelsky, D.; Athey, S.; Hirshberg, D. A.; Imbens, G. W.; Wager, S., Synthetic difference-in-differences, Amer. Econ. Rev., 111, 12, 4088-4118 (2021) |
| [8] | Athey, S.; Bayati, M.; Doudchenko, N.; Imbens, G.; Khosravi, K., Matrix completion methods for causal panel data models, J. Amer. Statist. Assoc., 116, 536, 1716-1730 (2021) · Zbl 1506.15030 |
| [9] | Athey, S.; Bayati, M.; Imbens, G.; Qu, Z., Ensemble methods for causal effects in panel data settings, (AEA Papers and Proceedings, vol. 109 (2019)), 65-70 |
| [10] | Athey, S.; Imbens, G. W., Machine learning methods that economists should know about, Annu. Rev. Econ., 11, 685-725 (2019) |
| [11] | Athey, S.; Imbens, G. W., Design-based analysis in difference-in-differences settings with staggered adoption, J. Econometrics, 226, 1, 62-79 (2022) · Zbl 07471885 |
| [12] | Bai, J., Panel data models with interactive fixed effects, Econometrica, 77, 4, 1229-1279 (2009) · Zbl 1183.62196 |
| [13] | Bai, C.; Li, Q.; Ouyang, M., Property taxes and home prices: A tale of two cities, J. Econometrics, 180, 1, 1-15 (2014) · Zbl 1298.91122 |
| [14] | Baicker, K.; Taubman, S. L.; Allen, H. L.; Bernstein, M.; Gruber, J. H.; Newhouse, J. P.; Schneider, E. C.; Wright, B. J.; Zaslavsky, A. M.; Finkelstein, A. N., The Oregon experiment—effects of Medicaid on clinical outcomes, N. Engl. J. Med., 368, 18, 1713-1722 (2013) |
| [15] | Belloni, A.; Chernozhukov, V.; Fernández-Val, I.; Hansen, C., Program evaluation and causal inference with high-dimensional data, Econometrica, 85, 1, 233-298 (2017) · Zbl 1410.62197 |
| [16] | Ben-Michael, E.; Feller, A.; Rothstein, J., The augmented synthetic control method, J. Amer. Statist. Assoc., 116, 536, 1789-1803 (2021) · Zbl 1506.62484 |
| [17] | Billmeier, A.; Nannicini, T., Assessing economic liberalization episodes: A synthetic control approach, Rev. Econ. Stat., 95, 3, 983-1001 (2013) |
| [18] | Blackwell, M.; Glynn, A. N., How to make causal inferences with time-series cross-sectional data under selection on observables, Am. Political Sci. Rev., 112, 4, 1067-1082 (2018) |
| [19] | Bojinov, I.; Shephard, N., Time series experiments and causal estimands: exact randomization tests and trading, J. Amer. Statist. Assoc., 114, 528, 1665-1682 (2019) · Zbl 1428.62385 |
| [20] | Bottmer, L.; Imbens, G.; Spiess, J.; Warnick, M., A design-based perspective on synthetic control methods (2021), arXiv preprint arXiv:2101.09398 |
| [21] | Brodersen, K. H.; Gallusser, F.; Koehler, J.; Remy, N.; Scott, S. L., Inferring causal impact using Bayesian structural time-series models, Ann. Appl. Stat., 9, 1, 247-274 (2015) · Zbl 1454.62473 |
| [22] | Carrasco, M.; Chen, X., Mixing and moment properties of various GARCH and stochastic volatility models, Econom. Theory, 18, 1, 17-39 (2002) · Zbl 1181.62125 |
| [23] | Carvalho, C.; Masini, R.; Medeiros, M. C., Arco: an artificial counterfactual approach for high-dimensional panel time-series data, J. Econometrics (2018) · Zbl 1452.62891 |
| [24] | Cavallo, E.; Galiani, S.; Noy, I.; Pantano, J., Catastrophic natural disasters and economic growth, Rev. Econ. Stat., 95, 5, 1549-1561 (2013) |
| [25] | Cesa-Bianchi, N.; Freund, Y.; Haussler, D.; Helmbold, D. P.; Schapire, R. E.; Warmuth, M. K., How to use expert advice, J. ACM, 44, 3, 427-485 (1997) · Zbl 0890.68066 |
| [26] | Cesa-Bianchi, N.; Lugosi, G., Prediction, Learning, and Games (2006), Cambridge University Press · Zbl 1114.91001 |
| [27] | Cesa-Bianchi, N.; Lugosi, G., On prediction of individual sequences, Ann. Statist., 27, 6, 1865-1895 (1999) · Zbl 0961.62081 |
| [28] | Chernozhukov, V.; Chetverikov, D.; Demirer, M.; Duflo, E.; Hansen, C.; Newey, W.; Robins, J., Double/debiased machine learning for treatment and structural parameters, Econom. J., 21, 1, C1-C68 (2018) · Zbl 07565928 |
| [29] | Chernozhukov, V.; Wüthrich, K.; Yinchu, Z., Exact and robust conformal inference methods for predictive machine learning with dependent data, (Conference on Learning Theory (2018), PMLR), 732-749 |
| [30] | Chernozhukov, V.; Wuthrich, K.; Zhu, Y., A \(t\)-test for synthetic controls (2018), arXiv preprint arXiv:1812.10820 |
| [31] | Chernozhukov, V.; Wüthrich, K.; Zhu, Y., An exact and robust conformal inference method for counterfactual and synthetic controls, J. Amer. Statist. Assoc., 116, 536, 1849-1864 (2021) · Zbl 1506.62246 |
| [32] | Doudchenko, N.; Imbens, G. W., Balancing, Regression, Difference-In-Differences and Synthetic Control Methods: A SynthesisTechnical Report (2016), National Bureau of Economic Research |
| [33] | Elliott, G.; Timmermann, A., Optimal forecast combinations under general loss functions and forecast error distributions, J. Econometrics, 122, 1, 47-79 (2004) · Zbl 1282.91266 |
| [34] | Fang, Z.; Santos, A., Inference on directionally differentiable functions, Rev. Econom. Stud., 86, 1, 377-412 (2018) · Zbl 1414.62498 |
| [35] | Ferman, B.; Pinto, C., Revisiting the synthetic control estimator (2016) |
| [36] | Firpo, S.; Possebom, V., Synthetic control method: Inference, sensitivity analysis and confidence sets, J. Causal Inference, 6, 2 (2018) |
| [37] | Garthwaite, C.; Gross, T.; Notowidigdo, M. J., Public health insurance, labor supply, and employment lock, Q. J. Econ., 129, 2, 653-696 (2014) |
| [38] | Gunsilius, F., Distributional synthetic controls (2020), arXiv preprint arXiv:2001.06118 |
| [39] | Hazlett, C.; Xu, Y., Trajectory balancing: A general reweighting approach to causal inference with time-series cross-sectional data (2018), https://Ssrn.Com/Abstract=3214231 |
| [40] | Hsiao, C.; Steve Ching, H.; Ki Wan, S., A panel data approach for program evaluation: measuring the benefits of political and economic integration of Hong Kong with mainland China, J. Appl. Econometrics, 27, 5, 705-740 (2012) |
| [41] | Hsiao, C.; Zhou, Q., Panel parametric, semiparametric, and nonparametric construction of counterfactuals, J. Appl. Econometrics, 34, 4, 463-481 (2019) |
| [42] | Iavor Bojinov, N. S., Time series experiments and causal estimands: exact randomization tests and trading, J. Amer. Statist. Assoc. (2019) · Zbl 1428.62385 |
| [43] | Imai, K.; Kim, I. S., On the use of two-way fixed effects regression models for causal inference with panel data, Political Anal., 29, 3, 405-415 (2021) |
| [44] | Imai, K.; Kim, I. S.; Wang, E., Matching methods for causal inference with time-series cross-section data (2018), https://imai.fas.harvard.edu/research/files/tscs.pdf |
| [45] | Imai, K.; Ratkovic, M., Estimating treatment effect heterogeneity in randomized program evaluation, Ann. Appl. Stat., 7, 1, 443-470 (2013) · Zbl 1376.62036 |
| [46] | Imbens, G. W.; Rubin, D. B., Causal Inference in Statistics, Social, and Biomedical Sciences (2015), Cambridge University Press · Zbl 1355.62002 |
| [47] | Kolstad, J. T.; Kowalski, A. E., The impact of health care reform on hospital and preventive care: evidence from Massachusetts, J. Publ. Econ., 96, 11-12, 909-929 (2012) |
| [48] | Künzel, S. R.; Sekhon, J. S.; Bickel, P. J.; Yu, B., Metalearners for estimating heterogeneous treatment effects using machine learning, Proc. Natl. Acad. Sci., 116, 10, 4156-4165 (2019) |
| [49] | Lange, T.; Rahbek, A.; Jensen, S.r. T., Estimation and asymptotic inference in the AR-ARCH model, Econometric Rev., 30, 2, 129-153 (2011) · Zbl 1209.62201 |
| [50] | Lee, O., Probabilistic properties of a nonlinear ARMA process with Markov switching, Comm. Statist. Theory Methods, 34, 1, 193-204 (2005) · Zbl 1062.62190 |
| [51] | Li, K. T., Estimating Average Treatment Effects Using a Modified Synthetic Control Method: Theory and Applications (2017), The Wharton School, the University of Pennsylvania |
| [52] | Li, K. T., Statistical inference for average treatment effects estimated by synthetic control methods, J. Amer. Statist. Assoc., 115, 532, 2068-2083 (2020) · Zbl 1453.62330 |
| [53] | Li, K. T.; Bell, D. R., Estimation of average treatment effects with panel data: Asymptotic theory and implementation, J. Econometrics, 197, 1, 65-75 (2017) · Zbl 1443.62488 |
| [54] | Long, S. K.; Stockley, K.; Yemane, A., Another look at the impacts of health reform in Massachusetts: evidence using new data and a stronger model, Amer. Econ. Rev., 99, 2, 508-511 (2009) |
| [55] | Lunde, R.; Shalizi, C. R., Bootstrapping generalization error bounds for time series (2017), arXiv preprint arXiv:1711.02834 |
| [56] | Maclean, J. C.; Tello-Trillo, S.; Webber, D., Losing Insurance and Behavioral Health Inpatient Care: Evidence from a Large-Scale Medicaid DisenrollmentTechnical Report (2019), National Bureau of Economic Research |
| [57] | Pham, T. D.; Tran, L. T., Some mixing properties of time series models, Stochastic Process. Appl., 19, 2, 297-303 (1985) · Zbl 0564.62068 |
| [58] | Politis, D. N.; Romano, J. P., A circular block-resampling procedure for stationary data, Explor. Limits Bootstrap, 2635270 (1992) · Zbl 0845.62036 |
| [59] | Politis, D. N.; Romano, J. P., The stationary bootstrap, J. Amer. Statist. Assoc., 89, 428, 1303-1313 (1994) · Zbl 0814.62023 |
| [60] | Polley, E. C.; Van Der Laan, M. J., Super learner in prediction (2010) |
| [61] | Potrafke, N.; Wuthrich, K., Green governments (2020), arXiv preprint arXiv:2012.09906 |
| [62] | Rambachan, A.; Shephard, N., A nonparametric dynamic causal model for macroeconometrics (2019), arXiv preprint arXiv:1903.01637 |
| [63] | Rigollet, P.; Tsybakov, A. B., Sparse estimation by exponential weighting, Statist. Sci., 27, 4, 558-575 (2012) · Zbl 1331.62351 |
| [64] | Rinaldo, A.; Wasserman, L.; G’Sell, M., Bootstrapping and sample splitting for high-dimensional, assumption-lean inference, Ann. Statist., 47, 6, 3438-3469 (2019) · Zbl 1436.62107 |
| [65] | Robins, J., A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect, Math. Model., 7, 9-12, 1393-1512 (1986) · Zbl 0614.62136 |
| [66] | Rubin, D. B., Formal mode of statistical inference for causal effects, J. Statist. Plann. Inference, 25, 3, 279-292 (1990) |
| [67] | Schapire, R. E.; Freund, Y., Boosting: Foundations and Algorithms (2012), MIT Press · Zbl 1278.68021 |
| [68] | Shaikh, A.; Toulis, P., Randomization Tests in Observational Studies with Staggered Adoption of TreatmentWorking Paper 2019-144 (2019), University of Chicago, Becker Friedman Institute for Economics |
| [69] | Tello-Trillo, D. S., Effects of losing public health insurance on preventative care, health, and emergency department use: Evidence from the TennCare disenrollment, Southern Econ. J., 88, 1, 322-366 (2021) |
| [70] | Timmermann, A., Forecast combinations, Handb. Econ. Forecast., 1, 135-196 (2006) |
| [71] | Varian, H. R., Causal inference in economics and marketing, Proc. Natl. Acad. Sci., 113, 27, 7310-7315 (2016) |
| [72] | Xie, S.; Huang, J., The impact of index futures on spot market volatility in China, Emerg. Mark. Finance Trade, 50, Suppl. 1, 167-177 (2014) |
| [73] | Xu, Y., Generalized synthetic control method: Causal inference with interactive fixed effects models, Political Anal., 25, 1, 57-76 (2017) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
