When is a Match Sufficient? A Score-based Balance Metric for the Synthetic Control Method

Journal of Causal Inference

article

Layla Parast¹  Priscillia Hunt¹  Beth Ann Griffin¹  David Powell¹ 

[1] RAND Corporation;Institute of Labor Economics (IZA)

关键词: Synthetic control method;    Non-experimental study;    Matching methods;    Gun violence;   

DOI  :  10.1515/jci-2020-0013

来源: De Gruyter

【 摘 要 】

In some applications, researchers using the synthetic control method (SCM) to evaluate the effect of a policy may struggle to determine whether they have identified a “good match” between the control group and treated group. In this paper, we demonstrate the utility of the mean and maximum Absolute Standardized Mean Difference (ASMD) as a test of balance between a synthetic control unit and treated unit, and provide guidance on what constitutes a poor fit when using a synthetic control. We explore and compare other potential metrics using a simulation study. We provide an application of our proposed balance metric to the 2013 Los Angeles (LA) Firearm Study [9]. Using Uniform Crime Report data, we apply the SCM to obtain a counterfactual for the LA firearm-related crime rate based on a weighted combination of control units in a donor pool of cities. We use this counterfactual to estimate the effect of the LA Firearm Study intervention and explore the impact of changing the donor pool and pre-intervention duration period on resulting matches and estimated effects. We demonstrate how decision-making about the quality of a synthetic control can be improved by using ASMD. The mean and max ASMD clearly differentiate between poor matches and good matches. Researchers need better guidance on what is a meaningful imbalance between synthetic control and treated groups. In addition to the use of gap plots, the proposed balance metric can provide an objective way of determining fit.

【 授权许可】

CC BY