New graphical tool helps researchers check when Difference‑in‑Differences comparisons are valid
This paper is about why a common method for estimating causal effects, called Difference‑in‑Differences (DiD), can give misleading answers unless researchers condition on the right variables. DiD compares outcomes over time for groups that do and do not get a treatment. It usually relies on a conditional parallel trends (CPT) assumption. CPT says that, after you control for some observed variables, the treated and untreated groups would have followed the same outcome path if no one had been treated. The authors ask: how can we justify which variables to control for in complex settings with many time periods and variables that change over time?
The researchers introduce a graphical tool they call Δ‑SWIGs, short for transformed Single World Intervention Graphs. These graphs are a way to draw the causal structure of a problem so that you can read off which conditional independencies must hold for CPT to be true. In plain terms, Δ‑SWIGs let researchers see, from the causal picture, when differences in potential outcomes will be unrelated to treatment after conditioning on certain variables. The graphs use a standard rule called d‑separation, which is a visual test for whether two pieces of the graph are independent once you condition on some nodes.
Using Δ‑SWIGs, the paper studies settings with many time periods and with covariates that change over time. One clear finding is that if time‑varying covariates affect the outcome, then researchers must control for post‑treatment variables to identify effects. The authors also show that pre‑treatment checks — for instance, tests of parallel trends before treatment starts — only speak to some of the assumptions needed for unbiased post‑treatment estimates. In other words, seeing parallel pre‑treatment trends does not prove that the CPT assumption holds for later periods.
To make these points concrete, the authors report simulation results. They compare three conditioning strategies: using only pre‑treatment covariates, using covariates measured just before each outcome comparison ("pre‑outcome" controls), and using the full sequence of time‑varying covariates. The simulations reveal several patterns. First, using only pre‑treatment covariates can give unbiased short‑term effects but biased dynamic effects. Second, pre‑outcome controls give the same results as using the full sequence. Third, if the treatment does not affect the covariates, including post‑treatment variables can be unbiased. Fourth, if the treatment feeds back into covariates, dynamic effect estimates are biased, a form of omitted‑variable or "wrong‑world control" bias. Fifth, pre‑trend tests do not diagnose post‑treatment violations of CPT, though short‑term effects can remain unbiased in some cases.