New sensitivity test for Difference‑in‑Differences checks when pretrend tests are not enough
This paper proposes a new way to check how sensitive Difference‑in‑Differences (DID) results are to a key assumption called parallel trends. The author, Thomas Leavitt, shows that the usual check — testing whether the treated and control groups moved together before treatment — can miss important problems. He develops a “discordance‑based” sensitivity analysis that flags cases where different ways of guessing the untreated outcome would lead to different conclusions.
In a standard DID study, researchers observe a treated group and a comparison group before and after a treatment. To estimate the average treatment effect on the treated (ATT), they use the control group’s change after treatment to guess how the treated group would have changed without the treatment. That guess is valid only under the parallel trends assumption: that the two groups would have followed the same path over time in the absence of treatment. An alternative guess uses the treated group’s own pre‑treatment change. Each guess has a weakness: the control‑based guess can be wrong if groups differ (between‑group differences), while the treated‑based guess can be wrong if the treated group’s trend shifts over time (temporal shifts).
Leavitt’s idea is to compare these different imputations directly. If the two imputations agree (concordance), then the ATT estimate is likely robust. If they disagree (discordance), then conclusions that rely on parallel trends are more fragile. The paper builds a formal model that measures the expected difference between the ATT under parallel trends and the ATT under alternative assumptions, weighting those differences by how plausible the alternatives are. It also gives a decision‑theory argument for benchmarking violations by the worst‑case discordance between the parallel‑trends guess and other plausible guesses. The framework is developed for the simple two‑group, one‑post‑period case and is extended to settings with multiple post periods and staggered treatment timing.