Research article · Econometrics, policy evaluation & research design

Who supplies
the comparison?

A policy arrives in different places at different times. Aligning those places around their adoption dates produces an appealing before-and-after chart. Understanding its shape requires another step: identify who supplies each comparison and whose experience each point represents.

My January 2026 public preprint on design-robust event studies examines the relationship between adoption timing, regression weights and sensitivity to assumptions about untreated outcomes. It treats the design of the comparison as part of the research result.

Keep calendar time and policy time together

Consider a hypothetical programme introduced in one group of regions in 2020 and another in 2022. In 2022, the later group has just entered the programme while the earlier group has already experienced it for two years. A common calendar year therefore contains different stages of exposure.

An event-study chart reorganises observations around time since adoption. Its first-year point might combine both groups, drawing on different calendar years. A third-year point needs a longer follow-up and may have fewer groups available. Neither the calendar comparison nor the population behind the plotted point should disappear when the observations are aligned.

Inspect what the regression compares

Liyang Sun and Sarah Abraham's research shows why conventional two-way fixed-effects event-study coefficients need care when adoption dates and treatment effects differ. A coefficient labelled for one relative time can contain contributions from effects at other relative times. Apparent differences before adoption can even arise from heterogeneous treatment effects elsewhere in the design.

For the hypothetical regions, using an already-exposed group as part of the comparison can bring its evolving programme effect into the estimate for a newly exposed group. Removing permanent regional differences and common calendar shocks does not, by itself, make that comparison an untreated benchmark.

This is a question about the meaning of the coefficient. More observations can estimate the regression's mixture more precisely without turning it into the particular policy effect the reader expects. The estimator and the intended effect need to be matched explicitly.

Keep the population behind the average visible

Brantly Callaway and Pedro H. C. Sant'Anna develop a framework that starts with effects for an adoption group at a particular time. Under the relevant assumptions, comparisons use suitable never-treated or not-yet-treated observations. Those group-time effects can then be combined in ways that answer different questions.

A simple constructed example shows why the aggregation matters. Suppose the early group has an effect of 2 units at every observed post-adoption horizon and the later group has an effect of 6. An equally weighted first-year average is 4. If the dataset ends before the later group reaches its third year, the available third-year average is 2.

The plotted average has fallen although neither group's effect has changed. These are invented values, not findings from the preprint. The example isolates changing composition. Reporting which groups contribute at each horizon, or comparing a common set where feasible, makes that interpretation visible. Restricting the sample also changes whose experience the estimate describes.

Show how much the trend assumption carries

Even a clearly chosen comparison needs a defensible account of what would have happened without the programme. The parallel-trends assumption concerns those unobserved outcomes. A quiet-looking pre-adoption chart cannot establish their later path.

Ashesh Rambachan and Jonathan Roth develop inference that allows specified departures from parallel trends. Their approach asks what remains learnable when post-adoption differences in untreated trends are restricted relative to earlier differences, or constrained in how smoothly they change. The restrictions require substantive justification.

Varying those restrictions produces a sensitivity analysis. Readers can see which conclusions survive a wider range of counterfactual trends. The resulting uncertainty reflects both noisy estimates and limits on identifying the causal effect under the chosen assumptions.

Make the design part of the result

My preprint brings these concerns into a reporting framework, including diagnostics for negative weights and contributions from other event times. It also considers methods designed to reduce sensitivity to errors in estimated auxiliary models when conditioning on observed characteristics.

The public source is a preprint. This article explains its methodological questions and the literature it builds on; it reports no new policy-effect estimate or independent replication. The practical output is a more inspectable chart: adoption groups, eligible comparisons, aggregation weights and sensitivity assumptions presented alongside the estimated effects.

Before asking why an effect grows or fades, establish that the plotted points are answering the intended question. The comparison is part of the evidence.