Changes to the schema for the MVP

mvp-rows.xlsx is a working copy of the schema spreadsheet (261005 Schema for empirical strategies.xlsx). It keeps only the 18 rows the MVP test cases use, plus the Token glossary tab (trimmed to the tokens those rows use; see section 5). This file records every change made to the copy so far. The original spreadsheet has not been changed.

Rows kept

Staggered rollout (II.J, II.J.1) was dropped from the MVP. II.J.1 is a placeholder: its strategy cells are empty and it names the problem ("forbidden comparisons; modern estimators") without specifying the check. Filling it in needs the staggered-DiD playbook and a ruling on II.J's open note about splitting DiD into staggered / non-staggered columns. The two staggered test cases were dropped with it; the remaining five cases still exercise every part of the MVP.

1. Tokens are written as identifiers

Every [needs: …] and [finding: …] token is now written as a single identifier: lowercase, with words joined by underscores. This makes it clear that a token is a name that must be spelled identically everywhere it appears, not a phrase to paraphrase. A trigger fires only when a token matches exactly.

The same renames were applied in the Token glossary tab.

Source notes removed. Where a trigger named the row a finding comes from, that note was inside the token, for example [finding: large baseline outcome gap between T and C (from I.A.1)] in II.N. That made the token differ from the one I.A.1 produces. The notes were first moved outside the brackets, then removed: the Possible findings column (section 2) now records which row produces each finding. This applies to II.K and II.N in the schema tab and to the anticipation token in the glossary tab.

2. New column: Possible findings

A new column, Possible findings (column E), sits between Trigger conditions and the strategy columns. It lists the findings a row's check can produce, one token per line. The strategy columns move from E–I to F–J.

Why a separate column. Trigger conditions say when a row applies. What a check can produce is a different kind of information, an output rather than a condition, and a check can produce more than one finding (I.A.2 produces three). Previously I.A.1's trigger cell held both: [needs: …]; produces [finding: …]. The "produces" part has moved to the new column, so I.A.1's trigger now reads [needs: baseline_predetermined_covariates].

What the column is for. The code that runs a check decides which finding to emit; this column does not drive that. It serves as the contract: every finding that some row's trigger waits on should appear in some row's Possible findings, spelled the same way. That makes it easy to catch a typo that would leave a row waiting on a finding nothing produces.

3. Interpretation text moved out of the trigger column

Column D (Trigger conditions) should say only when a row applies. II.N's trigger also carried guidance on how to read the finding:

[finding: large_baseline_outcome_gap_between_t_and_c] — level gaps are not disqualifying (absorbed by unit FE); they flag trend risk

The guidance moved, word for word, to the end of II.N's Critique / Question cell (column B), which now reads:

Are baseline level differences generating differential trends? (convergence / mean reversion / functional form) — level gaps are not disqualifying (absorbed by unit FE); they flag trend risk

II.N's trigger is now just [finding: large_baseline_outcome_gap_between_t_and_c].

4. Combining conditions: AND

When a cell has more than one condition, they are joined with AND. (The sheet had used both + and AND.) Needs are not conditions and are listed separately, after a ;, as in II.D.2's trigger: [finding: pre_trends_imprecise_or_violated]; [needs: …].

I.A.2's IV cell had two lines: the original condition in prose, then an illustration (added during review) of how IF conditions could be structured:

IF instrument varies over time and there is a conceptual pre-period

IF [needs: instrument_varies_over_time] AND [needs: there is ...]

Both conditions describe the study design rather than data that could be obtained, so they are design facts, not needs (a missing need greys a card out; a design fact that doesn't hold means the row doesn't apply). The cell now reads:

IF [instrument_timing: time_varying] AND [pre_period: conceptual]

The key is instrument_timing rather than a plain instrument, so that each fact about the instrument has its own key (as with instrument_claim). Its values: time_varying (the instrument's value changes over time for the same unit, e.g., exposure to a price shock) or time_invariant (fixed per unit, e.g., distance to a college, quarter of birth).

II.K.1's and II.K.2's IV cells read IF time-varying instrument, the same condition in prose. Both now read IF [instrument_timing: time_varying]. The condition is also implied by the chain: II.K fires only on I.A.2's anticipation finding, and for IV, I.A.2 applies only when the instrument is time-varying. It is kept explicit anyway.

5. Token glossary trimmed; Legend tab dropped

The glossary is the registry of tokens: every token the rows use should be defined there, spelled the same way, so that a misspelled token can be caught. In the copy it now lists exactly the 14 tokens the 18 rows use, in the order they first appear, and its "Used at" column lists every row that uses each one. Entries for tokens used only by other rows were removed.

Seven tokens had no entry and were added. Two were already missing from the original glossary; five were introduced during this review. Their definitions were written during the review and should be checked:

The Legend tab (cell vocabulary and row conventions) was dropped from the copy. Its rules apply to every row and will be built into the code rather than read from the sheet; the original keeps it.

Open questions

II.D.2: "often triggered by"

II.D.2 (unit-specific trends) has this trigger:

[needs: enough_periods_per_unit_to_separate_trend_from_treatment_effect] — often triggered by [finding: pre_trends_imprecise_or_violated]

The [needs: …] part is a clear condition: the check can run only if there are enough periods per unit. "Often triggered by" is not. The matcher can only evaluate yes/no conditions, so it has to treat the finding one of two ways:

1. The finding is the trigger. The row fires only once I.A.2 returns pre_trends_imprecise_or_violated. I.A.2b already uses the same finding this way.
2. The finding is a priority signal. The row applies to every DiD regardless, and the finding raises it from "consider" to "recommended."

"Often" sits between the two. One has to be chosen for the matcher to use the finding.

Decision (for the MVP): option 1. The finding is the trigger: II.D.2 fires only when I.A.2 returns pre_trends_imprecise_or_violated. The need stays as the input the check requires: if the finding is present but there are too few periods per unit, the row still applies and shows as greyed out. The cell now reads:

[finding: pre_trends_imprecise_or_violated]; [needs: enough_periods_per_unit_to_separate_trend_from_treatment_effect]

No "(from I.A.2)" note: the Possible findings column already records which row produces the finding.

II.D.2: is "panel" a design fact or a need?

II.D.2's IV cell read IF panel and now reads IF [data: panel], reusing the glossary's existing data key (as in [data: aggregated-below-survey-design-level]).

Written as an IF condition, "panel" is treated as a design fact: without a panel, the row doesn't apply and its card disappears. But panel structure describes the dataset, and a researcher could in principle obtain panel data, which makes it behave more like a need. The row already has that need: [needs: enough_periods_per_unit_to_separate_trend_from_treatment_effect] can only be met with a panel. If "panel" were left to the need, the IV cell could simply be Y, and a study without a panel would see the card greyed out (with the missing input named) rather than not see it at all.

Open question: keep IF [data: panel], or make the IV cell Y and let the need handle it?

II.B.1: an N with an exception

II.B.1's Experiment cell read N — unless rollout correlates with other programs. The "unless" makes the N conditional (the same problem as I.G.1 and II.G.2 below), so it is really an IF cell. It now reads:

IF [rival_policies: correlated_with_rollout]

The key reuses the row's own term, "rival policies" (as in its trick and its [needs: data_on_candidate_rival_policies]).

II.N.1 and II.N.2: "panel with pre-period (as I.A.2)"

The IV cells of II.N.1 and II.N.2 read IF panel with pre-period (as I.A.2) and IF panel with pre-period. In II.N.1 the prose and the reference disagree: "panel with pre-period" is about the data, while I.A.2's IV condition is about the instrument ([instrument_timing: time_varying] AND [pre_period: conceptual]). Both cells now follow the prose:

IF [data: panel] AND [pre_period: conceptual]

Open question: is that what was meant, or should these cells carry I.A.2's condition instead?

I.A.2: the "IF continuous treatment" variant in the Diff-in-Diff cell

I.A.2's Diff-in-Diff cell reads:

Y — Note: low-power pre-tests → I.A.2a/b.

IF continuous treatment: no event-time pre-period — plot co-movement by discretized exposure groups (e.g., above/below median); formal leads falsification → II.H.2 (autocorrelation Note)

Three issues:

1. IF is doing two jobs. The Legend defines a leading IF as "applicable only when…". Here the cell's value is Y (the check applies to every DiD), and the later IF means "done differently when…". Several such variants in one cell would be hard to read and impossible to parse. If variants stay, they need one schema-wide convention: either a distinct label in the cell, or their own rows.
2. The condition is ambiguous between two glossary tokens. "Continuous treatment" suggests [treatment: continuous dose] (treatment varies in intensity). But "no event-time pre-period" is the definition of [shock: continuous time-varying] (variation arrives continuously over time, e.g., world prices). A continuous dose with a clear start date still has a pre-period, so the two are different situations.
3. The continuous-shock case may already have a home. The glossary says that under [shock: continuous time-varying], "pre-trend-style checks take the leads/falsification form (II.H.2)", and II.H.2 already fires on that token. If the variant means the shock token, a clean structure would be: I.A.2 does not fire under that token (its event-study leads need a start date); the exposure-group plot moves into II.H.2 beside the leads falsification; and I.A.2's cell drops the variant.

Open question: which token does the variant mean? If the shock token, (3) works with no new row. If it really means a continuous dose with a discrete start date, a separate row (triggered by [treatment: continuous dose]) may be the cleaner home.

Decision (for the MVP): the variant paragraph was dropped from the copy; no MVP test case has a continuous treatment. The cell now reads Y — Note: low-power pre-tests → I.A.2a/b. Its attached note, the exemplar for the variant (Dube & Vargas, REStud 2013: discretized coffee-land groups), was removed with it. The original keeps both until the question above is resolved.

Issues found in the full spreadsheet (rows not in the MVP copy)

These rows are not in mvp-rows.xlsx; they are noted here for fixing in the original.

Y cells whose value is really conditional

The Legend defines a leading IF [condition] as "applicable only when the stated design feature holds." Two Experiment cells start with Y but make it conditional later in the cell, so a reader (or matcher) taking the leading value sees an unconditional Y:

IF conditions are free text, not tokens

Of the 68 cells in the full sheet that start with IF, 67 state their condition in free text. A matcher can only evaluate a condition written as a token, so as written none of these cells can be applied automatically. Example, I.A.1's IV cell:

IF instrument claimed as-good-as-random (lottery, judge leniency, shift-share): balance of covariates w.r.t. Z

Possible form, with the condition as a design-fact token (name to be decided, and added to the glossary):

IF [instrument_claim: as_good_as_random] (e.g., lottery, judge leniency, shift-share): balance of covariates w.r.t. Z

This form has been applied to I.A.1's IV cell in the copy.

instrument_claim rather than instrument_assignment: whether an instrument really is as good as random cannot be observed; what the researcher declares is the claim, which checks like this one then test. "Assignment" also already refers to treatment assignment throughout the sheet (II.A, I.C, I.D). Candidate values: as_good_as_random, conditionally_random (random only within strata or after controls), cutoff, rule_based, natural (e.g., rainfall, distance), model_based (e.g., lags in panel GMM).

This is the largest single cleanup item: each distinct condition needs one token, defined once in the glossary and reused everywhere. The two cells above (I.G.1, II.G.2) would follow the same pattern.