core/docs/handoff/PHASE-3.1-FOLLOWUP-RECOGNIZER-EXPANSION.md

Phase 3.1 Follow-up — Verb-coverage bottleneck on train_sample/v1

Status: Open recommendation Date: 2026-05-28 Author: Shay (analysis surfaced during ADR-0174 Phase 3a) Parent: ADR-0174 — Held-Hypothesis Comprehension Related ADRs: ADR-0163 (path to GSM8K mastery), ADR-0167 (audit-as-teaching-evidence), ADR-0150/0152/0155/0161 (HITL corridor)

---

Context

ADR-0174 Phase 3 specified a correct ≥ 8 lift target on evals/gsm8k_math/train_sample/v1 (≥ 5 of the 21 currently-empty discrete_count_statement anchors admitted via lookback). Empirical analysis during Phase 3a implementation found this target is not achievable through lookback alone on this corpus. The substrate is built correctly; the bottleneck is elsewhere.

What Phase 3a shipped

• generate/comprehension/lookback.py — the reevaluate operator, PronounResolution refinement type, ReevaluateResult dataclass.
• Held-anchor emission in recognizer_match._try_extract_discrete_count_anchor (pronoun-subject statements carry requires_pronoun_resolution=True rather than refusing).
• Lookback wiring at math_candidate_graph.parse_and_solve's recognizer-injection branch — applies PronounResolution against the existing _discourse_prior_subjects map; emits lookback JSON trace events with outcome ∈ {admitted, eliminated, no_antecedent}.
• 17 acceptance tests proving the wiring works on synthetic problems (tests/test_adr_0174_phase3_lookback.py).
• wrong = 0 invariant preserved; score unchanged at 3/47/0.

Why Phase 3a did not lift the score

The 21 empty-anchor discrete_count_statement refusals on train_sample/v1 break down as:

Structural cause	Cases
Pronoun-only (no compound clause)	2 — 0002, 0034
Compound-only	8
Pronoun + compound	5
Other narrowness fail (verb/structure)	6

For Phase 3a to lift any case, three conditions must all hold:

1. The matcher's recognizer registry recognises the statement.
2. The extractor passes every narrowness layer before the pronoun check. Specifically the verb must be in _POSSESSION_VERBS (has, have, had) or _ACQUISITION_VERBS (collected, collects, collect, received, receives, receive, bought, buys, buy, got, gets, get).
3. The candidate-graph's regex path (_filtered_statement_choices) must return empty for the same statement — otherwise the regex path commits the candidate (with the pronoun still as actor) and the recognizer-injection branch never runs.

Verb checks against the 13 cases with compound/pronoun structure:

Case	Statement (excerpt)	Verb	In whitelist?
0002	She splits it up...	splits	No
0034	He can run 40 yards...	run	No
0020	Two puppies, two kittens... were for sale...	were	No
0021	He bench presses 15 pounds...	presses	No
0027	Malcolm has 240 followers...	has	Yes
0033	Rachel is 12 years old...	is	No
0040	He now has 2 horses...	has	Yes
0041	Troy bakes 2 pans...	bakes	No
0044	John invests in a bank...	invests	No
0045	On Monday he finished 3 surveys...	finished	No
0047	John bakes 12 coconut macaroons...	bakes	No
...

Only two cases (0027, 0040) cross the verb whitelist. Both also fail at the compound-clause narrowness layer (which comes earlier than the pronoun check), so even adding compound-clause held hypotheses (Phase 3b) would have to fire first.

Conclusion: the empirical bottleneck on train_sample/v1 is verb-set coverage, not lookback or held hypotheses. ADR-0174 is the wrong tool for moving this score.

Recommended path forward

ADR-0163 is the correct scope for verb-coverage expansion via the HITL corridor. The path:

1. Run core eval math-contemplation on the 11 failing verbs — splits, run, bench presses, is, bakes, invests, finished, donated, wants, gained, eat. These surface as MathReaderRefusalEvidence audit rows that the contemplation lane already consumes (ADR-0167).

2. Operator review in workbench — categorise each verb:

   • Acquisition-class (engine should treat as add): received, bought, etc. — verbs that grammatically gain quantity to actor. Candidates from list: gained, won, earned, saved, accumulated, acquired.
   • Depletion-class (engine should treat as subtract): gives, loses, spends. Candidates: donated, gave, eats, consumed, lost, spent.
   • Non-arithmetic verbs (engine should refuse and ask): is, wants, bench presses, splits, run, bakes, invests. These do not carry possession/acquisition semantics; the right answer is a different intent (rate / capacity / descriptive), not a wider add/subtract whitelist.

   The first two classes ratify into the registry via the existing ADR-0150/0152 corridor (proposal → review → packed). The third class becomes refusal-typed evidence that informs whether a separate recognizer category is needed (e.g. a capacity_statement recognizer for "He can run 40 yards in 5 seconds" rather than forcing it into discrete_count_statement).

3. After verb widening lands — re-run Phase 3a's lookback wiring on the corpus. The cases that were previously verb-blocked now reach the pronoun-check layer, and the held-hypothesis path admits them. Expected lift from this combination: roughly the 13 cases with pronoun/compound structure that have an arithmetic-class verb under the widened whitelist.

What this means for ADR-0174

The held-hypothesis substrate (Phase 1 + 2 + 3a) is correct architecture and load-bearing for Phase 4 (in-loop contemplation) and Phase 5 (legacy-parser removal). Its eval impact depends on upstream recognizer coverage maturing through the ADR-0163.x corridor. These two efforts are complementary, not competing — the substrate makes lookback possible, the recognizer expansion gives lookback something to fire on.

The cleanest sequencing is:

1. ADR-0174 Phase 3a (this PR) — substrate landed.
2. ADR-0163.x verb expansion (this brief's recommendation) — widens the corpus surface that the substrate can act on.
3. ADR-0174 Phase 3b — compound-clause held hypotheses. Once the verb-coverage bottleneck is gone, compound-clause expansion surfaces real cases. Currently it would surface zero on train_sample for the same reason Phase 3a does: most compound cases also fail the verb check before reaching the clause-split narrowness layer.
4. ADR-0174 Phase 4 — in-loop contemplation. Builds on Phase 3 substrate.
5. ADR-0174 Phase 5 — legacy parser removal.

Decision needed (from operator)

• Authorise the ADR-0163.x verb-expansion contemplation pass? Concretely: run core eval math-contemplation against the 11 failing verbs above; review the proposals in workbench; ratify acquisition/depletion entries that are unambiguous.

• Re-scope ADR-0174 Phase 3b to "post-recognizer-expansion re-measurement" rather than "compound-clause held hypotheses"? Phase 3b should land only after verb expansion exposes cases that exercise its compound-clause logic.

No timelines are proposed; this is a sequencing recommendation. The substrate work in Phase 3a is already merged on its own merits (correctness and Phase 4/5 prerequisite); Phase 3b waits on recognizer coverage.

Cross-references

• ADR-0174 §Phase 3 acceptance — the criteria this brief documents as unmet (with structural-cause analysis).
• tests/test_adr_0174_phase3_lookback.py — proves the substrate works on synthetic problems even though no train_sample case exercises it.
• feedback-wrong-zero-hazard-case-0050 memory — verb expansion must preserve the case-0050 canary; the recommended depletion-class additions should be reviewed against this hazard before ratification.
• thesis-decoding-not-generating — the verb-class contemplation/HITL path is the right "teach the engine to find better" mechanism; widening the static whitelist directly would be "storing another found thing."