From 4beda0bfd74c2f8e956af20e3289d1a876ba4a91 Mon Sep 17 00:00:00 2001 From: Shay Date: Wed, 27 May 2026 09:51:40 -0700 Subject: [PATCH] =?UTF-8?q?docs(handoff):=20Wave-Next=20injector=20briefs?= =?UTF-8?q?=20=E2=80=94=20A1-A4=20+=20B1=20+=20D1=20+=20optional=20audit?= =?UTF-8?q?=20(#367)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Six dispatchable briefs covering the next-progress path on GSM8K (correct from 3 → 10+, ADR-0163 Round-1 gate). Wave A — four parallel recognizer-injector PRs: - A1 currency_amount → Sonnet (2-4 cases lift) - A2 rate_with_currency → Opus (schema decision required) - A3 multiplicative_aggregation → Sonnet (first CandidateOperation) - A4 temporal_aggregation → Sonnet (structural sanity) Wave B — orchestrator-handled inline: - B1 lexical-entry closure for 3 remaining cases Wave D — sequential after A2 lands: - D1 ADR-0169 CompositionClaim scoping → Opus Optional background: - Gemini recognizer registry audit (GPT-5.5 Task 3 unclaimed) Dispatch gated on the #362→#366 cascade fully merging. Codex is offline (rate limits); allocation reflects Sonnet + Opus + Gemini. Each brief carries explicit case 0050 hazard pins, narrow-form constraints, wrong=0 verification commands, and report-back questions. --- docs/handoff/WAVE-NEXT-INJECTORS.md | 408 ++++++++++++++++++++++++++++ 1 file changed, 408 insertions(+) create mode 100644 docs/handoff/WAVE-NEXT-INJECTORS.md diff --git a/docs/handoff/WAVE-NEXT-INJECTORS.md b/docs/handoff/WAVE-NEXT-INJECTORS.md new file mode 100644 index 00000000..d3ca7e47 --- /dev/null +++ b/docs/handoff/WAVE-NEXT-INJECTORS.md @@ -0,0 +1,408 @@ +# Wave-Next — Recognizer Injectors + Lexical Closure + CompositionClaim Scoping + +**Date:** 2026-05-27 +**Goal:** Lift GSM8K `correct` from 3 → 10+ (ADR-0163 Round-1 gate) +via the recognizer-injector path identified in the post-eval analysis. +**Risk profile:** Low. Each brief is a focused single-category injector +with explicit `wrong=0` pinning. Composition / Frame work is deferred +to subsequent waves with their own ADRs. + +--- + +## Operator pool (as of 2026-05-27) + +- **Sonnet 4.6** — workhorse for mechanical injector work; can run 3+ parallel agents +- **Opus 4.6/4.7** — deepest reasoning; reserved for briefs with real design calls +- **Gemini** — long-context surveys only (per `feedback-parallel-dispatch-pattern`) +- **GitHub Copilot** — held in reserve; less proven for this workflow +- **Codex** — OFFLINE (rate limits, several days) + +--- + +## Dispatch timeline + +### Gate 1 — cascade complete + +**Wait for #362 → #363 → #364 → #365 → #366 to all merge.** That puts on `main`: +- partition-test behavioral invariant (#362) — unblocks future ADR-0167 PRs +- domain-aware contemplation routing (#363) — partitions cognition vs math +- ADR-0168 FrameClaim scoping (#364) — names the next major sub-type ADR +- ADR-0168.1 adapter bridge (#365) — resolves the ADR-0057 evidence floor tension +- DCS injector spec (#366) — the methodology document A1–A4 reference + +Verify the cascade with `git fetch origin main && git log origin/main --oneline -6`. + +### Gate 2 — dispatch the parallel injector wave + +**Single message → 4 parallel Agent calls:** + +| Order | Brief | Operator | Branch | +|---|---|---|---| +| 1 | A1 currency_amount injector | Sonnet | `feat/injector-currency-amount` | +| 2 | A3 multiplicative_aggregation injector | Sonnet | `feat/injector-multiplicative-aggregation` | +| 3 | A4 temporal_aggregation injector | Sonnet | `feat/injector-temporal-aggregation` | +| 4 | A2 rate_with_currency injector | Opus | `feat/injector-rate-with-currency` | + +All 4 touch `generate/recognizer_anchor_inject.py`. First to push opens +clean; the other 3 need a union-merge rebase. Each rebase is trivial +(adding a function + a dispatch-table line). + +**In parallel with that wave**, the orchestrator (me) handles **B1** +inline — too small to dispatch. + +### Gate 3 — sequential after the injector wave settles + +| Brief | Operator | Branch | +|---|---|---| +| D1 ADR-0169 CompositionClaim scoping | Opus | `docs/adr-0169-compositionclaim-scoping` | + +D1 is docs-only, no code conflicts. Can technically run in parallel +with the injector wave; sequencing it after lets Opus give A2 full +attention first. + +### Gate 4 — optional background research + +| Brief | Operator | Branch | +|---|---|---| +| GPT-5.5 dispatch Task 3 (recognizer registry audit) | Gemini | `docs/gemini-recognizer-registry-audit` | + +Pure long-context survey of all 7 ratified recognizers. No code, no +risk. Informs future injector PRs. Run if you want background research +while the injector wave executes; skip if you don't want the noise. + +--- + +## Shared constraints (every brief inherits these) + +- Open a dedicated `git worktree add` (parallel-agent worktree rule) +- Branch off **current `main`** after Gate 1 confirms the cascade is in +- `wrong == 0` non-negotiable — verify against case `gsm8k-train-sample-v1-0050` + in every test suite +- ADR-0166 — no new canonical eval lanes; reuse `gsm8k_math/train_sample/v1` +- No teaching-store / pack mutation as a side effect of injector work +- `uv venv` / `uv pip install` / `uv run` — never `--break-system-packages` +- Stage explicit files; never `git add -A`; NEVER commit `engine_state/` +- Each PR runs the full regression suite (see Validation block per brief) +- CLAUDE.md §"Documentation Discipline" — pure markdown, no standalone HTML +- CLAUDE.md §"Schema-Defined Proof Obligations" — every new injector + must come with a test that can meaningfully fail under the wrong=0 + violations the injector is written to catch + +--- + +## A1 — `currency_amount` injector + +**Recommended operator:** Sonnet 4.6 +**Branch:** `feat/injector-currency-amount` +**Expected lift:** 2–4 cases (4 currently refused as `currency_amount`) +**Blocked by:** Gate 1 (cascade complete) + +### Context to read first + +- `generate/recognizer_anchor_inject.py:79` — `inject_discrete_count_statement` + (the existing template; do not reuse logic, just shape) +- `generate/recognizer_match.py` — the `currency_amount` match logic +- `engine_state/recognizers.jsonl` (read-only) — the ratified + `currency_amount` canonical pattern +- `docs/handoff/discrete_count_statement-injector-spec.md` (post-#366) + — the methodology for "narrow first, broaden later" +- `evals/gsm8k_math/train_sample/v1/report.json` — filter for + `category=currency_amount`; these are the 4 cases you target +- `evals/gsm8k_math/train_sample/v1/cases.jsonl` — the original problem + text for each + +### Setup + +```bash +git worktree add /tmp/wt-a1 -b feat/injector-currency-amount origin/main +cd /tmp/wt-a1 +uv venv && source .venv/bin/activate +uv pip install -e . +``` + +### Deliverables + +1. **`generate/recognizer_anchor_inject.py`** — new + `inject_currency_amount(match) -> tuple[CandidateInitial | CandidateOperation, ...]` + function. Add entry to the dispatch table at the bottom of the file. + Must: + - extract `currency` + `amount` + `entity` from `match.parsed_anchors` + - emit ONE `CandidateInitial` per match in the narrow canonical form + ` has|earns|charges $` + - return `()` (preserve refusal) for any shape outside that narrow + form — broadening is a follow-up PR + - never emit a `CandidateOperation` (those are FrameClaim territory) +2. **`tests/test_injector_currency_amount.py`** (new) — 8+ tests: + - happy path: narrow canonical form admits a complete graph + - sub-shape rejection: 2+ variant shapes the injector deliberately + skips (must return `()`, not raise) + - hazard pin: case `gsm8k-train-sample-v1-0050` remains refused at + `sentence_index=0` + - determinism: same `RecognizerMatch` → byte-identical injector output + - wrong=0 invariant: any admitted graph passes + `assert_graph_complete` and the existing solver's verifier +3. **Eval delta artifact** — append a new section to + `evals/gsm8k_math/train_sample/v1/audit_brief_11.md` documenting: + - which N cases moved from `currency_amount` refusal to admission + - which cases remained refused on a different bottleneck class + - confirmation that `wrong` count remains 0 + +### Hard constraints + +- The narrow form is non-negotiable. **Do not** match comparatives, + rate compositions, or multi-currency arithmetic in this PR +- Reject any shape where the entity is anonymous (`The store earns ...` + vs `Sam earns ...`) +- Manifest checksums unchanged (no pack file edits) +- Reader path remains the priority — flag-on reader still runs before + recognizer; this injector only fires on reader refusal + +### Verification + +```bash +uv run pytest tests/test_injector_currency_amount.py -q +uv run pytest tests/test_brief_11b_audit_artifact.py tests/test_brief_11b_step2_lexicon.py tests/test_recognizer_skip_wrong_zero.py -q +uv run pytest tests/ -k "teaching or contemplation or candidate or correction or store or review" -q +PYTHONPATH=. uv run python evals/gsm8k_math/train_sample/v1/runner.py +``` + +Capture the before/after `report.json` counts in the PR body. + +### PR body must include + +- Before/after refusal taxonomy for the `currency_amount` row +- Case-by-case verdict for the 4 currently-refused cases (admitted / + refused-on-different-class) +- Explicit case 0050 hazard verification line +- `wrong=0` invariant statement + +### Report back + +- PR URL +- Lift count (cases moved from refused → admitted) +- Hazard pin evidence +- Any sub-shapes you noticed that need follow-up injector PRs + +--- + +## A2 — `rate_with_currency` injector + +**Recommended operator:** Opus 4.6/4.7 +**Branch:** `feat/injector-rate-with-currency` +**Expected lift:** 1–3 cases (3 currently refused) +**Blocked by:** Gate 1 + +### Why Opus instead of Sonnet + +This brief has a real schema decision: does the existing +`Quantity` type in `generate/math_problem_graph.py` structurally model +a per-unit rate? If yes, the injector emits a `Rate`-shaped +`CandidateInitial` analogous to A1. If no, the injector must +**explicitly refuse** rather than invent a new type — flag for +follow-up. That decision needs judgment, not pattern-matching. + +### Setup, context, deliverables, hard constraints + +Identical structure to A1, but for `rate_with_currency`. Canonical +narrow form: ` earns|charges|pays $ per ` or +` earns|charges|pays $ for `. + +Specific differences from A1: + +- Check `generate/math_problem_graph.py` for the `Quantity` type + structure; if it doesn't model rates, the injector returns `()` + and the PR body writes an explicit follow-up note +- If `Quantity` does model rates (e.g. via a composite unit or a + separate `Rate` type), use that — DO NOT invent a new type +- Hazard pin: case 0050 still refused + +### Report back must include + +- The schema decision (does `Quantity` model rates?) and your evidence +- If "no," the follow-up note for whoever ships the `Rate` schema + extension +- Lift count (will be 0 if schema decision is "no" — that's still a + successful PR; documenting the gap is the deliverable) + +--- + +## A3 — `multiplicative_aggregation` injector + +**Recommended operator:** Sonnet 4.6 +**Branch:** `feat/injector-multiplicative-aggregation` +**Expected lift:** 2–4 cases (5 currently refused) +**Blocked by:** Gate 1 + +### Why this needs care + +This is the **first injector that emits `CandidateOperation`** (not +just `CandidateInitial`). Multiplicative operations widen the case +0050 hazard surface — if the operand isn't the right unit, the +solver computes a wrong product. + +### Canonical narrow form + +` has in each ` or +` per `. The injector emits a +`CandidateOperation` of kind `multiply` when the count, noun, and +container all extract cleanly from `parsed_anchors`. + +### Extra hazard pinning (beyond A1's spec) + +Reject any shape where: +- the container isn't a `count_unit_noun` +- the multiplier isn't a determinate integer or word-form integer +- the result unit doesn't match the original count unit + +The `tests/test_injector_multiplicative_aggregation.py` must include +a parameterized test confirming each of those rejection paths +returns `()` rather than admitting a wrong-product graph. + +### Otherwise identical to A1's structure + +Same deliverables, hard constraints, verification, PR body, report-back. + +--- + +## A4 — `temporal_aggregation` injector + +**Recommended operator:** Sonnet 4.6 +**Branch:** `feat/injector-temporal-aggregation` +**Expected lift:** 1–2 cases (2 currently refused) +**Blocked by:** Gate 1 + +### Why this is the structural sanity check + +Smallest injector in the wave. If a focused PR can lift the 2 cases, +the recognizer-injector pattern is operational and the larger sub-shape +work (especially DCS sub-shapes) can follow with confidence. + +### Canonical narrow form + +` per ` (e.g. `5 hours per day`, +`3 days per week`). Emits a `Rate`-shaped or multiplicative-shaped +candidate depending on context. + +### Coordinate with A3 + +Both A3 and A4 may produce multiplicative-kind operations. If the +`Quantity`/`Operation` schema doesn't distinguish them cleanly, +flag in the PR body for shared follow-up. + +### Otherwise identical to A1's structure + +--- + +## B1 — Lexical-entry closure: remaining 3 cases + +**Recommended operator:** Orchestrator (me) — too small to dispatch +**Branch:** `feat/lexicon-closure-wave-3` +**Expected lift:** 1–3 cases +**Blocked by:** Gate 1 (cascade complete) + +Three `lexicon_entry` refusals remain after #348: +- case 0001: `+` (arithmetic literal — DO NOT add as drain_token) +- case 0040: `sees` (perception verb — drain_token candidate) +- case 0049: `path` (noun — drain_token candidate) + +This is small (12 lines of edits, 3 test additions) and I'll handle +it in-line while the injector wave runs. Decision-making for `+` +documented in PR body (it's a structural issue, not a lexical gap). + +--- + +## D1 — ADR-0169 CompositionClaim scoping + +**Recommended operator:** Opus 4.6/4.7 +**Branch:** `docs/adr-0169-compositionclaim-scoping` +**Output:** `docs/decisions/ADR-0169-compositionclaim-ratification.md` +**Blocked by:** ADR-0168 (#364) merged — Gate 1 +**Sequencing:** Run after A2 lands (Opus needs full attention on A2 first) + +### Deliverable shape + +A scoping ADR analogous to ADR-0168 (#364), answering the same six +open questions for `CompositionClaim`: + +1. Sub-types of CompositionClaim needed? +2. SAFE_CATEGORIES allowlist applicable? +3. Concrete answer to how the ratification prevents the case 0050 + hazard (multi-quantity is exactly the hazard surface — a wrong + composition rule could admit `5 apples + 3 oranges = 8 things`) +4. Evidence signature normalisation needed +5. Graph completeness gating +6. Ablation test that proves the handler doesn't admit a partial + composition + +Plus ADR-0166 three-question test, plus compatibility audit against +ADR-0056/0057/0114a/0164/0165/0166/0167/0168, plus implementation +wave outline. + +### Hard constraints + +- Docs-only; no code, no test, no eval, no pack change +- Must explicitly address: "is CompositionClaim safer or riskier than + FrameClaim?" — argue from data, not intuition +- If "riskier," propose deferring CompositionClaim until FrameClaim + ships a clean second-sub-type precedent + +--- + +## (Optional) Background research — Gemini recognizer registry audit + +**Recommended operator:** Gemini +**Branch:** `docs/gemini-recognizer-registry-audit` +**Output:** `docs/handoff/ratified-recognizer-registry-audit.md` +**Blocked by:** Nothing — pure read-only survey + +This is GPT-5.5 dispatch Task 3 from the prior session that wasn't +picked up. Pure long-context audit of all 7 ratified recognizers in +`engine_state/recognizers.jsonl`. Output is a table-driven survey +naming: match-logic precision, injector presence/absence, GSM8K +refusal count, lift potential, hazard class. + +Informs future injector PRs. Independent of A1–A4. Skip if you don't +want background research running in parallel. + +--- + +## What this wave does NOT do + +- It does not implement `discrete_count_statement` sub-shapes (21 + largest bucket). That's Wave C, informed by #366's spec post-merge. +- It does not implement FrameClaim (Wave E, requires ADR-0168 merged + AND its own W1-W3 sub-wave). +- It does not add new eval lanes (ADR-0166 still gates). +- It does not touch workbench wiring (ADR-0167 §Q4, deferred). +- It does not propose any non-deterministic / non-decoding mechanism. + +## Expected aggregate lift + +If A1–A4 all ship cleanly: **6–13 cases lifted** out of the 14 +across those four categories. Plus B1: **1–3 cases**. + +That puts `correct` at **10–19**, clearing ADR-0163 Round-1 +(`correct ≥ 10`) and potentially nudging Round-2 (`correct ≥ 25`). + +A2's lift may be 0 if the `Quantity` schema doesn't model rates — +in that case the PR's value is documenting the gap, and the lift +shifts to A3 + A4. + +--- + +## Dispatch protocol summary + +```text +1. Wait for cascade #362→#366 (Gate 1) +2. Single message with 4 Agent calls: + - subagent_type=general-purpose, model=sonnet → A1 + - subagent_type=general-purpose, model=sonnet → A3 + - subagent_type=general-purpose, model=sonnet → A4 + - subagent_type=general-purpose, model=opus → A2 +3. Orchestrator handles B1 inline +4. After A2 lands: subagent_type=planner, model=opus → D1 +5. (Optional) subagent_type=general-purpose, model=sonnet (or Gemini) → Task 3 audit +``` + +Each operator gets pointed at this file's section header for their +brief. Shared constraints at the top apply to everyone.