From ecc0072ea1873dd332122539716dbc599ca15195 Mon Sep 17 00:00:00 2001 From: Shay Date: Wed, 27 May 2026 11:08:14 -0700 Subject: [PATCH] chore: remove stub injector + superseded docs (cleanup-as-you-find) (#373) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three concrete cleanup items from the day's work, per the cleanup-as-you-find memory principle. ## 1. Remove inject_rate_with_currency stub PR #369 (A2 rate_with_currency) shipped a function that always returns () with an extensive docstring documenting the Rate-not-in-SentenceChoice schema gap. The function is dead at runtime — `_INJECTORS.get(category)` returning None has the same downstream behavior as the function returning (). The 16 tests pinned the empty-tuple return; the case-0050 hazard pin is duplicated in test_recognizer_skip_wrong_zero.py and test_brief_11b_step2_lexicon.py. The schema gap is now properly documented in ADR-0170 (PR #372). A dispatch-table comment at the removal site retains the at-code pointer to that ADR for anyone wiring a new injector. Removed: - `inject_rate_with_currency` function in generate/recognizer_anchor_inject.py - Its `_INJECTORS` dispatch table entry - Its `__all__` export - tests/test_injector_rate_with_currency.py (371 lines, 16 tests) ## 2. Remove docs/handoff/GPT55-MOBILE-DISPATCH.md Single-session travel-time scaffolding. The 5 tasks it named are complete or superseded by ADR-0170's findings. Pure historical artifact. ## 3. Remove docs/handoff/WAVE-NEXT-INJECTORS.md Superseded by docs/handoff/WAVE-NEXT-REVISED.md, which captures everything load-bearing from the original brief in its A1–A4 findings table. The "kept for history" justification didn't survive scrutiny: the document was misframed (over-promised lift; misframed schema work as injector work). Lessons captured in REVISED + ADR-0170. Updated cross-references: - WAVE-NEXT-REVISED.md: removed the "supersedes ... kept for history" pointer; tightened cross-reference list - ADR-0167-FOLLOWUPS.md §7: rewrote pointer to name ADR-0170 + REVISED as the live plan rather than "the original is retained" ## Test plan - 219 tests passed across G.2/G.4/G.5/S1/Brief 11/B1/B11A/wiring/partition/DCS-D.2 - evals/gsm8k_math/train_sample/v1/report.json untouched (regen surfaces a separate stale-baseline test issue — out of cleanup scope) - No runtime behavior change ## Net impact - 5 files removed (~1200 lines) - 1 file modified for explanatory comment (~30 lines) - 2 doc files updated to remove dangling cross-references - 0 behavioral change --- docs/handoff/ADR-0167-FOLLOWUPS.md | 11 +- docs/handoff/GPT55-MOBILE-DISPATCH.md | 372 -------------------- docs/handoff/WAVE-NEXT-INJECTORS.md | 408 ---------------------- docs/handoff/WAVE-NEXT-REVISED.md | 6 +- generate/recognizer_anchor_inject.py | 85 ++--- tests/test_injector_rate_with_currency.py | 371 -------------------- 6 files changed, 30 insertions(+), 1223 deletions(-) delete mode 100644 docs/handoff/GPT55-MOBILE-DISPATCH.md delete mode 100644 docs/handoff/WAVE-NEXT-INJECTORS.md delete mode 100644 tests/test_injector_rate_with_currency.py diff --git a/docs/handoff/ADR-0167-FOLLOWUPS.md b/docs/handoff/ADR-0167-FOLLOWUPS.md index cc8efb7a..6b6c29e8 100644 --- a/docs/handoff/ADR-0167-FOLLOWUPS.md +++ b/docs/handoff/ADR-0167-FOLLOWUPS.md @@ -246,12 +246,11 @@ version below. - **A4 temporal_aggregation** — schema gap: needs `apply_rate` primitive that doesn't exist in the algebra -The actually-tractable next wave is **DCS sub-shape expansion** — one -focused PR per sub-shape against the existing v1 injector from #315. -See `WAVE-NEXT-REVISED.md` for the sub-shape sequence. - -The original `docs/handoff/WAVE-NEXT-INJECTORS.md` is retained for -history but superseded. +The actually-tractable next wave is **ADR-0170 injector contract +widening** + per-category injector follow-up PRs. See +`WAVE-NEXT-REVISED.md` and `ADR-0170-injector-contract-widening.md` +for the full plan; `DCS-S1-FINDING.md` for the investigation that +surfaced the contract gap. --- diff --git a/docs/handoff/GPT55-MOBILE-DISPATCH.md b/docs/handoff/GPT55-MOBILE-DISPATCH.md deleted file mode 100644 index 4676db9f..00000000 --- a/docs/handoff/GPT55-MOBILE-DISPATCH.md +++ /dev/null @@ -1,372 +0,0 @@ -# GPT-5.5 Mobile/GitHub-Connector Dispatch — In-Flight Work - -**Audience:** GPT-5.5 accessed via mobile + GitHub connector while Shay travels -**Mode:** Read-only execution surface (no test runs, no eval runs, no Python -interpreter). Output is markdown files committed via the connector and PRs -opened against `AssetOverflow/core`. -**Risk profile:** Zero — every task is docs-only, no code paths touched, no -pack mutations, no runtime change. `wrong=0` cannot be violated. -**Cadence:** Pick one task. Complete it fully (including the PR open). Move -to the next. Don't parallelize — mobile + connector tooling is single-thread. - ---- - -## Shared constraints - -- Open one PR per task. Each is a separate branch off `origin/main`. -- Branch naming: `docs/gpt55-task-N-` where `N` is the task number. -- File staging: explicit. **Never** `git add -A`. **Never** commit - `engine_state/`. -- Markdown-only output (CLAUDE.md §Documentation Discipline — no standalone - HTML, no dashboards). Mermaid + `
` collapsibles permitted. -- Honour CLAUDE.md's existing doctrine sections; specifically: - - §"Schema-Defined Proof Obligations" — any new schema you propose must - name an executing test that can meaningfully fail - - §"Non-Negotiable Field Invariant" — never propose anything that - weakens `wrong=0` or the field invariant - - §"Validation Through CLI" — refer to CLI lanes rather than ad-hoc - pytest invocations -- Cite filenames + line numbers (`path/file.py:LINE`) for every code - reference. Verify each reference resolves before committing. -- If a task's deliverable requires a code change (not docs), **stop and - flag it in the PR body** — do not attempt code edits via the connector. - ---- - -## Task 1 — Draft ADR-0168 (FrameClaim scoping) - -**Branch:** `docs/gpt55-task-1-adr-0168-frameclaim` -**Output:** `docs/decisions/ADR-0168-frameclaim-ratification.md` -**Priority:** Highest (this is the next gate after the LexicalClaim slice) - -### Context to read first - -- `docs/decisions/ADR-0167-audit-as-teaching-evidence.md` — the parent - scoping ADR with the five sub-types proposed -- `docs/handoff/ADR-0167-FOLLOWUPS.md` §1 — the queued sub-type work, - specifically the FrameClaim row -- `teaching/math_lexical_ratification.py` — the LexicalClaim handler - template (what your ADR's analogous handler would look like) -- `teaching/math_evidence.py` — `SUB_TYPE_FOR_OPERATOR` table; FrameClaim - maps from `pre_frame_filler_sentence` and `multi_subject_sentence` -- `evals/gsm8k_math/train_sample/v1/audit_brief_11.json` — the 9 - `pre_frame_filler_sentence` cases your ADR will eventually resolve -- `evals/gsm8k_math/train_sample/v1/audit_brief_11.md` §"design tension" - — the rejected one-line fixes and why they fail wrong=0; FrameClaim is - the structural answer -- `language_packs/data/en_core_math_v1/lexicon/` — pack mutation surface - for verb-category reclassification - -### Deliverable shape - -ADR-0168 must answer for FrameClaim what ADR-0167 answered for the -overall wire: - -1. **Scope.** FrameClaim ratifies a verb-category reclassification. - Specifically: when the operator reviews a `pre_frame_filler_sentence` - refusal, FrameClaim's handler reclassifies the unrecognised verb - from `drain_token` (or its current category) to a frame-opener - category (`accumulation_verb` / `depletion_verb` / `transfer_verb` / - `possession_verb` / `capacity_verb`). -2. **Why this is not LexicalClaim.** Reclassification is structurally - different from adding a new lemma: it changes the frame-opening - behaviour of an EXISTING entry. The hazard is real — reclassifying - `does` to `accumulation_verb` would re-introduce the case 0050 - hazard (W2-D pinned this in `SAFE_CATEGORIES`). -3. **Six open questions (analogous to ADR-0167's).** Answer each in - the ADR draft, not in code: - - (Q1) What sub-types of FrameClaim are needed? (E.g. distinct - handlers per target category, or one parameterised handler?) - - (Q2) What new SAFE_CATEGORIES allowlist applies? - - (Q3) How does the ratification prevent the case 0050 hazard? - Concrete answer required, not hand-waved. - - (Q4) What evidence signature normalisation does FrameClaim need? - (Token-only, or token+target-category?) - - (Q5) How does graph completeness gate this category change at the - downstream solver level? - - (Q6) What ablation test would prove this handler doesn't admit - a graph for a sentence whose verb the operator declined to - reclassify? -4. **Three-question test (ADR-0166).** Answer Q1/Q2/Q3 of ADR-0166 - for FrameClaim explicitly. If any of the three doesn't pass cleanly, - say so — the ADR can defer rather than pretend. -5. **Implementation outline.** A wave structure analogous to - ADR-0167's: which W1/W2/W3 deliverables, what operator-to-brief - matching, what's parallelisable. - -### PR body must include - -- Link to ADR-0167 and FOLLOWUPS §1 -- Quote the case 0050 hazard text from - `feedback-wrong-zero-hazard-case-0050` memory (Shay can paste it) -- Explicit "docs-only; no code change" callout -- The recommendation: ship or defer? Whichever, defend it. - -### Out of scope for this task - -- Implementing FrameClaim. ADR is scoping only. -- Touching `teaching/`, `language_packs/`, or any test file. -- New eval lanes (ADR-0166 still gates). - ---- - -## Task 2 — `discrete_count_statement` injector specification audit - -**Branch:** `docs/gpt55-task-2-dcs-injector-spec` -**Output:** `docs/handoff/discrete_count_statement-injector-spec.md` -**Priority:** Highest-leverage (21/47 GSM8K refusals are this one category) - -### Context to read first - -- `evals/gsm8k_math/train_sample/v1/report.json` — the post-eval - refusal records; filter for - `"category=discrete_count_statement"` (21 cases) -- `evals/gsm8k_math/train_sample/v1/cases.jsonl` — original problem - text for each of those 21 cases -- `generate/recognizer_match.py` — the `match` function that's - over-matching -- `generate/recognizer_anchor_inject.py` — the `inject_from_match` - function; the empty-tuple return path is the bug surface -- `engine_state/recognizers.jsonl` (read-only — **never commit this**) - — the ratified recognizer specs including the - `discrete_count_statement` canonical pattern -- `docs/decisions/ADR-0163-gsm8k-path-to-mastery.md` — the roadmap - that introduced this recognizer -- `docs/decisions/ADR-0163.D.2-discrete-count-statement.md` (if it - exists — locate and read it) - -### Deliverable shape - -A specification document, not an implementation. The document must: - -1. **Categorise the 21 cases.** Read each problem text; group by - sub-structure. Common shapes likely include: - - "X has N " pure initial-state - - "X has N and M " multi-quantity initial - - "There are N " subject-anonymous initial - - "N are " attribute-on-count - - Comparatives ("N more than M ") - The grouping is the load-bearing part — exact buckets aren't pre- - determined; let the data dictate. -2. **For each sub-shape**, propose: - - What `parsed_anchors` shape an injector would have to produce - - What `CandidateInitial` / `CandidateOperation` it maps to - - What admissibility check would catch wrong>0 admissions - - Which sub-shapes are LexicalClaim-resolvable (e.g. just a missing - noun) and which need FrameClaim / CompositionClaim -3. **Identify the over-matching root cause.** The recognizer's - canonical pattern matches any number+noun. Propose specific - tightening conditions (e.g. require a frame-opener verb, require - the noun to be in a count-noun whitelist). -4. **Quantify the lift potential.** Of the 21, how many would resolve - under each sub-shape's hypothetical injector? Be honest about - which ones still wouldn't resolve even with the injector (they - have downstream barriers — pronoun, fraction, etc.). -5. **Sequencing recommendation.** Which sub-shape's injector should - ship first? Lift-per-risk, not raw count. - -### PR body must include - -- Per-sub-shape lift estimate (table) -- A statement that NO injector implementation is being proposed — - this PR is specification only -- Cross-reference to ADR-0167-FOLLOWUPS §1 (FrameClaim) and - §"discrete_count_statement over-matching" - -### Out of scope for this task - -- Implementing any injector -- Modifying the recognizer canonical pattern -- Touching `language_packs/` or `teaching/` -- Running the eval (you can't anyway) - ---- - -## Task 3 — Recognizer registry audit - -**Branch:** `docs/gpt55-task-3-recognizer-audit` -**Output:** `docs/handoff/ratified-recognizer-registry-audit.md` -**Priority:** Medium (informs Task 2 and future injector work) - -### Context to read first - -- `engine_state/recognizers.jsonl` (read-only) — the 7 ratified - recognizers from #315 onward -- `generate/recognizer_match.py` — match logic -- `generate/recognizer_anchor_inject.py` — injection logic, including - which categories have injectors and which return `()` -- The eval report from Task 2 — refusal-class counts per recognizer - category - -### Deliverable shape - -A table-driven survey: - -| Recognizer category | Match logic precision | Injector present? | GSM8K refusal count | Lift potential | Risk class | -|---|---|---|---:|---|---| -| `discrete_count_statement` | over-broad | no | 21 | high | high (case 0050 class) | -| `currency_amount` | ? | ? | 4 | ? | ? | -| `rate_with_currency` | ? | ? | 3 | ? | ? | -| ... | ... | ... | ... | ... | ... | - -For each row, write a one-paragraph commentary explaining: -- What the recognizer is supposed to catch -- What it actually catches (the over-broadness or precision) -- Whether the injector is feasible (lexical-only? structural? multi-pack?) -- The case 0050 hazard analogue for THIS category - -### PR body must include - -- A "promote injector / tighten match / retire recognizer" recommendation - for each row -- An "if you fix one, fix this one first" prioritisation - -### Out of scope for this task - -- Implementing any recognizer change -- Retiring any recognizer (proposal-only) -- Touching `engine_state/` directly — read-only - ---- - -## Task 4 — FOLLOWUPS §6 ablation test specification - -**Branch:** `docs/gpt55-task-4-holonomy-ablation-spec` -**Output:** `docs/handoff/holonomy-ablation-test-spec.md` -**Priority:** Low-urgency, high-information - -### Context to read first - -- `docs/handoff/ADR-0167-FOLLOWUPS.md` §6 (when merged from PR #360) -- `language_packs/compiler.py:558` — `_apply_mounted_primary_domain_resonance` - (the architectural-invariant comment names the gap) -- `tests/test_alignment_graph.py:73` — - `test_holonomy_alignment_case_positive_closer_than_negative` (the - existing proof) -- `language_packs/schema.py:181` — `HolonomyAlignmentCase` schema - -### Deliverable shape - -A specification (not an implementation) for an ablation test that -isolates *structurally-derived* convergence from *blend-induced* -convergence. The spec must: - -1. **Name the ablation surface.** What part of - `_apply_mounted_primary_domain_resonance` needs to be temporarily - disabled or parameterised for the test? -2. **Name the test contract.** With ablation active (blend factor = 0), - does the positive-closer-than-negative assertion still hold? If - yes, structural derivation is real; if no, the test is gated by the - blend. -3. **Name the predicted outcome.** Best guess: blend-gated. Document - why (the 40% nudge is sizeable; without it, the morphology rotors - alone may not produce enough convergence). -4. **Name the honest reframing path.** If the ablation fails, the - `HolonomyAlignmentCase` contract should be reframed from "proves - structural divergence with coherent convergence" to "proves - endpoint similarity under the mount-time blend." Suggest the exact - docstring/schema text. - -### PR body must include - -- Cross-reference to FOLLOWUPS §6 and CLAUDE.md §"Schema-Defined Proof - Obligations" -- Explicit "spec only; no test implementation in this PR" callout - -### Out of scope - -- Implementing the ablation test -- Modifying the holonomy test or schema -- Modifying `_apply_mounted_primary_domain_resonance` - ---- - -## Task 5 — Cognition contemplation partition fix specification - -**Branch:** `docs/gpt55-task-5-contemplation-partition-spec` -**Output:** `docs/handoff/contemplation-pack-indexing-partition-spec.md` -**Priority:** Medium (this is FOLLOWUPS §5a) - -### Context to read first - -- `docs/handoff/ADR-0167-FOLLOWUPS.md` §5a -- `docs/handoff/ADR-0167-W2C-cross-domain-audit.md` — Gemini's W2-C - audit; the specific partition risks -- `teaching/contemplation.py::contemplate()` — the function that uses - hardcoded cognition pack/corpus indexes -- `teaching/discovery.py` — `DiscoveryCandidate` with the `domain` - field added by W2-C -- `language_packs/data/en_core_math_v1/` — what a math pack looks like - (for the alternate-domain branch) - -### Deliverable shape - -A surgical patch specification (not an implementation): - -1. **Inventory.** Which exact lines in `teaching/contemplation.py` - assume cognition? -2. **Patch surface.** Minimum change to make those lines respect - `candidate.domain`. -3. **Test surface.** What test(s) would catch a regression where a - math candidate silently fetches cognition pack data? -4. **Backwards compatibility.** Confirm the default (`domain="cognition"`) - preserves current behaviour byte-identically. - -### PR body must include - -- Cross-reference to W2-C audit and FOLLOWUPS §5a -- "Spec only; implementation in a follow-up PR" callout - -### Out of scope - -- Implementing the patch -- Touching `teaching/contemplation.py` -- Running cognition regression tests (you can't anyway) - ---- - -## Operational notes - -- **Pace yourself.** Mobile + connector tooling has latency. One task per - session is honourable; trying to finish all five in one go invites errors. -- **Cite line numbers.** Every code reference must include `path:LINE` and - be verified to resolve. If you can't verify via the connector, drop the - specific line number and reference the function name instead. -- **No code edits.** If a task starts feeling like it needs a code change, - flag it in the PR body and stop. Do not attempt code edits via the - connector — the test discipline can't be honoured from mobile. -- **Honest progress reports.** Each PR's body should report what you - actually concluded — including any sub-shapes you couldn't categorise, - any line numbers you couldn't verify, any open questions that need - Shay's input. -- **If you finish all five.** Open a meta-PR adding a section to - `docs/handoff/ADR-0167-FOLLOWUPS.md` linking to all five spec docs. - -## What NOT to attempt from the connector - -- Implementing FrameClaim, CompositionClaim, or any other handler -- Implementing any injector for `discrete_count_statement` or any other - recognizer category -- Implementing the holonomy ablation test -- Implementing the contemplation partition fix -- Running `core test --suite *` or `core eval cognition` (mobile cannot) -- Mutating any pack file under `language_packs/data/` -- Committing anything under `engine_state/` -- Force-pushing or rewriting history on any branch - -If in doubt, the rule is: **specs and audits, not implementation.** - ---- - -## Cross-references (for context) - -- `CLAUDE.md` — project doctrine -- `docs/decisions/ADR-0166-measurement-capability-sequencing.md` — the - three-question test every spec must answer -- `docs/decisions/ADR-0167-audit-as-teaching-evidence.md` — the parent - wire all five sub-types extend -- `docs/handoff/ADR-0167-FOLLOWUPS.md` — the canonical follow-up queue; - Tasks 1, 2, 4, 5 all extend items already named there -- `docs/decisions/SESSION-2026-05-27-adr-0167-parallel-dispatch.md` — - the wave narrative; reading this gives the full context for why each - task is shaped the way it is diff --git a/docs/handoff/WAVE-NEXT-INJECTORS.md b/docs/handoff/WAVE-NEXT-INJECTORS.md deleted file mode 100644 index d3ca7e47..00000000 --- a/docs/handoff/WAVE-NEXT-INJECTORS.md +++ /dev/null @@ -1,408 +0,0 @@ -# Wave-Next — Recognizer Injectors + Lexical Closure + CompositionClaim Scoping - -**Date:** 2026-05-27 -**Goal:** Lift GSM8K `correct` from 3 → 10+ (ADR-0163 Round-1 gate) -via the recognizer-injector path identified in the post-eval analysis. -**Risk profile:** Low. Each brief is a focused single-category injector -with explicit `wrong=0` pinning. Composition / Frame work is deferred -to subsequent waves with their own ADRs. - ---- - -## Operator pool (as of 2026-05-27) - -- **Sonnet 4.6** — workhorse for mechanical injector work; can run 3+ parallel agents -- **Opus 4.6/4.7** — deepest reasoning; reserved for briefs with real design calls -- **Gemini** — long-context surveys only (per `feedback-parallel-dispatch-pattern`) -- **GitHub Copilot** — held in reserve; less proven for this workflow -- **Codex** — OFFLINE (rate limits, several days) - ---- - -## Dispatch timeline - -### Gate 1 — cascade complete - -**Wait for #362 → #363 → #364 → #365 → #366 to all merge.** That puts on `main`: -- partition-test behavioral invariant (#362) — unblocks future ADR-0167 PRs -- domain-aware contemplation routing (#363) — partitions cognition vs math -- ADR-0168 FrameClaim scoping (#364) — names the next major sub-type ADR -- ADR-0168.1 adapter bridge (#365) — resolves the ADR-0057 evidence floor tension -- DCS injector spec (#366) — the methodology document A1–A4 reference - -Verify the cascade with `git fetch origin main && git log origin/main --oneline -6`. - -### Gate 2 — dispatch the parallel injector wave - -**Single message → 4 parallel Agent calls:** - -| Order | Brief | Operator | Branch | -|---|---|---|---| -| 1 | A1 currency_amount injector | Sonnet | `feat/injector-currency-amount` | -| 2 | A3 multiplicative_aggregation injector | Sonnet | `feat/injector-multiplicative-aggregation` | -| 3 | A4 temporal_aggregation injector | Sonnet | `feat/injector-temporal-aggregation` | -| 4 | A2 rate_with_currency injector | Opus | `feat/injector-rate-with-currency` | - -All 4 touch `generate/recognizer_anchor_inject.py`. First to push opens -clean; the other 3 need a union-merge rebase. Each rebase is trivial -(adding a function + a dispatch-table line). - -**In parallel with that wave**, the orchestrator (me) handles **B1** -inline — too small to dispatch. - -### Gate 3 — sequential after the injector wave settles - -| Brief | Operator | Branch | -|---|---|---| -| D1 ADR-0169 CompositionClaim scoping | Opus | `docs/adr-0169-compositionclaim-scoping` | - -D1 is docs-only, no code conflicts. Can technically run in parallel -with the injector wave; sequencing it after lets Opus give A2 full -attention first. - -### Gate 4 — optional background research - -| Brief | Operator | Branch | -|---|---|---| -| GPT-5.5 dispatch Task 3 (recognizer registry audit) | Gemini | `docs/gemini-recognizer-registry-audit` | - -Pure long-context survey of all 7 ratified recognizers. No code, no -risk. Informs future injector PRs. Run if you want background research -while the injector wave executes; skip if you don't want the noise. - ---- - -## Shared constraints (every brief inherits these) - -- Open a dedicated `git worktree add` (parallel-agent worktree rule) -- Branch off **current `main`** after Gate 1 confirms the cascade is in -- `wrong == 0` non-negotiable — verify against case `gsm8k-train-sample-v1-0050` - in every test suite -- ADR-0166 — no new canonical eval lanes; reuse `gsm8k_math/train_sample/v1` -- No teaching-store / pack mutation as a side effect of injector work -- `uv venv` / `uv pip install` / `uv run` — never `--break-system-packages` -- Stage explicit files; never `git add -A`; NEVER commit `engine_state/` -- Each PR runs the full regression suite (see Validation block per brief) -- CLAUDE.md §"Documentation Discipline" — pure markdown, no standalone HTML -- CLAUDE.md §"Schema-Defined Proof Obligations" — every new injector - must come with a test that can meaningfully fail under the wrong=0 - violations the injector is written to catch - ---- - -## A1 — `currency_amount` injector - -**Recommended operator:** Sonnet 4.6 -**Branch:** `feat/injector-currency-amount` -**Expected lift:** 2–4 cases (4 currently refused as `currency_amount`) -**Blocked by:** Gate 1 (cascade complete) - -### Context to read first - -- `generate/recognizer_anchor_inject.py:79` — `inject_discrete_count_statement` - (the existing template; do not reuse logic, just shape) -- `generate/recognizer_match.py` — the `currency_amount` match logic -- `engine_state/recognizers.jsonl` (read-only) — the ratified - `currency_amount` canonical pattern -- `docs/handoff/discrete_count_statement-injector-spec.md` (post-#366) - — the methodology for "narrow first, broaden later" -- `evals/gsm8k_math/train_sample/v1/report.json` — filter for - `category=currency_amount`; these are the 4 cases you target -- `evals/gsm8k_math/train_sample/v1/cases.jsonl` — the original problem - text for each - -### Setup - -```bash -git worktree add /tmp/wt-a1 -b feat/injector-currency-amount origin/main -cd /tmp/wt-a1 -uv venv && source .venv/bin/activate -uv pip install -e . -``` - -### Deliverables - -1. **`generate/recognizer_anchor_inject.py`** — new - `inject_currency_amount(match) -> tuple[CandidateInitial | CandidateOperation, ...]` - function. Add entry to the dispatch table at the bottom of the file. - Must: - - extract `currency` + `amount` + `entity` from `match.parsed_anchors` - - emit ONE `CandidateInitial` per match in the narrow canonical form - ` has|earns|charges $` - - return `()` (preserve refusal) for any shape outside that narrow - form — broadening is a follow-up PR - - never emit a `CandidateOperation` (those are FrameClaim territory) -2. **`tests/test_injector_currency_amount.py`** (new) — 8+ tests: - - happy path: narrow canonical form admits a complete graph - - sub-shape rejection: 2+ variant shapes the injector deliberately - skips (must return `()`, not raise) - - hazard pin: case `gsm8k-train-sample-v1-0050` remains refused at - `sentence_index=0` - - determinism: same `RecognizerMatch` → byte-identical injector output - - wrong=0 invariant: any admitted graph passes - `assert_graph_complete` and the existing solver's verifier -3. **Eval delta artifact** — append a new section to - `evals/gsm8k_math/train_sample/v1/audit_brief_11.md` documenting: - - which N cases moved from `currency_amount` refusal to admission - - which cases remained refused on a different bottleneck class - - confirmation that `wrong` count remains 0 - -### Hard constraints - -- The narrow form is non-negotiable. **Do not** match comparatives, - rate compositions, or multi-currency arithmetic in this PR -- Reject any shape where the entity is anonymous (`The store earns ...` - vs `Sam earns ...`) -- Manifest checksums unchanged (no pack file edits) -- Reader path remains the priority — flag-on reader still runs before - recognizer; this injector only fires on reader refusal - -### Verification - -```bash -uv run pytest tests/test_injector_currency_amount.py -q -uv run pytest tests/test_brief_11b_audit_artifact.py tests/test_brief_11b_step2_lexicon.py tests/test_recognizer_skip_wrong_zero.py -q -uv run pytest tests/ -k "teaching or contemplation or candidate or correction or store or review" -q -PYTHONPATH=. uv run python evals/gsm8k_math/train_sample/v1/runner.py -``` - -Capture the before/after `report.json` counts in the PR body. - -### PR body must include - -- Before/after refusal taxonomy for the `currency_amount` row -- Case-by-case verdict for the 4 currently-refused cases (admitted / - refused-on-different-class) -- Explicit case 0050 hazard verification line -- `wrong=0` invariant statement - -### Report back - -- PR URL -- Lift count (cases moved from refused → admitted) -- Hazard pin evidence -- Any sub-shapes you noticed that need follow-up injector PRs - ---- - -## A2 — `rate_with_currency` injector - -**Recommended operator:** Opus 4.6/4.7 -**Branch:** `feat/injector-rate-with-currency` -**Expected lift:** 1–3 cases (3 currently refused) -**Blocked by:** Gate 1 - -### Why Opus instead of Sonnet - -This brief has a real schema decision: does the existing -`Quantity` type in `generate/math_problem_graph.py` structurally model -a per-unit rate? If yes, the injector emits a `Rate`-shaped -`CandidateInitial` analogous to A1. If no, the injector must -**explicitly refuse** rather than invent a new type — flag for -follow-up. That decision needs judgment, not pattern-matching. - -### Setup, context, deliverables, hard constraints - -Identical structure to A1, but for `rate_with_currency`. Canonical -narrow form: ` earns|charges|pays $ per ` or -` earns|charges|pays $ for `. - -Specific differences from A1: - -- Check `generate/math_problem_graph.py` for the `Quantity` type - structure; if it doesn't model rates, the injector returns `()` - and the PR body writes an explicit follow-up note -- If `Quantity` does model rates (e.g. via a composite unit or a - separate `Rate` type), use that — DO NOT invent a new type -- Hazard pin: case 0050 still refused - -### Report back must include - -- The schema decision (does `Quantity` model rates?) and your evidence -- If "no," the follow-up note for whoever ships the `Rate` schema - extension -- Lift count (will be 0 if schema decision is "no" — that's still a - successful PR; documenting the gap is the deliverable) - ---- - -## A3 — `multiplicative_aggregation` injector - -**Recommended operator:** Sonnet 4.6 -**Branch:** `feat/injector-multiplicative-aggregation` -**Expected lift:** 2–4 cases (5 currently refused) -**Blocked by:** Gate 1 - -### Why this needs care - -This is the **first injector that emits `CandidateOperation`** (not -just `CandidateInitial`). Multiplicative operations widen the case -0050 hazard surface — if the operand isn't the right unit, the -solver computes a wrong product. - -### Canonical narrow form - -` has in each ` or -` per `. The injector emits a -`CandidateOperation` of kind `multiply` when the count, noun, and -container all extract cleanly from `parsed_anchors`. - -### Extra hazard pinning (beyond A1's spec) - -Reject any shape where: -- the container isn't a `count_unit_noun` -- the multiplier isn't a determinate integer or word-form integer -- the result unit doesn't match the original count unit - -The `tests/test_injector_multiplicative_aggregation.py` must include -a parameterized test confirming each of those rejection paths -returns `()` rather than admitting a wrong-product graph. - -### Otherwise identical to A1's structure - -Same deliverables, hard constraints, verification, PR body, report-back. - ---- - -## A4 — `temporal_aggregation` injector - -**Recommended operator:** Sonnet 4.6 -**Branch:** `feat/injector-temporal-aggregation` -**Expected lift:** 1–2 cases (2 currently refused) -**Blocked by:** Gate 1 - -### Why this is the structural sanity check - -Smallest injector in the wave. If a focused PR can lift the 2 cases, -the recognizer-injector pattern is operational and the larger sub-shape -work (especially DCS sub-shapes) can follow with confidence. - -### Canonical narrow form - -` per ` (e.g. `5 hours per day`, -`3 days per week`). Emits a `Rate`-shaped or multiplicative-shaped -candidate depending on context. - -### Coordinate with A3 - -Both A3 and A4 may produce multiplicative-kind operations. If the -`Quantity`/`Operation` schema doesn't distinguish them cleanly, -flag in the PR body for shared follow-up. - -### Otherwise identical to A1's structure - ---- - -## B1 — Lexical-entry closure: remaining 3 cases - -**Recommended operator:** Orchestrator (me) — too small to dispatch -**Branch:** `feat/lexicon-closure-wave-3` -**Expected lift:** 1–3 cases -**Blocked by:** Gate 1 (cascade complete) - -Three `lexicon_entry` refusals remain after #348: -- case 0001: `+` (arithmetic literal — DO NOT add as drain_token) -- case 0040: `sees` (perception verb — drain_token candidate) -- case 0049: `path` (noun — drain_token candidate) - -This is small (12 lines of edits, 3 test additions) and I'll handle -it in-line while the injector wave runs. Decision-making for `+` -documented in PR body (it's a structural issue, not a lexical gap). - ---- - -## D1 — ADR-0169 CompositionClaim scoping - -**Recommended operator:** Opus 4.6/4.7 -**Branch:** `docs/adr-0169-compositionclaim-scoping` -**Output:** `docs/decisions/ADR-0169-compositionclaim-ratification.md` -**Blocked by:** ADR-0168 (#364) merged — Gate 1 -**Sequencing:** Run after A2 lands (Opus needs full attention on A2 first) - -### Deliverable shape - -A scoping ADR analogous to ADR-0168 (#364), answering the same six -open questions for `CompositionClaim`: - -1. Sub-types of CompositionClaim needed? -2. SAFE_CATEGORIES allowlist applicable? -3. Concrete answer to how the ratification prevents the case 0050 - hazard (multi-quantity is exactly the hazard surface — a wrong - composition rule could admit `5 apples + 3 oranges = 8 things`) -4. Evidence signature normalisation needed -5. Graph completeness gating -6. Ablation test that proves the handler doesn't admit a partial - composition - -Plus ADR-0166 three-question test, plus compatibility audit against -ADR-0056/0057/0114a/0164/0165/0166/0167/0168, plus implementation -wave outline. - -### Hard constraints - -- Docs-only; no code, no test, no eval, no pack change -- Must explicitly address: "is CompositionClaim safer or riskier than - FrameClaim?" — argue from data, not intuition -- If "riskier," propose deferring CompositionClaim until FrameClaim - ships a clean second-sub-type precedent - ---- - -## (Optional) Background research — Gemini recognizer registry audit - -**Recommended operator:** Gemini -**Branch:** `docs/gemini-recognizer-registry-audit` -**Output:** `docs/handoff/ratified-recognizer-registry-audit.md` -**Blocked by:** Nothing — pure read-only survey - -This is GPT-5.5 dispatch Task 3 from the prior session that wasn't -picked up. Pure long-context audit of all 7 ratified recognizers in -`engine_state/recognizers.jsonl`. Output is a table-driven survey -naming: match-logic precision, injector presence/absence, GSM8K -refusal count, lift potential, hazard class. - -Informs future injector PRs. Independent of A1–A4. Skip if you don't -want background research running in parallel. - ---- - -## What this wave does NOT do - -- It does not implement `discrete_count_statement` sub-shapes (21 - largest bucket). That's Wave C, informed by #366's spec post-merge. -- It does not implement FrameClaim (Wave E, requires ADR-0168 merged - AND its own W1-W3 sub-wave). -- It does not add new eval lanes (ADR-0166 still gates). -- It does not touch workbench wiring (ADR-0167 §Q4, deferred). -- It does not propose any non-deterministic / non-decoding mechanism. - -## Expected aggregate lift - -If A1–A4 all ship cleanly: **6–13 cases lifted** out of the 14 -across those four categories. Plus B1: **1–3 cases**. - -That puts `correct` at **10–19**, clearing ADR-0163 Round-1 -(`correct ≥ 10`) and potentially nudging Round-2 (`correct ≥ 25`). - -A2's lift may be 0 if the `Quantity` schema doesn't model rates — -in that case the PR's value is documenting the gap, and the lift -shifts to A3 + A4. - ---- - -## Dispatch protocol summary - -```text -1. Wait for cascade #362→#366 (Gate 1) -2. Single message with 4 Agent calls: - - subagent_type=general-purpose, model=sonnet → A1 - - subagent_type=general-purpose, model=sonnet → A3 - - subagent_type=general-purpose, model=sonnet → A4 - - subagent_type=general-purpose, model=opus → A2 -3. Orchestrator handles B1 inline -4. After A2 lands: subagent_type=planner, model=opus → D1 -5. (Optional) subagent_type=general-purpose, model=sonnet (or Gemini) → Task 3 audit -``` - -Each operator gets pointed at this file's section header for their -brief. Shared constraints at the top apply to everyone. diff --git a/docs/handoff/WAVE-NEXT-REVISED.md b/docs/handoff/WAVE-NEXT-REVISED.md index 3e63f145..fc00ee61 100644 --- a/docs/handoff/WAVE-NEXT-REVISED.md +++ b/docs/handoff/WAVE-NEXT-REVISED.md @@ -1,7 +1,8 @@ # Wave-Next Revised — DCS Sub-Shapes + Schema-Gap Backlog **Date:** 2026-05-27 -**Supersedes:** `docs/handoff/WAVE-NEXT-INJECTORS.md` (kept for history) +**Supersedes:** the original Wave-Next injector briefs (removed in +cleanup PR; the four A-findings and pivot rationale are captured here) **Why revised:** The A1–A4 dispatch surfaced findings that invalidate three of the four briefs' lift assumptions. This doc replaces them with the actually-tractable next steps. @@ -173,15 +174,12 @@ No timelines. Order is by leverage, not calendar. ## What this document does NOT do - It does not dispatch any agents (per `feedback-no-self-dispatch-of-subagents`) -- It does not retire `WAVE-NEXT-INJECTORS.md` (kept for history; future - readers should consult this document for current state) - It does not modify any runtime code - It does not add new eval lanes (ADR-0166) - It does not propose any non-deterministic mechanism ## Cross-references -- `docs/handoff/WAVE-NEXT-INJECTORS.md` — original (now-superseded) brief - `docs/handoff/discrete_count_statement-injector-spec.md` — the DCS sub-shape spec - `docs/handoff/ADR-0167-FOLLOWUPS.md` — parent follow-up queue - `docs/decisions/ADR-0168-frameclaim-ratification.md` — FrameClaim scoping diff --git a/generate/recognizer_anchor_inject.py b/generate/recognizer_anchor_inject.py index da96d662..9c4a5f19 100644 --- a/generate/recognizer_anchor_inject.py +++ b/generate/recognizer_anchor_inject.py @@ -230,75 +230,36 @@ def _locate_possession_verb(sentence: str) -> str | None: # registers its injector. No global state, no side effects. # --------------------------------------------------------------------------- -def inject_rate_with_currency( - match: RecognizerMatch, - sentence: str, -) -> tuple[CandidateInitial, ...]: - """Inject a ``rate_with_currency`` recognizer match. - - Schema decision (Wave-Next A2): the :class:`Rate` type in - :mod:`generate.math_problem_graph` (ADR-0122) DOES structurally - model a per-unit rate via ``(value, numerator_unit, - denominator_unit)``. However, ``Rate`` is the operand of an - ``Operation(kind='apply_rate')`` — it is not, and cannot be - coerced into, a :class:`CandidateInitial` or a standalone - :class:`CandidateOperation` that the per-sentence choice - dispatcher consumes: - - - :class:`InitialPossession` requires a :class:`Quantity` (scalar - + unit) and an entity. A rate carries two units; the - rate-declaration sentence alone ("Tina makes $18.00 an hour.") - does not establish how many hours Tina worked, so the - denominator quantity is unknown. - - ``Operation(kind='apply_rate')`` requires the denominator - quantity (the "how many X" the rate multiplies) to be present - in the same sentence; an isolated rate-declaration sentence - does not carry it. - - ``SentenceChoice = Union[CandidateInitial, CandidateOperation]`` - (see :mod:`generate.math_candidate_graph`) — there is no - ``CandidateRate`` variant for the injector to deposit. - - The existing - :func:`generate.math_candidate_parser.extract_earnings_candidates` - handles rate-declaration sentences via a separate short-circuit - path keyed on a sibling :class:`CandidateEarningsRate` type that - is NOT part of ``SentenceChoice``. It is consumed by a special - earnings short-circuit in - :func:`generate.math_candidate_graph.parse_and_solve` BEFORE the - per-sentence-choices Cartesian product runs. - - Conclusion: the recognizer-injector contract (return a tuple of - :class:`CandidateInitial`) cannot meaningfully express a per-unit - rate without a wider ``SentenceChoice`` union. v1 therefore - RETURNS ``()`` (refusal-preferring, wrong=0 doctrine) and - documents the gap. This is the explicit-refusal A2 outcome. - - Follow-up (separate PR): extend ``SentenceChoice`` with a - ``CandidateRate`` variant carrying a :class:`Rate` operand keyed - by actor, and teach - :func:`generate.math_candidate_graph.parse_and_solve` to compose - a ``CandidateRate`` with a downstream apply_rate/multiply-shaped - question. Only at that point can a recognizer-injector emit - useful state for ``rate_with_currency``. - """ - return () - - _INJECTORS: Mapping[ShapeCategory, "type"] = { ShapeCategory.DISCRETE_COUNT_STATEMENT: inject_discrete_count_statement, # type: ignore[dict-item] - ShapeCategory.RATE_WITH_CURRENCY: inject_rate_with_currency, # type: ignore[dict-item] - # The four other recognizer categories route to the empty-tuple - # fallback (skip-only) until their D.2.x injector lands: + # All other recognizer categories route to the empty-tuple fallback + # in ``inject_from_match`` — `_INJECTORS.get(category)` returns + # ``None`` and the dispatcher returns ``()``, which the + # candidate-graph then treats as "recognizer matched but produced + # no injection" → explicit refusal (the wrong=0 fix from #359). # - # ShapeCategory.DESCRIPTIVE_SETUP_NO_QUANTITY — by design (no quantity) - # ShapeCategory.TEMPORAL_AGGREGATION — D.2.3 follow-up - # ShapeCategory.MULTIPLICATIVE_AGGREGATION — D.2.4 follow-up - # ShapeCategory.CURRENCY_AMOUNT — D.2.5 follow-up + # Categories deferred to follow-up PRs: + # + # ShapeCategory.DESCRIPTIVE_SETUP_NO_QUANTITY — by design (no quantity) + # ShapeCategory.RATE_WITH_CURRENCY — needs CandidateRate + # (SentenceChoice union + # extension; ADR-0171) + # ShapeCategory.TEMPORAL_AGGREGATION — needs apply_rate primitive + # in the algebra + # ShapeCategory.MULTIPLICATIVE_AGGREGATION — emits + # CandidateInitial(product) + # after ADR-0170 widens + # return type + # ShapeCategory.CURRENCY_AMOUNT — A1 currency_amount; + # CandidateInitial-shaped, + # ships after ADR-0170 + # + # See docs/decisions/ADR-0170-injector-contract-widening.md for the + # contract widening that unblocks DCS-S1 / A1 / A3. } __all__ = [ "inject_from_match", "inject_discrete_count_statement", - "inject_rate_with_currency", ] diff --git a/tests/test_injector_rate_with_currency.py b/tests/test_injector_rate_with_currency.py deleted file mode 100644 index ddd7b433..00000000 --- a/tests/test_injector_rate_with_currency.py +++ /dev/null @@ -1,371 +0,0 @@ -"""Wave-Next A2 — ``rate_with_currency`` injector. - -This test file is the load-bearing artifact for the A2 injector. The -A2 outcome is an explicit, documented schema-refusal: ``Rate`` (ADR-0122) -DOES structurally model a per-unit rate, but it is not a member of the -``SentenceChoice = Union[CandidateInitial, CandidateOperation]`` union -the per-sentence injector contract requires. The injector therefore -returns ``()`` and the load-bearing assertions in this file pin: - - a. SCHEMA EVIDENCE — ``Rate`` exists and structurally models - a (value, numerator_unit, denominator_unit) - per-unit rate, distinct from ``Quantity``. - b. SCHEMA REFUSAL — the injector returns ``()`` for every - shape (broad and narrow canonical forms). - c. DISPATCH WIRED — dispatch table routes - ``RATE_WITH_CURRENCY`` to the injector - (no longer the empty-tuple default). - d. CASE 0050 HAZARD PIN — case - ``gsm8k-train-sample-v1-0050`` remains - refused at sentence_index=0 (sentence - carries no currency, so it neither - matches nor would be lifted by A2). - e. DETERMINISM — identical ``(match, sentence)`` → - byte-identical injector output. - f. NO STATE INJECTED — the injector never produces a - ``CandidateInitial`` (would be a wrong=0 - hazard, since Rate ≠ Quantity). -""" - -from __future__ import annotations - -import json -from pathlib import Path - -from evals.refusal_taxonomy.shape_categories import ShapeCategory -from generate.math_candidate_graph import SentenceChoice, parse_and_solve -from generate.math_problem_graph import Quantity, Rate -from generate.recognizer_anchor_inject import ( - _INJECTORS, - inject_from_match, - inject_rate_with_currency, -) -from generate.recognizer_match import RecognizerMatch -from generate.recognizer_registry import RatifiedRecognizer - - -# --------------------------------------------------------------------------- -# Synthetic match builder — mirrors the A1/D.2 test pattern. -# --------------------------------------------------------------------------- - - -def _make_match(parsed_anchors: tuple[dict, ...]) -> RecognizerMatch: - rec = RatifiedRecognizer( - proposal_id="test-rate-with-currency", - shape_category=ShapeCategory.RATE_WITH_CURRENCY, - canonical_pattern={ - "anchor_kind": "currency_per_unit_rate", - "shape_category": "rate_with_currency", - "graph_intent": "rate", - "anchor_count_min": 1, - "anchor_count_max": 1, - "outcome": "admissible", - "observed_currency_symbols": ["$"], - "observed_per_units": ["hour", "day", "week"], - }, - spec_digest="test-digest", - review_date="2026-05-27", - ratified_at_revision="test", - ) - return RecognizerMatch( - recognizer=rec, - category=ShapeCategory.RATE_WITH_CURRENCY, - outcome="admissible", - graph_intent="rate", - parsed_anchors=parsed_anchors, - ) - - -# --------------------------------------------------------------------------- -# (a) Schema evidence — Rate models a per-unit rate. -# --------------------------------------------------------------------------- - - -class TestSchemaEvidence: - """The schema decision: does ``Quantity`` model a per-unit rate? - - Answer: NO — ``Quantity`` is a scalar+unit pair, not a rate. BUT - a separate ``Rate`` type (ADR-0122) DOES structurally model the - per-unit rate via ``numerator_unit`` / ``denominator_unit``. The - A2 schema-refusal hinges not on the absence of ``Rate`` but on - its absence from the ``SentenceChoice`` union. - """ - - def test_quantity_does_not_model_a_rate(self) -> None: - # Quantity is value + unit; no numerator/denominator distinction. - q = Quantity(value=18.0, unit="dollars") - assert q.value == 18.0 - assert q.unit == "dollars" - # No rate-shaped attributes: this is exactly the gap A2 documents. - assert not hasattr(q, "numerator_unit") - assert not hasattr(q, "denominator_unit") - - def test_rate_type_exists_and_models_per_unit_rate(self) -> None: - # Rate(18, "dollars", "hour") means "18 dollars per hour". - r = Rate(value=18.0, numerator_unit="dollars", denominator_unit="hour") - assert r.value == 18.0 - assert r.numerator_unit == "dollars" - assert r.denominator_unit == "hour" - - def test_sentence_choice_union_excludes_rate(self) -> None: - # The per-sentence injector contract is - # ``SentenceChoice = Union[CandidateInitial, CandidateOperation]``. - # No CandidateRate exists. This is the load-bearing reason the - # A2 injector cannot meaningfully emit a rate primitive. - from generate.math_candidate_parser import CandidateInitial - from generate.math_roundtrip import CandidateOperation - - # The Union is realised structurally — every SentenceChoice - # must be one of these two types. The test pins the closed - # set; expanding it is the explicit follow-up. - allowed = {CandidateInitial, CandidateOperation} - # Best-effort introspection of the Union type. Python's - # ``Union`` exposes its members via ``__args__`` on the alias. - import typing - - args = set(typing.get_args(SentenceChoice)) - assert args == allowed - - -# --------------------------------------------------------------------------- -# (b) Schema refusal — every shape returns (). -# --------------------------------------------------------------------------- - - -class TestSchemaRefusal: - """A2 v1 refuses every input shape, by design.""" - - def test_canonical_per_form_refuses(self) -> None: - m = _make_match(( - { - "kind": "currency_per_unit_rate", - "currency_symbol": "$", - "amount": "18.00", - "amount_kind": "decimal", - "per_unit": "hour", - }, - )) - out = inject_rate_with_currency(m, "Tina makes $18.00 an hour.") - assert out == () - - def test_canonical_for_form_refuses(self) -> None: - m = _make_match(( - { - "kind": "currency_per_unit_rate", - "currency_symbol": "$", - "amount": "30", - "amount_kind": "integer", - "per_unit": "hour", - }, - )) - out = inject_rate_with_currency(m, "Sam charges $30 for each hour.") - assert out == () - - def test_empty_parsed_anchors_refuses(self) -> None: - # No anchors → no possible state regardless of schema. - m = _make_match(()) - out = inject_rate_with_currency(m, "Tina makes $18.00 an hour.") - assert out == () - - def test_returns_empty_tuple_never_raises(self) -> None: - # Adversarial input: malformed anchor payload. The injector - # MUST NOT raise; it MUST return ``()``. - m = _make_match(({"kind": "currency_per_unit_rate", "junk": True},)) - out = inject_rate_with_currency(m, "") - assert out == () - - def test_injector_never_emits_candidate_initial(self) -> None: - # Iterate over a spread of shapes; none may admit any candidate. - sentences = ( - "Tina makes $18.00 an hour.", - "Sam charges $30 for each hour.", - "Bob pays $5 per cup.", - "Alice earns $100 a day.", - ) - for s in sentences: - m = _make_match(( - { - "kind": "currency_per_unit_rate", - "currency_symbol": "$", - "amount": "1", - "amount_kind": "integer", - "per_unit": "hour", - }, - )) - assert inject_rate_with_currency(m, s) == () - - -# --------------------------------------------------------------------------- -# (c) Dispatch wired — registry routes RATE_WITH_CURRENCY to injector. -# --------------------------------------------------------------------------- - - -class TestDispatchWired: - def test_injector_table_registers_rate_with_currency(self) -> None: - # Before A2: RATE_WITH_CURRENCY was absent from _INJECTORS - # (default empty-tuple skip). After A2: present and routed - # to inject_rate_with_currency. - assert ShapeCategory.RATE_WITH_CURRENCY in _INJECTORS - assert _INJECTORS[ShapeCategory.RATE_WITH_CURRENCY] is inject_rate_with_currency - - def test_dispatch_returns_empty_tuple_via_registry(self) -> None: - m = _make_match(( - { - "kind": "currency_per_unit_rate", - "currency_symbol": "$", - "amount": "18.00", - "amount_kind": "decimal", - "per_unit": "hour", - }, - )) - out = inject_from_match(m, "Tina makes $18.00 an hour.") - assert out == () - - def test_dispatch_equals_direct_call(self) -> None: - m = _make_match(( - { - "kind": "currency_per_unit_rate", - "currency_symbol": "$", - "amount": "18.00", - "amount_kind": "decimal", - "per_unit": "hour", - }, - )) - s = "Tina makes $18.00 an hour." - assert inject_from_match(m, s) == inject_rate_with_currency(m, s) - - -# --------------------------------------------------------------------------- -# (d) Case 0050 hazard pin — sentence_index=0 stays refused. -# --------------------------------------------------------------------------- - - -_CASES_PATH = ( - Path(__file__).resolve().parent.parent - / "evals" - / "gsm8k_math" - / "train_sample" - / "v1" - / "cases.jsonl" -) - - -def _load_case_0050() -> dict: - """Look up case 0050 from the fixed eval cases file.""" - with _CASES_PATH.open() as f: - for line in f: - c = json.loads(line) - if c["case_id"] == "gsm8k-train-sample-v1-0050": - return c - raise AssertionError("case gsm8k-train-sample-v1-0050 not found in cases.jsonl") - - -class TestCase0050HazardPin: - """Case 0050: "Mark does a gig every other day for 2 weeks. ..." - - Sentence 0 carries no currency symbol — rate_with_currency never - matches it. Even if A2 v1 emitted state (it doesn't), this case - would not be reachable through the A2 path. This test makes the - invariant explicit so a future A2 widening cannot silently lift - the case 0050 hazard. - """ - - def test_sentence_zero_has_no_currency_symbol(self) -> None: - case = _load_case_0050() - # The case's question text is the full problem; split into - # sentences on '.' the same way the candidate-graph does. - sentences = [s.strip() for s in case["question"].split(".") if s.strip()] - sentence_zero = sentences[0] - assert "Mark does a gig" in sentence_zero - for symbol in "$£€¥": - assert symbol not in sentence_zero - - def test_case_0050_remains_refused_end_to_end(self) -> None: - case = _load_case_0050() - r = parse_and_solve(case["question"]) - assert r.answer is None - # The wrong reading (~3 minutes, per the audit_brief_11 note) - # MUST never appear. 280 is the correct expected answer; the - # injector must not regress to either. - assert r.is_admitted is False - - -# --------------------------------------------------------------------------- -# (e) Determinism — same input, byte-identical output. -# --------------------------------------------------------------------------- - - -class TestDeterminism: - def test_injection_is_deterministic(self) -> None: - m = _make_match(( - { - "kind": "currency_per_unit_rate", - "currency_symbol": "$", - "amount": "18.00", - "amount_kind": "decimal", - "per_unit": "hour", - }, - )) - s = "Tina makes $18.00 an hour." - out1 = inject_rate_with_currency(m, s) - out2 = inject_rate_with_currency(m, s) - assert out1 == out2 - assert out1 == () # explicitly pinned: refusal - - def test_dispatch_is_deterministic(self) -> None: - m = _make_match(( - { - "kind": "currency_per_unit_rate", - "currency_symbol": "$", - "amount": "18.00", - "amount_kind": "decimal", - "per_unit": "hour", - }, - )) - s = "Tina makes $18.00 an hour." - out1 = inject_from_match(m, s) - out2 = inject_from_match(m, s) - assert out1 == out2 - - -# --------------------------------------------------------------------------- -# (f) Wrong=0 invariant — no candidate is ever produced. -# --------------------------------------------------------------------------- - - -class TestWrongZeroInvariant: - """The strongest possible wrong=0 statement: the injector emits - nothing. A wider follow-up must replace this assertion with a - grounded admissibility check on the (Rate, Quantity) composition. - """ - - def test_no_candidate_emitted_for_any_known_shape(self) -> None: - # Every shape the existing matcher could produce. - anchor_variants = ( - { - "kind": "currency_per_unit_rate", - "currency_symbol": "$", - "amount": "18.00", - "amount_kind": "decimal", - "per_unit": "hour", - }, - { - "kind": "currency_per_unit_rate", - "currency_symbol": "$", - "amount": "5", - "amount_kind": "integer", - "per_unit": "cup", - }, - { - "kind": "currency_per_unit_rate", - "currency_symbol": "$", - "amount": "1/2", - "amount_kind": "word", - "per_unit": "pound", - }, - ) - for anchor in anchor_variants: - m = _make_match((anchor,)) - out = inject_rate_with_currency(m, "irrelevant under refusal") - assert out == () - assert len(out) == 0