chore: remove stub injector + superseded docs (cleanup-as-you-find) (#373)
Three concrete cleanup items from the day's work, per the cleanup-as-you-find memory principle. ## 1. Remove inject_rate_with_currency stub PR #369 (A2 rate_with_currency) shipped a function that always returns () with an extensive docstring documenting the Rate-not-in-SentenceChoice schema gap. The function is dead at runtime — `_INJECTORS.get(category)` returning None has the same downstream behavior as the function returning (). The 16 tests pinned the empty-tuple return; the case-0050 hazard pin is duplicated in test_recognizer_skip_wrong_zero.py and test_brief_11b_step2_lexicon.py. The schema gap is now properly documented in ADR-0170 (PR #372). A dispatch-table comment at the removal site retains the at-code pointer to that ADR for anyone wiring a new injector. Removed: - `inject_rate_with_currency` function in generate/recognizer_anchor_inject.py - Its `_INJECTORS` dispatch table entry - Its `__all__` export - tests/test_injector_rate_with_currency.py (371 lines, 16 tests) ## 2. Remove docs/handoff/GPT55-MOBILE-DISPATCH.md Single-session travel-time scaffolding. The 5 tasks it named are complete or superseded by ADR-0170's findings. Pure historical artifact. ## 3. Remove docs/handoff/WAVE-NEXT-INJECTORS.md Superseded by docs/handoff/WAVE-NEXT-REVISED.md, which captures everything load-bearing from the original brief in its A1–A4 findings table. The "kept for history" justification didn't survive scrutiny: the document was misframed (over-promised lift; misframed schema work as injector work). Lessons captured in REVISED + ADR-0170. Updated cross-references: - WAVE-NEXT-REVISED.md: removed the "supersedes ... kept for history" pointer; tightened cross-reference list - ADR-0167-FOLLOWUPS.md §7: rewrote pointer to name ADR-0170 + REVISED as the live plan rather than "the original is retained" ## Test plan - 219 tests passed across G.2/G.4/G.5/S1/Brief 11/B1/B11A/wiring/partition/DCS-D.2 - evals/gsm8k_math/train_sample/v1/report.json untouched (regen surfaces a separate stale-baseline test issue — out of cleanup scope) - No runtime behavior change ## Net impact - 5 files removed (~1200 lines) - 1 file modified for explanatory comment (~30 lines) - 2 doc files updated to remove dangling cross-references - 0 behavioral change
This commit is contained in:
parent
ce6b2f5173
commit
ecc0072ea1
6 changed files with 30 additions and 1223 deletions
|
|
@ -246,12 +246,11 @@ version below.
|
|||
- **A4 temporal_aggregation** — schema gap: needs `apply_rate`
|
||||
primitive that doesn't exist in the algebra
|
||||
|
||||
The actually-tractable next wave is **DCS sub-shape expansion** — one
|
||||
focused PR per sub-shape against the existing v1 injector from #315.
|
||||
See `WAVE-NEXT-REVISED.md` for the sub-shape sequence.
|
||||
|
||||
The original `docs/handoff/WAVE-NEXT-INJECTORS.md` is retained for
|
||||
history but superseded.
|
||||
The actually-tractable next wave is **ADR-0170 injector contract
|
||||
widening** + per-category injector follow-up PRs. See
|
||||
`WAVE-NEXT-REVISED.md` and `ADR-0170-injector-contract-widening.md`
|
||||
for the full plan; `DCS-S1-FINDING.md` for the investigation that
|
||||
surfaced the contract gap.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,372 +0,0 @@
|
|||
# GPT-5.5 Mobile/GitHub-Connector Dispatch — In-Flight Work
|
||||
|
||||
**Audience:** GPT-5.5 accessed via mobile + GitHub connector while Shay travels
|
||||
**Mode:** Read-only execution surface (no test runs, no eval runs, no Python
|
||||
interpreter). Output is markdown files committed via the connector and PRs
|
||||
opened against `AssetOverflow/core`.
|
||||
**Risk profile:** Zero — every task is docs-only, no code paths touched, no
|
||||
pack mutations, no runtime change. `wrong=0` cannot be violated.
|
||||
**Cadence:** Pick one task. Complete it fully (including the PR open). Move
|
||||
to the next. Don't parallelize — mobile + connector tooling is single-thread.
|
||||
|
||||
---
|
||||
|
||||
## Shared constraints
|
||||
|
||||
- Open one PR per task. Each is a separate branch off `origin/main`.
|
||||
- Branch naming: `docs/gpt55-task-N-<slug>` where `N` is the task number.
|
||||
- File staging: explicit. **Never** `git add -A`. **Never** commit
|
||||
`engine_state/`.
|
||||
- Markdown-only output (CLAUDE.md §Documentation Discipline — no standalone
|
||||
HTML, no dashboards). Mermaid + `<details>` collapsibles permitted.
|
||||
- Honour CLAUDE.md's existing doctrine sections; specifically:
|
||||
- §"Schema-Defined Proof Obligations" — any new schema you propose must
|
||||
name an executing test that can meaningfully fail
|
||||
- §"Non-Negotiable Field Invariant" — never propose anything that
|
||||
weakens `wrong=0` or the field invariant
|
||||
- §"Validation Through CLI" — refer to CLI lanes rather than ad-hoc
|
||||
pytest invocations
|
||||
- Cite filenames + line numbers (`path/file.py:LINE`) for every code
|
||||
reference. Verify each reference resolves before committing.
|
||||
- If a task's deliverable requires a code change (not docs), **stop and
|
||||
flag it in the PR body** — do not attempt code edits via the connector.
|
||||
|
||||
---
|
||||
|
||||
## Task 1 — Draft ADR-0168 (FrameClaim scoping)
|
||||
|
||||
**Branch:** `docs/gpt55-task-1-adr-0168-frameclaim`
|
||||
**Output:** `docs/decisions/ADR-0168-frameclaim-ratification.md`
|
||||
**Priority:** Highest (this is the next gate after the LexicalClaim slice)
|
||||
|
||||
### Context to read first
|
||||
|
||||
- `docs/decisions/ADR-0167-audit-as-teaching-evidence.md` — the parent
|
||||
scoping ADR with the five sub-types proposed
|
||||
- `docs/handoff/ADR-0167-FOLLOWUPS.md` §1 — the queued sub-type work,
|
||||
specifically the FrameClaim row
|
||||
- `teaching/math_lexical_ratification.py` — the LexicalClaim handler
|
||||
template (what your ADR's analogous handler would look like)
|
||||
- `teaching/math_evidence.py` — `SUB_TYPE_FOR_OPERATOR` table; FrameClaim
|
||||
maps from `pre_frame_filler_sentence` and `multi_subject_sentence`
|
||||
- `evals/gsm8k_math/train_sample/v1/audit_brief_11.json` — the 9
|
||||
`pre_frame_filler_sentence` cases your ADR will eventually resolve
|
||||
- `evals/gsm8k_math/train_sample/v1/audit_brief_11.md` §"design tension"
|
||||
— the rejected one-line fixes and why they fail wrong=0; FrameClaim is
|
||||
the structural answer
|
||||
- `language_packs/data/en_core_math_v1/lexicon/` — pack mutation surface
|
||||
for verb-category reclassification
|
||||
|
||||
### Deliverable shape
|
||||
|
||||
ADR-0168 must answer for FrameClaim what ADR-0167 answered for the
|
||||
overall wire:
|
||||
|
||||
1. **Scope.** FrameClaim ratifies a verb-category reclassification.
|
||||
Specifically: when the operator reviews a `pre_frame_filler_sentence`
|
||||
refusal, FrameClaim's handler reclassifies the unrecognised verb
|
||||
from `drain_token` (or its current category) to a frame-opener
|
||||
category (`accumulation_verb` / `depletion_verb` / `transfer_verb` /
|
||||
`possession_verb` / `capacity_verb`).
|
||||
2. **Why this is not LexicalClaim.** Reclassification is structurally
|
||||
different from adding a new lemma: it changes the frame-opening
|
||||
behaviour of an EXISTING entry. The hazard is real — reclassifying
|
||||
`does` to `accumulation_verb` would re-introduce the case 0050
|
||||
hazard (W2-D pinned this in `SAFE_CATEGORIES`).
|
||||
3. **Six open questions (analogous to ADR-0167's).** Answer each in
|
||||
the ADR draft, not in code:
|
||||
- (Q1) What sub-types of FrameClaim are needed? (E.g. distinct
|
||||
handlers per target category, or one parameterised handler?)
|
||||
- (Q2) What new SAFE_CATEGORIES allowlist applies?
|
||||
- (Q3) How does the ratification prevent the case 0050 hazard?
|
||||
Concrete answer required, not hand-waved.
|
||||
- (Q4) What evidence signature normalisation does FrameClaim need?
|
||||
(Token-only, or token+target-category?)
|
||||
- (Q5) How does graph completeness gate this category change at the
|
||||
downstream solver level?
|
||||
- (Q6) What ablation test would prove this handler doesn't admit
|
||||
a graph for a sentence whose verb the operator declined to
|
||||
reclassify?
|
||||
4. **Three-question test (ADR-0166).** Answer Q1/Q2/Q3 of ADR-0166
|
||||
for FrameClaim explicitly. If any of the three doesn't pass cleanly,
|
||||
say so — the ADR can defer rather than pretend.
|
||||
5. **Implementation outline.** A wave structure analogous to
|
||||
ADR-0167's: which W1/W2/W3 deliverables, what operator-to-brief
|
||||
matching, what's parallelisable.
|
||||
|
||||
### PR body must include
|
||||
|
||||
- Link to ADR-0167 and FOLLOWUPS §1
|
||||
- Quote the case 0050 hazard text from
|
||||
`feedback-wrong-zero-hazard-case-0050` memory (Shay can paste it)
|
||||
- Explicit "docs-only; no code change" callout
|
||||
- The recommendation: ship or defer? Whichever, defend it.
|
||||
|
||||
### Out of scope for this task
|
||||
|
||||
- Implementing FrameClaim. ADR is scoping only.
|
||||
- Touching `teaching/`, `language_packs/`, or any test file.
|
||||
- New eval lanes (ADR-0166 still gates).
|
||||
|
||||
---
|
||||
|
||||
## Task 2 — `discrete_count_statement` injector specification audit
|
||||
|
||||
**Branch:** `docs/gpt55-task-2-dcs-injector-spec`
|
||||
**Output:** `docs/handoff/discrete_count_statement-injector-spec.md`
|
||||
**Priority:** Highest-leverage (21/47 GSM8K refusals are this one category)
|
||||
|
||||
### Context to read first
|
||||
|
||||
- `evals/gsm8k_math/train_sample/v1/report.json` — the post-eval
|
||||
refusal records; filter for
|
||||
`"category=discrete_count_statement"` (21 cases)
|
||||
- `evals/gsm8k_math/train_sample/v1/cases.jsonl` — original problem
|
||||
text for each of those 21 cases
|
||||
- `generate/recognizer_match.py` — the `match` function that's
|
||||
over-matching
|
||||
- `generate/recognizer_anchor_inject.py` — the `inject_from_match`
|
||||
function; the empty-tuple return path is the bug surface
|
||||
- `engine_state/recognizers.jsonl` (read-only — **never commit this**)
|
||||
— the ratified recognizer specs including the
|
||||
`discrete_count_statement` canonical pattern
|
||||
- `docs/decisions/ADR-0163-gsm8k-path-to-mastery.md` — the roadmap
|
||||
that introduced this recognizer
|
||||
- `docs/decisions/ADR-0163.D.2-discrete-count-statement.md` (if it
|
||||
exists — locate and read it)
|
||||
|
||||
### Deliverable shape
|
||||
|
||||
A specification document, not an implementation. The document must:
|
||||
|
||||
1. **Categorise the 21 cases.** Read each problem text; group by
|
||||
sub-structure. Common shapes likely include:
|
||||
- "X has N <noun>" pure initial-state
|
||||
- "X has N <noun> and M <other-noun>" multi-quantity initial
|
||||
- "There are N <noun>" subject-anonymous initial
|
||||
- "N <noun> are <attribute>" attribute-on-count
|
||||
- Comparatives ("N more <noun> than M <noun>")
|
||||
The grouping is the load-bearing part — exact buckets aren't pre-
|
||||
determined; let the data dictate.
|
||||
2. **For each sub-shape**, propose:
|
||||
- What `parsed_anchors` shape an injector would have to produce
|
||||
- What `CandidateInitial` / `CandidateOperation` it maps to
|
||||
- What admissibility check would catch wrong>0 admissions
|
||||
- Which sub-shapes are LexicalClaim-resolvable (e.g. just a missing
|
||||
noun) and which need FrameClaim / CompositionClaim
|
||||
3. **Identify the over-matching root cause.** The recognizer's
|
||||
canonical pattern matches any number+noun. Propose specific
|
||||
tightening conditions (e.g. require a frame-opener verb, require
|
||||
the noun to be in a count-noun whitelist).
|
||||
4. **Quantify the lift potential.** Of the 21, how many would resolve
|
||||
under each sub-shape's hypothetical injector? Be honest about
|
||||
which ones still wouldn't resolve even with the injector (they
|
||||
have downstream barriers — pronoun, fraction, etc.).
|
||||
5. **Sequencing recommendation.** Which sub-shape's injector should
|
||||
ship first? Lift-per-risk, not raw count.
|
||||
|
||||
### PR body must include
|
||||
|
||||
- Per-sub-shape lift estimate (table)
|
||||
- A statement that NO injector implementation is being proposed —
|
||||
this PR is specification only
|
||||
- Cross-reference to ADR-0167-FOLLOWUPS §1 (FrameClaim) and
|
||||
§"discrete_count_statement over-matching"
|
||||
|
||||
### Out of scope for this task
|
||||
|
||||
- Implementing any injector
|
||||
- Modifying the recognizer canonical pattern
|
||||
- Touching `language_packs/` or `teaching/`
|
||||
- Running the eval (you can't anyway)
|
||||
|
||||
---
|
||||
|
||||
## Task 3 — Recognizer registry audit
|
||||
|
||||
**Branch:** `docs/gpt55-task-3-recognizer-audit`
|
||||
**Output:** `docs/handoff/ratified-recognizer-registry-audit.md`
|
||||
**Priority:** Medium (informs Task 2 and future injector work)
|
||||
|
||||
### Context to read first
|
||||
|
||||
- `engine_state/recognizers.jsonl` (read-only) — the 7 ratified
|
||||
recognizers from #315 onward
|
||||
- `generate/recognizer_match.py` — match logic
|
||||
- `generate/recognizer_anchor_inject.py` — injection logic, including
|
||||
which categories have injectors and which return `()`
|
||||
- The eval report from Task 2 — refusal-class counts per recognizer
|
||||
category
|
||||
|
||||
### Deliverable shape
|
||||
|
||||
A table-driven survey:
|
||||
|
||||
| Recognizer category | Match logic precision | Injector present? | GSM8K refusal count | Lift potential | Risk class |
|
||||
|---|---|---|---:|---|---|
|
||||
| `discrete_count_statement` | over-broad | no | 21 | high | high (case 0050 class) |
|
||||
| `currency_amount` | ? | ? | 4 | ? | ? |
|
||||
| `rate_with_currency` | ? | ? | 3 | ? | ? |
|
||||
| ... | ... | ... | ... | ... | ... |
|
||||
|
||||
For each row, write a one-paragraph commentary explaining:
|
||||
- What the recognizer is supposed to catch
|
||||
- What it actually catches (the over-broadness or precision)
|
||||
- Whether the injector is feasible (lexical-only? structural? multi-pack?)
|
||||
- The case 0050 hazard analogue for THIS category
|
||||
|
||||
### PR body must include
|
||||
|
||||
- A "promote injector / tighten match / retire recognizer" recommendation
|
||||
for each row
|
||||
- An "if you fix one, fix this one first" prioritisation
|
||||
|
||||
### Out of scope for this task
|
||||
|
||||
- Implementing any recognizer change
|
||||
- Retiring any recognizer (proposal-only)
|
||||
- Touching `engine_state/` directly — read-only
|
||||
|
||||
---
|
||||
|
||||
## Task 4 — FOLLOWUPS §6 ablation test specification
|
||||
|
||||
**Branch:** `docs/gpt55-task-4-holonomy-ablation-spec`
|
||||
**Output:** `docs/handoff/holonomy-ablation-test-spec.md`
|
||||
**Priority:** Low-urgency, high-information
|
||||
|
||||
### Context to read first
|
||||
|
||||
- `docs/handoff/ADR-0167-FOLLOWUPS.md` §6 (when merged from PR #360)
|
||||
- `language_packs/compiler.py:558` — `_apply_mounted_primary_domain_resonance`
|
||||
(the architectural-invariant comment names the gap)
|
||||
- `tests/test_alignment_graph.py:73` —
|
||||
`test_holonomy_alignment_case_positive_closer_than_negative` (the
|
||||
existing proof)
|
||||
- `language_packs/schema.py:181` — `HolonomyAlignmentCase` schema
|
||||
|
||||
### Deliverable shape
|
||||
|
||||
A specification (not an implementation) for an ablation test that
|
||||
isolates *structurally-derived* convergence from *blend-induced*
|
||||
convergence. The spec must:
|
||||
|
||||
1. **Name the ablation surface.** What part of
|
||||
`_apply_mounted_primary_domain_resonance` needs to be temporarily
|
||||
disabled or parameterised for the test?
|
||||
2. **Name the test contract.** With ablation active (blend factor = 0),
|
||||
does the positive-closer-than-negative assertion still hold? If
|
||||
yes, structural derivation is real; if no, the test is gated by the
|
||||
blend.
|
||||
3. **Name the predicted outcome.** Best guess: blend-gated. Document
|
||||
why (the 40% nudge is sizeable; without it, the morphology rotors
|
||||
alone may not produce enough convergence).
|
||||
4. **Name the honest reframing path.** If the ablation fails, the
|
||||
`HolonomyAlignmentCase` contract should be reframed from "proves
|
||||
structural divergence with coherent convergence" to "proves
|
||||
endpoint similarity under the mount-time blend." Suggest the exact
|
||||
docstring/schema text.
|
||||
|
||||
### PR body must include
|
||||
|
||||
- Cross-reference to FOLLOWUPS §6 and CLAUDE.md §"Schema-Defined Proof
|
||||
Obligations"
|
||||
- Explicit "spec only; no test implementation in this PR" callout
|
||||
|
||||
### Out of scope
|
||||
|
||||
- Implementing the ablation test
|
||||
- Modifying the holonomy test or schema
|
||||
- Modifying `_apply_mounted_primary_domain_resonance`
|
||||
|
||||
---
|
||||
|
||||
## Task 5 — Cognition contemplation partition fix specification
|
||||
|
||||
**Branch:** `docs/gpt55-task-5-contemplation-partition-spec`
|
||||
**Output:** `docs/handoff/contemplation-pack-indexing-partition-spec.md`
|
||||
**Priority:** Medium (this is FOLLOWUPS §5a)
|
||||
|
||||
### Context to read first
|
||||
|
||||
- `docs/handoff/ADR-0167-FOLLOWUPS.md` §5a
|
||||
- `docs/handoff/ADR-0167-W2C-cross-domain-audit.md` — Gemini's W2-C
|
||||
audit; the specific partition risks
|
||||
- `teaching/contemplation.py::contemplate()` — the function that uses
|
||||
hardcoded cognition pack/corpus indexes
|
||||
- `teaching/discovery.py` — `DiscoveryCandidate` with the `domain`
|
||||
field added by W2-C
|
||||
- `language_packs/data/en_core_math_v1/` — what a math pack looks like
|
||||
(for the alternate-domain branch)
|
||||
|
||||
### Deliverable shape
|
||||
|
||||
A surgical patch specification (not an implementation):
|
||||
|
||||
1. **Inventory.** Which exact lines in `teaching/contemplation.py`
|
||||
assume cognition?
|
||||
2. **Patch surface.** Minimum change to make those lines respect
|
||||
`candidate.domain`.
|
||||
3. **Test surface.** What test(s) would catch a regression where a
|
||||
math candidate silently fetches cognition pack data?
|
||||
4. **Backwards compatibility.** Confirm the default (`domain="cognition"`)
|
||||
preserves current behaviour byte-identically.
|
||||
|
||||
### PR body must include
|
||||
|
||||
- Cross-reference to W2-C audit and FOLLOWUPS §5a
|
||||
- "Spec only; implementation in a follow-up PR" callout
|
||||
|
||||
### Out of scope
|
||||
|
||||
- Implementing the patch
|
||||
- Touching `teaching/contemplation.py`
|
||||
- Running cognition regression tests (you can't anyway)
|
||||
|
||||
---
|
||||
|
||||
## Operational notes
|
||||
|
||||
- **Pace yourself.** Mobile + connector tooling has latency. One task per
|
||||
session is honourable; trying to finish all five in one go invites errors.
|
||||
- **Cite line numbers.** Every code reference must include `path:LINE` and
|
||||
be verified to resolve. If you can't verify via the connector, drop the
|
||||
specific line number and reference the function name instead.
|
||||
- **No code edits.** If a task starts feeling like it needs a code change,
|
||||
flag it in the PR body and stop. Do not attempt code edits via the
|
||||
connector — the test discipline can't be honoured from mobile.
|
||||
- **Honest progress reports.** Each PR's body should report what you
|
||||
actually concluded — including any sub-shapes you couldn't categorise,
|
||||
any line numbers you couldn't verify, any open questions that need
|
||||
Shay's input.
|
||||
- **If you finish all five.** Open a meta-PR adding a section to
|
||||
`docs/handoff/ADR-0167-FOLLOWUPS.md` linking to all five spec docs.
|
||||
|
||||
## What NOT to attempt from the connector
|
||||
|
||||
- Implementing FrameClaim, CompositionClaim, or any other handler
|
||||
- Implementing any injector for `discrete_count_statement` or any other
|
||||
recognizer category
|
||||
- Implementing the holonomy ablation test
|
||||
- Implementing the contemplation partition fix
|
||||
- Running `core test --suite *` or `core eval cognition` (mobile cannot)
|
||||
- Mutating any pack file under `language_packs/data/`
|
||||
- Committing anything under `engine_state/`
|
||||
- Force-pushing or rewriting history on any branch
|
||||
|
||||
If in doubt, the rule is: **specs and audits, not implementation.**
|
||||
|
||||
---
|
||||
|
||||
## Cross-references (for context)
|
||||
|
||||
- `CLAUDE.md` — project doctrine
|
||||
- `docs/decisions/ADR-0166-measurement-capability-sequencing.md` — the
|
||||
three-question test every spec must answer
|
||||
- `docs/decisions/ADR-0167-audit-as-teaching-evidence.md` — the parent
|
||||
wire all five sub-types extend
|
||||
- `docs/handoff/ADR-0167-FOLLOWUPS.md` — the canonical follow-up queue;
|
||||
Tasks 1, 2, 4, 5 all extend items already named there
|
||||
- `docs/decisions/SESSION-2026-05-27-adr-0167-parallel-dispatch.md` —
|
||||
the wave narrative; reading this gives the full context for why each
|
||||
task is shaped the way it is
|
||||
|
|
@ -1,408 +0,0 @@
|
|||
# Wave-Next — Recognizer Injectors + Lexical Closure + CompositionClaim Scoping
|
||||
|
||||
**Date:** 2026-05-27
|
||||
**Goal:** Lift GSM8K `correct` from 3 → 10+ (ADR-0163 Round-1 gate)
|
||||
via the recognizer-injector path identified in the post-eval analysis.
|
||||
**Risk profile:** Low. Each brief is a focused single-category injector
|
||||
with explicit `wrong=0` pinning. Composition / Frame work is deferred
|
||||
to subsequent waves with their own ADRs.
|
||||
|
||||
---
|
||||
|
||||
## Operator pool (as of 2026-05-27)
|
||||
|
||||
- **Sonnet 4.6** — workhorse for mechanical injector work; can run 3+ parallel agents
|
||||
- **Opus 4.6/4.7** — deepest reasoning; reserved for briefs with real design calls
|
||||
- **Gemini** — long-context surveys only (per `feedback-parallel-dispatch-pattern`)
|
||||
- **GitHub Copilot** — held in reserve; less proven for this workflow
|
||||
- **Codex** — OFFLINE (rate limits, several days)
|
||||
|
||||
---
|
||||
|
||||
## Dispatch timeline
|
||||
|
||||
### Gate 1 — cascade complete
|
||||
|
||||
**Wait for #362 → #363 → #364 → #365 → #366 to all merge.** That puts on `main`:
|
||||
- partition-test behavioral invariant (#362) — unblocks future ADR-0167 PRs
|
||||
- domain-aware contemplation routing (#363) — partitions cognition vs math
|
||||
- ADR-0168 FrameClaim scoping (#364) — names the next major sub-type ADR
|
||||
- ADR-0168.1 adapter bridge (#365) — resolves the ADR-0057 evidence floor tension
|
||||
- DCS injector spec (#366) — the methodology document A1–A4 reference
|
||||
|
||||
Verify the cascade with `git fetch origin main && git log origin/main --oneline -6`.
|
||||
|
||||
### Gate 2 — dispatch the parallel injector wave
|
||||
|
||||
**Single message → 4 parallel Agent calls:**
|
||||
|
||||
| Order | Brief | Operator | Branch |
|
||||
|---|---|---|---|
|
||||
| 1 | A1 currency_amount injector | Sonnet | `feat/injector-currency-amount` |
|
||||
| 2 | A3 multiplicative_aggregation injector | Sonnet | `feat/injector-multiplicative-aggregation` |
|
||||
| 3 | A4 temporal_aggregation injector | Sonnet | `feat/injector-temporal-aggregation` |
|
||||
| 4 | A2 rate_with_currency injector | Opus | `feat/injector-rate-with-currency` |
|
||||
|
||||
All 4 touch `generate/recognizer_anchor_inject.py`. First to push opens
|
||||
clean; the other 3 need a union-merge rebase. Each rebase is trivial
|
||||
(adding a function + a dispatch-table line).
|
||||
|
||||
**In parallel with that wave**, the orchestrator (me) handles **B1**
|
||||
inline — too small to dispatch.
|
||||
|
||||
### Gate 3 — sequential after the injector wave settles
|
||||
|
||||
| Brief | Operator | Branch |
|
||||
|---|---|---|
|
||||
| D1 ADR-0169 CompositionClaim scoping | Opus | `docs/adr-0169-compositionclaim-scoping` |
|
||||
|
||||
D1 is docs-only, no code conflicts. Can technically run in parallel
|
||||
with the injector wave; sequencing it after lets Opus give A2 full
|
||||
attention first.
|
||||
|
||||
### Gate 4 — optional background research
|
||||
|
||||
| Brief | Operator | Branch |
|
||||
|---|---|---|
|
||||
| GPT-5.5 dispatch Task 3 (recognizer registry audit) | Gemini | `docs/gemini-recognizer-registry-audit` |
|
||||
|
||||
Pure long-context survey of all 7 ratified recognizers. No code, no
|
||||
risk. Informs future injector PRs. Run if you want background research
|
||||
while the injector wave executes; skip if you don't want the noise.
|
||||
|
||||
---
|
||||
|
||||
## Shared constraints (every brief inherits these)
|
||||
|
||||
- Open a dedicated `git worktree add` (parallel-agent worktree rule)
|
||||
- Branch off **current `main`** after Gate 1 confirms the cascade is in
|
||||
- `wrong == 0` non-negotiable — verify against case `gsm8k-train-sample-v1-0050`
|
||||
in every test suite
|
||||
- ADR-0166 — no new canonical eval lanes; reuse `gsm8k_math/train_sample/v1`
|
||||
- No teaching-store / pack mutation as a side effect of injector work
|
||||
- `uv venv` / `uv pip install` / `uv run` — never `--break-system-packages`
|
||||
- Stage explicit files; never `git add -A`; NEVER commit `engine_state/`
|
||||
- Each PR runs the full regression suite (see Validation block per brief)
|
||||
- CLAUDE.md §"Documentation Discipline" — pure markdown, no standalone HTML
|
||||
- CLAUDE.md §"Schema-Defined Proof Obligations" — every new injector
|
||||
must come with a test that can meaningfully fail under the wrong=0
|
||||
violations the injector is written to catch
|
||||
|
||||
---
|
||||
|
||||
## A1 — `currency_amount` injector
|
||||
|
||||
**Recommended operator:** Sonnet 4.6
|
||||
**Branch:** `feat/injector-currency-amount`
|
||||
**Expected lift:** 2–4 cases (4 currently refused as `currency_amount`)
|
||||
**Blocked by:** Gate 1 (cascade complete)
|
||||
|
||||
### Context to read first
|
||||
|
||||
- `generate/recognizer_anchor_inject.py:79` — `inject_discrete_count_statement`
|
||||
(the existing template; do not reuse logic, just shape)
|
||||
- `generate/recognizer_match.py` — the `currency_amount` match logic
|
||||
- `engine_state/recognizers.jsonl` (read-only) — the ratified
|
||||
`currency_amount` canonical pattern
|
||||
- `docs/handoff/discrete_count_statement-injector-spec.md` (post-#366)
|
||||
— the methodology for "narrow first, broaden later"
|
||||
- `evals/gsm8k_math/train_sample/v1/report.json` — filter for
|
||||
`category=currency_amount`; these are the 4 cases you target
|
||||
- `evals/gsm8k_math/train_sample/v1/cases.jsonl` — the original problem
|
||||
text for each
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
git worktree add /tmp/wt-a1 -b feat/injector-currency-amount origin/main
|
||||
cd /tmp/wt-a1
|
||||
uv venv && source .venv/bin/activate
|
||||
uv pip install -e .
|
||||
```
|
||||
|
||||
### Deliverables
|
||||
|
||||
1. **`generate/recognizer_anchor_inject.py`** — new
|
||||
`inject_currency_amount(match) -> tuple[CandidateInitial | CandidateOperation, ...]`
|
||||
function. Add entry to the dispatch table at the bottom of the file.
|
||||
Must:
|
||||
- extract `currency` + `amount` + `entity` from `match.parsed_anchors`
|
||||
- emit ONE `CandidateInitial` per match in the narrow canonical form
|
||||
`<ProperNoun> has|earns|charges $<amount>`
|
||||
- return `()` (preserve refusal) for any shape outside that narrow
|
||||
form — broadening is a follow-up PR
|
||||
- never emit a `CandidateOperation` (those are FrameClaim territory)
|
||||
2. **`tests/test_injector_currency_amount.py`** (new) — 8+ tests:
|
||||
- happy path: narrow canonical form admits a complete graph
|
||||
- sub-shape rejection: 2+ variant shapes the injector deliberately
|
||||
skips (must return `()`, not raise)
|
||||
- hazard pin: case `gsm8k-train-sample-v1-0050` remains refused at
|
||||
`sentence_index=0`
|
||||
- determinism: same `RecognizerMatch` → byte-identical injector output
|
||||
- wrong=0 invariant: any admitted graph passes
|
||||
`assert_graph_complete` and the existing solver's verifier
|
||||
3. **Eval delta artifact** — append a new section to
|
||||
`evals/gsm8k_math/train_sample/v1/audit_brief_11.md` documenting:
|
||||
- which N cases moved from `currency_amount` refusal to admission
|
||||
- which cases remained refused on a different bottleneck class
|
||||
- confirmation that `wrong` count remains 0
|
||||
|
||||
### Hard constraints
|
||||
|
||||
- The narrow form is non-negotiable. **Do not** match comparatives,
|
||||
rate compositions, or multi-currency arithmetic in this PR
|
||||
- Reject any shape where the entity is anonymous (`The store earns ...`
|
||||
vs `Sam earns ...`)
|
||||
- Manifest checksums unchanged (no pack file edits)
|
||||
- Reader path remains the priority — flag-on reader still runs before
|
||||
recognizer; this injector only fires on reader refusal
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
uv run pytest tests/test_injector_currency_amount.py -q
|
||||
uv run pytest tests/test_brief_11b_audit_artifact.py tests/test_brief_11b_step2_lexicon.py tests/test_recognizer_skip_wrong_zero.py -q
|
||||
uv run pytest tests/ -k "teaching or contemplation or candidate or correction or store or review" -q
|
||||
PYTHONPATH=. uv run python evals/gsm8k_math/train_sample/v1/runner.py
|
||||
```
|
||||
|
||||
Capture the before/after `report.json` counts in the PR body.
|
||||
|
||||
### PR body must include
|
||||
|
||||
- Before/after refusal taxonomy for the `currency_amount` row
|
||||
- Case-by-case verdict for the 4 currently-refused cases (admitted /
|
||||
refused-on-different-class)
|
||||
- Explicit case 0050 hazard verification line
|
||||
- `wrong=0` invariant statement
|
||||
|
||||
### Report back
|
||||
|
||||
- PR URL
|
||||
- Lift count (cases moved from refused → admitted)
|
||||
- Hazard pin evidence
|
||||
- Any sub-shapes you noticed that need follow-up injector PRs
|
||||
|
||||
---
|
||||
|
||||
## A2 — `rate_with_currency` injector
|
||||
|
||||
**Recommended operator:** Opus 4.6/4.7
|
||||
**Branch:** `feat/injector-rate-with-currency`
|
||||
**Expected lift:** 1–3 cases (3 currently refused)
|
||||
**Blocked by:** Gate 1
|
||||
|
||||
### Why Opus instead of Sonnet
|
||||
|
||||
This brief has a real schema decision: does the existing
|
||||
`Quantity` type in `generate/math_problem_graph.py` structurally model
|
||||
a per-unit rate? If yes, the injector emits a `Rate`-shaped
|
||||
`CandidateInitial` analogous to A1. If no, the injector must
|
||||
**explicitly refuse** rather than invent a new type — flag for
|
||||
follow-up. That decision needs judgment, not pattern-matching.
|
||||
|
||||
### Setup, context, deliverables, hard constraints
|
||||
|
||||
Identical structure to A1, but for `rate_with_currency`. Canonical
|
||||
narrow form: `<ProperNoun> earns|charges|pays $<amount> per <unit>` or
|
||||
`<ProperNoun> earns|charges|pays $<amount> for <unit>`.
|
||||
|
||||
Specific differences from A1:
|
||||
|
||||
- Check `generate/math_problem_graph.py` for the `Quantity` type
|
||||
structure; if it doesn't model rates, the injector returns `()`
|
||||
and the PR body writes an explicit follow-up note
|
||||
- If `Quantity` does model rates (e.g. via a composite unit or a
|
||||
separate `Rate` type), use that — DO NOT invent a new type
|
||||
- Hazard pin: case 0050 still refused
|
||||
|
||||
### Report back must include
|
||||
|
||||
- The schema decision (does `Quantity` model rates?) and your evidence
|
||||
- If "no," the follow-up note for whoever ships the `Rate` schema
|
||||
extension
|
||||
- Lift count (will be 0 if schema decision is "no" — that's still a
|
||||
successful PR; documenting the gap is the deliverable)
|
||||
|
||||
---
|
||||
|
||||
## A3 — `multiplicative_aggregation` injector
|
||||
|
||||
**Recommended operator:** Sonnet 4.6
|
||||
**Branch:** `feat/injector-multiplicative-aggregation`
|
||||
**Expected lift:** 2–4 cases (5 currently refused)
|
||||
**Blocked by:** Gate 1
|
||||
|
||||
### Why this needs care
|
||||
|
||||
This is the **first injector that emits `CandidateOperation`** (not
|
||||
just `CandidateInitial`). Multiplicative operations widen the case
|
||||
0050 hazard surface — if the operand isn't the right unit, the
|
||||
solver computes a wrong product.
|
||||
|
||||
### Canonical narrow form
|
||||
|
||||
`<ProperNoun> has <count> <noun> in each <container>` or
|
||||
`<count> <noun> per <container>`. The injector emits a
|
||||
`CandidateOperation` of kind `multiply` when the count, noun, and
|
||||
container all extract cleanly from `parsed_anchors`.
|
||||
|
||||
### Extra hazard pinning (beyond A1's spec)
|
||||
|
||||
Reject any shape where:
|
||||
- the container isn't a `count_unit_noun`
|
||||
- the multiplier isn't a determinate integer or word-form integer
|
||||
- the result unit doesn't match the original count unit
|
||||
|
||||
The `tests/test_injector_multiplicative_aggregation.py` must include
|
||||
a parameterized test confirming each of those rejection paths
|
||||
returns `()` rather than admitting a wrong-product graph.
|
||||
|
||||
### Otherwise identical to A1's structure
|
||||
|
||||
Same deliverables, hard constraints, verification, PR body, report-back.
|
||||
|
||||
---
|
||||
|
||||
## A4 — `temporal_aggregation` injector
|
||||
|
||||
**Recommended operator:** Sonnet 4.6
|
||||
**Branch:** `feat/injector-temporal-aggregation`
|
||||
**Expected lift:** 1–2 cases (2 currently refused)
|
||||
**Blocked by:** Gate 1
|
||||
|
||||
### Why this is the structural sanity check
|
||||
|
||||
Smallest injector in the wave. If a focused PR can lift the 2 cases,
|
||||
the recognizer-injector pattern is operational and the larger sub-shape
|
||||
work (especially DCS sub-shapes) can follow with confidence.
|
||||
|
||||
### Canonical narrow form
|
||||
|
||||
`<count> <time_unit> per <time_unit>` (e.g. `5 hours per day`,
|
||||
`3 days per week`). Emits a `Rate`-shaped or multiplicative-shaped
|
||||
candidate depending on context.
|
||||
|
||||
### Coordinate with A3
|
||||
|
||||
Both A3 and A4 may produce multiplicative-kind operations. If the
|
||||
`Quantity`/`Operation` schema doesn't distinguish them cleanly,
|
||||
flag in the PR body for shared follow-up.
|
||||
|
||||
### Otherwise identical to A1's structure
|
||||
|
||||
---
|
||||
|
||||
## B1 — Lexical-entry closure: remaining 3 cases
|
||||
|
||||
**Recommended operator:** Orchestrator (me) — too small to dispatch
|
||||
**Branch:** `feat/lexicon-closure-wave-3`
|
||||
**Expected lift:** 1–3 cases
|
||||
**Blocked by:** Gate 1 (cascade complete)
|
||||
|
||||
Three `lexicon_entry` refusals remain after #348:
|
||||
- case 0001: `+` (arithmetic literal — DO NOT add as drain_token)
|
||||
- case 0040: `sees` (perception verb — drain_token candidate)
|
||||
- case 0049: `path` (noun — drain_token candidate)
|
||||
|
||||
This is small (12 lines of edits, 3 test additions) and I'll handle
|
||||
it in-line while the injector wave runs. Decision-making for `+`
|
||||
documented in PR body (it's a structural issue, not a lexical gap).
|
||||
|
||||
---
|
||||
|
||||
## D1 — ADR-0169 CompositionClaim scoping
|
||||
|
||||
**Recommended operator:** Opus 4.6/4.7
|
||||
**Branch:** `docs/adr-0169-compositionclaim-scoping`
|
||||
**Output:** `docs/decisions/ADR-0169-compositionclaim-ratification.md`
|
||||
**Blocked by:** ADR-0168 (#364) merged — Gate 1
|
||||
**Sequencing:** Run after A2 lands (Opus needs full attention on A2 first)
|
||||
|
||||
### Deliverable shape
|
||||
|
||||
A scoping ADR analogous to ADR-0168 (#364), answering the same six
|
||||
open questions for `CompositionClaim`:
|
||||
|
||||
1. Sub-types of CompositionClaim needed?
|
||||
2. SAFE_CATEGORIES allowlist applicable?
|
||||
3. Concrete answer to how the ratification prevents the case 0050
|
||||
hazard (multi-quantity is exactly the hazard surface — a wrong
|
||||
composition rule could admit `5 apples + 3 oranges = 8 things`)
|
||||
4. Evidence signature normalisation needed
|
||||
5. Graph completeness gating
|
||||
6. Ablation test that proves the handler doesn't admit a partial
|
||||
composition
|
||||
|
||||
Plus ADR-0166 three-question test, plus compatibility audit against
|
||||
ADR-0056/0057/0114a/0164/0165/0166/0167/0168, plus implementation
|
||||
wave outline.
|
||||
|
||||
### Hard constraints
|
||||
|
||||
- Docs-only; no code, no test, no eval, no pack change
|
||||
- Must explicitly address: "is CompositionClaim safer or riskier than
|
||||
FrameClaim?" — argue from data, not intuition
|
||||
- If "riskier," propose deferring CompositionClaim until FrameClaim
|
||||
ships a clean second-sub-type precedent
|
||||
|
||||
---
|
||||
|
||||
## (Optional) Background research — Gemini recognizer registry audit
|
||||
|
||||
**Recommended operator:** Gemini
|
||||
**Branch:** `docs/gemini-recognizer-registry-audit`
|
||||
**Output:** `docs/handoff/ratified-recognizer-registry-audit.md`
|
||||
**Blocked by:** Nothing — pure read-only survey
|
||||
|
||||
This is GPT-5.5 dispatch Task 3 from the prior session that wasn't
|
||||
picked up. Pure long-context audit of all 7 ratified recognizers in
|
||||
`engine_state/recognizers.jsonl`. Output is a table-driven survey
|
||||
naming: match-logic precision, injector presence/absence, GSM8K
|
||||
refusal count, lift potential, hazard class.
|
||||
|
||||
Informs future injector PRs. Independent of A1–A4. Skip if you don't
|
||||
want background research running in parallel.
|
||||
|
||||
---
|
||||
|
||||
## What this wave does NOT do
|
||||
|
||||
- It does not implement `discrete_count_statement` sub-shapes (21
|
||||
largest bucket). That's Wave C, informed by #366's spec post-merge.
|
||||
- It does not implement FrameClaim (Wave E, requires ADR-0168 merged
|
||||
AND its own W1-W3 sub-wave).
|
||||
- It does not add new eval lanes (ADR-0166 still gates).
|
||||
- It does not touch workbench wiring (ADR-0167 §Q4, deferred).
|
||||
- It does not propose any non-deterministic / non-decoding mechanism.
|
||||
|
||||
## Expected aggregate lift
|
||||
|
||||
If A1–A4 all ship cleanly: **6–13 cases lifted** out of the 14
|
||||
across those four categories. Plus B1: **1–3 cases**.
|
||||
|
||||
That puts `correct` at **10–19**, clearing ADR-0163 Round-1
|
||||
(`correct ≥ 10`) and potentially nudging Round-2 (`correct ≥ 25`).
|
||||
|
||||
A2's lift may be 0 if the `Quantity` schema doesn't model rates —
|
||||
in that case the PR's value is documenting the gap, and the lift
|
||||
shifts to A3 + A4.
|
||||
|
||||
---
|
||||
|
||||
## Dispatch protocol summary
|
||||
|
||||
```text
|
||||
1. Wait for cascade #362→#366 (Gate 1)
|
||||
2. Single message with 4 Agent calls:
|
||||
- subagent_type=general-purpose, model=sonnet → A1
|
||||
- subagent_type=general-purpose, model=sonnet → A3
|
||||
- subagent_type=general-purpose, model=sonnet → A4
|
||||
- subagent_type=general-purpose, model=opus → A2
|
||||
3. Orchestrator handles B1 inline
|
||||
4. After A2 lands: subagent_type=planner, model=opus → D1
|
||||
5. (Optional) subagent_type=general-purpose, model=sonnet (or Gemini) → Task 3 audit
|
||||
```
|
||||
|
||||
Each operator gets pointed at this file's section header for their
|
||||
brief. Shared constraints at the top apply to everyone.
|
||||
|
|
@ -1,7 +1,8 @@
|
|||
# Wave-Next Revised — DCS Sub-Shapes + Schema-Gap Backlog
|
||||
|
||||
**Date:** 2026-05-27
|
||||
**Supersedes:** `docs/handoff/WAVE-NEXT-INJECTORS.md` (kept for history)
|
||||
**Supersedes:** the original Wave-Next injector briefs (removed in
|
||||
cleanup PR; the four A-findings and pivot rationale are captured here)
|
||||
**Why revised:** The A1–A4 dispatch surfaced findings that invalidate three
|
||||
of the four briefs' lift assumptions. This doc replaces them with the
|
||||
actually-tractable next steps.
|
||||
|
|
@ -173,15 +174,12 @@ No timelines. Order is by leverage, not calendar.
|
|||
## What this document does NOT do
|
||||
|
||||
- It does not dispatch any agents (per `feedback-no-self-dispatch-of-subagents`)
|
||||
- It does not retire `WAVE-NEXT-INJECTORS.md` (kept for history; future
|
||||
readers should consult this document for current state)
|
||||
- It does not modify any runtime code
|
||||
- It does not add new eval lanes (ADR-0166)
|
||||
- It does not propose any non-deterministic mechanism
|
||||
|
||||
## Cross-references
|
||||
|
||||
- `docs/handoff/WAVE-NEXT-INJECTORS.md` — original (now-superseded) brief
|
||||
- `docs/handoff/discrete_count_statement-injector-spec.md` — the DCS sub-shape spec
|
||||
- `docs/handoff/ADR-0167-FOLLOWUPS.md` — parent follow-up queue
|
||||
- `docs/decisions/ADR-0168-frameclaim-ratification.md` — FrameClaim scoping
|
||||
|
|
|
|||
|
|
@ -230,75 +230,36 @@ def _locate_possession_verb(sentence: str) -> str | None:
|
|||
# registers its injector. No global state, no side effects.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def inject_rate_with_currency(
|
||||
match: RecognizerMatch,
|
||||
sentence: str,
|
||||
) -> tuple[CandidateInitial, ...]:
|
||||
"""Inject a ``rate_with_currency`` recognizer match.
|
||||
|
||||
Schema decision (Wave-Next A2): the :class:`Rate` type in
|
||||
:mod:`generate.math_problem_graph` (ADR-0122) DOES structurally
|
||||
model a per-unit rate via ``(value, numerator_unit,
|
||||
denominator_unit)``. However, ``Rate`` is the operand of an
|
||||
``Operation(kind='apply_rate')`` — it is not, and cannot be
|
||||
coerced into, a :class:`CandidateInitial` or a standalone
|
||||
:class:`CandidateOperation` that the per-sentence choice
|
||||
dispatcher consumes:
|
||||
|
||||
- :class:`InitialPossession` requires a :class:`Quantity` (scalar
|
||||
+ unit) and an entity. A rate carries two units; the
|
||||
rate-declaration sentence alone ("Tina makes $18.00 an hour.")
|
||||
does not establish how many hours Tina worked, so the
|
||||
denominator quantity is unknown.
|
||||
- ``Operation(kind='apply_rate')`` requires the denominator
|
||||
quantity (the "how many X" the rate multiplies) to be present
|
||||
in the same sentence; an isolated rate-declaration sentence
|
||||
does not carry it.
|
||||
- ``SentenceChoice = Union[CandidateInitial, CandidateOperation]``
|
||||
(see :mod:`generate.math_candidate_graph`) — there is no
|
||||
``CandidateRate`` variant for the injector to deposit.
|
||||
|
||||
The existing
|
||||
:func:`generate.math_candidate_parser.extract_earnings_candidates`
|
||||
handles rate-declaration sentences via a separate short-circuit
|
||||
path keyed on a sibling :class:`CandidateEarningsRate` type that
|
||||
is NOT part of ``SentenceChoice``. It is consumed by a special
|
||||
earnings short-circuit in
|
||||
:func:`generate.math_candidate_graph.parse_and_solve` BEFORE the
|
||||
per-sentence-choices Cartesian product runs.
|
||||
|
||||
Conclusion: the recognizer-injector contract (return a tuple of
|
||||
:class:`CandidateInitial`) cannot meaningfully express a per-unit
|
||||
rate without a wider ``SentenceChoice`` union. v1 therefore
|
||||
RETURNS ``()`` (refusal-preferring, wrong=0 doctrine) and
|
||||
documents the gap. This is the explicit-refusal A2 outcome.
|
||||
|
||||
Follow-up (separate PR): extend ``SentenceChoice`` with a
|
||||
``CandidateRate`` variant carrying a :class:`Rate` operand keyed
|
||||
by actor, and teach
|
||||
:func:`generate.math_candidate_graph.parse_and_solve` to compose
|
||||
a ``CandidateRate`` with a downstream apply_rate/multiply-shaped
|
||||
question. Only at that point can a recognizer-injector emit
|
||||
useful state for ``rate_with_currency``.
|
||||
"""
|
||||
return ()
|
||||
|
||||
|
||||
_INJECTORS: Mapping[ShapeCategory, "type"] = {
|
||||
ShapeCategory.DISCRETE_COUNT_STATEMENT: inject_discrete_count_statement, # type: ignore[dict-item]
|
||||
ShapeCategory.RATE_WITH_CURRENCY: inject_rate_with_currency, # type: ignore[dict-item]
|
||||
# The four other recognizer categories route to the empty-tuple
|
||||
# fallback (skip-only) until their D.2.x injector lands:
|
||||
# All other recognizer categories route to the empty-tuple fallback
|
||||
# in ``inject_from_match`` — `_INJECTORS.get(category)` returns
|
||||
# ``None`` and the dispatcher returns ``()``, which the
|
||||
# candidate-graph then treats as "recognizer matched but produced
|
||||
# no injection" → explicit refusal (the wrong=0 fix from #359).
|
||||
#
|
||||
# ShapeCategory.DESCRIPTIVE_SETUP_NO_QUANTITY — by design (no quantity)
|
||||
# ShapeCategory.TEMPORAL_AGGREGATION — D.2.3 follow-up
|
||||
# ShapeCategory.MULTIPLICATIVE_AGGREGATION — D.2.4 follow-up
|
||||
# ShapeCategory.CURRENCY_AMOUNT — D.2.5 follow-up
|
||||
# Categories deferred to follow-up PRs:
|
||||
#
|
||||
# ShapeCategory.DESCRIPTIVE_SETUP_NO_QUANTITY — by design (no quantity)
|
||||
# ShapeCategory.RATE_WITH_CURRENCY — needs CandidateRate
|
||||
# (SentenceChoice union
|
||||
# extension; ADR-0171)
|
||||
# ShapeCategory.TEMPORAL_AGGREGATION — needs apply_rate primitive
|
||||
# in the algebra
|
||||
# ShapeCategory.MULTIPLICATIVE_AGGREGATION — emits
|
||||
# CandidateInitial(product)
|
||||
# after ADR-0170 widens
|
||||
# return type
|
||||
# ShapeCategory.CURRENCY_AMOUNT — A1 currency_amount;
|
||||
# CandidateInitial-shaped,
|
||||
# ships after ADR-0170
|
||||
#
|
||||
# See docs/decisions/ADR-0170-injector-contract-widening.md for the
|
||||
# contract widening that unblocks DCS-S1 / A1 / A3.
|
||||
}
|
||||
|
||||
|
||||
__all__ = [
|
||||
"inject_from_match",
|
||||
"inject_discrete_count_statement",
|
||||
"inject_rate_with_currency",
|
||||
]
|
||||
|
|
|
|||
|
|
@ -1,371 +0,0 @@
|
|||
"""Wave-Next A2 — ``rate_with_currency`` injector.
|
||||
|
||||
This test file is the load-bearing artifact for the A2 injector. The
|
||||
A2 outcome is an explicit, documented schema-refusal: ``Rate`` (ADR-0122)
|
||||
DOES structurally model a per-unit rate, but it is not a member of the
|
||||
``SentenceChoice = Union[CandidateInitial, CandidateOperation]`` union
|
||||
the per-sentence injector contract requires. The injector therefore
|
||||
returns ``()`` and the load-bearing assertions in this file pin:
|
||||
|
||||
a. SCHEMA EVIDENCE — ``Rate`` exists and structurally models
|
||||
a (value, numerator_unit, denominator_unit)
|
||||
per-unit rate, distinct from ``Quantity``.
|
||||
b. SCHEMA REFUSAL — the injector returns ``()`` for every
|
||||
shape (broad and narrow canonical forms).
|
||||
c. DISPATCH WIRED — dispatch table routes
|
||||
``RATE_WITH_CURRENCY`` to the injector
|
||||
(no longer the empty-tuple default).
|
||||
d. CASE 0050 HAZARD PIN — case
|
||||
``gsm8k-train-sample-v1-0050`` remains
|
||||
refused at sentence_index=0 (sentence
|
||||
carries no currency, so it neither
|
||||
matches nor would be lifted by A2).
|
||||
e. DETERMINISM — identical ``(match, sentence)`` →
|
||||
byte-identical injector output.
|
||||
f. NO STATE INJECTED — the injector never produces a
|
||||
``CandidateInitial`` (would be a wrong=0
|
||||
hazard, since Rate ≠ Quantity).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from evals.refusal_taxonomy.shape_categories import ShapeCategory
|
||||
from generate.math_candidate_graph import SentenceChoice, parse_and_solve
|
||||
from generate.math_problem_graph import Quantity, Rate
|
||||
from generate.recognizer_anchor_inject import (
|
||||
_INJECTORS,
|
||||
inject_from_match,
|
||||
inject_rate_with_currency,
|
||||
)
|
||||
from generate.recognizer_match import RecognizerMatch
|
||||
from generate.recognizer_registry import RatifiedRecognizer
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Synthetic match builder — mirrors the A1/D.2 test pattern.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_match(parsed_anchors: tuple[dict, ...]) -> RecognizerMatch:
|
||||
rec = RatifiedRecognizer(
|
||||
proposal_id="test-rate-with-currency",
|
||||
shape_category=ShapeCategory.RATE_WITH_CURRENCY,
|
||||
canonical_pattern={
|
||||
"anchor_kind": "currency_per_unit_rate",
|
||||
"shape_category": "rate_with_currency",
|
||||
"graph_intent": "rate",
|
||||
"anchor_count_min": 1,
|
||||
"anchor_count_max": 1,
|
||||
"outcome": "admissible",
|
||||
"observed_currency_symbols": ["$"],
|
||||
"observed_per_units": ["hour", "day", "week"],
|
||||
},
|
||||
spec_digest="test-digest",
|
||||
review_date="2026-05-27",
|
||||
ratified_at_revision="test",
|
||||
)
|
||||
return RecognizerMatch(
|
||||
recognizer=rec,
|
||||
category=ShapeCategory.RATE_WITH_CURRENCY,
|
||||
outcome="admissible",
|
||||
graph_intent="rate",
|
||||
parsed_anchors=parsed_anchors,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# (a) Schema evidence — Rate models a per-unit rate.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSchemaEvidence:
|
||||
"""The schema decision: does ``Quantity`` model a per-unit rate?
|
||||
|
||||
Answer: NO — ``Quantity`` is a scalar+unit pair, not a rate. BUT
|
||||
a separate ``Rate`` type (ADR-0122) DOES structurally model the
|
||||
per-unit rate via ``numerator_unit`` / ``denominator_unit``. The
|
||||
A2 schema-refusal hinges not on the absence of ``Rate`` but on
|
||||
its absence from the ``SentenceChoice`` union.
|
||||
"""
|
||||
|
||||
def test_quantity_does_not_model_a_rate(self) -> None:
|
||||
# Quantity is value + unit; no numerator/denominator distinction.
|
||||
q = Quantity(value=18.0, unit="dollars")
|
||||
assert q.value == 18.0
|
||||
assert q.unit == "dollars"
|
||||
# No rate-shaped attributes: this is exactly the gap A2 documents.
|
||||
assert not hasattr(q, "numerator_unit")
|
||||
assert not hasattr(q, "denominator_unit")
|
||||
|
||||
def test_rate_type_exists_and_models_per_unit_rate(self) -> None:
|
||||
# Rate(18, "dollars", "hour") means "18 dollars per hour".
|
||||
r = Rate(value=18.0, numerator_unit="dollars", denominator_unit="hour")
|
||||
assert r.value == 18.0
|
||||
assert r.numerator_unit == "dollars"
|
||||
assert r.denominator_unit == "hour"
|
||||
|
||||
def test_sentence_choice_union_excludes_rate(self) -> None:
|
||||
# The per-sentence injector contract is
|
||||
# ``SentenceChoice = Union[CandidateInitial, CandidateOperation]``.
|
||||
# No CandidateRate exists. This is the load-bearing reason the
|
||||
# A2 injector cannot meaningfully emit a rate primitive.
|
||||
from generate.math_candidate_parser import CandidateInitial
|
||||
from generate.math_roundtrip import CandidateOperation
|
||||
|
||||
# The Union is realised structurally — every SentenceChoice
|
||||
# must be one of these two types. The test pins the closed
|
||||
# set; expanding it is the explicit follow-up.
|
||||
allowed = {CandidateInitial, CandidateOperation}
|
||||
# Best-effort introspection of the Union type. Python's
|
||||
# ``Union`` exposes its members via ``__args__`` on the alias.
|
||||
import typing
|
||||
|
||||
args = set(typing.get_args(SentenceChoice))
|
||||
assert args == allowed
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# (b) Schema refusal — every shape returns ().
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSchemaRefusal:
|
||||
"""A2 v1 refuses every input shape, by design."""
|
||||
|
||||
def test_canonical_per_form_refuses(self) -> None:
|
||||
m = _make_match((
|
||||
{
|
||||
"kind": "currency_per_unit_rate",
|
||||
"currency_symbol": "$",
|
||||
"amount": "18.00",
|
||||
"amount_kind": "decimal",
|
||||
"per_unit": "hour",
|
||||
},
|
||||
))
|
||||
out = inject_rate_with_currency(m, "Tina makes $18.00 an hour.")
|
||||
assert out == ()
|
||||
|
||||
def test_canonical_for_form_refuses(self) -> None:
|
||||
m = _make_match((
|
||||
{
|
||||
"kind": "currency_per_unit_rate",
|
||||
"currency_symbol": "$",
|
||||
"amount": "30",
|
||||
"amount_kind": "integer",
|
||||
"per_unit": "hour",
|
||||
},
|
||||
))
|
||||
out = inject_rate_with_currency(m, "Sam charges $30 for each hour.")
|
||||
assert out == ()
|
||||
|
||||
def test_empty_parsed_anchors_refuses(self) -> None:
|
||||
# No anchors → no possible state regardless of schema.
|
||||
m = _make_match(())
|
||||
out = inject_rate_with_currency(m, "Tina makes $18.00 an hour.")
|
||||
assert out == ()
|
||||
|
||||
def test_returns_empty_tuple_never_raises(self) -> None:
|
||||
# Adversarial input: malformed anchor payload. The injector
|
||||
# MUST NOT raise; it MUST return ``()``.
|
||||
m = _make_match(({"kind": "currency_per_unit_rate", "junk": True},))
|
||||
out = inject_rate_with_currency(m, "")
|
||||
assert out == ()
|
||||
|
||||
def test_injector_never_emits_candidate_initial(self) -> None:
|
||||
# Iterate over a spread of shapes; none may admit any candidate.
|
||||
sentences = (
|
||||
"Tina makes $18.00 an hour.",
|
||||
"Sam charges $30 for each hour.",
|
||||
"Bob pays $5 per cup.",
|
||||
"Alice earns $100 a day.",
|
||||
)
|
||||
for s in sentences:
|
||||
m = _make_match((
|
||||
{
|
||||
"kind": "currency_per_unit_rate",
|
||||
"currency_symbol": "$",
|
||||
"amount": "1",
|
||||
"amount_kind": "integer",
|
||||
"per_unit": "hour",
|
||||
},
|
||||
))
|
||||
assert inject_rate_with_currency(m, s) == ()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# (c) Dispatch wired — registry routes RATE_WITH_CURRENCY to injector.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestDispatchWired:
|
||||
def test_injector_table_registers_rate_with_currency(self) -> None:
|
||||
# Before A2: RATE_WITH_CURRENCY was absent from _INJECTORS
|
||||
# (default empty-tuple skip). After A2: present and routed
|
||||
# to inject_rate_with_currency.
|
||||
assert ShapeCategory.RATE_WITH_CURRENCY in _INJECTORS
|
||||
assert _INJECTORS[ShapeCategory.RATE_WITH_CURRENCY] is inject_rate_with_currency
|
||||
|
||||
def test_dispatch_returns_empty_tuple_via_registry(self) -> None:
|
||||
m = _make_match((
|
||||
{
|
||||
"kind": "currency_per_unit_rate",
|
||||
"currency_symbol": "$",
|
||||
"amount": "18.00",
|
||||
"amount_kind": "decimal",
|
||||
"per_unit": "hour",
|
||||
},
|
||||
))
|
||||
out = inject_from_match(m, "Tina makes $18.00 an hour.")
|
||||
assert out == ()
|
||||
|
||||
def test_dispatch_equals_direct_call(self) -> None:
|
||||
m = _make_match((
|
||||
{
|
||||
"kind": "currency_per_unit_rate",
|
||||
"currency_symbol": "$",
|
||||
"amount": "18.00",
|
||||
"amount_kind": "decimal",
|
||||
"per_unit": "hour",
|
||||
},
|
||||
))
|
||||
s = "Tina makes $18.00 an hour."
|
||||
assert inject_from_match(m, s) == inject_rate_with_currency(m, s)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# (d) Case 0050 hazard pin — sentence_index=0 stays refused.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
_CASES_PATH = (
|
||||
Path(__file__).resolve().parent.parent
|
||||
/ "evals"
|
||||
/ "gsm8k_math"
|
||||
/ "train_sample"
|
||||
/ "v1"
|
||||
/ "cases.jsonl"
|
||||
)
|
||||
|
||||
|
||||
def _load_case_0050() -> dict:
|
||||
"""Look up case 0050 from the fixed eval cases file."""
|
||||
with _CASES_PATH.open() as f:
|
||||
for line in f:
|
||||
c = json.loads(line)
|
||||
if c["case_id"] == "gsm8k-train-sample-v1-0050":
|
||||
return c
|
||||
raise AssertionError("case gsm8k-train-sample-v1-0050 not found in cases.jsonl")
|
||||
|
||||
|
||||
class TestCase0050HazardPin:
|
||||
"""Case 0050: "Mark does a gig every other day for 2 weeks. ..."
|
||||
|
||||
Sentence 0 carries no currency symbol — rate_with_currency never
|
||||
matches it. Even if A2 v1 emitted state (it doesn't), this case
|
||||
would not be reachable through the A2 path. This test makes the
|
||||
invariant explicit so a future A2 widening cannot silently lift
|
||||
the case 0050 hazard.
|
||||
"""
|
||||
|
||||
def test_sentence_zero_has_no_currency_symbol(self) -> None:
|
||||
case = _load_case_0050()
|
||||
# The case's question text is the full problem; split into
|
||||
# sentences on '.' the same way the candidate-graph does.
|
||||
sentences = [s.strip() for s in case["question"].split(".") if s.strip()]
|
||||
sentence_zero = sentences[0]
|
||||
assert "Mark does a gig" in sentence_zero
|
||||
for symbol in "$£€¥":
|
||||
assert symbol not in sentence_zero
|
||||
|
||||
def test_case_0050_remains_refused_end_to_end(self) -> None:
|
||||
case = _load_case_0050()
|
||||
r = parse_and_solve(case["question"])
|
||||
assert r.answer is None
|
||||
# The wrong reading (~3 minutes, per the audit_brief_11 note)
|
||||
# MUST never appear. 280 is the correct expected answer; the
|
||||
# injector must not regress to either.
|
||||
assert r.is_admitted is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# (e) Determinism — same input, byte-identical output.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestDeterminism:
|
||||
def test_injection_is_deterministic(self) -> None:
|
||||
m = _make_match((
|
||||
{
|
||||
"kind": "currency_per_unit_rate",
|
||||
"currency_symbol": "$",
|
||||
"amount": "18.00",
|
||||
"amount_kind": "decimal",
|
||||
"per_unit": "hour",
|
||||
},
|
||||
))
|
||||
s = "Tina makes $18.00 an hour."
|
||||
out1 = inject_rate_with_currency(m, s)
|
||||
out2 = inject_rate_with_currency(m, s)
|
||||
assert out1 == out2
|
||||
assert out1 == () # explicitly pinned: refusal
|
||||
|
||||
def test_dispatch_is_deterministic(self) -> None:
|
||||
m = _make_match((
|
||||
{
|
||||
"kind": "currency_per_unit_rate",
|
||||
"currency_symbol": "$",
|
||||
"amount": "18.00",
|
||||
"amount_kind": "decimal",
|
||||
"per_unit": "hour",
|
||||
},
|
||||
))
|
||||
s = "Tina makes $18.00 an hour."
|
||||
out1 = inject_from_match(m, s)
|
||||
out2 = inject_from_match(m, s)
|
||||
assert out1 == out2
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# (f) Wrong=0 invariant — no candidate is ever produced.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestWrongZeroInvariant:
|
||||
"""The strongest possible wrong=0 statement: the injector emits
|
||||
nothing. A wider follow-up must replace this assertion with a
|
||||
grounded admissibility check on the (Rate, Quantity) composition.
|
||||
"""
|
||||
|
||||
def test_no_candidate_emitted_for_any_known_shape(self) -> None:
|
||||
# Every shape the existing matcher could produce.
|
||||
anchor_variants = (
|
||||
{
|
||||
"kind": "currency_per_unit_rate",
|
||||
"currency_symbol": "$",
|
||||
"amount": "18.00",
|
||||
"amount_kind": "decimal",
|
||||
"per_unit": "hour",
|
||||
},
|
||||
{
|
||||
"kind": "currency_per_unit_rate",
|
||||
"currency_symbol": "$",
|
||||
"amount": "5",
|
||||
"amount_kind": "integer",
|
||||
"per_unit": "cup",
|
||||
},
|
||||
{
|
||||
"kind": "currency_per_unit_rate",
|
||||
"currency_symbol": "$",
|
||||
"amount": "1/2",
|
||||
"amount_kind": "word",
|
||||
"per_unit": "pound",
|
||||
},
|
||||
)
|
||||
for anchor in anchor_variants:
|
||||
m = _make_match((anchor,))
|
||||
out = inject_rate_with_currency(m, "irrelevant under refusal")
|
||||
assert out == ()
|
||||
assert len(out) == 0
|
||||
Loading…
Reference in a new issue