Merge pull request #534 from AssetOverflow/codex/ntimes-completeness-guard

Harden no-reference n-times comparative guard
This commit is contained in:
Shay 2026-06-03 15:38:21 -07:00 committed by GitHub
commit 8327c6bfa1
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 877 additions and 0 deletions

View file

@ -0,0 +1,49 @@
# HANDOFF gpt55 2026-06-03
Branch: `codex/ntimes-completeness-guard`
Final commit SHA: reported in the thread final after commit creation.
## File Status
| file | path | status | purpose | ready-to-merge |
|---|---|---|---|---|
| recognizer anchor injector | `generate/recognizer_anchor_inject.py` | FINAL | Refusal-only guard preventing no-reference `<N> times ...` comparative multipliers from being injected as `CandidateInitial(value=N, unit="times")`. | Y |
| completeness guard tests | `tests/test_candidate_graph_completeness_guard.py` | FINAL | Pins the §9 hard negative matrix, with solve/refusal controls. | Y |
| question-layer gap survey | `docs/analysis/question-layer-gap-survey.md` | FINAL | Canonical audited 44-refusal partition and backlog interpretation. | Y |
| solver operation coverage | `docs/analysis/solver-operation-coverage.md` | FINAL | Canonical read-only audit of existing solver op coverage and representation gaps. | Y |
| composition capability scope | `docs/analysis/composition-capability-scope.md` | FINAL | Canonical execution-lane v2 scope for ADR-0174 Phase 5b emission/representation and §9 guard precondition. | Y |
| comprehension primitive inventory | `docs/analysis/comprehension-primitive-inventory.md` | FINAL | Canonical consolidated inventory and cross-subject leverage map from the execution lane. | Y |
| handoff | `HANDOFF-gpt55-2026-06-03.md` | FINAL | Merge/readiness manifest for this thread. | Y |
## Validation
Completed on branch `codex/ntimes-completeness-guard`:
- `uv run python -m pytest tests/test_candidate_graph_completeness_guard.py -q`
- `21 passed`
- `uv run python -m pytest tests/test_adr_0131_G2_comparatives.py tests/test_adr_0131_G2a_comparative_verb_widening.py -q`
- `30 passed`
- `uv run python -m pytest tests/test_aggregate_total_question_forms.py tests/test_discrete_count_open_noun_class.py tests/test_adr_0163_d2_discrete_count_injection.py -q`
- `59 passed`
- Train-sample probe through `generate.math_candidate_graph.parse_and_solve`
- `6 correct / 44 refused / 0 wrong`
- `refused_nonzero_count = 0`
- admitted IDs: `0003`, `0014`, `0018`, `0021`, `0024`, `0042`
## New Since Last Message
- Broadened the §9 guard from phrase-specific `times as many|more` matching to the structural captured shape: cardinal immediately followed by `times` inside the discrete-count injector.
- Added no-reference hard negatives for `3 times the number of apples` and `3 times the apples`.
- Added explicit safe-refusal controls for no-reference `twice`, no-reference `double`, and case 605.
- Added canonical execution-lane `composition-capability-scope.md`.
- Added canonical execution-lane `comprehension-primitive-inventory.md`.
- Added this handoff manifest.
## Not Done / WIP
- No 5b emission/representation slice was implemented.
- No <=20-case validation sub-corpus was authored.
- No solver operation kinds or binding-graph node types were added.
- No serving/eval/claims-ledger files were changed.
- Superseded `0001` / `0002` / `0003` patch files from the Opus handoff directory were not committed.

View file

@ -0,0 +1,244 @@
<!-- CANONICAL | composition-capability-scope.md | updated 2026-06-03 | execution lane | supersedes prior copies -->
# Composition-Capability Scope — v2 (re-anchored to shipped reality)
Status: Proposed analysis / build-scoping. No serving or eval edits. Source of
truth: `docs/claims_ledger.md`, `evals/gsm8k_math/train_sample/v1/report.json`,
ADR-0174 **read in full (644 lines, incl. Implementation Notes + amended Phase 5,
lines 300533)**. Every empirical claim below reproduced live against `main` @ `3e29559`.
> **v1 correction.** v1 aimed the plan at "land Phase 1; the unlanded value is
> Phase 3/4." That was wrong — scoped from ADR-0174's forward sections only,
> missing the ~225 lines documenting that **Phases 14 already shipped and are
> wired into serving**. Verified on `main`:
> `generate/comprehension/{state,lookback,contemplate,constraint_propagation}.py`
> exist with `HYPOTHESIS_CAP`/`open_hypotheses`/`Hypothesis`/`UnknownHeld`/
> `reevaluate`/`contemplate`, and `reevaluate`+`contemplate` are imported and
> invoked inside `generate/math_candidate_graph.py::parse_and_solve`. Metric still
> **6/44/0**. v2 re-anchors to the ADR's own amended Phase 5.
## 0. The finding that scopes everything (corrected)
The held-hypothesis machinery (ADR-0174 Phases 14) is **live in serving and the
metric did not move.** A live `parse_and_solve` audit of all 44 refusals (§1)
locates the wall precisely: **44/44 refuse at `branches_enumerated = 0` — upstream
of the solver, which is never reached.** So the lift is **not** more reader
machinery, and **not** solver operation coverage (the 8 op-kinds are never reached;
see §1). The wall is **emission/representation**: the recognizer/parser/binding
layer cannot construct an admissible multi-step candidate for these shapes, so
nothing is fed to the (already-capable) solver. ADR-0174's "removing the 35
narrowness layers per case" is that emission/representation work.
This inverts v1's thesis *and* supersedes the intermediate "operation coverage"
thesis: the composition wall is at **emission/representation upstream of the
solver**, not at the operation layer.
## 1. Verified current state on `main`
| Layer | State |
|---|---|
| Held-hypothesis reader (P1P4: state, constraint-propagation, lookback, contemplate) | **Shipped, wired into serving** candidate-graph |
| `HYPOTHESIS_CAP`, vault>packs>audit precedence | already set/enforced (0174 OQ#1/#3 moot) |
| `lifecycle.py` GSM8K-scoring dispatch | **inert** (admits 0/50) — retirable in 5a |
| `lifecycle.py` / `audit.py` reader surface | **load-bearing** for the ADR-0172 teaching corridor — **keep** |
| Solver operation kinds | **8 already exist** (verified by discovery): `add`, `subtract`, `transfer`, `multiply`, `divide`, `apply_rate`, `compare_additive`, `compare_multiplicative`. Missing arithmetic op kinds is **not** the blocker. |
| **Live audit of all 44 refusals (ran `parse_and_solve` on `main`)** | **44/44 refuse at `branches_enumerated = 0`** — i.e. *upstream of the solver*. No branch is built, so the 8 operation kinds are **never reached**. `rate`/`ratio` also exist as first-class `SEMANTIC_ROLES`/`QUESTION_FORMS` (`binding_graph/model.py:4265`); only `percent`/`accumulate` lack a named op-kind — moot while nothing reaches the solver. |
| train_sample metric | **6 / 44 / 0** (`correct_min=10` not yet passed) |
## 2. The actual remaining work (ADR-0174 amended Phase 5)
### Phase 5a — retire the inert parallel parser (structural, ~0 lift)
Retire the GSM8K-scoring-only inert dispatch (`_try_comprehension_reader` /
`_try_reader_for_question`) + adapter + `use_reader` plumbing. Net **1,038 LOC**
(code + tests). **Keep** `lifecycle.py`/`audit.py` — their reader *refusals* feed
the ADR-0172 math-contemplation teaching corridor (`teaching/math_*`,
`evals/flywheel_demo`, `core/cli.py`). Low-risk; single-path serving.
Acceptance: 6/44/0 byte-identical, capability-axis lanes 100% `wrong=0`,
pinned-SHAs pass.
### Phase 5b — emission / representation buildout (semantic — the real lift)
"This is where `correct` climbs toward 25. It is not a refactor." **Verified
diagnosis (live `parse_and_solve` audit of all 44, on `main`):** every refusal
occurs at `branches_enumerated = 0`*upstream of the solver*. The 8 operation
kinds are never reached. So 5b is **not** an operation-execution problem and
**not** chiefly a new-primitive problem. It is an **emission/representation**
problem: the recognizer/parser/binding layer cannot construct an admissible
multi-step candidate for these shapes, so nothing is fed to the (already-capable)
solver.
The 44 split into two emission failure modes (verified, matches `report.json`):
- **32 "recognizer matched but produced no injection"** — an anchor fired but the
hypothesis-emitter declined to construct a candidate (cannot represent the
multi-clause / derived / scoped structure).
- **12 "no admissible candidate"** — the parser produced nothing admissible.
**Sequencing implication:** fix emission/representation first; the
`percent`/`accumulate` primitive question is **secondary** — answerable only after
cases actually reach the solver. This *lowers* the wrong=0 risk: emitting existing
ops behind the same admissibility predicates is lower-risk than new solver math.
**5b first PR (smallest verified-tractable slice):** pick the emission failure mode
with the highest count whose representation is bounded — and prove it reaches branch
enumeration on ≥1 case without breaking `wrong=0`. **But the §9 completeness-gate
precondition lands before any emission PR.**
## 3. Test anchors (reusable, relabeled to 5a/5b)
1. **Serving gate (`gsm8k_math`):** 5a byte-identical 6/44/0; 5b climbs
(`correct ≥ 10` clears the ADR-0126 exit, target → 15 → 25), `wrong = 0` at
every step.
2. **Regression nets (100% `wrong=0`):** G1G5, S1; `anti_regression`; case-0050 pin.
3. **Determinism:** `trace_hash` over `open_hypotheses`; replay-equivalence.
4. **Serving freeze:** `verify_lane_shas.py` passes each step.
5. **Still-needed new asset (0174 OQ#5):** the curated ≤20-case multi-step
validation sub-corpus — **unbuilt, still required for 5b measurement.** The §9
hard negatives are the first deposits into it. Author before 5b measurement.
## 4. Corrected sequencing
0. **Precondition (§9):** harden the comparative-multiplicative completeness tripwire so a dropped `<N>×` clause refuses (as `twice` does). 5b emission on these shapes is unsafe until this lands. Driver test is FINAL and RED on `main` (see §9).
1. **5b gating analysis — done (§8):** the **emission/representation** audit ran and the 32 no-injection cases are sub-classified by representation. Highest-count bounded gap = **R1 derived/intermediate symbol (24/44)**.
2. **Author the ≤20-case multi-step validation sub-corpus** (test anchor for 5b).
3. **Phase 5a** (optional, can run in parallel — structural, low-risk).
4. **Phase 5b** — build the **emitter/representation** for R1 (derived-symbol) on the §8 near-pure exemplars (prove ≥1 case reaches `branches_enumerated > 0` and admits), each behind §3 gates **and the §9 precondition**. Not new operations — the 8 exist and are unreached.
(Do **not** "land Phase 1" — shipped. Do **not** treat P3/P4 as the lever — shipped, metric flat.)
## 5. Relation to cross-subject testing
5b maturing the math `DomainSolver`'s emission/representation is still the precondition
for wiring symbolic_logic as arena #2; the held-hypothesis reader + disagreement
gate remain the shared, arena-portable primitives. `cross_domain_transfer` /
`monotonic_learning` (both exist, with `contract.md`/`holdouts/`) become live tests
once arena #2 exists.
## 6. Honest ceiling (corrected)
The ≥15 climb is a **5b** outcome (emission/representation), **not** a P4 outcome — P4
shipped and we are at 6. Cases needing world knowledge (0040 legs) or representations 5b
does not add stay refused = `wrong = 0` holding, not failure.
## 7. Open questions for the build lane
1. **Answered by the live audit:** the binding constraint is emission upstream of
the solver (44/44 at `branches_enumerated=0`), not operation execution. Remaining:
rank the §8 R-classes by the size of the bounded-representation slice per class.
2. Of the 44, which are emission-fixable with existing ops vs (a) genuinely need a
`percent`/`accumulate` primitive *after* emission, vs (b) world-knowledge/out-of-
scope permanent refusals (e.g. 0040 legs-per-animal). The audit gives the upstream
verdict; this split needs per-case reader-trace reads.
3. 5a pre-flight: confirm the exact inert-dispatch LOC and that no teaching-corridor
consumer breaks when the GSM8K-scoring dispatch is retired.
---
## 8. Verified per-case representation classification (live audit, all 44)
Structural reading of each of the 44 (what representation the emitter must build for
emission to succeed), grounded in the case text + the live `parse_and_solve` locus.
Multi-tagged; a case usually needs several. Frequency across the 44:
| Representation gap | # cases (multi-tagged) |
|---|---:|
| R5 — multi-step rate/duration/scalar | 27 |
| R1 — derived/intermediate symbol | 24 |
| R6 — percent/fraction mutation (no op-kind) | 18 |
| R4 — accumulation/residual | 10 |
| R2 — inverse target | 6 |
| R3 — subset/partition scope | 3 |
| R7 — world-knowledge (permanent refusal) | 1 |
**Highest-leverage gap: R1 (derived/intermediate symbol)** — 24/44 need to compute an
intermediate quantity and reuse it downstream. Its lowest-arity exemplars are nearly
pure R1: 0027 (Twitter = half of IG+FB), 0008 (total beads → ÷ per bracelet),
0029 (keyboard = 3× mouse → sum), 0038 (×3 → sum). Contrastive proof it is the emission
gap (parseable aggregate question form): `"Nicole has 400 cards. Cindy has 800 cards.
How many cards do they have together?"` **admits (1200)**; `"…Cindy has twice as many
cards. How many cards do they have together?"` reaches `branches_enumerated=1` but
**refuses** (completeness: scalar `2.0` unconsumed). The stated-sum reaches the solver;
the derived form does not. R7 (0040 legs-per-animal) is a permanent refusal — world
knowledge.
**First 5b slice (recommended):** derived/intermediate-symbol emission, validated on the
four near-pure exemplars above — move them from refused to admitted, `wrong=0` preserved.
**But see §9 first — it is a hard precondition.**
## 9. ⚠ Latent `wrong=0` hazard surfaced by the live audit (gate gap)
Contrastive probes (not in the 50-case sample) surface a reproducible **admitted-wrong**
path. **Use the parseable aggregate question form `"...do they have together?"`** — the
short form `"How many apples together?"` refuses upstream at question-parse
(`branches_enumerated=0`) and does NOT exercise this `be=1` completeness gap.
```
"Tom has 7 apples. Jerry has 3 times as many apples. How many apples do they have together?"
→ admitted=True, answer=7, branches_enumerated=1 (correct = 28) # base returned, clause dropped
"...Jerry has five times as many apples. ..." → admitted, answer=7 (correct = 42)
"...Jerry has twice as many apples. ..." → refused, be=1 (completeness: scalar 2.0 unconsumed) # safe
```
**Broadened surface (all verified RED on `main`).** The hazard is the **no-reference**
comparative-multiplier surface across all four connectives and both cardinal forms:
```
<N> times as many <unit>
<N> times more <unit>
<N> times the number of <unit>
<N> times the <unit>
```
…for `<N>` ∈ {digit 2/3/5/…, word two/three/five/…, N≥2}. Every one admits the base.
`twice` / `double` / `double the` (no digit/cardinal) refuse safely.
**Root cause (verified by reading the emitted `MathProblemGraph`).** The admitted graph
contains a spurious initial:
`InitialPossession(entity='Jerry', quantity=Quantity(value=3, unit='times'))`.
1. `quantity_values_in_text` is **symmetric** (registers 2.0 for `twice` *and* 3.0 for
`3 times`) — **not** a quantity-extraction asymmetry (this overturns an earlier guess).
2. For the no-ref surface, **neither** serving comparative regex fires
(`_COMPARE_MULT_ANCHOR_RE` / `_COMPARE_MULT_NTIMES_RE` both require an "as `<REF>`"
tail the probe lacks). `comparatives.py::_N_TIMES_RE` is the **disjoint** `derivation/`
reader — off the serving path — so it is *not* the locus either.
3. Instead `recognizer_match.py::_match_discrete_count_statement` (open regex
`_extract_discrete_count_re_open`) captures the multiplier cardinal as a **count**, and
`recognizer_anchor_inject.py::_build_initial_from_discrete_count` builds
`CandidateInitial(value=N, unit='times', entity=<actor>)` — note the unit is the literal
`'times'` token, **not** the counted unit.
4. That bogus initial **consumes** the scalar N, so completeness sees
`uncovered = {N, base} {N, base} = ∅` → admits. The answer sums only the counted unit
(`apples`) and returns the base. `twice`/`double` carry no cardinal the discrete-count
regex grabs → no spurious initial → scalar unconsumed → existing guard refuses.
**Recommended guard shape (refusal-only, `wrong=0` preserving).** Make the discrete-count
recognizer **decline** when its cardinal sits in a
`<N> times {as many | more | the number of | the} <unit>` comparative-multiplier context
— equivalently, refuse to build a discrete-count initial whose unit is the literal
`'times'`. The no-ref form then refuses (like no-ref `twice`) until real no-ref
comparative-multiplicative *emission* lands in 5b. The fix MUST NOT regress the controls
below.
**Controls (verified on `main`).**
- **With-ref `<N> times` already solves — must stay green.** train_sample **case 0024**:
*"Sidney does 20 jumping jacks on Monday, 36 on Tuesday, 40 on Wednesday, and 50 on
Thursday. Brooke does three times as many jumping jacks as Sidney. How many jumping
jacks did Brooke do?"* → **438** (committed verdict: correct). Also `dice 3×` → 80.
- **Existing safe refusals — must stay refused.** no-ref `twice`, no-ref `double the`,
with-ref `twice` (Ivan/Jerry dice).
**Driver test (FINAL, RED on `main`).** `test_completeness_guard_ntimes_noref_hazard.py`
**10 no-ref hazard cases** (the broadened surface × digit/word) that MUST refuse, plus
**2 with-ref must-solve controls** (0024 → 438, dice → 80) and **3 must-still-refuse
controls**. Verified live: **10 failed, 5 passed** on `main` (hazard RED, controls green).
This is the first concrete 5b PR, ahead of any R1 emission work.
**Scope — stated precisely.** This is a **latent gate gap, not a live-metric violation.**
The two layers are distinct: the §1 wall is at `branches_enumerated=0`
(emission/question-parse); this hazard lives at `branches_enumerated=1` (the completeness
gate). The real 44 all refuse at `be=0` — upstream of `be=1` — so none reach this gap
today; that is *why* it is latent. train_sample 6/44/0 is reproduced live. The full-test
`0/0/1319` is **not** re-verified here — it is a sealed, recorded measurement per
`docs/claims_ledger.md` row A / `ADR-0119.7` (ciphertext
`evals/gsm8k_math/holdouts/v1/cases.jsonl.age`), not a CI-reproducible artifact; cited,
not re-verified (the citation and ciphertext both exist on `main`).
**Why it gates 5b.** Phase 5b's explicit goal is to make comparative-multiplicative cases
*reach the graph*. If emission improves while this completeness gap remains, 5b would
**convert latent into live** — admitting wrong answers on exactly the `<N> times` shape it
is trying to solve. Therefore: **hardening the comparative-multiplicative completeness
tripwire (so a dropped `<N>×` clause refuses, as `twice` does) is a hard precondition for
any 5b emission PR.** The §9 driver test is the ready-made hard negative for the ADR-0174
OQ#5 validation sub-corpus.

View file

@ -0,0 +1,211 @@
<!-- CANONICAL | comprehension-primitive-inventory.md | updated 2026-06-03 | execution lane | supersedes prior copies -->
# Comprehension-Primitive Inventory & Cross-Subject Leverage Map
Status: draft / proposal-only
Scope: read-only analysis from `main`, **verified** in the Claude lane (see appendix)
Task: Task A from `docs/handoff/NEXT-SUBJECTS-CHATGPT-HANDOFF.md`
## Operating constraints observed
This artifact is analysis only. It proposes no serving-path edits, no eval edits, no ADR number, and no empirical claims in the inventory body. Any correctness, coverage, or `wrong=0` claim in the body is a structural reading of the code. The **Claude-lane verification appendix** at the end records what was checked against `main` with real reads of the committed report and source; it is the only section permitted to assert empirical state.
Read surfaces:
- `docs/handoff/NEXT-SUBJECTS-CHATGPT-HANDOFF.md`
- `CLAUDE.md`
- `generate/derivation/model.py`
- `generate/derivation/extract.py`
- `generate/derivation/clauses.py`
- `generate/derivation/comparatives.py`
- `generate/derivation/search.py`
- `generate/derivation/multistep.py`
- `generate/derivation/target.py`
- `generate/derivation/compose.py`
- `generate/derivation/accumulate.py`
- `generate/derivation/pool.py`
- `generate/derivation/product_bridge.py`
- `generate/derivation/state/bind.py`
- `generate/derivation/state/change.py`
- `generate/math_candidate_parser.py`
- `generate/math_candidate_graph.py`
- `generate/recognizer_anchor_inject.py`
- Skimmed referenced ADR surface in code/docstrings, especially ADR-0126, ADR-0131, ADR-0136, ADR-0163, ADR-0170, ADR-0175, ADR-0176, ADR-0178, ADR-0182, ADR-0184, ADR-0186, ADR-0189a, ADR-0191, ADR-0193, ADR-0194, ADR-0195.
## Inventory table
| # | Primitive | File / function(s) actually read | One-line description | Subject-general vs math-specific? |
|---:|---|---|---|---|
| 1 | Grounded quantity object | `generate/derivation/model.py::Quantity` | Represents a text-sourced numeric value with unit and source token provenance. | **Subject-general core, math-shaped payload.** Provenance-bearing "observed fact" objects transfer broadly; the `value/unit` fields are math-specific. |
| 2 | Grounded derivation step | `generate/derivation/model.py::Step` | Represents one operation, its operand, and the licensing cue that must ground in text. | **Subject-general.** The pattern "claim/action must carry its own evidence cue" transfers to logic, reading comprehension, measurement, and any rule-bound subject. |
| 3 | Deterministic left-fold derivation | `generate/derivation/model.py::GroundedDerivation.answer` | Computes a candidate result by left-folding validated steps over a start quantity. | **Mostly math-specific.** The arithmetic fold is math-specific; the generalizable primitive is ordered, evidence-carrying state transition. |
| 4 | Primary-unit answer tracking | `generate/derivation/model.py::GroundedDerivation.answer_unit` | Carries the start quantity's unit as the answer unit under current derivation assumptions. | **Math-specific.** It is specifically dimensional arithmetic; the cross-subject analogue is "result type/class propagation." |
| 5 | Digit quantity extraction | `generate/derivation/extract.py::extract_quantities`, `_QTY_RE`, `_quantity` | Extracts digit values followed by single unit tokens into `Quantity` records. | **Subject-general extraction pattern, math-specific symbols.** Literal-span extraction with provenance transfers; numeric parsing/unit attachment is math-specific. |
| 6 | Word-number extraction | `generate/derivation/extract.py::_WORD_QTY_RE`, `_resolve_word_number`, `extract_quantities` | Resolves closed-set word numerals and conservative hyphen compounds into quantities. | **Broadly reusable.** Any subject with controlled lexical facts can use the same closed-vocabulary grounding discipline. |
| 7 | Function-word unit hygiene | `generate/derivation/extract.py::_NON_UNIT_WORDS`, `_clean_unit` | Blanks function words that would otherwise be misread as units. | **Subject-general.** This is a lexical false-positive suppression primitive; math uses it for units, but other subjects need equivalent stop-token guards. |
| 8 | List-unit inheritance | `generate/derivation/extract.py::_LIST_WITH_TRAILING_UNIT_RE`, `extract_quantities` | Assigns a trailing unit to every number in a same-list numeric sequence. | **Mixed.** The list inheritance pattern transfers to reading/measurement; the inherited object is math-specific. |
| 9 | Sentence-final bare number extraction | `generate/derivation/extract.py::_FINAL_NUMBER_RE`, `extract_quantities` | Keeps terminal numbers available with unknown/empty unit rather than inventing a unit. | **Subject-general refusal-first grounding.** It preserves observed evidence without hallucinating missing attributes. |
| 10 | Hyphen-bonded quantity extraction | `generate/derivation/extract.py::_HYPHEN_QTY_RE`, `extract_quantities` | Extracts tight `number-unit` surfaces such as `25-foot` without admitting open-ended multi-word units. | **Mixed.** Hyphenated modifier handling transfers; the payload is measurement/math-specific. |
| 11 | Clause segmentation | `generate/derivation/clauses.py::segment_clauses` | Splits problem text into sentence-level clauses using terminal punctuation. | **Subject-general.** Clause segmentation is a foundational reading primitive; the implementation is intentionally orthographic and conservative. |
| 12 | Clause-local sub-derivation | `generate/derivation/clauses.py::clause_local_results` | Derives each clause's local contribution or holds unresolved on ambiguity. | **Subject-general.** "Resolve locally before composing globally" transfers directly to reading comprehension, logic proof steps, and multi-sentence science/measurement tasks. |
| 13 | Comparative scalar extraction | `generate/derivation/comparatives.py::extract_comparative_scalars`, `_load_comparatives`, `_N_TIMES_RE` | Maps closed comparative lexemes and `<N> times` phrases into scalar operations. | **Mixed.** Closed lexical relation extraction is subject-general; scalar multiplication is math-specific. |
| 14 | Comparative-to-step bridge | `generate/derivation/comparatives.py::comparative_step` | Converts a comparative scalar into a derivation step whose grounding comes from the cue, not necessarily a literal numeric token. | **Subject-general.** The idea that an irreducible lexical fact licenses a typed transformation transfers strongly; the concrete operation is math-specific. |
| 15 | Multiplicative cue hypothesis | `generate/derivation/search.py::MULTIPLICATIVE_CUES`, `_sentence_candidates` | Uses a closed cue set to propose in-clause product candidates only when a multiplicative cue is present. | **Mixed.** Cue-licensed candidate generation is general; multiplication/product semantics are math-specific. |
| 16 | Bounded candidate generation | `generate/derivation/search.py::MAX_QUANTITIES`, `multiplicative_candidates`, `search_multiplicative`; `generate/derivation/multistep.py::MAX_QUANTITIES`, `candidate_chains` | Refuses rather than enumerating unbounded candidate spaces. | **Subject-general.** This is a core safety/performance primitive for any new subject. |
| 17 | Target extraction from question clause | `generate/derivation/target.py::_question_clause`, `extract_target` | Extracts question quantities, aggregation cues, and units named in the question. | **Strongly subject-general.** Every subject lane needs "what is being asked?" extraction; current fields are math-specific. |
| 18 | Prior-state question guard | `generate/derivation/target.py::asks_prior_state`, `_PRIOR_STATE_RE` | Detects questions asking for an earlier temporal state that forward derivation does not compute. | **Subject-general.** Temporal target mismatch is common across reading comprehension, science word problems, and procedural reasoning. |
| 19 | Aggregation hint extraction | `generate/derivation/target.py::_AGG_WORDS`, `_AGG_PHRASES`, `extract_target` | Detects aggregation words/phrases such as `total`, `combined`, and `in all`. | **Mixed.** Aggregation-cue extraction transfers; summation semantics are math-specific. |
| 20 | Question unit intersection | `generate/derivation/target.py::extract_target` | Treats asked units as body-known units that appear in the question. | **Mixed.** Target-slot/body-slot intersection transfers; unit semantics are math-specific. |
| 21 | Shape-based multi-step chain enumeration | `generate/derivation/multistep.py::_candidate_chains`, `_chain`, `candidate_chains` | Builds a small deterministic set of product/sum chains, optionally followed by comparative tail steps. | **Mixed.** Shape-pruned candidate enumeration is general; product/sum chain templates are math-specific. |
| 22 | Same-unit list-sum composition | `generate/derivation/compose.py::compose_sequential`, `_same_unit`, `_ADDITIVE_CUES` | Composes same-unit quantities within one clause using additive cues, with comparative tail application. | **Mixed.** Same-scope list composition transfers to reading/logic lists; same-unit arithmetic is math-specific. |
| 23 | Clause-scoped referent guard | `generate/derivation/compose.py::compose_sequential` | Refuses when a list-sum structure spans multiple quantity-bearing clauses or has out-of-clause comparatives. | **Subject-general.** Scope containment is a central comprehension primitive and directly transfers to reading comprehension. |
| 24 | Single-referent accumulation chaining | `generate/derivation/accumulate.py::_build_accumulation`, `compose_accumulation` | Chains gain/loss changes across clauses only when a later clause safely continues the anchor referent. | **Strongly subject-general.** This is state tracking over discourse; math uses numeric state, but the primitive is broadly useful. |
| 25 | Foreign-distractor candidate handling | `generate/derivation/accumulate.py::_build_accumulation`, `accumulation_candidates`; `generate/derivation/verify.py::classify_derivation` | Allows isolated foreign quantities to enter as disagreement-only/exempt readings rather than commit candidates. | **Subject-general safety primitive.** Distractor evidence handling transfers to all comprehension lanes with irrelevant details. |
| 26 | Sub-clause splitting | `generate/derivation/accumulate.py::_sub_clauses`, `_CONJUNCTION_SPLIT`, `_build_accumulation_anchor_skip` | Locally splits clauses on conjunctions for anchor/change discovery without changing the global segmenter. | **Subject-general.** Local structural refinement under a narrow caller-owned scope transfers well. |
| 27 | Leading-subject extraction | `generate/derivation/state/bind.py::leading_subject_token` | Extracts a clause's leading word token as a loose subject signal. | **Subject-general.** It is a minimal discourse entity cue. |
| 28 | Conservative same-referent continuation | `generate/derivation/state/bind.py::continues_anchor_referent`, `PRONOUNS` | Allows pronouns/same subjects/lowercase continuations and refuses new capitalized actor hazards. | **Subject-general.** This is directly reusable for reading comprehension and logic story-state tracking. |
| 29 | Change polarity classification | `generate/derivation/state/change.py::classify_change_polarity`, `GAIN_VERBS`, `LOSS_VERBS` | Maps closed gain/loss cue sets to `+1`, `-1`, or refusal on ambiguity. | **Mixed.** Polarity classification is subject-general; gain/loss inventory is math-story specific. |
| 30 | Grounded change cue selection | `generate/derivation/state/change.py::select_change_cue` | Chooses the actual cue lexeme that will be checked by the verifier. | **Subject-general.** Separating classification from evidence-cue selection is broadly valuable. |
| 31 | Operand grounding gate | `generate/derivation/verify.py::self_verifies`, `_base_reasons` | Requires every non-comparative operand value token to ground in the problem text. | **Subject-general.** No invented evidence is a cross-domain invariant. |
| 32 | Operation-cue grounding gate | `generate/derivation/verify.py::_base_reasons` | Requires every operation's licensing cue to appear in the text. | **Subject-general.** Every subject lane should require transformation rules to be evidence-licensed. |
| 33 | Unit consistency gate | `generate/derivation/verify.py::_base_reasons`, `_SAME_UNIT_REQUIRED` | Requires same units for add/subtract while allowing multiply/divide composition. | **Math-specific with transferable type discipline.** The gate's type-checking role transfers; the unit rules are math-specific. |
| 34 | Completeness gate | `generate/derivation/verify.py::_unused_quantities`, `self_verifies` | Refuses derivations that leave problem quantities unused. | **Subject-general.** "Account for all salient evidence" is central to reading, logic, measurement, and science tasks. |
| 35 | Branch disagreement / uniqueness gate | `generate/derivation/verify.py::select_self_verified`; `generate/derivation/pool.py::resolve_pooled`; `generate/math_candidate_graph.py::parse_and_solve` | Commits only when verified candidates collapse to one distinct answer; otherwise refuses. | **Strongly subject-general.** This is one of the most transferable wrong=0-preserving primitives. |
| 36 | Commit-eligible vs exempt classification | `generate/derivation/verify.py::classify_derivation` | Classifies readings as complete, exempt, or invalid; exempt readings can force disagreement but cannot commit alone. | **Subject-general.** "Counter-reading can block commitment without becoming an answer" is broadly useful. |
| 37 | Repeated-unit product hazard detector | `generate/derivation/verify.py::_is_repeated_unit_product` | Marks pure products that repeat non-empty dimensions as commit-ineligible. | **Math-specific.** The general form is domain-type impossibility detection. |
| 38 | Cross-composer pooling | `generate/derivation/pool.py::pooled_candidates`, `resolve_pooled` | Pools accumulation, multiplicative, and target-guided chain readings before applying disagreement/commit rules. | **Subject-general architecture.** Multiple independent readers should meet at a common disagreement gate. |
| 39 | Serving promotion bridge | `generate/derivation/product_bridge.py::resolve_promotable_product`, `_has_hazard_surface`, `_has_product_target` | Promotes only complete pure-product readings whose question target and blocker checks make them safe for serving exposure. | **Mixed.** Promotion-boundary pattern is subject-general; current target/hazard surfaces are math-specific. |
| 40 | Candidate initial-state extraction | `generate/math_candidate_parser.py::extract_initial_candidates`, `CandidateInitial` | Emits initial possession/state candidates with source-span provenance. | **Subject-general.** Initial state extraction is foundational for any story/world model; possession quantity is math-specific. |
| 41 | Value-slot resolution | `generate/math_candidate_parser.py::_resolve_value`, `_resolve_currency`, `_is_indefinite_quantifier` | Resolves digits, money, fractions, word numbers, and hyphenated cardinals; refuses indefinite/unparseable values. | **Mixed.** Refusal-first lexical resolution transfers; supported value types are math-specific. |
| 42 | Unit canonicalization | `generate/math_candidate_parser.py::_canonicalize_unit`, `_money_unit_normalization` | Maps surface unit tokens to canonical/plural units, including money normalization. | **Math/measurement-specific with transferable normalization boundary.** Other subjects need similar canonicalization for entities, predicates, or labels. |
| 43 | Operation candidate extraction | `generate/math_candidate_parser.py::extract_operation_candidates`, `_op_pattern`, `_build_op_candidate` | Emits add/subtract/transfer operation candidates from canonical subject-verb-value-unit shapes. | **Mixed.** Typed event extraction transfers; arithmetic operation kinds are math-specific. |
| 44 | Comparative operation extraction | `generate/math_candidate_parser.py::_compare_additive_candidates`, `_compare_multiplicative_candidates`, `_compare_nested_candidates`, `_resolve_reference_token` | Emits comparison candidates using closed comparison anchors and reference grounding. | **Mixed.** Comparative relation extraction transfers strongly; numeric delta/factor semantics are math-specific. |
| 45 | Question candidate extraction | `generate/math_candidate_parser.py::extract_question_candidates`, `CandidateUnknown` | Emits unknown target candidates from closed question shapes. | **Subject-general.** Question-frame parsing is a primary cross-subject bottleneck. |
| 46 | Aggregate question frames | `generate/math_candidate_parser.py::_Q_TOTAL_RE`, `_Q_THERE_RE`, `extract_question_candidates` | Maps total-across question surfaces to `Unknown(entity=None, unit=...)`. | **Mixed.** Aggregate target framing transfers; "unit total" is math-specific. |
| 47 | Activity question frame | `generate/math_candidate_parser.py::_Q_DID_RE`, `extract_question_candidates` | Handles `How many <unit> did <Entity> <verb>?` activity-count questions. | **Mixed.** Activity target extraction transfers; counted activity quantity is math-specific. |
| 48 | Conditional-prefix stripping | `generate/math_candidate_graph.py::_strip_conditional_prefix`, `_filtered_question_choices` | Retries question parsing after removing an `If X,` prefix. | **Subject-general.** Conditional-wrapper removal is broadly useful across logic and reading comprehension. |
| 49 | Comparative-question refusal detector | `generate/math_candidate_parser.py::_pattern_b_comparative_candidates`, `_pattern_b_detects` | Recognizes "how many more" questions but emits no candidate until solver semantics exist. | **Subject-general safety primitive.** Detection-only recognizers can force clean refusal without pretending capability. |
| 50 | Pronoun question resolution | `generate/math_candidate_parser.py::_resolve_pronoun_entity`, `_resolve_question_entity`, `_pattern_c_pronoun_verb_candidates` | Resolves gendered pronoun question entities only when exactly one whitelisted antecedent is present. | **Subject-general, implementation narrow.** The refuse-on-ambiguity pattern transfers; current name lists are GSM8K-specific. |
| 51 | Statement context classifier | `generate/math_candidate_parser.py::has_numeric_token`, `classify_sentence` | Skips non-numeric context statements while preserving numeric-state-bearing statements as required parse/refuse inputs. | **Mixed.** Context filtering transfers; numeric-token criterion is math-specific. |
| 52 | Capacity/rate extraction | `generate/math_candidate_parser.py::extract_capacity_candidates`, `extract_capacity_question_candidates`, `_to_seconds`; `generate/math_candidate_graph.py::parse_and_solve` | Extracts capacity per time and matching time-target questions, then computes scaled rate answers in a guarded short-circuit. | **Math/measurement-specific.** The broader primitive is matched statement/question rate-frame binding. |
| 53 | Earnings-rate extraction | `generate/math_candidate_parser.py::extract_earnings_candidates`, `extract_earnings_question_candidates`; `generate/math_candidate_graph.py::parse_and_solve` | Extracts currency-per-time statements and matching money-over-time questions. | **Math/measurement-specific.** Transfers mainly to measurement/finance-like lanes. |
| 54 | Conditional operation question | `generate/math_candidate_parser.py::extract_conditional_op_question_candidates`; `generate/math_candidate_graph.py::parse_and_solve` | Handles `If entity changes by N, how many ... left/now?` by matching one existing initial state and applying polarity. | **Mixed.** Conditional hypothetical target binding transfers strongly; arithmetic update is math-specific. |
| 55 | Sentence splitting / one-question invariant | `generate/math_candidate_graph.py::_split_sentences`, `parse_and_solve` | Splits text, requires exactly one question sentence, and refuses otherwise. | **Subject-general.** Most subject lanes need explicit problem/question segmentation and clean refusal on malformed tasks. |
| 56 | Per-sentence round-trip filtering | `generate/math_candidate_graph.py::_filtered_statement_choices`, `_filtered_question_choices`, `_initial_admissible`, `_question_admissible` | Filters emitted candidates by structural grounding before graph assembly. | **Subject-general.** Candidate emission and admissibility must remain separate in every subject. |
| 57 | Most-grounded-slots tiebreaker | `generate/math_candidate_graph.py::_slot_count`, `_collapse_per_sentence_ties` | Collapses same-sentence candidates to the most grounded candidate when appropriate. | **Subject-general but hazardous if overused.** It transfers as a deterministic tiebreaker, but each subject must prove it cannot mask ambiguity. |
| 58 | Graph construction with referential integrity | `generate/math_candidate_graph.py::_build_graph` | Builds a `MathProblemGraph`, rejecting branches whose question references unknown entities or violate graph invariants. | **Subject-general architecture, math-specific graph type.** Every subject needs typed graph construction with integrity checks. |
| 59 | Cartesian branch enumeration cap | `generate/math_candidate_graph.py::MAX_TOTAL_BRANCHES`, `parse_and_solve` | Bounds branch enumeration and refuses when the space would exceed the cap. | **Subject-general.** Essential for deterministic safety and performance. |
| 60 | Recognizer registry fallback | `generate/math_candidate_graph.py::_load_ratified_registry_or_empty`, `parse_and_solve` | Consults ratified recognizers only when parser choices are empty, and treats registry failures as empty. | **Subject-general.** Reviewed recognizer fallback with fail-closed behavior transfers directly. |
| 61 | Anchor injection dispatch | `generate/recognizer_anchor_inject.py::inject_from_match` | Converts recognized anchors into typed solver primitives or returns empty on unsupported/unsafe categories. | **Subject-general.** This is a reusable boundary between recognizers and solver primitives. |
| 62 | Composition registry consultation | `generate/recognizer_anchor_inject.py::_consult_composition_registry` | Admits pre-composed payloads only when the composition registry affirms their surface shape. | **Subject-general.** Reviewed structural-shape admission is reusable for logic, reading, and geometry. |
| 63 | Discrete-count anchor injection | `generate/recognizer_anchor_inject.py::inject_discrete_count_statement`, `_build_initial_from_discrete_count`, `_build_operation_from_discrete_count_acquisition` | Builds initial-state or add-operation candidates from discrete-count recognizer anchors. | **Mixed.** Anchor-to-typed-fact injection is general; discrete count semantics are math-specific. |
| 64 | Sealed injector lane | `generate/recognizer_anchor_inject.py::_SEALED_INJECTORS`, `inject_from_match`; `generate/math_candidate_graph.py::parse_and_solve(sealed=...)` | Keeps in-development injectors out of default serving until reviewed promotion. | **Subject-general.** This is a major reusable safety boundary for new subject lanes. |
| 65 | Lookback pronoun resolution / ambiguity defense | `generate/math_candidate_graph.py::parse_and_solve` recognizer-injection section | Holds pronoun-requiring injected candidates until a discourse antecedent or pack-backed disambiguation is available; otherwise drops them. | **Strongly subject-general.** This is directly relevant to reading-comprehension and story-state subjects. |
| 66 | Reader trace events | `generate/math_candidate_graph.py::CandidateGraphResult.reader_trace`, pronoun/lookback trace appends in `parse_and_solve` | Carries JSON-encoded trace events for reader phases and elimination/refusal causes. | **Subject-general.** Traceability/replay evidence is central to every future lane. |
## Cross-subject leverage map
### Strong transfer primitives
These are the highest-leverage primitives for new subjects because they are not inherently arithmetic:
1. **Evidence-carrying candidate objects** — anchors: `Quantity`, `Step`, `CandidateInitial`, `CandidateOperation`, `CandidateUnknown`. Cross-subject use: claims, propositions, logical premises, reading-comprehension facts, geometry givens.
2. **Candidate emission separated from admissibility** — anchors: `extract_*_candidates`, `_initial_admissible`, `_question_admissible`, `roundtrip_admissible`, `self_verifies`. Cross-subject use: emit possible readings, then require grounding/type/consistency before commitment.
3. **Refusal-first ambiguity handling** — anchors: `select_self_verified`, `resolve_pooled`, `parse_and_solve` decision rule. Cross-subject use: when multiple interpretations remain, refuse instead of choosing.
4. **Scope/referent guards** — anchors: `segment_clauses`, `compose_sequential` clause-local guard, `continues_anchor_referent`, `_resolve_pronoun_entity`, lookback ambiguity defense. Cross-subject use: reading comprehension, narrative state tracking, logic variable binding.
5. **Question/target extraction** — anchors: `extract_target`, `extract_question_candidates`, conditional prefix stripping, capacity/earnings/conditional question extractors. Cross-subject use: target-frame parsing is the obvious shared bottleneck across math, logic, reading, and measurement.
6. **Completeness and distractor classification** — anchors: `_unused_quantities`, `classify_derivation`, exempt readings, context classifier. Cross-subject use: all subjects need "account for all relevant evidence" without forcing irrelevant distractors into the committed answer.
7. **Promotion boundaries** — anchors: `resolve_promotable_product`, sealed injectors, ratified registry fallback. Cross-subject use: experimental readers can exist without becoming served behavior.
### Math-specific primitives with reusable analogues
| Math-specific primitive | Why math-specific | Reusable analogue |
|---|---|---|
| Unit consistency | Depends on dimensional arithmetic rules. | Type consistency / sort checking. |
| Product/sum chain enumeration | Depends on arithmetic operator semantics. | Bounded proof/action sequence enumeration. |
| Comparative scalar multiplication | Numeric scalar operation. | Relation-strength or predicate-transform facts from closed packs. |
| Capacity/earnings rate short-circuits | Rate arithmetic over time/currency. | Matched statement-target frame with deterministic transformation. |
| Repeated-unit product hazard | Dimensional impossibility. | Domain-type impossibility detector. |
| Money/currency normalization | Numeric unit system. | Canonical symbol/entity normalization. |
## Observed composition wall
The current substrate already has many individually strong primitives. The bottleneck is not lack of primitives; it is safe composition among them:
- Clause-local reasoning exists, but cross-clause reasoning remains guarded and narrow.
- Question target extraction exists, but many target frames still require closed shape support.
- Referent continuation exists, but pronoun/coreference resolution is intentionally conservative.
- Candidate pooling exists, but promotion to serving requires narrow target/hazard gates.
- Completeness is strong, but it can over-force distractors unless exempt/disagreement paths are present.
This confirms the brief's framing: the next-subject work should exercise the same composition primitives without creating live serving risk.
## What transfers to other subjects
- **Reading comprehension should reuse the most math-relevant primitives immediately:** clause segmentation, referent guards, pronoun ambiguity refusal, target-frame parsing, completeness, and branch disagreement are already the exact pain points behind the math composition wall.
- **Symbolic/deductive logic can reuse the candidate/admissibility/disagreement architecture:** premises become evidence-bearing candidates, inference rules become cue- or schema-licensed steps, and ambiguous proof branches refuse rather than commit.
- **Measurement/geometry can reuse the most math-specific substrate with low conceptual impedance:** quantity extraction, unit canonicalization, unit/type consistency, target-unit matching, rate/measurement frames, and dimensional impossibility checks are already close to that domain.
- **All future subjects should preserve the sealed/promotion boundary pattern:** draft readers and recognizers can be explored only as proposal-only or sealed lanes until the Claude lane verifies the relevant invariants.
- **The highest cross-subject ROI is not a new corpus first; it is a small capability-axis spec that stresses target extraction, referent binding, completeness, and disagreement without weakening `wrong=0`.**
## Open questions for the Claude lane
1. Verify whether any functions above are currently serving-active vs sealed/practice-only on `main`; this read-only pass did not run lane-sha checks or tests.
2. Confirm the exact current serving count and wrong/refusal distribution through the pinned eval lane before using this document as planning evidence.
3. Decide whether Task B should treat `product_bridge.resolve_promotable_product` as part of the active question layer or as a promotion boundary around the derivation reader.
4. Inspect coverage for the "most-grounded-slots-wins" tiebreaker before reusing that pattern in any new subject; it is powerful but could mask ambiguity if applied too broadly.
5. For Task C, compare candidate subject ordering against the actual contents of `evals/symbolic_logic/` and `evals/math_capability_axes/` before drafting any subject-specific axes.
---
## Claude-lane verification (landed)
Verified against `main` at commit `3e29559` by reading the committed serving report and source. Method note: the inventory above was authored read-only; the checks below resolve its five open questions. The full `core test`/MLX/Rust suite was **not** re-run in this lane (Apple-Silicon/MLX substrate unavailable here); the serving metric cited is the committed, pinned report — the authoritative source of truth for the frozen serving path — not a fresh run.
**Definition-of-done check (Task A):** all 66 primitives resolve to real files on `main`. Every referenced module exists (`generate/derivation/{model,extract,clauses,comparatives,search,multistep,target,compose,accumulate,pool,product_bridge,verify}.py`, `generate/derivation/state/{bind,change}.py`, `generate/{math_candidate_parser,math_candidate_graph,recognizer_anchor_inject}.py`). No invented APIs found.
### Q1 — serving-active vs sealed
- `_SEALED_INJECTORS = {}` is **empty** on `main`. Nothing is currently sealed. Inventory row #64 describes a real mechanism, but it is presently inert — so "sealed lane" is not what is suppressing any current behavior.
- `discrete_count_statement` is **serving-active**: it is wired directly into the live dispatch map (`ShapeCategory.DISCRETE_COUNT_STATEMENT: inject_discrete_count_statement`). Its empty injections (see Q2) are genuine conservatism in the active injector, not sealing.
- The frozen-serving gate (`scripts/verify_lane_shas.py`) pins the **SHA-256 of report outputs** for 8 eval lanes (reviewer_registry, miner_loop_closure, curriculum_loop_closure, domain_contract_validation, fabrication_control, demo_composition, public_demo, math_teaching_corpus). It freezes serving by making any drift in those outputs detectable; it does not pin a static list of serving source files.
### Q2 — exact serving distribution (CONFIRMED)
Pinned report `evals/gsm8k_math/train_sample/v1/report.json` (ADR-0126, sample_count=50):
- **6 correct / 44 refused / 0 wrong.** `wrong=0` holds. `exit_criterion.correct_min=10``passed: false`.
The 44 non-correct cases decompose as:
| Failure mode | Count |
|---|---:|
| Recognizer matched but produced **no injection** | 32 |
| **No admissible candidate** (parser emitted nothing usable) | 12 |
Locus of the 44: statement (recognizer) 32 · statement (parser) 7 · question (parser) 5.
Recognizer-fired-but-empty-injection (32) by category:
| Category | Count |
|---|---:|
| `discrete_count_statement` | 18 |
| `descriptive_setup_no_quantity` | 4 |
| `rate_with_currency` | 3 |
| `multiplicative_aggregation` | 3 |
| `currency_amount` | 3 |
| `temporal_aggregation` | 1 |
**Headline:** the single largest refusal bucket is `discrete_count_statement`**18 of 44 (41%)** — where the serving-active recognizer fires on a count-like token but the injector returns empty. **This marks *where* the composition wall surfaces; it is not a lever to widen.** As the corrected Net-read below establishes, all 18 are 24 capability compositions the injector *correctly* declines (emitting an initial-state there is metric-inert). The concentration is diagnostic — the most common surface form of the wall — not a backlog item, and it touches the entity/initial-state primitives (#40, #63) only as evidence that the wall sits *downstream* of extraction, in composition.
### Q3 — `product_bridge.resolve_promotable_product` classification (RESOLVED)
It is part of the **active serving question layer, behaving as a promotion boundary around the derivation reader** — both, not either/or. Its module docstring places it on "the serving candidate-graph path," and it returns a "serving-safe product resolution" only after passing `_has_hazard_surface` and `_has_product_target`. Recommendation for Task B: treat it as the guarded gate by which derivation-reader products reach serving, i.e. a promotion boundary that is itself live — not a sealed/practice-only reader.
### Q4 — "most-grounded-slots-wins" tiebreaker coverage (CAUTION CONFIRMED, scope corrected)
`_collapse_per_sentence_ties` / `_slot_count` are invoked at two serving sites in `parse_and_solve` (lines 958, 999). No test references those functions **by name** (no white-box test). However — correcting an earlier overstatement in this appendix — the collapse **is** behaviorally covered on the happy path: `tests/test_math_candidate_graph.py::TestAmbiguityResolution::test_gives_with_target_resolves_to_transfer` exercises the slot-count collapse ("Sam gives 3 apples to Tom" → transfer reading wins on more grounded slots) and would fail if the collapse broke. The accurate, narrower gap is therefore: happy-path collapse is covered; what is **missing** is (a) a white-box test naming the functions and (b) an **adversarial "high-slot-but-wrong vs low-slot-but-right"** case — the scenario where "more slots = better" selects the wrong reading. Recommendation: add both before reusing this pattern in any new subject.
### Q5 — Task C input (DEFERRED to Task C execution)
Not resolved here; Task C explicitly requires comparing candidate subject ordering against the live contents of `evals/symbolic_logic/` and `evals/math_capability_axes/`. Flagged for the Task C pass so it is not double-counted as Task A scope.
### Net read for planning (corrected)
An earlier version of this section recommended widening the serving-active `discrete_count_statement` injector as "the highest-count, lowest-risk math lever (18/44)." **That conclusion was wrong and is retracted.** Reading all 18 of those cases in full shows they are **24 capability compositions** (ratio chains 0020/0029/0033, multi-step rate/percent 0032/0034/0044, accumulate-against-target 0037/0039, and 0040 which needs per-entity attribute lookup before any arithmetic). The recognizer fires on the first count token ("2 horses"); the injector **correctly declines** because the surrounding problem is not a bare count. Emitting an initial-state there is **metric-inert** — the graph still cannot compose to the answer. The 18/44 concentration is the **composition wall surfacing at the most common recognizer category**, not an injector to widen. This is reinforced by **ADR-0174 (Proposed)**, which deprecates the per-category injector dispatch table as the runtime admission path (injectors become hypothesis-emitters in a held-hypothesis reader), and by the wrong=0 hazard of that surface (case-0050 canary on the same serving path).
Corrected steer: primitives are not the bottleneck; **safe composition is.** The honest next lever is a **composition capability** over the existing grounded primitives — multi-quantity chains (ratio, multi-step rate/percent, accumulate-against-target). The direct GSM8K-metric lever is **ADR-0174's held-hypothesis reader (Proposed)**; the adjacent proof-DAG substrate — binding-graph acyclicity, proof-graph builder, modus-ponens disagreement — is already **Accepted** (ADR-0203/0204/0205, proof_chain phase 2.12.3). So the work is composition through the held-hypothesis reader on an accepted proof substrate, **not** category-dispatch widening. For Task B: group all 44 and rank by **composition-arity** (1-capability gaps = tractable; 24-capability compositions = the wall), not by raw recognizer-category count.

View file

@ -0,0 +1,133 @@
<!-- FILE: docs/analysis/question-layer-gap-survey.md -->
# Question-Layer Gap Survey
Status: Proposed analysis draft. No serving behavior is changed.
Source of truth for the metric is `docs/claims_ledger.md` and
`evals/gsm8k_math/train_sample/v1/report.json`: the current real
`train_sample` result is 6 correct / 44 refused / 0 wrong. This survey assigns
each of the 44 refused case ids to exactly one current failure group. It does
not claim that any group would pass if widened; it names the layer that refuses
today and estimates whether the missing capability is single-step or
composition-bound.
The requested Task A file, `docs/analysis/comprehension-primitive-inventory.md`,
was not present in this worktree. I used the embedded older handoff only as
orientation and grounded the grouping below in the report plus the live parser /
recognizer code.
## Code Map
Current refusal topology:
- `generate/math_candidate_graph.py` first extracts statement choices, then
consults the ratified recognizer registry when a numeric statement has no
parser candidate. If recognition succeeds but injection yields no typed
solver primitive, it now refuses explicitly instead of dropping the statement.
- `generate/recognizer_anchor_inject.py` is still a category dispatch surface.
Only `discrete_count_statement` and a narrow `multiplicative_aggregation`
entry can emit today; other categories return empty and therefore become
explicit refusals.
- `generate/math_candidate_parser.py` admits a closed question grammar:
total-across `How many ... do they have ...`, existential aggregate
`How many ... are there ...`, entity possession, activity `did`, and three
ADR-0163.D.4 patterns. If these emit no `CandidateUnknown`, the graph refuses
before branch enumeration.
- ADR-0174 explicitly deprecates the per-category injector dispatch table as
the long-term runtime admission path. Injector widening should therefore be
treated as stopgap or hypothesis-emitter work, not the primary strategic
direction.
## Assignment Table
Tractability scoring is case-level and arity-aware:
- High: one closed primitive or one local parser frame is plausibly missing.
- Medium: one local frame/schema is clear, but downstream composition is still
likely needed.
- Low: the current refusal is an early stop in a 2-4 capability derivation.
| Group | Count | Case ids | Current failing layer | Tractability | Representative report reason excerpts | Interpretation |
|---|---:|---|---|---|---|---|
| DCS high-arity composition wall | 18 | 0002, 0015, 0016, 0020, 0029, 0031, 0032, 0033, 0034, 0036, 0037, 0038, 0039, 0040, 0041, 0044, 0047, 0049 | Matcher -> injector -> composition | Low | `candidate_graph: recognizer matched but produced no injection for statement: 'She splits it up into 25-foot sections.' (category=discrete_count_statement)`; `candidate_graph: recognizer matched but produced no injection for statement: 'Malcolm is trying to find the fastest walk to school and is currently comparing two routes.' (category=discrete_count_statement)` | The recognizer often fires on a count-like token, but the actual derivation needs division, rest-state, route comparison, percent/rate, chained comparisons, target residuals, or per-entity attributes. Widening the discrete-count injector alone is metric-inert and risks incomplete readings. |
| Missing inverse/residual/comparative question frames | 5 | 0007, 0008, 0009, 0025, 0035 | Question parser / admissibility | Low | `candidate_graph: no admissible candidate for question: 'How many more boxes do they need if Francine has a total of 85 crayons?'`; `candidate_graph: no admissible candidate for question: 'How many more apples would Martha need to give away to be left with only 4 of them?'` | These are not just new surface phrasings for `Unknown(entity, unit)`. They ask for a missing operand, inverse relation, target residual, or conditional total. The question layer must bind the requested slot to a derivation, not merely extract a unit. |
| Rate, currency-rate, and tariff statements | 4 | 0001, 0011, 0017, 0022 | Recognizer -> injector/schema | Medium | `candidate_graph: recognizer matched but produced no injection for statement: 'Tina makes $18.00 an hour.' (category=rate_with_currency)`; `candidate_graph: recognizer matched but produced no injection for statement: 'Hes charging $50.00 per day or $500.00 for 14 days.' (category=temporal_aggregation)` | The local schema gap is visible: `rate_with_currency` and `temporal_aggregation` need typed rate/tariff hypotheses. Case-level admission still needs overtime, profit, historical+today aggregation, or piecewise tariff composition. |
| Non-quantitative relation category used as an early stop | 4 | 0012, 0023, 0027, 0046 | Matcher -> injector | Low | `candidate_graph: recognizer matched but produced no injection for statement: 'He put all of them in his aquarium but his fish ate half of them.' (category=descriptive_setup_no_quantity)`; `candidate_graph: recognizer matched but produced no injection for statement: 'Half of the students are girls, the other half are boys.' (category=descriptive_setup_no_quantity)` | The category name is accurate for the injector: it cannot emit a concrete primitive from the matched surface. The cases need fraction-of-prior, combined-total binding, partition, and percentage-of-subgroup reasoning. |
| Multiplicative aggregate beyond the narrow emitted shapes | 3 | 0006, 0013, 0045 | Recognizer -> injector / composition registry | Medium | `candidate_graph: recognizer matched but produced no injection for statement: 'Mandy started reading books with only 8 pages when she was 6 years old.' (category=multiplicative_aggregation)`; `candidate_graph: recognizer matched but produced no injection for statement: 'Each survey has 10 questions.' (category=multiplicative_aggregation)` | There is already a narrow product injector, but these rows need time/age chains, month segmentation, doubled rates, or survey-count composition. Single product emission may help local state, but full cases still depend on multi-step composition. |
| Financial currency amount / percent mutation statements | 3 | 0019, 0028, 0043 | Recognizer -> injector / mutation schema | Low | `candidate_graph: recognizer matched but produced no injection for statement: 'After the first appointment, John paid $100 for pet insurance that covers 80% of the subsequent visits.' (category=currency_amount)`; `candidate_graph: recognizer matched but produced no injection for statement: 'Her mother gave her an additional $4, and her father twice as much as her mother.' (category=currency_amount)` | The surface contains currency, but the required reading is coverage after first event, percent daily operating cost, revenue target, or comparative gift amount. A `CandidateInitial` currency emission would be an incomplete graph. |
| Fractional relational statements with no parser candidate | 3 | 0004, 0005, 0010 | Statement parser / admissibility | Low | `candidate_graph: no admissible candidate for statement: 'Half of the kids are going to soccer camp, and 1/4 of the kids going to soccer camp are going to soccer camp in the morning.'`; `candidate_graph: no admissible candidate for statement: 'Marion has 1/4 more than what Yun currently has, plus 7.'` | The parser has fraction literals and comparative operations, but these surfaces are relational fractions over prior or unknown quantities. They require held equations or derivation nodes, not a flat possession/operation candidate. |
| Duration and recurrence statement frames | 3 | 0030, 0048, 0050 | Statement parser / temporal composition | Medium | `candidate_graph: no admissible candidate for statement: 'It is a 2-hour drive each way.'`; `candidate_graph: no admissible candidate for statement: 'Mark does a gig every other day for 2 weeks.'` | These expose bounded duration-multiplier or temporal-frequency frames. Each frame is local and deterministic, but the cases still need composition across trip stages, weekly deltas to target, or per-event song duration totals. |
| Relational conjoined-subject each initial | 1 | 0026 | Statement parser / entity binding | Medium | `candidate_graph: no admissible candidate for statement: 'Aaron and his brother Carson each saved up $40 to go to dinner.'` | The parser has an `each` extractor for two named subjects, but this sentence uses a possessive relational subject (`his brother Carson`) and a purpose tail. The local parse is probably narrower than the concept; the full case remains multi-step because bill fraction and shared scoop count follow. |
Total assigned: 18 + 5 + 4 + 4 + 3 + 3 + 3 + 3 + 1 = 44.
## Backlog Interpretation
The audited partition is the stable artifact. A single `count x tractability`
sort is misleading here because count ranges from 1 to 18 while tractability is
coarse; it would place the known composition wall at the top as if it were an
incremental injector work order. In this survey, count is impact evidence, not
the sort key for near-term changes.
### Composition-Bound Work
These groups should feed the ADR-0174 held-hypothesis / derivation-composer
scope, not category-specific injector widening. They need multi-clause state,
referent binding, ratio/fraction relations, target-slot questions, or
event-scope composition before any answer can be safe.
| Group | Count | How to use the count |
|---|---:|---|
| DCS high-arity composition wall | 18 | Main evidence that discrete-count recognition is surfacing a composition wall, not an injector backlog. |
| Missing inverse/residual/comparative question frames | 5 | Question-layer evidence that unknowns must bind to derivation slots, not just noun units. |
| Rate, currency-rate, and tariff statements | 4 | Rate/tariff hypotheses are useful only if downstream overtime, profit, history, or piecewise composition can refuse partial readings. |
| Non-quantitative relation category used as an early stop | 4 | Evidence for relation/partition composition and for avoiding hard-stop loss of later structure. |
| Multiplicative aggregate beyond the narrow emitted shapes | 3 | Product-like anchors need day/month, age-chain, or survey-count composition before admission. |
| Financial currency amount / percent mutation statements | 3 | Currency is not enough; these need percent coverage, operating-cost, or spend/residual mutation. |
| Fractional relational statements with no parser candidate | 3 | Needs relational fraction/equation hypotheses, not flat possession candidates. |
| Duration and recurrence statement frames | 3 | Duration-multiplier and recurrence frames are local signals whose answers require temporal composition. |
### Bounded Near-Term Fixes
These are smaller, lower-blast-radius probes that may be useful as executable
follow-ups or regression tests. They should still preserve refusal-first
admission and should not be represented as the main path to the metric.
| Candidate fix | Case ids | Why bounded |
|---|---|---|
| Relational conjoined-subject `each` binding | 0026 | One row exposes a local entity-binding gap around `Aaron and his brother Carson each ...`; useful as a narrow parser/binding probe even though the full problem remains multi-step. |
| Descriptive/no-quantity early-stop handling | 0012, 0023, 0027, 0046 | The bounded mechanism is to avoid losing later structure when a descriptive relation cannot emit state; the cases behind it still require composition, so this is not a direct answer path. |
| Single-slot question-frame probes | subset of 0007, 0008, 0035 | A few question surfaces can become narrow probes for residual or divisor target-slot binding. They must be held as unknown slots and refused unless the derivation is complete. |
## Layer Notes
The high-count `discrete_count_statement` bucket should not be read as "add more
discrete-count injectors." The report is saying the matcher saw something
count-shaped before the engine had a safe composed reading. In the live code,
recognized-but-uninjected statements refuse explicitly because dropping them
would permit incomplete graphs. ADR-0174 points in the same direction:
injectors should become hypothesis emitters inside a held-hypothesis reader,
where branch disagreement, constraint propagation, and completeness can reject
partial readings.
The five question refusals are genuinely question-layer refusals in the narrow
code sense: `extract_question_candidates` emits no `CandidateUnknown`. But all
five ask for an inverse or residual target. A wider regex that only extracts
the noun would not identify the unknown slot and would still be unsafe under
`wrong = 0`.
## Open Questions for the Claude Lane
- Confirm whether `docs/analysis/comprehension-primitive-inventory.md` was meant
to be landed in this worktree or only supplied out-of-band; this draft could
be amended to cite its exact primitive table once present.
- Run an instrumented read of the 44 refused cases to distinguish "injector
returned empty because no parsed anchors existed" from "anchors existed but
constraint propagation eliminated them."
- For the five question refusals, collect the intended `Unknown` slot shape
without using gold answers: missing operand, residual-to-target, inverse
divisor, or aggregate total.
- Decide whether the next executable lane should prototype rate/tariff
hypotheses or question-target slots first; both are safer as held hypotheses
than as direct category admissions.

View file

@ -0,0 +1,148 @@
<!-- FILE: docs/analysis/solver-operation-coverage.md -->
# Solver Operation Coverage Audit
Status: Proposed analysis draft. No serving behavior is changed. This is a
read-only structural audit to de-risk ADR-0174 Phase 5b / Phase-4-style
composition work. Verdicts below are code-reading conclusions only; they must be
verified in the Claude lane with executable solver/binding-graph cases before
any promotion claim.
## Scope
Read surfaces:
- `generate/math_problem_graph.py`
- `generate/math_solver.py`
- `generate/binding_graph/{model,adapter,admissibility,allocation,question_target,units}.py`
- ADR-0116, ADR-0117, ADR-0132, ADR-0133, ADR-0134, ADR-0135
- Skimmed ADR-0174 Phase 5b and ADR-0203/0204/0205 to confirm current
composition/proof-DAG framing.
Important correction to the relay: the current solver/graph vocabulary is not
only `{add, subtract, transfer, multiply, divide}`. `MathProblemGraph` and
`math_solver` both include the eight operation kinds:
```text
add, subtract, transfer, multiply, divide,
apply_rate, compare_additive, compare_multiplicative
```
ADR-0174 Phase 5b states the same: the solver is already waiting for these
operations; the gap is reader -> injector -> `Operation` front-end wiring plus
composition.
## Existing Operation Substrate
| Surface | Evidence | Consequence |
|---|---|---|
| Closed operation vocabulary | `generate/math_problem_graph.py` defines `VALID_OPERATION_KINDS` with eight kinds. | New arithmetic verbs are not needed for ordinary multiply/divide/rate/comparison chains. |
| Pack-bound solver dispatch | `generate/math_solver.py` maps all eight kinds to `en_arithmetic_v1` lemmas before solving. | Missing pack lemmas fail loudly; no hidden operation fallback. |
| Stateful solver semantics | `_apply` mutates `(actor, unit)` terminal state for `add`, `subtract`, `transfer`, `multiply`, `divide`; `_apply_rate` produces numerator-unit state from denominator-unit state; comparisons derive an actor state from a reference actor. | The solver is good at forward state trajectories, but weak at keeping multiple same-unit derived intermediates alive under one actor. |
| Unknown shape | `Unknown(entity, unit)` resolves either terminal state for one entity or total-across all entities with that unit. | Target questions that ask for a missing operand, number of iterations, or intermediate state are not represented by `Unknown` alone. |
| Binding-graph equation/data model | `BoundEquation(operation_kind=...)`, `BoundUnknown(question_form=...)`, semantic roles include `rate`, `duration`, `difference`, `ratio`. | The graph can name richer forms than the solver's final `Unknown`, but current adapter still comes from existing `MathProblemGraph` operation chains. |
| Unit admissibility | `check_admissibility` covers additive, multiplicative, divide, `apply_rate`, and comparison kinds. | Dimensional proofs exist for the current eight operation kinds; new node types would need explicit admissibility rules. |
| Question-target binding | `infer_question_form` recognizes `ratio`, `difference`, `rate`, `total`, and `count` from operation kinds touching the unknown. | It can label answer form, but it does not solve inverse targets or select intermediate operation indices yet. |
| Acyclicity | ADR-0203 adds `circular_dependency` refusal to the shared binding-graph constructor. | Any new equation/intermediate-node extension must remain a DAG, not a cyclic algebra system hidden inside the graph. |
## Phase-4 Target Chains
| Target chain | Verdict | Existing operations that can carry it | What is missing / risk |
|---|---|---|---|
| Multi-step rate-sum | Expressible via composition for straight-line rate applications and sums. Piecewise tariffs/conditionals need scoped selection before admission. | `apply_rate` computes `X/Y * Y -> X`; `add`/`subtract` can aggregate generated same-unit totals. Binding admissibility has an `apply_rate` rule requiring one rate dep plus one duration/count dep. | The reader must emit the base duration/count quantities, rate hypotheses, and sum operations without clobbering unrelated same-unit state. Piecewise tariffs such as "$50/day or $500/14 days" need a tariff/choice scope or explicit branch-disagreement gate; that is a binding/composition problem, not a missing arithmetic verb. |
| Ratio chain | Expressible via composition for forward ratio chains. Inverse ratio equations need a target/equation extension. | `compare_multiplicative` supports "actor = factor * reference"; `multiply`/`divide` support scalar transformations; `infer_question_form` maps touching `compare_multiplicative` to `ratio`. | Forward chains like `A`, `B = 2A`, `C = 1/2(A+B)` can be represented if the reader emits the right reference actors and order. Inverse forms such as "ducks are 10 more than 4x chickens; ducks = 150; find total birds" require solving for an unknown reference, not just applying a forward operation. That needs equation/target solving or a new binding node shape, not a new `ratio` operation kind. |
| Accumulate-against-target | Needs new binding-graph target/equation capability; not expressible as a fixed existing operation chain in the current solver. | `add`, `subtract`, `multiply`, `divide`, and `apply_rate` can express the arithmetic once the iteration count or missing operand is known. | The current solver consumes a fully specified graph in source order, then resolves terminal `Unknown(entity, unit)`. It cannot represent "after how many weeks", "how many cups to sell to reach profit", or "how long to make back cost" as an unknown operand/iteration count. This needs a first-class target-slot/equation node or bounded inverse solver with refusal/disagreement, plus proof that no cyclic dependency is introduced. |
| Percent/fraction mutation | Mostly expressible via composition for forward mutations; needs intermediate-symbol/scope support for same-unit derived amounts and event subsets. | Percent/fraction values can be scalar `multiply`/`divide` factors, with `add`/`subtract` for mutation and `compare_multiplicative` for relative quantities. Binding admissibility already covers multiply/divide dimensions. | Some cases are safe forward mutations ("eat 75% of a pan", "lose half"). Others require original and derived same-unit quantities to coexist, e.g. principal plus interest, insured vs uninsured portions, operating cost as percent of startup cost. The current solver overwrites `(actor, unit)` for multiply/divide and `apply_rate`, so these need derived intermediate symbols/event scopes or separate binding nodes. A new `percent` operation kind is not structurally necessary; a new scoped intermediate/equation representation may be. |
## Structural Verdict
Phase 4/5b is mostly **not** blocked by missing arithmetic operation kinds. The
current operation vocabulary already covers the primitive arithmetic field:
addition/subtraction/transfer, scalar multiply/divide, rate application, and
additive/multiplicative comparison.
The real scope risk is representation:
- Can the reader emit a chain of typed operations from scattered clauses while
preserving all grounded quantities?
- Can the graph retain intermediate derived quantities when they share the same
actor/unit as their source?
- Can a question bind to a missing operand, iteration count, or intermediate
state instead of only terminal `Unknown(entity, unit)`?
- Can inverse/equation targets be solved under a disagreement rule and the
ADR-0203 acyclicity invariant?
That makes the likely build a **binding-target / intermediate-symbol /
derivation-composer extension**, not a broad new solver primitive pack.
## Chain-Specific Notes
### Multi-step rate-sum
`apply_rate` is first-class: the solver's `_apply_rate` reads the actor's
denominator-unit state and writes numerator-unit state. The binding-graph
adapter synthesizes a rate symbol with composite unit `<num>_per_<denom>`, and
admissibility checks that the denominator cancels. Therefore simple rate-sum is
expressible as:
```text
duration/count fact -> apply_rate -> produced total
produced totals -> add/subtract -> final total
```
The unsafe part is not the operation. It is branch selection and scope:
overtime, tariffs, and "including today" require deciding which rate applies to
which event subset.
### Ratio chain
`compare_multiplicative` already gives a forward ratio operation. It refuses
when the reference actor has no quantity or multiple ambiguous units, which is
the right wrong=0 boundary. Binding target can label ratio-form questions.
The gap is inverse ratio. Current `Operation` is directional: it mutates the
actor from a known reference. If the reference is the unknown and the actor is
given, the solver has no inverse-equation mode.
### Accumulate-against-target
This is the clearest "needs new graph shape" target. A terminal state solver can
answer "what is the total after N weeks"; it cannot answer "what N reaches total
T" unless N is already present as a quantity and a divide operation has been
materialized by the reader. A safe implementation needs a target slot that names
the missing operand/iteration count and an admissibility/disagreement rule for
the inverse derivation.
### Percent/fraction mutation
Percent/fraction does not need a new arithmetic verb. It needs:
- scalar extraction (`75% -> 0.75`, `1/4 -> 0.25`);
- complement derivation when the text says "covers 80%" but the cost asks for
the uncovered part;
- event/subset scoping ("after the first appointment", "subsequent visits");
- intermediate symbols when original principal/cost and derived interest/cost
must both remain available.
The binding graph is the natural home for these intermediates because it already
has `BoundEquation`, dependency sets, unit proofs, question forms, and
acyclicity checks.
## Open Questions for the Claude Lane
- Does `apply_rate` overwriting `(actor, numerator_unit)` create a concrete
hazard in rate-sum cases where the actor already holds money? If yes, the
build needs derived result symbols before promotion.
- Should inverse target solving be introduced as a new `BoundUnknown` form, a
new `BoundEquation` operation kind, or a separate proof/derivation rule over
existing equations?
- Can forward ratio chains be admitted through `compare_multiplicative` without
widening the parser's reference-actor ambiguity beyond the existing refusal
discipline?
- What is the smallest executable probe set that distinguishes "new operation
kind required" from "same operation, new intermediate symbol required" for
percent/fraction mutations?
- When proof-DAG consumers and math binding graphs share
`SemanticSymbolicBindingGraph`, should proposition-specific operation kinds
remain isolated from math admissibility, or should the admissibility dispatcher
split into math/proof entrypoints before more equation kinds are added?

View file

@ -304,6 +304,17 @@ def _build_initial_from_discrete_count(
if value is None:
return None
# A surface like "Jerry has 3 times as many apples", "3 times more
# apples", or "3 times the apples" is not an initial possession of
# "3 times"; it is an incomplete comparative-multiplicative clause.
# Letting this through as an initial consumes the scalar token and
# defeats the ADR-0191 completeness guard. Refuse here until a real
# compare_multiplicative operation can be emitted.
if counted_noun.lower() == "times" and _count_token_followed_by_times(
sentence, count_token
):
return None
# CandidateInitial requires an anchor verb token recognized in its
# post-init whitelist (has/have/had/owns/owned/holds/held/contains/
# contained — matched by the recognizer's narrowness rule). We pick
@ -433,6 +444,26 @@ def _locate_token(sentence: str, target_lc: str) -> str | None:
return None
def _count_token_followed_by_times(sentence: str, count_token: str) -> bool:
"""True when the count surface is immediately followed by ``times``.
The discrete-count recognizer can otherwise misread comparative
multiplier surfaces as an initial possession of ``<N> times``. This
check intentionally sits at the injector boundary: it only suppresses
the malformed initial candidate and does not create any new
admitting path.
"""
target = count_token.lower()
tokens = [
raw.strip(".,;:!?\"'()[]{}").lower()
for raw in sentence.split()
]
for i, tok in enumerate(tokens[:-1]):
if tok == target and tokens[i + 1] == "times":
return True
return False
def _resolve_count_value(count_token: str, count_kind: str) -> int | None:
"""Map ``count_token`` to a numeric value.

View file

@ -89,3 +89,64 @@ def test_guard_is_refusal_only_not_answer_changing() -> None:
# Same value, same unit-bearing graph — guard does not mutate solving.
assert res.answer == 438.0
assert res.selected_graph is not None
@pytest.mark.parametrize(
("factor", "comparative"),
[
("2", "as many"),
("3", "as many"),
("5", "as many"),
("two", "as many"),
("three", "as many"),
("five", "as many"),
("3", "more"),
("three", "more"),
("3", "the number of"),
("3", "the"),
],
)
def test_n_times_without_reference_refuses(
factor: str, comparative: str
) -> None:
"""A dropped comparative multiplier must not admit as an initial
possession of ``<N> times``.
This is the Phase-5b hard negative: if future emission work makes
comparative-multiplicative surfaces reach the graph, the gate must
refuse incomplete ``<N> times ...`` clauses instead of consuming
the scalar as an ordinary count.
"""
res = parse_and_solve(
"Tom has 7 apples. "
f"Jerry has {factor} times {comparative} apples. "
"How many apples do they have together?"
)
assert res.answer is None
def test_n_times_as_many_with_reference_still_solves() -> None:
"""The guard only blocks incomplete comparative clauses; a fully
referenced compare_multiplicative graph still solves."""
res = parse_and_solve(
"Tom has 7 apples. Jerry has 3 times as many apples as Tom. "
"How many apples do they have together?"
)
assert res.answer == 28.0
@pytest.mark.parametrize(
"question",
[
"Tom has 7 apples. Jerry has twice as many apples. "
"How many apples do they have together?",
"Tom has 7 apples. Jerry has double the apples. "
"How many apples do they have together?",
"Ivan has 20 dice. Jerry has twice as many dice as Ivan. "
"How many dice do they have altogether?",
],
)
def test_existing_multiplier_refusals_stay_refused(question: str) -> None:
"""Existing safe multiplier refusals must not become admissions."""
res = parse_and_solve(question)
assert res.answer is None