From e1bcdf6286b6b9671af628eb9233d077632bccb3 Mon Sep 17 00:00:00 2001 From: Shay Date: Wed, 3 Jun 2026 15:22:42 -0700 Subject: [PATCH 1/2] Harden no-reference n-times comparative guard --- HANDOFF-gpt55-2026-06-03.md | 47 ++++++ docs/analysis/question-layer-gap-survey.md | 133 ++++++++++++++++ docs/analysis/solver-operation-coverage.md | 148 ++++++++++++++++++ generate/recognizer_anchor_inject.py | 31 ++++ ...test_candidate_graph_completeness_guard.py | 61 ++++++++ 5 files changed, 420 insertions(+) create mode 100644 HANDOFF-gpt55-2026-06-03.md create mode 100644 docs/analysis/question-layer-gap-survey.md create mode 100644 docs/analysis/solver-operation-coverage.md diff --git a/HANDOFF-gpt55-2026-06-03.md b/HANDOFF-gpt55-2026-06-03.md new file mode 100644 index 00000000..dbdf2ffb --- /dev/null +++ b/HANDOFF-gpt55-2026-06-03.md @@ -0,0 +1,47 @@ +# HANDOFF gpt55 2026-06-03 + +Branch: `codex/ntimes-completeness-guard` + +Final commit SHA: reported in the thread final after commit creation. + +## File Status + +| file | path | status | purpose | ready-to-merge | +|---|---|---|---|---| +| recognizer anchor injector | `generate/recognizer_anchor_inject.py` | FINAL | Refusal-only guard preventing no-reference ` times ...` comparative multipliers from being injected as `CandidateInitial(value=N, unit="times")`. | Y | +| completeness guard tests | `tests/test_candidate_graph_completeness_guard.py` | FINAL | Pins the §9 hard negative matrix, with solve/refusal controls. | Y | +| question-layer gap survey | `docs/analysis/question-layer-gap-survey.md` | FINAL | Canonical audited 44-refusal partition and backlog interpretation. | Y | +| solver operation coverage | `docs/analysis/solver-operation-coverage.md` | FINAL | Canonical read-only audit of existing solver op coverage and representation gaps. | Y | +| composition capability scope | `docs/analysis/composition-capability-scope.md` | SUPERSEDED | Superseded by the execution lane canonical copy; intentionally not committed here. | N | +| handoff | `HANDOFF-gpt55-2026-06-03.md` | FINAL | Merge/readiness manifest for this thread. | Y | + +## Validation + +Completed on branch `codex/ntimes-completeness-guard`: + +- `uv run python -m pytest tests/test_candidate_graph_completeness_guard.py -q` + - `21 passed` +- `uv run python -m pytest tests/test_adr_0131_G2_comparatives.py tests/test_adr_0131_G2a_comparative_verb_widening.py -q` + - `30 passed` +- `uv run python -m pytest tests/test_aggregate_total_question_forms.py tests/test_discrete_count_open_noun_class.py tests/test_adr_0163_d2_discrete_count_injection.py -q` + - `59 passed` +- Train-sample probe through `generate.math_candidate_graph.parse_and_solve` + - `6 correct / 44 refused / 0 wrong` + - `refused_nonzero_count = 0` + - admitted IDs: `0003`, `0014`, `0018`, `0021`, `0024`, `0042` + +## New Since Last Message + +- Broadened the §9 guard from phrase-specific `times as many|more` matching to the structural captured shape: cardinal immediately followed by `times` inside the discrete-count injector. +- Added no-reference hard negatives for `3 times the number of apples` and `3 times the apples`. +- Added explicit safe-refusal controls for no-reference `twice`, no-reference `double`, and case 605. +- Removed the local superseded `composition-capability-scope.md` from this branch's merge set. +- Added this handoff manifest. + +## Not Done / WIP + +- No 5b emission/representation slice was implemented. +- No <=20-case validation sub-corpus was authored. +- No solver operation kinds or binding-graph node types were added. +- No serving/eval/claims-ledger files were changed. +- The superseded composition scope is not part of this commit; use the execution lane canonical copy instead. diff --git a/docs/analysis/question-layer-gap-survey.md b/docs/analysis/question-layer-gap-survey.md new file mode 100644 index 00000000..2f37e4d5 --- /dev/null +++ b/docs/analysis/question-layer-gap-survey.md @@ -0,0 +1,133 @@ + + +# Question-Layer Gap Survey + +Status: Proposed analysis draft. No serving behavior is changed. + +Source of truth for the metric is `docs/claims_ledger.md` and +`evals/gsm8k_math/train_sample/v1/report.json`: the current real +`train_sample` result is 6 correct / 44 refused / 0 wrong. This survey assigns +each of the 44 refused case ids to exactly one current failure group. It does +not claim that any group would pass if widened; it names the layer that refuses +today and estimates whether the missing capability is single-step or +composition-bound. + +The requested Task A file, `docs/analysis/comprehension-primitive-inventory.md`, +was not present in this worktree. I used the embedded older handoff only as +orientation and grounded the grouping below in the report plus the live parser / +recognizer code. + +## Code Map + +Current refusal topology: + +- `generate/math_candidate_graph.py` first extracts statement choices, then + consults the ratified recognizer registry when a numeric statement has no + parser candidate. If recognition succeeds but injection yields no typed + solver primitive, it now refuses explicitly instead of dropping the statement. +- `generate/recognizer_anchor_inject.py` is still a category dispatch surface. + Only `discrete_count_statement` and a narrow `multiplicative_aggregation` + entry can emit today; other categories return empty and therefore become + explicit refusals. +- `generate/math_candidate_parser.py` admits a closed question grammar: + total-across `How many ... do they have ...`, existential aggregate + `How many ... are there ...`, entity possession, activity `did`, and three + ADR-0163.D.4 patterns. If these emit no `CandidateUnknown`, the graph refuses + before branch enumeration. +- ADR-0174 explicitly deprecates the per-category injector dispatch table as + the long-term runtime admission path. Injector widening should therefore be + treated as stopgap or hypothesis-emitter work, not the primary strategic + direction. + +## Assignment Table + +Tractability scoring is case-level and arity-aware: + +- High: one closed primitive or one local parser frame is plausibly missing. +- Medium: one local frame/schema is clear, but downstream composition is still + likely needed. +- Low: the current refusal is an early stop in a 2-4 capability derivation. + +| Group | Count | Case ids | Current failing layer | Tractability | Representative report reason excerpts | Interpretation | +|---|---:|---|---|---|---|---| +| DCS high-arity composition wall | 18 | 0002, 0015, 0016, 0020, 0029, 0031, 0032, 0033, 0034, 0036, 0037, 0038, 0039, 0040, 0041, 0044, 0047, 0049 | Matcher -> injector -> composition | Low | `candidate_graph: recognizer matched but produced no injection for statement: 'She splits it up into 25-foot sections.' (category=discrete_count_statement)`; `candidate_graph: recognizer matched but produced no injection for statement: 'Malcolm is trying to find the fastest walk to school and is currently comparing two routes.' (category=discrete_count_statement)` | The recognizer often fires on a count-like token, but the actual derivation needs division, rest-state, route comparison, percent/rate, chained comparisons, target residuals, or per-entity attributes. Widening the discrete-count injector alone is metric-inert and risks incomplete readings. | +| Missing inverse/residual/comparative question frames | 5 | 0007, 0008, 0009, 0025, 0035 | Question parser / admissibility | Low | `candidate_graph: no admissible candidate for question: 'How many more boxes do they need if Francine has a total of 85 crayons?'`; `candidate_graph: no admissible candidate for question: 'How many more apples would Martha need to give away to be left with only 4 of them?'` | These are not just new surface phrasings for `Unknown(entity, unit)`. They ask for a missing operand, inverse relation, target residual, or conditional total. The question layer must bind the requested slot to a derivation, not merely extract a unit. | +| Rate, currency-rate, and tariff statements | 4 | 0001, 0011, 0017, 0022 | Recognizer -> injector/schema | Medium | `candidate_graph: recognizer matched but produced no injection for statement: 'Tina makes $18.00 an hour.' (category=rate_with_currency)`; `candidate_graph: recognizer matched but produced no injection for statement: 'He’s charging $50.00 per day or $500.00 for 14 days.' (category=temporal_aggregation)` | The local schema gap is visible: `rate_with_currency` and `temporal_aggregation` need typed rate/tariff hypotheses. Case-level admission still needs overtime, profit, historical+today aggregation, or piecewise tariff composition. | +| Non-quantitative relation category used as an early stop | 4 | 0012, 0023, 0027, 0046 | Matcher -> injector | Low | `candidate_graph: recognizer matched but produced no injection for statement: 'He put all of them in his aquarium but his fish ate half of them.' (category=descriptive_setup_no_quantity)`; `candidate_graph: recognizer matched but produced no injection for statement: 'Half of the students are girls, the other half are boys.' (category=descriptive_setup_no_quantity)` | The category name is accurate for the injector: it cannot emit a concrete primitive from the matched surface. The cases need fraction-of-prior, combined-total binding, partition, and percentage-of-subgroup reasoning. | +| Multiplicative aggregate beyond the narrow emitted shapes | 3 | 0006, 0013, 0045 | Recognizer -> injector / composition registry | Medium | `candidate_graph: recognizer matched but produced no injection for statement: 'Mandy started reading books with only 8 pages when she was 6 years old.' (category=multiplicative_aggregation)`; `candidate_graph: recognizer matched but produced no injection for statement: 'Each survey has 10 questions.' (category=multiplicative_aggregation)` | There is already a narrow product injector, but these rows need time/age chains, month segmentation, doubled rates, or survey-count composition. Single product emission may help local state, but full cases still depend on multi-step composition. | +| Financial currency amount / percent mutation statements | 3 | 0019, 0028, 0043 | Recognizer -> injector / mutation schema | Low | `candidate_graph: recognizer matched but produced no injection for statement: 'After the first appointment, John paid $100 for pet insurance that covers 80% of the subsequent visits.' (category=currency_amount)`; `candidate_graph: recognizer matched but produced no injection for statement: 'Her mother gave her an additional $4, and her father twice as much as her mother.' (category=currency_amount)` | The surface contains currency, but the required reading is coverage after first event, percent daily operating cost, revenue target, or comparative gift amount. A `CandidateInitial` currency emission would be an incomplete graph. | +| Fractional relational statements with no parser candidate | 3 | 0004, 0005, 0010 | Statement parser / admissibility | Low | `candidate_graph: no admissible candidate for statement: 'Half of the kids are going to soccer camp, and 1/4 of the kids going to soccer camp are going to soccer camp in the morning.'`; `candidate_graph: no admissible candidate for statement: 'Marion has 1/4 more than what Yun currently has, plus 7.'` | The parser has fraction literals and comparative operations, but these surfaces are relational fractions over prior or unknown quantities. They require held equations or derivation nodes, not a flat possession/operation candidate. | +| Duration and recurrence statement frames | 3 | 0030, 0048, 0050 | Statement parser / temporal composition | Medium | `candidate_graph: no admissible candidate for statement: 'It is a 2-hour drive each way.'`; `candidate_graph: no admissible candidate for statement: 'Mark does a gig every other day for 2 weeks.'` | These expose bounded duration-multiplier or temporal-frequency frames. Each frame is local and deterministic, but the cases still need composition across trip stages, weekly deltas to target, or per-event song duration totals. | +| Relational conjoined-subject each initial | 1 | 0026 | Statement parser / entity binding | Medium | `candidate_graph: no admissible candidate for statement: 'Aaron and his brother Carson each saved up $40 to go to dinner.'` | The parser has an `each` extractor for two named subjects, but this sentence uses a possessive relational subject (`his brother Carson`) and a purpose tail. The local parse is probably narrower than the concept; the full case remains multi-step because bill fraction and shared scoop count follow. | + +Total assigned: 18 + 5 + 4 + 4 + 3 + 3 + 3 + 3 + 1 = 44. + +## Backlog Interpretation + +The audited partition is the stable artifact. A single `count x tractability` +sort is misleading here because count ranges from 1 to 18 while tractability is +coarse; it would place the known composition wall at the top as if it were an +incremental injector work order. In this survey, count is impact evidence, not +the sort key for near-term changes. + +### Composition-Bound Work + +These groups should feed the ADR-0174 held-hypothesis / derivation-composer +scope, not category-specific injector widening. They need multi-clause state, +referent binding, ratio/fraction relations, target-slot questions, or +event-scope composition before any answer can be safe. + +| Group | Count | How to use the count | +|---|---:|---| +| DCS high-arity composition wall | 18 | Main evidence that discrete-count recognition is surfacing a composition wall, not an injector backlog. | +| Missing inverse/residual/comparative question frames | 5 | Question-layer evidence that unknowns must bind to derivation slots, not just noun units. | +| Rate, currency-rate, and tariff statements | 4 | Rate/tariff hypotheses are useful only if downstream overtime, profit, history, or piecewise composition can refuse partial readings. | +| Non-quantitative relation category used as an early stop | 4 | Evidence for relation/partition composition and for avoiding hard-stop loss of later structure. | +| Multiplicative aggregate beyond the narrow emitted shapes | 3 | Product-like anchors need day/month, age-chain, or survey-count composition before admission. | +| Financial currency amount / percent mutation statements | 3 | Currency is not enough; these need percent coverage, operating-cost, or spend/residual mutation. | +| Fractional relational statements with no parser candidate | 3 | Needs relational fraction/equation hypotheses, not flat possession candidates. | +| Duration and recurrence statement frames | 3 | Duration-multiplier and recurrence frames are local signals whose answers require temporal composition. | + +### Bounded Near-Term Fixes + +These are smaller, lower-blast-radius probes that may be useful as executable +follow-ups or regression tests. They should still preserve refusal-first +admission and should not be represented as the main path to the metric. + +| Candidate fix | Case ids | Why bounded | +|---|---|---| +| Relational conjoined-subject `each` binding | 0026 | One row exposes a local entity-binding gap around `Aaron and his brother Carson each ...`; useful as a narrow parser/binding probe even though the full problem remains multi-step. | +| Descriptive/no-quantity early-stop handling | 0012, 0023, 0027, 0046 | The bounded mechanism is to avoid losing later structure when a descriptive relation cannot emit state; the cases behind it still require composition, so this is not a direct answer path. | +| Single-slot question-frame probes | subset of 0007, 0008, 0035 | A few question surfaces can become narrow probes for residual or divisor target-slot binding. They must be held as unknown slots and refused unless the derivation is complete. | + +## Layer Notes + +The high-count `discrete_count_statement` bucket should not be read as "add more +discrete-count injectors." The report is saying the matcher saw something +count-shaped before the engine had a safe composed reading. In the live code, +recognized-but-uninjected statements refuse explicitly because dropping them +would permit incomplete graphs. ADR-0174 points in the same direction: +injectors should become hypothesis emitters inside a held-hypothesis reader, +where branch disagreement, constraint propagation, and completeness can reject +partial readings. + +The five question refusals are genuinely question-layer refusals in the narrow +code sense: `extract_question_candidates` emits no `CandidateUnknown`. But all +five ask for an inverse or residual target. A wider regex that only extracts +the noun would not identify the unknown slot and would still be unsafe under +`wrong = 0`. + +## Open Questions for the Claude Lane + +- Confirm whether `docs/analysis/comprehension-primitive-inventory.md` was meant + to be landed in this worktree or only supplied out-of-band; this draft could + be amended to cite its exact primitive table once present. +- Run an instrumented read of the 44 refused cases to distinguish "injector + returned empty because no parsed anchors existed" from "anchors existed but + constraint propagation eliminated them." +- For the five question refusals, collect the intended `Unknown` slot shape + without using gold answers: missing operand, residual-to-target, inverse + divisor, or aggregate total. +- Decide whether the next executable lane should prototype rate/tariff + hypotheses or question-target slots first; both are safer as held hypotheses + than as direct category admissions. diff --git a/docs/analysis/solver-operation-coverage.md b/docs/analysis/solver-operation-coverage.md new file mode 100644 index 00000000..95197b3b --- /dev/null +++ b/docs/analysis/solver-operation-coverage.md @@ -0,0 +1,148 @@ + + +# Solver Operation Coverage Audit + +Status: Proposed analysis draft. No serving behavior is changed. This is a +read-only structural audit to de-risk ADR-0174 Phase 5b / Phase-4-style +composition work. Verdicts below are code-reading conclusions only; they must be +verified in the Claude lane with executable solver/binding-graph cases before +any promotion claim. + +## Scope + +Read surfaces: + +- `generate/math_problem_graph.py` +- `generate/math_solver.py` +- `generate/binding_graph/{model,adapter,admissibility,allocation,question_target,units}.py` +- ADR-0116, ADR-0117, ADR-0132, ADR-0133, ADR-0134, ADR-0135 +- Skimmed ADR-0174 Phase 5b and ADR-0203/0204/0205 to confirm current + composition/proof-DAG framing. + +Important correction to the relay: the current solver/graph vocabulary is not +only `{add, subtract, transfer, multiply, divide}`. `MathProblemGraph` and +`math_solver` both include the eight operation kinds: + +```text +add, subtract, transfer, multiply, divide, +apply_rate, compare_additive, compare_multiplicative +``` + +ADR-0174 Phase 5b states the same: the solver is already waiting for these +operations; the gap is reader -> injector -> `Operation` front-end wiring plus +composition. + +## Existing Operation Substrate + +| Surface | Evidence | Consequence | +|---|---|---| +| Closed operation vocabulary | `generate/math_problem_graph.py` defines `VALID_OPERATION_KINDS` with eight kinds. | New arithmetic verbs are not needed for ordinary multiply/divide/rate/comparison chains. | +| Pack-bound solver dispatch | `generate/math_solver.py` maps all eight kinds to `en_arithmetic_v1` lemmas before solving. | Missing pack lemmas fail loudly; no hidden operation fallback. | +| Stateful solver semantics | `_apply` mutates `(actor, unit)` terminal state for `add`, `subtract`, `transfer`, `multiply`, `divide`; `_apply_rate` produces numerator-unit state from denominator-unit state; comparisons derive an actor state from a reference actor. | The solver is good at forward state trajectories, but weak at keeping multiple same-unit derived intermediates alive under one actor. | +| Unknown shape | `Unknown(entity, unit)` resolves either terminal state for one entity or total-across all entities with that unit. | Target questions that ask for a missing operand, number of iterations, or intermediate state are not represented by `Unknown` alone. | +| Binding-graph equation/data model | `BoundEquation(operation_kind=...)`, `BoundUnknown(question_form=...)`, semantic roles include `rate`, `duration`, `difference`, `ratio`. | The graph can name richer forms than the solver's final `Unknown`, but current adapter still comes from existing `MathProblemGraph` operation chains. | +| Unit admissibility | `check_admissibility` covers additive, multiplicative, divide, `apply_rate`, and comparison kinds. | Dimensional proofs exist for the current eight operation kinds; new node types would need explicit admissibility rules. | +| Question-target binding | `infer_question_form` recognizes `ratio`, `difference`, `rate`, `total`, and `count` from operation kinds touching the unknown. | It can label answer form, but it does not solve inverse targets or select intermediate operation indices yet. | +| Acyclicity | ADR-0203 adds `circular_dependency` refusal to the shared binding-graph constructor. | Any new equation/intermediate-node extension must remain a DAG, not a cyclic algebra system hidden inside the graph. | + +## Phase-4 Target Chains + +| Target chain | Verdict | Existing operations that can carry it | What is missing / risk | +|---|---|---|---| +| Multi-step rate-sum | Expressible via composition for straight-line rate applications and sums. Piecewise tariffs/conditionals need scoped selection before admission. | `apply_rate` computes `X/Y * Y -> X`; `add`/`subtract` can aggregate generated same-unit totals. Binding admissibility has an `apply_rate` rule requiring one rate dep plus one duration/count dep. | The reader must emit the base duration/count quantities, rate hypotheses, and sum operations without clobbering unrelated same-unit state. Piecewise tariffs such as "$50/day or $500/14 days" need a tariff/choice scope or explicit branch-disagreement gate; that is a binding/composition problem, not a missing arithmetic verb. | +| Ratio chain | Expressible via composition for forward ratio chains. Inverse ratio equations need a target/equation extension. | `compare_multiplicative` supports "actor = factor * reference"; `multiply`/`divide` support scalar transformations; `infer_question_form` maps touching `compare_multiplicative` to `ratio`. | Forward chains like `A`, `B = 2A`, `C = 1/2(A+B)` can be represented if the reader emits the right reference actors and order. Inverse forms such as "ducks are 10 more than 4x chickens; ducks = 150; find total birds" require solving for an unknown reference, not just applying a forward operation. That needs equation/target solving or a new binding node shape, not a new `ratio` operation kind. | +| Accumulate-against-target | Needs new binding-graph target/equation capability; not expressible as a fixed existing operation chain in the current solver. | `add`, `subtract`, `multiply`, `divide`, and `apply_rate` can express the arithmetic once the iteration count or missing operand is known. | The current solver consumes a fully specified graph in source order, then resolves terminal `Unknown(entity, unit)`. It cannot represent "after how many weeks", "how many cups to sell to reach profit", or "how long to make back cost" as an unknown operand/iteration count. This needs a first-class target-slot/equation node or bounded inverse solver with refusal/disagreement, plus proof that no cyclic dependency is introduced. | +| Percent/fraction mutation | Mostly expressible via composition for forward mutations; needs intermediate-symbol/scope support for same-unit derived amounts and event subsets. | Percent/fraction values can be scalar `multiply`/`divide` factors, with `add`/`subtract` for mutation and `compare_multiplicative` for relative quantities. Binding admissibility already covers multiply/divide dimensions. | Some cases are safe forward mutations ("eat 75% of a pan", "lose half"). Others require original and derived same-unit quantities to coexist, e.g. principal plus interest, insured vs uninsured portions, operating cost as percent of startup cost. The current solver overwrites `(actor, unit)` for multiply/divide and `apply_rate`, so these need derived intermediate symbols/event scopes or separate binding nodes. A new `percent` operation kind is not structurally necessary; a new scoped intermediate/equation representation may be. | + +## Structural Verdict + +Phase 4/5b is mostly **not** blocked by missing arithmetic operation kinds. The +current operation vocabulary already covers the primitive arithmetic field: +addition/subtraction/transfer, scalar multiply/divide, rate application, and +additive/multiplicative comparison. + +The real scope risk is representation: + +- Can the reader emit a chain of typed operations from scattered clauses while + preserving all grounded quantities? +- Can the graph retain intermediate derived quantities when they share the same + actor/unit as their source? +- Can a question bind to a missing operand, iteration count, or intermediate + state instead of only terminal `Unknown(entity, unit)`? +- Can inverse/equation targets be solved under a disagreement rule and the + ADR-0203 acyclicity invariant? + +That makes the likely build a **binding-target / intermediate-symbol / +derivation-composer extension**, not a broad new solver primitive pack. + +## Chain-Specific Notes + +### Multi-step rate-sum + +`apply_rate` is first-class: the solver's `_apply_rate` reads the actor's +denominator-unit state and writes numerator-unit state. The binding-graph +adapter synthesizes a rate symbol with composite unit `_per_`, and +admissibility checks that the denominator cancels. Therefore simple rate-sum is +expressible as: + +```text +duration/count fact -> apply_rate -> produced total +produced totals -> add/subtract -> final total +``` + +The unsafe part is not the operation. It is branch selection and scope: +overtime, tariffs, and "including today" require deciding which rate applies to +which event subset. + +### Ratio chain + +`compare_multiplicative` already gives a forward ratio operation. It refuses +when the reference actor has no quantity or multiple ambiguous units, which is +the right wrong=0 boundary. Binding target can label ratio-form questions. + +The gap is inverse ratio. Current `Operation` is directional: it mutates the +actor from a known reference. If the reference is the unknown and the actor is +given, the solver has no inverse-equation mode. + +### Accumulate-against-target + +This is the clearest "needs new graph shape" target. A terminal state solver can +answer "what is the total after N weeks"; it cannot answer "what N reaches total +T" unless N is already present as a quantity and a divide operation has been +materialized by the reader. A safe implementation needs a target slot that names +the missing operand/iteration count and an admissibility/disagreement rule for +the inverse derivation. + +### Percent/fraction mutation + +Percent/fraction does not need a new arithmetic verb. It needs: + +- scalar extraction (`75% -> 0.75`, `1/4 -> 0.25`); +- complement derivation when the text says "covers 80%" but the cost asks for + the uncovered part; +- event/subset scoping ("after the first appointment", "subsequent visits"); +- intermediate symbols when original principal/cost and derived interest/cost + must both remain available. + +The binding graph is the natural home for these intermediates because it already +has `BoundEquation`, dependency sets, unit proofs, question forms, and +acyclicity checks. + +## Open Questions for the Claude Lane + +- Does `apply_rate` overwriting `(actor, numerator_unit)` create a concrete + hazard in rate-sum cases where the actor already holds money? If yes, the + build needs derived result symbols before promotion. +- Should inverse target solving be introduced as a new `BoundUnknown` form, a + new `BoundEquation` operation kind, or a separate proof/derivation rule over + existing equations? +- Can forward ratio chains be admitted through `compare_multiplicative` without + widening the parser's reference-actor ambiguity beyond the existing refusal + discipline? +- What is the smallest executable probe set that distinguishes "new operation + kind required" from "same operation, new intermediate symbol required" for + percent/fraction mutations? +- When proof-DAG consumers and math binding graphs share + `SemanticSymbolicBindingGraph`, should proposition-specific operation kinds + remain isolated from math admissibility, or should the admissibility dispatcher + split into math/proof entrypoints before more equation kinds are added? diff --git a/generate/recognizer_anchor_inject.py b/generate/recognizer_anchor_inject.py index 698e9e00..63faab1a 100644 --- a/generate/recognizer_anchor_inject.py +++ b/generate/recognizer_anchor_inject.py @@ -304,6 +304,17 @@ def _build_initial_from_discrete_count( if value is None: return None + # A surface like "Jerry has 3 times as many apples", "3 times more + # apples", or "3 times the apples" is not an initial possession of + # "3 times"; it is an incomplete comparative-multiplicative clause. + # Letting this through as an initial consumes the scalar token and + # defeats the ADR-0191 completeness guard. Refuse here until a real + # compare_multiplicative operation can be emitted. + if counted_noun.lower() == "times" and _count_token_followed_by_times( + sentence, count_token + ): + return None + # CandidateInitial requires an anchor verb token recognized in its # post-init whitelist (has/have/had/owns/owned/holds/held/contains/ # contained — matched by the recognizer's narrowness rule). We pick @@ -433,6 +444,26 @@ def _locate_token(sentence: str, target_lc: str) -> str | None: return None +def _count_token_followed_by_times(sentence: str, count_token: str) -> bool: + """True when the count surface is immediately followed by ``times``. + + The discrete-count recognizer can otherwise misread comparative + multiplier surfaces as an initial possession of `` times``. This + check intentionally sits at the injector boundary: it only suppresses + the malformed initial candidate and does not create any new + admitting path. + """ + target = count_token.lower() + tokens = [ + raw.strip(".,;:!?\"'()[]{}").lower() + for raw in sentence.split() + ] + for i, tok in enumerate(tokens[:-1]): + if tok == target and tokens[i + 1] == "times": + return True + return False + + def _resolve_count_value(count_token: str, count_kind: str) -> int | None: """Map ``count_token`` to a numeric value. diff --git a/tests/test_candidate_graph_completeness_guard.py b/tests/test_candidate_graph_completeness_guard.py index 8207f973..b9fcd8f8 100644 --- a/tests/test_candidate_graph_completeness_guard.py +++ b/tests/test_candidate_graph_completeness_guard.py @@ -89,3 +89,64 @@ def test_guard_is_refusal_only_not_answer_changing() -> None: # Same value, same unit-bearing graph — guard does not mutate solving. assert res.answer == 438.0 assert res.selected_graph is not None + + +@pytest.mark.parametrize( + ("factor", "comparative"), + [ + ("2", "as many"), + ("3", "as many"), + ("5", "as many"), + ("two", "as many"), + ("three", "as many"), + ("five", "as many"), + ("3", "more"), + ("three", "more"), + ("3", "the number of"), + ("3", "the"), + ], +) +def test_n_times_without_reference_refuses( + factor: str, comparative: str +) -> None: + """A dropped comparative multiplier must not admit as an initial + possession of `` times``. + + This is the Phase-5b hard negative: if future emission work makes + comparative-multiplicative surfaces reach the graph, the gate must + refuse incomplete `` times ...`` clauses instead of consuming + the scalar as an ordinary count. + """ + res = parse_and_solve( + "Tom has 7 apples. " + f"Jerry has {factor} times {comparative} apples. " + "How many apples do they have together?" + ) + assert res.answer is None + + +def test_n_times_as_many_with_reference_still_solves() -> None: + """The guard only blocks incomplete comparative clauses; a fully + referenced compare_multiplicative graph still solves.""" + res = parse_and_solve( + "Tom has 7 apples. Jerry has 3 times as many apples as Tom. " + "How many apples do they have together?" + ) + assert res.answer == 28.0 + + +@pytest.mark.parametrize( + "question", + [ + "Tom has 7 apples. Jerry has twice as many apples. " + "How many apples do they have together?", + "Tom has 7 apples. Jerry has double the apples. " + "How many apples do they have together?", + "Ivan has 20 dice. Jerry has twice as many dice as Ivan. " + "How many dice do they have altogether?", + ], +) +def test_existing_multiplier_refusals_stay_refused(question: str) -> None: + """Existing safe multiplier refusals must not become admissions.""" + res = parse_and_solve(question) + assert res.answer is None From 018181c95b9a153e40bb17fd049b4cdda619511e Mon Sep 17 00:00:00 2001 From: Shay Date: Wed, 3 Jun 2026 15:35:58 -0700 Subject: [PATCH 2/2] Add canonical composition analysis docs --- HANDOFF-gpt55-2026-06-03.md | 8 +- docs/analysis/composition-capability-scope.md | 244 ++++++++++++++++++ .../comprehension-primitive-inventory.md | 211 +++++++++++++++ 3 files changed, 460 insertions(+), 3 deletions(-) create mode 100644 docs/analysis/composition-capability-scope.md create mode 100644 docs/analysis/comprehension-primitive-inventory.md diff --git a/HANDOFF-gpt55-2026-06-03.md b/HANDOFF-gpt55-2026-06-03.md index dbdf2ffb..ca5a143a 100644 --- a/HANDOFF-gpt55-2026-06-03.md +++ b/HANDOFF-gpt55-2026-06-03.md @@ -12,7 +12,8 @@ Final commit SHA: reported in the thread final after commit creation. | completeness guard tests | `tests/test_candidate_graph_completeness_guard.py` | FINAL | Pins the §9 hard negative matrix, with solve/refusal controls. | Y | | question-layer gap survey | `docs/analysis/question-layer-gap-survey.md` | FINAL | Canonical audited 44-refusal partition and backlog interpretation. | Y | | solver operation coverage | `docs/analysis/solver-operation-coverage.md` | FINAL | Canonical read-only audit of existing solver op coverage and representation gaps. | Y | -| composition capability scope | `docs/analysis/composition-capability-scope.md` | SUPERSEDED | Superseded by the execution lane canonical copy; intentionally not committed here. | N | +| composition capability scope | `docs/analysis/composition-capability-scope.md` | FINAL | Canonical execution-lane v2 scope for ADR-0174 Phase 5b emission/representation and §9 guard precondition. | Y | +| comprehension primitive inventory | `docs/analysis/comprehension-primitive-inventory.md` | FINAL | Canonical consolidated inventory and cross-subject leverage map from the execution lane. | Y | | handoff | `HANDOFF-gpt55-2026-06-03.md` | FINAL | Merge/readiness manifest for this thread. | Y | ## Validation @@ -35,7 +36,8 @@ Completed on branch `codex/ntimes-completeness-guard`: - Broadened the §9 guard from phrase-specific `times as many|more` matching to the structural captured shape: cardinal immediately followed by `times` inside the discrete-count injector. - Added no-reference hard negatives for `3 times the number of apples` and `3 times the apples`. - Added explicit safe-refusal controls for no-reference `twice`, no-reference `double`, and case 605. -- Removed the local superseded `composition-capability-scope.md` from this branch's merge set. +- Added canonical execution-lane `composition-capability-scope.md`. +- Added canonical execution-lane `comprehension-primitive-inventory.md`. - Added this handoff manifest. ## Not Done / WIP @@ -44,4 +46,4 @@ Completed on branch `codex/ntimes-completeness-guard`: - No <=20-case validation sub-corpus was authored. - No solver operation kinds or binding-graph node types were added. - No serving/eval/claims-ledger files were changed. -- The superseded composition scope is not part of this commit; use the execution lane canonical copy instead. +- Superseded `0001` / `0002` / `0003` patch files from the Opus handoff directory were not committed. diff --git a/docs/analysis/composition-capability-scope.md b/docs/analysis/composition-capability-scope.md new file mode 100644 index 00000000..77c77d52 --- /dev/null +++ b/docs/analysis/composition-capability-scope.md @@ -0,0 +1,244 @@ + + +# Composition-Capability Scope — v2 (re-anchored to shipped reality) + +Status: Proposed analysis / build-scoping. No serving or eval edits. Source of +truth: `docs/claims_ledger.md`, `evals/gsm8k_math/train_sample/v1/report.json`, +ADR-0174 **read in full (644 lines, incl. Implementation Notes + amended Phase 5, +lines 300–533)**. Every empirical claim below reproduced live against `main` @ `3e29559`. + +> **v1 correction.** v1 aimed the plan at "land Phase 1; the unlanded value is +> Phase 3/4." That was wrong — scoped from ADR-0174's forward sections only, +> missing the ~225 lines documenting that **Phases 1–4 already shipped and are +> wired into serving**. Verified on `main`: +> `generate/comprehension/{state,lookback,contemplate,constraint_propagation}.py` +> exist with `HYPOTHESIS_CAP`/`open_hypotheses`/`Hypothesis`/`UnknownHeld`/ +> `reevaluate`/`contemplate`, and `reevaluate`+`contemplate` are imported and +> invoked inside `generate/math_candidate_graph.py::parse_and_solve`. Metric still +> **6/44/0**. v2 re-anchors to the ADR's own amended Phase 5. + +## 0. The finding that scopes everything (corrected) + +The held-hypothesis machinery (ADR-0174 Phases 1–4) is **live in serving and the +metric did not move.** A live `parse_and_solve` audit of all 44 refusals (§1) +locates the wall precisely: **44/44 refuse at `branches_enumerated = 0` — upstream +of the solver, which is never reached.** So the lift is **not** more reader +machinery, and **not** solver operation coverage (the 8 op-kinds are never reached; +see §1). The wall is **emission/representation**: the recognizer/parser/binding +layer cannot construct an admissible multi-step candidate for these shapes, so +nothing is fed to the (already-capable) solver. ADR-0174's "removing the 3–5 +narrowness layers per case" is that emission/representation work. + +This inverts v1's thesis *and* supersedes the intermediate "operation coverage" +thesis: the composition wall is at **emission/representation upstream of the +solver**, not at the operation layer. + +## 1. Verified current state on `main` + +| Layer | State | +|---|---| +| Held-hypothesis reader (P1–P4: state, constraint-propagation, lookback, contemplate) | **Shipped, wired into serving** candidate-graph | +| `HYPOTHESIS_CAP`, vault>packs>audit precedence | already set/enforced (0174 OQ#1/#3 moot) | +| `lifecycle.py` GSM8K-scoring dispatch | **inert** (admits 0/50) — retirable in 5a | +| `lifecycle.py` / `audit.py` reader surface | **load-bearing** for the ADR-0172 teaching corridor — **keep** | +| Solver operation kinds | **8 already exist** (verified by discovery): `add`, `subtract`, `transfer`, `multiply`, `divide`, `apply_rate`, `compare_additive`, `compare_multiplicative`. Missing arithmetic op kinds is **not** the blocker. | +| **Live audit of all 44 refusals (ran `parse_and_solve` on `main`)** | **44/44 refuse at `branches_enumerated = 0`** — i.e. *upstream of the solver*. No branch is built, so the 8 operation kinds are **never reached**. `rate`/`ratio` also exist as first-class `SEMANTIC_ROLES`/`QUESTION_FORMS` (`binding_graph/model.py:42–65`); only `percent`/`accumulate` lack a named op-kind — moot while nothing reaches the solver. | +| train_sample metric | **6 / 44 / 0** (`correct_min=10` not yet passed) | + +## 2. The actual remaining work (ADR-0174 amended Phase 5) + +### Phase 5a — retire the inert parallel parser (structural, ~0 lift) +Retire the GSM8K-scoring-only inert dispatch (`_try_comprehension_reader` / +`_try_reader_for_question`) + adapter + `use_reader` plumbing. Net **−1,038 LOC** +(code + tests). **Keep** `lifecycle.py`/`audit.py` — their reader *refusals* feed +the ADR-0172 math-contemplation teaching corridor (`teaching/math_*`, +`evals/flywheel_demo`, `core/cli.py`). Low-risk; single-path serving. +Acceptance: 6/44/0 byte-identical, capability-axis lanes 100% `wrong=0`, +pinned-SHAs pass. + +### Phase 5b — emission / representation buildout (semantic — the real lift) +"This is where `correct` climbs toward 25. It is not a refactor." **Verified +diagnosis (live `parse_and_solve` audit of all 44, on `main`):** every refusal +occurs at `branches_enumerated = 0` — *upstream of the solver*. The 8 operation +kinds are never reached. So 5b is **not** an operation-execution problem and +**not** chiefly a new-primitive problem. It is an **emission/representation** +problem: the recognizer/parser/binding layer cannot construct an admissible +multi-step candidate for these shapes, so nothing is fed to the (already-capable) +solver. + +The 44 split into two emission failure modes (verified, matches `report.json`): +- **32 "recognizer matched but produced no injection"** — an anchor fired but the + hypothesis-emitter declined to construct a candidate (cannot represent the + multi-clause / derived / scoped structure). +- **12 "no admissible candidate"** — the parser produced nothing admissible. + +**Sequencing implication:** fix emission/representation first; the +`percent`/`accumulate` primitive question is **secondary** — answerable only after +cases actually reach the solver. This *lowers* the wrong=0 risk: emitting existing +ops behind the same admissibility predicates is lower-risk than new solver math. + +**5b first PR (smallest verified-tractable slice):** pick the emission failure mode +with the highest count whose representation is bounded — and prove it reaches branch +enumeration on ≥1 case without breaking `wrong=0`. **But the §9 completeness-gate +precondition lands before any emission PR.** + +## 3. Test anchors (reusable, relabeled to 5a/5b) +1. **Serving gate (`gsm8k_math`):** 5a byte-identical 6/44/0; 5b climbs + (`correct ≥ 10` clears the ADR-0126 exit, target → 15 → 25), `wrong = 0` at + every step. +2. **Regression nets (100% `wrong=0`):** G1–G5, S1; `anti_regression`; case-0050 pin. +3. **Determinism:** `trace_hash` over `open_hypotheses`; replay-equivalence. +4. **Serving freeze:** `verify_lane_shas.py` passes each step. +5. **Still-needed new asset (0174 OQ#5):** the curated ≤20-case multi-step + validation sub-corpus — **unbuilt, still required for 5b measurement.** The §9 + hard negatives are the first deposits into it. Author before 5b measurement. + +## 4. Corrected sequencing +0. **Precondition (§9):** harden the comparative-multiplicative completeness tripwire so a dropped `×` clause refuses (as `twice` does). 5b emission on these shapes is unsafe until this lands. Driver test is FINAL and RED on `main` (see §9). +1. **5b gating analysis — done (§8):** the **emission/representation** audit ran and the 32 no-injection cases are sub-classified by representation. Highest-count bounded gap = **R1 derived/intermediate symbol (24/44)**. +2. **Author the ≤20-case multi-step validation sub-corpus** (test anchor for 5b). +3. **Phase 5a** (optional, can run in parallel — structural, low-risk). +4. **Phase 5b** — build the **emitter/representation** for R1 (derived-symbol) on the §8 near-pure exemplars (prove ≥1 case reaches `branches_enumerated > 0` and admits), each behind §3 gates **and the §9 precondition**. Not new operations — the 8 exist and are unreached. + +(Do **not** "land Phase 1" — shipped. Do **not** treat P3/P4 as the lever — shipped, metric flat.) + +## 5. Relation to cross-subject testing +5b maturing the math `DomainSolver`'s emission/representation is still the precondition +for wiring symbolic_logic as arena #2; the held-hypothesis reader + disagreement +gate remain the shared, arena-portable primitives. `cross_domain_transfer` / +`monotonic_learning` (both exist, with `contract.md`/`holdouts/`) become live tests +once arena #2 exists. + +## 6. Honest ceiling (corrected) +The ≥15 climb is a **5b** outcome (emission/representation), **not** a P4 outcome — P4 +shipped and we are at 6. Cases needing world knowledge (0040 legs) or representations 5b +does not add stay refused = `wrong = 0` holding, not failure. + +## 7. Open questions for the build lane +1. **Answered by the live audit:** the binding constraint is emission upstream of + the solver (44/44 at `branches_enumerated=0`), not operation execution. Remaining: + rank the §8 R-classes by the size of the bounded-representation slice per class. +2. Of the 44, which are emission-fixable with existing ops vs (a) genuinely need a + `percent`/`accumulate` primitive *after* emission, vs (b) world-knowledge/out-of- + scope permanent refusals (e.g. 0040 legs-per-animal). The audit gives the upstream + verdict; this split needs per-case reader-trace reads. +3. 5a pre-flight: confirm the exact inert-dispatch LOC and that no teaching-corridor + consumer breaks when the GSM8K-scoring dispatch is retired. + +--- + +## 8. Verified per-case representation classification (live audit, all 44) +Structural reading of each of the 44 (what representation the emitter must build for +emission to succeed), grounded in the case text + the live `parse_and_solve` locus. +Multi-tagged; a case usually needs several. Frequency across the 44: + +| Representation gap | # cases (multi-tagged) | +|---|---:| +| R5 — multi-step rate/duration/scalar | 27 | +| R1 — derived/intermediate symbol | 24 | +| R6 — percent/fraction mutation (no op-kind) | 18 | +| R4 — accumulation/residual | 10 | +| R2 — inverse target | 6 | +| R3 — subset/partition scope | 3 | +| R7 — world-knowledge (permanent refusal) | 1 | + +**Highest-leverage gap: R1 (derived/intermediate symbol)** — 24/44 need to compute an +intermediate quantity and reuse it downstream. Its lowest-arity exemplars are nearly +pure R1: 0027 (Twitter = half of IG+FB), 0008 (total beads → ÷ per bracelet), +0029 (keyboard = 3× mouse → sum), 0038 (×3 → sum). Contrastive proof it is the emission +gap (parseable aggregate question form): `"Nicole has 400 cards. Cindy has 800 cards. +How many cards do they have together?"` **admits (1200)**; `"…Cindy has twice as many +cards. How many cards do they have together?"` reaches `branches_enumerated=1` but +**refuses** (completeness: scalar `2.0` unconsumed). The stated-sum reaches the solver; +the derived form does not. R7 (0040 legs-per-animal) is a permanent refusal — world +knowledge. + +**First 5b slice (recommended):** derived/intermediate-symbol emission, validated on the +four near-pure exemplars above — move them from refused to admitted, `wrong=0` preserved. +**But see §9 first — it is a hard precondition.** + +## 9. ⚠ Latent `wrong=0` hazard surfaced by the live audit (gate gap) + +Contrastive probes (not in the 50-case sample) surface a reproducible **admitted-wrong** +path. **Use the parseable aggregate question form `"...do they have together?"`** — the +short form `"How many apples together?"` refuses upstream at question-parse +(`branches_enumerated=0`) and does NOT exercise this `be=1` completeness gap. + +``` +"Tom has 7 apples. Jerry has 3 times as many apples. How many apples do they have together?" + → admitted=True, answer=7, branches_enumerated=1 (correct = 28) # base returned, clause dropped +"...Jerry has five times as many apples. ..." → admitted, answer=7 (correct = 42) +"...Jerry has twice as many apples. ..." → refused, be=1 (completeness: scalar 2.0 unconsumed) # safe +``` + +**Broadened surface (all verified RED on `main`).** The hazard is the **no-reference** +comparative-multiplier surface across all four connectives and both cardinal forms: + +``` + times as many + times more + times the number of + times the +``` +…for `` ∈ {digit 2/3/5/…, word two/three/five/…, N≥2}. Every one admits the base. +`twice` / `double` / `double the` (no digit/cardinal) refuse safely. + +**Root cause (verified by reading the emitted `MathProblemGraph`).** The admitted graph +contains a spurious initial: +`InitialPossession(entity='Jerry', quantity=Quantity(value=3, unit='times'))`. + +1. `quantity_values_in_text` is **symmetric** (registers 2.0 for `twice` *and* 3.0 for + `3 times`) — **not** a quantity-extraction asymmetry (this overturns an earlier guess). +2. For the no-ref surface, **neither** serving comparative regex fires + (`_COMPARE_MULT_ANCHOR_RE` / `_COMPARE_MULT_NTIMES_RE` both require an "as ``" + tail the probe lacks). `comparatives.py::_N_TIMES_RE` is the **disjoint** `derivation/` + reader — off the serving path — so it is *not* the locus either. +3. Instead `recognizer_match.py::_match_discrete_count_statement` (open regex + `_extract_discrete_count_re_open`) captures the multiplier cardinal as a **count**, and + `recognizer_anchor_inject.py::_build_initial_from_discrete_count` builds + `CandidateInitial(value=N, unit='times', entity=)` — note the unit is the literal + `'times'` token, **not** the counted unit. +4. That bogus initial **consumes** the scalar N, so completeness sees + `uncovered = {N, base} − {N, base} = ∅` → admits. The answer sums only the counted unit + (`apples`) and returns the base. `twice`/`double` carry no cardinal the discrete-count + regex grabs → no spurious initial → scalar unconsumed → existing guard refuses. + +**Recommended guard shape (refusal-only, `wrong=0` preserving).** Make the discrete-count +recognizer **decline** when its cardinal sits in a +` times {as many | more | the number of | the} ` comparative-multiplier context +— equivalently, refuse to build a discrete-count initial whose unit is the literal +`'times'`. The no-ref form then refuses (like no-ref `twice`) until real no-ref +comparative-multiplicative *emission* lands in 5b. The fix MUST NOT regress the controls +below. + +**Controls (verified on `main`).** +- **With-ref ` times` already solves — must stay green.** train_sample **case 0024**: + *"Sidney does 20 jumping jacks on Monday, 36 on Tuesday, 40 on Wednesday, and 50 on + Thursday. Brooke does three times as many jumping jacks as Sidney. How many jumping + jacks did Brooke do?"* → **438** (committed verdict: correct). Also `dice 3×` → 80. +- **Existing safe refusals — must stay refused.** no-ref `twice`, no-ref `double the`, + with-ref `twice` (Ivan/Jerry dice). + +**Driver test (FINAL, RED on `main`).** `test_completeness_guard_ntimes_noref_hazard.py` +— **10 no-ref hazard cases** (the broadened surface × digit/word) that MUST refuse, plus +**2 with-ref must-solve controls** (0024 → 438, dice → 80) and **3 must-still-refuse +controls**. Verified live: **10 failed, 5 passed** on `main` (hazard RED, controls green). +This is the first concrete 5b PR, ahead of any R1 emission work. + +**Scope — stated precisely.** This is a **latent gate gap, not a live-metric violation.** +The two layers are distinct: the §1 wall is at `branches_enumerated=0` +(emission/question-parse); this hazard lives at `branches_enumerated=1` (the completeness +gate). The real 44 all refuse at `be=0` — upstream of `be=1` — so none reach this gap +today; that is *why* it is latent. train_sample 6/44/0 is reproduced live. The full-test +`0/0/1319` is **not** re-verified here — it is a sealed, recorded measurement per +`docs/claims_ledger.md` row A / `ADR-0119.7` (ciphertext +`evals/gsm8k_math/holdouts/v1/cases.jsonl.age`), not a CI-reproducible artifact; cited, +not re-verified (the citation and ciphertext both exist on `main`). + +**Why it gates 5b.** Phase 5b's explicit goal is to make comparative-multiplicative cases +*reach the graph*. If emission improves while this completeness gap remains, 5b would +**convert latent into live** — admitting wrong answers on exactly the ` times` shape it +is trying to solve. Therefore: **hardening the comparative-multiplicative completeness +tripwire (so a dropped `×` clause refuses, as `twice` does) is a hard precondition for +any 5b emission PR.** The §9 driver test is the ready-made hard negative for the ADR-0174 +OQ#5 validation sub-corpus. diff --git a/docs/analysis/comprehension-primitive-inventory.md b/docs/analysis/comprehension-primitive-inventory.md new file mode 100644 index 00000000..65c67326 --- /dev/null +++ b/docs/analysis/comprehension-primitive-inventory.md @@ -0,0 +1,211 @@ + + +# Comprehension-Primitive Inventory & Cross-Subject Leverage Map + +Status: draft / proposal-only +Scope: read-only analysis from `main`, **verified** in the Claude lane (see appendix) +Task: Task A from `docs/handoff/NEXT-SUBJECTS-CHATGPT-HANDOFF.md` + +## Operating constraints observed + +This artifact is analysis only. It proposes no serving-path edits, no eval edits, no ADR number, and no empirical claims in the inventory body. Any correctness, coverage, or `wrong=0` claim in the body is a structural reading of the code. The **Claude-lane verification appendix** at the end records what was checked against `main` with real reads of the committed report and source; it is the only section permitted to assert empirical state. + +Read surfaces: + +- `docs/handoff/NEXT-SUBJECTS-CHATGPT-HANDOFF.md` +- `CLAUDE.md` +- `generate/derivation/model.py` +- `generate/derivation/extract.py` +- `generate/derivation/clauses.py` +- `generate/derivation/comparatives.py` +- `generate/derivation/search.py` +- `generate/derivation/multistep.py` +- `generate/derivation/target.py` +- `generate/derivation/compose.py` +- `generate/derivation/accumulate.py` +- `generate/derivation/pool.py` +- `generate/derivation/product_bridge.py` +- `generate/derivation/state/bind.py` +- `generate/derivation/state/change.py` +- `generate/math_candidate_parser.py` +- `generate/math_candidate_graph.py` +- `generate/recognizer_anchor_inject.py` +- Skimmed referenced ADR surface in code/docstrings, especially ADR-0126, ADR-0131, ADR-0136, ADR-0163, ADR-0170, ADR-0175, ADR-0176, ADR-0178, ADR-0182, ADR-0184, ADR-0186, ADR-0189a, ADR-0191, ADR-0193, ADR-0194, ADR-0195. + +## Inventory table + +| # | Primitive | File / function(s) actually read | One-line description | Subject-general vs math-specific? | +|---:|---|---|---|---| +| 1 | Grounded quantity object | `generate/derivation/model.py::Quantity` | Represents a text-sourced numeric value with unit and source token provenance. | **Subject-general core, math-shaped payload.** Provenance-bearing "observed fact" objects transfer broadly; the `value/unit` fields are math-specific. | +| 2 | Grounded derivation step | `generate/derivation/model.py::Step` | Represents one operation, its operand, and the licensing cue that must ground in text. | **Subject-general.** The pattern "claim/action must carry its own evidence cue" transfers to logic, reading comprehension, measurement, and any rule-bound subject. | +| 3 | Deterministic left-fold derivation | `generate/derivation/model.py::GroundedDerivation.answer` | Computes a candidate result by left-folding validated steps over a start quantity. | **Mostly math-specific.** The arithmetic fold is math-specific; the generalizable primitive is ordered, evidence-carrying state transition. | +| 4 | Primary-unit answer tracking | `generate/derivation/model.py::GroundedDerivation.answer_unit` | Carries the start quantity's unit as the answer unit under current derivation assumptions. | **Math-specific.** It is specifically dimensional arithmetic; the cross-subject analogue is "result type/class propagation." | +| 5 | Digit quantity extraction | `generate/derivation/extract.py::extract_quantities`, `_QTY_RE`, `_quantity` | Extracts digit values followed by single unit tokens into `Quantity` records. | **Subject-general extraction pattern, math-specific symbols.** Literal-span extraction with provenance transfers; numeric parsing/unit attachment is math-specific. | +| 6 | Word-number extraction | `generate/derivation/extract.py::_WORD_QTY_RE`, `_resolve_word_number`, `extract_quantities` | Resolves closed-set word numerals and conservative hyphen compounds into quantities. | **Broadly reusable.** Any subject with controlled lexical facts can use the same closed-vocabulary grounding discipline. | +| 7 | Function-word unit hygiene | `generate/derivation/extract.py::_NON_UNIT_WORDS`, `_clean_unit` | Blanks function words that would otherwise be misread as units. | **Subject-general.** This is a lexical false-positive suppression primitive; math uses it for units, but other subjects need equivalent stop-token guards. | +| 8 | List-unit inheritance | `generate/derivation/extract.py::_LIST_WITH_TRAILING_UNIT_RE`, `extract_quantities` | Assigns a trailing unit to every number in a same-list numeric sequence. | **Mixed.** The list inheritance pattern transfers to reading/measurement; the inherited object is math-specific. | +| 9 | Sentence-final bare number extraction | `generate/derivation/extract.py::_FINAL_NUMBER_RE`, `extract_quantities` | Keeps terminal numbers available with unknown/empty unit rather than inventing a unit. | **Subject-general refusal-first grounding.** It preserves observed evidence without hallucinating missing attributes. | +| 10 | Hyphen-bonded quantity extraction | `generate/derivation/extract.py::_HYPHEN_QTY_RE`, `extract_quantities` | Extracts tight `number-unit` surfaces such as `25-foot` without admitting open-ended multi-word units. | **Mixed.** Hyphenated modifier handling transfers; the payload is measurement/math-specific. | +| 11 | Clause segmentation | `generate/derivation/clauses.py::segment_clauses` | Splits problem text into sentence-level clauses using terminal punctuation. | **Subject-general.** Clause segmentation is a foundational reading primitive; the implementation is intentionally orthographic and conservative. | +| 12 | Clause-local sub-derivation | `generate/derivation/clauses.py::clause_local_results` | Derives each clause's local contribution or holds unresolved on ambiguity. | **Subject-general.** "Resolve locally before composing globally" transfers directly to reading comprehension, logic proof steps, and multi-sentence science/measurement tasks. | +| 13 | Comparative scalar extraction | `generate/derivation/comparatives.py::extract_comparative_scalars`, `_load_comparatives`, `_N_TIMES_RE` | Maps closed comparative lexemes and ` times` phrases into scalar operations. | **Mixed.** Closed lexical relation extraction is subject-general; scalar multiplication is math-specific. | +| 14 | Comparative-to-step bridge | `generate/derivation/comparatives.py::comparative_step` | Converts a comparative scalar into a derivation step whose grounding comes from the cue, not necessarily a literal numeric token. | **Subject-general.** The idea that an irreducible lexical fact licenses a typed transformation transfers strongly; the concrete operation is math-specific. | +| 15 | Multiplicative cue hypothesis | `generate/derivation/search.py::MULTIPLICATIVE_CUES`, `_sentence_candidates` | Uses a closed cue set to propose in-clause product candidates only when a multiplicative cue is present. | **Mixed.** Cue-licensed candidate generation is general; multiplication/product semantics are math-specific. | +| 16 | Bounded candidate generation | `generate/derivation/search.py::MAX_QUANTITIES`, `multiplicative_candidates`, `search_multiplicative`; `generate/derivation/multistep.py::MAX_QUANTITIES`, `candidate_chains` | Refuses rather than enumerating unbounded candidate spaces. | **Subject-general.** This is a core safety/performance primitive for any new subject. | +| 17 | Target extraction from question clause | `generate/derivation/target.py::_question_clause`, `extract_target` | Extracts question quantities, aggregation cues, and units named in the question. | **Strongly subject-general.** Every subject lane needs "what is being asked?" extraction; current fields are math-specific. | +| 18 | Prior-state question guard | `generate/derivation/target.py::asks_prior_state`, `_PRIOR_STATE_RE` | Detects questions asking for an earlier temporal state that forward derivation does not compute. | **Subject-general.** Temporal target mismatch is common across reading comprehension, science word problems, and procedural reasoning. | +| 19 | Aggregation hint extraction | `generate/derivation/target.py::_AGG_WORDS`, `_AGG_PHRASES`, `extract_target` | Detects aggregation words/phrases such as `total`, `combined`, and `in all`. | **Mixed.** Aggregation-cue extraction transfers; summation semantics are math-specific. | +| 20 | Question unit intersection | `generate/derivation/target.py::extract_target` | Treats asked units as body-known units that appear in the question. | **Mixed.** Target-slot/body-slot intersection transfers; unit semantics are math-specific. | +| 21 | Shape-based multi-step chain enumeration | `generate/derivation/multistep.py::_candidate_chains`, `_chain`, `candidate_chains` | Builds a small deterministic set of product/sum chains, optionally followed by comparative tail steps. | **Mixed.** Shape-pruned candidate enumeration is general; product/sum chain templates are math-specific. | +| 22 | Same-unit list-sum composition | `generate/derivation/compose.py::compose_sequential`, `_same_unit`, `_ADDITIVE_CUES` | Composes same-unit quantities within one clause using additive cues, with comparative tail application. | **Mixed.** Same-scope list composition transfers to reading/logic lists; same-unit arithmetic is math-specific. | +| 23 | Clause-scoped referent guard | `generate/derivation/compose.py::compose_sequential` | Refuses when a list-sum structure spans multiple quantity-bearing clauses or has out-of-clause comparatives. | **Subject-general.** Scope containment is a central comprehension primitive and directly transfers to reading comprehension. | +| 24 | Single-referent accumulation chaining | `generate/derivation/accumulate.py::_build_accumulation`, `compose_accumulation` | Chains gain/loss changes across clauses only when a later clause safely continues the anchor referent. | **Strongly subject-general.** This is state tracking over discourse; math uses numeric state, but the primitive is broadly useful. | +| 25 | Foreign-distractor candidate handling | `generate/derivation/accumulate.py::_build_accumulation`, `accumulation_candidates`; `generate/derivation/verify.py::classify_derivation` | Allows isolated foreign quantities to enter as disagreement-only/exempt readings rather than commit candidates. | **Subject-general safety primitive.** Distractor evidence handling transfers to all comprehension lanes with irrelevant details. | +| 26 | Sub-clause splitting | `generate/derivation/accumulate.py::_sub_clauses`, `_CONJUNCTION_SPLIT`, `_build_accumulation_anchor_skip` | Locally splits clauses on conjunctions for anchor/change discovery without changing the global segmenter. | **Subject-general.** Local structural refinement under a narrow caller-owned scope transfers well. | +| 27 | Leading-subject extraction | `generate/derivation/state/bind.py::leading_subject_token` | Extracts a clause's leading word token as a loose subject signal. | **Subject-general.** It is a minimal discourse entity cue. | +| 28 | Conservative same-referent continuation | `generate/derivation/state/bind.py::continues_anchor_referent`, `PRONOUNS` | Allows pronouns/same subjects/lowercase continuations and refuses new capitalized actor hazards. | **Subject-general.** This is directly reusable for reading comprehension and logic story-state tracking. | +| 29 | Change polarity classification | `generate/derivation/state/change.py::classify_change_polarity`, `GAIN_VERBS`, `LOSS_VERBS` | Maps closed gain/loss cue sets to `+1`, `-1`, or refusal on ambiguity. | **Mixed.** Polarity classification is subject-general; gain/loss inventory is math-story specific. | +| 30 | Grounded change cue selection | `generate/derivation/state/change.py::select_change_cue` | Chooses the actual cue lexeme that will be checked by the verifier. | **Subject-general.** Separating classification from evidence-cue selection is broadly valuable. | +| 31 | Operand grounding gate | `generate/derivation/verify.py::self_verifies`, `_base_reasons` | Requires every non-comparative operand value token to ground in the problem text. | **Subject-general.** No invented evidence is a cross-domain invariant. | +| 32 | Operation-cue grounding gate | `generate/derivation/verify.py::_base_reasons` | Requires every operation's licensing cue to appear in the text. | **Subject-general.** Every subject lane should require transformation rules to be evidence-licensed. | +| 33 | Unit consistency gate | `generate/derivation/verify.py::_base_reasons`, `_SAME_UNIT_REQUIRED` | Requires same units for add/subtract while allowing multiply/divide composition. | **Math-specific with transferable type discipline.** The gate's type-checking role transfers; the unit rules are math-specific. | +| 34 | Completeness gate | `generate/derivation/verify.py::_unused_quantities`, `self_verifies` | Refuses derivations that leave problem quantities unused. | **Subject-general.** "Account for all salient evidence" is central to reading, logic, measurement, and science tasks. | +| 35 | Branch disagreement / uniqueness gate | `generate/derivation/verify.py::select_self_verified`; `generate/derivation/pool.py::resolve_pooled`; `generate/math_candidate_graph.py::parse_and_solve` | Commits only when verified candidates collapse to one distinct answer; otherwise refuses. | **Strongly subject-general.** This is one of the most transferable wrong=0-preserving primitives. | +| 36 | Commit-eligible vs exempt classification | `generate/derivation/verify.py::classify_derivation` | Classifies readings as complete, exempt, or invalid; exempt readings can force disagreement but cannot commit alone. | **Subject-general.** "Counter-reading can block commitment without becoming an answer" is broadly useful. | +| 37 | Repeated-unit product hazard detector | `generate/derivation/verify.py::_is_repeated_unit_product` | Marks pure products that repeat non-empty dimensions as commit-ineligible. | **Math-specific.** The general form is domain-type impossibility detection. | +| 38 | Cross-composer pooling | `generate/derivation/pool.py::pooled_candidates`, `resolve_pooled` | Pools accumulation, multiplicative, and target-guided chain readings before applying disagreement/commit rules. | **Subject-general architecture.** Multiple independent readers should meet at a common disagreement gate. | +| 39 | Serving promotion bridge | `generate/derivation/product_bridge.py::resolve_promotable_product`, `_has_hazard_surface`, `_has_product_target` | Promotes only complete pure-product readings whose question target and blocker checks make them safe for serving exposure. | **Mixed.** Promotion-boundary pattern is subject-general; current target/hazard surfaces are math-specific. | +| 40 | Candidate initial-state extraction | `generate/math_candidate_parser.py::extract_initial_candidates`, `CandidateInitial` | Emits initial possession/state candidates with source-span provenance. | **Subject-general.** Initial state extraction is foundational for any story/world model; possession quantity is math-specific. | +| 41 | Value-slot resolution | `generate/math_candidate_parser.py::_resolve_value`, `_resolve_currency`, `_is_indefinite_quantifier` | Resolves digits, money, fractions, word numbers, and hyphenated cardinals; refuses indefinite/unparseable values. | **Mixed.** Refusal-first lexical resolution transfers; supported value types are math-specific. | +| 42 | Unit canonicalization | `generate/math_candidate_parser.py::_canonicalize_unit`, `_money_unit_normalization` | Maps surface unit tokens to canonical/plural units, including money normalization. | **Math/measurement-specific with transferable normalization boundary.** Other subjects need similar canonicalization for entities, predicates, or labels. | +| 43 | Operation candidate extraction | `generate/math_candidate_parser.py::extract_operation_candidates`, `_op_pattern`, `_build_op_candidate` | Emits add/subtract/transfer operation candidates from canonical subject-verb-value-unit shapes. | **Mixed.** Typed event extraction transfers; arithmetic operation kinds are math-specific. | +| 44 | Comparative operation extraction | `generate/math_candidate_parser.py::_compare_additive_candidates`, `_compare_multiplicative_candidates`, `_compare_nested_candidates`, `_resolve_reference_token` | Emits comparison candidates using closed comparison anchors and reference grounding. | **Mixed.** Comparative relation extraction transfers strongly; numeric delta/factor semantics are math-specific. | +| 45 | Question candidate extraction | `generate/math_candidate_parser.py::extract_question_candidates`, `CandidateUnknown` | Emits unknown target candidates from closed question shapes. | **Subject-general.** Question-frame parsing is a primary cross-subject bottleneck. | +| 46 | Aggregate question frames | `generate/math_candidate_parser.py::_Q_TOTAL_RE`, `_Q_THERE_RE`, `extract_question_candidates` | Maps total-across question surfaces to `Unknown(entity=None, unit=...)`. | **Mixed.** Aggregate target framing transfers; "unit total" is math-specific. | +| 47 | Activity question frame | `generate/math_candidate_parser.py::_Q_DID_RE`, `extract_question_candidates` | Handles `How many did ?` activity-count questions. | **Mixed.** Activity target extraction transfers; counted activity quantity is math-specific. | +| 48 | Conditional-prefix stripping | `generate/math_candidate_graph.py::_strip_conditional_prefix`, `_filtered_question_choices` | Retries question parsing after removing an `If X,` prefix. | **Subject-general.** Conditional-wrapper removal is broadly useful across logic and reading comprehension. | +| 49 | Comparative-question refusal detector | `generate/math_candidate_parser.py::_pattern_b_comparative_candidates`, `_pattern_b_detects` | Recognizes "how many more" questions but emits no candidate until solver semantics exist. | **Subject-general safety primitive.** Detection-only recognizers can force clean refusal without pretending capability. | +| 50 | Pronoun question resolution | `generate/math_candidate_parser.py::_resolve_pronoun_entity`, `_resolve_question_entity`, `_pattern_c_pronoun_verb_candidates` | Resolves gendered pronoun question entities only when exactly one whitelisted antecedent is present. | **Subject-general, implementation narrow.** The refuse-on-ambiguity pattern transfers; current name lists are GSM8K-specific. | +| 51 | Statement context classifier | `generate/math_candidate_parser.py::has_numeric_token`, `classify_sentence` | Skips non-numeric context statements while preserving numeric-state-bearing statements as required parse/refuse inputs. | **Mixed.** Context filtering transfers; numeric-token criterion is math-specific. | +| 52 | Capacity/rate extraction | `generate/math_candidate_parser.py::extract_capacity_candidates`, `extract_capacity_question_candidates`, `_to_seconds`; `generate/math_candidate_graph.py::parse_and_solve` | Extracts capacity per time and matching time-target questions, then computes scaled rate answers in a guarded short-circuit. | **Math/measurement-specific.** The broader primitive is matched statement/question rate-frame binding. | +| 53 | Earnings-rate extraction | `generate/math_candidate_parser.py::extract_earnings_candidates`, `extract_earnings_question_candidates`; `generate/math_candidate_graph.py::parse_and_solve` | Extracts currency-per-time statements and matching money-over-time questions. | **Math/measurement-specific.** Transfers mainly to measurement/finance-like lanes. | +| 54 | Conditional operation question | `generate/math_candidate_parser.py::extract_conditional_op_question_candidates`; `generate/math_candidate_graph.py::parse_and_solve` | Handles `If entity changes by N, how many ... left/now?` by matching one existing initial state and applying polarity. | **Mixed.** Conditional hypothetical target binding transfers strongly; arithmetic update is math-specific. | +| 55 | Sentence splitting / one-question invariant | `generate/math_candidate_graph.py::_split_sentences`, `parse_and_solve` | Splits text, requires exactly one question sentence, and refuses otherwise. | **Subject-general.** Most subject lanes need explicit problem/question segmentation and clean refusal on malformed tasks. | +| 56 | Per-sentence round-trip filtering | `generate/math_candidate_graph.py::_filtered_statement_choices`, `_filtered_question_choices`, `_initial_admissible`, `_question_admissible` | Filters emitted candidates by structural grounding before graph assembly. | **Subject-general.** Candidate emission and admissibility must remain separate in every subject. | +| 57 | Most-grounded-slots tiebreaker | `generate/math_candidate_graph.py::_slot_count`, `_collapse_per_sentence_ties` | Collapses same-sentence candidates to the most grounded candidate when appropriate. | **Subject-general but hazardous if overused.** It transfers as a deterministic tiebreaker, but each subject must prove it cannot mask ambiguity. | +| 58 | Graph construction with referential integrity | `generate/math_candidate_graph.py::_build_graph` | Builds a `MathProblemGraph`, rejecting branches whose question references unknown entities or violate graph invariants. | **Subject-general architecture, math-specific graph type.** Every subject needs typed graph construction with integrity checks. | +| 59 | Cartesian branch enumeration cap | `generate/math_candidate_graph.py::MAX_TOTAL_BRANCHES`, `parse_and_solve` | Bounds branch enumeration and refuses when the space would exceed the cap. | **Subject-general.** Essential for deterministic safety and performance. | +| 60 | Recognizer registry fallback | `generate/math_candidate_graph.py::_load_ratified_registry_or_empty`, `parse_and_solve` | Consults ratified recognizers only when parser choices are empty, and treats registry failures as empty. | **Subject-general.** Reviewed recognizer fallback with fail-closed behavior transfers directly. | +| 61 | Anchor injection dispatch | `generate/recognizer_anchor_inject.py::inject_from_match` | Converts recognized anchors into typed solver primitives or returns empty on unsupported/unsafe categories. | **Subject-general.** This is a reusable boundary between recognizers and solver primitives. | +| 62 | Composition registry consultation | `generate/recognizer_anchor_inject.py::_consult_composition_registry` | Admits pre-composed payloads only when the composition registry affirms their surface shape. | **Subject-general.** Reviewed structural-shape admission is reusable for logic, reading, and geometry. | +| 63 | Discrete-count anchor injection | `generate/recognizer_anchor_inject.py::inject_discrete_count_statement`, `_build_initial_from_discrete_count`, `_build_operation_from_discrete_count_acquisition` | Builds initial-state or add-operation candidates from discrete-count recognizer anchors. | **Mixed.** Anchor-to-typed-fact injection is general; discrete count semantics are math-specific. | +| 64 | Sealed injector lane | `generate/recognizer_anchor_inject.py::_SEALED_INJECTORS`, `inject_from_match`; `generate/math_candidate_graph.py::parse_and_solve(sealed=...)` | Keeps in-development injectors out of default serving until reviewed promotion. | **Subject-general.** This is a major reusable safety boundary for new subject lanes. | +| 65 | Lookback pronoun resolution / ambiguity defense | `generate/math_candidate_graph.py::parse_and_solve` recognizer-injection section | Holds pronoun-requiring injected candidates until a discourse antecedent or pack-backed disambiguation is available; otherwise drops them. | **Strongly subject-general.** This is directly relevant to reading-comprehension and story-state subjects. | +| 66 | Reader trace events | `generate/math_candidate_graph.py::CandidateGraphResult.reader_trace`, pronoun/lookback trace appends in `parse_and_solve` | Carries JSON-encoded trace events for reader phases and elimination/refusal causes. | **Subject-general.** Traceability/replay evidence is central to every future lane. | + +## Cross-subject leverage map + +### Strong transfer primitives + +These are the highest-leverage primitives for new subjects because they are not inherently arithmetic: + +1. **Evidence-carrying candidate objects** — anchors: `Quantity`, `Step`, `CandidateInitial`, `CandidateOperation`, `CandidateUnknown`. Cross-subject use: claims, propositions, logical premises, reading-comprehension facts, geometry givens. +2. **Candidate emission separated from admissibility** — anchors: `extract_*_candidates`, `_initial_admissible`, `_question_admissible`, `roundtrip_admissible`, `self_verifies`. Cross-subject use: emit possible readings, then require grounding/type/consistency before commitment. +3. **Refusal-first ambiguity handling** — anchors: `select_self_verified`, `resolve_pooled`, `parse_and_solve` decision rule. Cross-subject use: when multiple interpretations remain, refuse instead of choosing. +4. **Scope/referent guards** — anchors: `segment_clauses`, `compose_sequential` clause-local guard, `continues_anchor_referent`, `_resolve_pronoun_entity`, lookback ambiguity defense. Cross-subject use: reading comprehension, narrative state tracking, logic variable binding. +5. **Question/target extraction** — anchors: `extract_target`, `extract_question_candidates`, conditional prefix stripping, capacity/earnings/conditional question extractors. Cross-subject use: target-frame parsing is the obvious shared bottleneck across math, logic, reading, and measurement. +6. **Completeness and distractor classification** — anchors: `_unused_quantities`, `classify_derivation`, exempt readings, context classifier. Cross-subject use: all subjects need "account for all relevant evidence" without forcing irrelevant distractors into the committed answer. +7. **Promotion boundaries** — anchors: `resolve_promotable_product`, sealed injectors, ratified registry fallback. Cross-subject use: experimental readers can exist without becoming served behavior. + +### Math-specific primitives with reusable analogues + +| Math-specific primitive | Why math-specific | Reusable analogue | +|---|---|---| +| Unit consistency | Depends on dimensional arithmetic rules. | Type consistency / sort checking. | +| Product/sum chain enumeration | Depends on arithmetic operator semantics. | Bounded proof/action sequence enumeration. | +| Comparative scalar multiplication | Numeric scalar operation. | Relation-strength or predicate-transform facts from closed packs. | +| Capacity/earnings rate short-circuits | Rate arithmetic over time/currency. | Matched statement-target frame with deterministic transformation. | +| Repeated-unit product hazard | Dimensional impossibility. | Domain-type impossibility detector. | +| Money/currency normalization | Numeric unit system. | Canonical symbol/entity normalization. | + +## Observed composition wall + +The current substrate already has many individually strong primitives. The bottleneck is not lack of primitives; it is safe composition among them: + +- Clause-local reasoning exists, but cross-clause reasoning remains guarded and narrow. +- Question target extraction exists, but many target frames still require closed shape support. +- Referent continuation exists, but pronoun/coreference resolution is intentionally conservative. +- Candidate pooling exists, but promotion to serving requires narrow target/hazard gates. +- Completeness is strong, but it can over-force distractors unless exempt/disagreement paths are present. + +This confirms the brief's framing: the next-subject work should exercise the same composition primitives without creating live serving risk. + +## What transfers to other subjects + +- **Reading comprehension should reuse the most math-relevant primitives immediately:** clause segmentation, referent guards, pronoun ambiguity refusal, target-frame parsing, completeness, and branch disagreement are already the exact pain points behind the math composition wall. +- **Symbolic/deductive logic can reuse the candidate/admissibility/disagreement architecture:** premises become evidence-bearing candidates, inference rules become cue- or schema-licensed steps, and ambiguous proof branches refuse rather than commit. +- **Measurement/geometry can reuse the most math-specific substrate with low conceptual impedance:** quantity extraction, unit canonicalization, unit/type consistency, target-unit matching, rate/measurement frames, and dimensional impossibility checks are already close to that domain. +- **All future subjects should preserve the sealed/promotion boundary pattern:** draft readers and recognizers can be explored only as proposal-only or sealed lanes until the Claude lane verifies the relevant invariants. +- **The highest cross-subject ROI is not a new corpus first; it is a small capability-axis spec that stresses target extraction, referent binding, completeness, and disagreement without weakening `wrong=0`.** + +## Open questions for the Claude lane + +1. Verify whether any functions above are currently serving-active vs sealed/practice-only on `main`; this read-only pass did not run lane-sha checks or tests. +2. Confirm the exact current serving count and wrong/refusal distribution through the pinned eval lane before using this document as planning evidence. +3. Decide whether Task B should treat `product_bridge.resolve_promotable_product` as part of the active question layer or as a promotion boundary around the derivation reader. +4. Inspect coverage for the "most-grounded-slots-wins" tiebreaker before reusing that pattern in any new subject; it is powerful but could mask ambiguity if applied too broadly. +5. For Task C, compare candidate subject ordering against the actual contents of `evals/symbolic_logic/` and `evals/math_capability_axes/` before drafting any subject-specific axes. + +--- + +## Claude-lane verification (landed) + +Verified against `main` at commit `3e29559` by reading the committed serving report and source. Method note: the inventory above was authored read-only; the checks below resolve its five open questions. The full `core test`/MLX/Rust suite was **not** re-run in this lane (Apple-Silicon/MLX substrate unavailable here); the serving metric cited is the committed, pinned report — the authoritative source of truth for the frozen serving path — not a fresh run. + +**Definition-of-done check (Task A):** all 66 primitives resolve to real files on `main`. Every referenced module exists (`generate/derivation/{model,extract,clauses,comparatives,search,multistep,target,compose,accumulate,pool,product_bridge,verify}.py`, `generate/derivation/state/{bind,change}.py`, `generate/{math_candidate_parser,math_candidate_graph,recognizer_anchor_inject}.py`). No invented APIs found. + +### Q1 — serving-active vs sealed +- `_SEALED_INJECTORS = {}` is **empty** on `main`. Nothing is currently sealed. Inventory row #64 describes a real mechanism, but it is presently inert — so "sealed lane" is not what is suppressing any current behavior. +- `discrete_count_statement` is **serving-active**: it is wired directly into the live dispatch map (`ShapeCategory.DISCRETE_COUNT_STATEMENT: inject_discrete_count_statement`). Its empty injections (see Q2) are genuine conservatism in the active injector, not sealing. +- The frozen-serving gate (`scripts/verify_lane_shas.py`) pins the **SHA-256 of report outputs** for 8 eval lanes (reviewer_registry, miner_loop_closure, curriculum_loop_closure, domain_contract_validation, fabrication_control, demo_composition, public_demo, math_teaching_corpus). It freezes serving by making any drift in those outputs detectable; it does not pin a static list of serving source files. + +### Q2 — exact serving distribution (CONFIRMED) +Pinned report `evals/gsm8k_math/train_sample/v1/report.json` (ADR-0126, sample_count=50): + +- **6 correct / 44 refused / 0 wrong.** `wrong=0` holds. `exit_criterion.correct_min=10` → `passed: false`. + +The 44 non-correct cases decompose as: + +| Failure mode | Count | +|---|---:| +| Recognizer matched but produced **no injection** | 32 | +| **No admissible candidate** (parser emitted nothing usable) | 12 | + +Locus of the 44: statement (recognizer) 32 · statement (parser) 7 · question (parser) 5. + +Recognizer-fired-but-empty-injection (32) by category: + +| Category | Count | +|---|---:| +| `discrete_count_statement` | 18 | +| `descriptive_setup_no_quantity` | 4 | +| `rate_with_currency` | 3 | +| `multiplicative_aggregation` | 3 | +| `currency_amount` | 3 | +| `temporal_aggregation` | 1 | + +**Headline:** the single largest refusal bucket is `discrete_count_statement` — **18 of 44 (41%)** — where the serving-active recognizer fires on a count-like token but the injector returns empty. **This marks *where* the composition wall surfaces; it is not a lever to widen.** As the corrected Net-read below establishes, all 18 are 2–4 capability compositions the injector *correctly* declines (emitting an initial-state there is metric-inert). The concentration is diagnostic — the most common surface form of the wall — not a backlog item, and it touches the entity/initial-state primitives (#40, #63) only as evidence that the wall sits *downstream* of extraction, in composition. + +### Q3 — `product_bridge.resolve_promotable_product` classification (RESOLVED) +It is part of the **active serving question layer, behaving as a promotion boundary around the derivation reader** — both, not either/or. Its module docstring places it on "the serving candidate-graph path," and it returns a "serving-safe product resolution" only after passing `_has_hazard_surface` and `_has_product_target`. Recommendation for Task B: treat it as the guarded gate by which derivation-reader products reach serving, i.e. a promotion boundary that is itself live — not a sealed/practice-only reader. + +### Q4 — "most-grounded-slots-wins" tiebreaker coverage (CAUTION CONFIRMED, scope corrected) +`_collapse_per_sentence_ties` / `_slot_count` are invoked at two serving sites in `parse_and_solve` (lines 958, 999). No test references those functions **by name** (no white-box test). However — correcting an earlier overstatement in this appendix — the collapse **is** behaviorally covered on the happy path: `tests/test_math_candidate_graph.py::TestAmbiguityResolution::test_gives_with_target_resolves_to_transfer` exercises the slot-count collapse ("Sam gives 3 apples to Tom" → transfer reading wins on more grounded slots) and would fail if the collapse broke. The accurate, narrower gap is therefore: happy-path collapse is covered; what is **missing** is (a) a white-box test naming the functions and (b) an **adversarial "high-slot-but-wrong vs low-slot-but-right"** case — the scenario where "more slots = better" selects the wrong reading. Recommendation: add both before reusing this pattern in any new subject. + +### Q5 — Task C input (DEFERRED to Task C execution) +Not resolved here; Task C explicitly requires comparing candidate subject ordering against the live contents of `evals/symbolic_logic/` and `evals/math_capability_axes/`. Flagged for the Task C pass so it is not double-counted as Task A scope. + +### Net read for planning (corrected) +An earlier version of this section recommended widening the serving-active `discrete_count_statement` injector as "the highest-count, lowest-risk math lever (18/44)." **That conclusion was wrong and is retracted.** Reading all 18 of those cases in full shows they are **2–4 capability compositions** (ratio chains 0020/0029/0033, multi-step rate/percent 0032/0034/0044, accumulate-against-target 0037/0039, and 0040 which needs per-entity attribute lookup before any arithmetic). The recognizer fires on the first count token ("2 horses"); the injector **correctly declines** because the surrounding problem is not a bare count. Emitting an initial-state there is **metric-inert** — the graph still cannot compose to the answer. The 18/44 concentration is the **composition wall surfacing at the most common recognizer category**, not an injector to widen. This is reinforced by **ADR-0174 (Proposed)**, which deprecates the per-category injector dispatch table as the runtime admission path (injectors become hypothesis-emitters in a held-hypothesis reader), and by the wrong=0 hazard of that surface (case-0050 canary on the same serving path). + +Corrected steer: primitives are not the bottleneck; **safe composition is.** The honest next lever is a **composition capability** over the existing grounded primitives — multi-quantity chains (ratio, multi-step rate/percent, accumulate-against-target). The direct GSM8K-metric lever is **ADR-0174's held-hypothesis reader (Proposed)**; the adjacent proof-DAG substrate — binding-graph acyclicity, proof-graph builder, modus-ponens disagreement — is already **Accepted** (ADR-0203/0204/0205, proof_chain phase 2.1–2.3). So the work is composition through the held-hypothesis reader on an accepted proof substrate, **not** category-dispatch widening. For Task B: group all 44 and rank by **composition-arity** (1-capability gaps = tractable; 2–4-capability compositions = the wall), not by raw recognizer-category count.