diff --git a/docs/decisions/ADR-0184-scoped-semantic-state-transitions.md b/docs/decisions/ADR-0184-scoped-semantic-state-transitions.md new file mode 100644 index 00000000..1f21c95a --- /dev/null +++ b/docs/decisions/ADR-0184-scoped-semantic-state-transitions.md @@ -0,0 +1,551 @@ +# ADR-0184 — Scoped Semantic State Transitions for English Multi-Step Reasoning + +**Status:** Proposed +**Date:** 2026-05-29 +**Author:** ChatGPT / Josh direction +**Blueprint:** [`docs/handoff/SEMANTIC-STATE-TRANSITION-BLUEPRINT.md`](../handoff/SEMANTIC-STATE-TRANSITION-BLUEPRINT.md) +**Anchor:** [[thesis-decoding-not-generating]] +**Builds on:** ADR-0165, ADR-0174, ADR-0176, ADR-0177, ADR-0178, ADR-0179, ADR-0182 +**Supersedes:** no runtime path yet; this is a scope-setting ADR for the next implementation sequence. + +--- + +## 1. Decision + +CORE will introduce a sealed derivation-lane semantic-state substrate for English multi-step math reasoning. + +The substrate reads word problems into scoped, entity-owned state transitions before arithmetic derivation: + +```text +text + -> lexeme extraction + clause segmentation + -> semantic frames + -> entity-bound state transitions + -> semantic ledger / world candidates + -> GroundedDerivation replay + -> existing self-verification / classification + -> cross-composer pooling + -> answer or refusal +``` + +This ADR does **not** authorize a serving-path change. The first implementation must remain sealed under `generate/derivation/**` and must emit ordinary `GroundedDerivation` candidates for the existing verifier/pool to judge. + +The immediate implementation target is to promote the proven accumulation logic in `generate/derivation/accumulate.py` into a reusable semantic-state package before adding transfer, comparison, rate/container, temporal replay, or DAG logic. + +--- + +## 2. Why this is needed + +CORE's arithmetic verification layer is stronger than its semantic reading layer. + +The current derivation lane can verify candidate arithmetic traces using grounding, cue evidence, unit consistency, completeness, uniqueness, and commit eligibility. That is the right final judge. But hard English word problems fail earlier: the system often lacks a scoped representation of the situation the text describes. + +The missing layer is not more arithmetic. It is a typed world-state reading: + +```text +Sam has 14 apples. +He buys 9 more. +How many apples does Sam have now? +``` + +should first become: + +```text +SET_STATE(Sam.apples, 14) +GAIN(Sam.apples, 9) +TARGET(final Sam.apples) +``` + +and only then replay to: + +```text +14 + 9 = 23 +``` + +Without this layer, future work will keep reimplementing the same semantic logic inside local composers: + +- subject / referent binding; +- gain/loss polarity; +- question target scope; +- temporal scope; +- foreign distractor classification; +- transfer handling; +- comparison direction; +- rate/container binding; +- branch/DAG quantity reuse. + +That is the path to composer spaghetti and wrong=0 hazards. + +--- + +## 3. What already exists + +The new substrate should use the existing system rather than bypass it. + +### Existing pieces to preserve + +- `generate/derivation/extract.py` — lexeme-level quantity/unit/source-token extraction. +- `generate/derivation/clauses.py` — deterministic clause segmentation and local clause results. +- `generate/derivation/accumulate.py` — first working state-transition composer: single-referent gain/loss accumulation. +- `generate/derivation/target.py` — target extraction and prior-state question guard. +- `generate/derivation/verify.py` — final proof gate: grounding, cue, unit, completeness, classification, uniqueness. +- `generate/derivation/pool.py` — cross-composer candidate pooling, disagreement refusal, commit eligibility. +- `generate/comprehension/state.py` — immutable state, entity/quantity/question/refusal/hypothesis concepts and canonical hashing; useful as design precedent, not automatically the active scoring path. + +### Existing pieces to avoid misusing + +- Do **not** resurrect the old broad comprehension lifecycle as the active scoring reader unless a separate audit proves it is now load-bearing. +- Do **not** revive gender-blind most-recent antecedent pronoun resolution. +- Do **not** make semantic worlds commit directly. +- Do **not** replace `verify.py`. +- Do **not** let any composer pick first non-`None` when multiple competing readings exist; use pooling. + +--- + +## 4. Where this fits + +Add a new sealed derivation-lane package: + +```text +generate/derivation/state/ + __init__.py + model.py + bind.py + change.py + ledger.py + target.py + replay.py + refusals.py +``` + +Later phases may add: + +```text +generate/derivation/state/transfer.py +generate/derivation/state/compare.py +generate/derivation/state/rate.py +generate/derivation/state/time.py +generate/derivation/state/world.py +generate/derivation/state/dag.py +``` + +The initial integration path is deliberately conservative: + +```text +accumulate.py public functions + -> semantic-state helper modules + -> semantic ledger + -> replay to GroundedDerivation + -> existing verify/select/classify +``` + +`pool.py` continues to arbitrate final candidates. + +--- + +## 5. Initial object language + +The first version should model only what accumulation already needs. + +Suggested minimal types: + +```python +@dataclass(frozen=True, slots=True) +class EntityMention: + surface: str + canonical: str + clause_index: int + token_index: int + kind: str # proper | pronoun | implicit | unknown + +@dataclass(frozen=True, slots=True) +class QuantityMention: + value: float + unit: str + source_token: str + clause_index: int + role: str # state | delta | scalar | rate | target | unknown + +@dataclass(frozen=True, slots=True) +class CueMention: + surface: str + cue_kind: str # gain | loss | aggregate | multiplicative | temporal | ... + clause_index: int + +@dataclass(frozen=True, slots=True) +class StateKey: + entity: str + unit: str + +@dataclass(frozen=True, slots=True) +class StateTransition: + key: StateKey + op: str # set | gain | loss + quantity: QuantityMention + cue: CueMention + clause_index: int + +@dataclass(frozen=True, slots=True) +class SemanticLedger: + transitions: tuple[StateTransition, ...] +``` + +The first replay target is a standard `GroundedDerivation`. + +This ADR intentionally does not require all final types up front. It requires that types be frozen, deterministic, and covered by tests before they are used for candidate generation. + +--- + +## 6. Wrong=0 obligations + +Every implementation phase must preserve the following. + +### 6.1 Semantic worlds do not commit directly + +A semantic world may only produce a candidate. It cannot answer by itself. + +Commit path remains: + +```text +SemanticWorld / SemanticLedger + -> GroundedDerivation + -> self_verifies / classify_derivation + -> resolve_pooled +``` + +A test must fail if a semantic-world-only answer bypasses `verify.py` or `pool.py`. + +### 6.2 Existing complete-commit rules remain intact + +`complete` remains the only commit-eligible classification. Exempt/distractor readings may enter the pool only to force disagreement/refusal, never as sole committable answers. + +### 6.3 Ambiguous referents refuse + +If a state transition needs an entity and the entity cannot be safely bound, the semantic world refuses or is not emitted. + +Required tests: + +```text +Sam has 14 apples. He buys 9 more. -> same referent, candidate allowed +Sam has 14 apples. Tom buys 9 more. -> new actor, candidate refused +Alice has 5. Bob has 3. She buys 2. -> ambiguous, refused unless explicitly resolved by a future safe rule +``` + +### 6.4 Ambiguous polarity refuses + +A gain/loss event may be emitted only when polarity is unambiguous and cue-grounded. + +Required tests: + +```text +gets 4 more -> gain +loses 4 -> loss +owns 4 -> no change cue, refused +mixed gain/loss cue in same change frame -> refused +``` + +### 6.5 Unsupported temporal target refuses + +Until temporal replay exists, prior-state questions remain refused. + +Required tests: + +```text +How much did Lisa have before lunch? -> refuse until time replay exists +How much does Lisa have left? -> forward/net target allowed +before in body narrative does not trip prior-state refusal +used to make does not trip prior-state refusal +``` + +### 6.6 Determinism + +All emitted candidates must be deterministic: + +- no random ordering; +- no unordered set/dict iteration in outputs; +- stable candidate ordering; +- frozen dataclasses or canonical bytes for replay-critical state. + +--- + +## 7. Implementation sequence + +### S1 — Extract proven accumulation helpers + +Create: + +```text +generate/derivation/state/bind.py +generate/derivation/state/change.py +``` + +Move behavior-equivalent helpers from `accumulate.py`: + +```text +_subject_token -> leading_subject_token +_same_referent -> continues_anchor_referent +_polarity -> classify_change_polarity +_cue -> select_change_cue +``` + +Acceptance: + +- accumulation tests unchanged and green; +- pool tests unchanged and green; +- new helper tests prove non-vacuous referent and polarity guards; +- no serving imports; +- no runner changes; +- no behavior change. + +### S2 — Add accumulation semantic ledger + +Create: + +```text +generate/derivation/state/model.py +generate/derivation/state/ledger.py +generate/derivation/state/replay.py +``` + +Represent: + +```text +SET_STATE(entity, unit, value) +GAIN(entity, unit, value) +LOSS(entity, unit, value) +``` + +Then replay the ledger into `GroundedDerivation`. + +Acceptance: + +- `compose_accumulation()` output unchanged; +- `accumulation_candidates()` output unchanged or explicitly equivalent by candidate answer/key tests; +- direct manual construction of accumulation chains in `accumulate.py` is minimized or removed; +- replay tests prove ledger -> `GroundedDerivation` correctness. + +### S3 — Add semantic target wrapper + +Create: + +```text +generate/derivation/state/target.py +``` + +Wrap current target extraction with semantic fields: + +```text +entity: optional +unit: optional +time_scope: final | prior | unknown +relation: count | difference | aggregate | unknown +``` + +Acceptance: + +- current prior-state guard behavior preserved; +- forward/net questions still resolve; +- target wrapper is conservative and refuses unsupported scope. + +### S4 — Add semantic candidate source to pool + +Create public surface: + +```python +def semantic_state_candidates(problem_text: str) -> tuple[GroundedDerivation, ...]: ... +``` + +Initially it delegates to accumulation-backed worlds. + +Then update `pool.py` from: + +```text +accumulation_candidates, multiplicative_candidates, candidate_chains +``` + +to: + +```text +semantic_state_candidates, multiplicative_candidates, candidate_chains +``` + +Acceptance: + +- all ADR-0182 pool tests remain green; +- clean accumulation still commits; +- distractor cases still refuse; +- exempt-only still never commits; +- direct `pool.py` import of `accumulation_candidates` can be removed. + +### S5 — Transfer events + +Add source/target entity transitions: + +```text +Sam gives Tom 3 apples + -> Sam.apples -= 3 + -> Tom.apples += 3 +``` + +Acceptance: + +- source-target binding tests; +- ambiguous target refused; +- no initial state refused; +- target-specific question required for commit. + +### S6 — Comparison / difference frames + +Add difference questions: + +```text +Sam has 10 apples. Tom has 7 apples. How many more apples does Sam have than Tom? +``` + +Acceptance: + +- direction explicit -> commit; +- direction ambiguous -> refuse; +- same-unit aggregate does not become difference by accident. + +### S7 — Rate / container frames + +Add structurally bound products: + +```text +24 boxes with 12 erasers each +60 miles per hour for 2 hours +``` + +Acceptance: + +- legitimate products commit; +- distractor duration with unrelated target refuses; +- protected existing product positives do not regress; +- product-of-all fallback remains until semantic rate frames cover positives. + +### S8 — Temporal replay + +Use ledger history to answer: + +```text +before +after +initially +finally +left +now +``` + +Acceptance: + +- prior-state questions answer only when event boundary is unambiguous; +- ambiguous temporal scope refuses; +- current prior-state refusal tests update only when the new capability is proven. + +### S9 — Held worlds and DAGs + +Represent multiple possible semantic worlds and quantity reuse / branch structures. + +Acceptance: + +- linear cases unchanged; +- unresolved multi-world ambiguity refuses; +- no DAG candidate commits without proof obligations equivalent to current grounding/completeness/target requirements. + +--- + +## 8. Immediate next PR authorized by this ADR + +Branch: + +```text +feat/adr-0184-s1-semantic-state-helpers +``` + +Scope: + +1. Add `generate/derivation/state/__init__.py`. +2. Add `generate/derivation/state/bind.py`. +3. Add `generate/derivation/state/change.py`. +4. Move proven helper logic from `accumulate.py` into those modules. +5. Keep `compose_accumulation()` unchanged externally. +6. Keep `accumulation_candidates()` unchanged externally. +7. Add tests for extracted helpers. +8. Do not touch serving. +9. Do not change runners. +10. Do not delete old reader files. + +This is intentionally a behavior-equivalent refactor. It creates the seam for multi-step English reasoning without expanding capability in the same PR. + +--- + +## 9. Explicit non-actions + +Do **not** do any of the following in S1: + +- no serving import; +- no `chat/**` edit; +- no `generate/math_roundtrip.py` edit; +- no runner behavior change; +- no deletion of `generate/comprehension/lifecycle.py`; +- no deletion of `generate/math_parser.py`; +- no product-of-all fallback removal; +- no transfer/comparison/rate/temporal/DAG implementation; +- no broad grammar parser; +- no pronoun resolver beyond current conservative same-referent guard. + +These are future phases or separate ADRs. + +--- + +## 10. Dead-weight policy + +This ADR does not delete code immediately. It establishes deletion criteria. + +### Composer-local helper deletion + +After S1/S2, helper logic for subject extraction, referent continuity, polarity, cue selection, and accumulation replay should not remain private to `accumulate.py` except as thin compatibility wrappers. + +### Pool prior-state guard migration + +After S3, direct prior-state logic in `pool.py` should migrate into semantic target handling or become a compatibility call. + +### Product-of-all fallback demotion + +Only after S7 proves rate/container semantic frames protect current positives and refuse known distractors may product-of-all fallback be demoted or deleted. + +### Old comprehension lifecycle + +Do not delete casually. Demote or delete only after an audit proves it is inert relative to active scoring and no tests depend on it as a runtime path. + +--- + +## 11. Acceptance criteria for this ADR moving from Proposed to Accepted + +This ADR may move to Accepted when: + +1. S1 lands with no behavior change and tests prove helper extraction is non-vacuous. +2. S2 lands with semantic ledger replay for accumulation and accumulation behavior remains equivalent. +3. `resolve_pooled` can source accumulation through `semantic_state_candidates()` with ADR-0182 behavior preserved. +4. Serving remains unchanged. +5. Existing wrong=0 lanes remain wrong=0. +6. Documentation identifies all old code paths that are retained, demoted, or scheduled for later deletion. + +--- + +## 12. Final doctrine + +CORE should complete English multi-step reasoning by reading into scoped semantic state, not by accumulating more arithmetic shapes. + +The durable loop is: + +```text +read the world stated by the text + -> mutate scoped state under cue-licensed rules + -> bind the question to a state/relation/time + -> replay into arithmetic proof objects + -> let existing verifier and pool accept or refuse +``` + +The first safe step is not more coverage. It is extracting the already-proven accumulation semantics into a reusable substrate so the next capability layers can compose without becoming local patches. diff --git a/docs/handoff/RB-GSM-SOLVER-DESIGN-INPUT.md b/docs/handoff/RB-GSM-SOLVER-DESIGN-INPUT.md new file mode 100644 index 00000000..d61e4285 --- /dev/null +++ b/docs/handoff/RB-GSM-SOLVER-DESIGN-INPUT.md @@ -0,0 +1,399 @@ +# RB-GSM Solver Design Input Notes + +**Status:** design input / cautionary mapping +**Branch:** `docs/semantic-state-transition-blueprint` +**Related PR:** #489 +**Related docs:** + +- `docs/handoff/SEMANTIC-STATE-TRANSITION-BLUEPRINT.md` +- `docs/decisions/ADR-0184-scoped-semantic-state-transitions.md` + +--- + +## 1. Why this note exists + +Josh provided an external-style sketch of a pure rule-based GSM8K solver: loose regex extraction, keyword dictionaries, if-then rules, a state builder, an operation queue, and deterministic arithmetic execution. + +The sketch is directionally useful because it reinforces the same high-level move captured in ADR-0184: + +```text +English story + -> signal extraction + -> state machine + -> operation / transition list + -> exact arithmetic replay +``` + +However, the sketch also contains claims and shortcuts CORE must **not** import as doctrine: + +- “100% coverage with 30–50 rules” is an empirical claim, not an architectural fact. +- “Last actor works 99%” is not acceptable as a commit rule under `wrong = 0`. +- Regex and keyword hits may propose readings, but they must not decide answers. +- A first matching rule must not bypass pooling, completeness, target binding, or disagreement refusal. + +This document records what to use, what to reject, and how to fold the useful pieces into CORE's semantic-state transition plan. + +--- + +## 2. Useful principles to keep + +### 2.1 Loose extraction, strict interpretation + +The RB-GSM sketch correctly separates rough signal collection from final logic: + +```text +loose regex = candidate signal collector +keyword / cue table = deterministic interpretation proposal +state machine + verifier = final decision path +``` + +CORE already follows a related discipline in `generate/derivation/extract.py`: extraction should identify lexeme-level quantities and surface evidence, not decide sentence meaning. + +ADR-0184 should preserve this: + +- regex may collect numbers, word-numbers, units, cue candidates, names, and punctuation boundaries; +- regex must not encode whole problem templates; +- semantic rules decide whether a signal becomes a state transition; +- verifier/pool decides whether the candidate can commit. + +### 2.2 State machine before arithmetic + +The RB-GSM sketch's strongest idea is the same as ADR-0184: + +```text +text -> state builder -> operation queue -> executor +``` + +CORE should translate this as: + +```text +text -> semantic ledger -> GroundedDerivation replay -> verifier / pool +``` + +The difference is that CORE's operation queue should be a typed, scoped semantic ledger, not a loosely ordered list of calculator commands. + +### 2.3 Cue tables as proposal tables + +The sketch's keyword dictionary is useful, but in CORE these should be cue tables that propose frame candidates: + +```text +"has" / "had" -> possible SET_STATE +"buys" / "gets" -> possible GAIN +"loses" / "spends" -> possible LOSS +"more than" -> possible COMPARISON / DIFFERENCE +"altogether" -> possible AGGREGATE target +"left" -> possible final/net target or loss result cue +"each" / "per" -> possible RATE / CONTAINER binding +"half" / "twice" -> possible SCALAR / COMPARATIVE +``` + +They should not directly commit arithmetic. + +### 2.4 Executor remains simple + +Once the semantic ledger is correct, arithmetic execution should stay boring: + +```text +SET_STATE -> assign +GAIN -> add +LOSS -> subtract +RATE -> multiply when structurally bound +TRANSFER -> subtract from source, add to target +DIFFERENCE -> subtract two bound states +``` + +This matches CORE's preference: make the hard part the reading/proof, not the arithmetic. + +--- + +## 3. Claims to reject or downgrade + +### 3.1 Reject unproven 100% dataset coverage claims + +The provided sketch claims that 30–50 rules can solve every GSM8K problem after small tuning. CORE should not encode this as truth. + +Correct CORE posture: + +```text +Rule coverage is measured by lanes, not asserted. +``` + +Any coverage claim must be backed by: + +- train_sample report; +- practice report; +- confuser probe; +- sealed holdout where available; +- wrong=0 preservation; +- perturbation / paraphrase checks where available. + +### 3.2 Reject last-actor pronoun as a commit rule + +The sketch says a simple last-actor rule works for ambiguous pronouns because GSM8K stories are linear. + +CORE must not use that as a commit rule. + +Allowed: + +```text +single active actor + pronoun continuation -> candidate may emit +``` + +Forbidden: + +```text +multiple prior actors + pronoun -> pick most recent and commit +``` + +Required behavior: + +```text +multiple possible antecedents -> refuse / hold until a future safe resolver exists +``` + +### 3.3 Reject first-matching operation execution + +The sketch's operation queue implies sequential execution of matched operations. CORE should avoid any first-match priority that can hide competing readings. + +Required behavior: + +```text +multiple plausible readings -> pooled candidates -> disagreement refuses +``` + +This preserves ADR-0182's core lesson. + +### 3.4 Reject regex as final logic + +Regex should not decide: + +- final operation; +- actor binding; +- question target; +- temporal target; +- rate binding; +- comparison direction; +- transfer source/target. + +Regex may propose candidates only. + +--- + +## 4. Mapping RB-GSM components into CORE + +| RB-GSM sketch | CORE equivalent | Notes | +|---|---|---| +| `number_pattern` | `generate/derivation/extract.py` | Keep lexeme-level only. | +| `name_pattern` | `state/bind.py` entity mention collector | Capitalized names are signals, not proof. | +| `keyword_table` | `state/change.py`, `state/rate.py`, `state/compare.py` cue tables | Cue tables emit semantic frame candidates. | +| `variables` dict | `SemanticLedger` / entity-owned `StateKey` | Must include entity + unit/scope. | +| `op_queue` | ordered `StateTransition` tuple | Transitions remain typed and replayable. | +| `execute_operations` | `state/replay.py` -> `GroundedDerivation` | Existing verifier still judges. | +| `answer_question` | semantic target binding + pool | Target must bind before commit. | +| `readable_steps` | derivation trace / replay explanation | Deterministic, auditable. | + +--- + +## 5. Recommended adjustments to ADR-0184 implementation + +### 5.1 Add a cue-table policy + +ADR-0184 S1/S2 should explicitly define cue tables as **proposal tables**, not answer rules. + +Policy: + +```text +A cue table entry may create a candidate semantic frame. +A cue table entry may not directly commit an arithmetic operation. +Every emitted candidate must still pass replay, verification, classification, and pooling. +``` + +### 5.2 Add a loose-signal collector concept + +S1 can stay helper extraction only, but S2 should prepare the seam for: + +```text +SignalBundle: + names + quantities + cue_hits + question_markers + temporal_markers +``` + +This does not need to be implemented immediately, but the model should not prevent it. + +### 5.3 Keep accumulation as the first worked example + +Use the Natalia-style pattern from the sketch as a future fixture class, but implement in CORE terms: + +```text +Natalia sold 48 clips in April. +She sold half as many in May. +How many altogether? +``` + +Expected semantic reading: + +```text +SET_STATE(Natalia.clips.April, 48) +SCALE_COPY(Natalia.clips.May, Natalia.clips.April, 0.5) +TARGET(aggregate Natalia.clips over April+May) +``` + +Important: this is **not** S1 or S2. It requires scalar-copy and aggregate target support, likely after basic accumulation ledger and semantic target wrapper exist. + +### 5.4 Treat rate examples as S7, not S1 + +The Weng babysitting example: + +```text +Weng earns $12 an hour. Yesterday she babysat 50 minutes. How much did she earn? +``` + +requires rate binding and unit conversion. It belongs under ADR-0184 S7 or a later rate/unit sub-ADR, not under S1 helper extraction. + +### 5.5 Add anti-overfit language + +Every semantic-state ADR/sub-ADR should include: + +```text +Regex collects signals; it does not license final meaning. +Cue dictionaries emit candidates; they do not bypass verifier/pool. +New keyword entries require tests showing both positive use and at least one refusal guard. +``` + +--- + +## 6. Revised immediate S1 non-scope + +Even with the RB-GSM input, S1 should remain behavior-equivalent. + +Do not add yet: + +- broad keyword table; +- operation queue executor; +- scalar-copy / half-as-many logic; +- rate conversions; +- cross-month aggregation; +- all-problem solver loop; +- last-actor pronoun resolver; +- confidence string or any claim of 100% coverage. + +S1 remains: + +```text +extract proven helper logic from accumulate.py + -> state/bind.py + -> state/change.py + -> helper tests + -> no behavior change +``` + +This matters because if S1 expands capability, it becomes impossible to tell whether the new semantic-state seam is safe. + +--- + +## 7. Future capability ordering informed by RB-GSM + +The RB-GSM sketch is most useful as a reminder of common GSM8K frame families. The ordering should be: + +1. **Existing accumulation extraction** — already proven; refactor only. +2. **Semantic ledger replay** — `SET_STATE`, `GAIN`, `LOSS`. +3. **Semantic question target** — final/prior/count/aggregate skeleton. +4. **Scalar copy / comparative amount** — half/twice/as-many; Natalia-style cases. +5. **Aggregate target over scoped states** — altogether / total / in all across compatible states. +6. **Transfer** — source and target entity mutations. +7. **Difference questions** — how many more/fewer than. +8. **Rate/container binding** — each/per/hour/minutes. +9. **Temporal replay** — before/after/originally/finally. +10. **Held worlds / DAGs** — quantity reuse and branching. + +This ordering differs slightly from the first ADR-0184 sequence by inserting scalar-copy and scoped aggregation before transfer. That may be higher payoff for GSM8K because many problems ask for totals over derived period/entity states. + +This should be reviewed before coding S3+. + +--- + +## 8. New guardrail tests suggested by this input + +Add these once their phase begins: + +### Loose regex must not over-decide + +```text +"Alice has 5 apples. Bob has 3 apples. How many apples does Alice have?" +``` + +A loose name/number extractor may see `Alice`, `Bob`, `5`, `3`, and `apples`, but no rule may sum 5+3 for Alice. + +### Cue hit must not commit alone + +```text +"Kate studies for 3 hours and buys 5 pencils. How many pencils?" +``` + +A `for` cue may be collected, but it must not force multiplication into the pencil target. + +### Last actor is not proof + +```text +"Alice has 5 apples. Bob has 3 apples. She buys 2 more apples. How many apples does Alice have?" +``` + +If gender/antecedent resolution is not proven, refuse. + +### Half-as-many requires source binding + +```text +"Natalia sold 48 clips in April. She sold half as many in May. How many altogether?" +``` + +`half` is not merely `current_result *= 0.5`; it creates a state for May by copying from a bound prior April state, then aggregate target sums April+May. + +### Rate conversion requires dimensional binding + +```text +"Weng earns $12 an hour. She babysat 50 minutes. How much did she earn?" +``` + +`minutes` and `hour` require unit conversion only because the rate denominator is hour and the worked duration is minutes. No global minutes/hour rule should fire without a rate frame. + +--- + +## 9. Bottom line + +The RB-GSM sketch strengthens the ADR-0184 direction but does not replace CORE's safety discipline. + +Use: + +```text +loose signal collection +cue dictionaries +state machine +operation/transition replay +simple arithmetic execution +``` + +Do not use: + +```text +unproven 100% claims +last-actor commits +first-rule-wins execution +regex as final meaning +keyword hits that bypass verification +``` + +CORE's version should be: + +```text +loose lexeme signals + -> cue-table semantic frame proposals + -> scoped state ledger + -> GroundedDerivation replay + -> existing verifier/classifier + -> pooled disagreement/commit eligibility +``` + +That keeps the useful classic rule-based insight while preserving the project doctrine: deterministic, auditable, refusal-first, and wrong=0-safe. diff --git a/docs/handoff/SEMANTIC-STATE-TRANSITION-BLUEPRINT.md b/docs/handoff/SEMANTIC-STATE-TRANSITION-BLUEPRINT.md new file mode 100644 index 00000000..914ff568 --- /dev/null +++ b/docs/handoff/SEMANTIC-STATE-TRANSITION-BLUEPRINT.md @@ -0,0 +1,1196 @@ +# Semantic State Transition Blueprint + +**Status:** development blueprint / handoff document +**Branch:** `docs/semantic-state-transition-blueprint` +**Scope:** docs only; no runtime code changes +**Audience:** lead engineer / reviewer / future ADR author +**Purpose:** define how to fit a scoped semantic-state-transition reader into the existing CORE math derivation lane without weakening `wrong = 0`. + +--- + +## 0. Executive decision + +CORE should not add another parser or another arithmetic composer as the next architectural move. + +The next move is to promote the already-proven semantic transition behavior emerging in `generate/derivation/accumulate.py` into a small, sealed, reusable substrate: + +```text +text + -> lexeme extraction + clause segmentation + -> semantic frames + -> scoped entity-owned state transitions + -> semantic world / ledger candidates + -> GroundedDerivation replay + -> existing self-verification / classification + -> cross-composer pool + -> answer or refusal +``` + +The system already contains the pieces, but they are spread across composer-local helpers: + +- `generate/derivation/extract.py` lifts quantities and units. +- `generate/derivation/clauses.py` provides sentence-level clause segmentation and local clause results. +- `generate/derivation/accumulate.py` implements the first real state-transition reading: single-referent gain/loss accumulation. +- `generate/derivation/target.py` contains early question-target and temporal-scope guards. +- `generate/derivation/verify.py` owns the proof gate. +- `generate/derivation/pool.py` owns cross-composer disagreement and commit eligibility. +- `generate/comprehension/state.py` contains older, broader immutable reader-state types, held hypotheses, reader refusals, and canonical hashing. + +The blueprint recommendation: + +> Add a sealed derivation-lane semantic-state package, initially backed by accumulation only, that emits the same `GroundedDerivation` candidates the current gate already knows how to verify. + +This gives CORE a real place for entity scope, question scope, temporal scope, state mutation, and later transfer/rate/comparison/DAG logic without creating composer spaghetti. + +--- + +## 1. Why this is needed + +### 1.1 The present failure mode + +Current successful logic is mostly candidate arithmetic plus a strong verifier. That is good, but it is not enough. The system needs a pre-arithmetic reading step that understands: + +- who owns a quantity; +- whether a quantity is an initial state, delta, scalar, rate, comparison, distractor, or target; +- whether a later clause continues the same referent or introduces a new actor; +- whether a cue licenses gain, loss, multiplication, division, comparison, or no operation; +- whether the question asks for final state, prior state, difference, total, rate, or another relation; +- whether quantities are genuinely relevant or isolated foreign distractors; +- whether the derivation is a linear chain or needs branching / reuse. + +The current local composers encode some of that, but not as a shared object language. + +### 1.2 The exact architectural gap + +`GroundedDerivation` is intentionally small: + +```text +start Quantity + ordered Step tuple +``` + +That is the right arithmetic proof object. It is not the right semantic reading object. + +The missing layer is a semantic object language that can represent: + +```text +Sam.apples = 14 +Sam.apples += 9 +question target = final Sam.apples +``` + +before replaying the arithmetic as: + +```text +14 + 9 = 23 +``` + +### 1.3 Why now + +This is the right time because the repository has already proven three supporting facts: + +1. `accumulate.py` shows that semantic state transitions can flip useful cases while preserving refusal-first guards. +2. `pool.py` shows that multiple readings should be pooled and arbitrated by disagreement instead of composer priority. +3. `verify.py` is strong enough to remain the final commit gate; the semantic layer can stay as candidate generation. + +If we keep extending one composer at a time, the same ideas will be reimplemented repeatedly: referent binding, cue polarity, target scoping, temporal refusal, distractor classification, and eventually DAG handling. That is the path to spaghetti. + +--- + +## 2. Non-negotiable constraints + +### 2.1 Serving stays untouched + +The semantic-state substrate must live in the sealed derivation/practice lane first. + +Do not import it from: + +- `chat/**` +- serving response generation +- runtime surface selection +- shared grounding primitives unless separately gated + +The first implementation should preserve the current two-regime contract: + +```text +serving: unchanged +practice/confuser lanes: allowed to attempt, measure, and eliminate +``` + +### 2.2 Existing verifier remains authoritative + +Do not replace `generate/derivation/verify.py`. + +The semantic layer should produce candidates and refusal reasons. It should not become a second answer gate. + +Commit path should remain: + +```text +semantic candidate + -> GroundedDerivation replay + -> classify_derivation / self_verifies + -> pool uniqueness / disagreement + -> commit only if complete and unique +``` + +### 2.3 No hidden best-guessing + +The semantic layer must refuse rather than infer silently when: + +- referent binding is ambiguous; +- a pronoun has multiple possible antecedents; +- gain/loss polarity is ambiguous; +- a new actor appears in a single-referent chain; +- a temporal target is unsupported; +- a question target cannot be bound; +- more than one semantic world survives without eliminating evidence. + +### 2.4 No grammar-template backslide + +Keep ADR-0165 discipline: + +- lexeme-level extraction is allowed; +- closed cue sets are allowed; +- deterministic clause/sub-clause splitting is allowed when scoped and tested; +- broad sentence-template parsing is not allowed. + +This substrate should be a typed transition interpreter, not a new regex grammar parser. + +### 2.5 Dead-code removal is part of the plan + +Any path made obsolete by the semantic-state substrate must be marked and later removed. Do not leave parallel, inert, or duplicate readers unless explicitly retained as offline baselines. + +--- + +## 3. Current repository map + +### 3.1 `generate/derivation/extract.py` + +Current role: + +- lexeme-level quantity extraction; +- word numbers; +- list-unit inheritance; +- sentence-final numbers; +- unit hygiene; +- hyphen-bonded number-units; +- intentionally deferred multi-word-unit handling. + +Future role: + +- remains lexeme lifting only; +- should not learn semantic roles like initial state, delta, target, actor, or temporal scope; +- feeds semantic frame construction. + +Keep. + +### 3.2 `generate/derivation/clauses.py` + +Current role: + +- sentence-level splitting; +- local clause result calculation; +- local ambiguity holds/refuses. + +Future role: + +- remains the default deterministic clause stream; +- semantic-state package can consume `segment_clauses()`; +- local sub-clause splitting can exist in semantic-state modules only when tightly scoped and tested. + +Keep, but do not overload it with semantic roles. + +### 3.3 `generate/derivation/accumulate.py` + +Current role: + +- first real state-transition composer; +- anchor state + gain/loss changes; +- referent guard; +- polarity classification; +- cue selection; +- distractor skip / anchor skip candidates for pooling. + +Future role: + +- should become a thin public composer facade; +- semantic logic should move into reusable state modules; +- public functions should remain stable initially: + - `compose_accumulation(problem_text)` + - `accumulation_candidates(problem_text)` + +Keep, but slim. + +### 3.4 `generate/derivation/compose.py` + +Current role: + +- same-unit list-sum/comparative-scale slice; +- clause-local guard after earlier whole-problem hazards. + +Future role: + +- eventually should emit or consume semantic frames for list aggregation and comparative scaling; +- should not independently grow a second referent/temporal/question model. + +Keep, but prevent further semantic accretion without using the shared substrate. + +### 3.5 `generate/derivation/search.py` and `multistep.py` + +Current role: + +- multiplicative product candidates; +- target-guided bounded chain candidates; +- blunt but useful candidate sources; +- intentionally wrong=0-safe through verification and pooling. + +Future role: + +- remain candidate sources; +- over time, semantically grounded rate/product frames should reduce dependence on product-of-all shapes; +- do not delete until semantic replacements are measured and gated. + +Keep, but expect demotion. + +### 3.6 `generate/derivation/target.py` + +Current role: + +- question quantities; +- aggregation hints; +- asked-unit intersection; +- prior-state question guard. + +Future role: + +- become a lexeme-level target extractor used by semantic `QuestionTarget`; +- temporal target scope should move into the semantic-state target model. + +Keep, but wrap. + +### 3.7 `generate/derivation/verify.py` + +Current role: + +- operand grounding; +- cue grounding; +- unit consistency; +- divide-by-zero; +- completeness; +- commit eligibility classification; +- uniqueness via `select_self_verified`. + +Future role: + +- unchanged final gate; +- semantic-state layer should add preconditions, not replace this. + +Keep as authoritative. + +### 3.8 `generate/derivation/pool.py` + +Current role: + +- union candidate readings across composers; +- classify as complete/exempt/invalid; +- refuse on disagreement; +- commit only complete, unique answers. + +Future role: + +- should become the stable arbitration seam; +- semantic-world candidates should enter through this path. + +Keep as the integration point. + +### 3.9 `generate/comprehension/state.py` + +Current role: + +- older broad reader-state substrate; +- immutable dataclasses; +- entity/quantity/question/refusal/hypothesis types; +- canonical bytes / hashes; +- still useful as a design precedent. + +Future role: + +- do not wire directly into scoring yet; +- reuse discipline and possibly some types if import direction is clean; +- avoid resurrecting old all-or-nothing / ambiguous pronoun hazards. + +Keep, but treat as adjacent / partially inert. + +### 3.10 Potential dead weight / caution zones + +The following should be reviewed for eventual retirement or demotion: + +1. **Old lifecycle reader runtime assumptions** + The amended ADR-0174 notes that `lifecycle.py` is not the reader to promote if it admits `0/50` and remains inert relative to the scoring path. Do not build the semantic-state transition system by trying to revive that whole path wholesale. + +2. **Legacy parser runtime paths** + If any legacy parser path remains in runtime scoring, it should be removed only after a lane-SHA and wrong=0 proof. If it is already only an offline baseline, leave it alone until the cleanup phase. + +3. **Composer-local semantic helpers** + Helpers like referent binding, polarity classification, cue selection, temporal target guards, and foreign-distractor logic should not keep spreading across composers. Extract once. + +4. **Priority-ordered composer resolution** + Any runner or scorer that picks the first non-`None` composer result is suspect. Pooling should be preferred wherever multiple readings are possible. + +--- + +## 4. Target architecture + +### 4.1 New package + +Add: + +```text +generate/derivation/state/ + __init__.py + model.py + frames.py + bind.py + change.py + target.py + ledger.py + replay.py + refusals.py +``` + +This package is derivation-lane scoped. It must not be imported by serving. + +### 4.2 Model objects + +Initial minimal model: + +```python +@dataclass(frozen=True, slots=True) +class EntityMention: + surface: str + canonical: str + clause_index: int + token_index: int + kind: str # "proper", "pronoun", "implicit", "unknown" + +@dataclass(frozen=True, slots=True) +class QuantityMention: + value: float + unit: str + source_token: str + clause_index: int + role: str # "state", "delta", "scalar", "rate", "target", "unknown" + +@dataclass(frozen=True, slots=True) +class CueMention: + surface: str + cue_kind: str # "gain", "loss", "aggregate", "multiplicative", "temporal", ... + clause_index: int + +@dataclass(frozen=True, slots=True) +class StateKey: + entity: str + unit: str + +@dataclass(frozen=True, slots=True) +class StateTransition: + key: StateKey + op: str # "set", "gain", "loss" + quantity: QuantityMention + cue: CueMention + clause_index: int + +@dataclass(frozen=True, slots=True) +class SemanticLedger: + transitions: tuple[StateTransition, ...] + +@dataclass(frozen=True, slots=True) +class SemanticWorld: + ledger: SemanticLedger + question_target: object | None + unresolved: tuple[str, ...] + refusal_reasons: tuple[str, ...] +``` + +The first version can keep some fields as closed strings rather than enums if that matches current style, but closed sets should live near the model and be tested. + +### 4.3 Public surfaces + +Initial public surfaces: + +```python +def accumulation_world_candidates(problem_text: str) -> tuple[SemanticWorld, ...]: ... + +def replay_world(world: SemanticWorld) -> GroundedDerivation | None: ... + +def semantic_state_candidates(problem_text: str) -> tuple[GroundedDerivation, ...]: ... +``` + +First implementation: + +```text +semantic_state_candidates(problem_text) == accumulation_candidates(problem_text) +``` + +Behavior should be equivalent before expanding scope. + +### 4.4 Integration flow + +Final intended flow: + +```text +resolve_pooled(problem_text) + -> pooled_candidates(problem_text) + -> semantic_state_candidates(problem_text) + -> multiplicative_candidates(problem_text) + -> candidate_chains(problem_text) + -> classify_derivation(...) + -> disagreement / commit eligibility +``` + +Initially, `pool.py` can keep calling `accumulation_candidates`; later swap to `semantic_state_candidates` after equivalence tests pass. + +--- + +## 5. Implementation phases + +## Phase S0 — ADR / blueprint ratification + +### Goal + +Convert this blueprint into an ADR or ADR-scope document before code begins. + +### Why + +This touches several ADR lines: + +- ADR-0174 held-hypothesis comprehension; +- ADR-0178 compositional structure; +- ADR-0182 pooling; +- ADR-0165 regex scope; +- ADR-0176/0177 derivation search and cue precision. + +A small ADR prevents future code from treating this as another local composer tweak. + +### Deliverable + +Suggested file: + +```text +docs/decisions/ADR-0183-scoped-semantic-state-transitions.md +``` + +If ADR numbering is already occupied, use the next available number. + +### Acceptance + +- ADR explicitly says serving remains untouched. +- ADR explicitly says semantic worlds replay to `GroundedDerivation` and use existing gates. +- ADR explicitly identifies `accumulate.py` as the first migration target. +- ADR explicitly forbids reviving the old lifecycle reader as a scoring path without a separate proof. + +--- + +## Phase S1 — Extract proven helpers without behavior change + +### Where + +From: + +```text +generate/derivation/accumulate.py +``` + +To: + +```text +generate/derivation/state/bind.py +generate/derivation/state/change.py +``` + +### What moves + +Move / rename: + +```text +_subject_token -> leading_subject_token +_same_referent -> continues_anchor_referent +_polarity -> classify_change_polarity +_cue -> select_change_cue +``` + +### Why + +These helpers are no longer accumulation-specific. They are the beginning of semantic reading. + +### How + +- Copy behavior exactly. +- Preserve ordering and cue selection. +- Preserve tests. +- Add direct unit tests for the new helper names. +- Keep `accumulate.py` public behavior unchanged. + +### Acceptance + +- `tests/test_adr_0178_gb3b1_accumulation.py` unchanged and green. +- `tests/test_adr_0182_pool.py` unchanged and green. +- No new imports from serving. +- No change to practice counts unless explicitly measured as byte-identical. + +### Dead-weight check + +If any old helper remains in `accumulate.py` after extraction, it should be a thin import alias or deleted. + +--- + +## Phase S2 — Add minimal semantic ledger for accumulation + +### Where + +New files: + +```text +generate/derivation/state/model.py +generate/derivation/state/ledger.py +generate/derivation/state/replay.py +``` + +### What + +Represent accumulation as: + +```text +SET_STATE(entity, unit, value) +GAIN(entity, unit, value) +LOSS(entity, unit, value) +``` + +### Why + +This is the first true transition layer. It decouples semantic reading from arithmetic replay. + +### How + +Internal flow for accumulation: + +```text +problem_text + -> quantity-bearing clauses + -> anchor state transition + -> change transitions + -> SemanticLedger + -> GroundedDerivation replay + -> existing select_self_verified / classify_derivation +``` + +`compose_accumulation()` should still return `Resolution | None`. + +`accumulation_candidates()` should still return `tuple[GroundedDerivation, ...]`. + +### Acceptance + +- Clean gain/loss fixtures still resolve. +- New actor still refuses. +- No-change-cue still refuses. +- Multi-change in one clause still refuses. +- Anchor must still be single quantity for strict accumulation. +- Distractor-skip and anchor-skip candidates still classify as expected under pooling. + +### Dead-weight check + +After this phase, direct manual construction of accumulation `GroundedDerivation` inside `accumulate.py` should be minimal or gone. Replay should own that. + +--- + +## Phase S3 — Add semantic question target wrapper + +### Where + +New file: + +```text +generate/derivation/state/target.py +``` + +Existing file remains: + +```text +generate/derivation/target.py +``` + +### What + +Wrap existing `Target` with semantic target fields: + +```text +entity: optional +unit: optional +time_scope: final | prior | unknown +relation: count | difference | aggregate | unknown +``` + +First implementation can be conservative: + +- detect prior-state question using existing `asks_prior_state()`; +- detect final/net questions only when safe; +- leave entity binding as unknown unless the question clearly names the anchor entity; +- refuse unsupported prior targets before replay. + +### Why + +Temporal and question-scope logic should not live in `pool.py` or in individual composers forever. + +### How + +Initially: + +```text +resolve_pooled() prior-state guard + -> semantic target refuses prior-state worlds before candidate replay +``` + +Keep old `asks_prior_state()` as a compatibility helper until all callers migrate. + +### Acceptance + +- Prior-state minimal pair still refuses: + - “How much did Lisa have before lunch?” refuses. +- Forward/net twin still resolves: + - “How much money does Lisa have left?” resolves. +- Body narrative “before” does not trip prior-state refusal. +- “used to make” false positive stays guarded. + +### Dead-weight check + +Once all prior-state checks route through semantic target, remove direct prior-state guard from `pool.py` or reduce it to a compatibility call. + +--- + +## Phase S4 — Introduce `semantic_state_candidates()` and pool integration + +### Where + +New public surface: + +```text +generate/derivation/state/__init__.py +``` + +Modified: + +```text +generate/derivation/pool.py +generate/derivation/__init__.py +``` + +### What + +Add: + +```python +def semantic_state_candidates(problem_text: str) -> tuple[GroundedDerivation, ...]: + ... +``` + +Initial implementation delegates to accumulation-backed worlds. + +Then change pool ordering from: + +```text +accumulation_candidates, multiplicative_candidates, candidate_chains +``` + +to: + +```text +semantic_state_candidates, multiplicative_candidates, candidate_chains +``` + +### Why + +`pool.py` should not need to know every semantic composer. It should ask for semantic-state readings as one candidate source. + +### How + +Perform this only after equivalence tests prove the candidate set is unchanged for existing fixtures. + +### Acceptance + +- `pooled_candidates()` de-duplicates as before. +- All ADR-0182 pool tests remain green. +- Clean accumulation still commits. +- Distractor cases still refuse through disagreement. +- Exempt-only still never commits. + +### Dead-weight check + +Once `pool.py` calls `semantic_state_candidates`, direct import of `accumulation_candidates` from `pool.py` should be removed. + +--- + +## Phase S5 — Add transfer events + +### Where + +```text +generate/derivation/state/transfer.py +``` + +or inside: + +```text +generate/derivation/state/change.py +``` + +if small. + +### What + +Support: + +```text +Sam gives Tom 3 apples. +``` + +as: + +```text +Sam.apples -= 3 +Tom.apples += 3 +``` + +### Why + +Transfer is the first multi-entity state transition. It should be implemented only after entity-owned ledgers exist. + +### How + +Rules: + +- require source entity; +- require target entity; +- require grounded quantity; +- require unit; +- require transfer cue; +- refuse if source/target ambiguous; +- refuse if question target does not identify which resulting state is requested. + +### Acceptance + +Tests: + +```text +Sam has 10 apples. Sam gives Tom 3 apples. How many apples does Sam have? -> 7 +Tom has 2 apples. Sam gives Tom 3 apples. How many apples does Tom have? -> 5 +Sam gives Tom 3 apples. How many apples does Sam have? -> refuse, no initial state +Sam gives Tom 3 apples. How many apples total? -> refuse until aggregate target exists +``` + +### Dead-weight check + +Do not patch transfer into `accumulate.py`. If transfer needs new helpers, they belong in the semantic-state package. + +--- + +## Phase S6 — Add comparison / difference frames + +### Where + +```text +generate/derivation/state/compare.py +``` + +### What + +Support safe cases: + +```text +Sam has 10 apples. Tom has 7 apples. How many more apples does Sam have than Tom? +``` + +as: + +```text +difference(Sam.apples, Tom.apples) = 10 - 7 +``` + +### Why + +Difference questions require target relation binding, not just state replay. + +### How + +Rules: + +- both entity states must be known; +- units must match; +- question must request difference / “more than” / “less than”; +- relation direction must be explicit; +- ambiguous direction refuses. + +### Acceptance + +- `Sam more than Tom` resolves `Sam - Tom`. +- `Tom fewer than Sam` resolves `Sam - Tom` only if direction is unambiguous. +- Unknown relation direction refuses. +- Same-unit aggregate question does not accidentally become difference. + +--- + +## Phase S7 — Add rate / container frames + +### Where + +```text +generate/derivation/state/rate.py +``` + +### What + +Support safe product structures: + +```text +24 boxes, 12 erasers each -> 24 * 12 erasers +60 miles per hour for 2 hours -> 60 * 2 miles +``` + +### Why + +This is how CORE should eventually distinguish legitimate multiplicative binders from distractor products. + +### How + +Rules: + +- rate must bind two dimensions; +- container count must bind to rate denominator; +- output unit must be rate numerator; +- unrelated foreign state quantities must not be consumed; +- ambiguous “for” adjuncts should remain candidates only if structurally bound. + +### Acceptance + +- Legitimate rate/container products commit. +- Distractor duration with unrelated target refuses via disagreement or lack of binding. +- Existing correct product cases do not regress. +- Product-of-all fallback remains until semantic rate coverage is measured sufficient. + +--- + +## Phase S8 — Add temporal target replay + +### Where + +```text +generate/derivation/state/time.py +``` + +### What + +Support: + +```text +before +initially +originally +at first +after +now +left +finally +``` + +as target scope over ledger history. + +### Why + +Current forward composers compute final/net state. Prior-state questions are correctly refused. The semantic ledger makes prior-state replay possible. + +### How + +Ledger replay must support: + +```text +state at transition index N +state before event E +state after event E +final state +initial state +``` + +### Acceptance + +- “How much did Lisa have before lunch?” returns the pre-spend value only when the event boundary is unambiguous. +- “How much does Lisa have left?” returns final/net state. +- Ambiguous temporal target refuses. + +--- + +## Phase S9 — Held semantic worlds and DAGs + +### Where + +```text +generate/derivation/state/world.py +generate/derivation/state/dag.py +``` + +### What + +Represent more than one possible semantic world, and eventually derivations where quantities are reused across branches. + +### Why + +Some GSM8K problems cannot be represented as one left-fold chain. They require branch/reuse structures. + +### How + +Only after earlier layers are stable: + +```text +SemanticWorld candidates + -> eliminate by constraints + -> if one survives, replay + -> if multiple survive and disagree, refuse + -> if no linear replay possible, emit DAG candidate behind a new gate +``` + +### Acceptance + +- Existing left-fold cases unchanged. +- DAG cases remain sealed until verifier supports them. +- No DAG candidate can commit without a completeness/grounding/target proof equivalent to current `GroundedDerivation` requirements. + +--- + +## 6. Testing strategy + +### 6.1 Unit tests + +Add tests under: + +```text +tests/test_semantic_state_*.py +``` + +or ADR-numbered tests once an ADR exists. + +Required categories: + +- model validation; +- referent binding; +- cue polarity; +- semantic ledger construction; +- replay to `GroundedDerivation`; +- prior-state refusal; +- pool equivalence; +- deterministic replay. + +### 6.2 Regression tests + +Do not weaken existing tests: + +- `tests/test_adr_0178_gb3b1_accumulation.py` +- `tests/test_adr_0182_pool.py` +- `tests/test_adr_0175_phase3b_mult_search.py` +- `tests/test_adr_0177_cp2a_training.py` + +### 6.3 Lane tests + +Every implementation PR must state whether it affects: + +- serving; +- sealed practice; +- confuser probe; +- train_sample; +- cue precision reports. + +Default for early phases: + +```text +serving: unchanged +practice: equivalent until new semantic capability phase +confuser: equivalent until pool candidate source changes +``` + +### 6.4 Determinism tests + +Every semantic object must either: + +- be frozen and directly comparable, or +- expose canonical bytes / canonical hash. + +Avoid unordered set iteration in emitted outputs. Sets may be used only for boolean membership or cardinality checks, never output ordering. + +--- + +## 7. Documentation strategy + +### 7.1 ADR sequence + +Suggested: + +1. `ADR-0183` — scoped semantic-state transitions. +2. `ADR-0183.S1` — accumulation extraction/refactor into semantic-state substrate. +3. `ADR-0183.S2` — semantic target wrapper. +4. `ADR-0183.S3` — transfer events. +5. `ADR-0183.S4` — comparison/difference frames. +6. `ADR-0183.S5` — rate/container frames. +7. `ADR-0183.S6` — temporal replay. +8. `ADR-0183.S7` — semantic-world hypotheses / DAGs. + +### 7.2 Required ADR language + +Each ADR must include: + +- why this is not a regex grammar template; +- how it preserves wrong=0; +- which composer-local logic it replaces; +- which tests fail if the guard is removed; +- what remains out of scope; +- dead-code/deprecation notes. + +--- + +## 8. Dead weight / cleanup plan + +### 8.1 Composer-local semantic helpers + +After Phase S1/S2, the following should not remain as private accumulation-only concepts: + +- subject-token extraction; +- same-referent check; +- change polarity; +- cue selection; +- anchor-state construction; +- state-change replay. + +They should live in semantic-state modules. + +### 8.2 Direct prior-state guard in pool + +`pool.py` currently performs a prior-state guard because forward composers cannot compute prior target scope. Once semantic targets own temporal scope, this guard should move out of `pool.py`. + +### 8.3 Old broad comprehension lifecycle + +Do not delete casually. But do not treat it as the active path to promote unless a separate audit proves it has become load-bearing. + +Potential future state: + +```text +generate/comprehension/state.py keep / share types / canonical hashing +generate/comprehension/lifecycle.py retire or demote if still inert +generate/math_parser.py baseline only, eventual delete after gate +generate/math_candidate_graph.py thin dispatcher / eventual simplification +``` + +### 8.4 Product-of-all fallback + +Do not delete early. It still protects known product cases. But semantic rate/container frames should eventually reduce reliance on blunt product-of-all candidate generation. + +Retirement condition: + +- semantic rate/container frames cover protected positives; +- confuser probe remains wrong=0; +- train_sample protected correct cases do not regress; +- cue precision report confirms better candidate readings. + +--- + +## 9. Risks and mitigations + +### Risk 1 — resurrecting old pronoun hazards + +Mitigation: + +- do not use gender-blind most-recent antecedent as a resolver; +- new actor or multiple possible actors should refuse; +- same-referent continuation must be tested with minimal pairs. + +### Risk 2 — semantic layer bypasses verifier + +Mitigation: + +- semantic worlds emit `GroundedDerivation`; +- verifier remains authoritative; +- no semantic world commits directly. + +### Risk 3 — hidden composer priority + +Mitigation: + +- pool candidates across composers; +- refuse on disagreement; +- do not choose first non-`None` where multiple readings exist. + +### Risk 4 — grammar-template creep + +Mitigation: + +- closed lexeme sets only; +- scoped clause splitting only; +- every multi-token cue must be documented as a cue phrase, not a sentence template; +- no broad natural-language parse regexes. + +### Risk 5 — line-count growth without payoff + +Mitigation: + +- every semantic-state phase must identify what it makes obsolete; +- refactor phases must be behavior-equivalent; +- capability phases must show either correct-count increase or wrong-count decrease/refusal improvement. + +--- + +## 10. Recommended immediate next PR + +Branch: + +```text +feat/adr-0183-semantic-state-accumulation-substrate +``` + +Scope: + +1. Add ADR-0183. +2. Add `generate/derivation/state/__init__.py`. +3. Add `generate/derivation/state/bind.py`. +4. Add `generate/derivation/state/change.py`. +5. Move proven helper logic from `accumulate.py` into those modules. +6. Keep `compose_accumulation()` and `accumulation_candidates()` behavior unchanged. +7. Add tests for extracted helpers. +8. Do not touch serving. +9. Do not change runners. +10. Do not delete old reader files yet. + +Acceptance: + +```text +existing accumulation tests pass +existing pool tests pass +no serving imports +no behavior change +new helper tests prove referent/polarity guards are non-vacuous +``` + +--- + +## 11. Final doctrine + +CORE should read word problems as scoped semantic state, not as number bags. + +The correct internal progression is: + +```text +lexemes + -> frames + -> entity-bound state transitions + -> question-targeted ledger replay + -> arithmetic proof object + -> existing verifier + -> pooled uniqueness/disagreement +``` + +`accumulate.py` is the proof that this works. `pool.py` is the proof that competing readings should be arbitrated by disagreement. `verify.py` is the proof that wrong=0 can remain the floor. + +The next engineering task is to stop letting those ideas remain composer-local and promote them into a clean semantic-state substrate before transfer, comparison, rate, temporal, and DAG logic arrive.