diff --git a/docs/research/compare-increment-funnel-2026-07-18.md b/docs/research/compare-increment-funnel-2026-07-18.md new file mode 100644 index 00000000..aaecec0f --- /dev/null +++ b/docs/research/compare-increment-funnel-2026-07-18.md @@ -0,0 +1,224 @@ +# compare_multiplicative Increment — Per-Layer Attrition Funnel (diagnostic artifact) + +**Status**: DIAGNOSTIC — instrument-first, decides the fix order (PR #76 ruling) +**Date**: 2026-07-18 +**Scope**: the 61 compare-multiplicative-marked cases in the **tune** split of official `holdout_dev/v1` +(measure split untouched). Signals are the reader's own: `branches_enumerated`, `refusal_reason`, +`selected_graph`, then the corridor compiler. + +--- + +## The funnel + +Reader pipeline stages: recognizer-match → **injection** → enumeration → admissibility → graph → solve. + +| Layer | Cases (of 61) | +| :--- | ---: | +| Died **pre-enumeration** | **57** | +| Enumerated | 4 | +| …enumerated but no graph | 1 | +| …graph has a compare op | 3 | +| **Solved today** | **3** | + +Pre-enumeration refusal reasons (the 57): + +| Reason | Cases | +| :--- | ---: | +| **"recognizer matched but produced no injection"** | **45** | +| "no admissible candidate for statement" | 8 | +| "expected exactly one question sentence" | 3 | +| "no admissible candidate for question" | 1 | + +## The finding + +**The leverage is the injection layer: 45 / 61.** The recognizer already *matches* these compare +statements (e.g. 0000: "Aria has twice as many … as Emily, who has twice the number … as Spencer"); +the **injection** step then produces no candidate — the multi-clause / compare shapes don't inject. +This is where the mass is — not the candidate regexes (they fire on clean clauses but the real +sentences never reach them), and not segmentation writ large. + +## The "no injection" is intentional — the reader was waiting on the compiler + +Code archaeology (before treating "no injection" as a defect) found the cause is a **deliberate +narrowing**, not broken code. In `generate/recognizer_anchor_inject.py`, the discrete-count injector +carries explicit fail-closed guards for comparative surfaces: + +- Line ~324: *"…it is an incomplete comparative-multiplicative clause. Letting this through as an + initial … defeats the ADR-0191 completeness guard. **Refuse here until a real compare_multiplicative + operation can be emitted.**"* +- `_count_token_followed_by_times` (~460): *"it only suppresses the malformed initial candidate and + **does not create any new admitting path**."* + +This is the **lockstep thesis of the arc written into the code's own history**: the reader +deliberately *refused* compare shapes — to protect `wrong=0` and completeness — because nothing +downstream could compile them. **The compiler tier this increment just shipped is exactly that +"real compare_multiplicative operation."** So the fix is not repairing injection; it is **lifting a +now-obsolete guard whose reason has shipped** — turning the deliberate suppression into a compare +emission — which is a cleaner, safer change than patching around it. + +## Consequence for the fix (emitter-fix, not gate-loosening) + +Injection is an **emitter** (it produces candidates), not a validator — so making it inject +compare candidates for statements the recognizer already matches is the sanctioned kind of change +(fix emitters; never loosen the round-trip gate). The remaining piles are secondary and sequenced +after: "no admissible candidate for statement" (8), the one-question-sentence constraint (3). + +Fix order, by measured mass: **injection (45) → statement-admissibility (8) → question-sentence (3)**. +Each fix is developed against tune only, to `wrong=0` on tune; the measure split gets its single run +at the end. Refusal-by-reason on the measure split (frequency / nested / anaphora / temporal / +inverse-undeterminable) is reported with the delta as the fail-closed evidence + the practice-lane +curriculum. + +## Shared-layer reality — the whole-tune refusal histogram (pinned baseline) + +Across ALL 261 tune cases (not just compare-marked), the reader's refusal-reason histogram — pinned +**before** any shared-layer edit, asserted **stable-or-explained** after: + +| Refusal reason | Cases | +| :--- | ---: | +| **no injection** | **196** | +| no admissible candidate for statement | 48 | +| expected exactly one question | 9 | +| no admissible candidate for question | 4 | +| **PARSED** (all compare) | **3** | +| no branch produced a solvable | 1 | + +**"no injection" is 196/261 (75%) — a *shared* bottleneck across every family, not a compare defect.** +The reader recognizes statements but can't inject candidates for three-quarters of real GSM8K. A +guard-lift that silently changed *why* other cases refuse would corrupt the practice-lane curriculum +even with zero parse breakage, so the histogram is pinned; the only sanctioned post-edit delta is +compare cases moving from "no injection" → PARSED (or to a downstream layer), disclosed explicitly. + +## Funnel-conditioned escalation rule (PRE-COMMITTED, before the number exists) + +X = 15 was ratified assuming *family capability* was the wall. The baseline shows the wall may be the +**shared pipeline** (injection / enumeration / admissibility / round-trip) that every family crosses. +So a miss on X is read through the funnel: + +- **Mass dying in compare-specific logic** → the family's design well is dry → **practice-lane ruling** + (as designed). +- **Mass dying in a shared layer** → the next increment is a **shared-layer design increment**, *not* + a practice escalation. X is re-applied after the shared layer is fixed. + +Load-bearing reason: **the practice lane cannot learn past a deterministic wall.** A practice loop +generates better parse *candidates*, but they cross the same enumeration/admissibility/round-trip +pipeline; if a hard-coded shared layer refuses everything, practice fails identically — escalating into +a mechanism that can't help. Shared-layer bottlenecks are design work regardless of what X says. + +## Reframe (recorded) + +Compare is the **first slice of a much larger reader project**: the reader parses ~1% of real GSM8K +(3/261 tune), and "no injection" (75%) is a shared wall in front of every family. This is not scope +creep — it is the arc seeing its true size. The increments stay small and measured *precisely because* +the project is large. + +--- + +## MEASURED OUTCOME — the reader half (frame-anchored recognizer) + +> **STATUS: BUILT + MEASURED, then DEFERRED out of increment 1 (PR #77 ruling).** The code change +> (`c3aed13b`, frame-anchored recognizer) was reverted from the merged increment; only the compiler +> tier + this finding land. Frame-anchoring is now an **open design task owned by the rate-frame +> increment** — see *Deferral ruling* below. This section is the on-record measurement that motivated +> the split. + +### What was built and measured +Per the construction constraint (anchor on the comparative FRAME, not verb-whitelist growth), the two +multiplicative-frame regexes (`_COMPARE_MULT_ANCHOR_RE`, `_COMPARE_MULT_NTIMES_RE`) were re-anchored: +the closed-class frame — ` as many/much as ` — is the gate, the verb slot became a +verb-free bounded connector, and `as much` joined `as many`. `_comparison_anchor_verb()` (the whitelist) +is left in place for the ADDITIVE / NESTED frames (out of scope). Both frame regexes remain `$`-anchored, +so multi-clause / nested surfaces stay refused by construction. + +### The polarity refutation — the real lesson +Pure verb-free anchoring is **not** wrong=0-safe: a polarity-inverting / depletion verb inside the frame +("Alice **lost** twice as many apples as Bob") makes the compared quantity a loss, so +`actor = factor × reference` reads the relation backwards → wrong>0. The frame does NOT disambiguate this +(pinned by `test_adr_0131_G2a…::test_polarity_inverting_verb_not_admitted`, `…spend…`, +`test_recognizer_comparative_inject…[Alice lost…]`). The prototype fix was a closed-class polarity +**blocklist** the connector may not span — but the ruling named the deeper problem this exposes, and it +is decisive: **the verb carries a polarity bit that must be positively determined, not +assumed-safe-unless-blocklisted.** A blocklist is fail-*open* — it admits every unenumerated verb, so a +depletion verb nobody listed reaches a *complete* frame as a coherent-but-inverted graph: a **wrong** on +the run-once sealed arbiter (the round-trip net guards parse consistency, not gold-correctness). The +fail-closed doctrine never takes that trade — least of all for the **zero** measured benefit below. So +the wrong=0-safe resolution is genuine design work: frame **plus positive polarity determination** (a +neutral-polarity *allowlist*, or a small polarity classifier), exercised end-to-end where it earns its +keep — the rate-frame increment — not bolted on here. + +### Footprint — provable, tiny, safe +Develop-on-tune / measure-once discipline held. A new-vs-old frame-fire diff over all 500 official cases: + +| Split | cases | divergent frame-fires | wrong | PARSED delta | +| :--- | ---: | ---: | ---: | ---: | +| tune | 261 | **0** (byte-identical histogram) | 0 | 0 | +| measure | 239 | **1** — `0441` "Zig **wrote** four times as many books as Flo." | 0 | 0 | + +`0441` is the exact archetype (verb-blocked-only, single-clause, frame-at-end). Post-change its frame +fires correctly — and the case **still refuses**, one layer downstream: *"no admissible candidate for +question: 'how many books did they write altogether?'"*. It advanced frame-refusal → **question-refusal** +(the pre-registered sanctioned delta), confirming the frame now reads but the binding wall is the +**shared summation-question** layer. Same wall killed `0228` ("…how many boxes in both warehouses +combined?"). Smoke gate: **176 passed**. + +### X = 15 — MISSED, delta = 0 (and why it's a shared-layer miss, not a family-exhaustion miss) +The 38 tune compare-family cases, classified by their **actual** structural blocker (not the noisy +surface histogram): 16 multiclause / nested (`…as Emily, who has…`, `…as Shawna, but 3 fewer…`), +10 single-clause-frame-no-match (multi-word "the X Y" references, `than` vs `as`, decimal factors, +unit-ellipsis "as much as", subject-is-comparison), 5 "times the number/amount of" (self-referential / +possessive references), 5 "as ADJ as" (attribute register), 2 "double what / triple". End-to-end traces +of the cleanest candidates (`0228`, `0274`, `0491`, `0165`, `0299`) show **every one dies at a +non-compare layer**: summation-question reading, seeding-sentence injection (shared discrete-count), +additive-compare, attribute registers, simultaneous equations. Even the cleanest compare-family frame +extension (multi-word reference) converts **0** cases end-to-end — and introduces an actor-misbinding +hazard — so it was reverted. + +**Ruling per the pre-committed funnel-conditioned escalation rule:** the mass is dying in **shared +layers**, not compare-specific logic ⇒ the next increment is a **shared-layer design increment**, NOT a +practice-lane escalation, and NOT more compare-family frame surgery. **Shay's ruling (#77): next = +seeding-sentence injection** — the 135/261 "no injection (discrete_count)" wall, the dominant shared +bucket that gates every family (production/possession verbs like *made/baked/grew/wrote* that don't +inject an initial state). The compare-specific unblock (summation-question reader + **inverse compare** — +known=actor, solve reference; `0441` needs both) stays queued behind it. X is re-applied *after* the +shared layer lands. + +### Triple report +- **correct**: measure new-correct = **0**. The compiler tier solves the compare cases the reader can + already reach (`0101` "double what", `0108` "times the number of", `0411`) — all **pre-existing**, not + new from the frame-anchoring. +- **refused-by-reason (traps broken out)**: polarity-inverting (lost/spent) → refused fail-closed by the + `_comparison_anchor_verb()` whitelist (the merged state, after the frame-anchoring revert; the + blocklist prototype that would have admitted-then-guarded them is deferred with the frame change). + Family funnel: 16 multiclause/nested, 10 frame-no-match, 5 number/amount-of, 5 as-ADJ, 2 double-what — + all refuse-and-record. +- **wrong = 0**: held on tune (261) **and** measure (239) **and** smoke (176). No over-admission from + the verb-free widening; no branch-disagreement regression (0 tune divergences). + +### Deferral ruling (#77) — the split +Increment 1 merges the **compiler tier + this finding**; the **frame-anchoring reader change is held +out** (`c3aed13b` reverted from the PR). Three compounding reasons, on record: + +1. **Zero measured benefit today** — 0 conversions across all 500 official cases. +2. **Its consumer is a later increment** — frame-anchoring exists to serve rate-frame admission + (increment 2). The *next* increment is seeding-sentence injection, a shared layer that does not touch + it. Nothing on the critical path needs it now. +3. **Fail-open surface at the sealed test for that zero benefit** — the polarity blocklist admits every + unenumerated verb; an unlisted depletion verb in a complete frame is a coherent-but-inverted graph = + a wrong on the run-once arbiter. The fail-closed doctrine never risks that, least of all for no gain. + +Verified before merge: with the whitelist reader restored, the 5 real compare parses are **non-divergent** +— `test_real_compare_cases_solved_wrong_zero` still sees **5, solves 5, wrong 0** (corridor real-reach +`0→5/500` preserved). The tier keeps its win without the frame change. + +### OPEN DESIGN TASK (owned by the rate-frame increment) +**Frame-anchored comparative admission with wrong=0-safe polarity determination.** The frame is the right +gate; the open problem is the verb's polarity bit. It must be **positively determined**, not +assumed-safe-unless-blocklisted: + +- a **neutral-polarity allowlist** (fail-closed: only verbs affirmatively known non-inverting admit), or +- a small **polarity classifier / lexicon** (possession/production/acquisition = neutral → `factor×ref`; + depletion/transfer/loss = inverting → refuse or model as a delta). + +Deliverable when the rate-frame increment lands it: exercised end-to-end on rate frames, measured on the +disjoint split, wrong=0 preserved on the sealed arbiter. Until then the reader stays on the +`_comparison_anchor_verb()` whitelist (fail-closed) for all comparative frames. diff --git a/evals/gsm8k_math/holdout_dev/v1/split.py b/evals/gsm8k_math/holdout_dev/v1/split.py new file mode 100644 index 00000000..31c9f4ca --- /dev/null +++ b/evals/gsm8k_math/holdout_dev/v1/split.py @@ -0,0 +1,25 @@ +"""Deterministic tune / measure split of the official holdout_dev/v1 (ADR-0250 reader arc). + +Pinned **before any grammar work** (compare increment plan §5, AMENDMENT 3): the reader grammar +family is developed against the **tune** split only; every increment's coverage delta is measured on +the disjoint **measure** split only. The assignment is a pure function of the case id, so it is +reproducible and cannot drift — disjointness is enforced by construction, not by discipline. + +Rule (fixed 2026-07-18, never re-seeded): a case is ``measure`` iff ``sha256(id)`` is even, else +``tune`` (≈ 50 / 50). `wrong=0` is verified across the **full 500** regardless of split; only the +*coverage delta* is attributed to the measure split. +""" +from __future__ import annotations + +import hashlib + +__all__ = ["split_of", "TUNE", "MEASURE"] + +TUNE = "tune" +MEASURE = "measure" + + +def split_of(case_id: str) -> str: + """Return ``"tune"`` or ``"measure"`` for ``case_id`` — deterministic, id-only.""" + digest = int(hashlib.sha256(case_id.encode("utf-8")).hexdigest(), 16) + return MEASURE if digest % 2 == 0 else TUNE diff --git a/evals/multi_register_program.py b/evals/multi_register_program.py index 372f3c35..8878dc33 100644 --- a/evals/multi_register_program.py +++ b/evals/multi_register_program.py @@ -48,7 +48,7 @@ from core.physics.quantity_kernel import ( translate_quantity, ) from core.ports.residual_protocol import GENESIS_DIGEST -from generate.math_problem_graph import MathProblemGraph, Quantity +from generate.math_problem_graph import Comparison, MathProblemGraph, Quantity __all__ = [ "MultiRegisterError", @@ -63,6 +63,7 @@ __all__ = [ _AFFINE = frozenset({"add", "subtract", "multiply", "divide"}) _CONSERVATION_RTOL = 1e-6 +_COMPARE_MULT = "compare_multiply" # internal turn kind for actor := factor × reference class MultiRegisterError(ValueError): @@ -96,7 +97,8 @@ def _state_digest(psi: np.ndarray) -> str: @dataclass(frozen=True, slots=True) class RegisterTurn: - """One program step: a per-register affine op or a coupled transfer.""" + """One program step: a per-register affine op, a coupled transfer, or a + cross-register comparison (``actor := factor × reference``).""" kind: str actor: str @@ -104,6 +106,7 @@ class RegisterTurn: offset: float operand: float target: str | None = None + reference: str | None = None # compare_multiply: the source register read from @dataclass(frozen=True, slots=True) @@ -112,6 +115,8 @@ class MultiRegisterProgram: turns: tuple[RegisterTurn, ...] answer_entity: str | None # None ⇒ certified summation over all registers answer_unit: str + # Deterministic register order for summation: seed order, then compare-defined order. + register_order: tuple[str, ...] = () @dataclass(frozen=True, slots=True) @@ -124,7 +129,8 @@ class MultiRegisterRecord: converged: bool step: tuple[str, float, float] # (kind, scale, offset) prev_record_digest: str - operand_source_digest: str = "" # summation only: binds the operand to its source state + operand_source_digest: str = "" # summation/compare: binds the operand to its source state + source_entity: str = "" # compare only: the reference register the dilation read from def _payload(self) -> dict: payload = { @@ -135,9 +141,12 @@ class MultiRegisterRecord: "step": [self.step[0], repr(float(self.step[1])), repr(float(self.step[2]))], "prev_record_digest": self.prev_record_digest, } - # Present only for summation turns, so non-summation record digests are unchanged. + # Each field is present only when set, so affine/transfer/summation record + # digests are byte-identical to before compare landed. if self.operand_source_digest: payload["operand_source_digest"] = self.operand_source_digest + if self.source_entity: + payload["source_entity"] = self.source_entity return payload def record_digest(self) -> str: @@ -171,21 +180,48 @@ def compile_multi_register_program(graph: MathProblemGraph) -> MultiRegisterProg raise MultiRegisterError("no_initial_state") seeds: list[tuple[str, float]] = [] units: dict[str, str] = {} + order: list[str] = [] # deterministic register order: seeds first, defined later for possession in graph.initial_state: entity = possession.entity if entity in units: raise MultiRegisterError("duplicate_initial_entity", entity=entity) units[entity] = possession.quantity.unit + order.append(entity) seeds.append((entity, float(possession.quantity.value))) - # A concrete unknown decodes that register; a None unknown ("altogether") is - # the explicit signal for a certified summation over all registers. - answer_entity = graph.unknown.entity - if answer_entity is not None and answer_entity not in units: - raise MultiRegisterError("unknown_entity_not_a_register", entity=answer_entity) - turns: list[RegisterTurn] = [] for op in graph.operations: + if op.kind == "compare_multiplicative": + # actor := factor × reference — a cross-register definition (AMENDMENT 1/2). + comparison = op.operand + if not isinstance(comparison, Comparison): + raise MultiRegisterError("compare_operand_not_comparison", kind=op.kind) + if comparison.direction not in ("times", "fraction"): + raise MultiRegisterError("compare_additive_out_of_scope", direction=comparison.direction) + reference = comparison.reference_actor + if reference not in units: + raise MultiRegisterError("compare_reference_not_a_register", reference=reference) + if comparison.factor is None: # invariant: times/fraction carry a factor + raise MultiRegisterError("compare_missing_factor", direction=comparison.direction) + factor = float(comparison.factor) + if not math.isfinite(factor) or factor <= 0.0: + raise MultiRegisterError("compare_factor_not_positive", factor=factor) + if op.actor == reference: + raise MultiRegisterError("compare_self_reference", actor=op.actor) + # The comparison DEFINES the actor. If the actor is already known + # (seeded or defined earlier), this is a redefine — most often a + # mis-directed inverse compare (the known side used as the target). + # Refuse rather than overwrite: a reader direction error becomes a + # refusal, never a wrong (the reader must bind the unknown side as + # the target, with a fraction factor). + if op.actor in units: + raise MultiRegisterError("compare_redefines_register", actor=op.actor) + units[op.actor] = units[reference] # unit propagation from the reference + order.append(op.actor) + turns.append( + RegisterTurn(_COMPARE_MULT, op.actor, factor, 0.0, factor, reference=reference) + ) + continue if op.actor not in units: raise MultiRegisterError("actor_not_a_register", actor=op.actor) if op.kind == "transfer": @@ -223,11 +259,18 @@ def compile_multi_register_program(graph: MathProblemGraph) -> MultiRegisterProg raise MultiRegisterError("non_positive_scale", kind="divide", value=value) turns.append(RegisterTurn("divide", op.actor, 1.0 / value, 0.0, value)) + # Validate the answer target AFTER ops, so compare-defined entities count + # (a concrete unknown may be a compare-defined register). None ⇒ summation. + answer_entity = graph.unknown.entity + if answer_entity is not None and answer_entity not in units: + raise MultiRegisterError("unknown_entity_not_a_register", entity=answer_entity) + return MultiRegisterProgram( seeds=tuple(seeds), turns=tuple(turns), answer_entity=answer_entity, answer_unit=graph.unknown.unit, + register_order=tuple(order), ) @@ -284,6 +327,29 @@ def execute_multi_register_program(program: MultiRegisterProgram) -> MultiRegist prev = record.record_digest() records.append(record) index += 1 + elif turn.kind == _COMPARE_MULT: + # actor := factor × reference. Read the reference register's STATE, + # dilate, write the actor (a definition, not a transfer — no + # conservation pin). The record binds the reference entity + its state + # digest so re-verification reconstructs the source from records alone. + reference = turn.reference + if reference is None: # invariant: compile sets a reference for compare + raise MultiRegisterError("compare_missing_reference", actor=turn.actor) + reference_state = registers[reference] + target = _unit(dilate_quantity(reference_state, -math.log(turn.scale))) + new_state, cert = _relax_to(reference_state, target) + if not cert.converged: + raise MultiRegisterError("compare_nonconverged", actor=turn.actor, reference=reference) + registers = {**registers, turn.actor: new_state} + record = MultiRegisterRecord( + index, turn.actor, cert.certificate_id, cert.converged, + (_COMPARE_MULT, turn.scale, 0.0), prev, + operand_source_digest=_state_digest(reference_state), + source_entity=reference, + ) + prev = record.record_digest() + records.append(record) + index += 1 else: target = _unit( translate_quantity( @@ -308,7 +374,10 @@ def execute_multi_register_program(program: MultiRegisterProgram) -> MultiRegist # decode of a certified register state, bound by ``operand_source_digest`` # (= that state's psi_digest) — deterministic re-execution reproduces it, # so the Python layer cannot tamper with the intermediate value. - order = [entity for entity, _ in program.seeds] + # Registers-driven (AMENDMENT 1): sum ALL registers — seeds AND + # compare-defined — in the pinned deterministic order, so a defined + # register is never silently dropped from a total. + order = list(program.register_order) accumulator = registers[order[0]] for entity in order[1:]: source_state = registers[entity] diff --git a/tests/test_adr_0250_compare.py b/tests/test_adr_0250_compare.py new file mode 100644 index 00000000..57b9cfae --- /dev/null +++ b/tests/test_adr_0250_compare.py @@ -0,0 +1,200 @@ +"""ADR-0250 reader-arc increment 1 — compare_multiplicative compiler tier. + +`compare_multiplicative` (`actor = factor × reference`) as a cross-register +dilation: read the reference register's field state, dilate, write the actor. +The actor may be *defined* by the comparison (no seed); the registers-driven +summation must include it (amendment 1); the record binds the reference entity ++ its state digest for records-alone re-verification (amendment 2). Compare +*creates* a quantity, so it is exempt from the transfer conservation pin. +""" +from __future__ import annotations + +import dataclasses +import json +from pathlib import Path + +import pytest + +from generate.math_candidate_graph import parse_and_solve +from generate.math_problem_graph import ( + Comparison, + InitialPossession, + MathProblemGraph, + Operation, + Quantity, + Unknown, +) + +from evals.multi_register_program import ( + MultiRegisterError, + compile_multi_register_program, + execute_multi_register_program, + verify_multi_register_chain, +) + +_DEV = Path(__file__).resolve().parents[1] / "evals" / "gsm8k_math" / "holdout_dev" / "v1" / "cases.jsonl" + + +def _cmp_graph(actor, factor, *, direction="times", seed=5, unknown=None, unit="apples"): + """A→seed; actor = factor × A; unknown None (total) or a concrete entity.""" + return MathProblemGraph( + entities=("A", actor), + initial_state=(InitialPossession("A", Quantity(seed, unit)),), + operations=( + Operation(actor, "compare_multiplicative", Comparison("A", None, factor, direction)), + ), + unknown=Unknown(unknown, unit), + ) + + +def _solve(graph): + return execute_multi_register_program(compile_multi_register_program(graph)) + + +# --- The 5 real official compare parses solve end-to-end, wrong=0 ----------- + + +def test_real_compare_cases_solved_wrong_zero() -> None: + cases = [json.loads(line) for line in _DEV.read_text().splitlines() if line.strip()] + solved = wrong = seen = 0 + for case in cases: + try: + result = parse_and_solve(case["problem"]) + except Exception: + continue + graph = result.selected_graph + if graph is None or not any(op.kind == "compare_multiplicative" for op in graph.operations): + continue + seen += 1 + answer = _solve(graph).answer + if abs(answer - float(case["expected_answer"])) < 1e-4: + solved += 1 + else: + wrong += 1 + assert seen == 5 # the compare parses the reader emits today + assert wrong == 0 + assert solved == 5 # corridor real-reach 0/500 → 5/500 (loop-works proof) + + +# --- Amendment 1: compare-defined register in the certified sum + unit prop -- + + +def test_compare_defined_register_included_in_total() -> None: + # A=5, B=3×A=15, total = 20 — B is compare-defined (no seed) yet summed. + assert abs(_solve(_cmp_graph("B", 3.0, unknown=None)).answer - 20.0) < 1e-4 + + +def test_compare_defines_answer_target_with_unit_propagation() -> None: + outcome = _solve(_cmp_graph("B", 3.0, unknown="B", unit="dollars")) + assert abs(outcome.answer - 15.0) < 1e-4 + assert outcome.answer_unit == "dollars" # propagated from the reference + + +def test_fraction_direction() -> None: + assert abs(_solve(_cmp_graph("B", 0.5, direction="fraction", seed=8, unknown="B")).answer - 4.0) < 1e-4 + + +def test_compare_does_not_conserve_and_is_not_rejected() -> None: + # total (20) exceeds the reference (5): compare creates quantity — no pin fires. + outcome = _solve(_cmp_graph("B", 3.0, unknown=None)) + assert outcome.certified is True + + +# --- Amendment 2: the compare record binds the reference (records-alone) ----- + + +def test_compare_record_binds_reference_and_verifies() -> None: + outcome = _solve(_cmp_graph("B", 3.0, unknown="B")) + compare_records = [r for r in outcome.records if r.entity == "B"] + assert len(compare_records) == 1 + rec = compare_records[0] + assert rec.source_entity == "A" # the reference is reconstructable from the record + assert rec.operand_source_digest != "" # bound to the reference state's digest + assert verify_multi_register_chain(outcome.records) is True + + +def test_compare_chain_detects_tamper() -> None: + # 3-record chain (compare B, then a later op) so a non-terminal tamper breaks a link. + graph = MathProblemGraph( + entities=("A", "B"), + initial_state=(InitialPossession("A", Quantity(5, "x")),), + operations=( + Operation("B", "compare_multiplicative", Comparison("A", None, 3.0, "times")), + Operation("A", "add", Quantity(2, "x")), + ), + unknown=Unknown(None, "x"), + ) + outcome = _solve(graph) + tampered = list(outcome.records) + tampered[0] = dataclasses.replace(tampered[0], source_entity="forged") + assert verify_multi_register_chain(tampered) is False + + +# --- Criterion 4: non-compare records carry no source_entity ---------------- + + +def test_summation_record_has_no_source_entity() -> None: + # A summation record keeps only operand_source_digest — source_entity stays + # empty, so pre-compare record digests are byte-identical. + graph = MathProblemGraph( + entities=("A", "B"), + initial_state=( + InitialPossession("A", Quantity(5, "x")), + InitialPossession("B", Quantity(3, "x")), + ), + operations=(), + unknown=Unknown(None, "x"), + ) + outcome = _solve(graph) + for record in outcome.records: + assert record.source_entity == "" + + +# --- Fail-closed refusals ---------------------------------------------------- + + +def test_refuses_chained_reference_before_definition() -> None: + # A := 2×B, but B is never seeded or defined at that point (story order ≠ + # dependency order): fail-closed, don't guess. Pins the ordering rule. + graph = MathProblemGraph( + entities=("A", "B", "C"), + initial_state=(InitialPossession("C", Quantity(5, "x")),), + operations=(Operation("A", "compare_multiplicative", Comparison("B", None, 2.0, "times")),), + unknown=Unknown("A", "x"), + ) + with pytest.raises(MultiRegisterError): + compile_multi_register_program(graph) + + +def test_refuses_compare_redefining_known_register() -> None: + # A seeded; "B is twice A" mis-directed so the target is the KNOWN side — + # a redefine of a seeded register. Refuse (the reader must invert to the + # unknown side); never overwrite a known quantity. + graph = MathProblemGraph( + entities=("A", "B"), + initial_state=( + InitialPossession("A", Quantity(5, "x")), + InitialPossession("B", Quantity(9, "x")), + ), + operations=(Operation("B", "compare_multiplicative", Comparison("A", None, 2.0, "times")),), + unknown=Unknown(None, "x"), + ) + with pytest.raises(MultiRegisterError): + compile_multi_register_program(graph) + + +def test_refuses_compare_additive() -> None: + # compare_additive ("more"/"fewer") is a later increment — out of scope now. + graph = MathProblemGraph( + entities=("A", "B"), + initial_state=( + InitialPossession("A", Quantity(5, "x")), + InitialPossession("B", Quantity(1, "x")), + ), + operations=( + Operation("B", "compare_additive", Comparison("A", Quantity(2, "x"), None, "more")), + ), + unknown=Unknown("B", "x"), + ) + with pytest.raises(MultiRegisterError): + compile_multi_register_program(graph)