diff --git a/docs/adr/ADR-0250-tier2-multi-entity-arithmetic.md b/docs/adr/ADR-0250-tier2-multi-entity-arithmetic.md new file mode 100644 index 00000000..55ca327a --- /dev/null +++ b/docs/adr/ADR-0250-tier2-multi-entity-arithmetic.md @@ -0,0 +1,95 @@ +# ADR-0250: Tier-2 Multi-Entity Arithmetic — the Multi-Register Extension + +**Status**: **Proposed** +**Date**: 2026-07-18 +**Authors**: Joshua Shay + multi-model R&D (implemented Fable 5) +**Depends on**: ADR-0249 (reader→Hamiltonian compiler — quantity kernel, relation compiler, turn programs), ADR-0243/0244/0245 (relaxation, content-addressing, f64) +**Design record**: `docs/research/tier2-multi-entity-arithmetic-spike-2026-07-18.md` +**Acceptance evidence**: `docs/handoff/ADR-0250-Acceptance-Evidence.md` + +--- + +## 1. What this ADR is + +ADR-0249 closed Tier-1 (single-accumulator affine arithmetic) and recorded the 24 multi-entity +GSM8K dev-holdout cases as the Tier-2 frontier. This ADR crosses that frontier — and with it the +reader→Hamiltonian compiler solves the **entire** real GSM8K dev holdout (50/50) at wrong=0. It +extends the substrate mechanism only; no serving change, no gate activated, no manifold mutation. + +## 2. The multi-register model + +Multi-entity state is a **product of independent conformal lines**: one register (null point / +its own relaxation) per entity. This respects the Cl(4,1) boundary exactly — no dimensional +inflation (no `Cl(8,2)` to pack two points into one multivector), so the f64 rounding floor +(~6e-11) stays the only residual and every op is a native, exact versor action on a pure state. +Per-register ops (add/subtract/multiply/divide by constants) reuse the Tier-1 transport unchanged. + +## 3. Transfers + the conservation pin + +A constant-operand transfer ("actor gives `k` to target", ADR-0116 decomposition) is a **coupled +pair of translators** — `T₋ₖ` on the actor register, `T₊ₖ` on the target — acting on disjoint +registers, so exactness across the independent null points is inherited from ADR-0249 P1. + +**Conservation pin (hard-reject, relative).** `Σ decode(after) ≡ Σ decode(before)` within +`|Δ| ≤ rtol·max(1,|before|)`. The tolerance is *relative* because decoded-quantity error scales +with magnitude (found on a case chaining to 222); an absolute tolerance is brittle, while a +genuine non-conserving transfer is off by a whole `k` and is still caught. A conservation failure +is an algebraic failure, not a scope miss — fail closed. + +## 4. Transactional atomicity + +Registers are an **immutable snapshot**. A transfer runs **prepare → validate → commit**: both +candidate states are relaxed into locals, validated (both converged AND conservation), then — only +on full success — a *new* register mapping is produced. A failure anywhere means the new mapping is +never built, so the original stands untouched: no partial-mutation window, no rollback. Records +append only on commit; any abort raises a typed `MultiRegisterError` and aborts the whole program +(fail-closed — a partial answer is never emitted, preserving wrong=0). + +## 5. Certified summation + chain of custody + +A total-like unknown (`unknown.entity is None`, "how many altogether") is the **explicit typed +signal** for a certified summation over all registers — never inferred from an operation pattern. +Per the summation ruling it stays in the substrate: each addition is a certified relaxation turn +(`T_v` on the running accumulator), never a Python `sum()`. + +The operand of each summation turn is the decode of a certified register state, bound in the record +by `operand_source_digest` (the `psi_digest` byte convention). Deterministic re-execution +reproduces the state and its decode, so the Python layer cannot tamper with the intermediate value +— the chain of custody holds with no trusted-Python assumption. The field is added conditionally to +the record payload, so non-summation record digests are unchanged. + +## 6. Honest measurement (done-when) + +On the sealed real GSM8K dev holdout (50 cases): **50/50 solved, wrong=0** — Tier-1 (26) + T2a +single-entity (18) + summation (6). The instrument's `arithmetic-chain` domain routes each case to +the right tier and compares the corridor to a symbolic fold of the *same* compiled program: PARITY +(delta 0 — the field matches arithmetic, it does not beat it). The deliverable is a decidable +measurement + real-holdout coverage, per the honest-NULL protocol. + +## 7. Scope + recorded frontier + +Tier-2a: single-accumulator-per-entity affine + constant-operand transfers, positive scale, +matching units. 2b: the certified summation turn (real, exercised by the 6 total-like cases) and +the derived-operand-transfer path ("half of X" — designed + guarded, **0 on this holdout**). Still +refused and recorded: non-affine kinds (rate/comparison/fraction/partition — 0 on this holdout), +derived-operand transfers, and >5-atom deduction — never silently dropped. + +## 8. Governance + +Off-serving (A-04): `evals/multi_register_program.py` + the instrument domain; never imported by +`chat/runtime.py`. No gate activated; I-03 untouched; ratified contracts reused +(`compile_quadratic_well`, `quantity_kernel`, the Ring-2 chain pattern, `RelaxationCertificate`). +Local-first CI: smoke green in-worktree at every phase. + +## 9. What this ADR does NOT claim + +- Not a serving change; the symbolic `generate/` path remains the serve path. +- Not a lift over symbolic solvers on arithmetic (PARITY is the honest result). +- Not derived-operand-transfer coverage (0 on this holdout; designed + guarded only). +- Not an activation of any admission gate (ADR-0246 §3.7 stays uncalibrated). + +## 10. Ruling record (Shay) + +_Awaiting ratification._ The design rulings (multi-register product-of-lines; relative +conservation hard-reject; prepare→validate→commit atomicity; certified summation turn; ship 2b +designed+guarded) are recorded RESOLVED in the spike; acceptance is Shay's alone (no self-Accept). diff --git a/docs/handoff/ADR-0250-Acceptance-Evidence.md b/docs/handoff/ADR-0250-Acceptance-Evidence.md new file mode 100644 index 00000000..15a17d2a --- /dev/null +++ b/docs/handoff/ADR-0250-Acceptance-Evidence.md @@ -0,0 +1,59 @@ +# ADR-0250 — Acceptance Evidence + +**Status**: Evidence pack for Shay's ruling (no self-Accept) +**Date**: 2026-07-18 +**ADR**: `docs/adr/ADR-0250-tier2-multi-entity-arithmetic.md` +**Design record**: `docs/research/tier2-multi-entity-arithmetic-spike-2026-07-18.md` + +Tier-2 extends the reader→Hamiltonian compiler (ADR-0249) to multi-entity arithmetic. With it, +the corridor solves the **entire** real GSM8K dev holdout. Built in smoke-gated phases, each +merged under its own PR off `forgejo/main`. + +## Phases merged + +| Phase | PR | Merge commit | What landed | +|---|---|---|---| +| Design spike | #71 | `97b2f8b7` | multi-register model, atomicity, chain-of-custody design; probe of the 24 refused cases | +| T2a executor | #72 | `efd5335e` | `evals/multi_register_program.py` — multi-register + transfers + conservation; 18/18 single-entity | +| 2b summation | #73 | `3c3dcdbe` | certified summation turn + `operand_source_digest`; full holdout 50/50 | +| Instrument + ADR | this PR | — | `arithmetic-chain` domain routes Tier-2; ADR-0250 + this evidence | + +Smoke (`uv run core test --suite smoke -q`) green at every phase (176 passed). + +## Load-bearing result + +**Real GSM8K dev holdout (sealed, 50 cases):** + +| Tier | Cases | Corridor | +|---|---|---| +| Tier-1 single-accumulator (ADR-0249) | 26 | correct, wrong=0 | +| T2a single-entity multi-register | 18 | correct, wrong=0 | +| 2b certified summation ("altogether") | 6 | correct, wrong=0 | +| **Total** | **50 / 50** | **wrong = 0** | + +Corridor vs symbolic fold of the same compiled program: **PARITY** (delta 0) — the field matches +arithmetic, it does not beat it. The instrument's `arithmetic-chain` domain now records 50/50 +coverage, 0 refused. + +## Verified in-tree + +- **Transfer conservation + per-register exactness**: Ruth 36→31, Sara 19→24, sum conserved; + 200 random transfers exact + conserving to ~6e-11. +- **Relative conservation pin**: `gma-050` (chains to 222) tripped an absolute 1e-6 tolerance; + the relative check `|Δ| ≤ rtol·max(1,|before|)` fixes it and still catches real violations. +- **Chain of custody**: summation records carry `operand_source_digest`; tamper on any non-terminal + record breaks the successor link (`verify_multi_register_chain`). + +## Design decisions applied (honest register) + +- **Atomicity via immutability**: prepare→validate→commit; no partial-mutation window, no rollback. +- **Summation stays in the substrate** (ruling #1): each addition is a certified turn, never a + Python `sum()`; the total is grounded in `RelaxationCertificate`s. +- **The summation signal is the typed `unknown.entity is None`**, not an inferred op pattern. + +## What remains open (not claimed here) + +- Derived-operand transfers ("half of X") — designed + guarded, 0 on this holdout. +- Non-affine kinds (rate/comparison/fraction/partition) — 0 on this holdout, refused. +- >5-atom deduction via ROBDD-partitioned turn programs — future arc. +- ADR-0246 §3.7 admission-policy calibration — unchanged; no gate activated. diff --git a/evals/generalized_lift_instrument.py b/evals/generalized_lift_instrument.py index 74681f1a..4e9c61ae 100644 --- a/evals/generalized_lift_instrument.py +++ b/evals/generalized_lift_instrument.py @@ -66,6 +66,11 @@ from evals.turn_program import ( compile_turn_program, execute_turn_program, ) +from evals.multi_register_program import ( + MultiRegisterError, + compile_multi_register_program, + execute_multi_register_program, +) __all__ = [ "DomainOutcome", @@ -486,14 +491,51 @@ def _symbolic_fold(seed: float, steps) -> float: return answer -def run_arithmetic_chain_domain() -> DomainOutcome: - """Corridor turn-program executor vs symbolic fold on real GSM8K problems. +def _symbolic_multi_register(program) -> float: + """Baseline for Tier-2: solve the SAME multi-register program arithmetically. - Both paths consume the graph's `ground_truth_graph` compiled to a turn - program; non-Tier-1 graphs (multi-entity, transfer, rate, …) are refused by - BOTH and recorded as the composition frontier — never silently dropped. The - expected honest verdict is PARITY with wrong=0: the field does not beat - arithmetic at arithmetic; the result is real-holdout coverage, not a lift. + Mirror of the corridor executor with Python arithmetic instead of + relaxation — per-register affine folds, transfers move `operand` between + registers, a `None` answer entity sums all registers. + """ + registers = {entity: seed for entity, seed in program.seeds} + for turn in program.turns: + if turn.kind == "transfer": + registers[turn.actor] -= turn.operand + registers[turn.target] += turn.operand + else: + registers[turn.actor] = turn.scale * registers[turn.actor] + turn.offset + if program.answer_entity is None: + return float(sum(registers.values())) + return registers[program.answer_entity] + + +def _corridor_and_baseline(graph): + """Route a graph to Tier-1 (single-accumulator) or Tier-2 (multi-register), + returning (corridor_answer, baseline_answer) or None if genuinely un-ingestible. + Both paths consume the identical compiled program; the corridor relaxes, the + baseline folds arithmetically.""" + try: + program = compile_turn_program(graph) + return execute_turn_program(program).answer, _symbolic_fold(program.seed, program.steps) + except TurnProgramError: + pass + try: + program = compile_multi_register_program(graph) + except MultiRegisterError: + return None + return execute_multi_register_program(program).answer, _symbolic_multi_register(program) + + +def run_arithmetic_chain_domain() -> DomainOutcome: + """Corridor executor vs symbolic fold on real GSM8K problems (Tier-1 + Tier-2). + + Each `ground_truth_graph` is routed to the single-accumulator turn program or + the multi-register program; both corridor and baseline consume that same + compiled program. Anything neither tier ingests (derived-operand transfers, + >5-atom shapes) is refused by BOTH and recorded, never silently dropped. The + expected honest verdict is PARITY with wrong=0: the field matches arithmetic, + it does not beat it; the result is real-holdout coverage. """ corridor_correct = corridor_wrong = corridor_refused = 0 baseline_correct = baseline_wrong = baseline_refused = 0 @@ -508,23 +550,15 @@ def run_arithmetic_chain_domain() -> DomainOutcome: graph = graph_from_dict(graph_dict) if graph_dict else None except (MathGraphError, KeyError, TypeError, ValueError): graph = None - try: - program = compile_turn_program(graph) if graph is not None else None - except TurnProgramError as refusal: + answers = _corridor_and_baseline(graph) if graph is not None else None + if answers is None: corridor_refused += 1 baseline_refused += 1 - rows.append({"case_id": case_id, "ingested": False, "refusal": refusal.reason}) - continue - if program is None: - corridor_refused += 1 - baseline_refused += 1 - rows.append({"case_id": case_id, "ingested": False, "refusal": "no_graph"}) + rows.append({"case_id": case_id, "ingested": False}) continue + corridor_answer, baseline_answer = answers gold = None if gold_raw is None else float(gold_raw) - corridor_answer = execute_turn_program(program).answer - baseline_answer = _symbolic_fold(program.seed, program.steps) - corridor_ok = gold is not None and abs(corridor_answer - gold) < 1e-4 baseline_ok = gold is not None and abs(baseline_answer - gold) < 1e-4 corridor_correct += int(corridor_ok) @@ -553,10 +587,10 @@ def run_arithmetic_chain_domain() -> DomainOutcome: baseline_wrong=baseline_wrong, baseline_refused=baseline_refused, notes=( - f"Real GSM8K dev holdout: {ingested}/{len(cases)} problems are Tier-1 " - f"affine single-accumulator ingestible and solved wrong=0; the rest are " - f"refused (multi-entity/transfer/rate/…) — the recorded Tier-2 frontier.", - "Baseline is a symbolic fold of the SAME compiled turn program; PARITY " + f"Real GSM8K dev holdout: {ingested}/{len(cases)} problems ingestible " + f"(Tier-1 single-accumulator + Tier-2 multi-entity/transfer/summation) and " + f"solved wrong=0. Derived-operand transfers / >5-atom shapes remain the frontier.", + "Baseline is a symbolic fold of the SAME compiled program; PARITY " "with wrong=0 is the honest outcome — the field matches arithmetic, it " "does not beat it. The result is real-holdout coverage, not a lift.", ), @@ -579,11 +613,12 @@ def run_generalized_lift_instrument() -> LiftInstrumentReport: wrong_zero_guard_held=(propositional.corridor_wrong == 0 and arithmetic.corridor_wrong == 0), honest_null=all(o.delta_correct <= 0 for o in outcomes), scope_limitations=( - "The reader→Hamiltonian compiler now exists for the affine " - "single-accumulator arithmetic subset (ADR-0249): on the real GSM8K " - "dev holdout it solves the Tier-1 subset wrong=0 (see the " - "arithmetic-chain domain). Non-affine / multi-entity GSM8K problems " - "(transfer, rate, comparison, fraction, partition) remain the " - "recorded Tier-2 frontier — refused, never silently dropped.", + "The reader→Hamiltonian compiler now solves the FULL real GSM8K dev " + "holdout wrong=0 (Tier-1 single-accumulator, ADR-0249; Tier-2 " + "multi-entity + constant-operand transfers + certified summation, " + "ADR-0250) — see the arithmetic-chain domain. Remaining general " + "frontier: derived-operand transfers ('half of X') and non-affine " + "kinds (rate/comparison/fraction/partition), both 0 on this holdout; " + "and >5-atom deduction. Refused, never silently dropped.", ), ) diff --git a/tests/test_adr_0249_arithmetic_lift.py b/tests/test_adr_0249_arithmetic_lift.py index 42a1c1ba..854cf721 100644 --- a/tests/test_adr_0249_arithmetic_lift.py +++ b/tests/test_adr_0249_arithmetic_lift.py @@ -27,7 +27,8 @@ def test_domain_solves_real_gsm8k_wrong_zero(domain) -> None: assert domain.domain_id == "arithmetic-chain" assert domain.corridor_wrong == 0 # wrong=0 on real GSM8K, not just templates assert domain.corridor_correct > 0 # the corridor ingests and solves real problems - assert domain.corridor_refused > 0 # and honestly refuses what it can't ingest + # Tier-2 (ADR-0250) closed the multi-entity frontier: the full holdout ingests now. + assert domain.corridor_refused == 0 def test_every_ingested_case_is_correct(domain) -> None: @@ -50,10 +51,10 @@ def test_coverage_is_recorded_not_dropped(domain) -> None: ingested = domain.corridor_correct + domain.corridor_wrong assert ingested + domain.corridor_refused == domain.n_cases assert domain.n_cases > 0 - # A coverage tracker against the sealed dev holdout (50 cases, 26 Tier-1). - assert domain.corridor_correct == 26 - assert domain.corridor_refused == 24 - assert any("Tier-1" in note for note in domain.notes) + # Coverage tracker against the sealed dev holdout: Tier-1 (26) + Tier-2 (24) = 50/50. + assert domain.corridor_correct == 50 + assert domain.corridor_refused == 0 + assert any("Tier-2" in note for note in domain.notes) def test_domain_joins_full_report_and_scope_corrected() -> None: @@ -61,8 +62,8 @@ def test_domain_joins_full_report_and_scope_corrected() -> None: ids = [o.domain_id for o in report.outcomes] assert "arithmetic-chain" in ids assert report.wrong_zero_guard_held # deductive + arithmetic both wrong=0 - # The stale "no compiler exists" scope note must be gone; the compiler exists now. + # The scope note reflects the full-holdout closure; the stale "no compiler" line is gone. joined = " ".join(report.scope_limitations) - assert "now exists" in joined + assert "FULL real GSM8K dev holdout" in joined assert "no reader-to-Hamiltonian compiler exists" not in joined json.dumps(report.as_dict()) # JSON-safe artifact