feat(adr-0250): instrument routes Tier-2 + ADR-0250 (Proposed) — arc closure
Instrument: the arithmetic-chain domain now routes each real GSM8K dev case to Tier-1 (single-accumulator) or Tier-2 (multi-register / summation) and compares the corridor to a symbolic fold of the SAME compiled program. Recorded coverage moves 26/50 -> 50/50, 0 refused, PARITY (wrong=0) — the measurement catching up to the capability. Scope note corrected (full holdout closed; remaining frontier = derived-operand transfers + non-affine kinds + >5-atom deduction, all 0 on this holdout). Updated the ADR-0249 instrument test pins (26->50, refused 24->0). ADR-0250 (Status Proposed, no self-Accept; §10 ruling awaits Shay) + acceptance evidence: multi-register model, relative conservation, atomicity, certified summation, chain-of-custody, and the full-holdout 50/50 wrong=0 result. 10/10 instrument pins green. [Verification]: uv run python -m pytest tests/test_adr_0249_arithmetic_lift.py tests/test_generalized_lift_instrument.py -q
This commit is contained in:
parent
3c3dcdbe8f
commit
e6439bb70f
4 changed files with 226 additions and 36 deletions
95
docs/adr/ADR-0250-tier2-multi-entity-arithmetic.md
Normal file
95
docs/adr/ADR-0250-tier2-multi-entity-arithmetic.md
Normal file
|
|
@ -0,0 +1,95 @@
|
|||
# ADR-0250: Tier-2 Multi-Entity Arithmetic — the Multi-Register Extension
|
||||
|
||||
**Status**: **Proposed**
|
||||
**Date**: 2026-07-18
|
||||
**Authors**: Joshua Shay + multi-model R&D (implemented Fable 5)
|
||||
**Depends on**: ADR-0249 (reader→Hamiltonian compiler — quantity kernel, relation compiler, turn programs), ADR-0243/0244/0245 (relaxation, content-addressing, f64)
|
||||
**Design record**: `docs/research/tier2-multi-entity-arithmetic-spike-2026-07-18.md`
|
||||
**Acceptance evidence**: `docs/handoff/ADR-0250-Acceptance-Evidence.md`
|
||||
|
||||
---
|
||||
|
||||
## 1. What this ADR is
|
||||
|
||||
ADR-0249 closed Tier-1 (single-accumulator affine arithmetic) and recorded the 24 multi-entity
|
||||
GSM8K dev-holdout cases as the Tier-2 frontier. This ADR crosses that frontier — and with it the
|
||||
reader→Hamiltonian compiler solves the **entire** real GSM8K dev holdout (50/50) at wrong=0. It
|
||||
extends the substrate mechanism only; no serving change, no gate activated, no manifold mutation.
|
||||
|
||||
## 2. The multi-register model
|
||||
|
||||
Multi-entity state is a **product of independent conformal lines**: one register (null point /
|
||||
its own relaxation) per entity. This respects the Cl(4,1) boundary exactly — no dimensional
|
||||
inflation (no `Cl(8,2)` to pack two points into one multivector), so the f64 rounding floor
|
||||
(~6e-11) stays the only residual and every op is a native, exact versor action on a pure state.
|
||||
Per-register ops (add/subtract/multiply/divide by constants) reuse the Tier-1 transport unchanged.
|
||||
|
||||
## 3. Transfers + the conservation pin
|
||||
|
||||
A constant-operand transfer ("actor gives `k` to target", ADR-0116 decomposition) is a **coupled
|
||||
pair of translators** — `T₋ₖ` on the actor register, `T₊ₖ` on the target — acting on disjoint
|
||||
registers, so exactness across the independent null points is inherited from ADR-0249 P1.
|
||||
|
||||
**Conservation pin (hard-reject, relative).** `Σ decode(after) ≡ Σ decode(before)` within
|
||||
`|Δ| ≤ rtol·max(1,|before|)`. The tolerance is *relative* because decoded-quantity error scales
|
||||
with magnitude (found on a case chaining to 222); an absolute tolerance is brittle, while a
|
||||
genuine non-conserving transfer is off by a whole `k` and is still caught. A conservation failure
|
||||
is an algebraic failure, not a scope miss — fail closed.
|
||||
|
||||
## 4. Transactional atomicity
|
||||
|
||||
Registers are an **immutable snapshot**. A transfer runs **prepare → validate → commit**: both
|
||||
candidate states are relaxed into locals, validated (both converged AND conservation), then — only
|
||||
on full success — a *new* register mapping is produced. A failure anywhere means the new mapping is
|
||||
never built, so the original stands untouched: no partial-mutation window, no rollback. Records
|
||||
append only on commit; any abort raises a typed `MultiRegisterError` and aborts the whole program
|
||||
(fail-closed — a partial answer is never emitted, preserving wrong=0).
|
||||
|
||||
## 5. Certified summation + chain of custody
|
||||
|
||||
A total-like unknown (`unknown.entity is None`, "how many altogether") is the **explicit typed
|
||||
signal** for a certified summation over all registers — never inferred from an operation pattern.
|
||||
Per the summation ruling it stays in the substrate: each addition is a certified relaxation turn
|
||||
(`T_v` on the running accumulator), never a Python `sum()`.
|
||||
|
||||
The operand of each summation turn is the decode of a certified register state, bound in the record
|
||||
by `operand_source_digest` (the `psi_digest` byte convention). Deterministic re-execution
|
||||
reproduces the state and its decode, so the Python layer cannot tamper with the intermediate value
|
||||
— the chain of custody holds with no trusted-Python assumption. The field is added conditionally to
|
||||
the record payload, so non-summation record digests are unchanged.
|
||||
|
||||
## 6. Honest measurement (done-when)
|
||||
|
||||
On the sealed real GSM8K dev holdout (50 cases): **50/50 solved, wrong=0** — Tier-1 (26) + T2a
|
||||
single-entity (18) + summation (6). The instrument's `arithmetic-chain` domain routes each case to
|
||||
the right tier and compares the corridor to a symbolic fold of the *same* compiled program: PARITY
|
||||
(delta 0 — the field matches arithmetic, it does not beat it). The deliverable is a decidable
|
||||
measurement + real-holdout coverage, per the honest-NULL protocol.
|
||||
|
||||
## 7. Scope + recorded frontier
|
||||
|
||||
Tier-2a: single-accumulator-per-entity affine + constant-operand transfers, positive scale,
|
||||
matching units. 2b: the certified summation turn (real, exercised by the 6 total-like cases) and
|
||||
the derived-operand-transfer path ("half of X" — designed + guarded, **0 on this holdout**). Still
|
||||
refused and recorded: non-affine kinds (rate/comparison/fraction/partition — 0 on this holdout),
|
||||
derived-operand transfers, and >5-atom deduction — never silently dropped.
|
||||
|
||||
## 8. Governance
|
||||
|
||||
Off-serving (A-04): `evals/multi_register_program.py` + the instrument domain; never imported by
|
||||
`chat/runtime.py`. No gate activated; I-03 untouched; ratified contracts reused
|
||||
(`compile_quadratic_well`, `quantity_kernel`, the Ring-2 chain pattern, `RelaxationCertificate`).
|
||||
Local-first CI: smoke green in-worktree at every phase.
|
||||
|
||||
## 9. What this ADR does NOT claim
|
||||
|
||||
- Not a serving change; the symbolic `generate/` path remains the serve path.
|
||||
- Not a lift over symbolic solvers on arithmetic (PARITY is the honest result).
|
||||
- Not derived-operand-transfer coverage (0 on this holdout; designed + guarded only).
|
||||
- Not an activation of any admission gate (ADR-0246 §3.7 stays uncalibrated).
|
||||
|
||||
## 10. Ruling record (Shay)
|
||||
|
||||
_Awaiting ratification._ The design rulings (multi-register product-of-lines; relative
|
||||
conservation hard-reject; prepare→validate→commit atomicity; certified summation turn; ship 2b
|
||||
designed+guarded) are recorded RESOLVED in the spike; acceptance is Shay's alone (no self-Accept).
|
||||
59
docs/handoff/ADR-0250-Acceptance-Evidence.md
Normal file
59
docs/handoff/ADR-0250-Acceptance-Evidence.md
Normal file
|
|
@ -0,0 +1,59 @@
|
|||
# ADR-0250 — Acceptance Evidence
|
||||
|
||||
**Status**: Evidence pack for Shay's ruling (no self-Accept)
|
||||
**Date**: 2026-07-18
|
||||
**ADR**: `docs/adr/ADR-0250-tier2-multi-entity-arithmetic.md`
|
||||
**Design record**: `docs/research/tier2-multi-entity-arithmetic-spike-2026-07-18.md`
|
||||
|
||||
Tier-2 extends the reader→Hamiltonian compiler (ADR-0249) to multi-entity arithmetic. With it,
|
||||
the corridor solves the **entire** real GSM8K dev holdout. Built in smoke-gated phases, each
|
||||
merged under its own PR off `forgejo/main`.
|
||||
|
||||
## Phases merged
|
||||
|
||||
| Phase | PR | Merge commit | What landed |
|
||||
|---|---|---|---|
|
||||
| Design spike | #71 | `97b2f8b7` | multi-register model, atomicity, chain-of-custody design; probe of the 24 refused cases |
|
||||
| T2a executor | #72 | `efd5335e` | `evals/multi_register_program.py` — multi-register + transfers + conservation; 18/18 single-entity |
|
||||
| 2b summation | #73 | `3c3dcdbe` | certified summation turn + `operand_source_digest`; full holdout 50/50 |
|
||||
| Instrument + ADR | this PR | — | `arithmetic-chain` domain routes Tier-2; ADR-0250 + this evidence |
|
||||
|
||||
Smoke (`uv run core test --suite smoke -q`) green at every phase (176 passed).
|
||||
|
||||
## Load-bearing result
|
||||
|
||||
**Real GSM8K dev holdout (sealed, 50 cases):**
|
||||
|
||||
| Tier | Cases | Corridor |
|
||||
|---|---|---|
|
||||
| Tier-1 single-accumulator (ADR-0249) | 26 | correct, wrong=0 |
|
||||
| T2a single-entity multi-register | 18 | correct, wrong=0 |
|
||||
| 2b certified summation ("altogether") | 6 | correct, wrong=0 |
|
||||
| **Total** | **50 / 50** | **wrong = 0** |
|
||||
|
||||
Corridor vs symbolic fold of the same compiled program: **PARITY** (delta 0) — the field matches
|
||||
arithmetic, it does not beat it. The instrument's `arithmetic-chain` domain now records 50/50
|
||||
coverage, 0 refused.
|
||||
|
||||
## Verified in-tree
|
||||
|
||||
- **Transfer conservation + per-register exactness**: Ruth 36→31, Sara 19→24, sum conserved;
|
||||
200 random transfers exact + conserving to ~6e-11.
|
||||
- **Relative conservation pin**: `gma-050` (chains to 222) tripped an absolute 1e-6 tolerance;
|
||||
the relative check `|Δ| ≤ rtol·max(1,|before|)` fixes it and still catches real violations.
|
||||
- **Chain of custody**: summation records carry `operand_source_digest`; tamper on any non-terminal
|
||||
record breaks the successor link (`verify_multi_register_chain`).
|
||||
|
||||
## Design decisions applied (honest register)
|
||||
|
||||
- **Atomicity via immutability**: prepare→validate→commit; no partial-mutation window, no rollback.
|
||||
- **Summation stays in the substrate** (ruling #1): each addition is a certified turn, never a
|
||||
Python `sum()`; the total is grounded in `RelaxationCertificate`s.
|
||||
- **The summation signal is the typed `unknown.entity is None`**, not an inferred op pattern.
|
||||
|
||||
## What remains open (not claimed here)
|
||||
|
||||
- Derived-operand transfers ("half of X") — designed + guarded, 0 on this holdout.
|
||||
- Non-affine kinds (rate/comparison/fraction/partition) — 0 on this holdout, refused.
|
||||
- >5-atom deduction via ROBDD-partitioned turn programs — future arc.
|
||||
- ADR-0246 §3.7 admission-policy calibration — unchanged; no gate activated.
|
||||
|
|
@ -66,6 +66,11 @@ from evals.turn_program import (
|
|||
compile_turn_program,
|
||||
execute_turn_program,
|
||||
)
|
||||
from evals.multi_register_program import (
|
||||
MultiRegisterError,
|
||||
compile_multi_register_program,
|
||||
execute_multi_register_program,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"DomainOutcome",
|
||||
|
|
@ -486,14 +491,51 @@ def _symbolic_fold(seed: float, steps) -> float:
|
|||
return answer
|
||||
|
||||
|
||||
def run_arithmetic_chain_domain() -> DomainOutcome:
|
||||
"""Corridor turn-program executor vs symbolic fold on real GSM8K problems.
|
||||
def _symbolic_multi_register(program) -> float:
|
||||
"""Baseline for Tier-2: solve the SAME multi-register program arithmetically.
|
||||
|
||||
Both paths consume the graph's `ground_truth_graph` compiled to a turn
|
||||
program; non-Tier-1 graphs (multi-entity, transfer, rate, …) are refused by
|
||||
BOTH and recorded as the composition frontier — never silently dropped. The
|
||||
expected honest verdict is PARITY with wrong=0: the field does not beat
|
||||
arithmetic at arithmetic; the result is real-holdout coverage, not a lift.
|
||||
Mirror of the corridor executor with Python arithmetic instead of
|
||||
relaxation — per-register affine folds, transfers move `operand` between
|
||||
registers, a `None` answer entity sums all registers.
|
||||
"""
|
||||
registers = {entity: seed for entity, seed in program.seeds}
|
||||
for turn in program.turns:
|
||||
if turn.kind == "transfer":
|
||||
registers[turn.actor] -= turn.operand
|
||||
registers[turn.target] += turn.operand
|
||||
else:
|
||||
registers[turn.actor] = turn.scale * registers[turn.actor] + turn.offset
|
||||
if program.answer_entity is None:
|
||||
return float(sum(registers.values()))
|
||||
return registers[program.answer_entity]
|
||||
|
||||
|
||||
def _corridor_and_baseline(graph):
|
||||
"""Route a graph to Tier-1 (single-accumulator) or Tier-2 (multi-register),
|
||||
returning (corridor_answer, baseline_answer) or None if genuinely un-ingestible.
|
||||
Both paths consume the identical compiled program; the corridor relaxes, the
|
||||
baseline folds arithmetically."""
|
||||
try:
|
||||
program = compile_turn_program(graph)
|
||||
return execute_turn_program(program).answer, _symbolic_fold(program.seed, program.steps)
|
||||
except TurnProgramError:
|
||||
pass
|
||||
try:
|
||||
program = compile_multi_register_program(graph)
|
||||
except MultiRegisterError:
|
||||
return None
|
||||
return execute_multi_register_program(program).answer, _symbolic_multi_register(program)
|
||||
|
||||
|
||||
def run_arithmetic_chain_domain() -> DomainOutcome:
|
||||
"""Corridor executor vs symbolic fold on real GSM8K problems (Tier-1 + Tier-2).
|
||||
|
||||
Each `ground_truth_graph` is routed to the single-accumulator turn program or
|
||||
the multi-register program; both corridor and baseline consume that same
|
||||
compiled program. Anything neither tier ingests (derived-operand transfers,
|
||||
>5-atom shapes) is refused by BOTH and recorded, never silently dropped. The
|
||||
expected honest verdict is PARITY with wrong=0: the field matches arithmetic,
|
||||
it does not beat it; the result is real-holdout coverage.
|
||||
"""
|
||||
corridor_correct = corridor_wrong = corridor_refused = 0
|
||||
baseline_correct = baseline_wrong = baseline_refused = 0
|
||||
|
|
@ -508,23 +550,15 @@ def run_arithmetic_chain_domain() -> DomainOutcome:
|
|||
graph = graph_from_dict(graph_dict) if graph_dict else None
|
||||
except (MathGraphError, KeyError, TypeError, ValueError):
|
||||
graph = None
|
||||
try:
|
||||
program = compile_turn_program(graph) if graph is not None else None
|
||||
except TurnProgramError as refusal:
|
||||
answers = _corridor_and_baseline(graph) if graph is not None else None
|
||||
if answers is None:
|
||||
corridor_refused += 1
|
||||
baseline_refused += 1
|
||||
rows.append({"case_id": case_id, "ingested": False, "refusal": refusal.reason})
|
||||
continue
|
||||
if program is None:
|
||||
corridor_refused += 1
|
||||
baseline_refused += 1
|
||||
rows.append({"case_id": case_id, "ingested": False, "refusal": "no_graph"})
|
||||
rows.append({"case_id": case_id, "ingested": False})
|
||||
continue
|
||||
|
||||
corridor_answer, baseline_answer = answers
|
||||
gold = None if gold_raw is None else float(gold_raw)
|
||||
corridor_answer = execute_turn_program(program).answer
|
||||
baseline_answer = _symbolic_fold(program.seed, program.steps)
|
||||
|
||||
corridor_ok = gold is not None and abs(corridor_answer - gold) < 1e-4
|
||||
baseline_ok = gold is not None and abs(baseline_answer - gold) < 1e-4
|
||||
corridor_correct += int(corridor_ok)
|
||||
|
|
@ -553,10 +587,10 @@ def run_arithmetic_chain_domain() -> DomainOutcome:
|
|||
baseline_wrong=baseline_wrong,
|
||||
baseline_refused=baseline_refused,
|
||||
notes=(
|
||||
f"Real GSM8K dev holdout: {ingested}/{len(cases)} problems are Tier-1 "
|
||||
f"affine single-accumulator ingestible and solved wrong=0; the rest are "
|
||||
f"refused (multi-entity/transfer/rate/…) — the recorded Tier-2 frontier.",
|
||||
"Baseline is a symbolic fold of the SAME compiled turn program; PARITY "
|
||||
f"Real GSM8K dev holdout: {ingested}/{len(cases)} problems ingestible "
|
||||
f"(Tier-1 single-accumulator + Tier-2 multi-entity/transfer/summation) and "
|
||||
f"solved wrong=0. Derived-operand transfers / >5-atom shapes remain the frontier.",
|
||||
"Baseline is a symbolic fold of the SAME compiled program; PARITY "
|
||||
"with wrong=0 is the honest outcome — the field matches arithmetic, it "
|
||||
"does not beat it. The result is real-holdout coverage, not a lift.",
|
||||
),
|
||||
|
|
@ -579,11 +613,12 @@ def run_generalized_lift_instrument() -> LiftInstrumentReport:
|
|||
wrong_zero_guard_held=(propositional.corridor_wrong == 0 and arithmetic.corridor_wrong == 0),
|
||||
honest_null=all(o.delta_correct <= 0 for o in outcomes),
|
||||
scope_limitations=(
|
||||
"The reader→Hamiltonian compiler now exists for the affine "
|
||||
"single-accumulator arithmetic subset (ADR-0249): on the real GSM8K "
|
||||
"dev holdout it solves the Tier-1 subset wrong=0 (see the "
|
||||
"arithmetic-chain domain). Non-affine / multi-entity GSM8K problems "
|
||||
"(transfer, rate, comparison, fraction, partition) remain the "
|
||||
"recorded Tier-2 frontier — refused, never silently dropped.",
|
||||
"The reader→Hamiltonian compiler now solves the FULL real GSM8K dev "
|
||||
"holdout wrong=0 (Tier-1 single-accumulator, ADR-0249; Tier-2 "
|
||||
"multi-entity + constant-operand transfers + certified summation, "
|
||||
"ADR-0250) — see the arithmetic-chain domain. Remaining general "
|
||||
"frontier: derived-operand transfers ('half of X') and non-affine "
|
||||
"kinds (rate/comparison/fraction/partition), both 0 on this holdout; "
|
||||
"and >5-atom deduction. Refused, never silently dropped.",
|
||||
),
|
||||
)
|
||||
|
|
|
|||
|
|
@ -27,7 +27,8 @@ def test_domain_solves_real_gsm8k_wrong_zero(domain) -> None:
|
|||
assert domain.domain_id == "arithmetic-chain"
|
||||
assert domain.corridor_wrong == 0 # wrong=0 on real GSM8K, not just templates
|
||||
assert domain.corridor_correct > 0 # the corridor ingests and solves real problems
|
||||
assert domain.corridor_refused > 0 # and honestly refuses what it can't ingest
|
||||
# Tier-2 (ADR-0250) closed the multi-entity frontier: the full holdout ingests now.
|
||||
assert domain.corridor_refused == 0
|
||||
|
||||
|
||||
def test_every_ingested_case_is_correct(domain) -> None:
|
||||
|
|
@ -50,10 +51,10 @@ def test_coverage_is_recorded_not_dropped(domain) -> None:
|
|||
ingested = domain.corridor_correct + domain.corridor_wrong
|
||||
assert ingested + domain.corridor_refused == domain.n_cases
|
||||
assert domain.n_cases > 0
|
||||
# A coverage tracker against the sealed dev holdout (50 cases, 26 Tier-1).
|
||||
assert domain.corridor_correct == 26
|
||||
assert domain.corridor_refused == 24
|
||||
assert any("Tier-1" in note for note in domain.notes)
|
||||
# Coverage tracker against the sealed dev holdout: Tier-1 (26) + Tier-2 (24) = 50/50.
|
||||
assert domain.corridor_correct == 50
|
||||
assert domain.corridor_refused == 0
|
||||
assert any("Tier-2" in note for note in domain.notes)
|
||||
|
||||
|
||||
def test_domain_joins_full_report_and_scope_corrected() -> None:
|
||||
|
|
@ -61,8 +62,8 @@ def test_domain_joins_full_report_and_scope_corrected() -> None:
|
|||
ids = [o.domain_id for o in report.outcomes]
|
||||
assert "arithmetic-chain" in ids
|
||||
assert report.wrong_zero_guard_held # deductive + arithmetic both wrong=0
|
||||
# The stale "no compiler exists" scope note must be gone; the compiler exists now.
|
||||
# The scope note reflects the full-holdout closure; the stale "no compiler" line is gone.
|
||||
joined = " ".join(report.scope_limitations)
|
||||
assert "now exists" in joined
|
||||
assert "FULL real GSM8K dev holdout" in joined
|
||||
assert "no reader-to-Hamiltonian compiler exists" not in joined
|
||||
json.dumps(report.as_dict()) # JSON-safe artifact
|
||||
|
|
|
|||
Loading…
Reference in a new issue