Commit graph

2 commits

Author SHA1 Message Date
Shay
324db57373 feat(adr-0249): P5 arithmetic-chain lift domain on real GSM8K holdout
Wires the turn-program executor into the generalized-lift instrument and
measures it against the REAL GSM8K dev holdout (evals/gsm8k_math/dev, never the
templated cases), using each problem's ground_truth_graph. Baseline is a
symbolic fold of the SAME compiled turn program: both paths consume identical
problems, the corridor relaxes each step, the baseline folds it numerically.

Honest result on the sealed 50-case dev holdout: 26 problems are Tier-1 affine
single-accumulator ingestible and solved wrong=0; the corridor matches the
symbolic fold on every one (PARITY, delta=0) — the field matches arithmetic, it
does not beat it, so this is real-holdout coverage, NOT a manufactured lift. The
24 refused (all not_single_accumulator / multi-entity) are the recorded Tier-2
frontier, never silently dropped.

Corrects the now-stale scope-limitation note ('no reader-to-Hamiltonian compiler
exists…') — the compiler exists as of this arc; the note now records real-data
Tier-1 coverage and the multi-entity frontier. wrong_zero_guard_held extended to
bind both exact-regime domains (deductive flagship + arithmetic).

10/10 pins (5 new + 5 existing instrument tests still green).

[Verification]: uv run python -m pytest tests/test_adr_0249_arithmetic_lift.py tests/test_generalized_lift_instrument.py -q
2026-07-18 13:07:40 -07:00
Shay
62a72deba0 feat(evals): seams S4+S5 — generalized-lift instrument + ports/handoff evidence
S4: evals/generalized_lift_instrument.py — corridor vs symbolic baseline on
identical compiled problems, independent truth-table gold, honest-NULL
protocol, no-silent-caps (GSM8K non-ingestibility RECORDED as the
composition frontier). Live results: propositional PARITY (corridor 10/10 =
ROBDD 10/10, wrong=0 guard HELD), constrained-recognition LIFT +9 (relax+
readback 9/9 vs constraint-blind argmax 0/9; round-trip agreement 1.0),
multimodal-completion PARITY (vision token already resonates with the
audio partial — measured, disclosed).

S5: evals/lift_evidence_handoff.py — two REAL certified turns through
IdentityPort+PrecisionPort (Ring-2) and coordinate_handoff (Ring-3),
off-serving, flags untouched: identity-action turn PROCEEDs, frame-rotating
turn ABSTAINs with typed port-attributed reason (d_stab>epsilon_turn); both
replay chains verify. Evidence doc: docs/handoff/ADR-0246-Acceptance-
Evidence.md (for Shay's §8 rulings — no self-Accept). Plan doc phases 0-5
marked DONE.

[Verification]: Smoke suite passed locally (129s, 176 passed); 147 passed
across all touched+adjacent suites.
2026-07-18 10:34:26 -07:00