core-labs/core
core-labs/core
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

feat(adr-0250): instrument routes Tier-2 (50/50) + ADR-0250 — Tier-2 arc closure #74

Merged
core-labs merged 1 commit from feat/adr-0250-instrument-adr into main 2026-07-18 21:56:02 +00:00
Conversation 2 Commits 1 Files changed 4 +226 -36
core-labs commented 2026-07-18 21:48:23 +00:00
Owner
Copy link

Closes the Tier-2 arc (spike #71, T2a #72, 2b #73 all merged): the instrument now measures the full capability, and ADR-0250 lands for ratification.

Instrument routes Tier-2 (measurement catches up to capability)

evals/generalized_lift_instrument.py — the arithmetic-chain domain now routes each real GSM8K dev case to Tier-1 (single-accumulator) or Tier-2 (multi-register / summation) and compares the corridor to a symbolic fold of the same compiled program. Recorded coverage moves 26/50 → 50/50, 0 refused, PARITY (wrong=0 — the field matches arithmetic, it does not beat it). The scope note is corrected (full holdout closed; remaining frontier = derived-operand transfers + non-affine kinds + >5-atom deduction, all 0 on this holdout). The ADR-0249 instrument test pins are updated honestly (26→50, refused 24→0).

ADR-0250 (Proposed — no self-Accept)

docs/adr/ADR-0250-*.md documents the Tier-2 arc: the multi-register product-of-lines model, coupled-translator transfers, the relative conservation hard-reject pin, prepare→validate→commit atomicity, the certified summation turn, and the chain-of-custody (operand_source_digest). docs/handoff/ADR-0250-Acceptance-Evidence.md collects the phase SHAs, the full-holdout result, and the in-tree verifications. §10 ruling record awaits your ratification.

Headline

The reader→Hamiltonian compiler now solves the entire real GSM8K dev holdout — 50/50, wrong = 0 — by chained certified relaxation (26 Tier-1 + 18 T2a single-entity + 6 certified summation). Every step certified, conservation-gated, tamper-evidently recorded, off-serving.

Governance

Off-serving (A-04); no gate activated (ADR-0246 §3.7 stays uncalibrated); no manifold mutated (I-03); ratified contracts reused. Every phase smoke-gated green.

Tests

tests/test_adr_0249_arithmetic_lift.py (updated) + tests/test_generalized_lift_instrument.py — 10/10 green (50/50 coverage, PARITY, scope corrected, JSON-safe artifact).

[Verification]: uv run python -m pytest tests/test_adr_0249_arithmetic_lift.py tests/test_generalized_lift_instrument.py -q → 10 passed; uv run core test --suite smoke -q → 176 passed.

Closes the Tier-2 arc (spike #71, T2a #72, 2b #73 all merged): the instrument now measures the full capability, and ADR-0250 lands for ratification. ## Instrument routes Tier-2 (measurement catches up to capability) `evals/generalized_lift_instrument.py` — the `arithmetic-chain` domain now routes each real GSM8K dev case to Tier-1 (single-accumulator) or Tier-2 (multi-register / summation) and compares the corridor to a symbolic fold of the **same** compiled program. Recorded coverage moves **26/50 → 50/50, 0 refused, PARITY** (wrong=0 — the field matches arithmetic, it does not beat it). The scope note is corrected (full holdout closed; remaining frontier = derived-operand transfers + non-affine kinds + >5-atom deduction, all 0 on this holdout). The ADR-0249 instrument test pins are updated honestly (26→50, refused 24→0). ## ADR-0250 (Proposed — no self-Accept) `docs/adr/ADR-0250-*.md` documents the Tier-2 arc: the multi-register product-of-lines model, coupled-translator transfers, the **relative** conservation hard-reject pin, prepare→validate→commit **atomicity**, the **certified summation turn**, and the **chain-of-custody** (`operand_source_digest`). `docs/handoff/ADR-0250-Acceptance-Evidence.md` collects the phase SHAs, the full-holdout result, and the in-tree verifications. §10 ruling record awaits your ratification. ## Headline The reader→Hamiltonian compiler now solves the **entire real GSM8K dev holdout — 50/50, wrong = 0** — by chained certified relaxation (26 Tier-1 + 18 T2a single-entity + 6 certified summation). Every step certified, conservation-gated, tamper-evidently recorded, off-serving. ## Governance Off-serving (A-04); no gate activated (ADR-0246 §3.7 stays uncalibrated); no manifold mutated (I-03); ratified contracts reused. Every phase smoke-gated green. ## Tests `tests/test_adr_0249_arithmetic_lift.py` (updated) + `tests/test_generalized_lift_instrument.py` — **10/10 green** (50/50 coverage, PARITY, scope corrected, JSON-safe artifact). [Verification]: `uv run python -m pytest tests/test_adr_0249_arithmetic_lift.py tests/test_generalized_lift_instrument.py -q` → 10 passed; `uv run core test --suite smoke -q` → 176 passed.
core-labs added 1 commit 2026-07-18 21:48:24 +00:00 
Instrument: the arithmetic-chain domain now routes each real GSM8K dev case to
Tier-1 (single-accumulator) or Tier-2 (multi-register / summation) and compares
the corridor to a symbolic fold of the SAME compiled program. Recorded coverage
moves 26/50 -> 50/50, 0 refused, PARITY (wrong=0) — the measurement catching up
to the capability. Scope note corrected (full holdout closed; remaining frontier
= derived-operand transfers + non-affine kinds + >5-atom deduction, all 0 on this
holdout). Updated the ADR-0249 instrument test pins (26->50, refused 24->0).

ADR-0250 (Status Proposed, no self-Accept; §10 ruling awaits Shay) + acceptance
evidence: multi-register model, relative conservation, atomicity, certified
summation, chain-of-custody, and the full-holdout 50/50 wrong=0 result.

10/10 instrument pins green.

[Verification]: uv run python -m pytest tests/test_adr_0249_arithmetic_lift.py tests/test_generalized_lift_instrument.py -q
core-labs merged commit 09257ed32f into main 2026-07-18 21:56:02 +00:00
core-labs deleted branch feat/adr-0250-instrument-adr 2026-07-18 21:56:04 +00:00
core-labs commented 2026-07-18 21:56:38 +00:00
Author
Owner
Copy link

Review — approve (nothing blocking).

Verified in review: instrument↔executor API match (seeds/turns/answer_entity/.answer); _symbolic_multi_register is semantically faithful to execute_multi_register_program (affine scale·x+offset, transfer ∓k, seed-order summation on answer_entity is None); conservation pin relative rtol=1e-6 matches ADR §3; prepare→validate→commit with an immutable register mapping matches §4; conditional operand_source_digest payload matches §5; evidence SHAs line up (#71 97b2f8b7, #72 efd5335e, #73 3c3dcdbe = this PR's base). Test pins updated honestly (26→50, refused 24→0).

Findings — non-blocking, for post-merge housekeeping:

  1. Stale module docstring — the top of evals/generalized_lift_instrument.py still says "Three deterministic domains" and "GSM8K … NOT ingestible … there is no reader→Hamiltonian compiler". That contradicts the file's own arithmetic-chain domain since ADR-0249, and this is the "measurement catches up" PR — the header should catch up too. (The tests only pin the report's scope_limitations tuple, not the docstring, which is how it slipped.)
  2. Refused rows lost their reason — _corridor_and_baseline returns bare None; the old rows carried "refusal": reason / "no_graph". refused=0 on this holdout so no rows are affected today, but "recorded, never silently dropped" should stay literal the moment a derived-operand case appears: return/record MultiRegisterError.reason (and distinguish graph-parse failures).
  3. Tier-1 try-scope — the except TurnProgramError wraps execute_turn_program + _symbolic_fold, not just compile. execute_turn_program never raises it today (require_converged=False), but a future typed raise would silently reroute a Tier-1 execution failure into Tier-2 and mask a regression. Tighten to compile-only. Related one-line doc note worth adding: Tier-2 execution fails closed on non-convergence while Tier-1 records-only and the instrument reads .answer without .certified — the gold comparison is what binds wrong=0 on both paths.
**Review — approve (nothing blocking).** Verified in review: instrument↔executor API match (`seeds`/`turns`/`answer_entity`/`.answer`); `_symbolic_multi_register` is semantically faithful to `execute_multi_register_program` (affine `scale·x+offset`, transfer ∓k, seed-order summation on `answer_entity is None`); conservation pin relative rtol=1e-6 matches ADR §3; prepare→validate→commit with an immutable register mapping matches §4; conditional `operand_source_digest` payload matches §5; evidence SHAs line up (#71 `97b2f8b7`, #72 `efd5335e`, #73 `3c3dcdbe` = this PR's base). Test pins updated honestly (26→50, refused 24→0). Findings — non-blocking, for post-merge housekeeping: 1. **Stale module docstring** — the top of `evals/generalized_lift_instrument.py` still says "Three deterministic domains" and "GSM8K … NOT ingestible … there is no reader→Hamiltonian compiler". That contradicts the file's own `arithmetic-chain` domain since ADR-0249, and this is the "measurement catches up" PR — the header should catch up too. (The tests only pin the report's `scope_limitations` tuple, not the docstring, which is how it slipped.) 2. **Refused rows lost their reason** — `_corridor_and_baseline` returns bare `None`; the old rows carried `"refusal": reason` / `"no_graph"`. refused=0 on this holdout so no rows are affected today, but "recorded, never silently dropped" should stay literal the moment a derived-operand case appears: return/record `MultiRegisterError.reason` (and distinguish graph-parse failures). 3. **Tier-1 try-scope** — the `except TurnProgramError` wraps `execute_turn_program` + `_symbolic_fold`, not just compile. `execute_turn_program` never raises it today (`require_converged=False`), but a future typed raise would silently reroute a Tier-1 execution failure into Tier-2 and mask a regression. Tighten to compile-only. Related one-line doc note worth adding: Tier-2 execution fails closed on non-convergence while Tier-1 records-only and the instrument reads `.answer` without `.certified` — the gold comparison is what binds wrong=0 on both paths.
core-labs commented 2026-07-18 22:37:34 +00:00
Author
Owner
Copy link

Erratum (label correction). This PR's description and my earlier evidence called evals/gsm8k_math/dev / public the "real GSM8K dev holdout." Per ADR-0119.2 those are CORE-authored GSM8K-style problems, not real GSM8K. Corrected — reframe, not retraction: the mechanism results (wrong=0, PARITY, 50/50, 200/200) stand as the mechanism-correctness instrument; only the corpus label falls.

Measured real-GSM8K reach (official holdout_dev, 500): reader parses 5/500, corridor solves 0/500 — reader-gated. Prominent ERRATA sections added to ADR-0249, ADR-0250, and both acceptance-evidence docs in PR #75, with the full analysis in docs/research/dev2-frontier-measurement-spike-2026-07-18.md.

**Erratum (label correction).** This PR's description and my earlier evidence called `evals/gsm8k_math/dev` / `public` the "real GSM8K dev holdout." Per ADR-0119.2 those are **CORE-authored GSM8K-*style*** problems, **not** real GSM8K. Corrected — reframe, not retraction: the mechanism results (wrong=0, PARITY, 50/50, 200/200) stand as the *mechanism-correctness* instrument; only the corpus label falls. Measured real-GSM8K reach (official `holdout_dev`, 500): reader parses 5/500, corridor solves 0/500 — reader-gated. Prominent ERRATA sections added to ADR-0249, ADR-0250, and both acceptance-evidence docs in PR #75, with the full analysis in `docs/research/dev2-frontier-measurement-spike-2026-07-18.md`.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: core-labs/core#74
No description provided.
Delete branch "feat/adr-0250-instrument-adr"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?