Shay 639616b3e5 docs(dev2-spike): errata + two-instrument reframe + incidence + reader-arc design

Folds in Shay's rulings on the spike:
- ERRATA sections added to ADR-0249, ADR-0250, and both acceptance-evidence docs
  (statuses stay Accepted): corpus label 'real GSM8K dev holdout' -> 'CORE-authored
  GSM8K-style corpus (ADR-0119.2)'. Mechanism claims (wrong=0, PARITY, conservation,
  atomicity, chain-of-custody, 200/200) all stand; reframe not retraction. Covers the
  #74 review comment that repeated the label. (Also fixed ADR-0250 evidence status ->
  RATIFIED, which the ratification housekeeping missed.)
- Two instruments named + never-blended scoreboard: authored = mechanism-correctness
  (200/200 stands as that); official holdout = generalization (5/500 symbolic / 0/500
  corridor). public/dev-1 floors relabeled MECHANISM floors, composition-disclosed.
- Censored-incidence fix: parse-INDEPENDENT tagging of 100 official cases (documented
  heuristic). 70/100 carry frontier markers (rate 31, fraction 28, compare_mult 20) ->
  (c) refuted, frontier COMMON not rare; the table is the reader-arc curriculum by
  real-world frequency.
- Reader-practice-loop design with the four non-negotiable constraints (real held-out
  data only; wrong=0 gates consolidation; sealed test untouched; holdout_dev split
  disjoint from the measured metric). Sequencing: capability->practice->calibration->
  serve; reader is the critical path for all three; serve-landing de-prioritized (0/500).

Spike verdict for ruling; nothing built.

2026-07-18 15:37:01 -07:00

3.8 KiB

Raw Blame History

ADR-0249 — Acceptance Evidence

ERRATA (2026-07-18): "real GSM8K dev holdout" in this doc = CORE-authored GSM8K-style corpus (ADR-0119.2), the mechanism-correctness instrument — not real GSM8K. Mechanism results (wrong=0, PARITY, 200/200) stand; only the corpus label is corrected. Measured real-GSM8K reach: reader 5/500, corridor 0/500 (reader-gated). See ADR-0249 §ERRATA + docs/research/dev2-frontier-measurement-spike-2026-07-18.md.

Status: RATIFIED — ADR-0249 Accepted 2026-07-18 by Joshua Shay for commit 220039f7 Date: 2026-07-18 ADR: docs/adr/ADR-0249-reader-hamiltonian-compiler-composition-frontier.md Design record: docs/research/reader-hamiltonian-compiler-spike-2026-07-18.md

The reader→Hamiltonian compiler closes the composition frontier the generalized-lift instrument (ADR-0246 S4) was waiting to measure. Built in five smoke-gated phases, each merged under its own PR off forgejo/main.

Phases merged

Phase	PR	Merge commit	Module	Pins
P1 quantity kernel	#66	`aca98d85`	`core/physics/quantity_kernel.py`	35/35
P2 relation compiler	#67	`2575d0d9`	`core/physics/relation_compiler.py`	21/21
P3 CNF converter	#68	`309e0ef6`	`evals/logic_cnf_compiler.py`	32/32
P4 turn-program executor	#69	`805752db`	`evals/turn_program.py`	15/15
P5 instrument integration	this PR	—	`evals/generalized_lift_instrument.py`	10/10

Smoke (uv run core test --suite smoke -q) green at every phase boundary (176 passed).

Load-bearing results

Multi-step arithmetic by chained certified relaxation (P4): ((5+3)*2)-4 = 12, (10/2+7)*3 = 36, (100-40)/4 = 15 — every turn ground_state_certified; accumulator flows as a field state, decoded once.

Real GSM8K dev holdout (P5, sealed 50 cases, evals/gsm8k_math/dev):

Metric	Value
Tier-1 affine ingestible	26 / 50 (52%)
Corridor correct on ingested	26 / 26 — wrong = 0
Corridor vs symbolic fold	PARITY (delta = 0)
Refused (multi-entity)	24 / 50 — recorded Tier-2 frontier

The corridor solves real GSM8K problems it can ingest with zero wrong answers, and honestly refuses the rest. It does not beat arithmetic at arithmetic (PARITY is honest); the deliverable is real-data coverage + a decidable measurement, per the honest-NULL protocol.

Algebra verified in-tree (P1): conformal line embedding + translator/dilator exactness checked against algebra/cl41.py's own multiplication table (nullity < 3e-10; translator exact to f64 rounding; dilator sign convention pinned).

CNF soundness proved (P3): for a 14-formula panel, the compiled CNF has the same ROBDD identity as the source (canonicalize oracle); end-to-end entailment agrees with evaluate_entailment gold, wrong = 0 on consistent premises.

Design corrections applied (honest register)

Ring-2 is non-mutating. run_residual_protocol is zero-bound (stage-5 recertification refuses a moved witness), so it certifies fixed-state admissibility, not transitions. The turn ledger reuses the chain-integrity pattern (TurnRecord / verify_turn_chain), not the admission protocol.
cga_inner is the wrong metric for versor states — readback and quantity decode use phase_correlation/2, pinned in the P1 module.
Scope note corrected — the instrument's stale "no compiler exists" line is replaced with measured Tier-1 coverage + the multi-entity frontier.

What remains open (not claimed here)

Multi-entity / non-affine GSM8K (transfer, rate, comparison, fraction) — Tier-2, refused and recorded.
5-atom deduction via ROBDD-partitioned turn programs — next arc.
ADR-0246 §3.7 admission-policy calibration — unchanged; no gate activated.
generate/ ("core_logos") serve-path articulation upgrade — still deferred.

3.8 KiB Raw Blame History