docs(reader-arc): fold PR #76 amendments into compare increment plan; X=15 ratified

Four amendments (Shay, #76 review) folded:
1. Summation registers-driven (NEW CODE): summation orders by program.seeds
   (initial_state) today, so a compare-DEFINED register (no seed) is silently
   excluded from the total — all 5 real cases are totals. Fix: sum all registers
   at solve time, deterministic order (seed order then definition order);
   compile-side validation admits defined entities as answer targets with unit
   propagation from the reference. Pin: defined register in the certified sum.
2. Compare record spec: record identifies the reference entity + binds the
   reference state digest via the existing conditional operand_source_digest
   (non-compare/non-summation digests unchanged); deterministic re-verification
   reconstructs the dilation's source register from records alone.
3. Tune/measure split pinned deterministically (sha256(id)-ordered) FIRST,
   before any grammar work, documented + fixed in advance.
4. 'N times more than' pinned to N× as a disclosed convention; gold mismatches
   tracked as recorded findings, never silent wrongs.

X = 15 ratified as the arc escalation threshold (spike §7.3 updated). §6 risks
reclassified (risk 4 -> new code §3.1; risk 1 -> pinned convention §4). Build
proceeds under this ruling — no separate re-ruling.
This commit is contained in:
Shay 2026-07-18 16:07:41 -07:00
parent c88e52c6a9
commit 09172da204
2 changed files with 59 additions and 25 deletions

View file

@ -1,6 +1,7 @@
# Increment Plan — `compare_multiplicative` (reader arc, increment 1)
**Status**: PLAN — design-first, for ruling before any build
**Status**: APPROVED with four amendments (PR #76 ruling, folded below); **X = 15 ratified** as the
arc's escalation threshold. Build proceeds under this ruling — no separate re-ruling.
**Date**: 2026-07-18
**Base**: `forgejo/main @ dd1e4670` (dev2 spike + errata merged, #75)
**Branch**: `feat/compare-mult-increment-plan`
@ -52,10 +53,26 @@ the multi-register framework; two things are new and must be designed explicitly
- `direction='fraction'` (factor < 1, "half as many as") is the same dilator with `factor < 1` folded
in. `compare_additive` (`more`/`fewer`, a translator) is a **later** increment (frequency order §7.2).
This is the whole compiler half: a new op branch that dilates a reference register into an actor
register, plus admitting compare-defined registers. The certified record chain and the existing
summation turn carry the rest unchanged. Expected immediate effect: **corridor real-reach 0/500 → the
compare parses that already exist** — the §7.2 "loop works end-to-end on official data" proof.
Expected immediate effect: **corridor real-reach 0/500 → the compare parses that already exist** — the
§7.2 "loop works end-to-end on official data" proof. Two amendments (PR #76) make this correct rather
than "unchanged":
### 3.1 Summation becomes registers-driven (AMENDMENT 1 — new code, not a confirmation)
The certified summation turn today orders by `program.seeds`, which come from `initial_state`. A
compare-**defined** register has **no seed**, so as currently coded it would be **silently excluded
from the total** — and all 5 real cases are totals. Fix: the summation sums **all registers present at
solve time**, in a **deterministically pinned order** (seed order, then definition order). Compile-side
validation must admit **defined entities as answer targets** (a total or a concrete unknown may be a
compare-defined register), with **unit propagation from the reference** (the defined register inherits
the reference's unit). Pin test: a compare-defined register appears in the certified sum.
### 3.2 The compare record (AMENDMENT 2 — specify before build)
Cross-register provenance must live in the chain so deterministic re-verification can reconstruct
**which register fed the dilation from records alone** (same reasoning as the summation-ruling). The
`compare_multiplicative` record: identifies the **reference entity**, and binds the **reference state's
digest** via the existing conditional `operand_source_digest` (present only on cross-register records,
so non-compare/non-summation record digests stay byte-identical). Verification re-derives the source
register from the record and checks the dilation reproduces.
## 4. Reader grammar family — broaden `compare_multiplicative` parsing
@ -67,6 +84,12 @@ Today the reader emits `compare_multiplicative` for ~5/500 — narrow phrasings
many more official problems parse to a correct compare graph. Reader work lives in the
`generate/math_candidate_*` parser; this half is where most of the coverage delta comes from.
**"N times more than" — pinned convention (AMENDMENT 4).** The grammar family reads "N times more
than X" as **N×X** (not (N+1)×X), fixed as an **explicit disclosed convention** — the ambiguity is
real, so the reading is a stated choice, not an assumption. Every gold mismatch attributable to this
reading is **tracked as a recorded finding** (never a silent wrong); if the corpus systematically means
(N+1)×, the record will show it and the convention flips on evidence.
**Honest scope:** this increment solves problems whose only frontier kind is `compare_multiplicative`
(plus basic ops + summation). Multi-frontier problems (spike §3: 16/100 carry ≥2 markers) wait for their
other families' increments — recorded, not counted here.
@ -76,28 +99,39 @@ other families' increments — recorded, not counted here.
- **Signal = official `holdout_dev` only** — never the authored corpus.
- **`wrong=0` is the floor** — a compare parse counts only if `compile → execute → gold` agrees.
- **Sealed test (1,319) untouched.**
- **Disjoint split:** the reader grammar is tuned against one `holdout_dev` sub-split and the increment's
delta is measured on a **held-out disjoint** sub-split — "better" can only mean "generalizes."
- **Delta reported** = new-correct on the measured split; the increment must clear **X = 15** (spike
§7.3, pending Shay's ruling) or it triggers a practice-lane ruling. Compare incidence (~20%) means a
paying reader family should convert well above 15.
- **Disjoint split, pinned deterministically FIRST (AMENDMENT 3).** Before any grammar work, the 500
`holdout_dev` cases are partitioned by a **deterministic protocol**`sha256(id)`-ordered assignment
(e.g. even/odd rank → tune / measure), documented in the plan and **fixed in advance**. The reader
grammar is tuned only against the *tune* split; the increment's delta is measured only on the disjoint
*measure* split. Disjointness is only as strong as its enforcement, so it is committed before a line of
grammar changes — "better" can only mean "generalizes."
- **Delta reported** = new-correct on the measured split; the increment must clear **X = 15** (ratified,
spike §7.3) or it triggers a practice-lane ruling. Compare incidence (~20%) means a paying reader
family should convert well above 15.
## 6. Risks / open decisions
## 6. Risks / open decisions (post-amendment)
1. **"N times more than"** is linguistically ambiguous (N× vs (N+1)×); the reader currently reads it as
N× and gold agreed on the sampled 5, but the grammar family must pin one reading and measure, not
assume.
1. **"N times more than"** — resolved to a pinned N× convention with tracked mismatches (§4, AMENDMENT 4).
2. **Garbled entity spans** (0268's actor is a run-on string) — cosmetic today (answer still correct),
but the reader family should tighten entity extraction; a wrong *entity binding* could mis-route a
compare in a 3-entity problem.
compare in a 3-entity problem. Remaining open risk.
3. **Compare-then-op ordering** — if a defined register is later transferred/scaled, the compare must
resolve first; the executor already runs ops in story order, so this is a test to pin, not new code.
4. **Register-introduced-by-compare** interacts with the total summation (which sums all registers) —
confirm the defined register is included in the sum (it must be).
4. **Register-introduced-by-compare in the total** — reclassified: this is **new code**, not a
confirmation (§3.1, AMENDMENT 1). Summation is registers-driven with a pinned order; a defined
register that is the answer target propagates its unit from the reference.
## 7. Increment PR (what actually ships, after this plan is ruled on)
## 7. Increment PR (what ships next, under this ruling)
One smoke-gated PR: the compiler tier (§3) + the reader grammar family (§4) + tests
(the 5 real parses solved end-to-end wrong=0; compare-defines-register; no-conservation-on-compare;
fraction direction; the disjoint-split holdout delta) + the measured `holdout_dev` delta recorded
against X. Nothing else in it.
One smoke-gated PR: the compiler tier (§3, including §3.1 registers-driven summation + §3.2 compare
record) + the reader grammar family (§4) + tests + the measured delta. Test set:
- the 5 real parses solved end-to-end, `wrong=0`;
- **compare-defined register appears in the certified sum** (§3.1 pin);
- defined-entity answer target with unit propagated from the reference (§3.1);
- **compare record re-verification** — source register reconstructed from records alone (§3.2);
- no-conservation-on-compare; `fraction` direction;
- N× convention pinned, mismatches surfaced as findings (§4);
- the deterministic tune/measure split fixed before grammar work (§5, AMENDMENT 3);
- the measured `holdout_dev` **measure-split** delta recorded against **X = 15**.
Nothing else in it. Build proceeds now under this ruling.

View file

@ -139,9 +139,9 @@ the `wrong=0` floor held, and the sealed test untouched.
official data.
2. Then strict §3 frequency order: **rate (31) → fraction (28) → compare_additive (9) → partition (1)**.
### 7.3 Escalation threshold X — PROPOSED (Shay rules with the merge)
Fixed now, before anyone is attached, so "switch when design stops working" cannot drift.
**Proposed X = 15 new-correct on `holdout_dev` per full family increment.** When a completed increment
### 7.3 Escalation threshold X — **RATIFIED (Shay, 2026-07-18, PR #76): X = 15**
Fixed before anyone is attached, so "switch when design stops working" cannot drift.
**X = 15 new-correct on `holdout_dev` per full family increment.** When a completed increment
buys fewer than X, escalate to a practice-lane ruling. Rationale: these families carry 2031% incidence
(~100155/500 problems each); a design that is paying converts far more than 15, so falling below
15/500 (3%) means design has stopped efficiently converting known incidence — a number trips the