core/docs/analysis/gsm8k-lift-program-strategy-2026-06-04.md

GSM8K Lift Program — Strategy for Sizeable, Sealed-Verified Lift

Goal (stated honestly): move the real number — the sealed 1,319 (and full GSM8K), not the 50-case train_sample — by meaningful chunks, with wrong=0 preserved at every step. This is the program plan, sequenced from the substrate ADR-0207 ratified.

⚠ The brutal baseline (read this first)

The real external number is 0 / 0 / 1319 — sealed real GSM8K test (HuggingFace openai/gsm8k), claims_ledger row A: 0 correct out of 1,319. CORE solves nothing on held-out GSM8K. The train_sample 7/43/0 is on a 50-case unsealed sample CORE was effectively built against — the ledger's own rule: "never present A as an accuracy."

So "sizeable lift" means getting the sealed number off zero — a fundamentally harder thing than the train_sample increments suggest. The R4 win (cv-0005) moved a proxy; it may be +0 on the real 1,319 (goal-residual fires on 2/455 visible cases; whether it fires on any sealed case is unknown until Stream 0 runs). Every magnitude claim below is against this: we are at zero on the bar that counts.

Grounding (origin/main 3a72d69, 2026-06-04): train_sample 7/43/0 (R4 goal-residual landed, sealed-pending). Composition class frequencies across the 44 refusals (multi-tagged, composition-capability-scope §8): R5 multi-step 27, R1 derived-symbol 24, R6 percent 18, R4 residual 10, R2 inverse 6, R3 partition 3. Only R4 is landed.

---

0. The leverage equation (why this plan is shaped as it is)

lift ≈ Σ over shapes of [ class_frequency × tractability × sealed_transfer ], divided by per-shape cost — and per-shape cost is dominated by the hand-built promotion bridge, not the composer.

Three consequences fall straight out, and they set the streams:

1. R4 gave +1 because it is rare (10/44, and most are multi-referent). Chasing more R4-like rare shapes one-by-one is a trickle. The big frequencies are R1 (24) and R5 (27) — but both need hard productions (R1 = quantity reuse / DAG; R5 = multi-step rate/duration).
2. Per-shape cost is the bridge, not the reading. Today every shape needs a hand-coded serving promotion gate (product_bridge, resolve_promotable_goal_residual). That is the tax that makes lift one-shape-at-a-time. Removing it (Stream A) is the force multiplier.
3. No lift counts until the sealed set says so. train_sample is a 50-case proxy; the 7/43/0 win is unverified on the real bar. Sealed measurement is the prerequisite, not a formality (Stream 0).

Honest magnitude expectation: there is no single move that jumps the sealed number by a big chunk cheaply and safely. Sizeable lift is compounding: build the flywheel + the general consumption bridge so each subsequent shape is cheap, then spend the expensive research on the high-frequency shapes (R1/R5). The curve bends up when per-shape cost drops, not from any one production.

---

Stream 0 — Sealed baseline (PREREQUISITE, blocks everything)

Until this runs, every number below is train_sample theater.

• 0.1 Resolve docs/handoff/sealed-measurement-obligation-2026-06-04.md: operator/CI decrypts + runs the sealed 1,319 at HEAD, confirms sealed wrong==0 and records the sealed correct count. Did R4 (cv-0005) actually move the sealed number, or was it +0 on held-out? This answer calibrates the whole program.
• 0.2 Stand up a repeatable sealed-measurement gate the operator can run per-increment (decrypt → parse_and_solve → counts → ledger row). Without a per-increment sealed check, the program cannot tell real lift from overfitting. This is the single most important infrastructure item — the program's measuring stick.
• Exit: a known sealed baseline (correct, 0, refused) and a one-command sealed re-measure.

Stream A — The force multiplier: general composition-promotion consumer

The highest-leverage infrastructure in the program. Replace N hand-built promotion bridges with one gated consumer.

• A.1 Design a ratified-composition-frame → structural-promotion-gate → serving bridge: a single serving consumer that takes a ratified frame (shape + op-class + target signature) and promotes any reading that passes the generalized gate (extract_target + target_units
  • self-verify grounding∧unit∧completeness + the divergence-firewall pattern). product_bridge and goal_residual become instances of this consumer, not bespoke code.
• A.2 The wrong=0 firewall is the entire risk: an auto-promotion consumer that admits a wrong frame is the prime-directive violation. Gate it harder than any single bridge — every promoted frame carries its own divergence-firewall test (the goal-vs-possession pattern, generalized) and a sealed-gated ratification.
• A.3 Wire cue-precision (ADR-0177) as the ranking signal into this consumer (it is currently inert / consumed nowhere on serving) — it ranks which frame promotes when several self-verify, replacing per-shape disagreement-refusal with learned precision.
• Payoff: after A, a new shape is "ratify a frame + its firewall test," not "write a bridge." Per-shape cost collapses; the flywheel's output finally reaches serving.

Stream B — Harvest at scale (the trickle, industrialized)

The cheap wins, mined from the real corpus instead of the 50.

• B.1 Acquire full GSM8K train (7,473 cases, public) into a harvest lane (the repo has only the 50-sample + the encrypted sealed holdout — the harvest pool must be added).
• B.2 Point the existing practice / contemplation / propose loop (evals/gsm8k_math/practice/v1/, propose_runner.py) at the full train set: attempt → diagnose refusals → for each, classify structural match but lexeme/frame miss vs needs new production.
• B.3 The lexical-variant harvest (your idea, grounded): structurally-built shapes that refuse only on a closed-set miss — a goal verb, a progress verb (saved is a live example — not in the change-cue vocab), a residual cue. Each becomes a one-lexeme ratification through the contemplation→HITL corridor, firewall-gated, landed via Stream A. The visible proxy showed a ~58-case remainder-question family (9% of visible) — at full-train scale this is the steady trickle, if A makes landing cheap.
• Honest cap: B is a trickle, not a flood. Its value is steady + cheap, and it compounds only once Stream A removes the landing tax.

Stream C — The high-frequency research bets (where the big chunk lives)

The expensive, uncertain work — but the only path to a sizeable single move.

• C.1 — R1 derived/intermediate symbol (24/44, the biggest single class). Needs quantity reuse (base + multiplier×base, the value used twice) — a DAG, which the linear chain cannot express (flagged GB-5). This is the genuine research. Cracking it unblocks the largest class at once. Build the DAG composer + its promotion frame (via A), prove on the near-pure exemplars (0027/0008/0029/0038), measure on sealed.
• C.2 — R5 multi-step rate/duration/scalar (27/44, biggest overall). Multi-step chains with a scalar-of-a-prior-stage and referent binding (0030/0015). Medium-hard; large frequency.
• C.3 — Sequence by frequency × tractability: the Stage-C investigation (docs/handoff/stage-c-composition-investigation-2026-06-03.md) is the input that ranks the cheapest real entry into C; run it first to pick C.1-vs-C.2 ordering on evidence, not guess.
• Risk discipline: each C build is a hypothesis tested against the sealed set, not the corpus. A corpus flip that does not move sealed is overfitting (ADR-0207 §6) and is reverted.

Stream D — Measurement & wrong=0 discipline (the spine)

• Every increment: train_sample (fast proxy) and the Stream-0 sealed gate (the real bar).
• The serving metric "moves only via ratified PRs" (CLAUDE.md) — each lift PR carries its sealed delta or an explicit sealed-pending obligation note (as R4 did).
• Track the sealed correct count as the program metric. train_sample is a smoke proxy.

---

Sequencing & priorities

NOW    Stream 0  (sealed baseline + per-increment gate)         ── blocks all claims
THEN   Stream A  (general promotion consumer)  ║ Stream B.1-2  (full-train harvest lane)
        — A is the force multiplier; B feeds it cheap wins
NEXT   Stream C.3 → C.1/C.2  (the big-frequency research, ranked by the Stage-C investigation)
ALWAYS Stream D  (sealed-gated, wrong=0, ratified-PR discipline)

Priority order if forced to pick one: Stream 0, then Stream A. Reason: without 0 we are flying blind on the real number; without A every win costs a hand-built bridge and the flywheel never compounds. C is where the big chunk lives, but it is wasted effort until 0 can measure it and A can land it cheaply.

What "meaningful lift in sizeable numbers" honestly requires

1. The sealed gate exists and the R4 win is confirmed real (Stream 0).
2. The landing tax is gone (Stream A) — so the flywheel and each new shape are cheap.
3. The expensive research lands on the high-frequency shapes R1/R5 (Stream C), each sealed-verified.

That is the honest path to a curve that bends up. It is a program, not a patch — and the first real milestone is Stream 0, because we do not yet know whether today's 7/43/0 moved the number that actually counts.

Cross-references

• Substrate: ADR-0207 (ratify · freeze · execute), §5 lever order, §6 gates.
• Inputs: composition-capability-scope.md (§8 class frequencies), composition-wall-execution-plan-2026-06-03.md (stage taxonomy), stage-c-composition-investigation-2026-06-03.md (Stream C ranking), sealed-measurement-obligation-2026-06-04.md (Stream 0.1).
• Landed exemplar of the build-then-gate pattern Stream A generalizes: generate/derivation/goal_residual.py + resolve_promotable_goal_residual.