core/docs/decisions/ADR-0207-gsm8k-substrate-ratification.md

ADR-0207 — GSM8K Comprehension/Composition Substrate: Ratify · Freeze · Execute

Status: Accepted (ratified 2026-06-03) Date: 2026-06-03 Current execution state: R4 goal-residual production landed after ratification; serving train-sample is now 7/43/0, wrong=0. Author: Shay Anchor: thesis-decoding-not-generating Consolidates (does not replace): ADR-0164, ADR-0165, ADR-0174, ADR-0178 (Compositional Structure), ADR-0179. Supersedes: ADR-0163 §Phase B–E prescription and ADR-0136's regex sentence-template prescriptions (refusal taxonomies preserved as evidence).

---

1. This is not a new design

It is the ratification of a design that already exists and is already built, plus the closing of the fallback path that keeps re-opening it. It ends a process loop, not an architectural gap:

a structured substrate gets proposed → never ratified or finished → the old regex path stays live in parallel → someone sees overfitting on the live regex path and proposes a "new" structured substrate that already exists → repeat.

If anyone (human or agent) proposes a "semantic operation grammar compiler," a "new IR," or a "new reader" for GSM8K after this ADR: it is redundant with the five ADRs consolidated here. Redirect to this document. The next correct artifact on this surface is execution against the gates (§6), not another design.

2. Verified findings (origin/main 2cb0922)

Every claim below was reproduced read-only against the tree on 2026-06-03. This section is the ratification baseline. Later execution notes may update metrics without rewriting the baseline finding.

252 ADRs. The relevant design is on the books, not missing. Actual status lines (cited as they stand — these are not "design of record" labels until this ADR is ratified):

ADR	Title (verbatim)	Status (verbatim)
0164	Incremental Comprehension Reader (replaces regex sentence-template parsing)	Partially implemented (Phase 1 + 2 shipped; eval delta pending lexicon expansion)
0165	Regex Scope Rule: Lexemes Only, Never Grammar	Proposed
0174	Held-Hypothesis Comprehension with Lookback and In-Loop Contemplation	Proposed
0178	Compositional Structure: Comprehension-Guided Multi-Step Derivation (Gap B)	Proposed
0179	Extraction Richness	Proposed
0136	Statement Layer Corridor	Active — regex sentence-template prescription superseded by 0164

• ADR-0164 explicitly "replaces regex sentence-template parsing" and supersedes ADR-0163 §Phase B–E + ADR-0136 (+ the .S.1–.S.4 sub-family). Verified verbatim.
• ADR-0174 deprecates the per-category injector dispatch table at generate/recognizer_anchor_inject.py:233 as the runtime admission path — "the injectors become hypothesis-emitters within the held-hypothesis reader; they no longer route admission via category lookup." Verified verbatim.

The machinery is built, not theoretical. All present and importable at HEAD: generate/comprehension/{lifecycle,state,lookback,constraint_propagation,contemplate,composition_registry}.py and generate/derivation/{multistep,compose,accumulate,clauses,extract,pool}.py. Held-hypothesis primitives (lookback.reevaluate, contemplate) are invoked live inside generate/math_candidate_graph.py (ADR-0174 Phase 3 lookback block, ~line 685–743).

Topology is fragmented into three coexisting pieces:

1. The regex recognizer/injector path (generate/recognizer_match.py + generate/recognizer_anchor_inject.py) — the live serving admission path (imported by math_candidate_graph.py at lines 663, 685, 1004) and the overfitting vector.
2. The held-hypothesis primitives — bolted into the candidate-graph flow (above).
3. The derivation/ composer — built and already enriched (see §3), but disjoint from serving at the ratification baseline: chat/ imports zero derivation modules, and the serving candidate-graph path imports exactly one derivation symbol — generate.derivation.product_bridge.resolve_promotable_product (math_candidate_graph.py:530, the guarded product bridge behind the current lift). The composer proper (extract/clauses/compose/accumulate/multistep/search/verify) is not reached from serving. (cue_precision/, proof_chain/, and derivation.model type imports are inert w.r.t. serving.)

The reader-as-admission-dispatch was found inert and retired in Phase 5a (3fd3172, ancestor of the prior HEAD 8327c6b, −1,038 LOC); the primitives survived. So ADR-0174's intended topology (reader = the brain; injectors = emitters) was never realized — it fragmented.

Empirical signature of the unfinished baseline: train_sample 6/44/0 (reproduced live at ratification); sealed holdout 0/0/1319 (cited per docs/claims_ledger.md / ADR-0119.7 ciphertext — not CI-reproducible). The live regex path solves a few train surface forms and transfers to ~nothing unseen. Safe failure (refuses, never wrong), but narrow-and-non-transferring.

Current execution note (post-ratification). ADR-0207 §5 R4 execution added generate/derivation/goal_residual.py and a guarded promotion bridge for residual-to-goal readings. This moves train_sample to 7/43/0, still wrong=0. The lift is execution of this ADR, not a new architecture and not a relaxation of the §4 freeze.

Root cause of the wasted weeks is process, not architecture: five Proposed/partial ADRs never ratified or driven to completion, with the deprecated fallback left live.

3. RATIFY — the design of record

Upon ratification of this ADR, adopt the following as the single ratified GSM8K design (and move ADR-0164/0165/0174/0178/0179 out of Proposed/limbo, this ADR serving as the consolidating decision that cites them):

Doctrine (decoding-not-generating). GSM8K capability comes from a structured comprehension reader that builds typed hypotheses and a composer that assembles them into multi-step derivations. Regex recognizes lexemes only (currency, fractions, numerics) — never sentence/grammar structure (ADR-0165). The composer's incomplete productions refuse structurally, preserving wrong=0.

ADR-0163 §Phase B–E and ADR-0136's regex sentence-template prescriptions are superseded (their refusal taxonomies are kept as evidence; the regex prescription is dead — already recorded in ADR-0164's "Supersedes in part" block).

4. FREEZE — close the fallback

The regex recognizer/injector path on the serving candidate-graph (recognizer_match.py

• recognizer_anchor_inject.py) is lexeme-recognition + refusal-only.

• Permitted: refusal-only wrong=0 guards — the no-reference <N> times completeness tripwire (composition-capability-scope §9; tests/test_candidate_graph_completeness_guard.py) is the model: it removes wrong answers, it cannot overfit.
• Forbidden: any new positive capability as a bespoke recognizer/injector branch. Positive capability flows only through the comprehension/composition machinery (§5).

This is consistent with ADR-0174's own deprecation (the per-category injector dispatch is no longer the admission path) and is the line that, left open, re-creates the loop.

5. CONVERGE + EXECUTE — feed the built machinery (no new architecture)

The convergence target is ADR-0174's intent: the structured machinery is the brain; regex is demoted to lexemes + refusal. Realize it by connecting and feeding what is already built, not rewriting it.

Extraction is not the open lever (tree-verified, 2026-06-03). An earlier draft named extraction richness (ADR-0179) as primary. That is stale: EX-1/EX-2/EX-4/EX-5/EX-6

• unit-hygiene have landed in generate/derivation/extract.py; only EX-3 (multi-word units) is deliberately deferred with documented traps (TestEX3StillDeferred pins them), and ADR-0179's §Context "thin extractor" table no longer matches the code — e.g. case 0003 now solves 864 end-to-end through the derivation composer. See extraction-richness-audit-2026-06-03.

Execute these three levers — listed by role, not by sequence (lever 1 explains why the wire is the last, trivial step and composition the real first work):

1. WIRING — trivial, and already done for the safe shapes. The wire pattern (resolve(text) → CandidateGraphResult | fall-through) is proven, but it generalizes nothing by itself. The initial product_bridge promoted only via narrow target proof for 0003/0021; the post-ratification goal_residual bridge promotes only goal-language + residual-question + same-referent licensed-progress chains. Extending either bridge is not the lowest-risk first step. Verified on the tree (2026-06-03): the general composer (resolve_pooled) refuses all ten R1/R4/R5/R6 corpus positives — R1/R4 build zero candidates, R5 builds only wrong ones — so wiring it wholesale yields +0 correct and wrong-commits 0016 → 510, a live wrong=0 regression that disagreement does not catch. Therefore WIRING and COMPOSITION are the same task: the real first work is composer production for each target shape, each behind a structural promotion gate (op-class/target proof via extract_target + target_units, not disagreement alone). The wire is the last, trivial step once a shape builds-and-certifies.
2. COMPOSITION (ADR-0178) — the actual wall. Complete the derivation composer's multi-step assembly: which quantities group, via which ops, in what order (the R4/R5/R6 lever). This is the genuine research risk; extraction already feeds it.
3. LEXICON (ADR-0164) — category/lexicon expansion. So the serving reader stops emitting ReaderRefusal on the first unknown token (the "eval delta pending lexicon expansion" ADR-0164 itself flagged).

EX-3 multi-word units = narrow mop-up only, addressed minimally if at all — e.g. head-unit list inheritance for the 0024-class, sidestepping the deferred EX-3 traps (see the audit's P1). Not a lever.

Positive capability = wiring + composer enrichment + shared lexicon (composable, generalizing). Never a per-shape branch.

6. Acceptance gates (wrong=0 is the governor)

Every step must hold all of:

1. train_sample stays wrong=0 (current floor: 7/43/0; correct may only rise).
2. The no-reference <N> times hazard (composition-capability-scope §9) stays refused.
3. No partial graph admits while any source quantity is unbound (the completeness leg).
4. Progress is measured on the composition-typed validation sub-corpus (evals/gsm8k_math/composition_validation/v1/, now 22 cases) and on the sealed 1,319 — the sealed number is the real bar. A train_sample gain that does not move held-out is overfitting in a tidier wrapper and does not count.
5. Each gain lands as machinery enrichment (§5), audited to confirm it is not a new recognizer/injector branch (the §4 freeze).

Phase 5b is gated on the 22-case composition corpus AND the sealed 1,319 together.

7. What this explicitly is NOT

Not a new grammar compiler. Not a new IR. Not a new reader. Not a new ADR-numbered architecture beyond this consolidation. It is: ratify the five that exist, freeze the fallback, feed the built machinery, measure on held-out. Any proposal to build a "new" structured front-end for GSM8K is this, restated — point it here (§1).

8. Why this ends the loop

The loop had three drivers: propose (Proposed-limbo), leave the fallback live (an overfitting vector to rediscover), and re-propose (perpetual design instead of finish-to-instrument). This ADR removes all three at once — ratify (§3), freeze (§4), finish-to-instrument (§5–§6, measured on the sealed set, not on more proposals).

9. Documentation-hygiene flags (for the operator, non-blocking)

• Duplicate ADR-0178 number. Two files carry ADR-0178: ADR-0178-compositional-structure.md (the Gap-B composer this ADR consolidates) and ADR-0178-GB3b-referent-accumulation-scope.md (a scope-only sub-phase). They should be renumbered/renamed for unambiguous citation.
• ADR-0179 §Context drift. Its "thin extractor" table predates the landed EX-1/2/4/5/6 + unit-hygiene work; reconcile it to the tree (the audit in §5 is the reconciliation) when 0179 is moved out of Proposed.

10. Cross-references

• Consolidates: ADR-0164, ADR-0165, ADR-0174, ADR-0178 Compositional Structure, ADR-0179.
• Supersedes: ADR-0163 §Phase B–E prescription, ADR-0136 regex sentence-templates.
• Instruments: the 22-case composition validation corpus (evals/gsm8k_math/composition_validation/v1/) and the sealed holdout (evals/gsm8k_math/holdouts/v1/cases.jsonl.age, 0/0/1319).
• Evidence: composition-capability-scope, extraction-richness-audit-2026-06-03.
• Thesis: thesis-decoding-not-generating.

Governance Cross-Reference (ADR-0225)

This late-corpus ADR is governed by ADR-0225:

• Safety boundaries: changes must preserve ADR-0027/0028/0029 identity and safety-pack boundaries; no identity, safety, or policy mutation is implied unless explicitly reviewed.
• Versor closure: runtime field paths must preserve versor_condition(F) < 1e-6; this ADR does not authorize hidden normalization or hot-path drift repair.
• Reconstruction-over-storage: evidence must remain reconstructive and content-addressed rather than duplicating opaque state.
• Replay-equivalence: serving, teaching, promotion, or checkpoint changes require a named deterministic replay / byte-equivalence gate.
• Mutation standing: any durable corpus, pack, policy, or epistemic-status mutation remains reviewed, proposal-only until accepted, or proof-carrying as applicable.