core/docs/decisions/ADR-0207-gsm8k-substrate-ratification.md

# ADR-0207 — GSM8K Comprehension/Composition Substrate: Ratify · Freeze · Execute

**Status:** Accepted (ratified 2026-06-03)
**Date:** 2026-06-03
**Current execution state:** R4 goal-residual production landed after ratification; serving train-sample is now `7/43/0`, wrong=0.
**Author:** Shay
**Anchor:** [[thesis-decoding-not-generating]]
**Consolidates (does not replace):** ADR-0164, ADR-0165, ADR-0174,
ADR-0178 (Compositional Structure), ADR-0179.
**Supersedes:** ADR-0163 §Phase B–E *prescription* and ADR-0136's regex
sentence-template prescriptions (refusal taxonomies preserved as evidence).

---

## 1. This is not a new design

It is the ratification of a design that already exists and is already built, plus the
closing of the fallback path that keeps re-opening it. It ends a process loop, not an
architectural gap:

> a structured substrate gets proposed → never ratified or finished → the old regex path
> stays live in parallel → someone sees overfitting on the live regex path and proposes a
> "new" structured substrate that already exists → repeat.

**If anyone (human or agent) proposes a "semantic operation grammar compiler," a "new IR,"
or a "new reader" for GSM8K after this ADR: it is redundant with the five ADRs consolidated
here. Redirect to this document.** The next correct artifact on this surface is *execution
against the gates* (§6), not another design.

## 2. Verified findings (origin/main `2cb0922`)

Every claim below was reproduced read-only against the tree on 2026-06-03. This section is the
ratification baseline. Later execution notes may update metrics without rewriting the baseline
finding.

**252 ADRs.** The relevant design is on the books, not missing. Actual status lines (cited as
they stand — these are *not* "design of record" labels until this ADR is ratified):

| ADR | Title (verbatim) | Status (verbatim) |
|---|---|---|
| **0164** | Incremental Comprehension Reader (replaces regex sentence-template parsing) | *Partially implemented (Phase 1 + 2 shipped; eval delta pending lexicon expansion)* |
| **0165** | Regex Scope Rule: Lexemes Only, Never Grammar | *Proposed* |
| **0174** | Held-Hypothesis Comprehension with Lookback and In-Loop Contemplation | *Proposed* |
| **0178** | Compositional Structure: Comprehension-Guided Multi-Step Derivation (Gap B) | *Proposed* |
| **0179** | Extraction Richness | *Proposed* |
| 0136 | Statement Layer Corridor | *Active* — regex sentence-template prescription superseded by 0164 |

- **ADR-0164** explicitly *"replaces regex sentence-template parsing"* and supersedes
  ADR-0163 §Phase B–E + ADR-0136 (+ the .S.1–.S.4 sub-family). Verified verbatim.
- **ADR-0174** *deprecates* the per-category injector dispatch table at
  `generate/recognizer_anchor_inject.py:233` **as the runtime admission path** — *"the
  injectors become hypothesis-emitters within the held-hypothesis reader; they no longer
  route admission via category lookup."* Verified verbatim.

**The machinery is built, not theoretical.** All present and importable at HEAD:
`generate/comprehension/{lifecycle,state,lookback,constraint_propagation,contemplate,composition_registry}.py`
and `generate/derivation/{multistep,compose,accumulate,clauses,extract,pool}.py`.
Held-hypothesis primitives (`lookback.reevaluate`, `contemplate`) are invoked live inside
`generate/math_candidate_graph.py` (ADR-0174 Phase 3 lookback block, ~line 685–743).

**Topology is fragmented into three coexisting pieces:**
1. **The regex recognizer/injector path** (`generate/recognizer_match.py` +
   `generate/recognizer_anchor_inject.py`) — the **live serving admission path**
   (imported by `math_candidate_graph.py` at lines 663, 685, 1004) and the overfitting vector.
2. **The held-hypothesis primitives** — bolted into the candidate-graph flow (above).
3. **The `derivation/` composer** — built and *already enriched* (see §3), but **disjoint
   from serving** at the ratification baseline: `chat/` imports zero derivation modules, and
   the serving candidate-graph path imports **exactly one** derivation symbol —
   `generate.derivation.product_bridge.resolve_promotable_product`
   (`math_candidate_graph.py:530`, the guarded product bridge behind the current lift). The
   composer proper (`extract`/`clauses`/`compose`/`accumulate`/`multistep`/`search`/`verify`)
   is **not reached from serving**. (`cue_precision/`, `proof_chain/`, and `derivation.model`
   type imports are inert w.r.t. serving.)

The reader-*as-admission-dispatch* was found inert and **retired in Phase 5a** (`3fd3172`,
ancestor of the prior HEAD `8327c6b`, −1,038 LOC); the primitives survived. So ADR-0174's
intended topology (**reader = the brain; injectors = emitters**) **was never realized — it
fragmented.**

**Empirical signature of the unfinished baseline:** train_sample **6/44/0** (reproduced live at
ratification); sealed holdout **0/0/1319** (cited per `docs/claims_ledger.md` / ADR-0119.7
ciphertext — not CI-reproducible). The live regex path solves a few train surface forms and
transfers to ~nothing unseen. Safe failure (refuses, never wrong), but narrow-and-non-transferring.

**Current execution note (post-ratification).** ADR-0207 §5 R4 execution added
`generate/derivation/goal_residual.py` and a guarded promotion bridge for residual-to-goal
readings. This moves train_sample to **7/43/0**, still wrong=0. The lift is execution of this ADR,
not a new architecture and not a relaxation of the §4 freeze.

**Root cause of the wasted weeks is process, not architecture:** five `Proposed`/partial ADRs
never ratified or driven to completion, with the deprecated fallback left live.

## 3. RATIFY — the design of record

Upon ratification of this ADR, adopt the following as the single ratified GSM8K design (and
move ADR-0164/0165/0174/0178/0179 out of `Proposed`/limbo, this ADR serving as the
consolidating decision that cites them):

> **Doctrine (decoding-not-generating).** GSM8K capability comes from a structured
> comprehension reader that builds typed hypotheses and a composer that assembles them into
> multi-step derivations. Regex recognizes **lexemes only** (currency, fractions, numerics) —
> **never** sentence/grammar structure (ADR-0165). The composer's incomplete productions
> **refuse structurally**, preserving `wrong=0`.

ADR-0163 §Phase B–E and ADR-0136's regex sentence-template prescriptions are **superseded**
(their refusal taxonomies are kept as evidence; the regex *prescription* is dead — already
recorded in ADR-0164's "Supersedes in part" block).

## 4. FREEZE — close the fallback

The regex recognizer/injector path on the **serving candidate-graph** (`recognizer_match.py`
+ `recognizer_anchor_inject.py`) is **lexeme-recognition + refusal-only.**

- **Permitted:** refusal-only `wrong=0` guards — the no-reference `<N> times` completeness
  tripwire (composition-capability-scope §9; `tests/test_candidate_graph_completeness_guard.py`)
  is the model: it *removes* wrong answers, it cannot overfit.
- **Forbidden:** any new *positive* capability as a bespoke recognizer/injector branch.
  Positive capability flows only through the comprehension/composition machinery (§5).

This is consistent with ADR-0174's own deprecation (the per-category injector dispatch is no
longer the admission path) and is the line that, left open, re-creates the loop.

## 5. CONVERGE + EXECUTE — feed the built machinery (no new architecture)

The convergence target is ADR-0174's intent: the structured machinery is the brain; regex is
demoted to lexemes + refusal. Realize it by **connecting and feeding what is already built**,
not rewriting it.

> **Extraction is not the open lever (tree-verified, 2026-06-03).** An earlier draft named
> *extraction richness* (ADR-0179) as primary. That is **stale**: EX-1/EX-2/EX-4/EX-5/EX-6
> + unit-hygiene have **landed** in `generate/derivation/extract.py`; only EX-3 (multi-word
> units) is deliberately deferred with documented traps (`TestEX3StillDeferred` pins them),
> and ADR-0179's §Context "thin extractor" table no longer matches the code — e.g. case 0003
> now solves 864 end-to-end through the derivation composer. See
> [extraction-richness-audit-2026-06-03](../analysis/extraction-richness-audit-2026-06-03.md).

Execute these three levers — listed by role, not by sequence (lever 1 explains why the
wire is the *last*, trivial step and composition the real first work):

1. **WIRING — trivial, and already done for the safe shapes.** The wire pattern
   (`resolve(text) → CandidateGraphResult | fall-through`) is proven, but it generalizes
   nothing by itself. The initial `product_bridge` promoted only via narrow target proof for
   0003/0021; the post-ratification `goal_residual` bridge promotes only goal-language +
   residual-question + same-referent licensed-progress chains. Extending either bridge is
   **not** the lowest-risk first step. Verified on the tree (2026-06-03): the general composer
   (`resolve_pooled`) refuses **all ten** R1/R4/R5/R6 corpus positives — R1/R4 build zero
   candidates, R5 builds only wrong ones — so wiring it wholesale yields **+0 correct** and
   wrong-commits `0016 → 510`, a live `wrong=0` regression that disagreement does **not** catch.
   Therefore WIRING and COMPOSITION are the **same task**: the real first work is composer
   production for each target shape, each behind a structural promotion gate (op-class/target
   proof via `extract_target` + `target_units`, not disagreement alone). The wire is the last,
   trivial step once a shape builds-and-certifies.
2. **COMPOSITION (ADR-0178) — the actual wall.** Complete the derivation composer's
   multi-step assembly: which quantities group, via which ops, in what order (the R4/R5/R6
   lever). This is the genuine research risk; extraction already feeds it.
3. **LEXICON (ADR-0164) — category/lexicon expansion.** So the serving reader stops emitting
   `ReaderRefusal` on the first unknown token (the "eval delta pending lexicon expansion"
   ADR-0164 itself flagged).

**EX-3 multi-word units = narrow mop-up only**, addressed minimally if at all — e.g.
head-unit list inheritance for the 0024-class, sidestepping the deferred EX-3 traps
(see the audit's P1). **Not a lever.**

Positive capability = **wiring + composer enrichment + shared lexicon** (composable,
generalizing). Never a per-shape branch.

## 6. Acceptance gates (`wrong=0` is the governor)

Every step must hold all of:

1. train_sample stays **`wrong=0`** (current floor: 7/43/0; `correct` may only rise).
2. The no-reference `<N> times` hazard (composition-capability-scope §9) stays **refused**.
3. No partial graph admits while any source quantity is unbound (the completeness leg).
4. Progress is measured on the composition-typed **validation sub-corpus**
   ([`evals/gsm8k_math/composition_validation/v1/`](../../evals/gsm8k_math/composition_validation/v1/cases.jsonl),
   now **22 cases**) *and* on the **sealed 1,319** — the sealed number is the real bar. A
   train_sample gain that does not move held-out is overfitting in a tidier wrapper and does
   **not** count.
5. Each gain lands as machinery enrichment (§5), audited to confirm it is not a new
   recognizer/injector branch (the §4 freeze).

**Phase 5b is gated on the 22-case composition corpus AND the sealed 1,319 together.**

## 7. What this explicitly is NOT

Not a new grammar compiler. Not a new IR. Not a new reader. Not a new ADR-numbered
architecture beyond this consolidation. It is: ratify the five that exist, freeze the
fallback, feed the built machinery, measure on held-out. Any proposal to build a "new"
structured front-end for GSM8K is this, restated — point it here (§1).

## 8. Why this ends the loop

The loop had three drivers: **propose** (Proposed-limbo), **leave the fallback live** (an
overfitting vector to rediscover), and **re-propose** (perpetual design instead of
finish-to-instrument). This ADR removes all three at once — **ratify** (§3), **freeze** (§4),
**finish-to-instrument** (§5–§6, measured on the sealed set, not on more proposals).

## 9. Documentation-hygiene flags (for the operator, non-blocking)

- **Duplicate ADR-0178 number.** Two files carry `ADR-0178`:
  `ADR-0178-compositional-structure.md` (the Gap-B composer this ADR consolidates) and
  `ADR-0178-GB3b-referent-accumulation-scope.md` (a scope-only sub-phase). They should be
  renumbered/renamed for unambiguous citation.
- **ADR-0179 §Context drift.** Its "thin extractor" table predates the landed
  EX-1/2/4/5/6 + unit-hygiene work; reconcile it to the tree (the audit in §5 is the
  reconciliation) when 0179 is moved out of `Proposed`.

## 10. Cross-references

- **Consolidates:** [ADR-0164](./ADR-0164-incremental-comprehension-reader.md),
  [ADR-0165](./ADR-0165-regex-scope-rule.md),
  [ADR-0174](./ADR-0174-held-hypothesis-comprehension.md),
  [ADR-0178 Compositional Structure](./ADR-0178-compositional-structure.md),
  [ADR-0179](./ADR-0179-extraction-richness.md).
- **Supersedes:** ADR-0163 §Phase B–E prescription, ADR-0136 regex sentence-templates.
- **Instruments:** the 22-case composition validation corpus
  (`evals/gsm8k_math/composition_validation/v1/`) and the sealed holdout
  (`evals/gsm8k_math/holdouts/v1/cases.jsonl.age`, `0/0/1319`).
- **Evidence:** [composition-capability-scope](../analysis/composition-capability-scope.md),
  [extraction-richness-audit-2026-06-03](../analysis/extraction-richness-audit-2026-06-03.md).
- **Thesis:** [[thesis-decoding-not-generating]].