# ADR-0131.5 — GSM8K Coverage Probe: Retirement After G.x Axis Completion

**Status:** Accepted  
**Date:** 2026-05-23  
**Parent:** [ADR-0131.G — GSM8K Coverage Probe](ADR-0131.G-gsm8k-coverage-probe.md)  
**Amends:** ADR-0131.G iteration discipline section

---

## Context

ADR-0131.G introduced the GSM8K coverage probe as a diff-able baseline
(`admitted_wrong == 0`, `admission_rate` tracked across G.x iterations)
and pinned capability-first iteration discipline for the G.x capability
axes.  Five axes shipped and merged to `main`:

| Axis | ADR | PR | Axis cases | Wrong |
|------|-----|----|-----------|-------|
| G.1 — verb classes for initial-state | ADR-0131.G.1 | #187 | curated lane | 0 |
| G.2 — comparative operations | ADR-0131.G.2 | #196 | 31 cases | 0 |
| G.3 — numeric literals | ADR-0131.G.3 | #183 | curated lane | 0 |
| G.3.1 — numerics extensions | ADR-0131.G.3.1 | #186 | curated lane | 0 |
| G.4 — multi-clause composition | ADR-0131.G.4 | #185 | curated lane | 0 |
| G.5 — aggregate answer composition | ADR-0131.G.5 | #197 | 20 cases | 0 |

**GSM8K admission after all five axes: 0/50 (unchanged from baseline).**

This was expected and documented in each G.x PR.  The bottleneck for all
50 sample cases is statement-layer parsing — rate verbs, compound
sentences, implicit entities, multi-step operations.  The G.x axes
targeted the *question layer* (aggregate cues, comparative structure,
numeric literals, multi-clause subjects) and the *initial-state verb
layer*.  None of the 50 GSM8K sample cases fail at those layers; they
fail earlier.

The safety rail (`admitted_wrong == 0`) never wavered.

---

## Decision

### 1. Retire the GSM8K probe from the per-iteration gate

Effective immediately, `train_sample_coverage_report.json` is a **frozen
evidence artifact**, not a live gate for future iterations.  The reasons:

- Admission is still 0/50.  A gate that never moves does not gate.
- The G.x axes demonstrated that architecture-aligned capability growth
  does not necessarily lift GSM8K admission — and that is correct
  behavior, not a failure.  The 50 sample cases are hard precisely
  because of statement-layer features (rate verbs, implicit entities)
  that are deliberately out of scope for the closed bounded-grammar
  model.
- Keeping the probe as a per-iteration required re-run creates process
  overhead whose signal-to-noise ratio is now close to zero.

### 2. Preserve the `admitted_wrong == 0` invariant permanently

The safety rail is not retired.  It becomes a **standing architectural
invariant** rather than a per-iteration gate:

> Any future change to the parser, solver, or verifier must preserve
> `admitted_wrong == 0` on the GSM8K train sample.

The existing test
`tests/test_adr_0131_G_gsm8k_coverage_probe.py` already CI-pins this;
it is not removed.  Each per-axis test file (G.2, G.5, etc.) also
asserts `admitted_wrong == 0` as a regression guard.  This coverage is
sufficient; no per-iteration re-report is needed.

### 3. Conditions for re-activating the probe as a live gate

If a future ADR targets **statement-layer** expansion (rate verbs,
multi-step implicit operations, compound sentence parsing), re-run the
probe and include the delta.  The probe infrastructure remains in-repo
for this purpose.  The re-activation criterion is:

- The new capability plausibly changes the admission count on ≥1
  of the 50 sample cases.
- The PR diff includes evidence that it does (pre/post `admission_rate`
  delta ≥ 0.02, i.e. ≥1 additional admission).

### 4. Archive note in ADR-0131.G

ADR-0131.G iteration discipline section 4 ("Reject any expansion that
only moves GSM8K admission") remains valid.  This ADR adds a symmetric
note: expansions that do not move GSM8K admission are not defects — they
are working as intended when the bottleneck is orthogonal to the
capability axis being extended.

---

## Evidence

- `evals/gsm8k_math/train_sample/v1/train_sample_coverage_report.json`
  — frozen at `admission_rate: 0.0`, `admitted_wrong: 0`, `refused: 50`
  after all G.x axes.
- `tests/test_adr_0131_G_gsm8k_coverage_probe.py` — CI-pins
  `admitted_wrong == 0` as a standing regression guard.
- Per-axis tests (G.2, G.5) each include a `test_gsm8k_*_admitted_wrong_zero`
  assertion as belt-and-suspenders.

---

## Consequences

- Future capability-axis PRs do not need to re-run or delta the GSM8K
  probe unless their scope touches statement-layer parsing.
- The probe file and runner remain in-repo as infrastructure for the
  eventual statement-layer push.
- The ADR-0131.G "capability-first, not coverage-first" discipline is
  unchanged.  GSM8K admission is still a *symptom*, never a target.
  This ADR merely removes the requirement to measure the symptom on
  every iteration when the symptom cannot move.

---

## Out of scope

- Statement-layer expansion (rate verbs, multi-step implicit operations)
  — that is a separate ADR when the time comes.
- Promotion-gate amendment — ADR-0131.4 is the gate; it does not
  reference the GSM8K probe, and this ADR does not change it.
- Removal of the probe runner or report file — both stay in-repo.