# ADR-0119.8 — gsm8k_math Overall Lane Gate (`gsm8k_capability_shape`)

**Status:** Accepted
**Date:** 2026-05-23
**Author:** CORE agents + reviewers
**Depends on:** ADR-0109 (lane-shape registry), ADR-0114a, ADR-0119, ADR-0119.1, ADR-0119.2, ADR-0119.3, ADR-0119.4, ADR-0119.5, ADR-0119.6

---

## Context

Phase 5.8 of [ADR-0119](ADR-0119-gsm8k-eval-lane-roadmap.md). Composes
the per-sub-phase work (5.1..5.6) into a single per-split lane verdict
the eventual ADR-0120 `expert` promotion contract can consume.

Per the ADR-0119 §5.8 brief:

> "A new lane shape `gsm8k_capability_shape` is registered in
> `LANE_SHAPE_REGISTRY` with the above thresholds. ADR-0119.8 ships
> the shape; ADR-0120 invokes it."

The shape's load-bearing constraints (per ADR-0114a Obligation #4):

- **`wrong == 0`** — the misparse-discipline gate
- **`correct + refused == cases_total`** — accounting completeness
- **`cases_total > 0`** — non-empty input
- **`overall_pass == True`** (when present) — runner self-consistency

Substrate prerequisites (5.1..5.6 artifacts) are checked by the
companion test module, not by the shape checker itself — that keeps
shape checkers narrowly metrics-focused (consistent with ADR-0109's
existing checker pattern).

---

## Decision

### `LANE_SHAPE_REGISTRY` mapping

```python
LANE_SHAPE_REGISTRY["gsm8k_math"] = "gsm8k_capability_shape"
```

Distinct from the existing four shapes because the metrics keys
differ (`cases_total` / `correct` / `wrong` / `refused` /
`overall_pass`) and the composition rule
(`correct + refused == total AND wrong == 0`) is unique to this
lane's runner contract.

### `_check_gsm8k_capability_shape` checker

Refuses on any of:

| Condition | Reason |
|---|---|
| Any of `cases_total` / `correct` / `wrong` / `refused` missing | "missing required metric X" |
| `cases_total <= 0` | "cases_total=N (must be > 0)" |
| `wrong != 0` | "wrong=N (must be 0 — ADR-0114a Obligation #4)" |
| `correct + refused != cases_total` | "outcome accounting incomplete" |
| `overall_pass` present and false | "overall_pass is False despite wrong=0 and accounting balanced" |

Accepts on everything else. Edge case: **all-refused (0 correct,
0 wrong, N refused) PASSES the shape gate.** Whether that's acceptable
for an `expert` promotion is ADR-0120's job (it sets the
correct-rate-minimum); this layer just verifies the runner's
contract is internally consistent.

### Current measurement on main

Running the gate against the on-main `evals/gsm8k_math/` corpus:

| Split | cases_total | correct | wrong | refused | gate |
|---|---|---|---|---|---|
| dev | 50 | 50 | 0 | 0 | ✓ |
| public | 150 | 150 | 0 | 0 | ✓ |

---

## ADR-0114a obligation roll-up after Phase 5.8

All 10 obligations are now mechanically gated on `main`:

| # | Obligation | Discharged by |
|---|---|---|
| 1 | Sealed-holdout discipline | ADR-0119.1 (fab_control); ADR-0119.7 for the GSM8K test set |
| 2 | OOD surface variation | ADR-0118a |
| 3 | Replay-equal trace | ADR-0117 |
| 4 | Typed refusal + wrong==0 | ADR-0116 + ADR-0119.3 + **gate enforces here** |
| 5 | Reasoning-isolation perturbation suite | ADR-0125 |
| 6 | Compositional-depth curve | ADR-0119.6 (ε threshold to ADR-0120) |
| 7 | Frontier-baseline comparison | ADR-0119.4 |
| 8 | Adversarial generation; wrong==0 | ADR-0119.5 |
| 9 | Determinism | solver + verifier + realizer + runner + shape checker |
| 10 | Operation provenance via pack | ADR-0116 |

The only piece left before ADR-0120 (first `expert` promotion
contract) is **Phase 5.7** — sealing the real GSM8K test set into
`evals/gsm8k_math/holdouts/v1/cases.jsonl.age`. That requires a
human to (a) acquire the GSM8K test set and (b) encrypt it against
the existing fab_control recipient key (or a fresh GSM8K-specific
recipient). All other machinery is in place.

---

## Invariants

### `adr_0119_8_registry_mapping`

`LANE_SHAPE_REGISTRY["gsm8k_math"] == "gsm8k_capability_shape"`;
`SHAPE_CHECKERS["gsm8k_capability_shape"]` is callable.

### `adr_0119_8_live_runner_passes_gate`

Running the on-main lane runner against the dev or public split
produces metrics the shape checker accepts. Today: dev 50/50,
public 150/150, both green.

### `adr_0119_8_nonzero_wrong_refuses`

A metrics block with `wrong > 0` refuses the gate with a reason
naming ADR-0114a Obligation #4.

### `adr_0119_8_incomplete_accounting_refuses`

A metrics block where `correct + refused != cases_total` refuses
with a typed "outcome accounting incomplete" reason.

### `adr_0119_8_missing_field_refuses`

Removing any of `cases_total` / `correct` / `wrong` / `refused`
from the metrics refuses with a "missing required metric" reason
naming the field.

### `adr_0119_8_substrate_artifacts_present`

Phase 5.1..5.6 artifacts exist on disk (sealed holdout for
fab_control; corpus + verify.py; runner; frontier + comparison;
adversarial generator + score; depth-curve harness). Phase 5.7
placeholder is also present (empty cases.jsonl files reserved for
the eventual real-GSM8K-test seal).

---

## Acceptance evidence

- `core/capability/expert_demo.py` registers `gsm8k_math →
  gsm8k_capability_shape`; `SHAPE_CHECKERS` exports the checker
- `tests/test_adr_0119_8_lane_gate.py` (21 cases) green; pins all
  six invariants
- Live runner on dev + public both pass the gate
- 101 tests green across Phase 5 + lane-shape-thresholds +
  expert-demo-contract + capability-reports
- Smoke suite green (67/67)
- ADR linked from `docs/decisions/README.md` index + frontier

---

## Consequences

- The gsm8k_math lane is now first-class in the lane-shape registry.
  Adding it to a domain pack's `eval_lanes` manifest and signing an
  `audit_passed_claim` would route through the new gate cleanly.
- ADR-0120 (first `expert` promotion contract) can now reference
  `gsm8k_capability_shape` as the lane shape it requires. The only
  remaining gap is the real-GSM8K-test seal (Phase 5.7) and the
  numeric ε threshold for the depth curve (ADR-0120's call).
- The "all-refused" edge case (where the lane refuses every input)
  passes the shape gate intentionally. That makes the shape narrowly
  about *runner self-consistency*, not capability. ADR-0120 sets the
  capability bar.

---

## Out of scope

- Sealing the real GSM8K test set (ADR-0119.7).
- The numeric ε threshold for the depth curve (ADR-0120).
- A minimum `correct_rate` for the lane to qualify as `expert`
  evidence — that's a capability claim, lives in ADR-0120.
- Cross-split gates (e.g. holdout must score within Δ of public).
  Those are anti-overfitting properties already handled by
  Obligation #2 (OOD) and Obligation #6 (depth curve); the shape
  checker doesn't duplicate them.
