# GSM8K Train Proxy Rebaseline — 2026-06-20

**PR:** `chore/evals-ratify-gsm8k-train-proxy-rebaseline`
**Title:** `chore(evals): ratify GSM8K train proxy rebaseline`
**Date:** 2026-06-20
**Author:** Sonnet 4.6 (governance/rebaseline role)

---

## 1. Background: What Happened in #846 and #847

### PR #846 — Unauthorized Rebaseline

PR #846 (`feat(kernel): implement ProblemFrame proportional-change closure`) merged at
`a145b7c3d62ab97726f87c4a711b4b4dafe9f2b6` on 2026-06-21T02:19:07Z.

The PR was a legitimate capability PR delivering proportional-change closure work in
`generate/problem_frame.py`, `generate/problem_frame_builder.py`, and
`generate/problem_frame_contracts.py`. Its kernel and test changes were valid and
authorized.

However, the merge commit also included **two unauthorized artifact-class changes**:

1. **`evals/gsm8k_math/train_sample/v1/report.json`** — rebaselined from `6/44/0` to
   `30/20/0`. The live runner output was correct, but committing an updated `report.json`
   inside a capability PR violates governance doctrine: the committed proxy baseline is a
   sealed artifact that requires a dedicated, explicitly ratified rebaseline PR. No ADR,
   session note, or explicit PR section in #846 authorized committing the new counts.

2. **`teaching/proposals/comprehension_failures/34ce9254…json`** — a new teaching proposal
   file added without authorization. Committed via a `chore(kernel)` message; no review-gate
   passage; contradicted the HANDOFF claim that "teaching/review/proposal paths [were]
   untouched".

### PR #847 — Reconciliation

PR #847 (`reconcile: restore eval proxy and remove unauthorized teaching proposal from #846`)
merged at `b6d422834b243073fc748754ad78b1a956774065`.

#847 performed the following narrow reversions:
- Restored `evals/gsm8k_math/train_sample/v1/report.json` to the governed 6/44/0 pin.
- Deleted the unauthorized teaching proposal artifact.
- **Preserved** the valid kernel and test changes from #846.

The rationale: the live runner output (30/20/0) was already correct; only the committed
artifact was wrong. Reverting the artifact does not revert the underlying capability.

---

## 2. Why This PR Is the Explicit Authorization Boundary

The governance doctrine for `report.json` has been enforced consistently across prior
sessions:

- `gsm8k-workstream-a-gate-a1-comparative-multiplicative-lookback-2026-06-17.md`:
  "No report.json rebaseline" / "report.json rebaseline only via separate ratified PR"
- `gsm8k-workstream-a-increment-3-rate-followup-ratification-2026-06-17.md`:
  "No write of updated report.json in this increment"
- `gsm8k-capability-paradigm-sprint11-lookback-2026-06-17.md`: "report.json untouched"
- `gsm8k-capability-paradigm-sprint12-lookback-2026-06-17.md`: "report.json untouched"

This PR is the **explicit authorization** for the single `report.json` update. It:

- Is opened **after** #847 is merged to main (required base integrity).
- Is based on a **fresh worktree from `origin/main`** at the #847 merge commit
  (`b6d422834b243073fc748754ad78b1a956774065`).
- Changes **only** `evals/gsm8k_math/train_sample/v1/report.json` and this session note.
- Does **not** touch any kernel code, tests, packs, policy, identity, recall, vault, field,
  algebra, teaching proposals, or serving runtime paths.

---

## 3. Before/After Counts

### Before (governed 6/44/0 pin, restored by #847)

```json
"counts": {
  "correct": 6,
  "refused": 44,
  "wrong": 0
},
"exit_criterion": {
  "correct_min": 10,
  "passed": false,
  "wrong_max": 0
}
```

### After (ratified 30/20/0 rebaseline, this PR)

```json
"counts": {
  "correct": 30,
  "refused": 20,
  "wrong": 0
},
"exit_criterion": {
  "correct_min": 10,
  "passed": true,
  "wrong_max": 0
}
```

**`wrong_ids == []` (sacred invariant: no wrong answers committed)**

---

## 4. Live Runner Verification

The report.json was generated by running the canonical train-sample runner from the fresh
worktree at HEAD (`b6d42283`) immediately before this PR:

```bash
# Verification command (run in core-gsm8k-train-rebaseline worktree):
uv run python - <<'PY'
from evals.gsm8k_math.train_sample.v1.runner import _CASES_PATH, _load_cases, build_report
r = build_report(_load_cases(_CASES_PATH))
print("counts:", r["counts"])
print("wrong_ids:", [x["case_id"] for x in r["per_case"] if x["verdict"] == "wrong"])
print("exit_criterion:", r["exit_criterion"])
PY

# Output:
# counts: {'correct': 30, 'wrong': 0, 'refused': 20}
# wrong_ids: []
# exit_criterion: {'correct_min': 10, 'wrong_max': 0, 'passed': True}
```

The report was then written deterministically using the runner's own `write_report()`:

```bash
uv run python - <<'PY'
from evals.gsm8k_math.train_sample.v1.runner import _CASES_PATH, _load_cases, build_report, write_report, _REPORT_PATH
r = build_report(_load_cases(_CASES_PATH))
write_report(r)
print("Written to:", _REPORT_PATH)
PY
```

The runner uses `json.dumps(report, indent=2, sort_keys=True)`, so the output is
deterministic and key-order is alphabetical throughout.

---

## 5. Scope

**Changed files in this PR:**

| File | Change |
|---|---|
| `evals/gsm8k_math/train_sample/v1/report.json` | Rebaselined from 6/44/0 to 30/20/0 |
| `docs/sessions/gsm8k-train-proxy-rebaseline-2026-06-20.md` | This audit note (new) |

**No other files changed.**

---

## 6. Non-Goals

This PR explicitly does **not**:

- Change any runtime code, serving path, or derivation organ.
- Add or modify any test.
- Touch packs, policy, identity, recall, vault, field, or algebra.
- Create any teaching proposal or teaching artifact.
- Start or advance quantity-entity implementation work.
- Rebaseline any other eval artifact (holdout, composition validation, etc.).
- Add any new eval artifact.

The quantity-entity foundational slice implementation is authorized separately by
`docs/sessions/quantity-entity-foundational-slice-authorization-2026-06-20.md` and must
not begin until that document's own gating conditions are met.

---

## 7. Remaining Risk

The primary systemic risk exposed by this sequence is:

> **Report pins must not be casually updated inside implementation or capability PRs.**

The `report.json` proxy baseline is a sealed artifact. Even when the live runner output
is correct and the capability work is authorized, committing the updated report inside
a capability PR:
- Conflates two authorization domains (capability work ≠ rebaseline authorization).
- Makes the committed artifact hard to audit (which PR actually authorized it?).
- Undermines the governed rebaseline PR discipline that has protected wrong-zero invariance
  across all prior sessions.

**All future PRs must leave `evals/gsm8k_math/train_sample/v1/report.json` (and all other
`report.json` proxy baselines) untouched unless the PR is explicitly and solely dedicated
to a governed rebaseline.**

---

## 8. Authorization Chain

This rebaseline is authorized by the following chain:

1. **ADR-0126** — establishes the train-sample exit criterion (`correct >= 10`, `wrong == 0`).
2. **#846 kernel changes** — the capability work that produced the 30/20/0 live runner
   output (proportional-change closure in ProblemFrame). These changes are valid and merged.
3. **#847 reconciliation** — restored the committed artifact to 6/44/0, explicitly delegating
   authorization to a dedicated rebaseline PR (this PR).
4. **This PR** — the explicit, governed authorization to commit the 30/20/0 counts.

The `wrong_ids == []` invariant is preserved: no wrong answer is committed, and the
`wrong_max: 0` gate remains active.

---

## 9. Fresh-Base Verification (Pre-Edit)

```bash
# In /Users/kaizenpro/Projects/core:
git fetch origin --prune
# -> b6d42283 reconcile: restore eval proxy and remove unauthorized teaching proposal from #846 (#847)

git worktree add ../core-gsm8k-train-rebaseline origin/main \
  -b chore/evals-ratify-gsm8k-train-proxy-rebaseline

# In new worktree:
echo "HEAD=$(git rev-parse HEAD)"
# HEAD=b6d422834b243073fc748754ad78b1a956774065

echo "ORIGIN_MAIN=$(git rev-parse origin/main)"
# ORIGIN_MAIN=b6d422834b243073fc748754ad78b1a956774065

git status --porcelain
# (empty — clean)

git log --oneline -n 3
# b6d42283 reconcile: restore eval proxy and remove unauthorized teaching proposal from #846 (#847)
# a145b7c3 feat(kernel): implement ProblemFrame proportional-change closure (#846)
# ad0bae29 docs(kernel): authorize quantity-entity foundational implementation slice (#845)
```

HEAD == ORIGIN_MAIN == the #847 merge commit. Pre-edit state verified clean.