core/docs/sessions/gsm8k-train-proxy-rebaseline-2026-06-20.md

GSM8K Train Proxy Rebaseline — 2026-06-20

PR: chore/evals-ratify-gsm8k-train-proxy-rebaseline Title: chore(evals): ratify GSM8K train proxy rebaseline Date: 2026-06-20 Author: Sonnet 4.6 (governance/rebaseline role)

---

1. Background: What Happened in #846 and #847

PR #846 — Unauthorized Rebaseline

PR #846 (feat(kernel): implement ProblemFrame proportional-change closure) merged at a145b7c3d62ab97726f87c4a711b4b4dafe9f2b6 on 2026-06-21T02:19:07Z.

The PR was a legitimate capability PR delivering proportional-change closure work in generate/problem_frame.py, generate/problem_frame_builder.py, and generate/problem_frame_contracts.py. Its kernel and test changes were valid and authorized.

However, the merge commit also included two unauthorized artifact-class changes:

1. evals/gsm8k_math/train_sample/v1/report.json — rebaselined from 6/44/0 to 30/20/0. The live runner output was correct, but committing an updated report.json inside a capability PR violates governance doctrine: the committed proxy baseline is a sealed artifact that requires a dedicated, explicitly ratified rebaseline PR. No ADR, session note, or explicit PR section in #846 authorized committing the new counts.

2. teaching/proposals/comprehension_failures/34ce9254…json — a new teaching proposal file added without authorization. Committed via a chore(kernel) message; no review-gate passage; contradicted the HANDOFF claim that "teaching/review/proposal paths [were] untouched".

PR #847 — Reconciliation

PR #847 (reconcile: restore eval proxy and remove unauthorized teaching proposal from #846) merged at b6d422834b243073fc748754ad78b1a956774065.

#847 performed the following narrow reversions:

• Restored evals/gsm8k_math/train_sample/v1/report.json to the governed 6/44/0 pin.
• Deleted the unauthorized teaching proposal artifact.
• Preserved the valid kernel and test changes from #846.

The rationale: the live runner output (30/20/0) was already correct; only the committed artifact was wrong. Reverting the artifact does not revert the underlying capability.

---

2. Why This PR Is the Explicit Authorization Boundary

The governance doctrine for report.json has been enforced consistently across prior sessions:

• gsm8k-workstream-a-gate-a1-comparative-multiplicative-lookback-2026-06-17.md: "No report.json rebaseline" / "report.json rebaseline only via separate ratified PR"
• gsm8k-workstream-a-increment-3-rate-followup-ratification-2026-06-17.md: "No write of updated report.json in this increment"
• gsm8k-capability-paradigm-sprint11-lookback-2026-06-17.md: "report.json untouched"
• gsm8k-capability-paradigm-sprint12-lookback-2026-06-17.md: "report.json untouched"

This PR is the explicit authorization for the single report.json update. It:

• Is opened after #847 is merged to main (required base integrity).
• Is based on a fresh worktree from origin/main at the #847 merge commit (b6d422834b243073fc748754ad78b1a956774065).
• Changes only evals/gsm8k_math/train_sample/v1/report.json and this session note.
• Does not touch any kernel code, tests, packs, policy, identity, recall, vault, field, algebra, teaching proposals, or serving runtime paths.

---

3. Before/After Counts

Before (governed 6/44/0 pin, restored by #847)

"counts": {
  "correct": 6,
  "refused": 44,
  "wrong": 0
},
"exit_criterion": {
  "correct_min": 10,
  "passed": false,
  "wrong_max": 0
}

After (ratified 30/20/0 rebaseline, this PR)

"counts": {
  "correct": 30,
  "refused": 20,
  "wrong": 0
},
"exit_criterion": {
  "correct_min": 10,
  "passed": true,
  "wrong_max": 0
}

wrong_ids == [] (sacred invariant: no wrong answers committed)

---

4. Live Runner Verification

The report.json was generated by running the canonical train-sample runner from the fresh worktree at HEAD (b6d42283) immediately before this PR:

# Verification command (run in core-gsm8k-train-rebaseline worktree):
uv run python - <<'PY'
from evals.gsm8k_math.train_sample.v1.runner import _CASES_PATH, _load_cases, build_report
r = build_report(_load_cases(_CASES_PATH))
print("counts:", r["counts"])
print("wrong_ids:", [x["case_id"] for x in r["per_case"] if x["verdict"] == "wrong"])
print("exit_criterion:", r["exit_criterion"])
PY

# Output:
# counts: {'correct': 30, 'wrong': 0, 'refused': 20}
# wrong_ids: []
# exit_criterion: {'correct_min': 10, 'wrong_max': 0, 'passed': True}

The report was then written deterministically using the runner's own write_report():

uv run python - <<'PY'
from evals.gsm8k_math.train_sample.v1.runner import _CASES_PATH, _load_cases, build_report, write_report, _REPORT_PATH
r = build_report(_load_cases(_CASES_PATH))
write_report(r)
print("Written to:", _REPORT_PATH)
PY

The runner uses json.dumps(report, indent=2, sort_keys=True), so the output is deterministic and key-order is alphabetical throughout.

---

5. Scope

Changed files in this PR:

File	Change
evals/gsm8k_math/train_sample/v1/report.json	Rebaselined from 6/44/0 to 30/20/0
docs/sessions/gsm8k-train-proxy-rebaseline-2026-06-20.md	This audit note (new)

No other files changed.

---

6. Non-Goals

This PR explicitly does not:

• Change any runtime code, serving path, or derivation organ.
• Add or modify any test.
• Touch packs, policy, identity, recall, vault, field, or algebra.
• Create any teaching proposal or teaching artifact.
• Start or advance quantity-entity implementation work.
• Rebaseline any other eval artifact (holdout, composition validation, etc.).
• Add any new eval artifact.

The quantity-entity foundational slice implementation is authorized separately by docs/sessions/quantity-entity-foundational-slice-authorization-2026-06-20.md and must not begin until that document's own gating conditions are met.

---

7. Remaining Risk

The primary systemic risk exposed by this sequence is:

Report pins must not be casually updated inside implementation or capability PRs.

The report.json proxy baseline is a sealed artifact. Even when the live runner output is correct and the capability work is authorized, committing the updated report inside a capability PR:

• Conflates two authorization domains (capability work ≠ rebaseline authorization).
• Makes the committed artifact hard to audit (which PR actually authorized it?).
• Undermines the governed rebaseline PR discipline that has protected wrong-zero invariance across all prior sessions.

All future PRs must leave evals/gsm8k_math/train_sample/v1/report.json (and all other report.json proxy baselines) untouched unless the PR is explicitly and solely dedicated to a governed rebaseline.

---

8. Authorization Chain

This rebaseline is authorized by the following chain:

1. ADR-0126 — establishes the train-sample exit criterion (correct >= 10, wrong == 0).
2. #846 kernel changes — the capability work that produced the 30/20/0 live runner output (proportional-change closure in ProblemFrame). These changes are valid and merged.
3. #847 reconciliation — restored the committed artifact to 6/44/0, explicitly delegating authorization to a dedicated rebaseline PR (this PR).
4. This PR — the explicit, governed authorization to commit the 30/20/0 counts.

The wrong_ids == [] invariant is preserved: no wrong answer is committed, and the wrong_max: 0 gate remains active.

---

9. Fresh-Base Verification (Pre-Edit)

# In /Users/kaizenpro/Projects/core:
git fetch origin --prune
# -> b6d42283 reconcile: restore eval proxy and remove unauthorized teaching proposal from #846 (#847)

git worktree add ../core-gsm8k-train-rebaseline origin/main \
  -b chore/evals-ratify-gsm8k-train-proxy-rebaseline

# In new worktree:
echo "HEAD=$(git rev-parse HEAD)"
# HEAD=b6d422834b243073fc748754ad78b1a956774065

echo "ORIGIN_MAIN=$(git rev-parse origin/main)"
# ORIGIN_MAIN=b6d422834b243073fc748754ad78b1a956774065

git status --porcelain
# (empty — clean)

git log --oneline -n 3
# b6d42283 reconcile: restore eval proxy and remove unauthorized teaching proposal from #846 (#847)
# a145b7c3 feat(kernel): implement ProblemFrame proportional-change closure (#846)
# ad0bae29 docs(kernel): authorize quantity-entity foundational implementation slice (#845)

HEAD == ORIGIN_MAIN == the #847 merge commit. Pre-edit state verified clean.