From 6a7faa3fc28dfec6da06377bd51a720efbbbce13 Mon Sep 17 00:00:00 2001 From: Shay Date: Wed, 3 Jun 2026 11:27:10 -0700 Subject: [PATCH] =?UTF-8?q?docs:=20correct=20stale=20GSM8K=20serving=20met?= =?UTF-8?q?ric=20in=20CLAUDE.md=20(3/47/0=20=E2=86=92=206/44/0)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CLAUDE.md's GSM8K substrate header asserted "serving stays `3/47/0` until ratified". That figure is two ratified metric-moves stale: the official candidate-graph serving metric (evals/gsm8k_math/train_sample/v1/report.json, scored by `_score_one_candidate_graph` in train_sample/v1/runner.py:88) moved 3/47/0 → 4/46/0 (#488, ADR-0189/0189a) → 6/44/0 (#500, ADR-0195), each PR re-baselining report.json. Verified on main 7e98ad3: committed report.json == live `build_report()` == {correct: 6, refused: 44, wrong: 0}. wrong==0 preserved. The stale header was the source of recurring 3/47/0 confusion across sessions. Historical ADRs (0164/0166/0174/0175/0176/0177/0178/0179) that say "serving stays 3/47/0" are point-in-time decision records and are intentionally left unchanged. --- CLAUDE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index 3620db88..f24bde3c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -104,7 +104,7 @@ runtime path. Vault recall is exact and deterministic. - `calibration/*` — bounded replay-based calibration. - `docs/runtime_contracts.md` — response, telemetry, memory, identity, and testing contracts. -### GSM8K math comprehension substrate (sealed; serving stays `3/47/0` until ratified) +### GSM8K math comprehension substrate (sealed; serving `6/44/0`, wrong=0 — moves only via ratified PRs) - `core/reliability_gate/` — calibrated-learning ledger + gate (ADR-0175): `ClassTally` counts, `conservative_floor` (one-sided Wilson, N_MIN=10), θ ceilings. - `generate/derivation/` — the comprehension composer: `extract.py` (lexeme quantity extraction, EX-1/4/5 + function-word unit filter), `clauses.py` (GB-1 segmentation), `compose.py` (GB-2a list-sum + GB-3a clause-scoped referent guard), `accumulate.py` (GB-3b.1 single-referent gain/loss chaining), `multistep.py`/`search.py` (bounded search), `verify.py` (the wrong=0 self-verification gate: grounding ∧ cue ∧ unit ∧ completeness ∧ uniqueness).