core/evals/gsm8k_math/train_sample
Shay c655f2fcda fix(rescan-v4): recognize new refusal-reason format + record reader advances
build_rescan's _FIRST_REFUSAL_RE only matched 'no admissible candidate for
...' and required the quote at end-of-string. PR #359 added the
'recognizer matched but produced no injection ... (category=X)' shape, so
extraction returned None and manufactured spurious shifts. Extend the regex
to both shapes and tolerate the trailing (category=...).

With extraction fixed, the live reader legitimately diverges from the v3
baseline on 10 cases (was 2 at the S.4 cut): 5 more first-refusals shifted
one sentence deeper (0019/0023/0025/0027/0047 — overrides added) and 3
refusals became correct admissions (0003/0021/0024 — the +3 that moved
serving 3->6). wrong stays 0; no admission lost. Tests retargeted to the
live counts and artifacts regenerated.

Follow-up: the v4 tests re-run the live reader, so they will drift again on
the next advance; a frozen v5 snapshot (or artifact-derived expectations)
would decouple them.
2026-06-03 01:13:10 -07:00
..
v1 fix(rescan-v4): recognize new refusal-reason format + record reader advances 2026-06-03 01:13:10 -07:00
__init__.py feat(gsm8k): ADR-0126 P6 — train-sample runner + exit-criterion gate 2026-05-23 06:33:06 -07:00