core/evals/gsm8k_math/train_sample
Shay 2342564883
feat(ADR-0136.S.4): novel-initial-form parser extension + rescan v4 (#210)
S.4 extends initial-state parsing with two closed subject-slot widenings:
- Indefinite-article: `A <noun> has N <unit>` (gsm8k-0046 sentence 1)
- Prepositional-prefix existential: `In a <place>, there are N <unit>...`
  (gsm8k-0038 sentence 1)

Design choice: sibling regexes (_INITIAL_HAS_INDEF_RE,
_INITIAL_THERE_ARE_PREFIX_RE) rather than widening the global _ENTITY
pattern — preserves existing behavior across all other initial-state
extractors (cascade-safety).

Per the S.x corridor discipline: no new short-circuit; new candidates
flow through extract_initial_candidates and the existing graph machinery.
No solver/graph/verifier changes.

Honest delta:
- Direct admissions: 0 (admission set unchanged at {0014, 0018, 0042})
- Barrier shifts: +2 (gsm8k-0038: novel_initial_form → compound_comparative;
  gsm8k-0046: novel_initial_form → fraction_operand)
- wrong == 0 on every lane

Bundled with this PR for ledger currency:

1. tests/test_rescan_v3_invariants.py refactored to read frozen on-disk
   v3 artifacts only (no more re-running build_rescan against live
   parser). The previous design tied a historical snapshot to live code
   and broke the moment any new phase landed.

2. rescan_v4.py + refusal_rescan_v4.json + refusal_taxonomy_v4.json +
   tests/test_rescan_v4_invariants.py — the current live snapshot.
   Shifts: exactly 2 (0038, 0046). Same pattern as v3.

Sonnet wrote: S.4 parser/axis-lane/tests/ADR.
Opus wrote: rescan_v4.py + v3 test refactor + bundling.

Files:
- generate/math_candidate_parser.py (+142 lines)
- evals/math_capability_axes/S4_novel_initial_form/v1/ (20-case lane)
- tests/test_adr_0136_S4_novel_initial_form.py (40 tests)
- docs/decisions/ADR-0136.S.4-novel-initial-form.md
- evals/gsm8k_math/train_sample/v1/{rescan_v4.py, *_v4.json}
- tests/test_rescan_v4_invariants.py (8 tests)
- tests/test_rescan_v3_invariants.py (refactored to artifact-only)
2026-05-23 22:34:51 -07:00
..
v1 feat(ADR-0136.S.4): novel-initial-form parser extension + rescan v4 (#210) 2026-05-23 22:34:51 -07:00
__init__.py feat(gsm8k): ADR-0126 P6 — train-sample runner + exit-criterion gate 2026-05-23 06:33:06 -07:00