core/evals/math_capability_axes
Shay 2342564883
feat(ADR-0136.S.4): novel-initial-form parser extension + rescan v4 (#210)
S.4 extends initial-state parsing with two closed subject-slot widenings:
- Indefinite-article: `A <noun> has N <unit>` (gsm8k-0046 sentence 1)
- Prepositional-prefix existential: `In a <place>, there are N <unit>...`
  (gsm8k-0038 sentence 1)

Design choice: sibling regexes (_INITIAL_HAS_INDEF_RE,
_INITIAL_THERE_ARE_PREFIX_RE) rather than widening the global _ENTITY
pattern — preserves existing behavior across all other initial-state
extractors (cascade-safety).

Per the S.x corridor discipline: no new short-circuit; new candidates
flow through extract_initial_candidates and the existing graph machinery.
No solver/graph/verifier changes.

Honest delta:
- Direct admissions: 0 (admission set unchanged at {0014, 0018, 0042})
- Barrier shifts: +2 (gsm8k-0038: novel_initial_form → compound_comparative;
  gsm8k-0046: novel_initial_form → fraction_operand)
- wrong == 0 on every lane

Bundled with this PR for ledger currency:

1. tests/test_rescan_v3_invariants.py refactored to read frozen on-disk
   v3 artifacts only (no more re-running build_rescan against live
   parser). The previous design tied a historical snapshot to live code
   and broke the moment any new phase landed.

2. rescan_v4.py + refusal_rescan_v4.json + refusal_taxonomy_v4.json +
   tests/test_rescan_v4_invariants.py — the current live snapshot.
   Shifts: exactly 2 (0038, 0046). Same pattern as v3.

Sonnet wrote: S.4 parser/axis-lane/tests/ADR.
Opus wrote: rescan_v4.py + v3 test refactor + bundling.

Files:
- generate/math_candidate_parser.py (+142 lines)
- evals/math_capability_axes/S4_novel_initial_form/v1/ (20-case lane)
- tests/test_adr_0136_S4_novel_initial_form.py (40 tests)
- docs/decisions/ADR-0136.S.4-novel-initial-form.md
- evals/gsm8k_math/train_sample/v1/{rescan_v4.py, *_v4.json}
- tests/test_rescan_v4_invariants.py (8 tests)
- tests/test_rescan_v3_invariants.py (refactored to artifact-only)
2026-05-23 22:34:51 -07:00
..
G1_verb_classes/v1 feat(G1): verb-classes capability axis (ADR-0131.G.1) 2026-05-23 15:39:14 -07:00
G2_comparatives fix(ADR-0131.G.2): rebase + mastery hardening — quarter/third fraction anchors, gate regex, boundary refusals (#196) 2026-05-23 19:28:09 -07:00
G3_numerics feat(ADR-0131.G.3.1): numerics extensions — fractions + multi-currency + multi-token cardinals + word-num-adjective 2026-05-23 15:16:46 -07:00
G4_multi_clause
G5_aggregate feat(ADR-0131.G.5): aggregate answer composition — combined/together cues wired, axis lane 20/20, wrong==0 (#197) 2026-05-23 19:42:55 -07:00
S1_rate_events feat(ADR-0136.S.1): rate/event statement parsing — capacity + earnings shapes, axis lane 20/20, wrong==0, gsm8k-0014 admits (#201) 2026-05-23 20:36:01 -07:00
S3_compound_initial_mutation feat(ADR-0136.S.3): compound initial-mutation extractor — one shape, gsm8k-0010 barrier shift, wrong==0 (#207) 2026-05-23 21:58:55 -07:00
S4_novel_initial_form feat(ADR-0136.S.4): novel-initial-form parser extension + rescan v4 (#210) 2026-05-23 22:34:51 -07:00
__init__.py