Commit graph

2 commits

Author SHA1 Message Date
Shay
2342564883
feat(ADR-0136.S.4): novel-initial-form parser extension + rescan v4 (#210)
S.4 extends initial-state parsing with two closed subject-slot widenings:
- Indefinite-article: `A <noun> has N <unit>` (gsm8k-0046 sentence 1)
- Prepositional-prefix existential: `In a <place>, there are N <unit>...`
  (gsm8k-0038 sentence 1)

Design choice: sibling regexes (_INITIAL_HAS_INDEF_RE,
_INITIAL_THERE_ARE_PREFIX_RE) rather than widening the global _ENTITY
pattern — preserves existing behavior across all other initial-state
extractors (cascade-safety).

Per the S.x corridor discipline: no new short-circuit; new candidates
flow through extract_initial_candidates and the existing graph machinery.
No solver/graph/verifier changes.

Honest delta:
- Direct admissions: 0 (admission set unchanged at {0014, 0018, 0042})
- Barrier shifts: +2 (gsm8k-0038: novel_initial_form → compound_comparative;
  gsm8k-0046: novel_initial_form → fraction_operand)
- wrong == 0 on every lane

Bundled with this PR for ledger currency:

1. tests/test_rescan_v3_invariants.py refactored to read frozen on-disk
   v3 artifacts only (no more re-running build_rescan against live
   parser). The previous design tied a historical snapshot to live code
   and broke the moment any new phase landed.

2. rescan_v4.py + refusal_rescan_v4.json + refusal_taxonomy_v4.json +
   tests/test_rescan_v4_invariants.py — the current live snapshot.
   Shifts: exactly 2 (0038, 0046). Same pattern as v3.

Sonnet wrote: S.4 parser/axis-lane/tests/ADR.
Opus wrote: rescan_v4.py + v3 test refactor + bundling.

Files:
- generate/math_candidate_parser.py (+142 lines)
- evals/math_capability_axes/S4_novel_initial_form/v1/ (20-case lane)
- tests/test_adr_0136_S4_novel_initial_form.py (40 tests)
- docs/decisions/ADR-0136.S.4-novel-initial-form.md
- evals/gsm8k_math/train_sample/v1/{rescan_v4.py, *_v4.json}
- tests/test_rescan_v4_invariants.py (8 tests)
- tests/test_rescan_v3_invariants.py (refactored to artifact-only)
2026-05-23 22:34:51 -07:00
Shay
a7feda3c19
audit(ADR-0136.S.3): refusal rescan v3 — exactly 1 barrier shift (gsm8k-0010) (#208)
Re-runs parse_and_solve on the 50-case GSM8K train sample on current
main (post-S.3) and compares to v2. Result: admitted=3/50 (unchanged),
wrong=0, exactly 1 barrier shifted v2→v3.

Shift: gsm8k-0010 (compound_statement → fraction_operand). S.3's
_INIT_MUTATION_RE resolves "Yun had 20 paperclips initially, but then
lost 12" to InitialPossession(Yun, 8, paperclips). First refusal moved
to sentence 2: "Marion has 1/4 more than what Yun currently has, plus
7" — needs fraction-operand + coreference-quantity + comparative-additive
arithmetic.

Top blockers (v3):
  compound_statement   5  (was 6)
  novel_initial_form   5  (unchanged)
  fraction_operand     4  (was 3 — gsm8k-0010 moved here)
  novel_initial_verb   4  (unchanged)

Artifacts:
- evals/gsm8k_math/train_sample/v1/rescan_v3.py
- evals/gsm8k_math/train_sample/v1/refusal_rescan_v3.json
- evals/gsm8k_math/train_sample/v1/refusal_taxonomy_v3.json
- docs/decisions/ADR-0136.S3-post-rescan.md
- tests/test_rescan_v3_invariants.py (7 tests; determinism + admission
  set unchanged + exactly-one-shift + 0010-specific shift assertions)
2026-05-23 22:05:16 -07:00