core/docs/decisions/ADR-0136.S.4-novel-initial-form.md
Shay 2342564883
feat(ADR-0136.S.4): novel-initial-form parser extension + rescan v4 (#210)
S.4 extends initial-state parsing with two closed subject-slot widenings:
- Indefinite-article: `A <noun> has N <unit>` (gsm8k-0046 sentence 1)
- Prepositional-prefix existential: `In a <place>, there are N <unit>...`
  (gsm8k-0038 sentence 1)

Design choice: sibling regexes (_INITIAL_HAS_INDEF_RE,
_INITIAL_THERE_ARE_PREFIX_RE) rather than widening the global _ENTITY
pattern — preserves existing behavior across all other initial-state
extractors (cascade-safety).

Per the S.x corridor discipline: no new short-circuit; new candidates
flow through extract_initial_candidates and the existing graph machinery.
No solver/graph/verifier changes.

Honest delta:
- Direct admissions: 0 (admission set unchanged at {0014, 0018, 0042})
- Barrier shifts: +2 (gsm8k-0038: novel_initial_form → compound_comparative;
  gsm8k-0046: novel_initial_form → fraction_operand)
- wrong == 0 on every lane

Bundled with this PR for ledger currency:

1. tests/test_rescan_v3_invariants.py refactored to read frozen on-disk
   v3 artifacts only (no more re-running build_rescan against live
   parser). The previous design tied a historical snapshot to live code
   and broke the moment any new phase landed.

2. rescan_v4.py + refusal_rescan_v4.json + refusal_taxonomy_v4.json +
   tests/test_rescan_v4_invariants.py — the current live snapshot.
   Shifts: exactly 2 (0038, 0046). Same pattern as v3.

Sonnet wrote: S.4 parser/axis-lane/tests/ADR.
Opus wrote: rescan_v4.py + v3 test refactor + bundling.

Files:
- generate/math_candidate_parser.py (+142 lines)
- evals/math_capability_axes/S4_novel_initial_form/v1/ (20-case lane)
- tests/test_adr_0136_S4_novel_initial_form.py (40 tests)
- docs/decisions/ADR-0136.S.4-novel-initial-form.md
- evals/gsm8k_math/train_sample/v1/{rescan_v4.py, *_v4.json}
- tests/test_rescan_v4_invariants.py (8 tests)
- tests/test_rescan_v3_invariants.py (refactored to artifact-only)
2026-05-23 22:34:51 -07:00

7.7 KiB

ADR-0136.S.4 — Novel Initial-Form Subject-Slot Widenings

Status: Accepted
Parent: ADR-0136 (Statement Layer Corridor)
Date: 2026-05-23

Context

The v3 refusal taxonomy (post-S.3 rescan) identified 5 cases with novel_initial_form as their primary barrier. These cases have initial-state sentences that the existing _INITIAL_HAS_RE and _INITIAL_THERE_ARE_RE cannot parse because the subject slot uses a form neither pattern admits.

The 5 cases and their S.4 disposition:

Case Primary Barrier S.4 Target Shape Secondary Barriers S.4 Result
0028 novel_initial_form rate_price + temporal_frequency No change (multi-clause)
0030 novel_initial_form compound_comparative No change (impersonal "It" subject)
0038 novel_initial_form Shape B (prep-prefix existential) compound_comparative on sentence 2 Barrier shift
0045 novel_initial_form rate_price No change (rate-price shape)
0046 novel_initial_form Shape A (indefinite-article subject) fraction_operand on sentence 2 Barrier shift

Only 2 of the 5 cases are single-shape targets addressable by S.4. The remaining 3 have load-bearing secondary barriers that S.4 cannot resolve:

  • 0028: "It cost $100,000 to open a bakery" — impersonal "It" subject + rate_price + temporal_frequency. All three barriers must fall together.
  • 0030: "It takes 2 times as many" — impersonal "It" + compound_comparative.
  • 0045: Sentence 1 is actually a rate-price shape ("The price of 1 orange is $0.50"), misclassified as novel_initial_form in v2; S.4 does not target it.

Decision

Two sibling regexes, not a widened _ENTITY

_ENTITY = r"(?:[A-Z]\w+|[Tt]he\s+\w+)" is used by many patterns across the file. Widening it to include indefinite-article forms would cascade through all regex sites, breaking the closed-grammar discipline of every shape that depends on it.

Instead S.4 adds two sibling patterns (_INITIAL_HAS_INDEF_RE and _INITIAL_THERE_ARE_PREFIX_RE) with their own entity slots, applied in dedicated extractor functions that are wired at the end of extract_initial_candidates. All other regex sites are unchanged.

Shape A — indefinite-article subject

^[Aa]\s+<noun>\s+has\s+<N>(?:\s+<adjective>)?(?:\s+<unit>)?(?:\s+of\s+<substance>)?\.$

Restrictions:

  • [Aa]\s+ — capital or lowercase "A" followed by whitespace only. "An" is deliberately excluded: [Aa] does not match the "n" that follows in "An", so the \s+ fails. This avoids collision with money-amount or other shapes that may start with "an".
  • Anchor: has only (singular third-person). "A school have" is ungrammatical; widening to "have/had" is deferred until a real GSM8K case requires it.
  • Entity stored as the bare noun, lowercased (e.g., "school", "factory").
  • All existing value-slot widenings (_VALUE) and substance qualifiers apply.

gsm8k-0046 sentence 1: "A school has 100 students." → InitialPossession(entity="school", quantity=Quantity(100, "students"))

Shape B — prepositional-prefix existential

^In\s+[Aa]\s+<place>,?\s+there\s+(?:are|were|is|was)\s+(?:a\s+)?<N>\s+<unit>
(?:\s+on\s+the\s+\w+(?:-floor)?)?(?:\s+\w+ing)?\.$

Restrictions:

  • Prefix: In\s+[Aa]\s+ — "In a" or "In A" only. Other prepositions ("At a", "On a") are deferred.
  • (?:a\s+)? before the value slot: handles the "a hundred" construction where "a" is an article for the numeral phrase. The article is consumed without being captured; the value group sees "hundred" which resolves via WORD_NUMBERS.
  • Ordinal-floor qualifier (on the first-floor) and participial phrase (studying, shelving) are optional and discarded.
  • Entity stored as the bare place noun, lowercased (e.g., "building").
  • Participial pattern \w+ing matches present participles only; past participles ("shelved") are not matched — consistent with the only GSM8K target sentence (gsm8k-0038 uses "studying").

gsm8k-0038 sentence 1: "In a building, there are a hundred ladies on the first-floor studying." → InitialPossession(entity="building", quantity=Quantity(100, "ladies"))

No short-circuit path

Both extractors produce CandidateInitial objects that flow through the existing graph machinery. No bypass route is added in parse_and_solve.

Per-Case Impact

Case Pre-S.4 Barrier S.4 Shape S.4 Effect Post-S.4 Barrier
0028 novel_initial_form No change novel_initial_form + rate_price + temporal_frequency
0030 novel_initial_form No change novel_initial_form (impersonal "It")
0038 novel_initial_form Shape B Shifted compound_comparative (sentence 2: "three times that many")
0045 novel_initial_form No change rate_price (misclassification; sentence 1 is a rate-price shape)
0046 novel_initial_form Shape A Shifted fraction_operand (sentence 2: "Half of the students are girls")

Barrier shifts: +2 (0038 → compound_comparative; 0046 → fraction_operand).

GSM8K Admission Count

Metric Pre-S.4 Post-S.4
Admitted (correct) 3 3
Wrong 0 0
Refused 47 47
Barrier shifted 2 (0038, 0046)

Direct admission delta: 0. Both 0038 and 0046 shift their barrier from sentence 1 to sentence 2, which introduces a barrier that S.4 does not address.

Axis Lane

20 curated cases at evals/math_capability_axes/S4_novel_initial_form/v1/:

Category Count Gate
indefinite_article_canonical 4 answer == expected
indefinite_article_substance 3 answer == expected
prep_prefix_canonical 4 answer == expected
prep_prefix_ordinal_participial 3 answer == expected
refusal_missing_unit 2 answer is None
refusal_indefinite_quantity 2 answer is None
refusal_definite_article_regression 2 answer == expected (regression gate)

wrong == 0 across all 20 cases.

The refusal_definite_article_regression category verifies that definite-article subjects ("The school has 100 students.") still resolve through the existing _INITIAL_HAS_RE path and are not blocked by the new sibling regex.

Deferred

  • Impersonal "It" subjects — "It cost $100,000 to open." The subject slot "It" is not a named entity; resolving it requires coreference or a special implicit-entity rule. Cases 0028 and 0030 have this form.
  • "An" article prefix — "An apple has 10 seeds." Excluded from [Aa]\s+ to avoid collision with money-amount or other shapes; no GSM8K target requires it.
  • Global _ENTITY widening — cascade-unsafe; every shape depending on _ENTITY would need independent testing.
  • Multi-clause cases — 0028 has rate_price + temporal_frequency as co-barriers; addressing them requires S.1-level rate-parsing and multi-clause composition, both out of S.4 scope.
  • Other preposition prefixes — "At a school, there are..." / "On a floor, there are..." — no GSM8K targets require them in the current 50-case sample.

PR Checklist Answers

  • Capability added: Two closed subject-slot widenings for initial-state sentences; 20-case axis lane with wrong == 0.
  • Invariant: versor_condition(F) < 1e-6 is field-level; this change is pure parser — no algebra path touched.
  • CLI lane: S.4 axis runner + S.3 + S.1 + rescan-v3 gates all pass.
  • No hidden normalization: No drift repair, no approximate recall, no stochastic fallback introduced.
  • Trust boundary: User-supplied text enters via parse_and_solve (existing boundary). Both new patterns are closed-grammar (no wildcards); unmatched sentences refuse, not wrong.