Commit graph

2 commits

Author SHA1 Message Date
Shay
997a6eb043 fix(generate): plural-subject agreement, 26/26 (Phase 3)
_inflect_predicate applied base_form — a SINGLE-VERB function — to whole
humanized predicate phrases, stripping the final word's last character
class, and its plural branch never consulted `copular`. Nine of the 26 seed
predicates came out wrong and every multi-word one did:

  is defined as         -> "is defined a"     want "are defined as"
  has the following ... -> "...step"          want "have the following steps"
  belongs to            -> "belongs to"       want "belong to"
  causes                -> "caus"             want "cause"

Root causes, both single-point as §1.4 predicted:

1. morphology.agree_plural_phrase inflects the HEAD (the finite verb is the
   first token of every humanized predicate) and carries the rest through.
   base_form stays single-verb and is PINNED as still wrong on a phrase, so
   nobody "fixes" the symptom in the wrong place.
2. The plural branch now agrees the phrase. The plural NEGATED branch had the
   same defect ("do not is defined a") and is fixed with it: a plural copula
   takes a bare "not", everything else takes do-support.

Two further defects found while writing the eval cases, both fixed here:

  base_form("causes") -> "caus". The -es sibilant rule fired on a stem
  ending in a single "s"; it must require a doubled "ss" ("passes"->"pass"),
  because a single "s" is nearly always a stem ending in "e" that took a
  plain -s ("causes"->"cause").

  pluralize("proof") -> "prooves". f/fe -> ves is NOT productive in English
  (proof->proofs, chief->chiefs, roof->roofs); it is a closed set. Now
  derived as lexicon.VES_PLURAL_SINGULARS from the ves-rows of
  IRREGULAR_SINGULARS, so the rule cannot claim a word the table does not
  know. Phase 2B had put pluralize on the SERVING path, so this one was live
  — and the surface-hashing pin from #133 confirms no served surface moved,
  which is that guard's first real use.

New construction C14 quantified_copular, 13 cases: the combination that was
broken was the one never tested. Measured before: 152 corpus cases carry a
quantifier and ZERO combine it with a copular predicate.

MY FIRST DRAFT OF THOSE CASES COULD NOT FAIL. must_contain/word_order listed
quantifier, subject and object but NOT THE VERB, so "all molecules is
defined a compound" passed and reverting the fix left 47/47 green. Rewritten
with the agreed verb in both constraints plus reject_surfaces carrying the
ACTUAL pre-fix output, computed by running the reverted code rather than
guessed. Mutation now:

  baseline                      13/13 C14
  revert phrase-head agreement   4/13
  revert -es stem rule          12/13
  revert closed ves set         11/13

The 4 survivors of the first mutation are the mass-noun controls, which is
correct — "all evidence is grounded in truth" must NOT pluralize.

Same lesson as the lane pins in #133: a pin that cannot fail guards nothing,
and the only way to know is to break the thing on purpose.

grammar_roundtrip's graph corpus is harvested from the committed case files,
so it grew 280 -> 293. Updated exactly rather than loosened to an inequality:
a corpus that grows or shrinks should require a deliberate edit.

[Verification]: in-worktree on CPython 3.12.13, uv sync --locked —
agreement 26/26 (from 17/26); english_fluency_ood 117/117 + 39/39 + dev
13/13 unchanged; grammatical_coverage v1 49/49; smoke 621 unchanged;
deductive 405; scripts/verify_lane_shas.py 11/11, no pin edited.
2026-07-27 10:11:22 -07:00
Shay
cfda71dc1c feat(evals): grammar round-trip instrument (Phase 1)
The measurement foundation for docs/plans/grammar-unification-2026-07-26.md.

WHY: evals/deterministic_fluency reports 1.00 on all six predicates and
still passes "banana does the.", "wet ground rains the is." and
"is is is is." — it checks terminal punctuation, presence of a verb-shaped
token, and two anti-shape regexes. Heuristic predicates will always have
that failure mode, because grammaticality cannot be measured without a
grammar. So this lane measures agreement between the two halves of CORE
that already encode grammar, and requires the measurement to FAIL on salad.

Two directions, reported separately because they fail for different
reasons and have different remedies:

  G-round-trip  graph -> realize_target -> surface -> comprehend -> graph
  S-round-trip  surface -> comprehend -> graph -> categorical renderer -> surface

v1 baseline on main @ 9696443a:

  graph_cases            280      surface_cases            8
  g_write_rate         1.000      s_read_rate          1.000
  g_read_rate          0.000      s_renderable_rate    0.625
  g_exact_rate         0.000      s_surface_match_rate 0.000
                                  negative_cases          16
                                  reject_rate          1.000

g_read_rate and s_surface_match_rate are pins on measured DEFECTS, not
goals; they may be revised upward only. s_surface_match_rate = 0 is the
§1.7 categorical render defect caught by construction — the lane found it
without being told to look.

g_args_rate and g_predicates_rate are deliberately separate: high argument
agreement with low predicate agreement would mean the grammars align and
only the vocabulary is split, a materially different remedy from both
being low. That distinction decides the arc's direction (plan §6).

Design notes:
- The committed cases.jsonl is the SINGLE source for authored surfaces —
  no in-module duplicate, since a second copy of a corpus is the defect
  this arc exists to remove. Negative shuffles are DERIVED at run time so
  they cannot drift from the positives.
- The shuffles are lexically identical to positives (same vocabulary, same
  length, order destroyed) so the lane cannot pass by vocabulary-checking.
- Fixed rotation, not a PRNG, so reject_rate is byte-reproducible.
- The lane keeps a local copy of the reader's quantifier map ON PURPOSE so
  it never becomes a consumer of what it measures;
  test_quantifier_map_matches_reader fails loudly if the reader changes.
- _render_categorical deliberately reaches a private serving function: a
  lane measuring a private copy would measure what users never see.

Every guarantee is paired with a mutation test. The load-bearing one is
test_reject_rate_goes_red_when_the_reader_accepts_everything: an
accept-everything reader must drive reject_rate to 0.0. Without it,
reject_rate == 1.0 would be unfalsifiable — precisely the defect that
makes the existing fluency lane decoration.

Also documents plainly what round-trip does NOT prove: it measures mutual
intelligibility, not English quality. english_fluency_ood accepts "river
flows valley" and round-trip would be happy with it. No metric here may be
cited as evidence of prose quality.

scripts/measure_grammar_seam.py reproduces every number in the plan's §1
so a reader can check them instead of trusting them.

[Verification]: in-worktree on CPython 3.12.13, uv sync --locked —
smoke 621 (unchanged), deductive 364 (349 + 15 new). Lane SHA pins
verified separately. No serving code touched; new files plus one suite
registration line.
2026-07-26 16:30:49 -07:00