The measurement foundation for docs/plans/grammar-unification-2026-07-26.md.
WHY: evals/deterministic_fluency reports 1.00 on all six predicates and
still passes "banana does the.", "wet ground rains the is." and
"is is is is." — it checks terminal punctuation, presence of a verb-shaped
token, and two anti-shape regexes. Heuristic predicates will always have
that failure mode, because grammaticality cannot be measured without a
grammar. So this lane measures agreement between the two halves of CORE
that already encode grammar, and requires the measurement to FAIL on salad.
Two directions, reported separately because they fail for different
reasons and have different remedies:
G-round-trip graph -> realize_target -> surface -> comprehend -> graph
S-round-trip surface -> comprehend -> graph -> categorical renderer -> surface
v1 baseline on main @ 9696443a:
graph_cases 280 surface_cases 8
g_write_rate 1.000 s_read_rate 1.000
g_read_rate 0.000 s_renderable_rate 0.625
g_exact_rate 0.000 s_surface_match_rate 0.000
negative_cases 16
reject_rate 1.000
g_read_rate and s_surface_match_rate are pins on measured DEFECTS, not
goals; they may be revised upward only. s_surface_match_rate = 0 is the
§1.7 categorical render defect caught by construction — the lane found it
without being told to look.
g_args_rate and g_predicates_rate are deliberately separate: high argument
agreement with low predicate agreement would mean the grammars align and
only the vocabulary is split, a materially different remedy from both
being low. That distinction decides the arc's direction (plan §6).
Design notes:
- The committed cases.jsonl is the SINGLE source for authored surfaces —
no in-module duplicate, since a second copy of a corpus is the defect
this arc exists to remove. Negative shuffles are DERIVED at run time so
they cannot drift from the positives.
- The shuffles are lexically identical to positives (same vocabulary, same
length, order destroyed) so the lane cannot pass by vocabulary-checking.
- Fixed rotation, not a PRNG, so reject_rate is byte-reproducible.
- The lane keeps a local copy of the reader's quantifier map ON PURPOSE so
it never becomes a consumer of what it measures;
test_quantifier_map_matches_reader fails loudly if the reader changes.
- _render_categorical deliberately reaches a private serving function: a
lane measuring a private copy would measure what users never see.
Every guarantee is paired with a mutation test. The load-bearing one is
test_reject_rate_goes_red_when_the_reader_accepts_everything: an
accept-everything reader must drive reject_rate to 0.0. Without it,
reject_rate == 1.0 would be unfalsifiable — precisely the defect that
makes the existing fluency lane decoration.
Also documents plainly what round-trip does NOT prove: it measures mutual
intelligibility, not English quality. english_fluency_ood accepts "river
flows valley" and round-trip would be happy with it. No metric here may be
cited as evidence of prose quality.
scripts/measure_grammar_seam.py reproduces every number in the plan's §1
so a reader can check them instead of trusting them.
[Verification]: in-worktree on CPython 3.12.13, uv sync --locked —
smoke 621 (unchanged), deductive 364 (349 + 15 new). Lane SHA pins
verified separately. No serving code touched; new files plus one suite
registration line.
cfda71dc1c