fix(generate): CORE writes plural nouns in categorical clauses (Phase 2B)
The ratified, flag-ON v1b categorical band served "all dog are mammal" for
every noun: the A/E/I/O templates supply all/no/some + are, which demand a
plural, and the slots were filled with singular canonical entity ids. Four
of the 47 ratified corpus cases were affected, and wrong=0 never noticed
because every verdict was correct — only the prose was wrong.
Two fixes that compose into a closed round trip:
1. generate/morphology.py becomes the single owner of the number RULES
(Phase 2A gave the TABLES one owner). Both directions live there:
pluralize + singularize, over lexicon.IRREGULAR_PLURALS /
IRREGULAR_SINGULARS, with mass-noun and compound-head handling.
templates.pluralize and reader._singularize were two independent copies
of the regular suffix rules that disagreed about which irregulars they
knew; both now delegate.
2. render._display_noun re-inflects at render time and swaps _ for a space,
so compound ids read "guard dogs" rather than "guards dog" or
"guard_dog".
Result: reader gives wolves -> wolf, renderer gives wolf -> wolves. The
round trip closes.
0 malformed of 47 (from 4)
reader-vs-reader singularization 20/20 (from 8/20), 0 silently wrong
s_surface_match_rate 0.0 -> 0.625, now EQUAL to s_renderable_rate, so the
only remaining losses are surfaces the renderer cannot express at all
NEAR-MISS — widening a reader broke soundness, caught by the lane:
Routing the reader through the full 29-entry table first produced wrong=1 on
band v6-EX. ds-ex-0012 ("No fish are mammals. Therefore some fish are
mammals.") answered invalid where gold is refuted.
_singularize returning None makes the reader REFUSE, and the serving
composer tries the categorical band FIRST, falling through to more capable
bands. "fish" previously returned None (it matches no suffix rule), so v6-EX
got the case and decided it correctly. Resolving "fish" made v1b accept a
sentence it cannot decide.
Fixed by a principle, not a patch: a number-INVARIANT form is ambiguous in
number — "fish are mammals" is plural, "a fish is a mammal" is singular, and
the token cannot tell you which — so a reader that must not guess number
declines it. lexicon.INVARIANT_NUMBER is derived from the singularizer's own
key == value rows, so the rule cannot drift from the table.
=> Coverage and correctness are different axes. Fixing a corrupted VALUE is
safe; widening ACCEPTANCE changes which band answers, and in a
first-match composer that turns a right answer into a wrong one. Same
family as ADR-0261 5.1 refuse-don't-drop.
Also fixed, found by testing the inverse law: the two number tables were not
mutual inverses. The pluralizer lacked cactus->cacti, fungus->fungi,
die->dice and the invariants aircraft/means/offspring, so CORE could read
"cacti" but wrote "cactuses", and would have written "aircrafts", "meanses",
"offsprings". No round trip can close across a non-invertible table.
THE LANE SHA PINS ARE BLIND TO SERVED ENGLISH:
2B changed 4 served surfaces and the pins came back 11/11 byte-identical.
The deduction_serve_v1 hashed report holds only n/counts/by_gold/
correct_by_gold/all_cases_correct/mismatch_examples — no prose at all.
Confirmed by sabotage: with _display_noun returning "SABOTAGE_" + ..., so
every clause reads "all SABOTAGE_dogs are SABOTAGE_animals",
11 lane SHA pins -> 11/11 byte-identical, blind
test_deduction_serve_lane + _license -> 20 passed, blind
Phase 1 grammar_roundtrip -> 3 tests RED, caught it
This corrects #129: byte-identity of the pins is the arbiter for values and
verdicts, NOT for surface text, so the 2A/2B split is not a partition of
"changes users can see." Phase 2A's 11/11 conclusion still stands on its own
independent evidence (24/24 table-equality checks vs the pre-migration
literals), but the justification was thinner than stated.
Before Phase 1 no test in the tree would have noticed CORE's served prose
turning into word salad. That is exactly how "all dog are mammal" survived
on a ratified flag-ON band with wrong=0 intact.
Deliberately not fixed: mass-noun verb agreement ("all evidence ARE truth"
should be "is"). The copula is fixed text inside the templates, so agreeing
it changes the template shape rather than a slot, and no mass noun reaches a
categorical clause in any serve corpus. Recorded in render.py.
No lane pin edited — none moved.
[Verification]: in-worktree on CPython 3.12.13, uv sync --locked —
smoke 621 unchanged; deductive 403 passed (383 + 20 new/rewritten);
scripts/verify_lane_shas.py 11/11 (see blindness note above — gate on
grammar_roundtrip for surface work, not on these pins).
This commit is contained in:
Shay
parent
899a6831d0
commit
5ba1359a79
8 changed files with 478 additions and 135 deletions
|
|
@ -558,6 +558,12 @@ Deliver:
|
|||
`wolves`→`wolve`, `news`→`new`, `species`→`specy`. Its own comment claims it
|
||||
refuses; it does not. Those five are pinned as current behaviour in
|
||||
`tests/test_lexicon_single_source.py` and this phase must flip them.
|
||||
|
||||
*Delivered differently than written:* the reader no longer holds a table at
|
||||
all. `generate/morphology.py` became the single owner of the number **rules**
|
||||
(2A owned the tables), and the reader calls `singularize`. That removed a
|
||||
second copy of the regular suffix rules, which was the actual reason the two
|
||||
sides disagreed about coverage.
|
||||
3. Updated lane SHA pins — **surgical single-line edits only**, never
|
||||
`--update` — with the old and new hash recorded per lane.
|
||||
|
||||
|
|
@ -571,7 +577,98 @@ Deliver:
|
|||
| hash movement is intended | every moved pin explained; no pin changed that shouldn't move |
|
||||
|
||||
**Why gated:** this changes what users see. Fixing a defect is still a serving
|
||||
change, and serving changes are Shay's call.
|
||||
change, and serving changes are Shay's call. *Authorized 2026-07-26.*
|
||||
|
||||
**RESULT.**
|
||||
|
||||
| criterion | measured |
|
||||
|---|---|
|
||||
| surfaces well-formed | **0 malformed of 47** (from 4); `ds-v1-0023/0024/0026/0028` all render plurals |
|
||||
| readers agree | **20/20**, 0 silently wrong (from 8/20) |
|
||||
| soundness preserved | `wrong=0` holds on all 7 bands; deductive **403 passed**, smoke **621** |
|
||||
| round-trip | `s_surface_match_rate` **0.0 → 0.625**, and it now *equals* `s_renderable_rate` — every renderable surface returns its input exactly |
|
||||
| hash movement is intended | **criterion void — the pins cannot see surfaces.** See below. |
|
||||
|
||||
#### The lane SHA pins are blind to served English
|
||||
|
||||
2B changed 4 of 47 served surfaces and the pins came back **11/11
|
||||
byte-identical**. That is not evidence of no change; it is evidence the
|
||||
instrument cannot see this kind of change.
|
||||
|
||||
Verified two ways rather than argued:
|
||||
|
||||
1. **By inspection.** The `deduction_serve_v1` hashed report contains only
|
||||
`n`, `counts`, `by_gold`, `correct_by_gold`, `all_cases_correct`,
|
||||
`mismatch_examples`. **No surface prose at all** — verdict classifications
|
||||
only.
|
||||
2. **By sabotage** (the [[feedback-ask-what-if-the-thing-were-absent]] control).
|
||||
Making `render._display_noun` return `"SABOTAGE_" + …`, so every categorical
|
||||
clause reads `all SABOTAGE_dogs are SABOTAGE_animals`:
|
||||
|
||||
| guard | caught it? |
|
||||
|---|---|
|
||||
| 11 lane SHA pins | **no — 11/11 still byte-identical** |
|
||||
| `test_deduction_serve_lane` + `_license` (20 tests) | **no — 20 passed** |
|
||||
| Phase 1 `grammar_roundtrip` | **yes — 3 tests red** |
|
||||
|
||||
⇒ **Correction to #129.** That PR established the arc's arbiter as "make the
|
||||
change, run the 11 pinned lanes, let byte-identity decide." For *values and
|
||||
verdicts* that holds. For **surface text it decides nothing**, so the 2A/2B
|
||||
split it defines is not a partition of "changes users can see." Phase 2A's
|
||||
11/11 conclusion is still sound — 2A changed no value, confirmed independently
|
||||
by 24/24 table-equality checks against the pre-migration literals — but the
|
||||
justification was thinner than claimed, and 2B is what exposed it.
|
||||
|
||||
⇒ **Before Phase 1, no test in the tree would have noticed CORE's served prose
|
||||
turning into word salad.** That is precisely how `all dog are mammal` survived
|
||||
on a ratified, flag-ON band with `wrong=0` intact for the whole arc. The
|
||||
round-trip lane is not a nice-to-have measurement; it is the only guard that
|
||||
reads what the user reads.
|
||||
|
||||
⇒ **Carried forward:** any future phase touching surfaces must gate on
|
||||
`grammar_roundtrip`, not on the lane pins. Adding surface text to the pinned
|
||||
lane payloads would be the durable fix and is *not* done here — it would move
|
||||
every pin at once and deserves its own unit.
|
||||
|
||||
The Phase 1 instrument moved on its own, having been built with no knowledge of
|
||||
this fix. That is the whole argument for building the instrument first.
|
||||
|
||||
**The near-miss worth recording — widening a reader can break soundness.**
|
||||
Routing the reader through the full 29-entry singularizer initially produced
|
||||
**wrong=1 on band v6-EX**: `ds-ex-0012` ("No fish are mammals. Therefore some
|
||||
fish are mammals.") answered `invalid` where gold is `refuted`.
|
||||
|
||||
Cause: `_singularize` returning `None` makes the reader *refuse*, and the
|
||||
serving composer tries the categorical band **first**, falling through to later,
|
||||
more capable bands. `fish` previously returned `None` (it matches no suffix
|
||||
rule), so v6-EX got the case and decided it correctly. Resolving `fish` made
|
||||
**v1b accept a sentence it cannot decide** and answer wrongly.
|
||||
|
||||
Fix, and it is a principle rather than a patch: **a number-invariant form is
|
||||
ambiguous in number** — "fish are mammals" is plural, "a fish is a mammal" is
|
||||
singular, and the token cannot tell you which — so a reader that must not guess
|
||||
number declines it. That simultaneously restores the fall-through *and* is the
|
||||
honest reading. `lexicon.INVARIANT_NUMBER` is derived from the singularizer's
|
||||
own `key == value` rows, so the rule cannot drift from the table.
|
||||
|
||||
⇒ **Coverage and correctness are different axes.** Fixing a corrupted *value*
|
||||
(`wolves`→`wolve`) is safe; widening *acceptance* changes which band answers,
|
||||
and in a first-match composer that can convert a right answer into a wrong one.
|
||||
Same family as ADR-0261 §5.1 refuse-don't-drop. Bundling the two is how a
|
||||
plural fix silently becomes a soundness regression.
|
||||
|
||||
**Also fixed, found by testing the inverse law:** the two number tables were not
|
||||
mutual inverses — the pluralizer lacked `cactus→cacti`, `fungus→fungi`,
|
||||
`die→dice` and the invariants `aircraft`/`means`/`offspring`, so CORE could
|
||||
*read* `cacti` but wrote `cactuses`, and would have written "aircrafts",
|
||||
"meanses", "offsprings". No round trip can close across a table that is not
|
||||
invertible. Pinned by `test_the_two_number_directions_are_mutual_inverses`.
|
||||
|
||||
**Deliberately not fixed:** mass-noun verb agreement. "all evidence are truth"
|
||||
should be "all evidence **is** truth", but the copula is fixed text inside the
|
||||
A/E/I/O templates, so agreeing it changes the template shape rather than a slot.
|
||||
No mass noun reaches a categorical clause in any serve corpus today. Recorded in
|
||||
`render.py::_display_noun` rather than silently left.
|
||||
|
||||
### Phase 3 — Fix the 9-of-26 agreement defect
|
||||
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ call sites need different content, this module defines one literal and
|
|||
|
||||
Naming convention: a name is the linguistic fact, not the consumer. When a
|
||||
consumer needs a narrower view, the narrow name says whose view it is
|
||||
(``STRUCTURAL``, ``READER_SINGULAR_KEYS``).
|
||||
(``STRUCTURAL``, ``NEGATION_BEARING_WITH_NOT``).
|
||||
|
||||
**Deliberate non-goal:** this module does not decide anything. It holds
|
||||
tables and derivations; the refusal/agreement logic stays with each band.
|
||||
|
|
@ -256,6 +256,11 @@ IRREGULAR_PLURALS: Final[dict[str, str]] = {
|
|||
"analysis": "analyses", "axis": "axes", "basis": "bases",
|
||||
"thesis": "theses", "hypothesis": "hypotheses",
|
||||
"mitochondrion": "mitochondria",
|
||||
# Added in Phase 2B so the two directions are mutual inverses. The
|
||||
# singularizer knew cacti/fungi/dice; without these the pluralizer
|
||||
# produced "cactuses"/"funguses"/"dies" and CORE could read a word it
|
||||
# could not write back.
|
||||
"cactus": "cacti", "fungus": "fungi", "die": "dice",
|
||||
}
|
||||
|
||||
#: Plural -> singular, including invariants that map to themselves. The
|
||||
|
|
@ -277,29 +282,43 @@ IRREGULAR_SINGULARS: Final[dict[str, str]] = {
|
|||
"offspring": "offspring", "aircraft": "aircraft", "news": "news",
|
||||
}
|
||||
|
||||
#: The 8 keys ``meaning_graph/reader.py`` currently covers.
|
||||
#: Nouns whose plural equals their singular, **derived** from the invariant
|
||||
#: rows of :data:`IRREGULAR_SINGULARS` (the rows where key == value).
|
||||
#:
|
||||
#: **Recorded divergence (Phase 2A decision).** The reader's values are all
|
||||
#: *correct* — they agree with :data:`IRREGULAR_SINGULARS` entry for entry —
|
||||
#: so the values are derived below and only the key list is local. What is
|
||||
#: wrong is the **coverage**, and the consequence is not a refusal:
|
||||
#: ``reader._singularize`` falls through to a bare ``-s`` strip, so an
|
||||
#: uncovered plural is silently mis-singularized rather than declined
|
||||
#: (``wolves`` -> ``wolve``, ``news`` -> ``new``, ``species`` -> ``specy``).
|
||||
#: The reader's own comment claims it "REFUSES rather than guessing a wrong
|
||||
#: singular (wrong=0)"; the code does not implement that.
|
||||
#:
|
||||
#: Widening this to the full table changes minted entity ids and therefore
|
||||
#: served surfaces and trace hashes, so it is **Phase 2B** work behind the
|
||||
#: authorization gate. Recorded here, with the gap named, rather than fixed
|
||||
#: silently.
|
||||
READER_SINGULAR_KEYS: Final[tuple[str, ...]] = (
|
||||
"people", "men", "women", "children", "feet", "teeth", "mice", "geese",
|
||||
#: Derived rather than re-listed because a hand-written second list is exactly
|
||||
#: how the two directions drifted apart in the first place: before Phase 2B the
|
||||
#: pluralizer lacked ``aircraft``/``means``/``offspring`` and produced
|
||||
#: "aircrafts", "meanses", "offsprings" while the singularizer knew all three.
|
||||
#: ``news`` was saved only by coincidence — it is also a mass noun.
|
||||
INVARIANT_NUMBER: Final[frozenset[str]] = frozenset(
|
||||
plural for plural, singular in IRREGULAR_SINGULARS.items() if plural == singular
|
||||
)
|
||||
|
||||
#: Uncountable nouns — they take a quantifier but stay singular, so
|
||||
#: ``all evidence`` is right and ``all evidences`` is wrong. The verb still
|
||||
#: agrees singular ("all evidence supports truth"), which the categorical
|
||||
#: renderer does **not** yet handle; see the Phase 2B note in
|
||||
#: ``proof_chain/render.py``.
|
||||
#:
|
||||
#: Moved here in Phase 2B: it was a single copy in ``templates.py``, but the
|
||||
#: pluralizer needs it and the pluralizer now serves, so the table has to be
|
||||
#: reachable from both without a second literal. Covers the abstract/epistemic
|
||||
#: vocabulary in ``en_core_cognition_v1`` plus common English mass nouns.
|
||||
MASS_NOUNS: Final[frozenset[str]] = frozenset(
|
||||
{
|
||||
# epistemic / abstract (the seed-pack vocabulary)
|
||||
"evidence", "wisdom", "knowledge", "truth", "light", "darkness",
|
||||
"information", "data", "music", "art", "literature", "philosophy",
|
||||
"courage", "patience", "love", "hope", "fear", "grace",
|
||||
"meaning", "purpose", "beauty", "justice", "freedom",
|
||||
# physical mass
|
||||
"water", "air", "fire", "earth", "sand", "rain", "snow", "ice",
|
||||
"wood", "metal", "gold", "silver", "iron", "stone",
|
||||
"blood", "flesh", "bone",
|
||||
# collective / continuous
|
||||
"weather", "traffic", "furniture", "luggage", "advice",
|
||||
"equipment", "machinery", "scenery", "money", "news",
|
||||
"research", "progress", "feedback",
|
||||
}
|
||||
)
|
||||
|
||||
#: The reader's view of :data:`IRREGULAR_SINGULARS`, restricted to
|
||||
#: :data:`READER_SINGULAR_KEYS`. Derived, so the reader's 8 values cannot
|
||||
#: drift from the 29-entry table they are a subset of.
|
||||
READER_IRREGULAR_SINGULARS: Final[dict[str, str]] = {
|
||||
key: IRREGULAR_SINGULARS[key] for key in READER_SINGULAR_KEYS
|
||||
}
|
||||
|
|
|
|||
|
|
@ -51,7 +51,7 @@ from __future__ import annotations
|
|||
import re
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from generate.lexicon import READER_IRREGULAR_SINGULARS
|
||||
from generate.morphology import singularize as _shared_singularize
|
||||
from generate.meaning_graph.model import (
|
||||
Entity,
|
||||
MeaningGraph,
|
||||
|
|
@ -62,9 +62,17 @@ from generate.meaning_graph.model import (
|
|||
|
||||
_ARTICLES = frozenset({"a", "an"})
|
||||
|
||||
# Common irregular plurals the corpus exercises. Conservative + closed; an
|
||||
# unrecognized plural REFUSES rather than guessing a wrong singular (wrong=0).
|
||||
_IRREGULAR_PLURALS = READER_IRREGULAR_SINGULARS
|
||||
# Phase 2B: this reader used to carry its own 8-entry plural table plus a
|
||||
# private copy of the suffix rules. Both now live in generate/morphology.py,
|
||||
# which reads lexicon.IRREGULAR_SINGULARS — 29 entries, the widest number
|
||||
# table CORE has.
|
||||
#
|
||||
# The old comment here claimed "an unrecognized plural REFUSES rather than
|
||||
# guessing a wrong singular (wrong=0)". That was false: the code fell through
|
||||
# to a bare -s strip, so wolves -> wolve, news -> new, species -> specy, and
|
||||
# those corrupted ids reached served text ("all wolve are mammal"). The shared
|
||||
# singularizer treats the table as authoritative and returns None instead of
|
||||
# guessing, so the comment is now true of the code.
|
||||
|
||||
# Categorical quantifier -> the MeaningGraph predicate it mints. The predicate
|
||||
# vocabulary is shared between facts and the "therefore" conclusion query, and is
|
||||
|
|
@ -152,15 +160,7 @@ class _Reject(Exception):
|
|||
|
||||
def _singularize(word: str) -> str | None:
|
||||
"""Conservative plural -> singular. None when not confidently a plural."""
|
||||
if word in _IRREGULAR_PLURALS:
|
||||
return _IRREGULAR_PLURALS[word]
|
||||
if word.endswith("ies") and len(word) > 3:
|
||||
return word[:-3] + "y"
|
||||
if word.endswith(("ses", "xes", "zes", "ches", "shes")):
|
||||
return word[:-2]
|
||||
if word.endswith("s") and not word.endswith("ss") and len(word) > 1:
|
||||
return word[:-1]
|
||||
return None
|
||||
return _shared_singularize(word)
|
||||
|
||||
|
||||
def _chunk(words: list[str], detail: str) -> str:
|
||||
|
|
|
|||
|
|
@ -1,11 +1,27 @@
|
|||
"""Deterministic English morphology for the realizer.
|
||||
|
||||
Handles inflection of predicates for tense, aspect, and negation.
|
||||
Handles inflection of predicates for tense, aspect, and negation, and
|
||||
(Phase 2B) **noun number** in both directions.
|
||||
|
||||
This is intentionally rule-based and limited to the seed vocabulary.
|
||||
Irregular forms are listed explicitly; regular forms follow English rules.
|
||||
|
||||
Phase 2A gave every *table* one owner (``generate/lexicon.py``); this module
|
||||
is the single owner of the *rules* that read them. Before 2B the regular
|
||||
number rules were written twice — ``templates.pluralize`` and
|
||||
``meaning_graph/reader._singularize`` — and the two disagreed about which
|
||||
irregulars they knew, which is how CORE came to serve ``all wolve are
|
||||
mammal``.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from generate.lexicon import (
|
||||
INVARIANT_NUMBER,
|
||||
IRREGULAR_PLURALS,
|
||||
IRREGULAR_SINGULARS,
|
||||
MASS_NOUNS,
|
||||
)
|
||||
|
||||
|
||||
# Genuinely irregular English verbs (the previous tables held only
|
||||
# regular forms that the suffix rules already produce correctly).
|
||||
|
|
@ -210,5 +226,103 @@ def past_participle(verb_3sg: str) -> str:
|
|||
return past_tense(verb_3sg)
|
||||
|
||||
|
||||
_SIBILANT_PLURAL_ENDINGS = ("s", "sh", "ch", "x", "z")
|
||||
|
||||
|
||||
def is_mass_noun(noun: str) -> bool:
|
||||
"""Uncountable ⇒ never pluralized, even under a quantifier."""
|
||||
return noun.lower() in MASS_NOUNS
|
||||
|
||||
|
||||
def _head_and_prefix(noun: str) -> tuple[str, str]:
|
||||
"""Split a canonical id into (everything-before-head, head).
|
||||
|
||||
Reader ids are lowercase tokens joined with ``_`` (``guard_dog``), and
|
||||
number marks the **head**, which in English compounds is the last token:
|
||||
``guard_dog`` → ``guard_dogs``, never ``guards_dog``. Space-joined input is
|
||||
handled the same way so display strings and ids behave alike.
|
||||
"""
|
||||
for sep in ("_", " "):
|
||||
if sep in noun:
|
||||
prefix, _, head = noun.rpartition(sep)
|
||||
return prefix + sep, head
|
||||
return "", noun
|
||||
|
||||
|
||||
def pluralize(noun: str) -> str:
|
||||
"""Singular → plural. Table first, then closed regular rules.
|
||||
|
||||
Mass nouns and the empty string are returned unchanged. Compound ids
|
||||
inflect on the head only.
|
||||
"""
|
||||
if not noun:
|
||||
return noun
|
||||
if is_mass_noun(noun):
|
||||
return noun
|
||||
prefix, head = _head_and_prefix(noun)
|
||||
if head in IRREGULAR_PLURALS:
|
||||
return prefix + IRREGULAR_PLURALS[head]
|
||||
# Invariant number (sheep, aircraft, means, offspring, ...) — derived from
|
||||
# the singularizer's own invariant rows, so the two directions agree.
|
||||
if head in INVARIANT_NUMBER:
|
||||
return noun
|
||||
if is_mass_noun(head):
|
||||
return noun
|
||||
if head.endswith(_SIBILANT_PLURAL_ENDINGS):
|
||||
return prefix + head + "es"
|
||||
if head.endswith("y") and len(head) > 1 and head[-2] not in "aeiou":
|
||||
return prefix + head[:-1] + "ies"
|
||||
if head.endswith("fe"):
|
||||
return prefix + head[:-2] + "ves"
|
||||
if head.endswith("f"):
|
||||
return prefix + head[:-1] + "ves"
|
||||
return prefix + head + "s"
|
||||
|
||||
|
||||
def singularize(noun: str) -> str | None:
|
||||
"""Plural → singular, or ``None`` when not confidently a plural.
|
||||
|
||||
The table is consulted first and is **authoritative**: if the token
|
||||
appears there, its value wins and no suffix rule runs. That is what keeps
|
||||
``news``/``new`` and ``species``/``specy`` unlinked — a bare ``-s`` strip
|
||||
corrupts both, and did, in served text.
|
||||
|
||||
``None`` (rather than a guess) is returned for anything the closed rules
|
||||
do not confidently cover, so a caller can refuse instead of minting a
|
||||
corrupted id.
|
||||
"""
|
||||
if not noun:
|
||||
return None
|
||||
prefix, head = _head_and_prefix(noun)
|
||||
# An INVARIANT form is ambiguous in number — "fish are mammals" is plural,
|
||||
# "a fish is a mammal" is singular, and the token cannot tell you which. A
|
||||
# reader that must not guess number therefore has to DECLINE these, even
|
||||
# though the table technically maps fish -> fish.
|
||||
#
|
||||
# This is load-bearing for soundness, not tidiness. The serving composer
|
||||
# tries the categorical band (v1b) FIRST and falls back to later, more
|
||||
# capable bands. Resolving an invariant makes v1b ACCEPT a sentence it
|
||||
# cannot decide, which steals the case from the band that can: with
|
||||
# "No fish are mammals. Therefore some fish are mammals." v1b answers
|
||||
# "invalid" where v6-EX correctly answers "refuted" (ds-ex-0012). Declining
|
||||
# keeps the fall-through intact. Same family as ADR-0261 §5.1
|
||||
# refuse-don't-drop.
|
||||
if head in INVARIANT_NUMBER:
|
||||
return None
|
||||
if head in IRREGULAR_SINGULARS:
|
||||
return prefix + IRREGULAR_SINGULARS[head]
|
||||
# A singular that the pluralizer knows is irregular is not a plural at all
|
||||
# ("child" must not become "chil"), and neither is a known mass noun.
|
||||
if head in IRREGULAR_PLURALS or is_mass_noun(head):
|
||||
return None
|
||||
if head.endswith("ies") and len(head) > 3:
|
||||
return prefix + head[:-3] + "y"
|
||||
if head.endswith(("ses", "xes", "zes", "ches", "shes")):
|
||||
return prefix + head[:-2]
|
||||
if head.endswith("s") and not head.endswith("ss") and len(head) > 1:
|
||||
return prefix + head[:-1]
|
||||
return None
|
||||
|
||||
|
||||
def base_form(verb_3sg: str) -> str:
|
||||
return _base_form(verb_3sg)
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ renders realized-structure DETERMINE answers — a different gear entirely).
|
|||
|
||||
from __future__ import annotations
|
||||
|
||||
from generate.morphology import pluralize
|
||||
from generate.proof_chain.entail import (
|
||||
INCONSISTENT_PREMISES,
|
||||
Entailment,
|
||||
|
|
@ -18,6 +19,12 @@ from generate.proof_chain.entail import (
|
|||
)
|
||||
|
||||
#: A/E/I/O categorical form → an English sentence template over (subject, predicate).
|
||||
#:
|
||||
#: Every template takes a PLURAL noun on both sides — ``all``/``no``/``some``
|
||||
#: with ``are``. The slots are filled from canonical entity ids, which are
|
||||
#: SINGULAR lowercase lemmas (``dog``, ``mammal``), so filling them raw emitted
|
||||
#: ``all dog are mammal``. Phase 2B re-inflects at render time; see
|
||||
#: :func:`_display_noun`.
|
||||
_CATEGORICAL_PHRASE = {
|
||||
"A": "all {s} are {p}",
|
||||
"E": "no {s} are {p}",
|
||||
|
|
@ -208,10 +215,37 @@ def render_entailment_exist(
|
|||
return f"Given: {given}. I can't evaluate {query_text} from that as stated."
|
||||
|
||||
|
||||
def _display_noun(term: str) -> str:
|
||||
"""A canonical entity id → the plural noun phrase a categorical clause needs.
|
||||
|
||||
Two transformations, both required for well-formed output:
|
||||
|
||||
1. **Number.** Ids are singular lemmas but every A/E/I/O template supplies
|
||||
``all``/``no``/``some`` + ``are``, which demand a plural. Mass nouns are
|
||||
left alone by :func:`generate.morphology.pluralize` ("all evidence", not
|
||||
"all evidences").
|
||||
2. **Word separator.** Ids join tokens with ``_`` (``guard_dog``), which is
|
||||
machine syntax, not English. Compounds inflect on the head, so
|
||||
pluralizing before the swap gives "guard dogs" rather than "guards dog".
|
||||
|
||||
Deliberately NOT done here: mass-noun verb agreement. "all evidence are
|
||||
truth" should read "all evidence is truth", but the copula is fixed text
|
||||
inside the templates, and making it agree changes the template shape rather
|
||||
than the slot. Left as a separate, smaller defect — no mass noun currently
|
||||
reaches a categorical clause in any serve corpus.
|
||||
"""
|
||||
if not term:
|
||||
return term
|
||||
return pluralize(term).replace("_", " ")
|
||||
|
||||
|
||||
def _categorical_clause(prop: dict) -> str:
|
||||
"""One categorical proposition dict → its English clause."""
|
||||
template = _CATEGORICAL_PHRASE.get(prop.get("form", ""), "{s} ~ {p}")
|
||||
return template.format(s=prop.get("subject", "?"), p=prop.get("predicate", "?"))
|
||||
return template.format(
|
||||
s=_display_noun(prop.get("subject", "?")),
|
||||
p=_display_noun(prop.get("predicate", "?")),
|
||||
)
|
||||
|
||||
|
||||
def render_syllogism(trace: EntailmentTrace, structure: dict, query: dict) -> str:
|
||||
|
|
|
|||
|
|
@ -22,62 +22,30 @@ from generate.articulation_legality import (
|
|||
validate_finite_predicate_legality,
|
||||
)
|
||||
from generate.graph_planner import RhetoricalMove
|
||||
from generate.morphology import base_form, past_participle, past_tense, present_participle
|
||||
from generate.morphology import (
|
||||
base_form,
|
||||
is_mass_noun,
|
||||
past_participle,
|
||||
past_tense,
|
||||
pluralize,
|
||||
present_participle,
|
||||
)
|
||||
|
||||
|
||||
# Noun pluralisation — used under quantifiers (all/some/many/few/most).
|
||||
# Closes english_fluency_ood gaps.md G2 (plural agreement).
|
||||
#
|
||||
# Phase 2B: the rules moved to generate/morphology.py, which now owns number
|
||||
# in both directions. Re-exported here because this module's public surface
|
||||
# is consumed by the eval runners.
|
||||
_IRREGULAR_PLURALS: dict[str, str] = IRREGULAR_PLURALS
|
||||
|
||||
|
||||
def pluralize(noun: str) -> str:
|
||||
if not noun:
|
||||
return noun
|
||||
if noun in _IRREGULAR_PLURALS:
|
||||
return _IRREGULAR_PLURALS[noun]
|
||||
n = noun
|
||||
if n.endswith(("s", "sh", "ch", "x", "z")):
|
||||
return n + "es"
|
||||
if n.endswith("y") and len(n) > 1 and n[-2] not in "aeiou":
|
||||
return n[:-1] + "ies"
|
||||
if n.endswith("fe"):
|
||||
return n[:-2] + "ves"
|
||||
if n.endswith("f"):
|
||||
return n[:-1] + "ves"
|
||||
return n + "s"
|
||||
|
||||
|
||||
# Quantifiers that demand plural agreement on the subject + verb.
|
||||
# "the" / "a" stay singular; "every" / "each" are singular by English
|
||||
# rule even though semantically universal.
|
||||
_PLURAL_QUANTIFIERS: frozenset[str] = PLURAL_QUANTIFIERS
|
||||
|
||||
# Mass nouns — uncountable in English, so "all evidence", "some wisdom"
|
||||
# stay singular under quantifiers ("all evidences" is wrong). The
|
||||
# verb still agrees (singular: "all evidence supports truth").
|
||||
# This list covers the abstract/epistemic vocabulary in
|
||||
# en_core_cognition_v1 + common English mass nouns.
|
||||
_MASS_NOUNS: frozenset[str] = frozenset({
|
||||
# epistemic / abstract (the seed-pack vocabulary)
|
||||
"evidence", "wisdom", "knowledge", "truth", "light", "darkness",
|
||||
"information", "data", "music", "art", "literature", "philosophy",
|
||||
"courage", "patience", "love", "hope", "fear", "grace",
|
||||
"meaning", "purpose", "beauty", "justice", "freedom",
|
||||
# physical mass
|
||||
"water", "air", "fire", "earth", "sand", "rain", "snow", "ice",
|
||||
"wood", "metal", "gold", "silver", "iron", "stone",
|
||||
"blood", "flesh", "bone",
|
||||
# collective / continuous
|
||||
"weather", "traffic", "furniture", "luggage", "advice",
|
||||
"equipment", "machinery", "scenery", "money", "news",
|
||||
"research", "progress", "feedback",
|
||||
})
|
||||
|
||||
|
||||
def is_mass_noun(noun: str) -> bool:
|
||||
return noun.lower() in _MASS_NOUNS
|
||||
|
||||
|
||||
_PREDICATE_DISPLAY: dict[str, str] = PREDICATE_DISPLAY
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -147,28 +147,59 @@ def test_g_roundtrip_baseline_is_zero(report):
|
|||
assert report.metrics["g_exact_rate"] == 0.0
|
||||
|
||||
|
||||
def test_s_roundtrip_pins_the_categorical_render_defect(report):
|
||||
"""BASELINE PIN (a defect): nothing CORE reads renders back to its input.
|
||||
def test_s_roundtrip_closes_for_every_renderable_surface(report):
|
||||
"""Phase 2A pinned this at **0.0** — nothing CORE read rendered back to its
|
||||
input, because the serving renderer interpolated singularized entity ids
|
||||
into plural templates ("all dog are animal").
|
||||
|
||||
The reader comprehends all 8 positive surfaces, and the serving categorical
|
||||
renderer reproduces none of them, because it interpolates singularized
|
||||
entity ids into plural templates. Phase 2B fixes this and must raise this
|
||||
number.
|
||||
Phase 2B re-inflects at render time, and every surface the renderer can
|
||||
express now returns its input exactly. The two rates are equal on purpose:
|
||||
``s_surface_match_rate == s_renderable_rate`` says the *only* remaining
|
||||
round-trip losses are surfaces the categorical renderer cannot express at
|
||||
all, which is a coverage gap in a later phase, not a grammar defect.
|
||||
|
||||
Asserting equality rather than a literal keeps this honest if the corpus
|
||||
grows: adding an unrenderable positive lowers both numbers together, while
|
||||
reintroducing the plural defect lowers only the match rate.
|
||||
"""
|
||||
assert report.metrics["s_read_rate"] == 1.0
|
||||
assert report.metrics["s_surface_match_rate"] == 0.0
|
||||
assert report.metrics["s_surface_match_rate"] == report.metrics["s_renderable_rate"]
|
||||
assert report.metrics["s_surface_match_rate"] > 0.0
|
||||
|
||||
|
||||
def test_the_categorical_render_defect_concretely():
|
||||
"""The specific defect, stated as an example rather than a rate."""
|
||||
def test_the_categorical_render_defect_is_fixed_concretely():
|
||||
"""The 2A defect stated as an example rather than a rate, now inverted.
|
||||
|
||||
Both sides matter: the graph still carries the SINGULAR canonical id
|
||||
(``dog``), so the fix is in the renderer's inflection and not in the
|
||||
comprehension it was built to preserve.
|
||||
"""
|
||||
result = comprehend("All dogs are animals.", source_id="t")
|
||||
assert isinstance(result, Comprehension)
|
||||
props = sorted(from_meaning_graph(result.meaning_graph))
|
||||
assert props == [CanonicalProposition("subset", "dog", "animal", False)]
|
||||
rendered = rt_runner._render_categorical("subset", "dog", "animal")
|
||||
# "all dog are animal" — plural template, singular entity ids.
|
||||
assert rendered == "all dog are animal"
|
||||
assert rendered != "all dogs are animals"
|
||||
assert rendered == "all dogs are animals"
|
||||
|
||||
|
||||
def test_irregular_plurals_survive_the_full_round_trip():
|
||||
"""The reader and renderer now share one number table, so an irregular
|
||||
plural returns as itself. Before 2B this produced ``all wolve are
|
||||
mammals`` — the reader's bare ``-s`` strip leaking a corrupted id straight
|
||||
into served text."""
|
||||
cases = (
|
||||
("All wolves are mammals.", "wolf", "all wolves are mammals"),
|
||||
("All children are mammals.", "child", "all children are mammals"),
|
||||
("All men are mammals.", "man", "all men are mammals"),
|
||||
("All knives are tools.", "knife", "all knives are tools"),
|
||||
)
|
||||
for surface, singular, expected_clause in cases:
|
||||
result = comprehend(surface, source_id="t")
|
||||
assert isinstance(result, Comprehension), surface
|
||||
props = sorted(from_meaning_graph(result.meaning_graph))
|
||||
assert props[0].subject == singular, f"{surface} -> {props}"
|
||||
rendered = rt_runner._render_categorical("subset", singular, props[0].obj)
|
||||
assert rendered == expected_clause
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
|
|
|||
|
|
@ -67,9 +67,18 @@ RECORDED_CONSUMERS: dict[str, frozenset[str]] = {
|
|||
"generate.semantic_templates",
|
||||
}),
|
||||
"DISCOURSE_PREDICATE_DISPLAY": frozenset({"generate.discourse_planner"}),
|
||||
"IRREGULAR_PLURALS": frozenset({"generate.templates"}),
|
||||
"IRREGULAR_SINGULARS": frozenset({"generate.proof_chain.member"}),
|
||||
"READER_IRREGULAR_SINGULARS": frozenset({"generate.meaning_graph.reader"}),
|
||||
# Phase 2B: generate.morphology became the single owner of the number
|
||||
# RULES, so it is now the consumer of the number tables. The reader no
|
||||
# longer imports a table of its own — it calls morphology.singularize —
|
||||
# which is why READER_IRREGULAR_SINGULARS/READER_SINGULAR_KEYS were deleted
|
||||
# rather than left behind documenting a coverage gap that no longer exists.
|
||||
"IRREGULAR_PLURALS": frozenset({"generate.morphology", "generate.templates"}),
|
||||
"IRREGULAR_SINGULARS": frozenset({
|
||||
"generate.morphology",
|
||||
"generate.proof_chain.member",
|
||||
}),
|
||||
"INVARIANT_NUMBER": frozenset({"generate.morphology"}),
|
||||
"MASS_NOUNS": frozenset({"generate.morphology"}),
|
||||
}
|
||||
|
||||
#: The tables a duplicate literal would be a duplicate *of*. Frozen as
|
||||
|
|
@ -295,15 +304,17 @@ def test_the_two_number_tables_are_inverse_directions_not_copies() -> None:
|
|||
assert lexicon.IRREGULAR_SINGULARS[plur] == sing, f"{sing}/{plur} not inverse"
|
||||
|
||||
|
||||
def test_reader_number_table_is_a_derived_subset_that_cannot_drift() -> None:
|
||||
"""The reader's 8 values were measured to AGREE with the 29-entry table
|
||||
entry for entry — the set difference in the reader's favour was empty. So
|
||||
only its key list is local; the values are derived."""
|
||||
assert set(lexicon.READER_IRREGULAR_SINGULARS) == set(lexicon.READER_SINGULAR_KEYS)
|
||||
assert len(lexicon.READER_SINGULAR_KEYS) == 8
|
||||
for key, value in lexicon.READER_IRREGULAR_SINGULARS.items():
|
||||
assert lexicon.IRREGULAR_SINGULARS[key] == value
|
||||
assert set(lexicon.READER_IRREGULAR_SINGULARS) < set(lexicon.IRREGULAR_SINGULARS)
|
||||
def test_invariant_number_is_derived_from_the_singularizer() -> None:
|
||||
"""Invariants must not be a second hand-written list.
|
||||
|
||||
A hand-written list is exactly how the directions drifted: the pluralizer
|
||||
lacked ``aircraft``/``means``/``offspring`` and produced "aircrafts",
|
||||
"meanses", "offsprings" while the singularizer knew all three. Deriving
|
||||
from the ``key == value`` rows makes that class of gap unrepresentable."""
|
||||
expected = {p for p, s in lexicon.IRREGULAR_SINGULARS.items() if p == s}
|
||||
assert lexicon.INVARIANT_NUMBER == expected
|
||||
for word in ("sheep", "aircraft", "means", "offspring", "species", "series"):
|
||||
assert word in lexicon.INVARIANT_NUMBER
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
|
|
@ -312,41 +323,110 @@ def test_reader_number_table_is_a_derived_subset_that_cannot_drift() -> None:
|
|||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("plural", "current_wrong_singular"),
|
||||
("plural", "singular"),
|
||||
[
|
||||
("wolves", "wolve"),
|
||||
("leaves", "leave"),
|
||||
("knives", "knive"),
|
||||
("news", "new"),
|
||||
("species", "specy"),
|
||||
# Phase 2A pinned these as WRONG (wolve / leave / knive). Phase 2B
|
||||
# flipped them by routing the reader through the shared 29-entry
|
||||
# singularizer.
|
||||
("wolves", "wolf"),
|
||||
("leaves", "leaf"),
|
||||
("knives", "knife"),
|
||||
("halves", "half"),
|
||||
("thieves", "thief"),
|
||||
("children", "child"),
|
||||
("men", "man"),
|
||||
# Genuinely inflected irregulars the reader could not read before.
|
||||
("cacti", "cactus"),
|
||||
("fungi", "fungus"),
|
||||
("oxen", "ox"),
|
||||
],
|
||||
)
|
||||
def test_reader_silently_mis_singularizes_uncovered_plurals(
|
||||
plural: str, current_wrong_singular: str
|
||||
) -> None:
|
||||
"""CURRENT BEHAVIOUR, deliberately pinned as wrong.
|
||||
def test_reader_singularizes_irregulars_correctly(plural: str, singular: str) -> None:
|
||||
"""The defect Phase 2A pinned, now fixed.
|
||||
|
||||
``reader.py``'s comment claims an unrecognized plural "REFUSES rather
|
||||
than guessing a wrong singular (wrong=0)". The code does not implement
|
||||
that: ``_singularize`` falls through to a bare ``-s`` strip, so uncovered
|
||||
plurals mint corrupted entity ids. ``news`` -> ``new`` and ``species`` ->
|
||||
``specy`` are exactly the corruptions ``member.py``'s table comment says
|
||||
its table exists to prevent.
|
||||
|
||||
Phase 2B replaces the reader's 8-key view with the full singularizer and
|
||||
must flip these to ``wolf``/``leaf``/``knife``/``news``/``species``.
|
||||
Changing minted ids moves trace hashes, which is why it is gated.
|
||||
``reader.py``'s comment always claimed an unrecognized plural "REFUSES
|
||||
rather than guessing a wrong singular (wrong=0)". Until 2B the code did
|
||||
not implement it — ``_singularize`` fell through to a bare ``-s`` strip
|
||||
and minted corrupted ids that reached served text ("all wolve are
|
||||
mammal"). The comment is now true of the code.
|
||||
"""
|
||||
from generate.meaning_graph.reader import _singularize
|
||||
|
||||
assert _singularize(plural) == current_wrong_singular
|
||||
assert _singularize(plural) == singular
|
||||
|
||||
|
||||
def test_reader_covered_plurals_are_already_correct() -> None:
|
||||
"""The control for the test above: where the reader HAS an entry it is
|
||||
right. So the defect is coverage, not wrong values — which is why 2A
|
||||
could derive the values safely and leave coverage to 2B."""
|
||||
@pytest.mark.parametrize(
|
||||
"invariant", ["fish", "sheep", "deer", "species", "series", "news", "means"]
|
||||
)
|
||||
def test_reader_declines_number_invariant_forms(invariant: str) -> None:
|
||||
"""Invariants are AMBIGUOUS in number and must be declined, not resolved.
|
||||
|
||||
"fish are mammals" is plural; "a fish is a mammal" is singular; the token
|
||||
cannot tell you which. Two independent reasons this must decline:
|
||||
|
||||
1. **Honesty** — resolving it is a guess about number, and 2A measured what
|
||||
guessing costs: the old ``-s`` strip turned ``news`` into ``new`` and
|
||||
``species`` into ``specy``.
|
||||
2. **Soundness** — the serving composer tries the categorical band (v1b)
|
||||
first. Resolving an invariant makes v1b *accept* a sentence it cannot
|
||||
decide, stealing the case from a band that can. Measured: resolving
|
||||
``fish`` made ds-ex-0012 ("No fish are mammals. Therefore some fish are
|
||||
mammals.") answer ``invalid`` instead of ``refuted`` — **wrong=1 on a
|
||||
ratified band.** Declining restores the fall-through.
|
||||
"""
|
||||
from generate.meaning_graph.reader import _singularize
|
||||
|
||||
for plural, singular in lexicon.READER_IRREGULAR_SINGULARS.items():
|
||||
assert _singularize(plural) == singular
|
||||
assert _singularize(invariant) is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("not_a_plural", ["child", "evidence", "wolf", "", "ss"])
|
||||
def test_reader_declines_rather_than_guessing(not_a_plural: str) -> None:
|
||||
"""The other half of wrong=0: a singular, a mass noun, or anything the
|
||||
closed rules do not confidently cover returns ``None`` so the caller can
|
||||
refuse instead of minting a corrupted id. Without this, ``child`` would
|
||||
become ``chil``."""
|
||||
from generate.meaning_graph.reader import _singularize
|
||||
|
||||
assert _singularize(not_a_plural) is None
|
||||
|
||||
|
||||
def test_regular_plurals_still_singularize() -> None:
|
||||
"""Control: widening the irregular table must not break the regular rule
|
||||
that handles the overwhelming majority of real input."""
|
||||
from generate.meaning_graph.reader import _singularize
|
||||
|
||||
assert _singularize("cars") == "car"
|
||||
assert _singularize("glasses") == "glass"
|
||||
assert _singularize("cities") == "city"
|
||||
|
||||
|
||||
def test_the_two_number_directions_are_mutual_inverses() -> None:
|
||||
"""Every irregular the singularizer knows, the pluralizer can produce.
|
||||
|
||||
This is the law that makes read/write agreement possible at all: if CORE
|
||||
can read ``cacti`` but writes ``cactuses``, no round trip can close. Phase
|
||||
2B added the three missing inverses (cactus/fungus/die) and derived
|
||||
:data:`lexicon.INVARIANT_NUMBER` from the singularizer's own invariant
|
||||
rows, so ``aircraft``/``means``/``offspring`` stop becoming "aircrafts",
|
||||
"meanses", "offsprings".
|
||||
"""
|
||||
from generate.morphology import pluralize
|
||||
|
||||
broken = {
|
||||
singular: (pluralize(singular), plural)
|
||||
for plural, singular in lexicon.IRREGULAR_SINGULARS.items()
|
||||
if pluralize(singular) != plural
|
||||
}
|
||||
assert not broken, f"singular -> plural does not round-trip: {broken}"
|
||||
|
||||
|
||||
def test_mass_nouns_and_compounds_inflect_correctly() -> None:
|
||||
"""Two cases the categorical renderer depends on: mass nouns must not take
|
||||
a plural ("all evidence", not "all evidences"), and an English compound
|
||||
inflects on its HEAD ("guard dogs", not "guards dog")."""
|
||||
from generate.morphology import pluralize
|
||||
|
||||
assert pluralize("evidence") == "evidence"
|
||||
assert pluralize("knowledge") == "knowledge"
|
||||
assert pluralize("guard_dog") == "guard_dogs"
|
||||
assert pluralize("guard dog") == "guard dogs"
|
||||
|
|
|
|||
Loading…
Reference in a new issue