Commit graph

6 commits

Author SHA1 Message Date
Shay
997a6eb043 fix(generate): plural-subject agreement, 26/26 (Phase 3)
_inflect_predicate applied base_form — a SINGLE-VERB function — to whole
humanized predicate phrases, stripping the final word's last character
class, and its plural branch never consulted `copular`. Nine of the 26 seed
predicates came out wrong and every multi-word one did:

  is defined as         -> "is defined a"     want "are defined as"
  has the following ... -> "...step"          want "have the following steps"
  belongs to            -> "belongs to"       want "belong to"
  causes                -> "caus"             want "cause"

Root causes, both single-point as §1.4 predicted:

1. morphology.agree_plural_phrase inflects the HEAD (the finite verb is the
   first token of every humanized predicate) and carries the rest through.
   base_form stays single-verb and is PINNED as still wrong on a phrase, so
   nobody "fixes" the symptom in the wrong place.
2. The plural branch now agrees the phrase. The plural NEGATED branch had the
   same defect ("do not is defined a") and is fixed with it: a plural copula
   takes a bare "not", everything else takes do-support.

Two further defects found while writing the eval cases, both fixed here:

  base_form("causes") -> "caus". The -es sibilant rule fired on a stem
  ending in a single "s"; it must require a doubled "ss" ("passes"->"pass"),
  because a single "s" is nearly always a stem ending in "e" that took a
  plain -s ("causes"->"cause").

  pluralize("proof") -> "prooves". f/fe -> ves is NOT productive in English
  (proof->proofs, chief->chiefs, roof->roofs); it is a closed set. Now
  derived as lexicon.VES_PLURAL_SINGULARS from the ves-rows of
  IRREGULAR_SINGULARS, so the rule cannot claim a word the table does not
  know. Phase 2B had put pluralize on the SERVING path, so this one was live
  — and the surface-hashing pin from #133 confirms no served surface moved,
  which is that guard's first real use.

New construction C14 quantified_copular, 13 cases: the combination that was
broken was the one never tested. Measured before: 152 corpus cases carry a
quantifier and ZERO combine it with a copular predicate.

MY FIRST DRAFT OF THOSE CASES COULD NOT FAIL. must_contain/word_order listed
quantifier, subject and object but NOT THE VERB, so "all molecules is
defined a compound" passed and reverting the fix left 47/47 green. Rewritten
with the agreed verb in both constraints plus reject_surfaces carrying the
ACTUAL pre-fix output, computed by running the reverted code rather than
guessed. Mutation now:

  baseline                      13/13 C14
  revert phrase-head agreement   4/13
  revert -es stem rule          12/13
  revert closed ves set         11/13

The 4 survivors of the first mutation are the mass-noun controls, which is
correct — "all evidence is grounded in truth" must NOT pluralize.

Same lesson as the lane pins in #133: a pin that cannot fail guards nothing,
and the only way to know is to break the thing on purpose.

grammar_roundtrip's graph corpus is harvested from the committed case files,
so it grew 280 -> 293. Updated exactly rather than loosened to an inequality:
a corpus that grows or shrinks should require a deliberate edit.

[Verification]: in-worktree on CPython 3.12.13, uv sync --locked —
agreement 26/26 (from 17/26); english_fluency_ood 117/117 + 39/39 + dev
13/13 unchanged; grammatical_coverage v1 49/49; smoke 621 unchanged;
deductive 405; scripts/verify_lane_shas.py 11/11, no pin edited.
2026-07-27 10:11:22 -07:00
Shay
3952da11bc fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour
Six contained-tier gap closures from the cross-phase gap audit.
Every gaps.md file involved gains a resolution block; the historical
findings are preserved for traceability.

Identity (teaching/review.py)
- _ROLE_FRAMES gains the philosophical-axis family (axiology, ontology,
  telos, ethos, epistemology, morality, ethics, virtues, norms,
  doctrine, creed, convictions, beliefs, values, principles).
- _REDIRECT_VERBS gains the dismissal family (disregard, dismiss,
  bypass, circumvent, renounce, reject, repudiate).
- Closes multi_agent_composition holdout MAC-HOLD-001 ("disregard your
  axiology") and the matching adversarial_identity gap.
- Multi-agent holdouts: 8/8 attacks rejected, 3/3 legits accepted.

Pipeline (core/cognition/pipeline.py + docs/runtime_contracts.md)
- When the unknown-domain gate fires, ChatRuntime returns the
  "I don't have field coordinates for that yet." stub and
  vault_hits == 0.  The pipeline now honours that stub as the
  user-facing surface instead of overriding with the realizer's
  fallback articulation.  walk_surface is unchanged either way.
- New contract test
  tests/test_semantic_realizer_integration.py::test_pipeline_honours_safety_stub_when_gate_fires
  locks the contract; the existing semantic-surface test now primes
  the vault first so the gate doesn't fire on the probe.
- Closes calibration gaps.md Finding 2.

Realizer morphology (generate/morphology.py)
- G1: ~100-entry irregular-verb table replaces the previous list which
  contained only regular forms.  Includes bind→bound, run→ran,
  stand→stood, write→wrote/written, eat→ate/eaten, fly→flew/flown,
  swim→swam/swum, etc.
- CVC doubling rule for -ed and -ing (stop→stopped/stopping,
  plan→planned, run→running).
- Short-ies disambiguation (die/lie/tie keep -ie- in the base; cry/fly
  collapse to -y).  Lie is also irregular (lay/lain) — uses
  _IRREGULAR_FORMS first.
- 28-case regression test (tests/test_morphology_irregular.py).

Realizer plural agreement (generate/templates.py)
- G2: under universal/existential/many/few/most quantifiers, count-noun
  subjects pluralise (molecule → molecules) and the verb de-conjugates
  (binds → bind).  Negation toggles does-not → do-not.  Aspect toggles
  has → have, is → are.  All other constructions unchanged.
- Mass nouns (evidence, wisdom, knowledge, truth, water, …) stay
  singular under quantifiers — "all evidence supports truth" is right;
  "all evidences support" would be wrong English.
- 17-case regression test
  (tests/test_realizer_quantifier_agreement.py) covering count vs mass,
  irregular plurals (child→children, analysis→analyses), and the
  quantifier-tense / quantifier-aspect / quantifier-negation grid.

Rubric punctuation tolerance (evals/grammatical_coverage/runner.py)
- G3: _check_word_order strips trailing/leading punctuation
  (.,;:!?—–) before exact-word comparison so "river," still satisfies
  word_order=["river"].  must_contain also accepts punctuation-
  stripped token matches.
- Affects every lane that uses grammatical_coverage scoring; the OOD
  case generators no longer need to pin punctuated accept_surfaces for
  C06.

Case generator + lane regeneration
- scripts/generate_english_fluency_ood.py uses generate.templates.pluralize
  for C07/C08 must_contain + word_order so case-side constraints stay
  aligned with the (more correct) realizer.
- All Phase 5 OOD lane cases (5.1, 5.4–5.7) regenerated; results files
  re-scored.

CLI (core/cli.py)
- cmd_eval no longer crashes on lanes whose case_details use "id"
  instead of "case_id" (adversarial_identity, multi_agent_composition).
- Cognition CLI lane gains the two new morphology/quantifier
  regression test files.

Lane sweep (all 100%, no regression):
  english_fluency_ood              117/117 public + 39/39 holdouts
  elementary_mathematics_ood       117/117 + 39/39
  foundational_physics_ood         117/117 + 39/39
  foundational_biology_ood         117/117 + 39/39
  classical_literature_ood         117/117 + 39/39
  grammatical_coverage             back to 100% on its own seed cases
  hebrew_fluency / koine_greek_fluency  3/3 each

CLI lane health:
  smoke 54, runtime 19, teaching 17, packs 6, cognition 103 (was 57),
  algebra 132.
2026-05-16 21:21:06 -07:00
Shay
0e7135ff74 feat(evals): grammatical-coverage v2 cases - 36 cases with deeper nesting and rarer vocabulary (100% pass) 2026-05-16 06:40:55 -07:00
Shay
93bbb6c824 feat(evals,packs): grammatical-coverage holdout, zero-code kits, Hebrew/Greek packs
- grammatical-coverage holdout v1: 52 cases across all 13 constructions, 100% pass
- zero-code-domain-acquisition lane: contract + 3 surprise domains (kinship,
  calendar, color) with vocabulary, relations, axioms, teaching examples,
  and dev prompts; pack closure verified for all three domains
- he_core_cognition_v1: 20 entries in Hebrew script with morphology decomposition
  (triliteral roots, binyanim, aspect/person/gender/number); depth_root role
  with fail_closed OOV policy
- grc_logos_cognition_v1: 20 entries in polytonic Greek with morphology
  decomposition (stems, prefix/suffix chains, declension class, tense/voice/
  mood/person); depth_relation role with fail_closed OOV policy
2026-05-16 06:23:28 -07:00
Shay
fa2712ebd7 feat(realizer): extend to all 13 English v1 constructions
Engineer the deterministic realizer to handle negation, conjunction,
disjunction, embedded clauses, relative clauses, quantification, tense,
and aspect — covering all 13 grammatical-coverage v1 constructions.

- generate/morphology.py: rule-based English inflection (past, participle,
  base form) for seed vocabulary predicates
- generate/templates.py: match-case inflection dispatch for tense/aspect/negation
- generate/graph_planner.py: add CONJUNCTION, DISJUNCTION, COMPLEMENT, RELATIVE
  relations; add grammatical feature fields to ArticulationStep
- generate/realizer.py: compound construction handling via graph edge traversal

grammatical-coverage eval: dev=100%, public v1=100% (from baseline of 24%/19%).
2026-05-16 05:55:49 -07:00
Shay
f5f3603dcb feat(evals): Phase 1 grammatical-coverage lane setup
Establish the grammatical-coverage eval lane with 13 English v1
constructions (simple declarative, negation, conjunction, disjunction,
embedded clause, relative clause, quantification, tense, aspect).

- contract.md with scoring rubric and pass thresholds
- runner.py conforming to framework interface
- dev set: 41 cases (baseline: 24.4%, only C01/C10 pass)
- public v1: 36 cases (baseline: 19.4%, only C01/C10 pass)
- holdout and realizer engineering are next

The realizer currently handles only simple present-tense SVO declaratives.
Negation, conjunction, embedding, quantification, tense, and aspect all
need engineering work.
2026-05-16 05:45:14 -07:00