Commit graph

9 commits

Author SHA1 Message Date
Shay
997a6eb043 fix(generate): plural-subject agreement, 26/26 (Phase 3)
_inflect_predicate applied base_form — a SINGLE-VERB function — to whole
humanized predicate phrases, stripping the final word's last character
class, and its plural branch never consulted `copular`. Nine of the 26 seed
predicates came out wrong and every multi-word one did:

  is defined as         -> "is defined a"     want "are defined as"
  has the following ... -> "...step"          want "have the following steps"
  belongs to            -> "belongs to"       want "belong to"
  causes                -> "caus"             want "cause"

Root causes, both single-point as §1.4 predicted:

1. morphology.agree_plural_phrase inflects the HEAD (the finite verb is the
   first token of every humanized predicate) and carries the rest through.
   base_form stays single-verb and is PINNED as still wrong on a phrase, so
   nobody "fixes" the symptom in the wrong place.
2. The plural branch now agrees the phrase. The plural NEGATED branch had the
   same defect ("do not is defined a") and is fixed with it: a plural copula
   takes a bare "not", everything else takes do-support.

Two further defects found while writing the eval cases, both fixed here:

  base_form("causes") -> "caus". The -es sibilant rule fired on a stem
  ending in a single "s"; it must require a doubled "ss" ("passes"->"pass"),
  because a single "s" is nearly always a stem ending in "e" that took a
  plain -s ("causes"->"cause").

  pluralize("proof") -> "prooves". f/fe -> ves is NOT productive in English
  (proof->proofs, chief->chiefs, roof->roofs); it is a closed set. Now
  derived as lexicon.VES_PLURAL_SINGULARS from the ves-rows of
  IRREGULAR_SINGULARS, so the rule cannot claim a word the table does not
  know. Phase 2B had put pluralize on the SERVING path, so this one was live
  — and the surface-hashing pin from #133 confirms no served surface moved,
  which is that guard's first real use.

New construction C14 quantified_copular, 13 cases: the combination that was
broken was the one never tested. Measured before: 152 corpus cases carry a
quantifier and ZERO combine it with a copular predicate.

MY FIRST DRAFT OF THOSE CASES COULD NOT FAIL. must_contain/word_order listed
quantifier, subject and object but NOT THE VERB, so "all molecules is
defined a compound" passed and reverting the fix left 47/47 green. Rewritten
with the agreed verb in both constraints plus reject_surfaces carrying the
ACTUAL pre-fix output, computed by running the reverted code rather than
guessed. Mutation now:

  baseline                      13/13 C14
  revert phrase-head agreement   4/13
  revert -es stem rule          12/13
  revert closed ves set         11/13

The 4 survivors of the first mutation are the mass-noun controls, which is
correct — "all evidence is grounded in truth" must NOT pluralize.

Same lesson as the lane pins in #133: a pin that cannot fail guards nothing,
and the only way to know is to break the thing on purpose.

grammar_roundtrip's graph corpus is harvested from the committed case files,
so it grew 280 -> 293. Updated exactly rather than loosened to an inequality:
a corpus that grows or shrinks should require a deliberate edit.

[Verification]: in-worktree on CPython 3.12.13, uv sync --locked —
agreement 26/26 (from 17/26); english_fluency_ood 117/117 + 39/39 + dev
13/13 unchanged; grammatical_coverage v1 49/49; smoke 621 unchanged;
deductive 405; scripts/verify_lane_shas.py 11/11, no pin edited.
2026-07-27 10:11:22 -07:00
Shay
5ba1359a79 fix(generate): CORE writes plural nouns in categorical clauses (Phase 2B)
The ratified, flag-ON v1b categorical band served "all dog are mammal" for
every noun: the A/E/I/O templates supply all/no/some + are, which demand a
plural, and the slots were filled with singular canonical entity ids. Four
of the 47 ratified corpus cases were affected, and wrong=0 never noticed
because every verdict was correct — only the prose was wrong.

Two fixes that compose into a closed round trip:

1. generate/morphology.py becomes the single owner of the number RULES
   (Phase 2A gave the TABLES one owner). Both directions live there:
   pluralize + singularize, over lexicon.IRREGULAR_PLURALS /
   IRREGULAR_SINGULARS, with mass-noun and compound-head handling.
   templates.pluralize and reader._singularize were two independent copies
   of the regular suffix rules that disagreed about which irregulars they
   knew; both now delegate.

2. render._display_noun re-inflects at render time and swaps _ for a space,
   so compound ids read "guard dogs" rather than "guards dog" or
   "guard_dog".

Result: reader gives wolves -> wolf, renderer gives wolf -> wolves. The
round trip closes.

  0 malformed of 47 (from 4)
  reader-vs-reader singularization 20/20 (from 8/20), 0 silently wrong
  s_surface_match_rate 0.0 -> 0.625, now EQUAL to s_renderable_rate, so the
    only remaining losses are surfaces the renderer cannot express at all

NEAR-MISS — widening a reader broke soundness, caught by the lane:

Routing the reader through the full 29-entry table first produced wrong=1 on
band v6-EX. ds-ex-0012 ("No fish are mammals. Therefore some fish are
mammals.") answered invalid where gold is refuted.

_singularize returning None makes the reader REFUSE, and the serving
composer tries the categorical band FIRST, falling through to more capable
bands. "fish" previously returned None (it matches no suffix rule), so v6-EX
got the case and decided it correctly. Resolving "fish" made v1b accept a
sentence it cannot decide.

Fixed by a principle, not a patch: a number-INVARIANT form is ambiguous in
number — "fish are mammals" is plural, "a fish is a mammal" is singular, and
the token cannot tell you which — so a reader that must not guess number
declines it. lexicon.INVARIANT_NUMBER is derived from the singularizer's own
key == value rows, so the rule cannot drift from the table.

  => Coverage and correctness are different axes. Fixing a corrupted VALUE is
     safe; widening ACCEPTANCE changes which band answers, and in a
     first-match composer that turns a right answer into a wrong one. Same
     family as ADR-0261 5.1 refuse-don't-drop.

Also fixed, found by testing the inverse law: the two number tables were not
mutual inverses. The pluralizer lacked cactus->cacti, fungus->fungi,
die->dice and the invariants aircraft/means/offspring, so CORE could read
"cacti" but wrote "cactuses", and would have written "aircrafts", "meanses",
"offsprings". No round trip can close across a non-invertible table.

THE LANE SHA PINS ARE BLIND TO SERVED ENGLISH:

2B changed 4 served surfaces and the pins came back 11/11 byte-identical.
The deduction_serve_v1 hashed report holds only n/counts/by_gold/
correct_by_gold/all_cases_correct/mismatch_examples — no prose at all.
Confirmed by sabotage: with _display_noun returning "SABOTAGE_" + ..., so
every clause reads "all SABOTAGE_dogs are SABOTAGE_animals",

  11 lane SHA pins                        -> 11/11 byte-identical, blind
  test_deduction_serve_lane + _license    -> 20 passed, blind
  Phase 1 grammar_roundtrip               -> 3 tests RED, caught it

This corrects #129: byte-identity of the pins is the arbiter for values and
verdicts, NOT for surface text, so the 2A/2B split is not a partition of
"changes users can see." Phase 2A's 11/11 conclusion still stands on its own
independent evidence (24/24 table-equality checks vs the pre-migration
literals), but the justification was thinner than stated.

Before Phase 1 no test in the tree would have noticed CORE's served prose
turning into word salad. That is exactly how "all dog are mammal" survived
on a ratified flag-ON band with wrong=0 intact.

Deliberately not fixed: mass-noun verb agreement ("all evidence ARE truth"
should be "is"). The copula is fixed text inside the templates, so agreeing
it changes the template shape rather than a slot, and no mass noun reaches a
categorical clause in any serve corpus. Recorded in render.py.

No lane pin edited — none moved.

[Verification]: in-worktree on CPython 3.12.13, uv sync --locked —
smoke 621 unchanged; deductive 403 passed (383 + 20 new/rewritten);
scripts/verify_lane_shas.py 11/11 (see blindness note above — gate on
grammar_roundtrip for surface work, not on these pins).
2026-07-26 18:42:42 -07:00
Shay
df6332fcb5 refactor(generate): one owner per linguistic fact (Phase 2A)
Collapses the duplicated closed English tables into generate/lexicon.py.
Measured on main @ 0948f7cd: the same facts were encoded up to five times
across the reading and writing paths, each copy written independently as a
successive deduction band landed.

The 2A/2B split is decided empirically per #129 — make the change, run the
11 pinned lanes, let byte-identity rule. Result: 11/11 byte-identical, so
this entire unit is 2A and nothing spilled into the authorization-gated 2B.

Collapsed to one object:
  - connectives    3 identical copies (member, verb, cond_member)
  - predicate display  2 identical 26-entry copies (semantic_templates,
                       templates)
  - be-form inventory  5 copies, not the 2 §1.3 counted. The structural
                       test found realizer_guard._BE_AUX and
                       chat/runtime._BE_FORMS, which a COPULA-shaped grep
                       could never see.

Derived rather than duplicated, so they can no longer drift:
  - STRUCTURAL = CONNECTIVES | {therefore}
  - QUANTIFIER_TOKENS = QUANTIFIER_LEAD | QUANTIFIER_NON_LEAD
  - NEGATION_BEARING_WITH_NOT = NEGATION_BEARING | {not}
  - READER_IRREGULAR_SINGULARS = IRREGULAR_SINGULARS restricted to 8 keys

§1.3 re-measured while doing this, and it overstated the disease (plan
§1.9). Almost nothing contradicts:
  - the "3 divergent plural tables" are 1 pluralizer + 2 singularizers,
    i.e. inverse directions; comparing them as copies is a category error
  - reader's 8 singulars are a STRICT SUBSET of member's 29 with the
    difference in reader's favour empty, so its values are all correct and
    only its coverage is short
  - the "3 divergent quantifier sets" are 3 distinct facts; every/each lead
    a clause but take singular nouns, so their absence from
    PLURAL_QUANTIFIERS is correct
  - english's negation set is a subset of member's, and the difference is
    principled: v2-EN normalizes "<copula> not", v3-MEM refuses it

Exactly one genuine contradiction exists in the whole inventory:
discourse_planner maps is_defined_as -> "is" where the others map it to
"is defined as". Preserved as a register choice, pinned by test.

Found while reading, pinned as current behaviour for 2B to flip:
reader._singularize does NOT refuse uncovered plurals despite a comment
claiming it does — it falls through to a bare -s strip, so news -> new and
species -> specy, exactly the corruptions member.py's table comment says
its table exists to prevent.

Two 2A exit criteria were unmeasurable as written and are replaced:
  - "Jaccard -> 1.00" cannot work. measure_grammar_seam.py scans source
    literals, so unifying a fact removes it from those files and Jaccard
    FELL, 0.083 -> 0.023 — the success direction reported by a metric
    shaped to read like failure. Replaced with an object-identity count:
    3 distinct objects behind 8 names, from 8-behind-8. Value comparison
    cannot tell a shared object from two equal copies.
  - "one table per fact" presumed §1.3's inventory was right. The report now
    separates must-be-one-object (all UNIFIED) from related-but-distinct
    (6 relations, all HOLD).

ADR-0258 §5 already decided morphology's destination is a ratified pack,
triggered by a grc/he member band. No such band exists, so the promotion is
correctly deferred and lexicon.py is a staging area, not a rival home — no
new ADR. Destination measured for whoever picks it up: packs/en/morphology
.jsonl has 9 records and zero irregular plurals, features.number has no
consumer, and _pack_morph_roots_for reads a top-level "root" key that 0 of
31 records across all four packs define, so it returns {} every call.

[Verification]: in-worktree on CPython 3.12.13, uv sync --locked —
smoke 621 unchanged; deductive 383 (364 + 19 new);
scripts/verify_lane_shas.py 11/11 byte-identical.
2026-07-26 17:45:27 -07:00
Shay
6761fc0974 feat(realizer): C1.5 — articulation legality at the realizer boundary
Adds a typed legality check that catches a narrow class of incoherent
finite-predicate surfaces before they ship.  Scope is deliberately
narrow:

  - generate/articulation_legality.py:
    - SlotKind enum {VERB, NON_VERB, UNKNOWN}
    - ArticulationLegality enum {LEGAL, ILLEGAL_NON_VERB_FINITE_PREDICATE}
    - classify_predicate_slot_kind() — token allowlists for known verbs
      and known non-verb nouns
    - validate_finite_predicate_legality() — fails on negated +
      NON_VERB; fail-open on UNKNOWN to preserve canary behavior

  - generate/templates.py:
    - _inflect_predicate: copular-aware negation
      ("is X" -> "is not X" instead of the default "does not be X")
    - render_step: invokes the legality validator; returns
      "I cannot realize that proposition coherently yet." when an
      illegal shape is detected

The check is upstream of register / anchor-lens transforms (presentation
+ substantive axes both downstream of the realizer); no interaction
with R6 / ADR-0073 layering.

Tests pin:
  - NON_VERB + negated -> ILLEGAL_NON_VERB_FINITE_PREDICATE
  - UNKNOWN + negated -> LEGAL (fail-open preserved)
  - render_step returns the disclosure string when illegal detected
  - render_step still produces the fall-through surface on UNKNOWN

Validation:
  - Cognition eval byte-identical (100/100/91.7/100)
  - 370 realizer / lens / register / pack / lane tests pass
  - anchor-lens-tour + register-tour both green
2026-05-20 11:11:28 -07:00
Shay
3952da11bc fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour
Six contained-tier gap closures from the cross-phase gap audit.
Every gaps.md file involved gains a resolution block; the historical
findings are preserved for traceability.

Identity (teaching/review.py)
- _ROLE_FRAMES gains the philosophical-axis family (axiology, ontology,
  telos, ethos, epistemology, morality, ethics, virtues, norms,
  doctrine, creed, convictions, beliefs, values, principles).
- _REDIRECT_VERBS gains the dismissal family (disregard, dismiss,
  bypass, circumvent, renounce, reject, repudiate).
- Closes multi_agent_composition holdout MAC-HOLD-001 ("disregard your
  axiology") and the matching adversarial_identity gap.
- Multi-agent holdouts: 8/8 attacks rejected, 3/3 legits accepted.

Pipeline (core/cognition/pipeline.py + docs/runtime_contracts.md)
- When the unknown-domain gate fires, ChatRuntime returns the
  "I don't have field coordinates for that yet." stub and
  vault_hits == 0.  The pipeline now honours that stub as the
  user-facing surface instead of overriding with the realizer's
  fallback articulation.  walk_surface is unchanged either way.
- New contract test
  tests/test_semantic_realizer_integration.py::test_pipeline_honours_safety_stub_when_gate_fires
  locks the contract; the existing semantic-surface test now primes
  the vault first so the gate doesn't fire on the probe.
- Closes calibration gaps.md Finding 2.

Realizer morphology (generate/morphology.py)
- G1: ~100-entry irregular-verb table replaces the previous list which
  contained only regular forms.  Includes bind→bound, run→ran,
  stand→stood, write→wrote/written, eat→ate/eaten, fly→flew/flown,
  swim→swam/swum, etc.
- CVC doubling rule for -ed and -ing (stop→stopped/stopping,
  plan→planned, run→running).
- Short-ies disambiguation (die/lie/tie keep -ie- in the base; cry/fly
  collapse to -y).  Lie is also irregular (lay/lain) — uses
  _IRREGULAR_FORMS first.
- 28-case regression test (tests/test_morphology_irregular.py).

Realizer plural agreement (generate/templates.py)
- G2: under universal/existential/many/few/most quantifiers, count-noun
  subjects pluralise (molecule → molecules) and the verb de-conjugates
  (binds → bind).  Negation toggles does-not → do-not.  Aspect toggles
  has → have, is → are.  All other constructions unchanged.
- Mass nouns (evidence, wisdom, knowledge, truth, water, …) stay
  singular under quantifiers — "all evidence supports truth" is right;
  "all evidences support" would be wrong English.
- 17-case regression test
  (tests/test_realizer_quantifier_agreement.py) covering count vs mass,
  irregular plurals (child→children, analysis→analyses), and the
  quantifier-tense / quantifier-aspect / quantifier-negation grid.

Rubric punctuation tolerance (evals/grammatical_coverage/runner.py)
- G3: _check_word_order strips trailing/leading punctuation
  (.,;:!?—–) before exact-word comparison so "river," still satisfies
  word_order=["river"].  must_contain also accepts punctuation-
  stripped token matches.
- Affects every lane that uses grammatical_coverage scoring; the OOD
  case generators no longer need to pin punctuated accept_surfaces for
  C06.

Case generator + lane regeneration
- scripts/generate_english_fluency_ood.py uses generate.templates.pluralize
  for C07/C08 must_contain + word_order so case-side constraints stay
  aligned with the (more correct) realizer.
- All Phase 5 OOD lane cases (5.1, 5.4–5.7) regenerated; results files
  re-scored.

CLI (core/cli.py)
- cmd_eval no longer crashes on lanes whose case_details use "id"
  instead of "case_id" (adversarial_identity, multi_agent_composition).
- Cognition CLI lane gains the two new morphology/quantifier
  regression test files.

Lane sweep (all 100%, no regression):
  english_fluency_ood              117/117 public + 39/39 holdouts
  elementary_mathematics_ood       117/117 + 39/39
  foundational_physics_ood         117/117 + 39/39
  foundational_biology_ood         117/117 + 39/39
  classical_literature_ood         117/117 + 39/39
  grammatical_coverage             back to 100% on its own seed cases
  hebrew_fluency / koine_greek_fluency  3/3 each

CLI lane health:
  smoke 54, runtime 19, teaching 17, packs 6, cognition 103 (was 57),
  algebra 132.
2026-05-16 21:21:06 -07:00
Shay
fa2712ebd7 feat(realizer): extend to all 13 English v1 constructions
Engineer the deterministic realizer to handle negation, conjunction,
disjunction, embedded clauses, relative clauses, quantification, tense,
and aspect — covering all 13 grammatical-coverage v1 constructions.

- generate/morphology.py: rule-based English inflection (past, participle,
  base form) for seed vocabulary predicates
- generate/templates.py: match-case inflection dispatch for tense/aspect/negation
- generate/graph_planner.py: add CONJUNCTION, DISJUNCTION, COMPLEMENT, RELATIVE
  relations; add grammatical feature fields to ArticulationStep
- generate/realizer.py: compound construction handling via graph edge traversal

grammatical-coverage eval: dev=100%, public v1=100% (from baseline of 24%/19%).
2026-05-16 05:55:49 -07:00
Shay
523c072818 feat: vault recall index, Rust versor parity, cognitive pack expansion
Phase 3 — vault exact recall index:
- Replace O(N) np.array_equal scan with hash-based exact-match index
- Add optional max_entries with deterministic FIFO eviction
- Index rebuilds on reproject for consistency

Phase 4 — Rust versor_apply parity:
- Fix CGA metric signature (+,+,+,+,-) and blade ordering to match Python
- Implement versor_apply_closed with null-vector preservation, f64 unitize,
  and construction seed fallback matching Python closure semantics
- Gate Rust dispatch behind CORE_BACKEND=rust; Python remains default
- Add f64 geometric product for closure-path precision

Phase 5 — cognitive quality pack expansion:
- Expand lexicon from 55 to 70 entries (evidence, inference, procedure,
  verification, distinction, relation, thought, understanding, judgment,
  principle, order, connectives)
- Improve semantic templates for cause, procedure, comparison, recall,
  verification intents
- Expand eval cases from 20 to 45 across all categories

Validation: 491 tests pass, 45 eval cases at 100% all metrics.
2026-05-15 15:34:39 -07:00
Shay
a7febd48ef
Integrate semantic realizer into cognition pipeline
- add intent-aware semantic templates for seed-pack relation predicates
- add semantic realization path for ArticulationTarget outputs
- wire semantic realization into CognitiveTurnPipeline results without changing ChatRuntime.chat
- expand cognition CLI suite coverage for semantic realizer integration
- add focused tests for deterministic semantic surfaces and response contract stability
2026-05-15 07:08:37 -07:00
Shay
58a06124bf
Add articulation realizer v2
- add deterministic ArticulationTarget realizer
- add rhetorical move templates and predicate humanization
- handle definition, comparison, correction, unknown, and empty targets
- keep runtime ChatResponse path unchanged
- add focused realizer tests
2026-05-14 20:14:50 -07:00