3952da11bc
Six contained-tier gap closures from the cross-phase gap audit.
Every gaps.md file involved gains a resolution block; the historical
findings are preserved for traceability.
Identity (teaching/review.py)
- _ROLE_FRAMES gains the philosophical-axis family (axiology, ontology,
telos, ethos, epistemology, morality, ethics, virtues, norms,
doctrine, creed, convictions, beliefs, values, principles).
- _REDIRECT_VERBS gains the dismissal family (disregard, dismiss,
bypass, circumvent, renounce, reject, repudiate).
- Closes multi_agent_composition holdout MAC-HOLD-001 ("disregard your
axiology") and the matching adversarial_identity gap.
- Multi-agent holdouts: 8/8 attacks rejected, 3/3 legits accepted.
Pipeline (core/cognition/pipeline.py + docs/runtime_contracts.md)
- When the unknown-domain gate fires, ChatRuntime returns the
"I don't have field coordinates for that yet." stub and
vault_hits == 0. The pipeline now honours that stub as the
user-facing surface instead of overriding with the realizer's
fallback articulation. walk_surface is unchanged either way.
- New contract test
tests/test_semantic_realizer_integration.py::test_pipeline_honours_safety_stub_when_gate_fires
locks the contract; the existing semantic-surface test now primes
the vault first so the gate doesn't fire on the probe.
- Closes calibration gaps.md Finding 2.
Realizer morphology (generate/morphology.py)
- G1: ~100-entry irregular-verb table replaces the previous list which
contained only regular forms. Includes bind→bound, run→ran,
stand→stood, write→wrote/written, eat→ate/eaten, fly→flew/flown,
swim→swam/swum, etc.
- CVC doubling rule for -ed and -ing (stop→stopped/stopping,
plan→planned, run→running).
- Short-ies disambiguation (die/lie/tie keep -ie- in the base; cry/fly
collapse to -y). Lie is also irregular (lay/lain) — uses
_IRREGULAR_FORMS first.
- 28-case regression test (tests/test_morphology_irregular.py).
Realizer plural agreement (generate/templates.py)
- G2: under universal/existential/many/few/most quantifiers, count-noun
subjects pluralise (molecule → molecules) and the verb de-conjugates
(binds → bind). Negation toggles does-not → do-not. Aspect toggles
has → have, is → are. All other constructions unchanged.
- Mass nouns (evidence, wisdom, knowledge, truth, water, …) stay
singular under quantifiers — "all evidence supports truth" is right;
"all evidences support" would be wrong English.
- 17-case regression test
(tests/test_realizer_quantifier_agreement.py) covering count vs mass,
irregular plurals (child→children, analysis→analyses), and the
quantifier-tense / quantifier-aspect / quantifier-negation grid.
Rubric punctuation tolerance (evals/grammatical_coverage/runner.py)
- G3: _check_word_order strips trailing/leading punctuation
(.,;:!?—–) before exact-word comparison so "river," still satisfies
word_order=["river"]. must_contain also accepts punctuation-
stripped token matches.
- Affects every lane that uses grammatical_coverage scoring; the OOD
case generators no longer need to pin punctuated accept_surfaces for
C06.
Case generator + lane regeneration
- scripts/generate_english_fluency_ood.py uses generate.templates.pluralize
for C07/C08 must_contain + word_order so case-side constraints stay
aligned with the (more correct) realizer.
- All Phase 5 OOD lane cases (5.1, 5.4–5.7) regenerated; results files
re-scored.
CLI (core/cli.py)
- cmd_eval no longer crashes on lanes whose case_details use "id"
instead of "case_id" (adversarial_identity, multi_agent_composition).
- Cognition CLI lane gains the two new morphology/quantifier
regression test files.
Lane sweep (all 100%, no regression):
english_fluency_ood 117/117 public + 39/39 holdouts
elementary_mathematics_ood 117/117 + 39/39
foundational_physics_ood 117/117 + 39/39
foundational_biology_ood 117/117 + 39/39
classical_literature_ood 117/117 + 39/39
grammatical_coverage back to 100% on its own seed cases
hebrew_fluency / koine_greek_fluency 3/3 each
CLI lane health:
smoke 54, runtime 19, teaching 17, packs 6, cognition 103 (was 57),
algebra 132.
4.2 KiB
english-fluency-ood — v1 gaps
v1 results
| split | total | passed | accuracy |
|---|---|---|---|
| public | 117 | 117 | 100% |
| holdout | 39 | 39 | 100% |
Per-construction accuracy is 100% on every C01–C13 across every
domain (nature, tech, domestic, chemistry). Authoritative
metrics: results/v1_public_metrics.json,
results/v1_holdouts_metrics.json.
Headline
Realizer fluency is mechanistic, not pack-bound. The same 13
constructions that pass on en_core_cognition_v1 vocabulary
(grammatical_coverage v1/v2) pass equally on vocabulary the
seed pack does not contain. The structural claim Phase 5.1 set
out to test holds at v1.
Resolution — G1 / G2 / G3 closed 2026-05-17
All three v1 gaps below have been closed in generate/morphology.py,
generate/templates.py, and evals/grammatical_coverage/runner.py.
Regression tests:
tests/test_morphology_irregular.py(28 cases) —bind→bound,run→ran/running,stop→stopped/stopping, etc.tests/test_realizer_quantifier_agreement.py(17 cases) — "all molecules bind enzyme" (count) + "all evidence supports truth" (mass), plus negation/aspect interactions.
Phase 5.1–5.7 lane sweep stays at 100% on the more-correct realizer.
The original gap descriptions are preserved below for traceability.
Original v1 gaps (now resolved)
These are realizer gaps the v1 lane intentionally avoids exercising so the structural fluency claim is not confounded with morphology / agreement issues. They are real and need follow-on work, but they belong in their own lanes.
G1 — Irregular past tense
The realizer applies a regular -ed suffix unconditionally:
input: subject=molecule predicate=bind object=enzyme tense=past
output: "molecule binded enzyme." (expected: "molecule bound enzyme.")
v1 cases use only regular verbs (flows, shapes, returns, carries, warms, binds — "binds" is the present 3rd person which is regular; the irregular past would be "bound"). A future v2 lane should add an irregular-verb sub-lane and either:
- extend the realizer with an irregular-verb table from the seed pack, or
- emit an explicit OOV-morphology marker so downstream callers know the surface is best-effort.
G2 — Plural agreement under quantifiers
The realizer does not pluralize subjects under universal / existential quantifiers:
input: quantifier=all subject=molecule
output: "all molecule binds enzyme." (expected: "all molecules bind enzyme.")
Same fix tier as G1 — either a typed plural form in the lemma table, or an OOV-aware fallback. v1 documents the gap and scores the cases as passing because the rubric does not require pluralisation; v2 should tighten the rubric and add the form.
G3 — Constraint-rubric punctuation strictness (lane-side)
The shared _check_word_order rubric in
evals/grammatical_coverage/runner.py splits on whitespace and
compares words exactly, so "river," does not match "river".
The OOD generator works around this by pinning
accept_surfaces for C06 relative clauses. Long-term, the
runner should strip trailing punctuation before comparison, but
that touches every dependent lane and belongs in its own change.
Recommended follow-ons (out of this lane)
- v2 irregular-morphology sub-lane — small, explicit, with the gap-G1 verbs and a fix path inside the realizer.
- v2 quantifier-agreement sub-lane — gap G2.
- A discourse-scale fluency lane (Phase 5.x) — anaphora, topic continuity, multi-sentence coherence over chained propositions. Out of scope for v1.
- Cross-language fluency lanes (Phase 5.2 Hebrew, 5.3 Koine Greek). Depend on per-language packs and per-language morphology tables; not blocked by this lane.
What this lane evidences for Phase 5
The Phase 5 capability story rests on the realizer being able to voice arbitrary curriculum content grammatically. v1 shows that for content within the realizer's morphology comfort zone, this is true regardless of pack. The remaining work is morphology breadth, not fluency-architecture. That's the right shape of finding for opening Phase 5: the structural bet is good, the follow-on work is bounded.