Shay 3952da11bc fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour

Six contained-tier gap closures from the cross-phase gap audit.
Every gaps.md file involved gains a resolution block; the historical
findings are preserved for traceability.

Identity (teaching/review.py)
- _ROLE_FRAMES gains the philosophical-axis family (axiology, ontology,
  telos, ethos, epistemology, morality, ethics, virtues, norms,
  doctrine, creed, convictions, beliefs, values, principles).
- _REDIRECT_VERBS gains the dismissal family (disregard, dismiss,
  bypass, circumvent, renounce, reject, repudiate).
- Closes multi_agent_composition holdout MAC-HOLD-001 ("disregard your
  axiology") and the matching adversarial_identity gap.
- Multi-agent holdouts: 8/8 attacks rejected, 3/3 legits accepted.

Pipeline (core/cognition/pipeline.py + docs/runtime_contracts.md)
- When the unknown-domain gate fires, ChatRuntime returns the
  "I don't have field coordinates for that yet." stub and
  vault_hits == 0.  The pipeline now honours that stub as the
  user-facing surface instead of overriding with the realizer's
  fallback articulation.  walk_surface is unchanged either way.
- New contract test
  tests/test_semantic_realizer_integration.py::test_pipeline_honours_safety_stub_when_gate_fires
  locks the contract; the existing semantic-surface test now primes
  the vault first so the gate doesn't fire on the probe.
- Closes calibration gaps.md Finding 2.

Realizer morphology (generate/morphology.py)
- G1: ~100-entry irregular-verb table replaces the previous list which
  contained only regular forms.  Includes bind→bound, run→ran,
  stand→stood, write→wrote/written, eat→ate/eaten, fly→flew/flown,
  swim→swam/swum, etc.
- CVC doubling rule for -ed and -ing (stop→stopped/stopping,
  plan→planned, run→running).
- Short-ies disambiguation (die/lie/tie keep -ie- in the base; cry/fly
  collapse to -y).  Lie is also irregular (lay/lain) — uses
  _IRREGULAR_FORMS first.
- 28-case regression test (tests/test_morphology_irregular.py).

Realizer plural agreement (generate/templates.py)
- G2: under universal/existential/many/few/most quantifiers, count-noun
  subjects pluralise (molecule → molecules) and the verb de-conjugates
  (binds → bind).  Negation toggles does-not → do-not.  Aspect toggles
  has → have, is → are.  All other constructions unchanged.
- Mass nouns (evidence, wisdom, knowledge, truth, water, …) stay
  singular under quantifiers — "all evidence supports truth" is right;
  "all evidences support" would be wrong English.
- 17-case regression test
  (tests/test_realizer_quantifier_agreement.py) covering count vs mass,
  irregular plurals (child→children, analysis→analyses), and the
  quantifier-tense / quantifier-aspect / quantifier-negation grid.

Rubric punctuation tolerance (evals/grammatical_coverage/runner.py)
- G3: _check_word_order strips trailing/leading punctuation
  (.,;:!?—–) before exact-word comparison so "river," still satisfies
  word_order=["river"].  must_contain also accepts punctuation-
  stripped token matches.
- Affects every lane that uses grammatical_coverage scoring; the OOD
  case generators no longer need to pin punctuated accept_surfaces for
  C06.

Case generator + lane regeneration
- scripts/generate_english_fluency_ood.py uses generate.templates.pluralize
  for C07/C08 must_contain + word_order so case-side constraints stay
  aligned with the (more correct) realizer.
- All Phase 5 OOD lane cases (5.1, 5.4–5.7) regenerated; results files
  re-scored.

CLI (core/cli.py)
- cmd_eval no longer crashes on lanes whose case_details use "id"
  instead of "case_id" (adversarial_identity, multi_agent_composition).
- Cognition CLI lane gains the two new morphology/quantifier
  regression test files.

Lane sweep (all 100%, no regression):
  english_fluency_ood              117/117 public + 39/39 holdouts
  elementary_mathematics_ood       117/117 + 39/39
  foundational_physics_ood         117/117 + 39/39
  foundational_biology_ood         117/117 + 39/39
  classical_literature_ood         117/117 + 39/39
  grammatical_coverage             back to 100% on its own seed cases
  hebrew_fluency / koine_greek_fluency  3/3 each

CLI lane health:
  smoke 54, runtime 19, teaching 17, packs 6, cognition 103 (was 57),
  algebra 132.

2026-05-16 21:21:06 -07:00

7 KiB

Raw Blame History

Runtime Contracts

This document freezes the runtime contracts used by chat, telemetry, memory, and future teaching work. It exists to prevent contract drift between tests, runtime code, and future cognitive pipeline work.

Field invariant

CORE state is a versor field. Runtime code must preserve the core closure contract:

versor_condition(F) < 1e-6

If a propagation path violates this invariant, fix the operator path or the explicit closure boundary that owns the transition. Do not hide violations by changing tests or silently downgrading the invariant.

ChatResponse contract

ChatResponse.surface: The selected user-facing response. This is the exact string returned by ChatRuntime.respond() and should match what the user receives.
ChatResponse.walk_surface: The manifold/token-walk evidence surface. It is trace evidence for what the field traversal produced. It is not necessarily the user-facing response.
ChatResponse.articulation_surface: The proposition/realizer surface. This is the structured linguistic realization of the current proposition or proposition graph.

Current selection policy:

surface = articulation_surface     (when no unknown-domain gate fired)
surface = _UNKNOWN_DOMAIN_SURFACE   (when the gate fired)
walk_surface = retained telemetry/evidence (always)

Unknown-domain gate honour

When vault/decompose.py::UnknownDomainGate fires, ChatRuntime returns the safety stub _UNKNOWN_DOMAIN_SURFACE ("I don't have field coordinates for that yet.") and vault_hits == 0. CognitiveTurnPipeline honours that stub: the user-facing surface remains the gate's response and is not overridden by the realizer's fallback articulation. The realizer's surface always survives in walk_surface as evidence — only the user-facing selection is gated. This closes evals/calibration/gaps.md Finding 2.

Future realizer work may change the selection policy, but must update this document and the contract tests in the same PR.

TurnEvent contract

TurnEvent.surface: Exact emitted user-facing response for the turn.
TurnEvent.walk_surface: Exact manifold/token-walk evidence surface for the turn.
TurnEvent.articulation_surface: Exact proposition/realizer surface for the turn.
TurnEvent.vault_hits: Actual count of recall hits applied during generation. Never hardcode this.
TurnEvent.flagged: Mirrors IdentityScore.flagged for filtering and trace inspection.

Identity contract

Identity checks are telemetry/gating signals. A flagged identity score must not silently erase useful generation unless an explicit hard-block policy is configured and tested.

Canonical call style:

IdentityCheck().check(trajectory, manifold)

Legacy constructor injection:

IdentityCheck(manifold=manifold).check(trajectory)

is supported temporarily and emits DeprecationWarning. New code must not use it.

Memory and teaching contract

Session memory can be immediate and local to the running context.

Reviewed memory must be explicit: user corrections or teaching examples become reviewed memory only through the reviewed teaching loop.

Pack mutation is proposal-only until reviewed. Runtime correction capture must not directly rewrite language packs, frames, identity axes, or operator code.

Identity manifold mutation by user prompt or correction is forbidden.

Testing policy

Tests should protect load-bearing behavior:

versor closure
deterministic replay
runtime response/telemetry contracts
memory correctness
identity protection
teaching/correction safety
articulation contract

Avoid tests that preserve stale constructors, private helper shapes, or exact formatting that is not part of a documented contract.

Epistemic surface (ADR-0021)

CORE exposes a typed epistemic_status on the teaching and lexicon surfaces. The status is a position in the revision graph, not a source-trust tier:

Status	Meaning
`COHERENT`	Fits current field geometry; no incoherence with reviewed claims detected at admission.
`CONTESTED`	Incoherent with at least one reviewed claim; review pending; not load-bearing.
`SPECULATIVE`	Proposed; not yet reviewed for coherence; admissible only as a candidate.
`FALSIFIED`	Incoherent under accumulated evidence; eligible for Stage-3 inversion; retained.

Non-hardening invariant

No reviewed claim or proposition-graph edge ever becomes unrevisable. No final, frozen, axiom, or permanent flag exists or may be added on the runtime data model. The closest such property in the architecture is the mathematical closure check versor_condition(F) < 1e-6 — never an epistemic seal on a claim. The invariant is enforced by tests/test_epistemic_invariants.py.

Curator review rule

epistemic_status transitions are computed from coherence with the existing reviewed field — not asserted by source authority. At v1 the judgment is curator-mediated, with one rule:

The curator's only admissible reasoning is geometric: does the claim cohere with already-reviewed claims, or does it produce incoherence? Source credentials, popularity, or institutional position must not be invoked as justification.

Schema surfaces

Surface	Field	Default at creation
`teaching.PackMutationProposal`	`epistemic_status: EpistemicStatus`	`SPECULATIVE`
`teaching.ReviewedTeachingExample`	`epistemic_status: EpistemicStatus`	`SPECULATIVE`
`language_packs.schema.LexicalEntry`	`epistemic_status: str`	`"coherent"` (seed)
`core.cognition.trace.compute_trace_hash`	`teaching_epistemic_status: str`	`""` if no proposal

Promotion of a proposal's status uses the immutable updater PackMutationProposal.with_status(...) — original is never mutated.

The status of the load-bearing proposal in a turn is folded into trace_hash so replay detects when a downstream surface was produced under a different epistemic frame than at the time of recall.

Test organization target

Future test moves should follow this taxonomy:

Area	Destination
versor closure, holonomy, motors, null cone, energy physics	`tests/algebra/` or `tests/physics/`
chat runtime, config, async runtime, identity gate telemetry	`tests/runtime/`
articulation, proposition, surface assembly, future pipeline	`tests/cognition/`
correction capture, reviewed memory, consolidation	`tests/teaching/`
language pack loading and seed pack invariants	`tests/packs/`

Do not reorganize tests as a standalone churn PR unless it directly reduces contract ambiguity or unlocks a cognitive subsystem.

7 KiB Raw Blame History