Phase C of the gloss feature. Lands the natural-language gloss
content that the resolver (Phase B2) and the runtime composer
(Phase B3) were prepared for. This is the user-visible payoff:
cold-start DEFINITION / RECALL prompts on pack-resident lemmas now
emit fluent grounded sentences instead of dotted-domain disclosure.
Authoring: five parallel subagents in ONE message block (a single
parallel dispatch, ~20s wall-clock vs ~95s sequential). Each
subagent received its pack's complete lemma + POS list and a strict
JSON-shape exemplar. Total returned: 326 raw gloss entries.
Assembly (this commit): the raw entries were partitioned by
lexicon-residency lookup (the resolve_gloss invariant enforced at
storage time), deduplicated within pack, sorted by lemma, written
to ``language_packs/data/<pack>/glosses.jsonl``, and each pack's
manifest received a new ``glosses_checksum`` field. 323 glosses
landed clean; 0 rejected.
Per-pack distribution:
en_core_cognition_v1 78 glosses
en_core_meta_v1 72 glosses
en_core_attitude_v1 40 glosses
en_core_temporal_v1 28 glosses
en_core_action_v1 26 glosses
en_core_quantitative_v1 24 glosses
en_core_spatial_v1 24 glosses
en_core_polarity_v1 16 glosses
en_core_causation_v1 15 glosses
Live-probe lift (fresh ChatRuntime per prompt):
BEFORE:
truth — pack-grounded (en_core_cognition_v1):
cognition.truth; logos.core; epistemic.ground.
No session evidence yet.
AFTER:
Truth is a claim or state grounded by evidence and coherent
judgment. pack-grounded (en_core_cognition_v1).
Same provenance. Same audit-trail content (the dotted domains are
still in lexicon.jsonl, the resolver can still read them, the
candidate object carries them verbatim). But the user-facing
surface is a sentence the user can actually read.
Eval-lane lift:
deterministic_fluency BEFORE AFTER
no_dotted_inventory_rate 0.3333 → 1.0000
no_provenance_only_rate 1.0000 → 1.0000 (held)
no_placeholder_rate 1.0000 → 1.0000 (held)
complete_punctuation_rate 1.0000 → 1.0000 (held)
finite_predicate_shape 1.0000 → 1.0000 (held)
surface_provenance_match 1.0000 → 1.0000 (held)
cold_start_grounding all metrics held at 1.0
warmed_session_consistency no_placeholder + telemetry_match held at 1.0
(warm_grounding_stability still 0 — separate fix)
cognition eval public 100 / 100 / 91.7 / 100 (BYTE-IDENTICAL)
cognition eval holdout 100 / 100 / 83.3 / 100 (BYTE-IDENTICAL)
The cognition eval bytes-identity holds because the eval checks
substring containment (case-insensitive after the format change).
Every lemma still appears in its fluent surface.
Hardening this commit enforces:
Lexicon-residency at storage time
tests/test_pack_glosses_content.py::test_every_gloss_lemma_is_lexicon_resident
walks every glosses.jsonl and asserts every lemma is present in
the same pack's lexicon.jsonl. Drift in glosses (an unratified
lemma sneaking in) fails the lane immediately.
Dual-checksum discipline
tests/test_pack_glosses_content.py::test_every_glossed_pack_has_matching_checksum
re-hashes glosses.jsonl bytes-on-disk and compares against the
manifest's glosses_checksum. Any tampering fails.
Immutable-lexicon invariant
tests/test_pack_glosses_content.py::test_lexicon_checksum_unchanged_by_gloss_landing
re-hashes lexicon.jsonl and compares against the manifest's
(original) checksum. Proves that adding glosses did NOT perturb
the lexicon seal.
High-freq lemma resolution
32 of the most-common conversational lemmas (truth, doubt,
fact, idea, self, true, important, now, place, make, effect,
always, ...) all resolve to a fluent surface end-to-end.
Test-suite drift this commit absorbed:
- tests/test_pack_grounding.py — three substring assertions
updated to be case-insensitive (gloss-backed surfaces capitalize
lemmas at sentence start, dotted-disclosure surfaces don't).
"No session evidence yet" assertion replaced with the
common-substring "pack-grounded" marker that BOTH forms emit.
- tests/test_pack_resolver_glosses.py — the back-compat test
pivots from en_core_cognition_v1 (now glossed) to en_minimal_v1
(deliberately unglossed). A new test pins the glossed case.
Files added:
language_packs/data/<pack>/glosses.jsonl (9 files, 323 entries)
tests/test_pack_glosses_content.py (9 contract tests)
Files modified:
language_packs/data/<pack>/manifest.json (9 files, glosses_checksum field)
chat/pack_grounding.py (lowercase "pack-grounded" tag)
tests/test_pack_grounding.py (3 substring assertions relaxed)
tests/test_pack_resolver_glosses.py (back-compat test pivoted)
Verification:
127/127 affected tests green.
9/9 new gloss-content tests green.
All three eval lanes report the lift documented above.
Cognition eval byte-identical.
158 lines
6.3 KiB
Python
158 lines
6.3 KiB
Python
"""ADR-0048 — pack-grounded fallback surface tests.
|
|
|
|
The contract these tests pin:
|
|
|
|
- Pack-grounded surfaces engage ONLY when the ``UnknownDomainGate``
|
|
fires with ``source="empty_vault"`` AND the intent is DEFINITION
|
|
or RECALL AND the subject lemma is in ``en_core_cognition_v1``.
|
|
- The surface is composed verbatim from the pack lexicon's
|
|
``semantic_domains`` and the lemma — no synthesis.
|
|
- The audit contract is preserved: ChatResponse and TurnEvent both
|
|
carry a ``grounding_source`` provenance tag set to ``"pack"`` on
|
|
the pack-grounded path, ``"none"`` on the universal disclosure,
|
|
and ``"vault"`` on the main walk path.
|
|
- Safety / ethics refusal still takes priority — pack-grounded
|
|
surfaces never bypass a SafetyVerdict violation.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import pytest
|
|
|
|
from chat.pack_grounding import (
|
|
PACK_ID,
|
|
is_pack_lemma,
|
|
pack_grounded_surface,
|
|
)
|
|
from chat.runtime import ChatRuntime, _UNKNOWN_DOMAIN_SURFACE
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# pack_grounding module — pure-function contracts
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.mark.parametrize("lemma", ["light", "knowledge", "meaning", "memory", "truth"])
|
|
def test_known_pack_lemmas_produce_grounded_surface(lemma: str) -> None:
|
|
surface = pack_grounded_surface(lemma)
|
|
assert surface is not None
|
|
# Case-insensitive: gloss-backed surfaces capitalize the lemma at
|
|
# the start of the sentence ("Truth is a claim..."), while the
|
|
# dotted-disclosure fallback keeps it lowercase. Both forms must
|
|
# contain the lemma.
|
|
assert lemma.lower() in surface.lower()
|
|
assert PACK_ID in surface
|
|
# Surface carries either the gloss-backed "pack-grounded (pack_id)."
|
|
# tag or the dotted-disclosure "No session evidence yet." trailer.
|
|
assert "pack-grounded" in surface
|
|
|
|
|
|
def test_unknown_lemma_returns_none() -> None:
|
|
assert pack_grounded_surface("nonexistentwordxyz") is None
|
|
|
|
|
|
@pytest.mark.parametrize("bad", ["", " ", None])
|
|
def test_empty_or_invalid_lemma_returns_none(bad) -> None:
|
|
assert pack_grounded_surface(bad) is None # type: ignore[arg-type]
|
|
|
|
|
|
def test_is_pack_lemma_round_trips() -> None:
|
|
assert is_pack_lemma("light") is True
|
|
assert is_pack_lemma("nonexistentwordxyz") is False
|
|
assert is_pack_lemma("") is False
|
|
|
|
|
|
def test_surface_is_deterministic() -> None:
|
|
"""Same lemma must produce byte-identical surfaces on repeat calls
|
|
— pack is immutable, no randomness, no synthesis."""
|
|
a = pack_grounded_surface("light")
|
|
b = pack_grounded_surface("light")
|
|
assert a == b
|
|
assert a is not None
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# ChatRuntime integration — cold-start path
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_cold_start_definition_returns_pack_grounded_surface() -> None:
|
|
"""Cold-start DEFINITION on a pack-known lemma routes through the
|
|
pack-grounded surface, not the universal disclosure."""
|
|
rt = ChatRuntime()
|
|
resp = rt.chat("What is light?")
|
|
assert "pack-grounded" in resp.surface
|
|
# Case-insensitive: fluent gloss surfaces capitalize at sentence start.
|
|
assert "light" in resp.surface.lower()
|
|
assert resp.grounding_source == "pack"
|
|
|
|
|
|
def test_cold_start_recall_returns_pack_grounded_surface() -> None:
|
|
"""RECALL intent on a pack-known lemma also engages the pack path."""
|
|
rt = ChatRuntime()
|
|
resp = rt.chat("Remember light")
|
|
assert resp.grounding_source == "pack"
|
|
assert "light" in resp.surface.lower()
|
|
|
|
|
|
def test_cold_start_unknown_lemma_routes_to_oov_invitation() -> None:
|
|
"""ADR-0065 / P2.1 — when the classifier extracts a clean subject
|
|
that is OOV, the runtime emits the OOV "teach me" invitation
|
|
surface instead of the universal disclosure.
|
|
|
|
``How can I quoxulate the wxyzabc?`` is PROCEDURE intent;
|
|
``quoxulate`` is OOV. Pre-P2.1 this produced the universal
|
|
disclosure; post-P2.1 it produces an OOV invitation naming the
|
|
unknown token + the mounted-pack list."""
|
|
rt = ChatRuntime()
|
|
resp = rt.chat("How can I quoxulate the wxyzabc?")
|
|
assert resp.grounding_source == "oov"
|
|
assert "quoxulate" in resp.surface or "wxyzabc" in resp.surface
|
|
assert "PackMutationProposal" in resp.surface
|
|
|
|
|
|
def test_cold_start_cause_on_oov_routes_to_oov_invitation() -> None:
|
|
"""ADR-0065 / P2.1 — CAUSE on an OOV subject also routes to the
|
|
OOV invitation, not the universal disclosure.
|
|
|
|
Pre-P2.1 these prompts went silent (CAUSE branch early-returned
|
|
None when no teaching chain existed). Post-P2.1 the runtime
|
|
explicitly names the gap."""
|
|
rt = ChatRuntime()
|
|
resp = rt.chat("Why does wxyzabc exist?")
|
|
assert resp.grounding_source == "oov"
|
|
assert "wxyzabc" in resp.surface
|
|
|
|
|
|
def test_turn_event_carries_grounding_source() -> None:
|
|
"""ADR-0048 provenance propagates to TurnEvent for downstream audit."""
|
|
rt = ChatRuntime()
|
|
rt.chat("What is light?")
|
|
last_event = rt.turn_log[-1]
|
|
assert getattr(last_event, "grounding_source", None) == "pack"
|
|
|
|
|
|
def test_chat_response_grounding_source_default_for_main_path() -> None:
|
|
"""When the walk path runs, the ChatResponse carries
|
|
``grounding_source="vault"``. We force the walk by priming the
|
|
vault with one turn so the second turn's gate clears."""
|
|
rt = ChatRuntime()
|
|
rt.chat("light truth") # seed vault with one known-token turn
|
|
resp = rt.chat("light truth")
|
|
# The second turn may or may not have vault hits depending on
|
|
# gate threshold; what we assert is that grounding_source is set
|
|
# to one of the documented values.
|
|
assert resp.grounding_source in {"vault", "pack", "none"}
|
|
|
|
|
|
def test_pack_grounded_surface_passes_safety_check() -> None:
|
|
"""Pack-grounded surfaces preserve the audit contract — safety
|
|
and ethics verdicts still surface and refusal still takes
|
|
priority above pack grounding when triggered."""
|
|
rt = ChatRuntime()
|
|
resp = rt.chat("What is light?")
|
|
# Safety verdict must be present on every stub-path response
|
|
# (ADR-0035). Specific verdict outcome depends on the safety
|
|
# pack — we only assert the audit contract holds.
|
|
assert resp.safety_verdict is not None
|
|
assert resp.ethics_verdict is not None
|