core/tests/test_pack_grounding.py
Shay 9f83b27a7c feat(adr-0063): cross-pack surface resolver — kinship lemmas ground on live path
ADR-0063 closes the ADR-0048/0050/0053/0061 hardcoded-cognition-pack
asymmetry. New chat/pack_resolver.py provides resolve_lemma(lemma,
pack_ids) → (resolving_pack_id, semantic_domains) across an ordered
tuple of mounted lexicon packs (first-match-wins, lru_cache per-pack).

Surface composers in chat/pack_grounding.py now consult the resolver
instead of a hardcoded en_core_cognition_v1. en_core_relations_v1
joins RuntimeConfig.input_packs defaults; kinship lemmas now ground
on the live path:

  > What is a parent?
  parent — pack-grounded (en_core_relations_v1):
  kinship.ascendant.direct; kinship.parent; biology.progenitor.
  No session evidence yet.

Cross-pack comparison (knowledge × parent) renders composite tag
(en_core_cognition_v1 × en_core_relations_v1). Cognition lane
remains byte-identical: cognition is resolved first and the surface
format for cognition lemmas is unchanged.

Cognition eval (byte-identical to pre-ADR baseline):
  public  → intent 100% / surface 100% / term 91.7% / closure 100%
  holdout → intent 100% / surface 100% / term 83.3% / closure 100%

Curated lanes green: smoke 67 / cognition 121 / teaching 17 /
packs 6 / runtime 19 / algebra 132.

New tests: test_pack_resolver.py (28) + test_cross_pack_grounding.py
(17). test_en_core_relations_v1_pack.py: default-input-packs guard
inverted. test_pack_grounding.py: two stale ADR-0048 tests rewritten
(premises invalidated by ADR-0052/0061; now use fully-out-of-pack
prompts).

chat/teaching_grounding.py UNCHANGED — cognition_chains_v1 corpus
stays cognition-only. Cross-pack teaching corpora are the natural
ADR-0064.
2026-05-18 15:00:58 -07:00

150 lines
5.8 KiB
Python

"""ADR-0048 — pack-grounded fallback surface tests.
The contract these tests pin:
- Pack-grounded surfaces engage ONLY when the ``UnknownDomainGate``
fires with ``source="empty_vault"`` AND the intent is DEFINITION
or RECALL AND the subject lemma is in ``en_core_cognition_v1``.
- The surface is composed verbatim from the pack lexicon's
``semantic_domains`` and the lemma — no synthesis.
- The audit contract is preserved: ChatResponse and TurnEvent both
carry a ``grounding_source`` provenance tag set to ``"pack"`` on
the pack-grounded path, ``"none"`` on the universal disclosure,
and ``"vault"`` on the main walk path.
- Safety / ethics refusal still takes priority — pack-grounded
surfaces never bypass a SafetyVerdict violation.
"""
from __future__ import annotations
import pytest
from chat.pack_grounding import (
PACK_ID,
is_pack_lemma,
pack_grounded_surface,
)
from chat.runtime import ChatRuntime, _UNKNOWN_DOMAIN_SURFACE
# ---------------------------------------------------------------------------
# pack_grounding module — pure-function contracts
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("lemma", ["light", "knowledge", "meaning", "memory", "truth"])
def test_known_pack_lemmas_produce_grounded_surface(lemma: str) -> None:
surface = pack_grounded_surface(lemma)
assert surface is not None
assert lemma in surface
assert PACK_ID in surface
assert "No session evidence yet." in surface
def test_unknown_lemma_returns_none() -> None:
assert pack_grounded_surface("nonexistentwordxyz") is None
@pytest.mark.parametrize("bad", ["", " ", None])
def test_empty_or_invalid_lemma_returns_none(bad) -> None:
assert pack_grounded_surface(bad) is None # type: ignore[arg-type]
def test_is_pack_lemma_round_trips() -> None:
assert is_pack_lemma("light") is True
assert is_pack_lemma("nonexistentwordxyz") is False
assert is_pack_lemma("") is False
def test_surface_is_deterministic() -> None:
"""Same lemma must produce byte-identical surfaces on repeat calls
— pack is immutable, no randomness, no synthesis."""
a = pack_grounded_surface("light")
b = pack_grounded_surface("light")
assert a == b
assert a is not None
# ---------------------------------------------------------------------------
# ChatRuntime integration — cold-start path
# ---------------------------------------------------------------------------
def test_cold_start_definition_returns_pack_grounded_surface() -> None:
"""Cold-start DEFINITION on a pack-known lemma routes through the
pack-grounded surface, not the universal disclosure."""
rt = ChatRuntime()
resp = rt.chat("What is light?")
assert "pack-grounded" in resp.surface
assert "light" in resp.surface
assert resp.grounding_source == "pack"
def test_cold_start_recall_returns_pack_grounded_surface() -> None:
"""RECALL intent on a pack-known lemma also engages the pack path."""
rt = ChatRuntime()
resp = rt.chat("Remember light")
assert resp.grounding_source == "pack"
assert "light" in resp.surface
def test_cold_start_unknown_lemma_returns_universal_disclosure() -> None:
"""When the gate fires AND no lemma in the utterance resolves in any
mounted pack, we fall through to the universal disclosure unchanged.
ADR-0061 + ADR-0063 — the PROCEDURE composer scans the subject
phrase for any mounted-pack lemma. This test deliberately uses
a fully out-of-pack prompt so neither the cognition nor the
relations pack catches a topic anchor."""
rt = ChatRuntime()
resp = rt.chat("How can I quoxulate the wxyzabc?")
assert resp.surface == _UNKNOWN_DOMAIN_SURFACE
assert resp.grounding_source == "none"
def test_cold_start_non_definition_intent_no_pack_grounding() -> None:
"""CAUSE on a non-pack subject lemma does not engage the
pack-grounded DEFINITION path — pack-grounded surfaces require a
pack-resident lemma in any mounted lexicon (ADR-0048 + ADR-0063).
ADR-0052 teaching-grounded surfaces handle CAUSE on subjects that
appear in the reviewed cognition chains corpus; ``wxyzabc`` is in
neither the pack nor the corpus, so the universal disclosure fires."""
rt = ChatRuntime()
resp = rt.chat("Why does wxyzabc exist?")
assert resp.grounding_source == "none"
assert resp.surface == _UNKNOWN_DOMAIN_SURFACE
def test_turn_event_carries_grounding_source() -> None:
"""ADR-0048 provenance propagates to TurnEvent for downstream audit."""
rt = ChatRuntime()
rt.chat("What is light?")
last_event = rt.turn_log[-1]
assert getattr(last_event, "grounding_source", None) == "pack"
def test_chat_response_grounding_source_default_for_main_path() -> None:
"""When the walk path runs, the ChatResponse carries
``grounding_source="vault"``. We force the walk by priming the
vault with one turn so the second turn's gate clears."""
rt = ChatRuntime()
rt.chat("light truth") # seed vault with one known-token turn
resp = rt.chat("light truth")
# The second turn may or may not have vault hits depending on
# gate threshold; what we assert is that grounding_source is set
# to one of the documented values.
assert resp.grounding_source in {"vault", "pack", "none"}
def test_pack_grounded_surface_passes_safety_check() -> None:
"""Pack-grounded surfaces preserve the audit contract — safety
and ethics verdicts still surface and refusal still takes
priority above pack grounding when triggered."""
rt = ChatRuntime()
resp = rt.chat("What is light?")
# Safety verdict must be present on every stub-path response
# (ADR-0035). Specific verdict outcome depends on the safety
# pack — we only assert the audit contract holds.
assert resp.safety_verdict is not None
assert resp.ethics_verdict is not None