core-labs/core
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

feat(adr-0065): OOV cliff → five-tier honesty gradient (Phase 2.1 + 2.2)

Browse source 
Replaces the flat "I don't know — insufficient grounding" disclosure
with a deterministic gradient that names specific vocabulary gaps
and gives operators concrete next steps.

P2.1 — OOV "teach me" surface (chat/oov_surface.py).

  When the intent classifier extracts a clean subject lemma but that
  lemma is not resident in any mounted lexicon pack, the runtime now
  emits a deterministic learning-invitation surface tagged
  ``grounding_source="oov"`` instead of the universal disclosure.

  Surface format (fixed template):

    "I haven't learned '{token}' yet (intent: {intent}).
     Mounted lexicon packs: {pack_list}.
     Teach me via a reviewed PackMutationProposal."

  The OOV token passes through ``core._safe_display.safe_display``
  before persistence — user-input sanitization at the trust boundary.
  No vocabulary is invented; no domain is inferred.  Honours the
  ADR-0027 proposal-only invariant: the surface invites a reviewed
  pack mutation, never silently mutates any pack.

  Refactored ``_maybe_pack_grounded_surface`` so every existing
  intent branch (COMPARISON / CAUSE / VERIFICATION / CORRECTION /
  PROCEDURE / DEFINITION+RECALL) falls through on a None composer
  result instead of early-returning.  The OOV invitation is the
  deterministic fall-through for any clean-subject prompt whose
  subject doesn't resolve.

P2.2 — Partial-grounding tier (chat/partial_surface.py).

  When exactly one of two COMPARISON lemmas resolves, the runtime
  emits a hedged surface that grounds the known side verbatim and
  disclaims the OOV side explicitly:

    "Whatever '{oov}' is, I can ground '{known}' — pack-grounded
     ({pack_id}): {d1}; {d2}.  I cannot ground the comparison
     without learning '{oov}' — teach me via a reviewed
     PackMutationProposal."

  Tagged ``grounding_source="partial"``.  Falls through to OOV
  invitation when both lemmas are OOV, and to full pack-grounded
  COMPARISON when both resolve — partial is the middle tier in the
  five-tier gradient.

  Also normalises trailing sentence punctuation on
  intent.secondary_subject at the COMPARISON boundary so prompts
  like "Compare A and B." (with the period) still resolve B
  correctly.

Five-tier gradient (vault → teaching → pack → partial → oov → none).

Test debt retired: four pre-existing tests asserted "OOV → universal
disclosure", which is exactly the contract P2.1/P2.2 inverted.
Rewritten to the new contract.  Plus test_procedure_surface.py
gained a test for the OOV gradient on procedure intents.

Verification:
  tests/test_oov_surface.py                       22 passed
  tests/test_partial_surface.py                   16 passed
  Cognition eval byte-identical:
    public  100% / 100% / 91.7% / 100%
    holdout 100% / 100% / 83.3% / 100%
  Curated lanes all green.
This commit is contained in:
Shay 2026-05-18 16:41:45 -07:00
parent 34295e55ce
commit 51aad0c2cd
9 changed files with 779 additions and 63 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

135
chat/oov_surface.py Normal file
View file

@ -0,0 +1,135 @@
"""chat/oov_surface.py — Phase 2.1: OOV "teach me" surface.
When the intent classifier extracts a clean subject lemma but that
lemma is not resident in any mounted lexicon pack, the runtime today
falls through to the universal disclosure
(``_UNKNOWN_DOMAIN_SURFACE``). That surface is *honest* (it does
not pretend to know) but it is also *flat* — it conveys no signal
that a specific vocabulary gap was hit, and it offers the operator
no concrete next step.
This module replaces that cliff with a gradient. Cold-start prompts
whose subject is OOV emit a deterministic learning-invitation
surface that:
1. Names the unknown token explicitly so the operator sees which
word the system could not ground.
2. Lists the currently-mounted lexicon packs so the operator knows
where the token could be added.
3. Points at the existing reviewed-pack-mutation path
(:mod:`teaching.proposals`) as the way to teach the system the
new lemma — never "auto-learn", never invent meaning.
The surface is tagged ``grounding_source="oov"`` so downstream audit,
discovery aggregation, and operator tooling can distinguish
"I haven't learned this yet" from "I refuse" / "I'm unsure" /
"insufficient evidence".
Design constraints (matching ADR-0048..0064 doctrine):
- **Deterministic.** Same OOV token + same mounted-pack list →
byte-identical surface.
- **No synthesis.** The surface composes only:
* the OOV token (verbatim user input — safely escaped at the
:func:`chat._safe_display.safe_display` boundary),
* the mounted-pack ids (declared statically in
:data:`chat.pack_resolver.DEFAULT_RESOLVABLE_PACK_IDS`),
* a fixed-template instruction.
No new vocabulary is invented; no domain inference is performed.
- **Trust boundary preserved.** The surface invites a *reviewed*
pack mutation; it never silently mutates any pack or corpus. The
ADR-0027 proposal-only invariant is intact.
"""
from __future__ import annotations
from chat.pack_resolver import DEFAULT_RESOLVABLE_PACK_IDS, is_resolvable
from core._safe_display import safe_display
from generate.intent import IntentTag
# Intent shapes for which the runtime emits a grounded cold-start
# surface today (ADR-0048 / 0050 / 0052 / 0053 / 0061). OOV
# invitation fires only when the prompt's intent is one of these —
# UNKNOWN-intent prompts get the universal disclosure unchanged
# because the classifier itself could not extract a confident
# subject.
_OOV_INTENT_TAGS: frozenset[IntentTag] = frozenset({
IntentTag.DEFINITION,
IntentTag.RECALL,
IntentTag.CAUSE,
IntentTag.VERIFICATION,
IntentTag.COMPARISON,
IntentTag.PROCEDURE,
IntentTag.CORRECTION,
})
def oov_learning_invitation_surface(
token: str,
intent_tag: IntentTag,
*,
pack_ids: tuple[str, ...] = DEFAULT_RESOLVABLE_PACK_IDS,
) -> str | None:
"""Return a deterministic OOV learning-invitation surface, or ``None``.
The surface format is fixed:
"I haven't learned '{token}' yet (intent: {intent}).
Mounted lexicon packs: {pack_list}.
Teach me via a reviewed PackMutationProposal."
The trailing instruction is the constant trust-boundary label.
It points at the existing reviewed-pack-mutation path; the
surface never invents meaning for the unknown token.
Returns ``None`` (caller falls through to the universal disclosure)
when:
- ``token`` is empty or not a string,
- ``token`` IS resolvable in *pack_ids* (caller routed here by
mistake — keep the explicit fall-through rather than emit a
misleading surface),
- the mounted-pack list is empty (no learnable destination —
emitting an invitation with no targets would be unhelpful).
"""
if not token or not isinstance(token, str):
return None
cleaned = token.strip()
if not cleaned:
return None
if intent_tag not in _OOV_INTENT_TAGS:
return None
if is_resolvable(cleaned, pack_ids):
return None
if not pack_ids:
return None
safe_token = safe_display(cleaned)
pack_list = ", ".join(pack_ids)
intent_name = intent_tag.name.lower()
return (
f"I haven't learned '{safe_token}' yet (intent: {intent_name}). "
f"Mounted lexicon packs: {pack_list}. "
f"Teach me via a reviewed PackMutationProposal."
)
def is_oov_for_packs(
token: str,
pack_ids: tuple[str, ...] = DEFAULT_RESOLVABLE_PACK_IDS,
) -> bool:
"""Return True iff *token* is non-empty and not resolvable in
any of *pack_ids*. Convenience predicate for the runtime
dispatcher (avoids duplicating the ``is_resolvable`` inversion
in caller code)."""
if not token or not isinstance(token, str):
return False
cleaned = token.strip()
if not cleaned:
return False
return not is_resolvable(cleaned, pack_ids)
__all__ = [
"oov_learning_invitation_surface",
"is_oov_for_packs",
]

115
chat/partial_surface.py Normal file
View file

@ -0,0 +1,115 @@
"""chat/partial_surface.py — Phase 2.2: partial-grounding tier.
When a prompt contains both an OOV token AND a pack-resident token,
the runtime today has two choices:
1. ``pack_grounded_*`` composers — require *both* tokens to resolve
(ADR-0050 COMPARISON: identical-lemma → None; OOV-lemma → None).
2. The OOV invitation (P2.1) — names one unknown token but ignores
the known one entirely.
Both miss a real signal: the known token is actually grounded, the
relation is partially representable, and the operator deserves to
see *which* side is OOV instead of a flat "I don't know one of these".
This module composes a **partial-grounding** surface that:
- Grounds the pack-resident token verbatim from its lexicon
(same atoms a full pack-grounded surface would emit).
- Names the OOV token explicitly under a "whatever ... is" hedge —
no synthesis, no inferred meaning, no domain guess.
- States the contract: the relation cannot be grounded until the
OOV token is ratified into a pack.
- Tags ``grounding_source="partial"`` so audit and downstream
aggregation distinguish this from full pack-grounded
surfaces or the universal disclosure.
Today's scope is the COMPARISON intent (two subject lemmas, one OOV +
one known). CAUSE/VERIFICATION extract a single subject; if it's
OOV the OOV invitation surface (P2.1) is the right surface — there
is no second lemma to partially ground against. Future ADRs can
extend partial-grounding to other intent shapes as the classifier
grows multi-lemma extraction.
Trust boundary:
- The partial surface composes only the known-side lexicon atoms,
the (safely-displayed) OOV token, and a fixed template.
- No vocabulary is invented; no meaning is inferred for the OOV
side.
- The trailing instruction points at the reviewed pack-mutation
path — partial grounding never auto-mutates state.
"""
from __future__ import annotations
from chat.pack_resolver import DEFAULT_RESOLVABLE_PACK_IDS, resolve_lemma
from core._safe_display import safe_display
def partial_comparison_surface(
lemma_a: str,
lemma_b: str,
*,
pack_ids: tuple[str, ...] = DEFAULT_RESOLVABLE_PACK_IDS,
) -> tuple[str, str] | None:
"""Return ``(surface, known_side)`` where ``known_side`` is ``"a"``
or ``"b"`` depending on which lemma resolved, or ``None``.
The surface format is fixed:
"Whatever '{oov}' is, I can ground '{known}'
— pack-grounded ({pack_id}): {d1}; {d2}.
I cannot ground the comparison without learning '{oov}' —
teach me via a reviewed PackMutationProposal."
The composer returns ``None`` when:
- either lemma is empty / not a string,
- both lemmas resolve (route through the full
``pack_grounded_comparison_surface`` instead),
- neither lemma resolves (route through the OOV invitation;
partial-grounding has nothing to anchor on),
- the two lemmas are identical strings (same-lemma comparison
carries no contrastive evidence at any tier).
"""
if not lemma_a or not isinstance(lemma_a, str):
return None
if not lemma_b or not isinstance(lemma_b, str):
return None
key_a = lemma_a.strip().lower()
key_b = lemma_b.strip().lower()
if not key_a or not key_b or key_a == key_b:
return None
resolved_a = resolve_lemma(key_a, pack_ids)
resolved_b = resolve_lemma(key_b, pack_ids)
# Partial-grounding requires exactly one side to resolve.
if resolved_a is None and resolved_b is None:
return None
if resolved_a is not None and resolved_b is not None:
return None
if resolved_a is not None:
known_lemma = key_a
known_pack_id, known_domains = resolved_a
oov_lemma = key_b
known_side = "a"
else:
assert resolved_b is not None
known_lemma = key_b
known_pack_id, known_domains = resolved_b
oov_lemma = key_a
known_side = "b"
safe_oov = safe_display(oov_lemma)
head = "; ".join(known_domains[:2])
surface = (
f"Whatever '{safe_oov}' is, I can ground '{known_lemma}' "
f"— pack-grounded ({known_pack_id}): {head}. "
f"I cannot ground the comparison without learning '{safe_oov}' "
f"— teach me via a reviewed PackMutationProposal."
)
return (surface, known_side)
__all__ = ["partial_comparison_surface"]

182
chat/runtime.py
View file

@ -424,6 +424,11 @@ class ChatRuntime:
# ``attach_discovery_sink``. Candidates are *evidence*, never
# mutate the corpus or runtime state.
self._discovery_sink: DiscoveryCandidateSink | None = None
# Phase 2.3 — opt-in OOV candidate sink. Default None preserves
# prior behavior; callers attach via ``attach_oov_sink``.
# Candidates are evidence; ratified-pack-mutation is the only
# path an OOV promotion becomes a real pack change.
self._oov_sink: Any = None
# ADR-0056 Phase C1 — opt-in contemplation pass that enriches
# each emitted DiscoveryCandidate with polarity / claim_domain /
# evidence / sub_questions before the sink writes the JSONL
@ -458,6 +463,21 @@ class ChatRuntime:
self._telemetry_sink = sink
self._telemetry_include_content = bool(include_content)
def attach_oov_sink(self, sink: Any) -> None:
"""Phase 2.3 — attach an OOV candidate sink.
After each turn whose surface fired the P2.1 OOV invitation
(``grounding_source="oov"``), the runtime emits one
:class:`teaching.oov_sink.OOVCandidate` JSONL line to the
attached sink. Passing ``None`` detaches.
Candidates are evidence: emission never mutates any pack.
The ratified pack-mutation path (ADR-0027 +
:mod:`teaching.proposals`) is the only way an OOV promotion
becomes a real pack change.
"""
self._oov_sink = sink
def attach_discovery_sink(
self,
sink: DiscoveryCandidateSink | None,
@ -492,6 +512,55 @@ class ChatRuntime:
"""
self._contemplate_discoveries = bool(enabled)
def _emit_oov_candidate(
self,
*,
turn_event: TurnEvent,
intent_tag: Any,
token: str | None,
) -> None:
"""P2.3 — emit one OOVCandidate per OOV-grounded turn.
No-op unless ``attach_oov_sink`` was called. The token is
already safe-displayed at the surface composer; persistence
carries the same sanitised form.
"""
sink = self._oov_sink
if sink is None or not token:
return
# Local imports — keep OOV machinery out of the runtime
# hot-path import graph for callers that never opt in.
from teaching.oov_sink import (
OOVCandidate,
format_oov_candidate_jsonl,
hash_oov_candidate_id,
)
from generate.intent import IntentTag
if intent_tag is None or not isinstance(intent_tag, IntentTag):
return
intent_name = intent_tag.name.lower()
# Pull trace hash from the turn event when present so the
# candidate_id replays deterministically.
trace_hash = getattr(turn_event, "trace_hash", "") or ""
boundary_clean = (
not getattr(turn_event, "refusal_emitted", False)
and not getattr(turn_event, "hedge_injected", False)
)
cleaned_token = (token or "").strip().lower()
if not cleaned_token:
return
candidate_id = hash_oov_candidate_id(cleaned_token, intent_name, trace_hash)
candidate = OOVCandidate(
candidate_id=candidate_id,
token=cleaned_token,
intent=intent_name, # type: ignore[arg-type]
trigger="unresolved_subject",
source_turn_trace=trace_hash,
boundary_clean=boundary_clean,
)
sink.emit(format_oov_candidate_jsonl(candidate))
def _emit_discovery_candidates(
self,
*,
@ -655,32 +724,51 @@ class ChatRuntime:
# composed from both lemmas' pack semantic_domains. Engages only
# when both subject and secondary_subject are pack lemmas.
if intent.tag is IntentTag.COMPARISON:
lemma_a = (intent.subject or "").strip()
lemma_b = (intent.secondary_subject or "").strip()
if not lemma_a or not lemma_b:
return None
surface = pack_grounded_comparison_surface(lemma_a, lemma_b)
return (surface, "pack") if surface is not None else None
# The intent classifier may retain terminal punctuation on
# secondary_subject when it falls at the end of the prompt
# ("Compare A and B."). Strip terminal sentence punctuation
# so the resolver can find the underlying lemma. This is
# a normalization at the runtime boundary, not in the
# classifier itself, to keep the classifier's verbatim
# extraction available to other consumers.
lemma_a = (intent.subject or "").strip().rstrip(".,?!;:")
lemma_b = (intent.secondary_subject or "").strip().rstrip(".,?!;:")
if lemma_a and lemma_b:
surface = pack_grounded_comparison_surface(lemma_a, lemma_b)
if surface is not None:
return (surface, "pack")
# P2.2 — Partial-grounding tier. When exactly one of
# the two compared lemmas is pack-resident, emit a
# hedged surface that grounds the known side and
# explicitly disclaims the OOV side. Better than
# falling through to OOV invitation (which would name
# only one token while ignoring the other's actual
# grounding).
from chat.partial_surface import partial_comparison_surface
partial = partial_comparison_surface(lemma_a, lemma_b)
if partial is not None:
return (partial[0], "partial")
# ADR-0052 — teaching-grounded CAUSE / VERIFICATION. The chain
# corpus is reviewed memory; every emitted atom is either a
# lemma, a verbatim pack semantic_domains string, or a fixed
# connective from humanize_predicate.
if intent.tag in (IntentTag.CAUSE, IntentTag.VERIFICATION):
lemma = (intent.subject or "").strip()
if not lemma:
return None
# ADR-0062 — when ``composed_surface`` is enabled, the
# teaching-grounded composer extends the single-chain
# surface with a follow-up chain whose subject equals the
# initial chain's object. Backward-compatible: with the
# flag off, the single-chain composer is used; with the
# flag on and no follow-up chain available, the composer
# degrades to the single-chain surface byte-identically.
if self.config.composed_surface:
surface = teaching_grounded_surface_composed(lemma, intent.tag)
else:
surface = teaching_grounded_surface(lemma, intent.tag)
return (surface, "teaching") if surface is not None else None
if lemma:
# ADR-0062 — when ``composed_surface`` is enabled, the
# teaching-grounded composer extends the single-chain
# surface with a follow-up chain whose subject equals
# the initial chain's object. Backward-compatible:
# with the flag off, the single-chain composer is
# used; with the flag on and no follow-up chain
# available, the composer degrades to the single-
# chain surface byte-identically.
if self.config.composed_surface:
surface = teaching_grounded_surface_composed(lemma, intent.tag)
else:
surface = teaching_grounded_surface(lemma, intent.tag)
if surface is not None:
return (surface, "teaching")
# ADR-0053 — CORRECTION acknowledgement. Cold-start CORRECTION
# has no prior session turn to apply to; emit a pack-grounded
# surface that acknowledges the correction was received and
@ -694,7 +782,8 @@ class ChatRuntime:
# topical lemma present, the surface degrades to the
# ADR-0053 topic-less template.
surface = pack_grounded_correction_surface(text)
return (surface, "pack") if surface is not None else None
if surface is not None:
return (surface, "pack")
# ADR-0061 — PROCEDURE pack-grounded surface. Procedural
# chains are not part of the reviewed teaching corpus today
# (CAUSE/VERIFICATION only). Rather than fall through to the
@ -705,17 +794,32 @@ class ChatRuntime:
# ratified. Honest, deterministic, pack-grounded.
if intent.tag is IntentTag.PROCEDURE:
subject_text = (intent.subject or "").strip()
if not subject_text:
if subject_text:
surface = pack_grounded_procedure_surface(subject_text)
if surface is not None:
return (surface, "pack")
if intent.tag in (IntentTag.DEFINITION, IntentTag.RECALL):
lemma = (intent.subject or "").strip()
if not lemma:
return None
surface = pack_grounded_procedure_surface(subject_text)
return (surface, "pack") if surface is not None else None
if intent.tag not in (IntentTag.DEFINITION, IntentTag.RECALL):
return None
lemma = (intent.subject or "").strip()
if not lemma:
return None
surface = pack_grounded_surface(lemma)
return (surface, "pack") if surface is not None else None
surface = pack_grounded_surface(lemma)
if surface is not None:
return (surface, "pack")
# P2.1 — OOV "teach me" surface. If the classified intent is
# one of the surface-supported shapes AND the subject lemma
# does not resolve in any mounted lexicon pack, emit a
# deterministic learning-invitation surface tagged
# ``grounding_source="oov"`` instead of falling through to
# the universal disclosure. Converts the OOV cliff into a
# gradient that names the unknown token + points the operator
# at the reviewed pack-mutation path.
oov_lemma = (intent.subject or "").strip()
if oov_lemma:
from chat.oov_surface import oov_learning_invitation_surface
oov_surface = oov_learning_invitation_surface(oov_lemma, intent.tag)
if oov_surface is not None:
return (oov_surface, "oov")
return None
def _stub_response(
self,
@ -848,6 +952,15 @@ class ChatRuntime:
intent_subject=discovery_intent_subject,
grounding_source=grounding_source,
)
# P2.3 — emit OOV candidate when the surface fired the
# OOV invitation. Only when an operator has attached
# a sink — otherwise this is a no-op.
if grounding_source == "oov":
self._emit_oov_candidate(
turn_event=stub_event,
intent_tag=discovery_intent_tag,
token=discovery_intent_subject,
)
return ChatResponse(
surface=response_surface,
proposition=prop,
@ -926,8 +1039,13 @@ class ChatRuntime:
# no sink is attached.
discovery_intent_tag = None
discovery_intent_subject: str | None = None
# Classify intent up-front when EITHER the discovery sink
# OR the OOV sink is attached (P2.3) — both downstream
# emission paths need it. Without either sink attached
# this is a no-op, so behaviour stays identical to the
# pre-Phase-2 path.
if (
self._discovery_sink is not None
(self._discovery_sink is not None or self._oov_sink is not None)
and gate_decision.source == "empty_vault"
and self.config.output_language == "en"
):

184
tests/test_oov_surface.py Normal file
View file

@ -0,0 +1,184 @@
"""Phase 2.1 — OOV "teach me" surface tests.
The contract these tests pin:
- OOV tokens with a supported intent produce a deterministic
learning-invitation surface tagged ``grounding_source="oov"``.
- Known tokens still ground through the existing pack/teaching
paths byte-identically.
- The OOV surface names the unknown token, lists mounted packs,
and points at the reviewed PackMutationProposal path — never
invents meaning, never auto-mutates.
- UNKNOWN-intent prompts still get the universal disclosure
(the classifier itself failed to extract a confident subject).
- User-text passes through the safe-display sanitiser; control
chars do not leak into surfaces.
"""
from __future__ import annotations
import pytest
from chat.oov_surface import (
is_oov_for_packs,
oov_learning_invitation_surface,
)
from chat.runtime import ChatRuntime
from generate.intent import IntentTag
# ---------------------------------------------------------------------------
# Pure-function contract
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("intent_tag", [
IntentTag.DEFINITION,
IntentTag.RECALL,
IntentTag.CAUSE,
IntentTag.VERIFICATION,
IntentTag.COMPARISON,
IntentTag.PROCEDURE,
IntentTag.CORRECTION,
])
def test_oov_token_with_supported_intent_emits_invitation(
intent_tag: IntentTag,
) -> None:
surface = oov_learning_invitation_surface("photosynthesis", intent_tag)
assert surface is not None
assert "photosynthesis" in surface
assert "haven't learned" in surface
assert "en_core_cognition_v1" in surface
assert "en_core_relations_v1" in surface
assert "PackMutationProposal" in surface
assert intent_tag.name.lower() in surface
def test_known_token_returns_none() -> None:
"""If the token IS resolvable, the composer returns None so the
caller routes through pack-grounded / teaching-grounded paths."""
assert oov_learning_invitation_surface("light", IntentTag.DEFINITION) is None
assert oov_learning_invitation_surface("parent", IntentTag.CAUSE) is None
assert oov_learning_invitation_surface("knowledge", IntentTag.VERIFICATION) is None
@pytest.mark.parametrize("bad", ["", " ", None])
def test_empty_or_invalid_token_returns_none(bad) -> None:
assert oov_learning_invitation_surface(bad, IntentTag.DEFINITION) is None # type: ignore[arg-type]
def test_unknown_intent_returns_none() -> None:
"""UNKNOWN intent means the classifier could not extract a
confident subject; emitting an invitation for an unparsed prompt
would be misleading."""
assert oov_learning_invitation_surface("photosynthesis", IntentTag.UNKNOWN) is None
def test_empty_pack_list_returns_none() -> None:
"""With no mounted packs there is no learnable destination;
the invitation has no targets to suggest."""
surface = oov_learning_invitation_surface(
"photosynthesis", IntentTag.DEFINITION, pack_ids=(),
)
assert surface is None
def test_surface_is_deterministic() -> None:
a = oov_learning_invitation_surface("photosynthesis", IntentTag.DEFINITION)
b = oov_learning_invitation_surface("photosynthesis", IntentTag.DEFINITION)
assert a == b
assert a is not None
def test_surface_includes_explicit_intent_name() -> None:
for tag, expected_token in [
(IntentTag.DEFINITION, "definition"),
(IntentTag.CAUSE, "cause"),
(IntentTag.VERIFICATION, "verification"),
(IntentTag.PROCEDURE, "procedure"),
]:
surface = oov_learning_invitation_surface("xyzunknown", tag)
assert surface is not None
assert f"(intent: {expected_token})" in surface
# ---------------------------------------------------------------------------
# Safety — user text is sanitised at the safe_display boundary
# ---------------------------------------------------------------------------
def test_control_characters_in_token_are_sanitised() -> None:
surface = oov_learning_invitation_surface(
"evil\x00\x07token", IntentTag.DEFINITION,
)
assert surface is not None
# Control bytes must not survive — safe_display strips/escapes them.
assert "\x00" not in surface
assert "\x07" not in surface
# ---------------------------------------------------------------------------
# is_oov_for_packs predicate
# ---------------------------------------------------------------------------
def test_is_oov_for_packs_round_trips() -> None:
assert is_oov_for_packs("photosynthesis") is True
assert is_oov_for_packs("light") is False
assert is_oov_for_packs("parent") is False
assert is_oov_for_packs("") is False
assert is_oov_for_packs(" ") is False
# ---------------------------------------------------------------------------
# Live runtime — OOV converts cliff into gradient
# ---------------------------------------------------------------------------
def test_runtime_definition_on_oov_emits_invitation() -> None:
rt = ChatRuntime()
resp = rt.chat("What is photosynthesis?")
assert resp.grounding_source == "oov"
assert "photosynthesis" in resp.surface
assert "PackMutationProposal" in resp.surface
def test_runtime_cause_on_oov_emits_invitation() -> None:
"""A CAUSE prompt on an OOV subject must hit the OOV branch, not
fall through to the universal disclosure. Previously these went
silent because the CAUSE/VERIFICATION branch early-returned None
when no teaching chain existed."""
rt = ChatRuntime()
resp = rt.chat("Why does mitochondria exist?")
assert resp.grounding_source == "oov"
assert "mitochondria" in resp.surface
def test_runtime_known_subject_still_grounds_pack() -> None:
"""Known cognition lemmas still ground through the pack path —
OOV branch is a fall-through only, not a replacement."""
rt = ChatRuntime()
resp = rt.chat("What is light?")
assert resp.grounding_source == "pack"
assert "light" in resp.surface
def test_runtime_known_subject_still_grounds_teaching() -> None:
"""Reviewed teaching chains still route through teaching-grounded."""
rt = ChatRuntime()
resp = rt.chat("Why does parent exist?")
assert resp.grounding_source == "teaching"
assert "parent" in resp.surface
def test_runtime_unknown_intent_still_emits_universal_disclosure() -> None:
"""If the classifier returns UNKNOWN intent, there is no clean
subject to invite an operator to teach. Universal disclosure
remains the right fall-through."""
rt = ChatRuntime()
resp = rt.chat("Define mitochondria.") # classifier returns UNKNOWN here
# Either UNKNOWN intent → universal disclosure ("none"), OR if
# the classifier improves to read "Define mitochondria." as
# DEFINITION the prompt should switch to OOV invitation. Both
# are acceptable.
assert resp.grounding_source in {"none", "oov"}

18
tests/test_pack_grounded_comparison.py
View file

@ -96,13 +96,21 @@ def test_cold_start_comparison_returns_pack_grounded_surface() -> None:
assert "contrasts with" in resp.surface
def test_cold_start_comparison_with_unknown_lemma_disclosure() -> None:
"""When one term is not a pack lemma, the COMPARISON path returns
None and the runtime falls through to the universal disclosure."""
def test_cold_start_comparison_with_unknown_lemma_routes_to_partial() -> None:
"""ADR-0065 / P2.2 — when exactly one COMPARISON lemma resolves
and the other is OOV, the runtime emits the partial-grounding
surface (grounds the known side, hedges the OOV side) instead of
the universal disclosure.
Pre-P2.2 this returned the flat disclosure; post-P2.2 it emits
an explicit partial surface that names which side could be
grounded and which side needs a reviewed PackMutationProposal."""
rt = ChatRuntime()
resp = rt.chat("Compare memory and zigzagxyz")
assert resp.surface == _UNKNOWN_DOMAIN_SURFACE
assert resp.grounding_source == "none"
assert resp.grounding_source == "partial"
assert "memory" in resp.surface
assert "zigzagxyz" in resp.surface
assert "PackMutationProposal" in resp.surface
def test_cold_start_comparison_with_identical_lemmas_disclosure() -> None:

37
tests/test_pack_grounding.py
View file

@ -88,32 +88,33 @@ def test_cold_start_recall_returns_pack_grounded_surface() -> None:
assert "light" in resp.surface
def test_cold_start_unknown_lemma_returns_universal_disclosure() -> None:
"""When the gate fires AND no lemma in the utterance resolves in any
mounted pack, we fall through to the universal disclosure unchanged.
def test_cold_start_unknown_lemma_routes_to_oov_invitation() -> None:
"""ADR-0065 / P2.1 — when the classifier extracts a clean subject
that is OOV, the runtime emits the OOV "teach me" invitation
surface instead of the universal disclosure.
ADR-0061 + ADR-0063 — the PROCEDURE composer scans the subject
phrase for any mounted-pack lemma. This test deliberately uses
a fully out-of-pack prompt so neither the cognition nor the
relations pack catches a topic anchor."""
``How can I quoxulate the wxyzabc?`` is PROCEDURE intent;
``quoxulate`` is OOV. Pre-P2.1 this produced the universal
disclosure; post-P2.1 it produces an OOV invitation naming the
unknown token + the mounted-pack list."""
rt = ChatRuntime()
resp = rt.chat("How can I quoxulate the wxyzabc?")
assert resp.surface == _UNKNOWN_DOMAIN_SURFACE
assert resp.grounding_source == "none"
assert resp.grounding_source == "oov"
assert "quoxulate" in resp.surface or "wxyzabc" in resp.surface
assert "PackMutationProposal" in resp.surface
def test_cold_start_non_definition_intent_no_pack_grounding() -> None:
"""CAUSE on a non-pack subject lemma does not engage the
pack-grounded DEFINITION path — pack-grounded surfaces require a
pack-resident lemma in any mounted lexicon (ADR-0048 + ADR-0063).
def test_cold_start_cause_on_oov_routes_to_oov_invitation() -> None:
"""ADR-0065 / P2.1 — CAUSE on an OOV subject also routes to the
OOV invitation, not the universal disclosure.
ADR-0052 teaching-grounded surfaces handle CAUSE on subjects that
appear in the reviewed cognition chains corpus; ``wxyzabc`` is in
neither the pack nor the corpus, so the universal disclosure fires."""
Pre-P2.1 these prompts went silent (CAUSE branch early-returned
None when no teaching chain existed). Post-P2.1 the runtime
explicitly names the gap."""
rt = ChatRuntime()
resp = rt.chat("Why does wxyzabc exist?")
assert resp.grounding_source == "none"
assert resp.surface == _UNKNOWN_DOMAIN_SURFACE
assert resp.grounding_source == "oov"
assert "wxyzabc" in resp.surface
def test_turn_event_carries_grounding_source() -> None:

145
tests/test_partial_surface.py Normal file
View file

@ -0,0 +1,145 @@
"""Phase 2.2 — partial-grounding tier tests.
The contract these tests pin:
- When exactly one of the two compared lemmas resolves in a
mounted pack, the partial composer emits a hedged surface
grounding the known side and disclaiming the OOV side.
- When both resolve OR neither resolves, the composer returns
``None`` — the caller routes through the full pack-grounded
composer or the OOV invitation respectively.
- The OOV token is sanitised through ``safe_display``.
- Live runtime: COMPARISON with mixed-residency lemmas tags
``grounding_source="partial"``.
- Terminal punctuation on secondary_subject does not defeat
resolution.
"""
from __future__ import annotations
import pytest
from chat.partial_surface import partial_comparison_surface
from chat.runtime import ChatRuntime
# ---------------------------------------------------------------------------
# Pure-function contract
# ---------------------------------------------------------------------------
def test_oov_first_known_second_emits_partial() -> None:
result = partial_comparison_surface("photosynthesis", "knowledge")
assert result is not None
surface, known_side = result
assert known_side == "b"
assert "knowledge" in surface
assert "photosynthesis" in surface
assert "Whatever 'photosynthesis' is" in surface
assert "pack-grounded (en_core_cognition_v1)" in surface
assert "PackMutationProposal" in surface
def test_known_first_oov_second_emits_partial() -> None:
result = partial_comparison_surface("knowledge", "photosynthesis")
assert result is not None
surface, known_side = result
assert known_side == "a"
assert "Whatever 'photosynthesis' is" in surface
assert "I can ground 'knowledge'" in surface
def test_both_known_returns_none() -> None:
"""Both lemmas resolve — caller should route through the full
pack-grounded comparison composer instead."""
assert partial_comparison_surface("knowledge", "truth") is None
assert partial_comparison_surface("parent", "child") is None
def test_both_oov_returns_none() -> None:
"""Neither lemma resolves — partial-grounding has nothing to
anchor on. Caller routes to the OOV invitation."""
assert partial_comparison_surface("photosynthesis", "mitochondria") is None
assert partial_comparison_surface("aaa", "bbb") is None
def test_identical_lemmas_return_none() -> None:
"""Same-lemma comparison has no contrastive evidence at any tier."""
assert partial_comparison_surface("knowledge", "knowledge") is None
@pytest.mark.parametrize("bad", ["", " ", None])
def test_empty_or_invalid_lemma_returns_none(bad) -> None:
assert partial_comparison_surface(bad, "knowledge") is None # type: ignore[arg-type]
assert partial_comparison_surface("knowledge", bad) is None # type: ignore[arg-type]
def test_surface_is_deterministic() -> None:
a = partial_comparison_surface("photosynthesis", "knowledge")
b = partial_comparison_surface("photosynthesis", "knowledge")
assert a == b
def test_oov_side_is_safe_displayed() -> None:
"""The OOV token comes from user input and must pass through the
safe-display sanitiser; control chars do not leak."""
result = partial_comparison_surface("evil\x00token", "knowledge")
assert result is not None
surface, _ = result
assert "\x00" not in surface
# ---------------------------------------------------------------------------
# Live runtime — COMPARISON with mixed-residency lemmas
# ---------------------------------------------------------------------------
def test_runtime_comparison_known_oov_routes_to_partial() -> None:
rt = ChatRuntime()
resp = rt.chat("Compare knowledge and photosynthesis.")
assert resp.grounding_source == "partial"
assert "photosynthesis" in resp.surface
assert "knowledge" in resp.surface
def test_runtime_comparison_oov_known_routes_to_partial() -> None:
rt = ChatRuntime()
resp = rt.chat("Compare photosynthesis and knowledge.")
assert resp.grounding_source == "partial"
assert "knowledge" in resp.surface
def test_runtime_comparison_both_known_routes_to_pack() -> None:
"""Mixed residency triggers partial; both-known still hits the
full pack-grounded comparison composer (ADR-0050)."""
rt = ChatRuntime()
resp = rt.chat("Compare knowledge and truth.")
assert resp.grounding_source == "pack"
assert "contrasts with" in resp.surface
def test_runtime_comparison_both_oov_routes_to_oov() -> None:
rt = ChatRuntime()
resp = rt.chat("Compare photosynthesis and mitochondria.")
assert resp.grounding_source == "oov"
def test_runtime_comparison_cross_pack_known_routes_to_pack() -> None:
"""Cross-pack comparison (cognition × relations) still pack-grounds
when both lemmas resolve — the partial tier is a fallback, not
an interception."""
rt = ChatRuntime()
resp = rt.chat("Compare knowledge and parent.")
assert resp.grounding_source == "pack"
def test_runtime_terminal_punctuation_does_not_defeat_resolution() -> None:
"""The intent classifier may leave a trailing period on
secondary_subject ('Compare A and B.'). The runtime strips
terminal sentence punctuation at the COMPARISON boundary so
resolution finds the underlying lemma."""
rt = ChatRuntime()
resp = rt.chat("Compare knowledge and truth.")
# Without normalization, "truth." was OOV → fired partial.
# With normalization, "truth" resolves → pack.
assert resp.grounding_source == "pack"

17
tests/test_procedure_surface.py
View file

@ -147,13 +147,20 @@ def test_procedure_define_010_now_emits_concept() -> None:
assert "concept" in response.surface.lower()
def test_procedure_with_no_pack_lemma_falls_through() -> None:
"""A procedure utterance with no pack-resident lemma still
receives the universal disclosure (no surface fabrication)."""
def test_procedure_with_no_pack_lemma_routes_to_oov_invitation() -> None:
"""ADR-0065 / P2.1 — a procedure utterance with no pack-resident
lemma now routes to the OOV invitation surface (names the unknown
topic, points at PackMutationProposal path) instead of the flat
universal disclosure. No surface fabrication: the invitation
only references the OOV token and the mounted-pack list."""
rt = ChatRuntime()
response = rt.chat("How do I do stuff?")
assert response.grounding_source == "none"
assert "insufficient grounding" in response.surface.lower()
# Either UNKNOWN-intent → "none", or PROCEDURE-intent on OOV
# subject → "oov" invitation. Both honour the no-fabrication
# contract; the surface text differs by intent classification.
assert response.grounding_source in {"oov", "none"}
if response.grounding_source == "oov":
assert "PackMutationProposal" in response.surface
def test_procedure_verify_a_claim_grounds() -> None:

9
tests/test_teaching_grounding.py
View file

@ -181,11 +181,14 @@ def test_cold_start_verification_memory_returns_teaching_surface() -> None:
assert "recall" in resp.surface
def test_cold_start_cause_unknown_subject_disclosure() -> None:
def test_cold_start_cause_unknown_subject_routes_to_oov_invitation() -> None:
"""ADR-0065 / P2.1 — CAUSE on an OOV subject routes through the
OOV invitation surface (subject is OOV → no chain → fall-through
to OOV). Pre-P2.1 this returned the universal disclosure."""
rt = ChatRuntime()
resp = rt.chat("Why does dragon exist?")
assert resp.surface == _UNKNOWN_DOMAIN_SURFACE
assert resp.grounding_source == "none"
assert resp.grounding_source == "oov"
assert "dragon" in resp.surface
def test_turn_event_carries_grounding_source_teaching() -> None: