51aad0c2cd
Replaces the flat "I don't know — insufficient grounding" disclosure
with a deterministic gradient that names specific vocabulary gaps
and gives operators concrete next steps.
P2.1 — OOV "teach me" surface (chat/oov_surface.py).
When the intent classifier extracts a clean subject lemma but that
lemma is not resident in any mounted lexicon pack, the runtime now
emits a deterministic learning-invitation surface tagged
``grounding_source="oov"`` instead of the universal disclosure.
Surface format (fixed template):
"I haven't learned '{token}' yet (intent: {intent}).
Mounted lexicon packs: {pack_list}.
Teach me via a reviewed PackMutationProposal."
The OOV token passes through ``core._safe_display.safe_display``
before persistence — user-input sanitization at the trust boundary.
No vocabulary is invented; no domain is inferred. Honours the
ADR-0027 proposal-only invariant: the surface invites a reviewed
pack mutation, never silently mutates any pack.
Refactored ``_maybe_pack_grounded_surface`` so every existing
intent branch (COMPARISON / CAUSE / VERIFICATION / CORRECTION /
PROCEDURE / DEFINITION+RECALL) falls through on a None composer
result instead of early-returning. The OOV invitation is the
deterministic fall-through for any clean-subject prompt whose
subject doesn't resolve.
P2.2 — Partial-grounding tier (chat/partial_surface.py).
When exactly one of two COMPARISON lemmas resolves, the runtime
emits a hedged surface that grounds the known side verbatim and
disclaims the OOV side explicitly:
"Whatever '{oov}' is, I can ground '{known}' — pack-grounded
({pack_id}): {d1}; {d2}. I cannot ground the comparison
without learning '{oov}' — teach me via a reviewed
PackMutationProposal."
Tagged ``grounding_source="partial"``. Falls through to OOV
invitation when both lemmas are OOV, and to full pack-grounded
COMPARISON when both resolve — partial is the middle tier in the
five-tier gradient.
Also normalises trailing sentence punctuation on
intent.secondary_subject at the COMPARISON boundary so prompts
like "Compare A and B." (with the period) still resolve B
correctly.
Five-tier gradient (vault → teaching → pack → partial → oov → none).
Test debt retired: four pre-existing tests asserted "OOV → universal
disclosure", which is exactly the contract P2.1/P2.2 inverted.
Rewritten to the new contract. Plus test_procedure_surface.py
gained a test for the OOV gradient on procedure intents.
Verification:
tests/test_oov_surface.py 22 passed
tests/test_partial_surface.py 16 passed
Cognition eval byte-identical:
public 100% / 100% / 91.7% / 100%
holdout 100% / 100% / 83.3% / 100%
Curated lanes all green.
145 lines
5.5 KiB
Python
145 lines
5.5 KiB
Python
"""Phase 2.2 — partial-grounding tier tests.
|
||
|
||
The contract these tests pin:
|
||
|
||
- When exactly one of the two compared lemmas resolves in a
|
||
mounted pack, the partial composer emits a hedged surface
|
||
grounding the known side and disclaiming the OOV side.
|
||
- When both resolve OR neither resolves, the composer returns
|
||
``None`` — the caller routes through the full pack-grounded
|
||
composer or the OOV invitation respectively.
|
||
- The OOV token is sanitised through ``safe_display``.
|
||
- Live runtime: COMPARISON with mixed-residency lemmas tags
|
||
``grounding_source="partial"``.
|
||
- Terminal punctuation on secondary_subject does not defeat
|
||
resolution.
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
import pytest
|
||
|
||
from chat.partial_surface import partial_comparison_surface
|
||
from chat.runtime import ChatRuntime
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# Pure-function contract
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_oov_first_known_second_emits_partial() -> None:
|
||
result = partial_comparison_surface("photosynthesis", "knowledge")
|
||
assert result is not None
|
||
surface, known_side = result
|
||
assert known_side == "b"
|
||
assert "knowledge" in surface
|
||
assert "photosynthesis" in surface
|
||
assert "Whatever 'photosynthesis' is" in surface
|
||
assert "pack-grounded (en_core_cognition_v1)" in surface
|
||
assert "PackMutationProposal" in surface
|
||
|
||
|
||
def test_known_first_oov_second_emits_partial() -> None:
|
||
result = partial_comparison_surface("knowledge", "photosynthesis")
|
||
assert result is not None
|
||
surface, known_side = result
|
||
assert known_side == "a"
|
||
assert "Whatever 'photosynthesis' is" in surface
|
||
assert "I can ground 'knowledge'" in surface
|
||
|
||
|
||
def test_both_known_returns_none() -> None:
|
||
"""Both lemmas resolve — caller should route through the full
|
||
pack-grounded comparison composer instead."""
|
||
assert partial_comparison_surface("knowledge", "truth") is None
|
||
assert partial_comparison_surface("parent", "child") is None
|
||
|
||
|
||
def test_both_oov_returns_none() -> None:
|
||
"""Neither lemma resolves — partial-grounding has nothing to
|
||
anchor on. Caller routes to the OOV invitation."""
|
||
assert partial_comparison_surface("photosynthesis", "mitochondria") is None
|
||
assert partial_comparison_surface("aaa", "bbb") is None
|
||
|
||
|
||
def test_identical_lemmas_return_none() -> None:
|
||
"""Same-lemma comparison has no contrastive evidence at any tier."""
|
||
assert partial_comparison_surface("knowledge", "knowledge") is None
|
||
|
||
|
||
@pytest.mark.parametrize("bad", ["", " ", None])
|
||
def test_empty_or_invalid_lemma_returns_none(bad) -> None:
|
||
assert partial_comparison_surface(bad, "knowledge") is None # type: ignore[arg-type]
|
||
assert partial_comparison_surface("knowledge", bad) is None # type: ignore[arg-type]
|
||
|
||
|
||
def test_surface_is_deterministic() -> None:
|
||
a = partial_comparison_surface("photosynthesis", "knowledge")
|
||
b = partial_comparison_surface("photosynthesis", "knowledge")
|
||
assert a == b
|
||
|
||
|
||
def test_oov_side_is_safe_displayed() -> None:
|
||
"""The OOV token comes from user input and must pass through the
|
||
safe-display sanitiser; control chars do not leak."""
|
||
result = partial_comparison_surface("evil\x00token", "knowledge")
|
||
assert result is not None
|
||
surface, _ = result
|
||
assert "\x00" not in surface
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# Live runtime — COMPARISON with mixed-residency lemmas
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_runtime_comparison_known_oov_routes_to_partial() -> None:
|
||
rt = ChatRuntime()
|
||
resp = rt.chat("Compare knowledge and photosynthesis.")
|
||
assert resp.grounding_source == "partial"
|
||
assert "photosynthesis" in resp.surface
|
||
assert "knowledge" in resp.surface
|
||
|
||
|
||
def test_runtime_comparison_oov_known_routes_to_partial() -> None:
|
||
rt = ChatRuntime()
|
||
resp = rt.chat("Compare photosynthesis and knowledge.")
|
||
assert resp.grounding_source == "partial"
|
||
assert "knowledge" in resp.surface
|
||
|
||
|
||
def test_runtime_comparison_both_known_routes_to_pack() -> None:
|
||
"""Mixed residency triggers partial; both-known still hits the
|
||
full pack-grounded comparison composer (ADR-0050)."""
|
||
rt = ChatRuntime()
|
||
resp = rt.chat("Compare knowledge and truth.")
|
||
assert resp.grounding_source == "pack"
|
||
assert "contrasts with" in resp.surface
|
||
|
||
|
||
def test_runtime_comparison_both_oov_routes_to_oov() -> None:
|
||
rt = ChatRuntime()
|
||
resp = rt.chat("Compare photosynthesis and mitochondria.")
|
||
assert resp.grounding_source == "oov"
|
||
|
||
|
||
def test_runtime_comparison_cross_pack_known_routes_to_pack() -> None:
|
||
"""Cross-pack comparison (cognition × relations) still pack-grounds
|
||
when both lemmas resolve — the partial tier is a fallback, not
|
||
an interception."""
|
||
rt = ChatRuntime()
|
||
resp = rt.chat("Compare knowledge and parent.")
|
||
assert resp.grounding_source == "pack"
|
||
|
||
|
||
def test_runtime_terminal_punctuation_does_not_defeat_resolution() -> None:
|
||
"""The intent classifier may leave a trailing period on
|
||
secondary_subject ('Compare A and B.'). The runtime strips
|
||
terminal sentence punctuation at the COMPARISON boundary so
|
||
resolution finds the underlying lemma."""
|
||
rt = ChatRuntime()
|
||
resp = rt.chat("Compare knowledge and truth.")
|
||
# Without normalization, "truth." was OOV → fired partial.
|
||
# With normalization, "truth" resolves → pack.
|
||
assert resp.grounding_source == "pack"
|