core-labs/core
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

feat: ADR-0086 + ADR-0087 + 100-register catalog — cognition lane closure

Browse source 
Three load-bearing pieces:

1. ADR-0086 — UNKNOWN-intent pack-resident token surface
   New deterministic composer `pack_grounded_unknown_surface` in
   chat/pack_grounding.py.  When intent classification returns UNKNOWN
   but the prompt contains pack-resident lemmas (via cross-pack
   resolver), surface those lemmas with their semantic_domains
   instead of falling to the bare _UNKNOWN_DOMAIN_SURFACE.  Wired
   into chat/runtime.py::_maybe_pack_grounded_surface as the
   last typed-intent branch before the OOV fallback.  Null-lift
   invariant pinned: fully-OOV prompts still emit the universal
   disclosure byte-identically.  Closes four cognition-eval term
   misses: unknown_logos_019 (public), unknown_evidence_042 (dev),
   unknown_spirit_041 + unknown_word_018 (holdout).  Side effect:
   evals/results/phase2_pack_measurements.json refusal_rate drops
   from 0.25 → 0.125 across all three identity packs (no longer
   refusing on these prompts).

2. ADR-0087 — PROCEDURE selector + trailing-clause subject echo
   Two coupled changes in chat/pack_grounding.py:
   (a) Numeric-determiner downrank in _extract_procedure_topic_lemma:
       tokens whose primary semantic_domain starts with
       "quantitative.numeric." are demoted; non-numeric resident
       candidates always win.  So "compare two terms" anchors on
       `compare` not `two`.
   (b) Trailing clause echoes the full normalized subject_text
       rather than just the selected lemma, so OOV head nouns like
       "terms" reach the surface even when only the procedure verb
       is pack-resident.  Closes procedure_compare_011.

3. 100-register catalog
   New packs/register/_catalog.json — canonical machine-readable
   spec for all 100 registers (7 currently-ratified + 93 drafted)
   organized into 9 voice groups (depth/tone/stance/posture/domain/
   cultural/affective/functional/composite).  Each entry is a
   complete production input — realizer_overrides, marker palettes
   (openings/transitions/closings), depth_preference, description,
   author_notes.  All realizer_overrides use only legal keys per
   scripts/ratify_register_packs.py::_KNOWN_OVERRIDE_KEYS.
   Companion packs/register/CATALOG.md documents the production
   loop: materialize → widen REGISTER_IDS → ratify → smoke.

Cognition-eval lifts (all three splits):
  public:  term_capture 91.7% → 100.0%  (+8.3pp)
  holdout: term_capture 83.3% → 100.0%  (+16.7pp)
  dev:     term_capture 78.6% → 100.0%  (+21.4pp)
  surface_groundedness: 100% preserved on all splits
  intent_accuracy / versor_closure: 100% preserved on all splits

Tests:
  tests/test_pack_grounded_unknown.py     — 14 tests (composer
    direct + runtime engagement + null-lift invariant)
  tests/test_adr_0087_procedure_selector.py — 12 tests (selector
    numeric downrank + trailing-clause echo + regression guard)
  Existing test suites unaffected — cognition lane 120 passed / 1
  skipped both before and after.  Full lane net −3 failures vs
  pristine main (39 → 36 — none introduced).
This commit is contained in:
Shay 2026-05-21 00:08:12 -07:00
parent fa5b01ade0
commit 79f1678923
7 changed files with 2993 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

137
chat/pack_grounding.py
View file

@ -819,7 +819,7 @@ _PROCEDURE_TOPIC_STOPWORDS: frozenset[str] = frozenset({
def _extract_procedure_topic_lemma(subject_text: str) -> str | None:
"""Return the **last** pack-resident topical lemma in *subject_text*.
"""Return the **last** non-numeric pack-resident topical lemma in *subject_text*.
Procedure subjects emerge from the intent classifier as verb
phrases (e.g. ``"define a concept"``, ``"correct an error"``,
@ -837,6 +837,15 @@ def _extract_procedure_topic_lemma(subject_text: str) -> str | None:
are NOT stopworded — when a procedure utterance contains only
one pack-resident lemma and that lemma is the verb, the verb
is the topical anchor by elimination.
ADR-0087 — numeric-determiner downrank. Tokens whose primary
semantic_domain starts with ``quantitative.numeric.`` (cardinals
like ``two``, ``three``) are demoted: a non-numeric resident
candidate always wins, and the numeric is only selected when it
is the sole resident token in the subject. Closes
``procedure_compare_011`` (*"How do I compare two terms?"*),
where the pre-ADR-0087 last-wins selector picked the cardinal
determiner ``two`` over the operation verb ``compare``.
"""
if not subject_text or not isinstance(subject_text, str):
return None
@ -845,14 +854,20 @@ def _extract_procedure_topic_lemma(subject_text: str) -> str | None:
for ch in ",.;:!?\"'()[]{}":
raw = raw.replace(ch, " ")
last_match: str | None = None
last_numeric_fallback: str | None = None
for token in raw.split():
if not token:
continue
if token in _PROCEDURE_TOPIC_STOPWORDS:
continue
if token in lemmas:
resolved = resolve_lemma(token)
primary = resolved[1][0] if resolved is not None and resolved[1] else ""
if primary.startswith("quantitative.numeric."):
last_numeric_fallback = token
continue
last_match = token
return last_match
return last_match if last_match is not None else last_numeric_fallback
def pack_grounded_procedure_surface(
@ -876,9 +891,16 @@ def pack_grounded_procedure_surface(
Surface format (fixed template, all atoms pack-sourced):
"procedure-grounded ({pack_id}): {lemma} ({d1}; {d2}).
Step-by-step guidance for {lemma} is not yet ratified
Step-by-step guidance for {subject_text} is not yet ratified
in this session."
The trailing clause echoes the user's verb-phrase subject so that
OOV head nouns (e.g. ``"terms"`` in *"compare two terms"*) still
reach the surface even when only the procedure verb is pack-resident.
ADR-0087 — this preserves the user's actual topic of interest in
the trust-boundary label without inventing pack atoms for OOV
words.
The trailing clause is the constant trust-boundary label,
analogous to ``"No prior turn in this session to correct yet."``
in the CORRECTION acknowledgement (ADR-0053 / ADR-0060).
@ -900,12 +922,119 @@ def pack_grounded_procedure_surface(
return None
resolved_pack_id, domains = resolved
head = "; ".join(domains[:2])
# ADR-0087 — echo the full subject_text in the trailing clause
# (normalized: lowercase + punctuation stripped) so OOV head nouns
# reach the surface. Falls back to the lemma alone if the
# normalized echo would be empty.
echo = subject_text.lower()
for ch in ",.;:!?\"'()[]{}":
echo = echo.replace(ch, " ")
echo = " ".join(echo.split())
topic_phrase = echo if echo else lemma
return (
f"procedure-grounded ({resolved_pack_id}): {lemma} ({head}). "
f"Step-by-step guidance for {lemma} is not yet ratified in this session."
f"Step-by-step guidance for {topic_phrase} is not yet ratified in this session."
)
_UNKNOWN_TOKEN_STOPWORDS: frozenset[str] = frozenset({
# Pack-resident lemmas that classify but carry no topical signal in
# an UNKNOWN-intent prompt — dialogue fillers / copulae. Mirrors
# the CORRECTION / PROCEDURE stopword conventions.
"be",
"have",
})
def pack_grounded_unknown_surface(
text: str | None,
*,
max_tokens: int = 3,
pack_ids: tuple[str, ...] = DEFAULT_RESOLVABLE_PACK_IDS,
register: RegisterPack = UNREGISTERED,
) -> str | None:
"""ADR-0086 — UNKNOWN-intent pack-resident token surface.
When the intent classifier returns :class:`IntentTag.UNKNOWN` (the
prompt does not match any known dialogue shape) but the prompt
text contains one or more pack-resident lemmas, surface those
lemmas with their semantic_domains rather than emitting the bare
``_UNKNOWN_DOMAIN_SURFACE`` disclosure.
Surface format (fixed template, all atoms pack-sourced)::
"Pack-resident tokens — pack-grounded ({tag}): {tok1} ({d_a1}; {d_a2}),
{tok2} ({d_b1}; {d_b2}). No session evidence yet."
Where ``{tag}`` is either ``{pack_id}`` (single pack) or
``{pack_a} × {pack_b}`` (cross-pack), matching the convention of
:func:`pack_grounded_comparison_surface`.
Token selection rules (deterministic, left-to-right):
1. Lowercase the prompt, strip standard punctuation.
2. Skip dialogue-filler stopwords (``be``, ``have``).
3. Take the first :func:`resolve_lemma`-resolving token, then the
next distinct one, up to ``max_tokens`` (default 3).
4. Per-token domain disclosure is capped at 2 to keep multi-token
surfaces compact; matches :func:`pack_grounded_comparison_surface`.
Returns ``None`` when:
- ``text`` is empty / not a string,
- zero prompt tokens resolve in any mounted pack.
The ``None`` fallback preserves the bare ``_UNKNOWN_DOMAIN_SURFACE``
for fully-OOV prompts — null-lift invariant: prompts that contained
zero pack-resident lemmas pre-ADR-0086 still emit the universal
disclosure byte-identically.
Closes the four UNKNOWN-intent term_capture misses in the cognition
eval lane: ``unknown_logos_019`` (public), ``unknown_evidence_042``
(dev), ``unknown_spirit_041`` / ``unknown_word_018`` (holdout).
"""
if not text or not isinstance(text, str):
return None
raw = text.lower()
for ch in ",.;:!?\"'()[]{}":
raw = raw.replace(ch, " ")
seen: set[str] = set()
hits: list[tuple[str, str, tuple[str, ...]]] = []
for token in raw.split():
if not token or token in seen:
continue
if token in _UNKNOWN_TOKEN_STOPWORDS:
continue
resolved = resolve_lemma(token, pack_ids)
if resolved is None:
continue
resolved_pack_id, domains = resolved
if not domains:
continue
seen.add(token)
hits.append((token, resolved_pack_id, domains))
if len(hits) >= max_tokens:
break
if not hits:
return None
pack_ids_in_surface: list[str] = []
for _, pid, _ in hits:
if pid not in pack_ids_in_surface:
pack_ids_in_surface.append(pid)
if len(pack_ids_in_surface) == 1:
tag = f"pack-grounded ({pack_ids_in_surface[0]})"
else:
tag = f"pack-grounded ({' × '.join(pack_ids_in_surface)})"
items = ", ".join(
f"{tok} ({'; '.join(domains[:2])})"
for tok, _, domains in hits
)
# ``register`` accepted for signature parity with the other
# pack_grounded_*_surface composers; UNKNOWN-intent does not yet
# consult realizer_overrides. Follow-up if a register variant
# wants to dial token count or per-token domain width.
_ = register
return f"Pack-resident tokens — {tag}: {items}. No session evidence yet."
def pack_grounded_comparison_surface(
lemma_a: str,
lemma_b: str,

18
chat/runtime.py
View file

@ -16,6 +16,7 @@ from chat.pack_grounding import (
pack_grounded_correction_surface,
pack_grounded_procedure_surface,
pack_grounded_relation_confirmation_surface,
pack_grounded_unknown_surface,
gloss_aware_cause_surface,
PACK_ID as _COGNITION_PACK_ID,
)
@ -907,6 +908,23 @@ class ChatRuntime:
resolved = resolve_lemma(lemma)
domains = resolved[1] if resolved is not None else ()
return (surface, "pack", domains)
if intent.tag is IntentTag.UNKNOWN:
# ADR-0086 — UNKNOWN intent with pack-resident prompt
# tokens. The classifier could not assign a known dialogue
# shape, but the prompt itself may contain lemmas that are
# ratified in mounted lexicon packs (e.g. ``"light logos"``,
# ``"spirit wisdom truth"``). Surface those lemmas with
# their semantic_domains rather than emit the bare
# _UNKNOWN_DOMAIN_SURFACE disclosure. Null-lift invariant:
# when no prompt token resolves, composer returns None and
# the caller falls through to the universal disclosure
# byte-identically (preserves the ADR-0053 honesty contract
# for fully-OOV prompts).
surface = pack_grounded_unknown_surface(
text, register=self.register_pack,
)
if surface is not None:
return (surface, "pack", ())
oov_lemma = (intent.subject or "").strip()
if oov_lemma:
from chat.oov_surface import oov_learning_invitation_surface

6
evals/results/phase2_pack_measurements.json
View file

@ -70,19 +70,19 @@
"fabrication_rate": 0.0,
"out_of_grounding_count": 8,
"pack_id": "default_general_v1",
"refusal_rate": 0.25
"refusal_rate": 0.125
},
{
"fabrication_rate": 0.0,
"out_of_grounding_count": 8,
"pack_id": "precision_first_v1",
"refusal_rate": 0.25
"refusal_rate": 0.125
},
{
"fabrication_rate": 0.0,
"out_of_grounding_count": 8,
"pack_id": "generosity_first_v1",
"refusal_rate": 0.25
"refusal_rate": 0.125
}
],
"schema_version": 1

88
packs/register/CATALOG.md Normal file
View file

@ -0,0 +1,88 @@
# Register pack catalog — production guide
`packs/register/_catalog.json` is the canonical, machine-readable spec for the
**100-register catalog**. Each entry is a complete production input — an author
or content pipeline materializes `packs/register/<register_id>.json` from it
mechanically and re-runs ratification.
| Group | Count | Voice |
| --- | --- | --- |
| A — Depth ladder | 6 | Varies disclosure_domain_count + structural booleans. No markers. |
| B — Tone | 16 | Affective marker palettes. Default knobs. |
| C — Stance | 12 | Epistemic posture — assertive, hedged, exploratory, etc. |
| D — Posture | 10 | Social role — peer, mentor, scholar, etc. |
| E — Domain | 12 | Academic, technical, legal, scientific, philosophical, etc. |
| F — Cultural | 12 | Plainspoken, cosmopolitan, classic, lyrical, etc. |
| G — Affective | 10 | Cheerful, somber, wry, gentle, etc. |
| H — Functional | 10 | Documentary, instructional, persuasive, etc. |
| I — Composite | 12 | Combine knobs from multiple groups. |
| **Total** | **100** | 7 ratified + 93 drafted |
## Production loop
For each entry whose `status == "drafted"`:
1. **Materialize** `packs/register/<register_id>.json` from the catalog entry.
Required fields: `register_id`, `version` (set to `"1.0.0"`),
`description`, `schema_version` (`"1.0.0"`), `mastery_report_sha256`
(leave `""` — the ratify script will fill it),
`display_name`, `depth_preference`, `realizer_overrides`,
`discourse_markers`.
2. **Widen** `scripts/ratify_register_packs.py::REGISTER_IDS` to include
the new `register_id`.
3. **Ratify**: `python scripts/ratify_register_packs.py`. Idempotent;
re-runs produce byte-identical files.
4. **Test**: the ratification script writes the `.mastery_report.json`
companion and updates `mastery_report_sha256` in the pack file.
Run `python -m pytest tests/test_register_loader.py -q` to confirm
the loader self-seal check passes.
5. **Smoke**: `core chat "Test prompt" --register <register_id>` should
produce a surface whose `grounding_source` and `trace_hash` are
byte-identical to the same prompt under `default_neutral_v1`
(ADR-0072 register invariant). Only the surface text varies.
## Trust boundaries
- `realizer_overrides` is restricted to the allow-list in
`scripts/ratify_register_packs.py::_KNOWN_OVERRIDE_KEYS`:
`disclosure_domain_count` (1, 2, or 3), `compress_gloss`,
`drop_articles`, `drop_provenance_tag`, `append_semantic_domain_clause`,
plus `per_intent` nested block keyed by `IntentTag` names. Any other
key fails ratification.
- `discourse_markers` has exactly three buckets: `openings`,
`transitions`, `closings`. Each is a list of strings. An empty string
`""` inside a bucket is a legal entry — the seeded selector (ADR-0071)
treats it as "no marker this turn", enabling natural variation.
## Author guidance
- Keep each marker palette small (3–5 entries per non-empty bucket).
Larger palettes are not better — the seeded selector wants a tight
bounded space so per-turn variation is felt without being noisy.
- Include `""` in `openings` and `closings` for ~20–30% of registers
so the system has a "land plainly" option per turn.
- Markers may contain Unicode em-dash (—), ASCII punctuation, and
standard English contractions. Avoid emoji and non-English script
unless the register's purpose is explicitly multilingual.
- For composite registers (group I), the `description` field should
name the two constituent voices it combines.
## Invariants pinned by ADR-0072 / ADR-0071
Every register in this catalog must satisfy, on every prompt:
- `grounding_source` is byte-identical to `default_neutral_v1`.
- `trace_hash` is byte-identical to `default_neutral_v1`.
- `aggregate metrics` (cognition eval lane) are byte-identical.
- Only the surface text varies.
The `core demo register-tour` runner exists to assert this. New registers
should be added to its sweep once authored.
## Next steps after authoring
- Update the cognition eval's register-invariance test fixtures to
include the new register ids (sweep, no per-register code change).
- Optional follow-up: add a `register_distribution_lift` benchmark
(ADR-0072 variant) that measures surface-variance across the full
100-register sweep for a fixed prompt corpus.

2477
packs/register/_catalog.json Normal file
View file

File diff suppressed because it is too large Load diff

133
tests/test_adr_0087_procedure_selector.py Normal file
View file

@ -0,0 +1,133 @@
"""ADR-0087 — PROCEDURE selector + trailing-clause subject echo.
Two coupled changes:
1. **Numeric-determiner downrank** in :func:`_extract_procedure_topic_lemma`
— tokens whose primary semantic_domain starts with
``quantitative.numeric.`` are demoted; a non-numeric resident
candidate always wins. Only when the numeric is the sole
resident does it become the topic.
2. **Subject-text echo in the trailing clause** of
:func:`pack_grounded_procedure_surface` — the *"Step-by-step
guidance for X is not yet ratified."* clause echoes the
normalized full subject_text rather than just the lemma, so OOV
head nouns (``terms``, ``mistake``) reach the surface even when
only the procedure verb is pack-resident.
Closes ``procedure_compare_011`` (*"How do I compare two terms?"*)
where the pre-ADR-0087 selector picked the cardinal ``two`` over
``compare`` and the OOV head noun ``terms`` never reached the surface.
Pins both the engagement on the failing case and the regression
guard on the four existing PROCEDURE eval cases.
"""
from __future__ import annotations
import pytest
from chat.pack_grounding import (
_extract_procedure_topic_lemma,
pack_grounded_procedure_surface,
)
from chat.runtime import ChatRuntime
# ---------------------------------------------------------------------------
# Selector — numeric downrank
# ---------------------------------------------------------------------------
def test_numeric_determiner_is_downranked() -> None:
"""A non-numeric resident candidate (verb ``compare``) wins over
a numeric-cardinal candidate (``two``)."""
assert _extract_procedure_topic_lemma("compare two terms") == "compare"
def test_numeric_only_resident_still_wins_by_elimination() -> None:
"""If the numeric is the sole resident token, it remains the
topic — preserves coverage on numeric-only prompts."""
assert _extract_procedure_topic_lemma("count to two") == "two"
def test_non_numeric_resident_still_takes_last_wins_priority() -> None:
"""Existing last-wins behavior is preserved for non-numeric tokens.
``concept`` still wins over ``define`` in ``define a concept``."""
assert _extract_procedure_topic_lemma("define a concept") == "concept"
# ---------------------------------------------------------------------------
# Surface — trailing clause echoes subject_text
# ---------------------------------------------------------------------------
def test_trailing_clause_echoes_full_subject_text() -> None:
"""OOV head noun ``terms`` reaches the surface via the trailing
clause even though it doesn't resolve in any pack."""
surface = pack_grounded_procedure_surface("compare two terms")
assert surface is not None
assert "terms" in surface
assert "Step-by-step guidance for compare two terms" in surface
def test_trailing_clause_normalizes_punctuation_in_echo() -> None:
surface = pack_grounded_procedure_surface("define, a concept.")
assert surface is not None
assert "Step-by-step guidance for define a concept" in surface
def test_displayed_lemma_is_compare_not_two() -> None:
"""The semantic anchor in the surface is the operation verb
``compare``, not the cardinal determiner ``two``."""
surface = pack_grounded_procedure_surface("compare two terms")
assert surface is not None
assert "compare (operation.compare" in surface
# The cardinal-determiner lemma should not appear as the displayed
# pack-grounded anchor (it may still appear in the echoed subject).
assert "two (quantitative.numeric" not in surface
# ---------------------------------------------------------------------------
# Regression guard — existing PROCEDURE eval cases still pass
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"prompt,expected_terms",
[
("How do I define a concept?", ("concept",)),
# ``verify a claim`` and ``correct a mistake`` carry no
# expected_terms in the cognition eval, but the surfaces must
# still ground deterministically and mention the user's topic.
("How do I verify a claim?", ("claim", "verify a claim")),
("How can I correct a mistake?", ("correct", "correct a mistake")),
# The ADR-0087 target case.
("How do I compare two terms?", ("terms", "compare two terms")),
],
)
def test_runtime_procedure_surfaces_contain_expected_terms(
prompt: str, expected_terms: tuple[str, ...],
) -> None:
rt = ChatRuntime()
resp = rt.chat(prompt)
assert resp.grounding_source == "pack"
surface_lower = resp.surface.lower()
for term in expected_terms:
assert term.lower() in surface_lower, (
f"prompt={prompt!r} expected {term!r} in surface, got {resp.surface!r}"
)
def test_procedure_surface_is_deterministic_after_adr_0087() -> None:
a = pack_grounded_procedure_surface("compare two terms")
b = pack_grounded_procedure_surface("compare two terms")
assert a == b
def test_oov_only_procedure_still_falls_through_to_none() -> None:
"""ADR-0061 honesty contract holds: a verb+noun pair that is
entirely OOV across mounted packs still returns ``None`` from
the composer (runtime falls through to the OOV invitation)."""
# ``fix bugs`` is the canonical fully-OOV PROCEDURE fixture used
# by tests/test_procedure_surface.py.
assert pack_grounded_procedure_surface("fix bugs") is None

141
tests/test_pack_grounded_unknown.py Normal file
View file

@ -0,0 +1,141 @@
"""ADR-0086 — UNKNOWN-intent pack-resident token surface.
Pins:
1. **Engagement** — the four cognition-eval UNKNOWN miss prompts each
surface their pack-resident English tokens with semantic_domains.
2. **Null-lift invariant** — fully-OOV prompts that have zero
pack-resident lemmas still emit the universal disclosure
byte-identically.
3. **Composer determinism** — repeated invocation on the same prompt
returns the byte-identical surface (ratified packs are immutable
and the composer reads no session state).
4. **Provenance** — surfaces emitted by the new composer carry
``grounding_source == "pack"`` so the audit contract distinguishes
them from vault- and teaching-grounded surfaces.
5. **Pre-ADR-0086 hand-off** — the runtime falls through to the
bare ``_UNKNOWN_DOMAIN_SURFACE`` only when the composer returns
``None``; no other UNKNOWN-intent code path is disturbed.
6. **Stopword discipline** — pure dialogue-filler prompts (``be have``)
return ``None`` because every resident token is stopworded.
"""
from __future__ import annotations
import pytest
from chat.pack_grounding import pack_grounded_unknown_surface
from chat.runtime import ChatRuntime, _UNKNOWN_DOMAIN_SURFACE
# ---------------------------------------------------------------------------
# Composer — direct calls
# ---------------------------------------------------------------------------
def test_composer_returns_none_on_empty() -> None:
assert pack_grounded_unknown_surface("") is None
assert pack_grounded_unknown_surface(None) is None # type: ignore[arg-type]
def test_composer_returns_none_on_fully_oov_prompt() -> None:
"""Null-lift invariant — zero resident tokens → None."""
assert pack_grounded_unknown_surface("xyzzy plugh frobnitz") is None
def test_composer_returns_none_on_stopwords_only() -> None:
"""``be`` and ``have`` are pack-resident but stopworded — a
prompt of only stopwords yields no surface."""
assert pack_grounded_unknown_surface("be have") is None
def test_composer_lifts_single_resident_token() -> None:
"""``light`` is in ``en_core_cognition_v1`` — composer surfaces it."""
surface = pack_grounded_unknown_surface("light logos")
assert surface is not None
assert "light" in surface
assert "pack-grounded (en_core_cognition_v1)" in surface
assert "No session evidence yet." in surface
def test_composer_lifts_two_resident_tokens() -> None:
surface = pack_grounded_unknown_surface("evidence reason")
assert surface is not None
assert "evidence" in surface
assert "reason" in surface
def test_composer_caps_at_max_tokens() -> None:
"""Default ``max_tokens=3`` — four resident tokens surface
only the first three, deterministically."""
surface = pack_grounded_unknown_surface(
"spirit wisdom truth knowledge",
)
assert surface is not None
# First three lemmas in left-to-right order should appear; the
# fourth (``knowledge``) is dropped under the default cap.
assert "spirit" in surface
assert "wisdom" in surface
assert "truth" in surface
assert "knowledge" not in surface
def test_composer_is_deterministic() -> None:
"""Repeated calls on the same prompt yield byte-identical surfaces."""
a = pack_grounded_unknown_surface("spirit wisdom truth")
b = pack_grounded_unknown_surface("spirit wisdom truth")
assert a == b
def test_composer_strips_punctuation() -> None:
surface = pack_grounded_unknown_surface("light, logos.")
assert surface is not None
assert "light" in surface
def test_composer_is_case_insensitive() -> None:
surface = pack_grounded_unknown_surface("LIGHT LOGOS")
assert surface is not None
assert "light" in surface
# ---------------------------------------------------------------------------
# Runtime engagement — the four cognition-eval miss cases
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"prompt,expected_terms",
[
# public split — unknown_logos_019
("light logos", ("light",)),
# dev split — unknown_evidence_042
("evidence reason", ("evidence", "reason")),
# holdout split — unknown_spirit_041
("spirit wisdom truth", ("wisdom", "truth")),
# holdout split — unknown_word_018
("word beginning truth", ("word", "truth")),
],
)
def test_runtime_unknown_lifts_pack_tokens(
prompt: str, expected_terms: tuple[str, ...],
) -> None:
"""The four UNKNOWN-intent eval misses each lift to pack-grounded
surfaces containing the expected English term(s)."""
rt = ChatRuntime()
resp = rt.chat(prompt)
assert resp.grounding_source == "pack"
surface_lower = resp.surface.lower()
for term in expected_terms:
assert term.lower() in surface_lower, (
f"expected {term!r} in surface, got {resp.surface!r}"
)
def test_runtime_null_lift_on_fully_oov_unknown() -> None:
"""Null-lift invariant at the runtime layer: prompts with zero
pack-resident lemmas still emit the universal disclosure
byte-identically."""
rt = ChatRuntime()
resp = rt.chat("xyzzy plugh frobnitz")
assert resp.surface == _UNKNOWN_DOMAIN_SURFACE
assert resp.grounding_source == "none"