core-labs/core
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

feat(adr-0050): pack-grounded COMPARISON surface

Browse source 
Sibling to ADR-0048's DEFINITION/RECALL pack-grounded surface for
the COMPARISON intent.  `pack_grounded_comparison_surface(a, b)` in
`chat/pack_grounding.py` composes a deterministic side-by-side
surface from both lemmas' pack `semantic_domains`, joined by the
fixed connective "contrasts with":

  "{a} (d_a1; d_a2) contrasts with {b} (d_b1; d_b2) — pack-grounded
   ({pack_id}). No session evidence yet."

`chat/runtime.py:_maybe_pack_grounded_surface` gains a COMPARISON
branch that runs before the DEFINITION/RECALL check.  Engages only
when both `intent.subject` and `intent.secondary_subject` are pack
lemmas and differ (identical-lemma comparison defers to disclosure).
Order-sensitive by design — matches the graph-layer's directional
CONTRAST edge.

Cognition eval (13-case public split):
  surface_groundedness  61.5% → 69.2%  (+7.7 pp)
  term_capture_rate     50.0% → 58.3%  (+8.3 pp)
  intent_accuracy            100.0%        (=)
  versor_closure_rate        100.0%        (=)

Case lifted: comparison_memory_recall_030 ("Compare memory and
recall").  Remaining unlift cases (CAUSE×2, VERIFICATION×1,
CORRECTION×1) need teaching-store chains or operator-driven
inference — pack lookup cannot supply causal explanations,
verifications, or corrections without fabrication.

Tests: tests/test_pack_grounded_comparison.py (15 tests).
Lanes green: smoke (67), cognition (121), runtime (19), algebra
(132), teaching (17), packs (6).
This commit is contained in:
Shay 2026-05-18 06:59:53 -07:00
parent c8037cfa0d
commit ecd580479a
5 changed files with 472 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

48
chat/pack_grounding.py
View file

@ -118,3 +118,51 @@ def is_pack_lemma(lemma: str) -> bool:
if not lemma or not isinstance(lemma, str):
return False
return lemma.strip().lower() in _pack_index()
def pack_grounded_comparison_surface(
lemma_a: str, lemma_b: str
) -> str | None:
"""ADR-0050 — deterministic pack-grounded surface for COMPARISON intent.
Returns a surface that composes each lemma's pack semantic_domains
side-by-side, with no rewording or inference:
"{a} (d_a1; d_a2) contrasts with {b} (d_b1; d_b2) — pack-grounded
({pack_id}). No session evidence yet."
Up to two semantic_domains per side are emitted to keep the surface
compact. All visible tokens are either the lemmas themselves or
verbatim pack strings; the verb "contrasts with" is the fixed
COMPARISON template constant (mirroring the relation predicate
`contrasts_with` already humanised by ``humanize_predicate``).
Returns ``None`` when:
- either lemma is empty or not a string,
- either lemma is not present in the pack,
- the two lemmas are identical (a comparison between a term
and itself carries no contrastive evidence — defer to the
single-lemma ``pack_grounded_surface`` path or to the
universal disclosure).
"""
if not lemma_a or not isinstance(lemma_a, str):
return None
if not lemma_b or not isinstance(lemma_b, str):
return None
key_a = lemma_a.strip().lower()
key_b = lemma_b.strip().lower()
if not key_a or not key_b:
return None
if key_a == key_b:
return None
index = _pack_index()
domains_a = index.get(key_a)
domains_b = index.get(key_b)
if not domains_a or not domains_b:
return None
head_a = "; ".join(domains_a[:2])
head_b = "; ".join(domains_b[:2])
return (
f"{key_a} ({head_a}) contrasts with {key_b} ({head_b}) "
f"— pack-grounded ({PACK_ID}). No session evidence yet."
)

15
chat/runtime.py
View file

@ -10,7 +10,11 @@ from typing import List
import numpy as np
from algebra.versor import versor_condition
from chat.pack_grounding import pack_grounded_surface, PACK_ID as _COGNITION_PACK_ID
from chat.pack_grounding import (
pack_grounded_surface,
pack_grounded_comparison_surface,
PACK_ID as _COGNITION_PACK_ID,
)
from chat.refusal import (
build_hedge_prefix,
build_refusal_surface,
@ -545,6 +549,15 @@ class ChatRuntime:
from generate.intent import IntentTag # local to avoid coupling at import time
from generate.intent_bridge import classify_intent_from_input
intent = classify_intent_from_input(text)
# ADR-0050 — COMPARISON path: deterministic side-by-side surface
# composed from both lemmas' pack semantic_domains. Engages only
# when both subject and secondary_subject are pack lemmas.
if intent.tag is IntentTag.COMPARISON:
lemma_a = (intent.subject or "").strip()
lemma_b = (intent.secondary_subject or "").strip()
if not lemma_a or not lemma_b:
return None
return pack_grounded_comparison_surface(lemma_a, lemma_b)
if intent.tag not in (IntentTag.DEFINITION, IntentTag.RECALL):
return None
lemma = (intent.subject or "").strip()

254
docs/decisions/ADR-0050-pack-grounded-comparison.md Normal file
View file

@ -0,0 +1,254 @@
# ADR-0050 — Pack-Grounded Surface for Cold-Start COMPARISON
**Status:** Accepted
**Date:** 2026-05-18
**Author:** Shay
---
## Context
[ADR-0048](./ADR-0048-pack-grounded-surface.md) added a pack-grounded
surface for cold-start DEFINITION / RECALL intents — a deterministic,
verbatim composition of the cognition pack's `semantic_domains`
for a single subject lemma. [ADR-0049](./ADR-0049-intent-subject-extraction.md)
then tightened intent classifier subject extraction so that prompts
like `"What is a procedure?"` produce a clean lemma the pack can
match.
The cognition lane's COMPARISON case
(`comparison_memory_recall_030` — `"Compare memory and recall"`)
still missed. Investigation:
- Intent classifier correctly tags the prompt `COMPARISON`,
`subject="memory"`, `secondary_subject="recall"`.
- Both lemmas are in `en_core_cognition_v1` with curated
`semantic_domains`:
- `memory` → `("cognition.memory", "memory.semantic", ...)`
- `recall` → `("operation.recall", "cognition.memory", ...)`
- `_maybe_pack_grounded_surface` was scoped to DEFINITION / RECALL
only; COMPARISON fell through to the universal disclosure.
The structure is exactly the situation ADR-0048's pattern was
designed for: two pack-known lemmas, no session evidence, deterministic
surface compositable entirely from pack atoms. The doctrinally
clean fix is a sibling pack-path branch for COMPARISON, with the
same trust-boundary discipline.
---
## Decision
Add a deterministic COMPARISON-shaped pack-grounded surface as a
second branch of `_maybe_pack_grounded_surface`, identical guardrails
to the DEFINITION / RECALL branch.
### Surface format
```text
{a} ({d_a1}; {d_a2}) contrasts with {b} ({d_b1}; {d_b2}) — pack-grounded ({pack_id}). No session evidence yet.
```
Up to two `semantic_domains` per side are emitted to keep the surface
compact. Every visible non-template token is either one of the two
lemmas or a verbatim pack `semantic_domains` string.
The connective `"contrasts with"` is a fixed-template constant —
identical to the human-readable form of the `contrasts_with` relation
predicate already produced by
`generate/semantic_templates.py:humanize_predicate("contrasts_with")`.
The COMPARISON intent's downstream graph node already uses this
predicate (`graph_planner.graph_from_intent` builds a
`Relation.CONTRAST` edge), so this ADR preserves the existing
COMPARISON connective vocabulary rather than introducing a new one.
### Engagement conditions
`pack_grounded_comparison_surface(a, b)` returns a non-`None` surface
**only** when **all** hold:
- both `a` and `b` are non-empty strings,
- both `a` and `b` are pack lemmas (with `semantic_domains`),
- `a ≠ b` after lowercasing.
`_maybe_pack_grounded_surface` invokes this path **only** when **all**
hold:
- gate fired with `source="empty_vault"` (cold-start session),
- `config.output_language == "en"`,
- intent is `COMPARISON`,
- both `intent.subject` and `intent.secondary_subject` are non-empty.
Any other condition returns `None` and the runtime falls through to
the universal disclosure unchanged. Safety / ethics refusal still
takes priority above this branch.
### Identical-lemma defer
`pack_grounded_comparison_surface("memory", "memory")` returns `None`.
A comparison between a term and itself carries no contrastive
evidence; the universal disclosure is the correct surface in that
case. Callers that want a single-lemma surface for `"X"` should use
`pack_grounded_surface("X")` directly via the DEFINITION / RECALL
path.
### Order sensitivity
The COMPARISON surface is **order-sensitive** by design:
`compare(a, b)` and `compare(b, a)` produce distinct surfaces because
the `"contrasts with"` connective is directional. This matches the
graph-layer behaviour where `Relation.CONTRAST(a → b)` and
`Relation.CONTRAST(b → a)` are distinct edges.
---
## Why this is doctrine-aligned
CLAUDE.md prohibits *opaque LLM fallbacks, stochastic sampling,
hidden normalisation, hot-path repair, and approximate recall*. This
ADR is:
- **Not opaque.** Every visible atom is either a lemma supplied by
the intent classifier or a verbatim pack `semantic_domains` string.
Pack ID is named in the surface.
- **Not stochastic.** Deterministic JSONL read, deterministic string
composition; identical input produces byte-identical output
(`test_comparison_surface_is_deterministic`).
- **Not hidden normalisation.** The pack lookup is a separate source
of grounding, not a normalisation step inside an existing operator.
No versor, no manifold, no field state touched.
- **Not hot-path repair.** `UnknownDomainGate` semantics are
unchanged — the gate still fires; this ADR only broadens what the
stub-path emits *after* the gate fires, in a narrow intent-typed
branch.
- **Not approximate recall.** Exact dictionary lookup on the pack
lexicon by lemma. No metric, no neighbourhood, no threshold.
The fundamental architectural move is the same as ADR-0048: the
cognition pack contributes a second source of grounding alongside
the session vault, with provenance preserved end-to-end. This ADR
extends that contribution from the DEFINITION / RECALL shape to the
COMPARISON shape.
---
## Characterisation — `core eval cognition`
A/B run on the 13-case public cognition split, identical
`RuntimeConfig` except for the merge of this ADR (build on top of
ADR-0049's article-stripping):
| Metric | Pre-ADR-0050 | Post-ADR-0050 | Δ |
|---------------------------|--------------|---------------|-------------|
| `intent_accuracy` | 100.0 % | 100.0 % | 0 |
| `surface_groundedness` | 61.5 % | **69.2 %** | **+7.7 pp** |
| `term_capture_rate` | 50.0 % | **58.3 %** | **+8.3 pp** |
| `versor_closure_rate` | 100.0 % | 100.0 % | 0 |
| `versor_condition < 1e-6` | preserved | preserved | invariant |
The case that lifts is exactly `comparison_memory_recall_030`:
```text
"Compare memory and recall"
-> intent.tag = COMPARISON, subject="memory", secondary_subject="recall"
-> both pack lemmas
-> "memory (cognition.memory; memory.semantic) contrasts with
recall (operation.recall; cognition.memory) — pack-grounded
(en_core_cognition_v1). No session evidence yet."
-> grounding_source = "pack"
```
The remaining unlift cases (CAUSE × 2, VERIFICATION × 1,
CORRECTION × 1) need either teaching-store chains (ADR-0018) or
operator-driven inference — pack lookup cannot supply causal
explanations, verifications, or corrections without fabrication.
---
## Consequences
### What changes
- `chat/pack_grounding.py` gains
`pack_grounded_comparison_surface(lemma_a, lemma_b) -> str | None`.
- `chat/runtime.py:_maybe_pack_grounded_surface` gains a COMPARISON
branch that runs before the DEFINITION / RECALL check.
- One new case lifts in the cognition eval.
### What does not change
- `_UNKNOWN_DOMAIN_SURFACE` constant retained; non-pack-lemma,
non-English, non-DEFINITION/RECALL/COMPARISON paths still return
the universal disclosure unchanged.
- `UnknownDomainGate` semantics unchanged.
- `ChatResponse.grounding_source` / `TurnEvent.grounding_source`
enum unchanged — COMPARISON pack-grounded surfaces carry the same
`"pack"` tag as single-lemma pack-grounded surfaces. Downstream
audit consumers do not need a new value to filter on; they already
distinguish via the surface contents if needed.
- Safety / ethics refusal still takes priority above pack grounding.
- `versor_condition(F) < 1e-6` invariant unaffected (no algebra
changes).
### Scope limits
- English only (`en_core_cognition_v1`). Same constraint as
ADR-0048; multilingual extension is a separate ADR.
- Exactly two operands. ``"Compare a, b, and c"`` is not yet
supported by the COMPARISON regex in `generate/intent.py` and is
out of scope here.
- Connective is fixed to `"contrasts with"` — matching the
`contrasts_with` predicate already in the relation vocabulary.
Other comparison flavours (`similar to`, `differs from`, `is like`)
are not modelled; they would need their own predicates in the
pack.
- Identical-lemma comparison defers to disclosure. Callers wanting
a single-lemma surface should use the DEFINITION / RECALL path.
---
## Cross-References
- [ADR-0018](./ADR-0018-tool-use-scope.md) — `DialogueIntent` and the
COMPARISON regex this ADR consumes.
- [ADR-0048](./ADR-0048-pack-grounded-surface.md) — the
DEFINITION / RECALL pack-grounded surface this ADR mirrors for
COMPARISON; same trust-boundary discipline, same `grounding_source`
tag.
- [ADR-0049](./ADR-0049-intent-subject-extraction.md) — clean
subject extraction the COMPARISON path inherits (though COMPARISON
uses its own named-group regex, not the post-processor;
``"memory"`` and ``"recall"`` are already clean from the
``_COMPARE_RE`` capture).
- [ADR-0029](./ADR-0029-safety-packs.md) /
[ADR-0033](./ADR-0033-ethics-packs.md) — sibling-pack-grounded
contributor pattern this ADR continues to extend from verdict
surfaces to the answer surface.
---
## Verification
```
tests/test_pack_grounded_comparison.py — 15 tests, all green
tests/test_pack_grounding.py — 18 pre-existing tests still green
tests/test_intent_subject_extraction.py — 30 pre-existing tests still green
tests/test_semantic_realizer_integration.py — pre-existing tests still green
Lanes (all green on this branch):
core test --suite smoke 67 passed
core test --suite cognition 121 passed
core test --suite runtime 19 passed
core test --suite packs 6 passed
core eval cognition (pre → post):
intent_accuracy 100.0% → 100.0% (=)
surface_groundedness 61.5% → 69.2% (+7.7 pp)
term_capture_rate 50.0% → 58.3% (+8.3 pp)
versor_closure_rate 100.0% → 100.0% (=)
```
The non-negotiable field invariant (`versor_condition(F) < 1e-6`) is
preserved: this ADR adds a surface-construction branch on the
existing stub path — no algebra changes, no rotor construction, no
field update.

1
docs/decisions/README.md
View file

@ -59,6 +59,7 @@ ADRs record significant architectural decisions: what was decided, why, what alt
| [ADR-0047](ADR-0047-wire-forward-graph-constraint.md) | Wire forward graph constraint into the chat hot path (opt-in) | Accepted (2026-05-18) |
| [ADR-0048](ADR-0048-pack-grounded-surface.md) | Pack-grounded surface for cold-start DEFINITION / RECALL | Accepted (2026-05-18) |
| [ADR-0049](ADR-0049-intent-subject-extraction.md) | Intent classifier head-noun subject extraction | Accepted (2026-05-18) |
| [ADR-0050](ADR-0050-pack-grounded-comparison.md) | Pack-grounded surface for cold-start COMPARISON | Accepted (2026-05-18) |
---

155
tests/test_pack_grounded_comparison.py Normal file
View file

@ -0,0 +1,155 @@
"""ADR-0050 — pack-grounded COMPARISON surface tests.
Contract pinned here:
- ``pack_grounded_comparison_surface(a, b)`` returns a deterministic
surface composed of both lemmas + their pack ``semantic_domains``
(up to two per side) joined by the fixed connective
``"contrasts with"``. No synthesis.
- Returns ``None`` when either lemma is missing, not a pack lemma,
or when the two lemmas are identical (no contrastive evidence).
- The runtime wiring engages only when:
- the gate fires with ``source="empty_vault"``,
- ``output_language == "en"``,
- intent is ``COMPARISON``,
- both ``subject`` and ``secondary_subject`` are pack lemmas.
- ``ChatResponse.grounding_source`` and ``TurnEvent.grounding_source``
both carry ``"pack"`` on this branch.
- Refusal still takes priority — pack-grounded comparison never
bypasses safety / ethics verdict refusal.
"""
from __future__ import annotations
import pytest
from chat.pack_grounding import (
PACK_ID,
pack_grounded_comparison_surface,
)
from chat.runtime import ChatRuntime, _UNKNOWN_DOMAIN_SURFACE
# ---------------------------------------------------------------------------
# pack_grounded_comparison_surface — pure-function contracts
# ---------------------------------------------------------------------------
def test_two_known_lemmas_produce_comparison_surface() -> None:
surface = pack_grounded_comparison_surface("memory", "recall")
assert surface is not None
assert "memory" in surface
assert "recall" in surface
assert "contrasts with" in surface
assert PACK_ID in surface
assert "No session evidence yet." in surface
def test_unknown_lemma_a_returns_none() -> None:
assert pack_grounded_comparison_surface("nonexistentxyz", "memory") is None
def test_unknown_lemma_b_returns_none() -> None:
assert pack_grounded_comparison_surface("memory", "nonexistentxyz") is None
def test_identical_lemmas_return_none() -> None:
"""``Compare X and X`` carries no contrastive evidence — defer."""
assert pack_grounded_comparison_surface("memory", "memory") is None
assert pack_grounded_comparison_surface("MEMORY", "memory") is None # case-insensitive
@pytest.mark.parametrize("bad", ["", " ", None])
def test_empty_or_invalid_returns_none(bad) -> None:
assert pack_grounded_comparison_surface(bad, "memory") is None # type: ignore[arg-type]
assert pack_grounded_comparison_surface("memory", bad) is None # type: ignore[arg-type]
def test_comparison_surface_is_deterministic() -> None:
"""Same input must produce byte-identical surface on repeat."""
a = pack_grounded_comparison_surface("memory", "recall")
b = pack_grounded_comparison_surface("memory", "recall")
assert a == b
assert a is not None
def test_comparison_surface_is_order_sensitive() -> None:
"""``compare(a, b)`` and ``compare(b, a)`` produce distinct surfaces —
the connective ``"contrasts with"`` orients the comparison."""
ab = pack_grounded_comparison_surface("memory", "recall")
ba = pack_grounded_comparison_surface("recall", "memory")
assert ab is not None and ba is not None
assert ab != ba
# ---------------------------------------------------------------------------
# ChatRuntime integration — cold-start COMPARISON path
# ---------------------------------------------------------------------------
def test_cold_start_comparison_returns_pack_grounded_surface() -> None:
rt = ChatRuntime()
resp = rt.chat("Compare memory and recall")
assert resp.grounding_source == "pack"
assert "memory" in resp.surface
assert "recall" in resp.surface
assert "contrasts with" in resp.surface
def test_cold_start_comparison_with_unknown_lemma_disclosure() -> None:
"""When one term is not a pack lemma, the COMPARISON path returns
None and the runtime falls through to the universal disclosure."""
rt = ChatRuntime()
resp = rt.chat("Compare memory and zigzagxyz")
assert resp.surface == _UNKNOWN_DOMAIN_SURFACE
assert resp.grounding_source == "none"
def test_cold_start_comparison_with_identical_lemmas_disclosure() -> None:
"""``Compare X and X`` defers to the universal disclosure."""
rt = ChatRuntime()
resp = rt.chat("Compare memory and memory")
assert resp.surface == _UNKNOWN_DOMAIN_SURFACE
assert resp.grounding_source == "none"
def test_turn_event_carries_grounding_source_on_comparison() -> None:
rt = ChatRuntime()
rt.chat("Compare memory and recall")
last_event = rt.turn_log[-1]
assert getattr(last_event, "grounding_source", None) == "pack"
def test_comparison_pack_grounded_passes_verdict_audit() -> None:
rt = ChatRuntime()
resp = rt.chat("Compare memory and recall")
assert resp.safety_verdict is not None
assert resp.ethics_verdict is not None
# ---------------------------------------------------------------------------
# Doctrine — no synthesis, no inference
# ---------------------------------------------------------------------------
def test_comparison_surface_atoms_are_verbatim_from_pack() -> None:
"""Every visible non-template token must be either the lemma or a
verbatim ``semantic_domains`` string from the pack — no rewording."""
from chat.pack_grounding import _pack_index
surface = pack_grounded_comparison_surface("memory", "recall")
assert surface is not None
index = _pack_index()
memory_domains = index["memory"][:2]
recall_domains = index["recall"][:2]
for domain in memory_domains:
assert domain in surface
for domain in recall_domains:
assert domain in surface
# The two fixed-template tokens
assert "contrasts with" in surface
assert "pack-grounded" in surface
assert "No session evidence yet." in surface