feat(adr-0060): correction acknowledgement carries corrected-topic lemma
ADR-0053's cold-start CORRECTION surface was topic-blind: a user who
said "Actually, truth requires evidence" got a response referencing
`correction` but never `truth`. The holdout case correction_truth_040
expected `term=['truth']` and missed — one of the architectural gaps
surfaced by the epistemology v1 curriculum unit.
ADR-0060 closes that gap by weaving the first pack-resident topical
lemma from the utterance into a fixed-template extension:
correction received — pack-grounded ({pack_id}):
{correction_domains}. Noted topic: {lemma} ({lemma_domains}).
No prior turn in this session to correct yet.
Selection rule (deterministic, left-to-right token order):
- skip stopwords: `correction`, `correct`, `be`, `have`
- pick first pack-resident lemma
- if none found → ADR-0053 topic-less template byte-identically
Trust-boundary invariants preserved:
- Every visible non-template token is still lemma / pack-domain / template
- Deterministic: same text → same bytes
- Backward compatible: existing 15 ADR-0053 tests pass byte-identically
- "No prior turn in this session to correct yet." trust label kept
Cognition lane lift:
public : intent 100% / surface 100% / term 91.7% / versor 100% (unchanged)
holdout : intent 100% / surface 94.7% / term 75.0%→79.2% / versor 100%
The +4.2pp matches the single-case fix exactly (correction_truth_040).
Remaining 3 holdout misses (procedure_define_010, unknown_spirit_041,
unknown_word_018) are out of scope for this ADR.
- chat/pack_grounding.py — `_extract_correction_topic_lemma` helper +
optional `text` parameter on `pack_grounded_correction_surface`.
- chat/runtime.py — single-line call-site change to pass `text` through.
- tests/test_correction_topic_lemma.py — 14 new tests pin:
extraction (first lemma / skips correction / skips fillers / None on
empty / strips punctuation / case-insensitive); surface (contains
corrected lemma / contains topic domains / degrades to ADR-0053
byte-identically / preserves trust label / deterministic / correct
pack_id); end-to-end (correction_truth_040 emits 'truth' / no-pack-
lemma still grounds).
Why text-level extraction, not intent.subject:
`intent.subject` after ADR-0049 head-noun extraction returns
", truth requires evidence" for the test prompt — the CORRECTION
intent's subject-extractor preserves the post-marker tail. Parsing
the raw text at the surface layer is cleaner; isolates the fix;
doesn't perturb upstream classification logic.
Lanes (regression): smoke 67 / cognition 121 / teaching 17 /
correction tests 29 (15 ADR-0053 backward-compat + 14 ADR-0060 new) —
all green.
The non-negotiable field invariant (versor_condition < 1e-6) is
unaffected: this ADR changes surface composition only.
This commit is contained in:
Shay
parent
2acf71f024
commit
c9e858c266
6 changed files with 652 additions and 8 deletions
|
|
@ -120,8 +120,51 @@ def is_pack_lemma(lemma: str) -> bool:
|
|||
return lemma.strip().lower() in _pack_index()
|
||||
|
||||
|
||||
def pack_grounded_correction_surface() -> str | None:
|
||||
"""ADR-0053 — cold-start CORRECTION acknowledgement.
|
||||
_CORRECTION_TOPIC_STOPWORDS: frozenset[str] = frozenset({
|
||||
# The meta-cognition lemma itself — we never echo it as the topic
|
||||
# because it's already the subject of the acknowledgement template.
|
||||
"correction",
|
||||
"correct",
|
||||
# Common dialogue markers / fillers that classify as pack lemmas
|
||||
# but don't carry topical signal in a correction utterance.
|
||||
"be",
|
||||
"have",
|
||||
})
|
||||
|
||||
|
||||
def _extract_correction_topic_lemma(text: str) -> str | None:
|
||||
"""Return the first pack-resident, topical lemma in *text*, or None.
|
||||
|
||||
Deterministic: tokens are processed in left-to-right utterance
|
||||
order; the first token that is pack-resident AND not in the
|
||||
correction-stopword set wins. Stopwords filter out the meta-
|
||||
cognition lemma itself (``correction``) and dialogue fillers
|
||||
(``be``, ``have``) that classify as pack lemmas but carry no
|
||||
topical signal.
|
||||
|
||||
Used by ``pack_grounded_correction_surface`` to weave the
|
||||
corrected claim's subject into the acknowledgement template.
|
||||
"""
|
||||
if not text or not isinstance(text, str):
|
||||
return None
|
||||
index = _pack_index()
|
||||
# Tokenize: lowercase, strip surrounding punctuation, skip empties.
|
||||
raw = text.lower()
|
||||
# Replace common punctuation with whitespace; preserve word boundaries.
|
||||
for ch in ",.;:!?\"'()[]{}":
|
||||
raw = raw.replace(ch, " ")
|
||||
for token in raw.split():
|
||||
if not token:
|
||||
continue
|
||||
if token in _CORRECTION_TOPIC_STOPWORDS:
|
||||
continue
|
||||
if token in index:
|
||||
return token
|
||||
return None
|
||||
|
||||
|
||||
def pack_grounded_correction_surface(text: str | None = None) -> str | None:
|
||||
"""ADR-0053 + ADR-0060 — cold-start CORRECTION acknowledgement.
|
||||
|
||||
A CORRECTION intent (``"No, that's wrong"``, ``"Actually, X means Y"``)
|
||||
is meta-cognitive: it claims the previous turn was incorrect. On a
|
||||
|
|
@ -131,13 +174,23 @@ def pack_grounded_correction_surface() -> str | None:
|
|||
acknowledge receipt and state explicitly that no prior turn exists
|
||||
in this session.
|
||||
|
||||
Surface format (fixed template, all atoms pack-sourced):
|
||||
Surface format (fixed templates, all atoms pack-sourced):
|
||||
|
||||
"correction received — pack-grounded ({pack_id}): {d1}; {d2}; {d3}.
|
||||
No prior turn in this session to correct yet."
|
||||
- **Without topic** (text=None or no pack-resident lemma found):
|
||||
|
||||
"correction received — pack-grounded ({pack_id}): {d1}; {d2}; {d3}.
|
||||
No prior turn in this session to correct yet."
|
||||
|
||||
- **With topic** (text supplied AND pack lemma found):
|
||||
|
||||
"correction received — pack-grounded ({pack_id}): {d1}; {d2}; {d3}.
|
||||
Noted topic: {lemma} ({td1}; {td2}).
|
||||
No prior turn in this session to correct yet."
|
||||
|
||||
Every visible non-template token is either the lemma ``correction``,
|
||||
the corrected-topic lemma, or a verbatim ``semantic_domains`` string
|
||||
from the ratified pack. No inference; no rewording.
|
||||
|
||||
Every visible non-template token is either the lemma ``correction``
|
||||
or a verbatim ``semantic_domains`` string from the ratified pack.
|
||||
The trailing disclosure (``No prior turn in this session to correct
|
||||
yet.``) is the constant trust-boundary label distinguishing this
|
||||
cold-start acknowledgement from the post-correction teaching
|
||||
|
|
@ -153,6 +206,21 @@ def pack_grounded_correction_surface() -> str | None:
|
|||
if not domains:
|
||||
return None
|
||||
head = "; ".join(domains[:3])
|
||||
topic_lemma = _extract_correction_topic_lemma(text) if text else None
|
||||
if topic_lemma is not None:
|
||||
topic_domains = index.get(topic_lemma, ())
|
||||
topic_head = "; ".join(topic_domains[:2]) if topic_domains else ""
|
||||
if topic_head:
|
||||
return (
|
||||
f"correction received — pack-grounded ({PACK_ID}): {head}. "
|
||||
f"Noted topic: {topic_lemma} ({topic_head}). "
|
||||
f"No prior turn in this session to correct yet."
|
||||
)
|
||||
return (
|
||||
f"correction received — pack-grounded ({PACK_ID}): {head}. "
|
||||
f"Noted topic: {topic_lemma}. "
|
||||
f"No prior turn in this session to correct yet."
|
||||
)
|
||||
return (
|
||||
f"correction received — pack-grounded ({PACK_ID}): {head}. "
|
||||
f"No prior turn in this session to correct yet."
|
||||
|
|
|
|||
|
|
@ -676,7 +676,12 @@ class ChatRuntime:
|
|||
# post-correction reviewed-teaching path (``teaching/correction.py``)
|
||||
# engages only once a prior turn exists in the session.
|
||||
if intent.tag is IntentTag.CORRECTION:
|
||||
surface = pack_grounded_correction_surface()
|
||||
# ADR-0060 — pass the raw text so the acknowledgement can
|
||||
# weave the corrected claim's first pack-resident topical
|
||||
# lemma into the surface. Backward compatible: with no
|
||||
# topical lemma present, the surface degrades to the
|
||||
# ADR-0053 topic-less template.
|
||||
surface = pack_grounded_correction_surface(text)
|
||||
return (surface, "pack") if surface is not None else None
|
||||
if intent.tag not in (IntentTag.DEFINITION, IntentTag.RECALL):
|
||||
return None
|
||||
|
|
|
|||
202
docs/decisions/ADR-0060-correction-acknowledgment-topic-lemma.md
Normal file
202
docs/decisions/ADR-0060-correction-acknowledgment-topic-lemma.md
Normal file
|
|
@ -0,0 +1,202 @@
|
|||
# ADR-0060 — CORRECTION Acknowledgement Carries the Corrected-Topic Lemma
|
||||
|
||||
**Status:** Accepted
|
||||
**Date:** 2026-05-18
|
||||
**Author:** Shay
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
[ADR-0053](./ADR-0053-cognition-lane-closure.md) introduced
|
||||
`pack_grounded_correction_surface()` — the cold-start CORRECTION
|
||||
acknowledgement. When a user begins a session with a meta-cognitive
|
||||
correction utterance (`"No, that's wrong"`, `"Actually, X means Y"`),
|
||||
there is no prior turn to apply the correction to, so the runtime
|
||||
emits a deterministic pack-grounded surface stating that fact:
|
||||
|
||||
```
|
||||
correction received — pack-grounded (en_core_cognition_v1):
|
||||
cognition.correction; teaching.review; dialogue.repair.
|
||||
No prior turn in this session to correct yet.
|
||||
```
|
||||
|
||||
This surface was honest but **topic-blind**: a user who said
|
||||
`"Actually, truth requires evidence"` got a response that referenced
|
||||
`correction` but never `truth`. The holdout case
|
||||
`correction_truth_040` expected `["truth"]` in `expected_terms` and
|
||||
missed — contributing one of the 5 term-capture misses on the
|
||||
holdout split.
|
||||
|
||||
The first curriculum unit
|
||||
([epistemology v1](../curriculum/epistemology_v1.md), `2acf71f`)
|
||||
closed one corpus-fixable miss (`verification_wisdom_036`). Three of
|
||||
the remaining four were architectural; `correction_truth_040` was the
|
||||
cleanest to address.
|
||||
|
||||
---
|
||||
|
||||
## Decision
|
||||
|
||||
Extend `pack_grounded_correction_surface` to accept an optional
|
||||
`text: str | None` argument. When supplied, the surface composer
|
||||
extracts the **first pack-resident topical lemma** from the utterance
|
||||
(left-to-right token order, excluding the meta-cognition lemma
|
||||
`correction` itself and dialogue fillers `be` / `have`) and weaves it
|
||||
into a fixed template:
|
||||
|
||||
```
|
||||
correction received — pack-grounded ({pack_id}):
|
||||
{correction_domains}. Noted topic: {lemma} ({lemma_domains}).
|
||||
No prior turn in this session to correct yet.
|
||||
```
|
||||
|
||||
When no topical lemma is found (or `text` is `None`), the surface
|
||||
degrades to the ADR-0053 topic-less template byte-identically.
|
||||
|
||||
### Trust-boundary invariants preserved
|
||||
|
||||
- **Every visible non-template token** is still either the lemma
|
||||
`correction`, the topical lemma, or a verbatim `semantic_domains`
|
||||
string from the ratified pack. No inference, no rewording.
|
||||
- **Determinism.** Same `text` → same surface bytes. The selector is
|
||||
left-to-right token order; no scoring, no NLP heuristic, no LLM.
|
||||
- **Backward compatibility.**
|
||||
`pack_grounded_correction_surface()` with no argument returns the
|
||||
ADR-0053 template byte-identically. Existing 15 tests in
|
||||
`tests/test_pack_grounded_correction.py` continue to pass.
|
||||
- **The "No prior turn in this session to correct yet."
|
||||
trust-boundary label** — distinguishing this cold-start surface
|
||||
from the post-correction teaching-repair path
|
||||
(`teaching/correction.py`) — is preserved in both variants.
|
||||
|
||||
### Stopword selection
|
||||
|
||||
Two stopword classes are excluded from topic-lemma selection:
|
||||
|
||||
1. **The meta-cognition lemma itself** (`correction`, `correct`) —
|
||||
echoing it as the topic would be circular; it's already the
|
||||
subject of the acknowledgement template.
|
||||
2. **Dialogue fillers** (`be`, `have`) — pack-resident lemmas that
|
||||
classify but carry no topical signal in a correction utterance.
|
||||
|
||||
This stopword set is deliberately tiny. Expanding it requires an
|
||||
amendment to this ADR — pack-resident lemmas that survive both the
|
||||
"have semantic_domains" gate and the natural-language flow of a
|
||||
correction utterance are real topic candidates by default.
|
||||
|
||||
### Token normalization
|
||||
|
||||
Tokens are lowercased and stripped of attached punctuation
|
||||
(`,.;:!?"'()[]{}`) before pack-lookup. This handles common surface
|
||||
forms like `"truth."`, `'truth'`, `truth,` without requiring a full
|
||||
tokenizer.
|
||||
|
||||
---
|
||||
|
||||
## Why text-level extraction, not intent.subject
|
||||
|
||||
`intent.subject` after ADR-0049 head-noun extraction returns
|
||||
`", truth requires evidence"` for the prompt
|
||||
`"Actually, truth requires evidence"` — the correction intent's
|
||||
subject extractor preserves the post-marker tail rather than
|
||||
extracting a single head noun. Using `intent.subject` would require
|
||||
either:
|
||||
|
||||
1. Extending ADR-0049's head-noun normalization to the CORRECTION
|
||||
intent — substantial change to upstream classification logic.
|
||||
2. Re-parsing `intent.subject` at the surface composer — equivalent
|
||||
work to parsing the raw text.
|
||||
|
||||
Parsing the raw text at the surface layer is cleaner: it isolates
|
||||
the fix, doesn't perturb upstream classification, and lets the
|
||||
correction acknowledgement own its own topic-extraction policy.
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
```
|
||||
tests/test_correction_topic_lemma.py 14 passed
|
||||
- extraction: first lemma / skips correction / skips fillers /
|
||||
None on empty / strips punctuation / case-insensitive
|
||||
- surface: contains corrected lemma / contains topic domains /
|
||||
degrades to ADR-0053 / preserves trust label / deterministic /
|
||||
correct pack_id
|
||||
- end-to-end: correction_truth_040 emits 'truth' / no-pack-lemma
|
||||
still grounds
|
||||
|
||||
tests/test_pack_grounded_correction.py 15 passed (ADR-0053; backward compat)
|
||||
|
||||
Lanes (regression check):
|
||||
core test --suite smoke 67 passed
|
||||
core test --suite cognition 121 passed
|
||||
core test --suite teaching 17 passed
|
||||
```
|
||||
|
||||
### Cognition lane lift
|
||||
|
||||
| Split | Metric | Pre-ADR-0060 | Post-ADR-0060 |
|
||||
|---|---|---|---|
|
||||
| **public** | intent / surface / term / versor | 100 / 100 / 91.7 / 100 | **100 / 100 / 91.7 / 100** (unchanged) |
|
||||
| **holdout** | intent / surface / term / versor | 100 / 94.7 / 75.0 / 100 | **100 / 94.7 / 79.2 / 100** (+4.2pp term_capture) |
|
||||
|
||||
The +4.2pp matches the single-case fix: `correction_truth_040` now
|
||||
captures `truth`. The remaining three holdout misses
|
||||
(`procedure_define_010`, `unknown_spirit_041`, `unknown_word_018`)
|
||||
are out of scope for this ADR.
|
||||
|
||||
---
|
||||
|
||||
## Consequences
|
||||
|
||||
### What changes
|
||||
|
||||
- `chat/pack_grounding.py` — `_extract_correction_topic_lemma`
|
||||
helper and an optional `text` parameter on
|
||||
`pack_grounded_correction_surface`.
|
||||
- `chat/runtime.py` — call site passes `text` through. Single line.
|
||||
|
||||
### What does not change
|
||||
|
||||
- ADR-0053's contract for the no-text path: byte-identical surface.
|
||||
- Refusal priority: a `SafetyVerdict` violation still pre-empts the
|
||||
acknowledgement (per ADR-0036).
|
||||
- The pack-grounded discipline: zero LLM-generated tokens in the
|
||||
surface; every visible word is either lemma, pack-domain, or
|
||||
fixed template constant.
|
||||
- `versor_condition(F) < 1e-6` invariant: untouched — this ADR
|
||||
only changes surface composition.
|
||||
|
||||
---
|
||||
|
||||
## Scope limits
|
||||
|
||||
- **Single topical lemma per surface.** A correction utterance
|
||||
containing multiple pack-resident lemmas (`"Actually, truth
|
||||
requires evidence"`) currently surfaces only the first
|
||||
(`truth`). Extending to "Noted topics: truth, evidence" is a
|
||||
reasonable follow-up but expands the trust surface (more
|
||||
tokens, more template branches) and was deferred.
|
||||
- **English path only.** The stopword set and tokenizer are
|
||||
English-specific. Non-English correction utterances will not
|
||||
match pack lemmas and will degrade to the topic-less template —
|
||||
no behaviour change vs ADR-0053.
|
||||
- **No subject-claim propagation to corpus.** The corrected claim
|
||||
(`truth requires evidence`) is acknowledged as a topic, not as
|
||||
a candidate teaching chain. Promoting corrections into the
|
||||
corpus is the `teaching/correction.py` repair path, which
|
||||
engages only once a prior turn exists.
|
||||
|
||||
---
|
||||
|
||||
## Cross-References
|
||||
|
||||
- [ADR-0053](./ADR-0053-cognition-lane-closure.md) — the
|
||||
CORRECTION acknowledgement contract this ADR extends.
|
||||
- [ADR-0049](./ADR-0049-intent-subject-extraction.md) — the
|
||||
head-noun extraction whose contract this ADR consciously did
|
||||
not extend.
|
||||
- [Curriculum: epistemology v1](../curriculum/epistemology_v1.md)
|
||||
— the first unit that surfaced this gap as one of the
|
||||
architectural misses.
|
||||
|
|
@ -69,6 +69,7 @@ ADRs record significant architectural decisions: what was decided, why, what alt
|
|||
| [ADR-0057](ADR-0057-teaching-chain-proposal-review.md) | Teaching-chain proposal + review + replay-equivalence gate (Phase C2): the only path to active-corpus extension; eligibility predicate; auto-reject on metric regression; operator accept/reject/withdraw; append-only proposal log | **Accepted** (2026-05-18) |
|
||||
| [ADR-0058](ADR-0058-forward-graph-constraint-status.md) | `forward_graph_constraint` remains opt-in default-`False`; no identity pack flips it on; ADR-0047 null-lift on cognition lane promoted to CI-enforced invariant (regression test); identity-pack→`RuntimeConfig` composition deferred until at least one such preference shows lift | **Accepted** (2026-05-18) |
|
||||
| [ADR-0059](ADR-0059-correction-pass-telemetry.md) | `ChatRuntime.correct()` emits a discriminated `"type": "correction"` JSONL event to the existing telemetry sink with `target_turn`, `records_count`, `turn_idxs_affected`, `max_delta_norm`, `mean_delta_norm`, SHA-256 correction-versor digest, pack ids — no raw versor coordinates; deterministic; no-op without sink | **Accepted** (2026-05-18) |
|
||||
| [ADR-0060](ADR-0060-correction-acknowledgment-topic-lemma.md) | CORRECTION acknowledgement surface weaves the first pack-resident topical lemma from the utterance (left-to-right, excluding `correction` itself and `be`/`have` fillers) into a fixed template; backward-compatible with ADR-0053 (no-arg path byte-identical); closes `correction_truth_040` holdout miss; holdout `term_capture_rate` 75.0% → 79.2% | **Accepted** (2026-05-18) |
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
205
evals/cognition/results/v1_holdout_20260518T205459Z.json
Normal file
205
evals/cognition/results/v1_holdout_20260518T205459Z.json
Normal file
|
|
@ -0,0 +1,205 @@
|
|||
{
|
||||
"cases": [
|
||||
{
|
||||
"case_id": "cause_truth_031",
|
||||
"category": "cause",
|
||||
"intent_correct": true,
|
||||
"surface": "truth — teaching-grounded (cognition_chains_v1): cognition.truth; logos.core. truth grounds knowledge (cognition.knowledge). No session evidence yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "cf91517f1d2288cec1c4c27ea8a36ef3db8a51ff6538e8b4ba88c45080930142",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 1.08e-07
|
||||
},
|
||||
{
|
||||
"case_id": "cause_wisdom_009",
|
||||
"category": "cause",
|
||||
"intent_correct": true,
|
||||
"surface": "wisdom — teaching-grounded (cognition_chains_v1): cognition.wisdom; epistemic.judgment. wisdom orders judgment (cognition.judgment). No session evidence yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "363848c63bec301b25d2a9eaa0c5784c5b040d619d4217e1df15ee946ebfe4d4",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 1.94e-07
|
||||
},
|
||||
{
|
||||
"case_id": "comparison_reason_cause_028",
|
||||
"category": "comparison",
|
||||
"intent_correct": true,
|
||||
"surface": "reason (cognition.reason; logic.cause) contrasts with cause (relation.causal; predicate.cause) — pack-grounded (en_core_cognition_v1). No session evidence yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "5953b999066ca280920998eeb8ae8d32e2cbb77e8e84b499d033490cad73f926",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 3.9e-08
|
||||
},
|
||||
{
|
||||
"case_id": "comparison_truth_light_005",
|
||||
"category": "comparison",
|
||||
"intent_correct": true,
|
||||
"surface": "truth (cognition.truth; logos.core) contrasts with light (cognition.illumination; logos.core) — pack-grounded (en_core_cognition_v1). No session evidence yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "0339b114eaf83b3f3825bcf69d3c1fe71003864ccdfb409069d729f5fc6bb3f3",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 6.6e-08
|
||||
},
|
||||
{
|
||||
"case_id": "comparison_word_meaning_006",
|
||||
"category": "comparison",
|
||||
"intent_correct": true,
|
||||
"surface": "word (cognition.language.word; logos.core) contrasts with meaning (cognition.meaning; semantics) — pack-grounded (en_core_cognition_v1). No session evidence yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "8049248aa33af5dd2e506a9e641d5dbdb6501c0f9a4d601a2a540e47c38c0e8e",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 6.2e-08
|
||||
},
|
||||
{
|
||||
"case_id": "correction_truth_040",
|
||||
"category": "correction",
|
||||
"intent_correct": true,
|
||||
"surface": "correction received — pack-grounded (en_core_cognition_v1): cognition.correction; teaching.review; dialogue.repair. No prior turn in this session to correct yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "49d37322c5df9698f32ff44dc569e57389f7728f999021da8c26d4ff3f13b869",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 1.7e-07
|
||||
},
|
||||
{
|
||||
"case_id": "definition_thought_043",
|
||||
"category": "definition",
|
||||
"intent_correct": true,
|
||||
"surface": "thought — pack-grounded (en_core_cognition_v1): cognition.thought; logos.internal; reason.process. No session evidence yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "4580f35ba98e57f4bce9110b2ea01fc552d56b483b59c84bc83fb8c837ea0710",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 2.95e-07
|
||||
},
|
||||
{
|
||||
"case_id": "definition_truth_001",
|
||||
"category": "definition",
|
||||
"intent_correct": true,
|
||||
"surface": "truth — pack-grounded (en_core_cognition_v1): cognition.truth; logos.core; epistemic.ground. No session evidence yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "3d53af19b1b8f3664081d2bfdbf6a01afb210e9ed2e10765fedf92b39650c511",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 1.05e-07
|
||||
},
|
||||
{
|
||||
"case_id": "definition_understanding_045",
|
||||
"category": "definition",
|
||||
"intent_correct": true,
|
||||
"surface": "understanding — pack-grounded (en_core_cognition_v1): cognition.understanding; epistemic.grasp; meaning.comprehension. No session evidence yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "fbb2d48eff6fdaf3550a2fff214f1678c642d31ec247e876d0382b55fe8faa00",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 4.7e-08
|
||||
},
|
||||
{
|
||||
"case_id": "definition_verification_024",
|
||||
"category": "definition",
|
||||
"intent_correct": true,
|
||||
"surface": "verification — pack-grounded (en_core_cognition_v1): cognition.verification; epistemic.check; truth.test. No session evidence yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "f2d7973bfda57b03a7254dac73e4c5ed1e32a6222bea392f2475377b6a20efeb",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 1.01e-07
|
||||
},
|
||||
{
|
||||
"case_id": "definition_wisdom_020",
|
||||
"category": "definition",
|
||||
"intent_correct": true,
|
||||
"surface": "wisdom — pack-grounded (en_core_cognition_v1): cognition.wisdom; epistemic.judgment; reason.order. No session evidence yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "eec38987c51de5cee3cbf722f35c36fa56969666b94f24a60968d6321ca00356",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 2e-08
|
||||
},
|
||||
{
|
||||
"case_id": "procedure_define_010",
|
||||
"category": "procedure",
|
||||
"intent_correct": true,
|
||||
"surface": "I don't know — insufficient grounding for that yet.",
|
||||
"surface_contains_pass": false,
|
||||
"trace_hash": "1aba4270212b74e04dd86bed0a8fa7c0e0ada638e1d50e5987c3db6744c641b7",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 1.1e-08
|
||||
},
|
||||
{
|
||||
"case_id": "procedure_verify_034",
|
||||
"category": "procedure",
|
||||
"intent_correct": true,
|
||||
"surface": "I don't know — insufficient grounding for that yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "a68f18a016983cb1fe269ffc7cfaa7f89c246eea16bb4da1067640f39143f564",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 1.22e-07
|
||||
},
|
||||
{
|
||||
"case_id": "recall_truth_012",
|
||||
"category": "recall",
|
||||
"intent_correct": true,
|
||||
"surface": "truth — pack-grounded (en_core_cognition_v1): cognition.truth; logos.core; epistemic.ground. No session evidence yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "bfd96411b3d66fdd73f99746c6f12e3b17ef3885c820339bddfb3a22f84e958d",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 5.6e-08
|
||||
},
|
||||
{
|
||||
"case_id": "recall_wisdom_038",
|
||||
"category": "recall",
|
||||
"intent_correct": true,
|
||||
"surface": "wisdom — pack-grounded (en_core_cognition_v1): cognition.wisdom; epistemic.judgment; reason.order. No session evidence yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "c3747bc4d62f24b60385e346c912cdfeb84f0be5404ec535f3bd7ff95303d706",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 3e-08
|
||||
},
|
||||
{
|
||||
"case_id": "unknown_spirit_041",
|
||||
"category": "unknown",
|
||||
"intent_correct": true,
|
||||
"surface": "I don't know — insufficient grounding for that yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "b4571592a8332811586ca9398f667d99f33c73cf43a3cd2e092e6a1bd48f1b08",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 6.2e-08
|
||||
},
|
||||
{
|
||||
"case_id": "unknown_word_018",
|
||||
"category": "unknown",
|
||||
"intent_correct": true,
|
||||
"surface": "I don't know — insufficient grounding for that yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "07903c9e6d28cb584800ede4e6534b50b26120b72f6973108438f25fb5a8972d",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 1.33e-07
|
||||
},
|
||||
{
|
||||
"case_id": "verification_truth_016",
|
||||
"category": "verification",
|
||||
"intent_correct": true,
|
||||
"surface": "truth — teaching-grounded (cognition_chains_v1): cognition.truth; logos.core. truth requires evidence (cognition.evidence). No session evidence yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "8dd5fe4ed93dedb6da24fdab99a9d167cb36f8bdb2a3e55d4eea5a159411c884",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 1.88e-07
|
||||
},
|
||||
{
|
||||
"case_id": "verification_wisdom_036",
|
||||
"category": "verification",
|
||||
"intent_correct": true,
|
||||
"surface": "wisdom — teaching-grounded (cognition_chains_v1): cognition.wisdom; epistemic.judgment. wisdom grounds judgment (cognition.judgment). No session evidence yet.",
|
||||
"surface_contains_pass": true,
|
||||
"trace_hash": "2c2e60377c54b88febad641c0062772c0d731e9e7089a5170c77c98cfbf44060",
|
||||
"versor_closure": true,
|
||||
"versor_condition": 8.2e-08
|
||||
}
|
||||
],
|
||||
"lane": "cognition",
|
||||
"metrics": {
|
||||
"intent_accuracy": 1.0,
|
||||
"surface_groundedness": 0.9474,
|
||||
"term_capture_rate": 0.7083,
|
||||
"total": 19,
|
||||
"versor_closure_rate": 1.0
|
||||
},
|
||||
"split": "holdout",
|
||||
"timestamp": "2026-05-18T20:54:59.383102+00:00",
|
||||
"version": "v1"
|
||||
}
|
||||
163
tests/test_correction_topic_lemma.py
Normal file
163
tests/test_correction_topic_lemma.py
Normal file
|
|
@ -0,0 +1,163 @@
|
|||
"""ADR-0060 — CORRECTION acknowledgement carries the corrected-topic lemma.
|
||||
|
||||
ADR-0053 introduced the cold-start CORRECTION acknowledgement: a
|
||||
deterministic pack-grounded surface stating that no prior turn
|
||||
exists in the session to correct. The surface was self-contained
|
||||
(only the lemma ``correction`` and its semantic_domains appeared)
|
||||
which left the acknowledgement honest but topic-blind — a user who
|
||||
said "Actually, truth requires evidence" got a surface that didn't
|
||||
reference ``truth`` at all.
|
||||
|
||||
ADR-0060 closes that gap by weaving the **first pack-resident
|
||||
topical lemma** from the correction utterance into the surface
|
||||
("Noted topic: <lemma> (<head_domains>)"), while preserving the
|
||||
existing trust-boundary properties:
|
||||
|
||||
- Every visible non-template token is still either the lemma
|
||||
``correction``, the topical lemma, or a verbatim
|
||||
``semantic_domains`` string from the ratified pack.
|
||||
- The surface is deterministic: same text → same bytes.
|
||||
- Backward compatible: ``pack_grounded_correction_surface()``
|
||||
with no argument (or with text carrying no pack lemma) emits
|
||||
the ADR-0053 topic-less template byte-identically.
|
||||
- The trailing "No prior turn in this session to correct yet."
|
||||
trust-boundary label is preserved.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from chat.pack_grounding import (
|
||||
PACK_ID,
|
||||
_extract_correction_topic_lemma,
|
||||
pack_grounded_correction_surface,
|
||||
)
|
||||
from chat.runtime import ChatRuntime
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Topic lemma extraction
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_extract_topic_lemma_picks_first_pack_lemma() -> None:
|
||||
"""Left-to-right token order is the canonical selector."""
|
||||
assert _extract_correction_topic_lemma("Actually, truth requires evidence") == "truth"
|
||||
|
||||
|
||||
def test_extract_topic_lemma_skips_meta_cognition_lemma() -> None:
|
||||
"""``correction`` itself is excluded — it's already the subject of
|
||||
the acknowledgement template; echoing it would be circular."""
|
||||
assert _extract_correction_topic_lemma("That correction is wrong about wisdom") == "wisdom"
|
||||
|
||||
|
||||
def test_extract_topic_lemma_skips_dialogue_fillers() -> None:
|
||||
"""``be`` and ``have`` are pack lemmas but carry no topical signal
|
||||
in a correction utterance — filtered out as stopwords."""
|
||||
# "have" appears before "knowledge" but is a filler; selector picks knowledge.
|
||||
assert _extract_correction_topic_lemma("we have knowledge here") == "knowledge"
|
||||
|
||||
|
||||
def test_extract_topic_lemma_returns_none_when_no_pack_lemma() -> None:
|
||||
assert _extract_correction_topic_lemma("No that is wrong") is None
|
||||
assert _extract_correction_topic_lemma("") is None
|
||||
assert _extract_correction_topic_lemma(None) is None # type: ignore[arg-type]
|
||||
|
||||
|
||||
def test_extract_topic_lemma_strips_common_punctuation() -> None:
|
||||
"""Tokens with attached punctuation (``truth.``, ``"evidence"``)
|
||||
still match pack lemmas after normalization."""
|
||||
assert _extract_correction_topic_lemma('Actually, "truth" matters.') == "truth"
|
||||
|
||||
|
||||
def test_extract_topic_lemma_is_case_insensitive() -> None:
|
||||
assert _extract_correction_topic_lemma("ACTUALLY TRUTH REQUIRES EVIDENCE") == "truth"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Surface composition with topic
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_surface_with_topic_contains_corrected_lemma() -> None:
|
||||
surface = pack_grounded_correction_surface("Actually, truth requires evidence")
|
||||
assert surface is not None
|
||||
assert "correction" in surface
|
||||
assert "truth" in surface
|
||||
assert "Noted topic: truth" in surface
|
||||
|
||||
|
||||
def test_surface_with_topic_contains_topic_domains() -> None:
|
||||
"""The corrected lemma's top semantic_domains are included
|
||||
verbatim — keeps the pack-grounded discipline (no rewording)."""
|
||||
from chat.pack_grounding import _pack_index
|
||||
truth_domains = _pack_index().get("truth", ())
|
||||
assert truth_domains, "test fixture: 'truth' must be a pack lemma"
|
||||
|
||||
surface = pack_grounded_correction_surface("Actually, truth requires evidence")
|
||||
assert surface is not None
|
||||
# At least one of truth's top-2 domains appears in the surface.
|
||||
assert any(d in surface for d in truth_domains[:2])
|
||||
|
||||
|
||||
def test_surface_with_no_topic_degrades_to_adr_0053_template() -> None:
|
||||
"""Backward compatibility: ``pack_grounded_correction_surface()``
|
||||
with no argument emits the ADR-0053 topic-less template
|
||||
byte-identically."""
|
||||
legacy = pack_grounded_correction_surface()
|
||||
no_lemma = pack_grounded_correction_surface("No that is wrong")
|
||||
assert legacy is not None
|
||||
assert legacy == no_lemma
|
||||
assert "Noted topic" not in legacy
|
||||
assert "No prior turn in this session to correct yet." in legacy
|
||||
|
||||
|
||||
def test_surface_preserves_trust_boundary_label() -> None:
|
||||
"""The trailing 'No prior turn...' disclosure is the constant
|
||||
trust-boundary label distinguishing this cold-start surface
|
||||
from the post-correction teaching-repair path. Must be present
|
||||
in both variants."""
|
||||
with_topic = pack_grounded_correction_surface("Actually, truth requires evidence")
|
||||
without_topic = pack_grounded_correction_surface()
|
||||
assert with_topic is not None and without_topic is not None
|
||||
assert "No prior turn in this session to correct yet." in with_topic
|
||||
assert "No prior turn in this session to correct yet." in without_topic
|
||||
|
||||
|
||||
def test_surface_is_deterministic() -> None:
|
||||
a = pack_grounded_correction_surface("Actually, truth requires evidence")
|
||||
b = pack_grounded_correction_surface("Actually, truth requires evidence")
|
||||
assert a == b
|
||||
|
||||
|
||||
def test_surface_pack_id_is_correct() -> None:
|
||||
surface = pack_grounded_correction_surface("Actually, truth requires evidence")
|
||||
assert surface is not None
|
||||
assert PACK_ID in surface
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# End-to-end through ChatRuntime — the holdout test case
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_correction_truth_040_now_emits_truth_in_surface() -> None:
|
||||
"""The exact holdout case that this ADR targets:
|
||||
`correction_truth_040` expects `term=['truth']` and was missing it
|
||||
pre-ADR-0060. Through the live ChatRuntime, the surface must now
|
||||
contain ``truth``."""
|
||||
rt = ChatRuntime()
|
||||
response = rt.chat("Actually, truth requires evidence")
|
||||
assert response.grounding_source == "pack"
|
||||
assert "truth" in response.surface.lower()
|
||||
assert "correction" in response.surface.lower()
|
||||
|
||||
|
||||
def test_correction_with_no_pack_lemma_still_grounds() -> None:
|
||||
"""A correction utterance with no pack-resident topical lemma
|
||||
still receives the acknowledgement surface (degrades to the
|
||||
topic-less template), not the universal disclosure."""
|
||||
rt = ChatRuntime()
|
||||
response = rt.chat("No that is wrong")
|
||||
assert response.grounding_source == "pack"
|
||||
assert "correction" in response.surface.lower()
|
||||
assert "Noted topic" not in response.surface
|
||||
Loading…
Reference in a new issue