feat(adr-0062): composed teaching-grounded surface (chain-of-chains)
Pre-ADR-0062, the teaching-grounded composer emitted exactly one
reviewed chain per surface — "light reveals truth" — even when the
corpus already contained an immediate follow-up "truth grounds
knowledge". With 21 active chains after curriculum saturation v2,
many grounded prompts had a corpus-ratified follow-up the composer
silently dropped.
ADR-0062 adds the composed composer + an opt-in config flag:
flag OFF (default):
light — teaching-grounded (cognition_chains_v1): cognition.illumination;
logos.core. light reveals truth (cognition.truth). No session evidence yet.
flag ON:
light — teaching-grounded (cognition_chains_v1): cognition.illumination;
logos.core. light reveals truth (cognition.truth), which grounds
knowledge (cognition.knowledge). No session evidence yet.
Follow-up resolution:
- prefer cause; fall back to verification (deterministic preference)
- cycle guard: 1-step cycles (A→B, B→A) blocked
- pack-residency guard: follow-up's object must be pack-resident
- bounded depth: v1 follows exactly one hop
- degrades to single-chain BYTE-IDENTICALLY when no follow-up
survives the guards (drop-in replacement)
Trust-boundary invariants preserved:
- Every visible non-template token is lemma / pack-domain /
humanize_predicate connective / template constant. Only added
template constant: ", which "
- Deterministic: same chains → same surface bytes
- Default-False flag pattern mirrors ADR-0047/0058
- `versor_condition < 1e-6` invariant untouched (surface composition only)
Cognition lane null-drop invariant CI-pinned:
Composed mode emits a strictly LONGER surface (extra follow-up
clause); every expected_term passing flag-OFF must still pass flag-ON.
Asserted in test_cognition_lane_metrics_unchanged_with_composed_flag
for both public and holdout splits. If a future change drops tokens,
the test fails as a deliberate regression.
public flag OFF: intent 100% / surface 100% / term 91.7% / versor 100%
public flag ON : intent 100% / surface 100% / term 91.7% / versor 100% (identical)
holdout flag OFF: intent 100% / surface 100% / term 83.3% / versor 100%
holdout flag ON : intent 100% / surface 100% / term 83.3% / versor 100% (identical)
Live-prompt lift visible on ~12 of 21 active chains; the rest hit
cycle or pack-residency guards. Saturation v2's clusters were
authored partly with composition in mind (thought→meaning→
understanding, inference→evidence→knowledge, etc.).
- core/config.py — `RuntimeConfig.composed_surface: bool = False`
- chat/teaching_grounding.py — `teaching_grounded_surface_composed`
sibling to `teaching_grounded_surface`
- chat/runtime.py — dispatch branch in `_maybe_pack_grounded_surface`
selects composed vs single-chain based on config flag
- tests/test_composed_surface.py — 11 tests pin: function-level
(None on no chain / degrades when no follow-up / two-clause when
follow-up exists / includes intermediate + final domains /
deterministic / cycle guard / trust label preserved); runtime
integration (default single-chain / flag-on composed / frozen
config); cognition-lane null-drop invariant.
Lanes (regression): smoke 67 / cognition 121 / teaching 17 /
composed-surface 11 — all green.
This commit is contained in:
Shay
parent
a0edbb4bdb
commit
c492014815
6 changed files with 510 additions and 1 deletions
|
|
@ -19,6 +19,7 @@ from chat.pack_grounding import (
|
|||
)
|
||||
from chat.teaching_grounding import (
|
||||
teaching_grounded_surface,
|
||||
teaching_grounded_surface_composed,
|
||||
TEACHING_CORPUS_ID as _TEACHING_CORPUS_ID,
|
||||
)
|
||||
from chat.refusal import (
|
||||
|
|
@ -668,7 +669,17 @@ class ChatRuntime:
|
|||
lemma = (intent.subject or "").strip()
|
||||
if not lemma:
|
||||
return None
|
||||
surface = teaching_grounded_surface(lemma, intent.tag)
|
||||
# ADR-0062 — when ``composed_surface`` is enabled, the
|
||||
# teaching-grounded composer extends the single-chain
|
||||
# surface with a follow-up chain whose subject equals the
|
||||
# initial chain's object. Backward-compatible: with the
|
||||
# flag off, the single-chain composer is used; with the
|
||||
# flag on and no follow-up chain available, the composer
|
||||
# degrades to the single-chain surface byte-identically.
|
||||
if self.config.composed_surface:
|
||||
surface = teaching_grounded_surface_composed(lemma, intent.tag)
|
||||
else:
|
||||
surface = teaching_grounded_surface(lemma, intent.tag)
|
||||
return (surface, "teaching") if surface is not None else None
|
||||
# ADR-0053 — CORRECTION acknowledgement. Cold-start CORRECTION
|
||||
# has no prior session turn to apply to; emit a pack-grounded
|
||||
|
|
|
|||
|
|
@ -224,6 +224,104 @@ def teaching_grounded_surface(
|
|||
)
|
||||
|
||||
|
||||
def teaching_grounded_surface_composed(
|
||||
subject_lemma: str, intent_tag: IntentTag,
|
||||
) -> str | None:
|
||||
"""ADR-0062 — chain-of-chains teaching-grounded surface.
|
||||
|
||||
When a chain ``(A, intent_A, conn_A, B)`` exists AND a follow-up
|
||||
chain ``(B, ?, conn_B, C)`` exists for either intent, compose a
|
||||
two-clause surface:
|
||||
|
||||
"{A} — teaching-grounded ({corpus_id}): {dA1}; {dA2}.
|
||||
{A} {conn_A} {B} ({dB1}), which {conn_B} {C} ({dC1}).
|
||||
No session evidence yet."
|
||||
|
||||
Cycle-safe: if ``C == A`` or ``C == B``, the composer falls back
|
||||
to the single-chain surface (no follow-up clause). Bounded depth:
|
||||
v1 follows exactly one hop; deeper chains require a future ADR.
|
||||
|
||||
Follow-up intent preference: prefer ``cause`` when both exist
|
||||
(causal continuation reads more naturally than a verification
|
||||
detour). This preference is deterministic and pack-agnostic.
|
||||
|
||||
Returns ``None`` under the same conditions as
|
||||
``teaching_grounded_surface``. When the initial chain exists
|
||||
but no follow-up does, the composer degrades to the single-chain
|
||||
surface byte-identically — drop-in replacement.
|
||||
"""
|
||||
if not subject_lemma or not isinstance(subject_lemma, str):
|
||||
return None
|
||||
key = subject_lemma.strip().lower()
|
||||
if not key:
|
||||
return None
|
||||
intent_name = _intent_name(intent_tag)
|
||||
if intent_name is None:
|
||||
return None
|
||||
corpus = _corpus_index()
|
||||
chain = corpus.get((key, intent_name))
|
||||
if chain is None:
|
||||
return None
|
||||
pack = _pack_index()
|
||||
subject_domains = pack.get(chain.subject, ())
|
||||
object_domains = pack.get(chain.object, ())
|
||||
if not subject_domains or not object_domains:
|
||||
return None
|
||||
head_subject = "; ".join(
|
||||
subject_domains[: max(1, chain.domains_subject_k)]
|
||||
)
|
||||
head_object_short = "; ".join(
|
||||
object_domains[: max(1, chain.domains_object_k)]
|
||||
)
|
||||
connective = humanize_predicate(chain.connective)
|
||||
|
||||
# Look for a follow-up chain whose subject equals the initial
|
||||
# chain's object. Prefer cause; fall back to verification.
|
||||
follow_up = None
|
||||
for next_intent in ("cause", "verification"):
|
||||
candidate = corpus.get((chain.object, next_intent))
|
||||
if candidate is None:
|
||||
continue
|
||||
# Cycle guard: don't follow if the next object is the initial
|
||||
# subject (1-step cycle) or the same as the current object
|
||||
# (degenerate same-cell mismatch).
|
||||
if candidate.object in (chain.subject, chain.object):
|
||||
continue
|
||||
follow_up = candidate
|
||||
break
|
||||
|
||||
if follow_up is None:
|
||||
# No follow-up available — degrade to single-chain surface
|
||||
# byte-identically with ``teaching_grounded_surface``.
|
||||
return (
|
||||
f"{chain.subject} — teaching-grounded ({TEACHING_CORPUS_ID}): "
|
||||
f"{head_subject}. {chain.subject} {connective} {chain.object} "
|
||||
f"({head_object_short}). No session evidence yet."
|
||||
)
|
||||
|
||||
follow_object_domains = pack.get(follow_up.object, ())
|
||||
if not follow_object_domains:
|
||||
# Follow-up's object isn't pack-resident with semantic domains
|
||||
# — degrade to single-chain surface rather than emit a
|
||||
# partially-grounded composition.
|
||||
return (
|
||||
f"{chain.subject} — teaching-grounded ({TEACHING_CORPUS_ID}): "
|
||||
f"{head_subject}. {chain.subject} {connective} {chain.object} "
|
||||
f"({head_object_short}). No session evidence yet."
|
||||
)
|
||||
|
||||
follow_head = "; ".join(
|
||||
follow_object_domains[: max(1, follow_up.domains_object_k)]
|
||||
)
|
||||
follow_connective = humanize_predicate(follow_up.connective)
|
||||
return (
|
||||
f"{chain.subject} — teaching-grounded ({TEACHING_CORPUS_ID}): "
|
||||
f"{head_subject}. {chain.subject} {connective} {chain.object} "
|
||||
f"({head_object_short}), which {follow_connective} {follow_up.object} "
|
||||
f"({follow_head}). No session evidence yet."
|
||||
)
|
||||
|
||||
|
||||
def has_teaching_chain(subject_lemma: str, intent_tag: IntentTag) -> bool:
|
||||
"""Return True iff a reviewed chain exists for (subject, intent)."""
|
||||
if not subject_lemma or not isinstance(subject_lemma, str):
|
||||
|
|
|
|||
|
|
@ -48,6 +48,16 @@ class RuntimeConfig:
|
|||
# disable to retain the pre-ADR-0046 unconstrained walk.
|
||||
forward_graph_constraint: bool = False
|
||||
|
||||
# ADR-0062 — composed teaching-grounded surface. When enabled,
|
||||
# the teaching-grounded composer extends a single-chain surface
|
||||
# with a follow-up chain whose subject equals the initial chain's
|
||||
# object — producing surfaces like "light reveals truth, which
|
||||
# grounds knowledge" instead of just "light reveals truth".
|
||||
# Default False preserves all pre-ADR-0062 behaviour. Cycle-safe
|
||||
# (won't follow if the next subject has been visited), bounded
|
||||
# depth (max one follow-up chain in v1).
|
||||
composed_surface: bool = False
|
||||
|
||||
|
||||
DEFAULT_IDENTITY_PACK: str = "default_general_v1"
|
||||
DEFAULT_ETHICS_PACK: str = "default_general_ethics_v1"
|
||||
|
|
|
|||
218
docs/decisions/ADR-0062-composed-teaching-grounded-surface.md
Normal file
218
docs/decisions/ADR-0062-composed-teaching-grounded-surface.md
Normal file
|
|
@ -0,0 +1,218 @@
|
|||
# ADR-0062 — Composed Teaching-Grounded Surface (Chain-of-Chains)
|
||||
|
||||
**Status:** Accepted
|
||||
**Date:** 2026-05-18
|
||||
**Author:** Shay
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
Pre-ADR-0062, `teaching_grounded_surface` emitted exactly **one
|
||||
reviewed chain** per surface:
|
||||
|
||||
```
|
||||
light — teaching-grounded (cognition_chains_v1): cognition.illumination;
|
||||
logos.core. light reveals truth (cognition.truth).
|
||||
No session evidence yet.
|
||||
```
|
||||
|
||||
Every grounded prompt produced a single-clause surface, regardless
|
||||
of how many follow-up chains the corpus already contained. With
|
||||
21 active chains in `cognition_chains_v1` (after curriculum
|
||||
saturation v2), many grounded prompts have an immediate corpus-
|
||||
ratified follow-up that the surface composer was silently dropping:
|
||||
|
||||
| Initial chain | Follow-up chain | What single-chain emits | What composed could emit |
|
||||
|---|---|---|---|
|
||||
| `light reveals truth` | `truth grounds knowledge` | `light reveals truth` | `light reveals truth, which grounds knowledge` |
|
||||
| `thought reveals meaning` | `meaning grounds understanding` | `thought reveals meaning` | `thought reveals meaning, which grounds understanding` |
|
||||
| `inference requires evidence` | `evidence grounds knowledge` | `inference requires evidence` | `inference requires evidence, which grounds knowledge` |
|
||||
|
||||
This is the *fluency-from-existing-corpus* gap I called out
|
||||
in the "more packs?" question: the rate-limiter on articulation
|
||||
isn't pack vocabulary, it's surface composition over chains that
|
||||
already exist.
|
||||
|
||||
---
|
||||
|
||||
## Decision
|
||||
|
||||
Add `teaching_grounded_surface_composed(subject, intent_tag)` to
|
||||
`chat/teaching_grounding.py` alongside the existing single-chain
|
||||
composer, and route it via a new opt-in
|
||||
`RuntimeConfig.composed_surface: bool = False`.
|
||||
|
||||
### Surface format
|
||||
|
||||
```
|
||||
"{A} — teaching-grounded ({corpus_id}): {dA1}; {dA2}.
|
||||
{A} {conn_A} {B} ({dB}), which {conn_B} {C} ({dC}).
|
||||
No session evidence yet."
|
||||
```
|
||||
|
||||
Every visible non-template token remains a lemma, a verbatim pack
|
||||
`semantic_domains` string, or a `humanize_predicate`-emitted
|
||||
connective. The new `, which ` linker is the only added template
|
||||
constant.
|
||||
|
||||
### Follow-up resolution rules
|
||||
|
||||
1. Look up an initial chain `(subject, intent)`.
|
||||
2. Look up a follow-up chain whose `subject` equals the initial
|
||||
chain's `object`. Prefer `cause`; fall back to `verification`.
|
||||
(Causal continuation reads more naturally than a verification
|
||||
detour; the preference is deterministic.)
|
||||
3. **Cycle guard.** If the follow-up's `object` equals the initial
|
||||
`subject` OR the initial `object`, do not follow (1-step cycle
|
||||
or degenerate same-cell mismatch).
|
||||
4. **Pack-residency guard.** If the follow-up's `object` is not
|
||||
pack-resident with `semantic_domains`, do not follow (would
|
||||
emit a partially-grounded composition).
|
||||
5. If no follow-up survives the guards, degrade to the
|
||||
single-chain surface **byte-identically**. Drop-in replacement.
|
||||
|
||||
### Bounded depth
|
||||
|
||||
v1 follows **exactly one hop**. Deeper compositions (A→B→C→D) are
|
||||
deferred to a future ADR. The cycle/pack-residency guards alone
|
||||
don't suffice for unbounded depth — a depth-2 chain can re-enter
|
||||
through a different intent. Bounded depth + visited-set check is
|
||||
the natural next step but adds template-shape complexity not
|
||||
needed today.
|
||||
|
||||
### Opt-in flag
|
||||
|
||||
`RuntimeConfig.composed_surface: bool = False`. Default preserves
|
||||
all pre-ADR-0062 behaviour byte-identically. Mirrors the
|
||||
ADR-0047/0058 `forward_graph_constraint` pattern: ship the
|
||||
capability behind a flag, characterise empirically, decide on
|
||||
default behaviour in a follow-up once downstream consumers have
|
||||
observed it on their workloads.
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
```
|
||||
tests/test_composed_surface.py 11 passed
|
||||
- Pure function: None when no chain / degrades when no follow-up /
|
||||
produces two-clause when follow-up exists / includes both
|
||||
intermediate and final domains / deterministic / cycle guard
|
||||
blocks 1-step cycle / preserves trust-boundary label.
|
||||
- Runtime: default keeps single-chain / flag-on uses composed /
|
||||
flag is observable on frozen config.
|
||||
- Null-drop invariant: cognition-lane metrics byte-identical
|
||||
flag OFF vs ON on both public and holdout splits.
|
||||
|
||||
Lanes (regression check):
|
||||
core test --suite smoke 67 passed
|
||||
core test --suite cognition 121 passed
|
||||
core test --suite teaching 17 passed
|
||||
```
|
||||
|
||||
### Cognition-lane null-drop invariant
|
||||
|
||||
Composed mode emits a **strictly longer** surface — every token
|
||||
in the single-chain surface still appears, plus one follow-up
|
||||
clause. So every `expected_term` and `expected_surface_contains`
|
||||
that passed flag-OFF must still pass flag-ON. The contract test
|
||||
`test_cognition_lane_metrics_unchanged_with_composed_flag` runs
|
||||
both public and holdout splits twice (flag OFF vs ON) and asserts
|
||||
all four watched metrics are pair-wise identical:
|
||||
|
||||
| Split | Flag OFF | Flag ON |
|
||||
|---|---|---|
|
||||
| public | 100 / 100 / 91.7 / 100 | **100 / 100 / 91.7 / 100** |
|
||||
| holdout | 100 / 100 / 83.3 / 100 | **100 / 100 / 83.3 / 100** |
|
||||
|
||||
If a future change ever drops tokens in composed mode (e.g.
|
||||
shortens the surface to omit the intermediate object), this test
|
||||
fails as the deliberate regression it is.
|
||||
|
||||
### Live-prompt observable lift
|
||||
|
||||
Composed mode visibly enriches the surface on prompts where
|
||||
follow-ups exist:
|
||||
|
||||
```
|
||||
flag OFF: "Why does light exist?"
|
||||
→ light — teaching-grounded (cognition_chains_v1):
|
||||
cognition.illumination; logos.core. light reveals truth
|
||||
(cognition.truth). No session evidence yet.
|
||||
|
||||
flag ON: "Why does light exist?"
|
||||
→ light — teaching-grounded (cognition_chains_v1):
|
||||
cognition.illumination; logos.core. light reveals truth
|
||||
(cognition.truth), which grounds knowledge
|
||||
(cognition.knowledge). No session evidence yet.
|
||||
```
|
||||
|
||||
Of the 21 active chains, the follow-up resolution succeeds for
|
||||
~12 of them (the rest hit cycle guards or pack-residency
|
||||
guards). Saturation v2's three coherent clusters were authored
|
||||
partly with this composition in mind — `thought reveals meaning`
|
||||
+ `meaning grounds understanding`, `definition grounds concept` +
|
||||
`concept requires definition` (cycle-blocked, degrades cleanly),
|
||||
etc.
|
||||
|
||||
---
|
||||
|
||||
## Consequences
|
||||
|
||||
### What changes
|
||||
|
||||
- `core/config.py` — `RuntimeConfig.composed_surface: bool = False`.
|
||||
- `chat/teaching_grounding.py` —
|
||||
`teaching_grounded_surface_composed(subject, intent_tag)` sibling
|
||||
to `teaching_grounded_surface`.
|
||||
- `chat/runtime.py` — dispatch branch in `_maybe_pack_grounded_surface`
|
||||
for `IntentTag.CAUSE` / `IntentTag.VERIFICATION` selects composed
|
||||
vs single-chain based on the config flag. Single-line change.
|
||||
|
||||
### What does not change
|
||||
|
||||
- The pack-grounded discipline: zero LLM-generated tokens; every
|
||||
visible word is lemma, pack-domain, connective, or template
|
||||
constant.
|
||||
- ADR-0053's cold-start contract: empty session + no chain still
|
||||
emits the universal disclosure.
|
||||
- Default runtime behaviour: byte-identical to pre-ADR-0062 main.
|
||||
- The non-negotiable field invariant
|
||||
(`versor_condition(F) < 1e-6`) is unaffected — this ADR only
|
||||
changes surface composition, not rotor construction or sandwich
|
||||
application.
|
||||
|
||||
---
|
||||
|
||||
## Scope limits
|
||||
|
||||
- **Depth-1 only.** v1 follows one hop. `light reveals truth,
|
||||
which grounds knowledge, which requires evidence` would require
|
||||
a depth-2 composer with visited-set tracking — out of scope
|
||||
here.
|
||||
- **No multi-claim aggregation.** When the same subject has
|
||||
multiple ratified chains (e.g. `knowledge requires evidence`
|
||||
AND `knowledge` is the object of three other chains), the
|
||||
composer still picks one initial chain. Aggregation across
|
||||
multiple grounded views is a separate ADR.
|
||||
- **English path only.** The `, which ` linker and the
|
||||
`humanize_predicate` connectives are English-specific.
|
||||
- **Flag stays off by default.** Operators must opt in. A follow-up
|
||||
ADR will decide on default behaviour after characterising the
|
||||
flag on more workloads (mirrors ADR-0047/0058).
|
||||
|
||||
---
|
||||
|
||||
## Cross-References
|
||||
|
||||
- [ADR-0052](./ADR-0052-teaching-grounded-surface.md) — the
|
||||
single-chain teaching-grounded composer this ADR extends.
|
||||
- [ADR-0053](./ADR-0053-cognition-lane-closure.md) — the cognition-
|
||||
lane closure that exposed the saturation headroom.
|
||||
- [ADR-0058](./ADR-0058-forward-graph-constraint-status.md) —
|
||||
the opt-in-default-False + null-lift-invariant pattern this ADR
|
||||
reuses.
|
||||
- [Curriculum: cognition saturation v2](../curriculum/cognition_saturation_v2.md)
|
||||
— the unit that produced the 21 chains this composer composes
|
||||
over.
|
||||
|
|
@ -71,6 +71,7 @@ ADRs record significant architectural decisions: what was decided, why, what alt
|
|||
| [ADR-0059](ADR-0059-correction-pass-telemetry.md) | `ChatRuntime.correct()` emits a discriminated `"type": "correction"` JSONL event to the existing telemetry sink with `target_turn`, `records_count`, `turn_idxs_affected`, `max_delta_norm`, `mean_delta_norm`, SHA-256 correction-versor digest, pack ids — no raw versor coordinates; deterministic; no-op without sink | **Accepted** (2026-05-18) |
|
||||
| [ADR-0060](ADR-0060-correction-acknowledgment-topic-lemma.md) | CORRECTION acknowledgement surface weaves the first pack-resident topical lemma from the utterance (left-to-right, excluding `correction` itself and `be`/`have` fillers) into a fixed template; backward-compatible with ADR-0053 (no-arg path byte-identical); closes `correction_truth_040` holdout miss; holdout `term_capture_rate` 75.0% → 79.2% | **Accepted** (2026-05-18) |
|
||||
| [ADR-0061](ADR-0061-procedure-intent-pack-grounded-surface.md) | PROCEDURE intent (`"How do I X?"`) routes to new `pack_grounded_procedure_surface`; selector picks **last** pack-resident lemma from verb-phrase subject (object > verb), falls back to verb when object is OOV, returns `None` (→ universal disclosure) for no-pack-lemma utterances; closes `procedure_define_010` (term `concept`) + `procedure_verify_034` (surface); holdout `surface_groundedness` 94.7% → 100.0%; `term_capture_rate` 79.2% → 83.3% | **Accepted** (2026-05-18) |
|
||||
| [ADR-0062](ADR-0062-composed-teaching-grounded-surface.md) | Composed teaching-grounded surface: when a chain `(A, intent_A, conn_A, B)` has a follow-up chain `(B, ?, conn_B, C)`, emit `"{A} {conn_A} {B}, which {conn_B} {C}"` instead of just `"{A} {conn_A} {B}"`; depth-1 (one hop) + cycle guard + pack-residency guard; degrades to single-chain byte-identically when no follow-up survives the guards; opt-in via `RuntimeConfig.composed_surface=False` default; cognition lane null-drop invariant (metrics byte-identical flag OFF/ON) CI-pinned | **Accepted** (2026-05-18) |
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
171
tests/test_composed_surface.py
Normal file
171
tests/test_composed_surface.py
Normal file
|
|
@ -0,0 +1,171 @@
|
|||
"""ADR-0062 — composed teaching-grounded surface (chain-of-chains).
|
||||
|
||||
When a chain ``(A, intent_A, conn_A, B)`` is grounded and a follow-up
|
||||
chain ``(B, ?, conn_B, C)`` exists in the corpus, the composed
|
||||
composer extends the single-chain surface with a second clause:
|
||||
|
||||
"{A} — teaching-grounded ({corpus_id}): {dA}. {A} {conn_A} {B}
|
||||
({dB}), which {conn_B} {C} ({dC}). No session evidence yet."
|
||||
|
||||
This test file pins:
|
||||
|
||||
- Default config keeps the flag off → byte-identical single-chain
|
||||
surface.
|
||||
- Flag-on with a follow-up available → composed two-clause surface.
|
||||
- Flag-on with no follow-up available → composer degrades to the
|
||||
single-chain surface (drop-in replacement; never errors).
|
||||
- Cycle guard: 1-step cycles (A→B, B→A) are not followed.
|
||||
- Determinism: same input → same surface bytes.
|
||||
- Cognition lane: metrics byte-identical flag OFF vs ON on both
|
||||
public and holdout splits (the null-lift invariant for composed
|
||||
surface — composition adds tokens but doesn't drop any).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import replace
|
||||
|
||||
from core.config import RuntimeConfig
|
||||
from chat.runtime import ChatRuntime
|
||||
from chat.teaching_grounding import (
|
||||
teaching_grounded_surface,
|
||||
teaching_grounded_surface_composed,
|
||||
)
|
||||
from generate.intent import IntentTag
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pure-function contract
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_composed_returns_none_when_no_chain() -> None:
|
||||
"""No chain for (subject, intent) → None, matching the
|
||||
single-chain composer's behaviour."""
|
||||
out = teaching_grounded_surface_composed("zzznotalemma", IntentTag.CAUSE)
|
||||
assert out is None
|
||||
|
||||
|
||||
def test_composed_degrades_to_single_chain_when_no_follow_up() -> None:
|
||||
"""``memory verification`` has no follow-up chain whose subject
|
||||
is its object (``recall``) that doesn't cycle back to ``memory``
|
||||
— composer degrades to the single-chain surface byte-identically."""
|
||||
composed = teaching_grounded_surface_composed("memory", IntentTag.VERIFICATION)
|
||||
single = teaching_grounded_surface("memory", IntentTag.VERIFICATION)
|
||||
assert composed is not None
|
||||
assert single is not None
|
||||
assert composed == single
|
||||
|
||||
|
||||
def test_composed_produces_two_clause_when_follow_up_exists() -> None:
|
||||
"""``light cause`` has ``light reveals truth`` AND there exists a
|
||||
follow-up ``truth cause`` (``truth grounds knowledge``). The
|
||||
composed surface must contain both ``light`` and ``knowledge``."""
|
||||
composed = teaching_grounded_surface_composed("light", IntentTag.CAUSE)
|
||||
single = teaching_grounded_surface("light", IntentTag.CAUSE)
|
||||
assert composed is not None
|
||||
assert single is not None
|
||||
assert composed != single
|
||||
# Surface must contain initial subject, intermediate object, and final object.
|
||||
assert "light" in composed
|
||||
assert "truth" in composed
|
||||
assert "knowledge" in composed
|
||||
# And the ", which " connective clause.
|
||||
assert ", which " in composed
|
||||
|
||||
|
||||
def test_composed_includes_intermediate_and_final_domains() -> None:
|
||||
"""Pack-grounded discipline: both the intermediate object's
|
||||
semantic_domains AND the final object's semantic_domains appear
|
||||
verbatim in the composed surface."""
|
||||
from chat.pack_grounding import _pack_index
|
||||
pack = _pack_index()
|
||||
truth_d = pack["truth"]
|
||||
knowledge_d = pack["knowledge"]
|
||||
|
||||
composed = teaching_grounded_surface_composed("light", IntentTag.CAUSE)
|
||||
assert composed is not None
|
||||
assert any(d in composed for d in truth_d[:1])
|
||||
assert any(d in composed for d in knowledge_d[:1])
|
||||
|
||||
|
||||
def test_composed_is_deterministic() -> None:
|
||||
a = teaching_grounded_surface_composed("light", IntentTag.CAUSE)
|
||||
b = teaching_grounded_surface_composed("light", IntentTag.CAUSE)
|
||||
assert a == b
|
||||
|
||||
|
||||
def test_composed_cycle_guard_blocks_one_step_cycle() -> None:
|
||||
"""``memory verification`` → ``memory requires recall``; the only
|
||||
follow-up candidate ``recall cause`` is ``recall reveals memory``
|
||||
which would re-introduce ``memory`` (1-step cycle). Composer
|
||||
must not follow. Surface == single-chain surface."""
|
||||
composed = teaching_grounded_surface_composed("memory", IntentTag.VERIFICATION)
|
||||
single = teaching_grounded_surface("memory", IntentTag.VERIFICATION)
|
||||
assert composed == single
|
||||
|
||||
|
||||
def test_composed_preserves_trust_label() -> None:
|
||||
"""The trailing ``No session evidence yet.`` trust-boundary label
|
||||
must be preserved in both single-chain and composed variants."""
|
||||
composed = teaching_grounded_surface_composed("light", IntentTag.CAUSE)
|
||||
assert composed is not None
|
||||
assert "No session evidence yet." in composed
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Runtime integration via the config flag
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_runtime_default_uses_single_chain() -> None:
|
||||
"""Default ``RuntimeConfig`` keeps ``composed_surface=False`` →
|
||||
runtime emits the single-chain surface for ``Why does light exist?``."""
|
||||
rt = ChatRuntime(config=RuntimeConfig())
|
||||
response = rt.chat("Why does light exist?")
|
||||
expected = teaching_grounded_surface("light", IntentTag.CAUSE)
|
||||
assert response.surface == expected
|
||||
|
||||
|
||||
def test_runtime_with_flag_on_uses_composed() -> None:
|
||||
rt = ChatRuntime(config=replace(RuntimeConfig(), composed_surface=True))
|
||||
response = rt.chat("Why does light exist?")
|
||||
expected = teaching_grounded_surface_composed("light", IntentTag.CAUSE)
|
||||
assert response.surface == expected
|
||||
# And the composed surface is observably different from single.
|
||||
assert response.surface != teaching_grounded_surface("light", IntentTag.CAUSE)
|
||||
|
||||
|
||||
def test_runtime_flag_is_observable_on_frozen_config() -> None:
|
||||
cfg = replace(RuntimeConfig(), composed_surface=True)
|
||||
assert cfg.composed_surface is True
|
||||
assert RuntimeConfig().composed_surface is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Cognition-lane null-lift invariant (composed mode adds tokens, never drops)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_cognition_lane_metrics_unchanged_with_composed_flag() -> None:
|
||||
"""Composed mode emits a strictly longer surface with one
|
||||
additional follow-up clause; every expected_term that passed
|
||||
flag-OFF must still pass flag-ON. Public + holdout splits.
|
||||
If a future change drops tokens in composed mode (e.g. omitting
|
||||
the intermediate object), this test fails as a regression."""
|
||||
from evals.framework import get_lane, run_lane
|
||||
|
||||
lane = get_lane("cognition")
|
||||
watched = ("intent_accuracy", "surface_groundedness",
|
||||
"term_capture_rate", "versor_closure_rate")
|
||||
for split in ("public", "holdout"):
|
||||
off = run_lane(lane, version="v1", split=split,
|
||||
config=RuntimeConfig()).metrics
|
||||
on = run_lane(lane, version="v1", split=split,
|
||||
config=replace(RuntimeConfig(), composed_surface=True)).metrics
|
||||
for m in watched:
|
||||
assert off[m] == on[m], (
|
||||
f"ADR-0062 null-drop invariant broken on split={split!r} "
|
||||
f"metric={m!r}: OFF={off[m]} vs ON={on[m]}. "
|
||||
f"Composed surface should add tokens, never drop them."
|
||||
)
|
||||
Loading…
Reference in a new issue