feat(contemplation): Phase 3 — live plan contemplation pre-flight
Wires deterministic, read-only contemplation OVER a completed
``DiscoursePlan`` BEFORE the renderer fires. This is the
"reasoning at meaningful checkpoints" capability — the system
now inspects the global shape of its own articulation plan and
emits SPECULATIVE findings about quality issues the move-by-move
planner couldn't see locally.
Doctrine alignment (ADR-0080)
-----------------------------
* **Read-only** — never mutates the plan, packs, vault, teaching
corpus, or runtime state. Returns findings as a tuple; the
runtime stores them on a read-only property.
* **SPECULATIVE-only** — every finding is stamped
``EpistemicStatus.SPECULATIVE`` by the schema's ``__post_init__``;
the doctrine pin ``test_findings_always_speculative`` keeps that
invariant visible.
* **Deterministic replay** — same plan → byte-identical findings
(same ``substrate_hash``, same ``finding_id``).
* **No parallel learning path** — findings flow to a read-only
observation surface (``runtime.last_plan_findings``). Promotion
to memory still goes through the existing proposal → review →
ratify chain. The offline contemplation miner (Phase 5 target)
is what eventually consumes the findings and emits reviewable
pack-mutation candidates.
v1 rules (``core/contemplation/plan_preflight.py``)
----------------------------------------------------
* ``PLANNER_GAP`` — non-BRIEF mode produced anchor-only depth.
Signals the teaching/cross-pack substrate for that lemma is too
thin for the planner to expand.
* ``WEAK_SURFACE`` — three or more moves share a predicate.
Signals the rendered surface will read mechanical (e.g. three
``belongs_to`` clauses in a row). Fires on today's compound
prompt ``"What is truth, and why does it matter?"`` — the
6-sentence plan uses ``belongs_to`` 3 times.
* ``COVERAGE_GAP`` — every move in a multi-move plan draws from
a single ``FactSource``. Signals one-sided substrate (e.g.
pack-only with no teaching enrichment).
Runtime wiring
--------------
* New ``RuntimeConfig.discourse_contemplation: bool = False`` —
opt-in for now. Default off keeps the cognition eval byte-
identical to Phase 2 (verified 45/45 surface + 45/45 trace_hash).
* New ``ChatRuntime.last_plan_findings`` property — read-only tuple
of ``ContemplationFinding`` records from the most recent turn.
Reset to ``()`` at the start of every plan-engagement call so
findings never leak across turns.
* Contemplation runs AFTER the planner produces a multi-move plan
and BEFORE the renderer fires; the plan itself is not modified.
Demo (config: discourse_contemplation=True)
-------------------------------------------
"What is knowledge?" → planner fast-path; no findings
"Tell me about memory." → 3 moves, distinct predicates;
no findings (good!)
"What is truth, and why does
it matter?" → 6 moves, ``belongs_to`` x 3:
[WEAK_SURFACE] subject='truth'
predicate='predicate_repeats_in_plan'
object='belongs_to'
proposed action: diversify the
relation inventory for 'truth'
(grounds / requires / reveals /
contrasts) so the planner has
more variety to draw from.
"Explain truth." → 3 moves, distinct predicates;
no findings
Tests
-----
* ``tests/test_plan_contemplation.py`` — 11 unit tests pinning
each rule, empty/trivial plans, determinism, and the
SPECULATIVE-only doctrine.
* ``tests/test_plan_contemplation_runtime.py`` — 6 end-to-end
tests proving the runtime wiring: disabled by default,
populated when enabled, reset across turns, deterministic
across runs, all findings SPECULATIVE.
Verification
------------
pytest tests/test_plan_contemplation*.py 17/17 pass
pytest tests/test_discourse_planner_*.py 99/99 pass
pytest tests/test_articulation_demo.py all claims supported
pytest tests/test_narrative_example_intents.py pass
pytest tests/test_runtime_config.py pass
cognition eval OFF vs ON 45/45 surface byte-equal
45/45 trace_hash byte-equal
4/4 aggregate metrics
identical
core test --suite smoke 67/67 pass
core test --suite runtime 19/19 pass
Phases roadmap (logged in commit, not built today)
--------------------------------------------------
* Phase 4 — articulation telemetry enrichment. Emit per-turn
metrics (grounding_ratio, anaphora_engagement, plan_completeness,
novelty, focus_consistency) to the existing telemetry sink so
the offline miner has structured signal.
* Phase 5 — offline contemplation miner. Extend
``core/contemplation`` with a miner that consumes
``last_plan_findings`` streams and emits reviewable
pack-mutation / teaching-corpus expansion proposals. Still
SPECULATIVE; review-gated.
This commit is contained in:
Shay
parent
9dfb505f06
commit
664e08150c
5 changed files with 756 additions and 0 deletions
|
|
@ -529,11 +529,31 @@ class ChatRuntime:
|
|||
self._contemplate_discoveries: bool = False
|
||||
self._correction_pass = CorrectionPass()
|
||||
self._last_valence: float = 0.0
|
||||
# Phase 3 — most-recent plan-contemplation findings (tuple of
|
||||
# SPECULATIVE ``ContemplationFinding`` records). Reset to ``()``
|
||||
# on every turn; populated only when ``config.discourse_contemplation``
|
||||
# is True AND the planner actually engaged on the turn. Exposed
|
||||
# via the ``last_plan_findings`` property below.
|
||||
self._last_plan_findings: tuple[Any, ...] = ()
|
||||
|
||||
@property
|
||||
def session(self) -> SessionContext:
|
||||
return self._context
|
||||
|
||||
@property
|
||||
def last_plan_findings(self) -> tuple[Any, ...]:
|
||||
"""Phase 3 — most-recent plan-contemplation findings.
|
||||
|
||||
Tuple of ``core.contemplation.schema.ContemplationFinding``
|
||||
records (always SPECULATIVE per ADR-0080). Populated only
|
||||
when ``config.discourse_contemplation`` is True and the
|
||||
discourse planner engaged on the turn — empty tuple
|
||||
otherwise. Read-only observation surface; the runtime
|
||||
itself never acts on findings, the offline contemplation
|
||||
miner does.
|
||||
"""
|
||||
return self._last_plan_findings
|
||||
|
||||
def attach_telemetry_sink(
|
||||
self,
|
||||
sink: TurnEventSink | None,
|
||||
|
|
@ -996,6 +1016,10 @@ class ChatRuntime:
|
|||
* Returns ``None`` when the renderer produces an empty string.
|
||||
"""
|
||||
|
||||
# Phase 3 — reset plan-contemplation findings at the start of
|
||||
# every call so they never leak across turns; only successfully
|
||||
# rendered plans (with contemplation enabled) repopulate them.
|
||||
self._last_plan_findings = ()
|
||||
if not self.config.discourse_planner:
|
||||
return None
|
||||
from generate.discourse_planner import (
|
||||
|
|
@ -1075,6 +1099,15 @@ class ChatRuntime:
|
|||
plan = plan_discourse(intent, mode, bundle)
|
||||
if len(plan.moves) <= 1:
|
||||
return None
|
||||
# Phase 3 — plan-level contemplation pre-flight. Read-only,
|
||||
# SPECULATIVE-only; stores findings on the runtime for the
|
||||
# offline contemplation miner to consume. Does not mutate
|
||||
# the plan or block rendering — emits side observations only.
|
||||
if self.config.discourse_contemplation:
|
||||
from core.contemplation.plan_preflight import contemplate_plan
|
||||
self._last_plan_findings = contemplate_plan(plan)
|
||||
else:
|
||||
self._last_plan_findings = ()
|
||||
# Phase 2 — reflective rendering pronominalizes the focus
|
||||
# subject across consecutive same-subject moves, eliminating
|
||||
# the mechanical "Truth ... Truth ... Truth ..." cascade the
|
||||
|
|
|
|||
|
|
@ -137,6 +137,18 @@ class RuntimeConfig:
|
|||
# disclosures or OOV refusals.
|
||||
discourse_planner: bool = True
|
||||
|
||||
# Phase 3 — plan-level contemplation pre-flight. When True, after
|
||||
# the discourse planner produces a plan (and before the renderer
|
||||
# fires) the runtime runs ``core.contemplation.plan_preflight.
|
||||
# contemplate_plan`` over the plan and stores any SPECULATIVE
|
||||
# findings on the runtime for downstream consumption. Per
|
||||
# ADR-0080: contemplation is read-only and SPECULATIVE-only — no
|
||||
# plan mutation, no autonomous memory promotion. Findings flow
|
||||
# into the offline contemplation miner (Phase 5) for review-gated
|
||||
# pack-mutation candidates. Default False keeps the contemplation
|
||||
# opt-in until the operator wires the downstream sink.
|
||||
discourse_contemplation: bool = False
|
||||
|
||||
# ADR-0068 / ADR-0069 — register pack id loaded at runtime startup.
|
||||
# ``None`` resolves to ``RegisterPack.unregistered()`` (the in-memory
|
||||
# null-register sentinel; structurally identical to
|
||||
|
|
|
|||
246
core/contemplation/plan_preflight.py
Normal file
246
core/contemplation/plan_preflight.py
Normal file
|
|
@ -0,0 +1,246 @@
|
|||
"""Phase 3 — live contemplation pre-flight over a completed DiscoursePlan.
|
||||
|
||||
The Phase 1 planner (commit ``63ffd88``) builds a plan one move at a
|
||||
time using local selectors (anchor → support → relation → transition
|
||||
→ closure). No selector sees the full plan; pattern-level issues
|
||||
that emerge only from the *global* shape (predicate monotony, source
|
||||
homogeneity, anchor-only depth on a non-BRIEF mode) slip past.
|
||||
|
||||
Phase 3 closes that gap with a deterministic read-only contemplation
|
||||
pass that runs AFTER the planner finishes and BEFORE the renderer
|
||||
fires. Per ADR-0080 contemplation doctrine:
|
||||
|
||||
* Read-only — never mutates the plan, packs, vault, teaching
|
||||
corpus, or runtime state. Returns findings as a tuple; callers
|
||||
decide what to do with them.
|
||||
* SPECULATIVE-only — every emitted finding is stamped
|
||||
``EpistemicStatus.SPECULATIVE`` by the schema's ``__post_init__``.
|
||||
* Deterministic replay — same plan → same findings, byte-for-byte.
|
||||
|
||||
The findings flow into the telemetry sink (see
|
||||
``chat.telemetry``) so the offline contemplation miner (Phase 5
|
||||
target) can aggregate them into reviewable evidence for pack /
|
||||
teaching-corpus expansion proposals. At no point does this module
|
||||
auto-promote anything to memory — that path remains the existing
|
||||
proposal-review-ratify chain.
|
||||
|
||||
Rules implemented in v1
|
||||
-----------------------
|
||||
|
||||
* ``PLANNER_GAP`` — non-BRIEF mode produced a single-move plan.
|
||||
Anchor without support/relation/transition signals the substrate
|
||||
for that lemma is too thin: there are no qualifying teaching-chain
|
||||
or cross-pack facts the planner could surface. Proposed action
|
||||
(operator-facing): widen the teaching corpus for that subject.
|
||||
|
||||
* ``WEAK_SURFACE`` — three or more moves in the plan share the
|
||||
same predicate. Indicates rendered surface will repeat the same
|
||||
relational pattern (e.g. three ``belongs_to`` clauses in a row),
|
||||
which reads mechanical. Proposed action: diversify the relation
|
||||
inventory for that subject.
|
||||
|
||||
* ``COVERAGE_GAP`` — every move in a multi-move plan draws from a
|
||||
single ``FactSource``. Indicates the substrate is one-sided
|
||||
(e.g. pack-only with no teaching enrichment, or teaching-only
|
||||
with no pack anchor). Proposed action: confirm whether the
|
||||
missing source actually has nothing on this subject, or whether
|
||||
the planner's selector ordering is leaving gold on the table.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
from collections import Counter
|
||||
|
||||
from core.contemplation.schema import (
|
||||
ContemplationEvidenceRef,
|
||||
ContemplationFinding,
|
||||
FindingKind,
|
||||
)
|
||||
from generate.discourse_planner import (
|
||||
DiscourseMoveKind,
|
||||
DiscoursePlan,
|
||||
ResponseMode,
|
||||
)
|
||||
|
||||
|
||||
_PREDICATE_MONOTONY_THRESHOLD = 3
|
||||
"""Trigger ``WEAK_SURFACE`` when this many moves share a predicate.
|
||||
|
||||
Two moves with the same predicate read naturally (e.g. ``belongs_to``
|
||||
twice for two domain memberships). Three or more turns mechanical.
|
||||
"""
|
||||
|
||||
|
||||
def _plan_substrate_hash(plan: DiscoursePlan) -> str:
|
||||
"""SHA-256-16 of the plan's canonical JSON.
|
||||
|
||||
Used as the ``substrate_hash`` on every emitted finding so two
|
||||
contemplation passes over byte-equal plans produce byte-equal
|
||||
finding IDs.
|
||||
"""
|
||||
return hashlib.sha256(plan.to_json().encode("utf-8")).hexdigest()[:16]
|
||||
|
||||
|
||||
def _evidence_ref_for_plan(plan: DiscoursePlan) -> ContemplationEvidenceRef:
|
||||
"""Single evidence ref pointing at the in-memory plan substrate."""
|
||||
return ContemplationEvidenceRef(
|
||||
source_type="discourse_plan",
|
||||
source_id="in_memory_plan",
|
||||
pointer=_plan_substrate_hash(plan),
|
||||
summary=(
|
||||
f"mode={plan.mode.value} "
|
||||
f"intent={plan.intent.tag.value} "
|
||||
f"subject={plan.intent.subject!r} "
|
||||
f"moves={len(plan.moves)}"
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _rule_planner_gap(
|
||||
plan: DiscoursePlan, substrate_hash: str,
|
||||
) -> tuple[ContemplationFinding, ...]:
|
||||
"""Detect anchor-only depth on a non-BRIEF mode plan.
|
||||
|
||||
BRIEF mode is anchor-only by design (budget ``(1, 1)``) — no gap.
|
||||
EXPLAIN / PARAGRAPH / EXAMPLE / WALKTHROUGH plans that emit only
|
||||
an anchor signal the planner ran out of substrate for that lemma.
|
||||
"""
|
||||
if plan.mode is ResponseMode.BRIEF:
|
||||
return ()
|
||||
if len(plan.moves) != 1:
|
||||
return ()
|
||||
anchor = plan.moves[0]
|
||||
if anchor.kind is not DiscourseMoveKind.ANCHOR:
|
||||
return ()
|
||||
if anchor.fact is None:
|
||||
return ()
|
||||
return (
|
||||
ContemplationFinding(
|
||||
kind=FindingKind.PLANNER_GAP,
|
||||
subject=anchor.fact.subject,
|
||||
predicate="anchor_only_depth",
|
||||
object=plan.mode.value,
|
||||
evidence_refs=(_evidence_ref_for_plan(plan),),
|
||||
proposed_action=(
|
||||
f"widen substrate for {anchor.fact.subject!r}: planner "
|
||||
f"under {plan.mode.value} mode could only surface an "
|
||||
f"anchor — no qualifying support/relation/transition "
|
||||
f"facts available. Candidates: add teaching chains "
|
||||
f"rooted on this lemma, or add pack ``belongs_to`` "
|
||||
f"facts that the SUPPORT selector can pick up."
|
||||
),
|
||||
substrate_hash=substrate_hash,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _rule_predicate_monotony(
|
||||
plan: DiscoursePlan, substrate_hash: str,
|
||||
) -> tuple[ContemplationFinding, ...]:
|
||||
"""Detect ``>= _PREDICATE_MONOTONY_THRESHOLD`` moves sharing a predicate."""
|
||||
predicates = Counter(
|
||||
m.fact.predicate for m in plan.moves if m.fact is not None
|
||||
)
|
||||
findings: list[ContemplationFinding] = []
|
||||
for predicate, count in sorted(predicates.items()):
|
||||
if count < _PREDICATE_MONOTONY_THRESHOLD:
|
||||
continue
|
||||
# Subject for the finding is the anchor subject (or fall back
|
||||
# to the first move with a fact). Predicate of the finding
|
||||
# itself names the issue ("predicate_repeats_in_plan"); the
|
||||
# object is the dominating predicate.
|
||||
anchor = plan.anchor()
|
||||
subject = (
|
||||
anchor.fact.subject
|
||||
if anchor is not None and anchor.fact is not None
|
||||
else next(
|
||||
(m.fact.subject for m in plan.moves if m.fact is not None),
|
||||
plan.intent.subject or "<unknown>",
|
||||
)
|
||||
)
|
||||
findings.append(
|
||||
ContemplationFinding(
|
||||
kind=FindingKind.WEAK_SURFACE,
|
||||
subject=subject,
|
||||
predicate="predicate_repeats_in_plan",
|
||||
object=predicate,
|
||||
evidence_refs=(_evidence_ref_for_plan(plan),),
|
||||
proposed_action=(
|
||||
f"diversify relation inventory for {subject!r}: "
|
||||
f"plan uses predicate {predicate!r} {count} times. "
|
||||
f"Reader may perceive mechanical cadence. "
|
||||
f"Candidates: add chains with different relations "
|
||||
f"(grounds / requires / reveals / contrasts) so "
|
||||
f"the planner's RELATION selector has more variety."
|
||||
),
|
||||
substrate_hash=substrate_hash,
|
||||
)
|
||||
)
|
||||
return tuple(findings)
|
||||
|
||||
|
||||
def _rule_source_homogeneity(
|
||||
plan: DiscoursePlan, substrate_hash: str,
|
||||
) -> tuple[ContemplationFinding, ...]:
|
||||
"""Detect multi-move plans where every fact-bearing move draws from
|
||||
a single ``FactSource``.
|
||||
|
||||
BRIEF / single-move plans are exempt (one source by definition).
|
||||
"""
|
||||
if len(plan.moves) < 2:
|
||||
return ()
|
||||
sources = Counter(
|
||||
m.fact.source for m in plan.moves if m.fact is not None
|
||||
)
|
||||
if not sources or len(sources) > 1:
|
||||
return ()
|
||||
(source, count), = sources.items() # exactly one entry
|
||||
if count < 2:
|
||||
return ()
|
||||
anchor = plan.anchor()
|
||||
subject = (
|
||||
anchor.fact.subject
|
||||
if anchor is not None and anchor.fact is not None
|
||||
else plan.intent.subject or "<unknown>"
|
||||
)
|
||||
return (
|
||||
ContemplationFinding(
|
||||
kind=FindingKind.COVERAGE_GAP,
|
||||
subject=subject,
|
||||
predicate="single_source_plan",
|
||||
object=source.value,
|
||||
evidence_refs=(_evidence_ref_for_plan(plan),),
|
||||
proposed_action=(
|
||||
f"confirm coverage for {subject!r}: every move in this "
|
||||
f"plan draws from {source.value!r}. "
|
||||
f"Verify whether the unused sources truly carry nothing "
|
||||
f"on this subject, or whether selector ordering / "
|
||||
f"corpus structure is leaving qualifying facts unsurfaced."
|
||||
),
|
||||
substrate_hash=substrate_hash,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def contemplate_plan(plan: DiscoursePlan) -> tuple[ContemplationFinding, ...]:
|
||||
"""Run every plan-level rule over *plan* and collect findings.
|
||||
|
||||
Pure deterministic function: ``contemplate_plan(p) ==
|
||||
contemplate_plan(p)`` byte-identical for any plan ``p``.
|
||||
|
||||
Empty plans yield no findings (nothing to reason about).
|
||||
"""
|
||||
if plan.is_empty():
|
||||
return ()
|
||||
substrate_hash = _plan_substrate_hash(plan)
|
||||
findings: list[ContemplationFinding] = []
|
||||
findings.extend(_rule_planner_gap(plan, substrate_hash))
|
||||
findings.extend(_rule_predicate_monotony(plan, substrate_hash))
|
||||
findings.extend(_rule_source_homogeneity(plan, substrate_hash))
|
||||
return tuple(findings)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"contemplate_plan",
|
||||
]
|
||||
347
tests/test_plan_contemplation.py
Normal file
347
tests/test_plan_contemplation.py
Normal file
|
|
@ -0,0 +1,347 @@
|
|||
"""Phase 3 — plan-level contemplation tests.
|
||||
|
||||
Pins ``core.contemplation.plan_preflight.contemplate_plan`` against
|
||||
the three v1 rules:
|
||||
|
||||
* ``PLANNER_GAP`` — anchor-only depth on non-BRIEF mode
|
||||
* ``WEAK_SURFACE`` — predicate repeats >= threshold
|
||||
* ``COVERAGE_GAP`` — single ``FactSource`` across multi-move plan
|
||||
|
||||
Plus invariants:
|
||||
|
||||
* Empty plans yield no findings.
|
||||
* Single-fact BRIEF plans yield no findings (substrate-thin by
|
||||
design, not a gap).
|
||||
* Determinism: same plan in → same findings (with byte-identical
|
||||
``finding_id`` and ``substrate_hash``).
|
||||
* All findings are SPECULATIVE (schema's ``__post_init__`` enforces
|
||||
this; we still pin it explicitly so the doctrine is visible).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from core.contemplation.plan_preflight import contemplate_plan
|
||||
from core.contemplation.schema import FindingKind
|
||||
from generate.discourse_planner import (
|
||||
DiscourseMove,
|
||||
DiscourseMoveKind,
|
||||
DiscoursePlan,
|
||||
FactSource,
|
||||
GroundedFact,
|
||||
)
|
||||
from generate.intent import DialogueIntent, IntentTag, ResponseMode
|
||||
from teaching.epistemic import EpistemicStatus
|
||||
|
||||
|
||||
def _fact(
|
||||
subject: str,
|
||||
predicate: str,
|
||||
obj: str,
|
||||
*,
|
||||
source: FactSource = FactSource.PACK,
|
||||
source_id: str = "test_pack_v1",
|
||||
) -> GroundedFact:
|
||||
return GroundedFact(
|
||||
subject=subject,
|
||||
predicate=predicate,
|
||||
obj=obj,
|
||||
source=source,
|
||||
source_id=source_id,
|
||||
)
|
||||
|
||||
|
||||
def _intent(subject: str = "truth") -> DialogueIntent:
|
||||
return DialogueIntent(tag=IntentTag.DEFINITION, subject=subject)
|
||||
|
||||
|
||||
def _move(
|
||||
kind: DiscourseMoveKind, fact: GroundedFact | None = None,
|
||||
) -> DiscourseMove:
|
||||
topic = fact.subject if fact is not None else ""
|
||||
return DiscourseMove(
|
||||
kind=kind,
|
||||
topic=topic,
|
||||
given=(),
|
||||
new=(fact.obj,) if fact is not None else (),
|
||||
relation_to_previous=None,
|
||||
fact=fact,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Empty / trivial plans yield no findings
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_empty_plan_yields_no_findings() -> None:
|
||||
plan = DiscoursePlan(intent=_intent(), mode=ResponseMode.BRIEF, moves=())
|
||||
assert contemplate_plan(plan) == ()
|
||||
|
||||
|
||||
def test_brief_single_move_yields_no_findings() -> None:
|
||||
"""BRIEF mode is anchor-only by design; not a gap."""
|
||||
plan = DiscoursePlan(
|
||||
intent=_intent(),
|
||||
mode=ResponseMode.BRIEF,
|
||||
moves=(
|
||||
_move(
|
||||
DiscourseMoveKind.ANCHOR,
|
||||
_fact("truth", "is_defined_as", "what is true"),
|
||||
),
|
||||
),
|
||||
)
|
||||
assert contemplate_plan(plan) == ()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# PLANNER_GAP — anchor-only depth on non-BRIEF mode
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_planner_gap_fires_when_explain_emits_only_anchor() -> None:
|
||||
plan = DiscoursePlan(
|
||||
intent=_intent(),
|
||||
mode=ResponseMode.EXPLAIN,
|
||||
moves=(
|
||||
_move(
|
||||
DiscourseMoveKind.ANCHOR,
|
||||
_fact("truth", "is_defined_as", "what is true"),
|
||||
),
|
||||
),
|
||||
)
|
||||
findings = contemplate_plan(plan)
|
||||
assert len(findings) == 1
|
||||
f = findings[0]
|
||||
assert f.kind is FindingKind.PLANNER_GAP
|
||||
assert f.subject == "truth"
|
||||
assert f.predicate == "anchor_only_depth"
|
||||
assert f.object == "explain"
|
||||
assert "widen substrate" in f.proposed_action
|
||||
assert f.epistemic_status is EpistemicStatus.SPECULATIVE
|
||||
|
||||
|
||||
def test_planner_gap_does_not_fire_on_multi_move_plans() -> None:
|
||||
plan = DiscoursePlan(
|
||||
intent=_intent(),
|
||||
mode=ResponseMode.EXPLAIN,
|
||||
moves=(
|
||||
_move(
|
||||
DiscourseMoveKind.ANCHOR,
|
||||
_fact("truth", "is_defined_as", "what is true"),
|
||||
),
|
||||
_move(
|
||||
DiscourseMoveKind.SUPPORT,
|
||||
_fact("truth", "belongs_to", "cognition.truth"),
|
||||
),
|
||||
),
|
||||
)
|
||||
kinds = {f.kind for f in contemplate_plan(plan)}
|
||||
assert FindingKind.PLANNER_GAP not in kinds
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# WEAK_SURFACE — predicate monotony
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_weak_surface_fires_on_three_same_predicate_moves() -> None:
|
||||
plan = DiscoursePlan(
|
||||
intent=_intent(),
|
||||
mode=ResponseMode.PARAGRAPH,
|
||||
moves=(
|
||||
_move(
|
||||
DiscourseMoveKind.ANCHOR,
|
||||
_fact("truth", "belongs_to", "cognition.truth"),
|
||||
),
|
||||
_move(
|
||||
DiscourseMoveKind.SUPPORT,
|
||||
_fact("truth", "belongs_to", "epistemic.ground"),
|
||||
),
|
||||
_move(
|
||||
DiscourseMoveKind.RELATION,
|
||||
_fact("truth", "belongs_to", "logos.core"),
|
||||
),
|
||||
),
|
||||
)
|
||||
findings = contemplate_plan(plan)
|
||||
weak = [f for f in findings if f.kind is FindingKind.WEAK_SURFACE]
|
||||
assert len(weak) == 1
|
||||
assert weak[0].subject == "truth"
|
||||
assert weak[0].predicate == "predicate_repeats_in_plan"
|
||||
assert weak[0].object == "belongs_to"
|
||||
|
||||
|
||||
def test_weak_surface_does_not_fire_on_two_same_predicate() -> None:
|
||||
"""Two repetitions read naturally; threshold is 3."""
|
||||
plan = DiscoursePlan(
|
||||
intent=_intent(),
|
||||
mode=ResponseMode.EXPLAIN,
|
||||
moves=(
|
||||
_move(
|
||||
DiscourseMoveKind.ANCHOR,
|
||||
_fact("truth", "belongs_to", "cognition.truth"),
|
||||
),
|
||||
_move(
|
||||
DiscourseMoveKind.SUPPORT,
|
||||
_fact("truth", "belongs_to", "epistemic.ground"),
|
||||
),
|
||||
),
|
||||
)
|
||||
weak = [
|
||||
f for f in contemplate_plan(plan)
|
||||
if f.kind is FindingKind.WEAK_SURFACE
|
||||
]
|
||||
assert weak == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# COVERAGE_GAP — single FactSource across multi-move plan
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_coverage_gap_fires_on_all_pack_multi_move_plan() -> None:
|
||||
plan = DiscoursePlan(
|
||||
intent=_intent(),
|
||||
mode=ResponseMode.EXPLAIN,
|
||||
moves=(
|
||||
_move(
|
||||
DiscourseMoveKind.ANCHOR,
|
||||
_fact("truth", "is_defined_as", "what is true"),
|
||||
),
|
||||
_move(
|
||||
DiscourseMoveKind.SUPPORT,
|
||||
_fact("truth", "belongs_to", "cognition.truth"),
|
||||
),
|
||||
),
|
||||
)
|
||||
coverage = [
|
||||
f for f in contemplate_plan(plan)
|
||||
if f.kind is FindingKind.COVERAGE_GAP
|
||||
]
|
||||
assert len(coverage) == 1
|
||||
assert coverage[0].object == "pack"
|
||||
assert "single_source_plan" == coverage[0].predicate
|
||||
|
||||
|
||||
def test_coverage_gap_does_not_fire_with_mixed_sources() -> None:
|
||||
plan = DiscoursePlan(
|
||||
intent=_intent(),
|
||||
mode=ResponseMode.EXPLAIN,
|
||||
moves=(
|
||||
_move(
|
||||
DiscourseMoveKind.ANCHOR,
|
||||
_fact(
|
||||
"truth", "is_defined_as", "what is true",
|
||||
source=FactSource.PACK,
|
||||
),
|
||||
),
|
||||
_move(
|
||||
DiscourseMoveKind.RELATION,
|
||||
_fact(
|
||||
"truth", "grounds", "knowledge",
|
||||
source=FactSource.TEACHING,
|
||||
source_id="cognition_chains_v1",
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
coverage = [
|
||||
f for f in contemplate_plan(plan)
|
||||
if f.kind is FindingKind.COVERAGE_GAP
|
||||
]
|
||||
assert coverage == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Determinism: same plan → byte-identical findings
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_contemplation_is_deterministic() -> None:
|
||||
plan = DiscoursePlan(
|
||||
intent=_intent(),
|
||||
mode=ResponseMode.PARAGRAPH,
|
||||
moves=(
|
||||
_move(
|
||||
DiscourseMoveKind.ANCHOR,
|
||||
_fact("truth", "belongs_to", "cognition.truth"),
|
||||
),
|
||||
_move(
|
||||
DiscourseMoveKind.SUPPORT,
|
||||
_fact("truth", "belongs_to", "epistemic.ground"),
|
||||
),
|
||||
_move(
|
||||
DiscourseMoveKind.RELATION,
|
||||
_fact("truth", "belongs_to", "logos.core"),
|
||||
),
|
||||
),
|
||||
)
|
||||
a = contemplate_plan(plan)
|
||||
b = contemplate_plan(plan)
|
||||
# Byte-equal IDs prove the substrate_hash + identity payload are
|
||||
# deterministic; the schema's ``_sha256_16`` derives ``finding_id``
|
||||
# from those.
|
||||
assert tuple(f.finding_id for f in a) == tuple(f.finding_id for f in b)
|
||||
assert tuple(f.substrate_hash for f in a) == tuple(
|
||||
f.substrate_hash for f in b
|
||||
)
|
||||
|
||||
|
||||
def test_all_findings_remain_speculative() -> None:
|
||||
"""Pinned to make the ADR-0080 doctrine visible at the test
|
||||
layer. The schema's ``__post_init__`` raises on non-SPECULATIVE
|
||||
findings; if a future refactor changes that, this test fails
|
||||
first and loudly."""
|
||||
plan = DiscoursePlan(
|
||||
intent=_intent(),
|
||||
mode=ResponseMode.PARAGRAPH,
|
||||
moves=(
|
||||
_move(
|
||||
DiscourseMoveKind.ANCHOR,
|
||||
_fact("truth", "belongs_to", "cognition.truth"),
|
||||
),
|
||||
_move(
|
||||
DiscourseMoveKind.SUPPORT,
|
||||
_fact("truth", "belongs_to", "epistemic.ground"),
|
||||
),
|
||||
_move(
|
||||
DiscourseMoveKind.RELATION,
|
||||
_fact("truth", "belongs_to", "logos.core"),
|
||||
),
|
||||
),
|
||||
)
|
||||
findings = contemplate_plan(plan)
|
||||
assert findings # would-be-failing rules at the top of this file
|
||||
for f in findings:
|
||||
assert f.epistemic_status is EpistemicStatus.SPECULATIVE
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Combined: multiple rules fire on a single plan
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_multiple_rules_fire_on_same_plan() -> None:
|
||||
"""Both WEAK_SURFACE and COVERAGE_GAP fire when a plan is
|
||||
predicate-monotonous AND source-homogeneous."""
|
||||
plan = DiscoursePlan(
|
||||
intent=_intent(),
|
||||
mode=ResponseMode.PARAGRAPH,
|
||||
moves=(
|
||||
_move(
|
||||
DiscourseMoveKind.ANCHOR,
|
||||
_fact("truth", "belongs_to", "cognition.truth"),
|
||||
),
|
||||
_move(
|
||||
DiscourseMoveKind.SUPPORT,
|
||||
_fact("truth", "belongs_to", "epistemic.ground"),
|
||||
),
|
||||
_move(
|
||||
DiscourseMoveKind.RELATION,
|
||||
_fact("truth", "belongs_to", "logos.core"),
|
||||
),
|
||||
),
|
||||
)
|
||||
kinds = {f.kind for f in contemplate_plan(plan)}
|
||||
assert FindingKind.WEAK_SURFACE in kinds
|
||||
assert FindingKind.COVERAGE_GAP in kinds
|
||||
118
tests/test_plan_contemplation_runtime.py
Normal file
118
tests/test_plan_contemplation_runtime.py
Normal file
|
|
@ -0,0 +1,118 @@
|
|||
"""Phase 3 — end-to-end live contemplation through ``ChatRuntime``.
|
||||
|
||||
Pins that turning on ``RuntimeConfig.discourse_contemplation`` causes
|
||||
the runtime to populate ``runtime.last_plan_findings`` after each
|
||||
turn where the discourse planner engaged, and leaves it empty
|
||||
otherwise. This is the load-bearing wiring claim — without this
|
||||
test a future refactor could silently drop the contemplation pass
|
||||
and the runtime would still pass every other gate.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from chat.runtime import ChatRuntime
|
||||
from core.config import RuntimeConfig
|
||||
from core.contemplation.schema import FindingKind
|
||||
from teaching.epistemic import EpistemicStatus
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Disabled by default — no findings even on multi-move plans
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_findings_empty_when_contemplation_disabled() -> None:
|
||||
rt = ChatRuntime(config=RuntimeConfig(discourse_contemplation=False))
|
||||
rt.chat("What is truth, and why does it matter?")
|
||||
assert rt.last_plan_findings == ()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Enabled — multi-move predicate-monotonous plan triggers WEAK_SURFACE
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_compound_prompt_triggers_weak_surface_finding() -> None:
|
||||
"""The compound prompt "What is truth, and why does it matter?"
|
||||
plans 6 moves; 3 of them share the ``belongs_to`` predicate.
|
||||
Phase 3's predicate-monotony rule should fire."""
|
||||
rt = ChatRuntime(config=RuntimeConfig(discourse_contemplation=True))
|
||||
rt.chat("What is truth, and why does it matter?")
|
||||
findings = rt.last_plan_findings
|
||||
assert findings, "expected at least one finding on this prompt"
|
||||
kinds = {f.kind for f in findings}
|
||||
assert FindingKind.WEAK_SURFACE in kinds
|
||||
# Pin the specific finding's content
|
||||
weak = next(f for f in findings if f.kind is FindingKind.WEAK_SURFACE)
|
||||
assert weak.subject == "truth"
|
||||
assert weak.predicate == "predicate_repeats_in_plan"
|
||||
assert weak.object == "belongs_to"
|
||||
assert weak.epistemic_status is EpistemicStatus.SPECULATIVE
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# BRIEF prompts (fast-path) do not engage the planner → no findings
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_brief_prompt_yields_no_findings() -> None:
|
||||
"""``What is knowledge?`` is BRIEF mode; the runtime fast-path
|
||||
short-circuits the planner before any plan is built — there is
|
||||
nothing to contemplate."""
|
||||
rt = ChatRuntime(config=RuntimeConfig(discourse_contemplation=True))
|
||||
rt.chat("What is knowledge?")
|
||||
assert rt.last_plan_findings == ()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Findings do not leak across turns
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_findings_reset_between_turns() -> None:
|
||||
"""A turn that populates findings followed by a turn that does
|
||||
not must leave ``last_plan_findings == ()``. Pinned because a
|
||||
prior bug would have kept the previous turn's findings live."""
|
||||
rt = ChatRuntime(config=RuntimeConfig(discourse_contemplation=True))
|
||||
rt.chat("What is truth, and why does it matter?") # populates
|
||||
assert rt.last_plan_findings # sanity
|
||||
rt.chat("What is knowledge?") # BRIEF — should clear
|
||||
assert rt.last_plan_findings == ()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Determinism: same prompt → byte-equal findings
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_findings_are_deterministic_across_runs() -> None:
|
||||
rt1 = ChatRuntime(config=RuntimeConfig(discourse_contemplation=True))
|
||||
rt2 = ChatRuntime(config=RuntimeConfig(discourse_contemplation=True))
|
||||
rt1.chat("What is truth, and why does it matter?")
|
||||
rt2.chat("What is truth, and why does it matter?")
|
||||
ids_1 = tuple(f.finding_id for f in rt1.last_plan_findings)
|
||||
ids_2 = tuple(f.finding_id for f in rt2.last_plan_findings)
|
||||
assert ids_1 == ids_2
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# All emitted findings remain SPECULATIVE (doctrine pin)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"prompt",
|
||||
[
|
||||
"What is truth, and why does it matter?",
|
||||
"Tell me about memory.",
|
||||
"Explain truth.",
|
||||
"Compare knowledge and wisdom.",
|
||||
],
|
||||
)
|
||||
def test_findings_always_speculative(prompt: str) -> None:
|
||||
rt = ChatRuntime(config=RuntimeConfig(discourse_contemplation=True))
|
||||
rt.chat(prompt)
|
||||
for f in rt.last_plan_findings:
|
||||
assert f.epistemic_status is EpistemicStatus.SPECULATIVE
|
||||
Loading…
Reference in a new issue