664e08150c
Wires deterministic, read-only contemplation OVER a completed
``DiscoursePlan`` BEFORE the renderer fires. This is the
"reasoning at meaningful checkpoints" capability — the system
now inspects the global shape of its own articulation plan and
emits SPECULATIVE findings about quality issues the move-by-move
planner couldn't see locally.
Doctrine alignment (ADR-0080)
-----------------------------
* **Read-only** — never mutates the plan, packs, vault, teaching
corpus, or runtime state. Returns findings as a tuple; the
runtime stores them on a read-only property.
* **SPECULATIVE-only** — every finding is stamped
``EpistemicStatus.SPECULATIVE`` by the schema's ``__post_init__``;
the doctrine pin ``test_findings_always_speculative`` keeps that
invariant visible.
* **Deterministic replay** — same plan → byte-identical findings
(same ``substrate_hash``, same ``finding_id``).
* **No parallel learning path** — findings flow to a read-only
observation surface (``runtime.last_plan_findings``). Promotion
to memory still goes through the existing proposal → review →
ratify chain. The offline contemplation miner (Phase 5 target)
is what eventually consumes the findings and emits reviewable
pack-mutation candidates.
v1 rules (``core/contemplation/plan_preflight.py``)
----------------------------------------------------
* ``PLANNER_GAP`` — non-BRIEF mode produced anchor-only depth.
Signals the teaching/cross-pack substrate for that lemma is too
thin for the planner to expand.
* ``WEAK_SURFACE`` — three or more moves share a predicate.
Signals the rendered surface will read mechanical (e.g. three
``belongs_to`` clauses in a row). Fires on today's compound
prompt ``"What is truth, and why does it matter?"`` — the
6-sentence plan uses ``belongs_to`` 3 times.
* ``COVERAGE_GAP`` — every move in a multi-move plan draws from
a single ``FactSource``. Signals one-sided substrate (e.g.
pack-only with no teaching enrichment).
Runtime wiring
--------------
* New ``RuntimeConfig.discourse_contemplation: bool = False`` —
opt-in for now. Default off keeps the cognition eval byte-
identical to Phase 2 (verified 45/45 surface + 45/45 trace_hash).
* New ``ChatRuntime.last_plan_findings`` property — read-only tuple
of ``ContemplationFinding`` records from the most recent turn.
Reset to ``()`` at the start of every plan-engagement call so
findings never leak across turns.
* Contemplation runs AFTER the planner produces a multi-move plan
and BEFORE the renderer fires; the plan itself is not modified.
Demo (config: discourse_contemplation=True)
-------------------------------------------
"What is knowledge?" → planner fast-path; no findings
"Tell me about memory." → 3 moves, distinct predicates;
no findings (good!)
"What is truth, and why does
it matter?" → 6 moves, ``belongs_to`` x 3:
[WEAK_SURFACE] subject='truth'
predicate='predicate_repeats_in_plan'
object='belongs_to'
proposed action: diversify the
relation inventory for 'truth'
(grounds / requires / reveals /
contrasts) so the planner has
more variety to draw from.
"Explain truth." → 3 moves, distinct predicates;
no findings
Tests
-----
* ``tests/test_plan_contemplation.py`` — 11 unit tests pinning
each rule, empty/trivial plans, determinism, and the
SPECULATIVE-only doctrine.
* ``tests/test_plan_contemplation_runtime.py`` — 6 end-to-end
tests proving the runtime wiring: disabled by default,
populated when enabled, reset across turns, deterministic
across runs, all findings SPECULATIVE.
Verification
------------
pytest tests/test_plan_contemplation*.py 17/17 pass
pytest tests/test_discourse_planner_*.py 99/99 pass
pytest tests/test_articulation_demo.py all claims supported
pytest tests/test_narrative_example_intents.py pass
pytest tests/test_runtime_config.py pass
cognition eval OFF vs ON 45/45 surface byte-equal
45/45 trace_hash byte-equal
4/4 aggregate metrics
identical
core test --suite smoke 67/67 pass
core test --suite runtime 19/19 pass
Phases roadmap (logged in commit, not built today)
--------------------------------------------------
* Phase 4 — articulation telemetry enrichment. Emit per-turn
metrics (grounding_ratio, anaphora_engagement, plan_completeness,
novelty, focus_consistency) to the existing telemetry sink so
the offline miner has structured signal.
* Phase 5 — offline contemplation miner. Extend
``core/contemplation`` with a miner that consumes
``last_plan_findings`` streams and emits reviewable
pack-mutation / teaching-corpus expansion proposals. Still
SPECULATIVE; review-gated.
347 lines
11 KiB
Python
347 lines
11 KiB
Python
"""Phase 3 — plan-level contemplation tests.
|
|
|
|
Pins ``core.contemplation.plan_preflight.contemplate_plan`` against
|
|
the three v1 rules:
|
|
|
|
* ``PLANNER_GAP`` — anchor-only depth on non-BRIEF mode
|
|
* ``WEAK_SURFACE`` — predicate repeats >= threshold
|
|
* ``COVERAGE_GAP`` — single ``FactSource`` across multi-move plan
|
|
|
|
Plus invariants:
|
|
|
|
* Empty plans yield no findings.
|
|
* Single-fact BRIEF plans yield no findings (substrate-thin by
|
|
design, not a gap).
|
|
* Determinism: same plan in → same findings (with byte-identical
|
|
``finding_id`` and ``substrate_hash``).
|
|
* All findings are SPECULATIVE (schema's ``__post_init__`` enforces
|
|
this; we still pin it explicitly so the doctrine is visible).
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from core.contemplation.plan_preflight import contemplate_plan
|
|
from core.contemplation.schema import FindingKind
|
|
from generate.discourse_planner import (
|
|
DiscourseMove,
|
|
DiscourseMoveKind,
|
|
DiscoursePlan,
|
|
FactSource,
|
|
GroundedFact,
|
|
)
|
|
from generate.intent import DialogueIntent, IntentTag, ResponseMode
|
|
from teaching.epistemic import EpistemicStatus
|
|
|
|
|
|
def _fact(
|
|
subject: str,
|
|
predicate: str,
|
|
obj: str,
|
|
*,
|
|
source: FactSource = FactSource.PACK,
|
|
source_id: str = "test_pack_v1",
|
|
) -> GroundedFact:
|
|
return GroundedFact(
|
|
subject=subject,
|
|
predicate=predicate,
|
|
obj=obj,
|
|
source=source,
|
|
source_id=source_id,
|
|
)
|
|
|
|
|
|
def _intent(subject: str = "truth") -> DialogueIntent:
|
|
return DialogueIntent(tag=IntentTag.DEFINITION, subject=subject)
|
|
|
|
|
|
def _move(
|
|
kind: DiscourseMoveKind, fact: GroundedFact | None = None,
|
|
) -> DiscourseMove:
|
|
topic = fact.subject if fact is not None else ""
|
|
return DiscourseMove(
|
|
kind=kind,
|
|
topic=topic,
|
|
given=(),
|
|
new=(fact.obj,) if fact is not None else (),
|
|
relation_to_previous=None,
|
|
fact=fact,
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Empty / trivial plans yield no findings
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_empty_plan_yields_no_findings() -> None:
|
|
plan = DiscoursePlan(intent=_intent(), mode=ResponseMode.BRIEF, moves=())
|
|
assert contemplate_plan(plan) == ()
|
|
|
|
|
|
def test_brief_single_move_yields_no_findings() -> None:
|
|
"""BRIEF mode is anchor-only by design; not a gap."""
|
|
plan = DiscoursePlan(
|
|
intent=_intent(),
|
|
mode=ResponseMode.BRIEF,
|
|
moves=(
|
|
_move(
|
|
DiscourseMoveKind.ANCHOR,
|
|
_fact("truth", "is_defined_as", "what is true"),
|
|
),
|
|
),
|
|
)
|
|
assert contemplate_plan(plan) == ()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# PLANNER_GAP — anchor-only depth on non-BRIEF mode
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_planner_gap_fires_when_explain_emits_only_anchor() -> None:
|
|
plan = DiscoursePlan(
|
|
intent=_intent(),
|
|
mode=ResponseMode.EXPLAIN,
|
|
moves=(
|
|
_move(
|
|
DiscourseMoveKind.ANCHOR,
|
|
_fact("truth", "is_defined_as", "what is true"),
|
|
),
|
|
),
|
|
)
|
|
findings = contemplate_plan(plan)
|
|
assert len(findings) == 1
|
|
f = findings[0]
|
|
assert f.kind is FindingKind.PLANNER_GAP
|
|
assert f.subject == "truth"
|
|
assert f.predicate == "anchor_only_depth"
|
|
assert f.object == "explain"
|
|
assert "widen substrate" in f.proposed_action
|
|
assert f.epistemic_status is EpistemicStatus.SPECULATIVE
|
|
|
|
|
|
def test_planner_gap_does_not_fire_on_multi_move_plans() -> None:
|
|
plan = DiscoursePlan(
|
|
intent=_intent(),
|
|
mode=ResponseMode.EXPLAIN,
|
|
moves=(
|
|
_move(
|
|
DiscourseMoveKind.ANCHOR,
|
|
_fact("truth", "is_defined_as", "what is true"),
|
|
),
|
|
_move(
|
|
DiscourseMoveKind.SUPPORT,
|
|
_fact("truth", "belongs_to", "cognition.truth"),
|
|
),
|
|
),
|
|
)
|
|
kinds = {f.kind for f in contemplate_plan(plan)}
|
|
assert FindingKind.PLANNER_GAP not in kinds
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# WEAK_SURFACE — predicate monotony
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_weak_surface_fires_on_three_same_predicate_moves() -> None:
|
|
plan = DiscoursePlan(
|
|
intent=_intent(),
|
|
mode=ResponseMode.PARAGRAPH,
|
|
moves=(
|
|
_move(
|
|
DiscourseMoveKind.ANCHOR,
|
|
_fact("truth", "belongs_to", "cognition.truth"),
|
|
),
|
|
_move(
|
|
DiscourseMoveKind.SUPPORT,
|
|
_fact("truth", "belongs_to", "epistemic.ground"),
|
|
),
|
|
_move(
|
|
DiscourseMoveKind.RELATION,
|
|
_fact("truth", "belongs_to", "logos.core"),
|
|
),
|
|
),
|
|
)
|
|
findings = contemplate_plan(plan)
|
|
weak = [f for f in findings if f.kind is FindingKind.WEAK_SURFACE]
|
|
assert len(weak) == 1
|
|
assert weak[0].subject == "truth"
|
|
assert weak[0].predicate == "predicate_repeats_in_plan"
|
|
assert weak[0].object == "belongs_to"
|
|
|
|
|
|
def test_weak_surface_does_not_fire_on_two_same_predicate() -> None:
|
|
"""Two repetitions read naturally; threshold is 3."""
|
|
plan = DiscoursePlan(
|
|
intent=_intent(),
|
|
mode=ResponseMode.EXPLAIN,
|
|
moves=(
|
|
_move(
|
|
DiscourseMoveKind.ANCHOR,
|
|
_fact("truth", "belongs_to", "cognition.truth"),
|
|
),
|
|
_move(
|
|
DiscourseMoveKind.SUPPORT,
|
|
_fact("truth", "belongs_to", "epistemic.ground"),
|
|
),
|
|
),
|
|
)
|
|
weak = [
|
|
f for f in contemplate_plan(plan)
|
|
if f.kind is FindingKind.WEAK_SURFACE
|
|
]
|
|
assert weak == []
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# COVERAGE_GAP — single FactSource across multi-move plan
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_coverage_gap_fires_on_all_pack_multi_move_plan() -> None:
|
|
plan = DiscoursePlan(
|
|
intent=_intent(),
|
|
mode=ResponseMode.EXPLAIN,
|
|
moves=(
|
|
_move(
|
|
DiscourseMoveKind.ANCHOR,
|
|
_fact("truth", "is_defined_as", "what is true"),
|
|
),
|
|
_move(
|
|
DiscourseMoveKind.SUPPORT,
|
|
_fact("truth", "belongs_to", "cognition.truth"),
|
|
),
|
|
),
|
|
)
|
|
coverage = [
|
|
f for f in contemplate_plan(plan)
|
|
if f.kind is FindingKind.COVERAGE_GAP
|
|
]
|
|
assert len(coverage) == 1
|
|
assert coverage[0].object == "pack"
|
|
assert "single_source_plan" == coverage[0].predicate
|
|
|
|
|
|
def test_coverage_gap_does_not_fire_with_mixed_sources() -> None:
|
|
plan = DiscoursePlan(
|
|
intent=_intent(),
|
|
mode=ResponseMode.EXPLAIN,
|
|
moves=(
|
|
_move(
|
|
DiscourseMoveKind.ANCHOR,
|
|
_fact(
|
|
"truth", "is_defined_as", "what is true",
|
|
source=FactSource.PACK,
|
|
),
|
|
),
|
|
_move(
|
|
DiscourseMoveKind.RELATION,
|
|
_fact(
|
|
"truth", "grounds", "knowledge",
|
|
source=FactSource.TEACHING,
|
|
source_id="cognition_chains_v1",
|
|
),
|
|
),
|
|
),
|
|
)
|
|
coverage = [
|
|
f for f in contemplate_plan(plan)
|
|
if f.kind is FindingKind.COVERAGE_GAP
|
|
]
|
|
assert coverage == []
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Determinism: same plan → byte-identical findings
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_contemplation_is_deterministic() -> None:
|
|
plan = DiscoursePlan(
|
|
intent=_intent(),
|
|
mode=ResponseMode.PARAGRAPH,
|
|
moves=(
|
|
_move(
|
|
DiscourseMoveKind.ANCHOR,
|
|
_fact("truth", "belongs_to", "cognition.truth"),
|
|
),
|
|
_move(
|
|
DiscourseMoveKind.SUPPORT,
|
|
_fact("truth", "belongs_to", "epistemic.ground"),
|
|
),
|
|
_move(
|
|
DiscourseMoveKind.RELATION,
|
|
_fact("truth", "belongs_to", "logos.core"),
|
|
),
|
|
),
|
|
)
|
|
a = contemplate_plan(plan)
|
|
b = contemplate_plan(plan)
|
|
# Byte-equal IDs prove the substrate_hash + identity payload are
|
|
# deterministic; the schema's ``_sha256_16`` derives ``finding_id``
|
|
# from those.
|
|
assert tuple(f.finding_id for f in a) == tuple(f.finding_id for f in b)
|
|
assert tuple(f.substrate_hash for f in a) == tuple(
|
|
f.substrate_hash for f in b
|
|
)
|
|
|
|
|
|
def test_all_findings_remain_speculative() -> None:
|
|
"""Pinned to make the ADR-0080 doctrine visible at the test
|
|
layer. The schema's ``__post_init__`` raises on non-SPECULATIVE
|
|
findings; if a future refactor changes that, this test fails
|
|
first and loudly."""
|
|
plan = DiscoursePlan(
|
|
intent=_intent(),
|
|
mode=ResponseMode.PARAGRAPH,
|
|
moves=(
|
|
_move(
|
|
DiscourseMoveKind.ANCHOR,
|
|
_fact("truth", "belongs_to", "cognition.truth"),
|
|
),
|
|
_move(
|
|
DiscourseMoveKind.SUPPORT,
|
|
_fact("truth", "belongs_to", "epistemic.ground"),
|
|
),
|
|
_move(
|
|
DiscourseMoveKind.RELATION,
|
|
_fact("truth", "belongs_to", "logos.core"),
|
|
),
|
|
),
|
|
)
|
|
findings = contemplate_plan(plan)
|
|
assert findings # would-be-failing rules at the top of this file
|
|
for f in findings:
|
|
assert f.epistemic_status is EpistemicStatus.SPECULATIVE
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Combined: multiple rules fire on a single plan
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_multiple_rules_fire_on_same_plan() -> None:
|
|
"""Both WEAK_SURFACE and COVERAGE_GAP fire when a plan is
|
|
predicate-monotonous AND source-homogeneous."""
|
|
plan = DiscoursePlan(
|
|
intent=_intent(),
|
|
mode=ResponseMode.PARAGRAPH,
|
|
moves=(
|
|
_move(
|
|
DiscourseMoveKind.ANCHOR,
|
|
_fact("truth", "belongs_to", "cognition.truth"),
|
|
),
|
|
_move(
|
|
DiscourseMoveKind.SUPPORT,
|
|
_fact("truth", "belongs_to", "epistemic.ground"),
|
|
),
|
|
_move(
|
|
DiscourseMoveKind.RELATION,
|
|
_fact("truth", "belongs_to", "logos.core"),
|
|
),
|
|
),
|
|
)
|
|
kinds = {f.kind for f in contemplate_plan(plan)}
|
|
assert FindingKind.WEAK_SURFACE in kinds
|
|
assert FindingKind.COVERAGE_GAP in kinds
|