664e08150c
Wires deterministic, read-only contemplation OVER a completed
``DiscoursePlan`` BEFORE the renderer fires. This is the
"reasoning at meaningful checkpoints" capability — the system
now inspects the global shape of its own articulation plan and
emits SPECULATIVE findings about quality issues the move-by-move
planner couldn't see locally.
Doctrine alignment (ADR-0080)
-----------------------------
* **Read-only** — never mutates the plan, packs, vault, teaching
corpus, or runtime state. Returns findings as a tuple; the
runtime stores them on a read-only property.
* **SPECULATIVE-only** — every finding is stamped
``EpistemicStatus.SPECULATIVE`` by the schema's ``__post_init__``;
the doctrine pin ``test_findings_always_speculative`` keeps that
invariant visible.
* **Deterministic replay** — same plan → byte-identical findings
(same ``substrate_hash``, same ``finding_id``).
* **No parallel learning path** — findings flow to a read-only
observation surface (``runtime.last_plan_findings``). Promotion
to memory still goes through the existing proposal → review →
ratify chain. The offline contemplation miner (Phase 5 target)
is what eventually consumes the findings and emits reviewable
pack-mutation candidates.
v1 rules (``core/contemplation/plan_preflight.py``)
----------------------------------------------------
* ``PLANNER_GAP`` — non-BRIEF mode produced anchor-only depth.
Signals the teaching/cross-pack substrate for that lemma is too
thin for the planner to expand.
* ``WEAK_SURFACE`` — three or more moves share a predicate.
Signals the rendered surface will read mechanical (e.g. three
``belongs_to`` clauses in a row). Fires on today's compound
prompt ``"What is truth, and why does it matter?"`` — the
6-sentence plan uses ``belongs_to`` 3 times.
* ``COVERAGE_GAP`` — every move in a multi-move plan draws from
a single ``FactSource``. Signals one-sided substrate (e.g.
pack-only with no teaching enrichment).
Runtime wiring
--------------
* New ``RuntimeConfig.discourse_contemplation: bool = False`` —
opt-in for now. Default off keeps the cognition eval byte-
identical to Phase 2 (verified 45/45 surface + 45/45 trace_hash).
* New ``ChatRuntime.last_plan_findings`` property — read-only tuple
of ``ContemplationFinding`` records from the most recent turn.
Reset to ``()`` at the start of every plan-engagement call so
findings never leak across turns.
* Contemplation runs AFTER the planner produces a multi-move plan
and BEFORE the renderer fires; the plan itself is not modified.
Demo (config: discourse_contemplation=True)
-------------------------------------------
"What is knowledge?" → planner fast-path; no findings
"Tell me about memory." → 3 moves, distinct predicates;
no findings (good!)
"What is truth, and why does
it matter?" → 6 moves, ``belongs_to`` x 3:
[WEAK_SURFACE] subject='truth'
predicate='predicate_repeats_in_plan'
object='belongs_to'
proposed action: diversify the
relation inventory for 'truth'
(grounds / requires / reveals /
contrasts) so the planner has
more variety to draw from.
"Explain truth." → 3 moves, distinct predicates;
no findings
Tests
-----
* ``tests/test_plan_contemplation.py`` — 11 unit tests pinning
each rule, empty/trivial plans, determinism, and the
SPECULATIVE-only doctrine.
* ``tests/test_plan_contemplation_runtime.py`` — 6 end-to-end
tests proving the runtime wiring: disabled by default,
populated when enabled, reset across turns, deterministic
across runs, all findings SPECULATIVE.
Verification
------------
pytest tests/test_plan_contemplation*.py 17/17 pass
pytest tests/test_discourse_planner_*.py 99/99 pass
pytest tests/test_articulation_demo.py all claims supported
pytest tests/test_narrative_example_intents.py pass
pytest tests/test_runtime_config.py pass
cognition eval OFF vs ON 45/45 surface byte-equal
45/45 trace_hash byte-equal
4/4 aggregate metrics
identical
core test --suite smoke 67/67 pass
core test --suite runtime 19/19 pass
Phases roadmap (logged in commit, not built today)
--------------------------------------------------
* Phase 4 — articulation telemetry enrichment. Emit per-turn
metrics (grounding_ratio, anaphora_engagement, plan_completeness,
novelty, focus_consistency) to the existing telemetry sink so
the offline miner has structured signal.
* Phase 5 — offline contemplation miner. Extend
``core/contemplation`` with a miner that consumes
``last_plan_findings`` streams and emits reviewable
pack-mutation / teaching-corpus expansion proposals. Still
SPECULATIVE; review-gated.
118 lines
4.8 KiB
Python
118 lines
4.8 KiB
Python
"""Phase 3 — end-to-end live contemplation through ``ChatRuntime``.
|
|
|
|
Pins that turning on ``RuntimeConfig.discourse_contemplation`` causes
|
|
the runtime to populate ``runtime.last_plan_findings`` after each
|
|
turn where the discourse planner engaged, and leaves it empty
|
|
otherwise. This is the load-bearing wiring claim — without this
|
|
test a future refactor could silently drop the contemplation pass
|
|
and the runtime would still pass every other gate.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import pytest
|
|
|
|
from chat.runtime import ChatRuntime
|
|
from core.config import RuntimeConfig
|
|
from core.contemplation.schema import FindingKind
|
|
from teaching.epistemic import EpistemicStatus
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Disabled by default — no findings even on multi-move plans
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_findings_empty_when_contemplation_disabled() -> None:
|
|
rt = ChatRuntime(config=RuntimeConfig(discourse_contemplation=False))
|
|
rt.chat("What is truth, and why does it matter?")
|
|
assert rt.last_plan_findings == ()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Enabled — multi-move predicate-monotonous plan triggers WEAK_SURFACE
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_compound_prompt_triggers_weak_surface_finding() -> None:
|
|
"""The compound prompt "What is truth, and why does it matter?"
|
|
plans 6 moves; 3 of them share the ``belongs_to`` predicate.
|
|
Phase 3's predicate-monotony rule should fire."""
|
|
rt = ChatRuntime(config=RuntimeConfig(discourse_contemplation=True))
|
|
rt.chat("What is truth, and why does it matter?")
|
|
findings = rt.last_plan_findings
|
|
assert findings, "expected at least one finding on this prompt"
|
|
kinds = {f.kind for f in findings}
|
|
assert FindingKind.WEAK_SURFACE in kinds
|
|
# Pin the specific finding's content
|
|
weak = next(f for f in findings if f.kind is FindingKind.WEAK_SURFACE)
|
|
assert weak.subject == "truth"
|
|
assert weak.predicate == "predicate_repeats_in_plan"
|
|
assert weak.object == "belongs_to"
|
|
assert weak.epistemic_status is EpistemicStatus.SPECULATIVE
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# BRIEF prompts (fast-path) do not engage the planner → no findings
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_brief_prompt_yields_no_findings() -> None:
|
|
"""``What is knowledge?`` is BRIEF mode; the runtime fast-path
|
|
short-circuits the planner before any plan is built — there is
|
|
nothing to contemplate."""
|
|
rt = ChatRuntime(config=RuntimeConfig(discourse_contemplation=True))
|
|
rt.chat("What is knowledge?")
|
|
assert rt.last_plan_findings == ()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Findings do not leak across turns
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_findings_reset_between_turns() -> None:
|
|
"""A turn that populates findings followed by a turn that does
|
|
not must leave ``last_plan_findings == ()``. Pinned because a
|
|
prior bug would have kept the previous turn's findings live."""
|
|
rt = ChatRuntime(config=RuntimeConfig(discourse_contemplation=True))
|
|
rt.chat("What is truth, and why does it matter?") # populates
|
|
assert rt.last_plan_findings # sanity
|
|
rt.chat("What is knowledge?") # BRIEF — should clear
|
|
assert rt.last_plan_findings == ()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Determinism: same prompt → byte-equal findings
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_findings_are_deterministic_across_runs() -> None:
|
|
rt1 = ChatRuntime(config=RuntimeConfig(discourse_contemplation=True))
|
|
rt2 = ChatRuntime(config=RuntimeConfig(discourse_contemplation=True))
|
|
rt1.chat("What is truth, and why does it matter?")
|
|
rt2.chat("What is truth, and why does it matter?")
|
|
ids_1 = tuple(f.finding_id for f in rt1.last_plan_findings)
|
|
ids_2 = tuple(f.finding_id for f in rt2.last_plan_findings)
|
|
assert ids_1 == ids_2
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# All emitted findings remain SPECULATIVE (doctrine pin)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"prompt",
|
|
[
|
|
"What is truth, and why does it matter?",
|
|
"Tell me about memory.",
|
|
"Explain truth.",
|
|
"Compare knowledge and wisdom.",
|
|
],
|
|
)
|
|
def test_findings_always_speculative(prompt: str) -> None:
|
|
rt = ChatRuntime(config=RuntimeConfig(discourse_contemplation=True))
|
|
rt.chat(prompt)
|
|
for f in rt.last_plan_findings:
|
|
assert f.epistemic_status is EpistemicStatus.SPECULATIVE
|