Flips ``RuntimeConfig.discourse_planner`` from ``False`` → ``True``
(the architectural intent the planner was designed for) AND adds a
fast-path early return so single-fact prompts pay no extra cost.
Why the flip
------------
The discourse planner apparatus has been fully wired in the codebase
for some time (``generate.discourse_planner.plan_discourse`` /
``plan_compound_discourse`` / ``render_plan``,
``generate.grounding_accessors.grounding_bundle_for``,
``chat.runtime._maybe_apply_discourse_planner``) but gated off behind
this flag. Investigation surfaced that:
* **Cognition eval (45 cases) is byte-identical OFF vs ON** across
both surface and trace_hash projections — the planner's
downstream ``len(plan.moves) <= 1`` gate correctly returns
``None`` for single-fact prompts, leaving them with the exact
existing pack-grounded surface.
* **NARRATIVE / EXAMPLE / EXPLAIN / PARAGRAPH and compound shapes
visibly lift.** ``"Tell me about memory."`` goes from a one-
fragment disclosure to a 3-sentence grounded discourse.
``"What is truth, and why does it matter?"`` — currently refused
as OOV because the flat classifier sees the polluted subject —
becomes a 6-sentence grounded articulation via the compound
bypass.
* **No quality regression on existing benches.** The full bench
suite (determinism / latency / speedup / versor / convergence /
realizer / teaching-loop / articulation) stays 8/8 PASS with
the flag on.
Why the fast-path
-----------------
Default-on uncovered a perf trap: the gate ran
``grounding_bundle_for(lemma)`` (pack + teaching + cross-pack queries)
AND ``plan_discourse(...)`` on EVERY turn, then discarded the
result when ``len(plan.moves) <= 1``. For BRIEF mode the budget
``_MODE_BUDGETS[BRIEF] = (1, 1)`` guarantees plans of length ≤ 1, so
the downstream gate is guaranteed to reject — pure waste. The
register matrix test runtime went from ~30s → ~14 minutes (28x
slowdown) under the naive default-flip before the fast-path landed.
The new short-circuit:
if mode is BRIEF and not compound.is_compound():
return None
skips the bundle query + plan run entirely for the common case.
Compound prompts still flow through (they get auto-upgraded BRIEF
→ EXPLAIN on the line above). Empirical post-fast-path
measurement on a 45-case eval (workers=1):
OFF: 23.31s (1.93 turns/sec)
ON : 17.74s (2.54 turns/sec)
slowdown : 0.76x (flag-ON is actually 24% FASTER — the bundle
work the OFF path also touches downstream is
short-circuited cleanly when not needed)
surface byte-equal: True
trace_hash byte-equal: True
Test updates
------------
* ``test_discourse_planner_render.py`` — invert
``test_default_runtime_config_has_flag_off`` →
``test_default_runtime_config_has_flag_on`` and rename
``test_flag_off_default_unchanged`` →
``test_flag_off_explicit_path_unchanged`` (the OFF path is still
a load-bearing invariant, just no longer the default).
* ``test_narrative_example_intents.py`` — three tests that assert
composer-level provenance tags (``narrative-grounded``,
``example-grounded``, ``relations_chains_v1``) now explicitly
set ``RuntimeConfig(discourse_planner=False)`` so they continue
to exercise the underlying composer. The runtime-level
multi-sentence behavior is pinned separately by
``tests/test_articulation_demo.py``.
Verified
--------
cognition eval (45 cases) OFF ≡ ON byte-identical
pytest tests/test_discourse_planner_* 132/132 pass
pytest tests/test_articulation_demo.py all claims supported
pytest tests/test_narrative_example_intents.py pass
pytest tests/test_runtime_config.py pass
core test --suite smoke 67/67 pass
core test --suite runtime 19/19 pass
core test --suite packs 6/6 pass
Live demo (default config):
"What is knowledge?" → single sentence (BRIEF, fast-path)
"Tell me about memory." → 3 grounded sentences
"What is truth, and why does
it matter?" → 6 grounded sentences (was: OOV)
"Explain truth." → 3 grounded sentences
214 lines
8.2 KiB
Python
214 lines
8.2 KiB
Python
"""Tests for ``render_plan`` and the runtime ``discourse_planner`` flag.
|
|
|
|
Step 5 split into two slices:
|
|
|
|
* The pure ``render_plan`` function — deterministic multi-clause
|
|
surface from a :class:`DiscoursePlan`.
|
|
* The runtime hook in ``chat/runtime.py`` — gated by
|
|
``RuntimeConfig.discourse_planner``, default False (flag off must be
|
|
byte-identical to the existing single-sentence path; verified
|
|
separately by the cognition eval).
|
|
|
|
Flag-on integration is exercised on a known cognition-pack lemma so
|
|
the assertions don't depend on private pack contents — only the
|
|
shape and structural properties (multi-sentence count, no walk
|
|
fragment, grounded source) are pinned.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from core.config import RuntimeConfig
|
|
from chat.runtime import ChatRuntime
|
|
from generate.discourse_planner import (
|
|
DialogueIntent,
|
|
DiscourseMove,
|
|
DiscourseMoveKind,
|
|
DiscoursePlan,
|
|
FactSource,
|
|
GroundedFact,
|
|
GroundingBundle,
|
|
IntentTag,
|
|
Relation,
|
|
ResponseMode,
|
|
plan_discourse,
|
|
render_plan,
|
|
)
|
|
|
|
|
|
def _intent() -> DialogueIntent:
|
|
return DialogueIntent(tag=IntentTag.DEFINITION, subject="truth")
|
|
|
|
|
|
def _full_bundle() -> GroundingBundle:
|
|
return GroundingBundle(
|
|
facts=(
|
|
GroundedFact(
|
|
subject="truth", predicate="is_defined_as",
|
|
obj="that which corresponds to reality",
|
|
source=FactSource.PACK,
|
|
source_id="en_core_cognition_v1:truth#gloss",
|
|
),
|
|
GroundedFact(
|
|
subject="truth", predicate="belongs_to",
|
|
obj="epistemic_domain", source=FactSource.PACK,
|
|
source_id="en_core_cognition_v1:truth#domain:0",
|
|
),
|
|
GroundedFact(
|
|
subject="truth", predicate="reveals", obj="knowledge",
|
|
source=FactSource.TEACHING,
|
|
source_id="cognition_chains_v1#cause_truth_reveals_knowledge",
|
|
),
|
|
GroundedFact(
|
|
subject="knowledge", predicate="requires", obj="evidence",
|
|
source=FactSource.TEACHING,
|
|
source_id="cognition_chains_v1#cause_knowledge_requires_evidence",
|
|
),
|
|
)
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# render_plan
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestRenderPlan:
|
|
def test_empty_plan_renders_empty(self) -> None:
|
|
plan = DiscoursePlan(intent=_intent(), mode=ResponseMode.PARAGRAPH)
|
|
assert render_plan(plan) == ""
|
|
|
|
def test_brief_renders_single_sentence(self) -> None:
|
|
plan = plan_discourse(_intent(), ResponseMode.BRIEF, _full_bundle())
|
|
rendered = render_plan(plan)
|
|
assert rendered.count(".") == 1
|
|
assert rendered.endswith(".")
|
|
|
|
def test_paragraph_renders_multi_sentence(self) -> None:
|
|
plan = plan_discourse(_intent(), ResponseMode.PARAGRAPH, _full_bundle())
|
|
rendered = render_plan(plan)
|
|
# PARAGRAPH plan has 5 moves but CLOSURE has no fact, so 4 clauses.
|
|
assert rendered.count(".") >= 2
|
|
|
|
def test_paragraph_uses_canonical_connectives(self) -> None:
|
|
plan = plan_discourse(_intent(), ResponseMode.PARAGRAPH, _full_bundle())
|
|
rendered = render_plan(plan)
|
|
# SUPPORT and RELATION clauses use fixed connectives.
|
|
assert "Furthermore," in rendered
|
|
assert "In turn," in rendered
|
|
|
|
def test_paragraph_transition_uses_consequently(self) -> None:
|
|
plan = plan_discourse(_intent(), ResponseMode.PARAGRAPH, _full_bundle())
|
|
rendered = render_plan(plan)
|
|
assert "Consequently," in rendered
|
|
|
|
def test_render_is_deterministic(self) -> None:
|
|
plan = plan_discourse(_intent(), ResponseMode.PARAGRAPH, _full_bundle())
|
|
a = render_plan(plan)
|
|
b = render_plan(plan)
|
|
assert a == b
|
|
|
|
def test_clause_uses_verbatim_fact_object(self) -> None:
|
|
# No synthesis: every fact's obj must appear verbatim in output.
|
|
plan = plan_discourse(_intent(), ResponseMode.PARAGRAPH, _full_bundle())
|
|
rendered = render_plan(plan)
|
|
for move in plan.moves:
|
|
if move.fact is None:
|
|
continue
|
|
assert move.fact.obj in rendered
|
|
|
|
def test_anchor_uses_is_for_is_defined_as(self) -> None:
|
|
# is_defined_as collapses to natural "is" connective.
|
|
plan = DiscoursePlan(
|
|
intent=_intent(),
|
|
mode=ResponseMode.BRIEF,
|
|
moves=(
|
|
DiscourseMove(
|
|
kind=DiscourseMoveKind.ANCHOR,
|
|
topic="truth",
|
|
new=("truth",),
|
|
fact=GroundedFact(
|
|
subject="truth", predicate="is_defined_as",
|
|
obj="reality-correspondence",
|
|
source=FactSource.PACK,
|
|
source_id="en_core_cognition_v1:truth#gloss",
|
|
),
|
|
),
|
|
),
|
|
)
|
|
rendered = render_plan(plan)
|
|
assert "Truth is reality-correspondence." == rendered
|
|
|
|
def test_closure_without_fact_is_skipped(self) -> None:
|
|
plan = DiscoursePlan(
|
|
intent=_intent(),
|
|
mode=ResponseMode.PARAGRAPH,
|
|
moves=(
|
|
DiscourseMove(
|
|
kind=DiscourseMoveKind.ANCHOR, topic="truth",
|
|
new=("truth",),
|
|
fact=GroundedFact(
|
|
subject="truth", predicate="is_defined_as",
|
|
obj="reality",
|
|
source=FactSource.PACK,
|
|
source_id="en_core_cognition_v1:truth#gloss",
|
|
),
|
|
),
|
|
DiscourseMove(
|
|
kind=DiscourseMoveKind.CLOSURE, topic="truth",
|
|
given=("truth",), relation_to_previous=Relation.ELABORATION,
|
|
fact=None,
|
|
),
|
|
),
|
|
)
|
|
rendered = render_plan(plan)
|
|
assert rendered == "Truth is reality."
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Runtime flag — default off
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestRuntimeFlagDefault:
|
|
def test_default_runtime_config_has_flag_on(self) -> None:
|
|
"""Default flipped to True 2026-05-21 after the cognition eval
|
|
(45 cases) was confirmed byte-identical OFF vs ON across both
|
|
surface and trace_hash projections — single-fact prompts get
|
|
the same output either way; the flag only differentiates
|
|
NARRATIVE / EXAMPLE / PARAGRAPH / EXPLAIN / compound shapes.
|
|
"""
|
|
cfg = RuntimeConfig()
|
|
assert cfg.discourse_planner is True
|
|
|
|
def test_runtime_config_field_exists(self) -> None:
|
|
assert "discourse_planner" in RuntimeConfig.__dataclass_fields__
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Runtime flag — on path engages on pack-grounded EXPLAIN/PARAGRAPH
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestRuntimeFlagOn:
|
|
def test_flag_on_lifts_multi_sentence_on_known_pack_lemma(self) -> None:
|
|
cfg = RuntimeConfig(discourse_planner=True)
|
|
runtime = ChatRuntime(config=cfg)
|
|
response = runtime.chat("Explain truth")
|
|
# When the planner engages, the surface contains a connective
|
|
# from the canonical table. When it doesn't (e.g. truth has no
|
|
# qualifying teaching chain in the live corpus), the test
|
|
# documents that fact rather than failing: lift is conditional
|
|
# on substrate availability.
|
|
if "Furthermore," in response.surface or "In turn," in response.surface:
|
|
assert response.surface.count(".") >= 2
|
|
|
|
def test_flag_off_explicit_path_unchanged(self) -> None:
|
|
"""When the operator explicitly disables the planner the surface
|
|
must remain in the existing single-sentence shape — no planner
|
|
connectives. This pins the OFF path even though the default
|
|
flipped to ON in 2026-05-21.
|
|
"""
|
|
runtime = ChatRuntime(config=RuntimeConfig(discourse_planner=False))
|
|
response = runtime.chat("Explain truth")
|
|
assert "Furthermore," not in response.surface
|
|
assert "Consequently," not in response.surface
|