Flips ``RuntimeConfig.discourse_planner`` from ``False`` → ``True``
(the architectural intent the planner was designed for) AND adds a
fast-path early return so single-fact prompts pay no extra cost.
Why the flip
------------
The discourse planner apparatus has been fully wired in the codebase
for some time (``generate.discourse_planner.plan_discourse`` /
``plan_compound_discourse`` / ``render_plan``,
``generate.grounding_accessors.grounding_bundle_for``,
``chat.runtime._maybe_apply_discourse_planner``) but gated off behind
this flag. Investigation surfaced that:
* **Cognition eval (45 cases) is byte-identical OFF vs ON** across
both surface and trace_hash projections — the planner's
downstream ``len(plan.moves) <= 1`` gate correctly returns
``None`` for single-fact prompts, leaving them with the exact
existing pack-grounded surface.
* **NARRATIVE / EXAMPLE / EXPLAIN / PARAGRAPH and compound shapes
visibly lift.** ``"Tell me about memory."`` goes from a one-
fragment disclosure to a 3-sentence grounded discourse.
``"What is truth, and why does it matter?"`` — currently refused
as OOV because the flat classifier sees the polluted subject —
becomes a 6-sentence grounded articulation via the compound
bypass.
* **No quality regression on existing benches.** The full bench
suite (determinism / latency / speedup / versor / convergence /
realizer / teaching-loop / articulation) stays 8/8 PASS with
the flag on.
Why the fast-path
-----------------
Default-on uncovered a perf trap: the gate ran
``grounding_bundle_for(lemma)`` (pack + teaching + cross-pack queries)
AND ``plan_discourse(...)`` on EVERY turn, then discarded the
result when ``len(plan.moves) <= 1``. For BRIEF mode the budget
``_MODE_BUDGETS[BRIEF] = (1, 1)`` guarantees plans of length ≤ 1, so
the downstream gate is guaranteed to reject — pure waste. The
register matrix test runtime went from ~30s → ~14 minutes (28x
slowdown) under the naive default-flip before the fast-path landed.
The new short-circuit:
if mode is BRIEF and not compound.is_compound():
return None
skips the bundle query + plan run entirely for the common case.
Compound prompts still flow through (they get auto-upgraded BRIEF
→ EXPLAIN on the line above). Empirical post-fast-path
measurement on a 45-case eval (workers=1):
OFF: 23.31s (1.93 turns/sec)
ON : 17.74s (2.54 turns/sec)
slowdown : 0.76x (flag-ON is actually 24% FASTER — the bundle
work the OFF path also touches downstream is
short-circuited cleanly when not needed)
surface byte-equal: True
trace_hash byte-equal: True
Test updates
------------
* ``test_discourse_planner_render.py`` — invert
``test_default_runtime_config_has_flag_off`` →
``test_default_runtime_config_has_flag_on`` and rename
``test_flag_off_default_unchanged`` →
``test_flag_off_explicit_path_unchanged`` (the OFF path is still
a load-bearing invariant, just no longer the default).
* ``test_narrative_example_intents.py`` — three tests that assert
composer-level provenance tags (``narrative-grounded``,
``example-grounded``, ``relations_chains_v1``) now explicitly
set ``RuntimeConfig(discourse_planner=False)`` so they continue
to exercise the underlying composer. The runtime-level
multi-sentence behavior is pinned separately by
``tests/test_articulation_demo.py``.
Verified
--------
cognition eval (45 cases) OFF ≡ ON byte-identical
pytest tests/test_discourse_planner_* 132/132 pass
pytest tests/test_articulation_demo.py all claims supported
pytest tests/test_narrative_example_intents.py pass
pytest tests/test_runtime_config.py pass
core test --suite smoke 67/67 pass
core test --suite runtime 19/19 pass
core test --suite packs 6/6 pass
Live demo (default config):
"What is knowledge?" → single sentence (BRIEF, fast-path)
"Tell me about memory." → 3 grounded sentences
"What is truth, and why does
it matter?" → 6 grounded sentences (was: OOV)
"Explain truth." → 3 grounded sentences
269 lines
10 KiB
Python
269 lines
10 KiB
Python
"""Phase 3.3 + 3.4 — NARRATIVE and EXAMPLE intent + composer tests.
|
|
|
|
The contracts pinned here:
|
|
|
|
NARRATIVE
|
|
- "Tell me about X" / "Describe X" / "What can you say about X"
|
|
classify as NARRATIVE before falling through to DEFINITION.
|
|
- Composer walks every reviewed chain rooted on X across all
|
|
registered teaching corpora; emits up to max_clauses unique
|
|
(predicate, object) clauses; deterministic ordering.
|
|
- Falls through to OOV invitation when X is unknown.
|
|
|
|
EXAMPLE
|
|
- "Give me an example of X" / "Show an instance of X" /
|
|
"Example of X" classify as EXAMPLE before DEFINITION.
|
|
- Composer surfaces chains where X is the OBJECT (reverse-chain
|
|
access pattern); dedupes by subject; deterministic ordering.
|
|
- Falls through to OOV invitation when X is unknown.
|
|
|
|
Both
|
|
- Surface composes only pack atoms + verbatim chain content +
|
|
fixed template — no content synthesis.
|
|
- Tagged ``grounding_source="teaching"`` (same provenance as
|
|
teaching_grounded_surface — both consume the reviewed corpora).
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import pytest
|
|
|
|
from chat.example_surface import example_grounded_surface
|
|
from chat.narrative_surface import narrative_grounded_surface
|
|
from chat.runtime import ChatRuntime
|
|
from core.config import RuntimeConfig
|
|
from generate.intent import IntentTag, classify_intent
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Intent classification
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.mark.parametrize("prompt", [
|
|
"Tell me about light.",
|
|
"Tell me about parent",
|
|
"Describe truth",
|
|
"Describe photosynthesis.",
|
|
"What can you say about wisdom?",
|
|
"What do you know about memory?",
|
|
])
|
|
def test_narrative_patterns_classify_narrative(prompt: str) -> None:
|
|
intent = classify_intent(prompt)
|
|
assert intent.tag is IntentTag.NARRATIVE
|
|
assert intent.subject
|
|
|
|
|
|
@pytest.mark.parametrize("prompt", [
|
|
"Give me an example of truth.",
|
|
"Show me an instance of knowledge.",
|
|
"Show an example of parent.",
|
|
"Example of meaning",
|
|
])
|
|
def test_example_patterns_classify_example(prompt: str) -> None:
|
|
intent = classify_intent(prompt)
|
|
assert intent.tag is IntentTag.EXAMPLE
|
|
assert intent.subject
|
|
|
|
|
|
def test_narrative_pattern_precedes_definition() -> None:
|
|
"""``What can you say about X?`` could match the generic
|
|
``what is/are X`` pattern — assert NARRATIVE wins on the more
|
|
specific pattern."""
|
|
intent = classify_intent("What can you say about light?")
|
|
assert intent.tag is IntentTag.NARRATIVE
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# NARRATIVE composer — pure function
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_narrative_aggregates_multiple_chains() -> None:
|
|
"""``truth`` appears as the subject of multiple cognition chains;
|
|
the narrative composer emits a clause for each."""
|
|
surface = narrative_grounded_surface("truth")
|
|
assert surface is not None
|
|
assert "narrative-grounded (cognition_chains_v1)" in surface
|
|
assert "truth grounds knowledge" in surface
|
|
assert "truth requires evidence" in surface
|
|
|
|
|
|
def test_narrative_dedupes_by_predicate_object() -> None:
|
|
"""When cause + verification carry the same (connective, object),
|
|
only one clause is emitted."""
|
|
surface = narrative_grounded_surface("light")
|
|
assert surface is not None
|
|
# (light, cause, reveals, truth) + (light, verification, reveals, truth)
|
|
# → one clause "light reveals truth", not two.
|
|
assert surface.count("light reveals truth") == 1
|
|
|
|
|
|
def test_narrative_handles_relations_pack_subject() -> None:
|
|
surface = narrative_grounded_surface("parent")
|
|
assert surface is not None
|
|
# ADR-0067 — ``parent`` is the subject of both the in-pack chain
|
|
# ``parent precedes child`` (relations_chains_v1) and the cross-
|
|
# pack chain ``parent grounds understanding`` (cross_pack_chains_v1).
|
|
# The narrative composer aggregates both; the corpus tag reflects
|
|
# both binding sources.
|
|
assert "relations_chains_v1" in surface
|
|
assert "parent precedes child" in surface
|
|
|
|
|
|
def test_narrative_handles_relations_v2_subject() -> None:
|
|
surface = narrative_grounded_surface("mother")
|
|
assert surface is not None
|
|
assert "narrative-grounded (relations_chains_v2)" in surface
|
|
assert "mother precedes daughter" in surface
|
|
|
|
|
|
def test_narrative_unknown_lemma_returns_none() -> None:
|
|
assert narrative_grounded_surface("photosynthesis") is None
|
|
assert narrative_grounded_surface("xyzunknown") is None
|
|
|
|
|
|
def test_narrative_empty_input_returns_none() -> None:
|
|
assert narrative_grounded_surface("") is None
|
|
assert narrative_grounded_surface(" ") is None
|
|
|
|
|
|
def test_narrative_is_deterministic() -> None:
|
|
a = narrative_grounded_surface("truth")
|
|
b = narrative_grounded_surface("truth")
|
|
assert a == b
|
|
|
|
|
|
def test_narrative_max_clauses_caps_output() -> None:
|
|
"""``max_clauses=1`` should emit just the lexicographically-first
|
|
clause for a multi-chain subject."""
|
|
full = narrative_grounded_surface("truth", max_clauses=8)
|
|
capped = narrative_grounded_surface("truth", max_clauses=1)
|
|
assert full is not None
|
|
assert capped is not None
|
|
assert capped != full
|
|
assert len(capped) < len(full)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# EXAMPLE composer — pure function
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_example_surfaces_reverse_chain() -> None:
|
|
"""``truth`` appears as the object of ``light reveals truth`` —
|
|
the example composer surfaces the chain inverted (X = object)."""
|
|
surface = example_grounded_surface("truth")
|
|
assert surface is not None
|
|
assert "example-grounded (cognition_chains_v1)" in surface
|
|
assert "light reveals truth" in surface
|
|
|
|
|
|
def test_example_aggregates_multiple_subjects() -> None:
|
|
"""``knowledge`` appears as the object of multiple chains; the
|
|
example composer dedupes by subject."""
|
|
surface = example_grounded_surface("knowledge")
|
|
assert surface is not None
|
|
# truth/understanding/evidence all relate to knowledge as object.
|
|
assert "knowledge" in surface
|
|
# Each is listed once at most.
|
|
subjects = ["truth", "understanding", "evidence"]
|
|
found = [s for s in subjects if f"{s}" in surface]
|
|
assert len(found) >= 1
|
|
|
|
|
|
def test_example_handles_relations_object() -> None:
|
|
"""``parent`` appears as object of ``child follows parent`` +
|
|
``family grounds parent`` — multiple examples. ADR-0067 added
|
|
``understanding requires parent`` (cross-pack), which is also
|
|
aggregated; the corpus tag widens to reflect both bindings."""
|
|
surface = example_grounded_surface("parent")
|
|
assert surface is not None
|
|
assert "relations_chains_v1" in surface
|
|
assert "parent" in surface
|
|
|
|
|
|
def test_example_unknown_object_returns_none() -> None:
|
|
assert example_grounded_surface("photosynthesis") is None
|
|
assert example_grounded_surface("xyzunknown") is None
|
|
|
|
|
|
def test_example_is_deterministic() -> None:
|
|
a = example_grounded_surface("truth")
|
|
b = example_grounded_surface("truth")
|
|
assert a == b
|
|
|
|
|
|
def test_example_max_examples_caps_output() -> None:
|
|
capped = example_grounded_surface("knowledge", max_examples=1)
|
|
full = example_grounded_surface("knowledge", max_examples=8)
|
|
assert capped is not None
|
|
assert full is not None
|
|
assert len(capped) <= len(full)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Live runtime — NARRATIVE
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_runtime_narrative_on_known_subject_routes_to_teaching() -> None:
|
|
"""Composer-level invariant: NARRATIVE intent on a pack-resident
|
|
subject routes to the teaching-grounded composer and the provenance
|
|
tag ``narrative-grounded`` lands on the surface.
|
|
|
|
Explicitly disables the discourse planner because the planner
|
|
(default-on as of 2026-05-21) intercepts on NARRATIVE shapes and
|
|
renders a multi-clause surface that drops the composer tag. The
|
|
multi-sentence runtime-level behavior is pinned separately by
|
|
``tests/test_articulation_demo.py``.
|
|
"""
|
|
rt = ChatRuntime(config=RuntimeConfig(discourse_planner=False))
|
|
resp = rt.chat("Tell me about truth.")
|
|
assert resp.grounding_source == "teaching"
|
|
assert "narrative-grounded" in resp.surface
|
|
assert "truth" in resp.surface
|
|
|
|
|
|
def test_runtime_narrative_on_oov_routes_to_oov_invitation() -> None:
|
|
rt = ChatRuntime()
|
|
resp = rt.chat("Describe photosynthesis.")
|
|
assert resp.grounding_source == "oov"
|
|
assert "photosynthesis" in resp.surface
|
|
assert "PackMutationProposal" in resp.surface
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Live runtime — EXAMPLE
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_runtime_example_on_known_object_routes_to_teaching() -> None:
|
|
"""Composer-level invariant for EXAMPLE intent on a pack-resident
|
|
object. Discourse planner explicitly disabled — see
|
|
``test_runtime_narrative_on_known_subject_routes_to_teaching``
|
|
for the rationale.
|
|
"""
|
|
rt = ChatRuntime(config=RuntimeConfig(discourse_planner=False))
|
|
resp = rt.chat("Give me an example of truth.")
|
|
assert resp.grounding_source == "teaching"
|
|
assert "example-grounded" in resp.surface
|
|
assert "light reveals truth" in resp.surface
|
|
|
|
|
|
def test_runtime_example_on_oov_routes_to_oov_invitation() -> None:
|
|
rt = ChatRuntime()
|
|
resp = rt.chat("Example of photosynthesis")
|
|
assert resp.grounding_source == "oov"
|
|
|
|
|
|
def test_runtime_example_on_relations_object() -> None:
|
|
"""Composer-level invariant — EXAMPLE intent on a relations-pack
|
|
object surfaces the relations corpus tag. Discourse planner
|
|
explicitly disabled (default-on overlays its own renderer that
|
|
drops the corpus tag).
|
|
"""
|
|
rt = ChatRuntime(config=RuntimeConfig(discourse_planner=False))
|
|
resp = rt.chat("Give me an example of parent.")
|
|
assert resp.grounding_source == "teaching"
|
|
assert "relations_chains_v1" in resp.surface
|