core/tests/test_compound_intent.py
Shay 7af7892dd8 feat(intent+discourse): CompoundIntent + sub-plan composition
Adds compound-intent decomposition for prompts that ask multiple
things in one turn ("What is X, and why does it matter?",
"Explain X, but how does it work?", "What is X, and what is Y?").

Three landings in one PR (rule says additive; the three pieces
are inseparable for the runtime hook to do anything useful):

1. generate/intent.py
   * New ``CompoundIntent`` frozen dataclass — ordered tuple of
     ``DialogueIntent`` parts + raw_text + ``.primary`` back-compat
     accessor + ``.is_compound()`` helper.
   * New ``classify_compound_intent(prompt)`` sibling to
     ``classify_intent``.  Pure, deterministic, byte-stable.  Splits
     on closed connector list (``,\s+(and|but|because|while)\s+``);
     anaphoric tails ("why does it matter") get the prior part's
     subject substituted ("why does truth matter") then are
     classified independently.
   * ``classify_intent`` return shape is untouched — every existing
     caller still receives ``DialogueIntent``.
   * No new ``IntentTag`` introduced.  v1 semantic approximation:
     "why does X matter" routes to ``CAUSE(X)``; "matter" means
     causal/relevance support, not metaphysical importance.

2. generate/discourse_planner.py
   * New ``plan_compound_discourse(compound, mode, bundles)`` —
     concatenates per-part sub-plans in source order with a
     ``TRANSITION`` bridge (fact=None) between consecutive parts.
     No cross-part re-sorting.
   * New private kw-only ``_exclude_facts`` parameter on
     ``plan_discourse`` so subsequent sub-plans can avoid emitting
     the same facts the prior sub-plans already used (prevents
     "Truth is X. Truth is X." duplicates on shared-subject
     compounds).  Public signature ``(intent, mode, bundle)`` is
     unchanged.

3. chat/runtime.py
   * Helper ``_maybe_apply_discourse_planner`` now consults the
     compound classifier first.  When the prompt is multi-part it
     builds per-part bundles and calls ``plan_compound_discourse``;
     otherwise it follows the previous single-intent path.
   * Compound bypass: when upstream tagged the surface ``oov`` /
     ``none`` because the flat classifier saw a polluted subject
     (e.g. ``"truth, and why does it matter"``), but the compound
     decomposition reveals a pack-resident primary subject, the
     planner engages on the decomposed parts.  This narrowly widens
     the gate exclusively for compound prompts with substrate.
   * BRIEF mode upgrades to EXPLAIN for compound prompts —
     single-anchor sub-plans on shared subjects would emit duplicate
     anchor sentences in BRIEF.
   * Return shape widened to ``tuple[str, str] | None`` —
     ``(rendered_surface, new_source_tag)``.  ``new_source_tag`` is
     ``"teaching"`` when the plan uses any teaching fact, else
     ``"pack"`` — so downstream labels reflect actual provenance
     even on the compound bypass.  Both cold and warm call sites
     updated to apply both fields.

24 new tests pin: compound decomposition correctness, source-order
preservation across sub-plans, anaphoric-followup rewriting,
deterministic byte-stable plans, no new IntentTag introduced,
fact-dedup across sub-plans, compound-bypass engagement, and
source-tag correction on planner-engaged surfaces.

Lane re-measurement after 3 compound cases added to cases.jsonl
(24 total cases):

  flag off: articulate=0.0833, disclosure=0.1667, unarticulate=0.7500
  flag on : articulate=0.9167, disclosure=0.0000, unarticulate=0.0833

Note: disclosure flag-on dropped to 0.0 because the source-tag
correction now correctly labels compound-bypass surfaces as
``pack/teaching`` instead of letting the upstream ``oov`` label
inflate disclosure.  The two remaining unarticulate cases flag-on
are the walkthrough prompts targeted by the next landing.

Critical gates all green:
* flag off cognition byte-identical: public 100/100/91.7/100
* smoke suite 67/67
* 32/32 planner tests pass (helper + render + compound)
* 18/18 compound classifier tests pass
2026-05-19 12:23:58 -07:00

193 lines
7.7 KiB
Python

"""Tests for ``CompoundIntent`` and ``classify_compound_intent``.
Pins:
* The compound classifier is purely additive — ``classify_intent``
return shape is untouched.
* Decomposition is deterministic, byte-stable, and preserves source
order (no cross-part re-sorting).
* Anaphoric follow-ups ("why does it matter") rewrite the pronoun
with the prior part's subject.
* Prompts without a recognised connector produce exactly one part
and are byte-equivalent to ``classify_intent``.
* The "matter" / "work" / "causes it" follow-ups map to existing
intent tags (CAUSE in v1) — no new IntentTag is introduced.
"""
from __future__ import annotations
import pytest
from generate.intent import (
CompoundIntent,
DialogueIntent,
IntentTag,
classify_compound_intent,
classify_intent,
)
# Imported for type clarity even when only used in assertion side-effects.
_ = CompoundIntent
# ---------------------------------------------------------------------------
# Back-compat: classify_intent untouched
# ---------------------------------------------------------------------------
class TestSingleIntentUntouched:
def test_classify_intent_still_returns_dialogue_intent(self) -> None:
result = classify_intent("What is truth?")
assert isinstance(result, DialogueIntent)
def test_compound_classifier_does_not_change_single_intent_shape(self) -> None:
# Identical input through both APIs — single-intent payload
# remains identical.
flat = classify_intent("What is truth?")
compound = classify_compound_intent("What is truth?")
assert compound.parts == (flat,)
assert compound.primary == flat
assert not compound.is_compound()
# ---------------------------------------------------------------------------
# Decomposition behavior
# ---------------------------------------------------------------------------
class TestCompoundDecomposition:
def test_what_is_x_and_why_does_it_matter(self) -> None:
result = classify_compound_intent("What is truth, and why does it matter?")
assert result.is_compound()
assert len(result.parts) == 2
assert result.parts[0].tag is IntentTag.DEFINITION
assert result.parts[0].subject == "truth"
# Anaphoric "it" rewritten to "truth"; CAUSE classification.
assert result.parts[1].tag is IntentTag.CAUSE
assert result.parts[1].subject == "truth"
def test_explain_x_but_also_how_does_it_work(self) -> None:
result = classify_compound_intent("Explain truth, but how does it work?")
assert result.is_compound()
assert result.parts[0].tag is IntentTag.DEFINITION
assert result.parts[0].subject == "truth"
# how-does-X-work routes to CAUSE per the existing _HOW_DOES_X_RE
# rule once the pronoun is rewritten.
assert result.parts[1].tag is IntentTag.CAUSE
assert result.parts[1].subject == "truth"
def test_what_is_x_because_y(self) -> None:
# Non-anaphoric trailing fragment — kept as-is, classified
# independently.
result = classify_compound_intent(
"What is truth, because what causes evidence?"
)
assert result.is_compound()
assert result.parts[0].tag is IntentTag.DEFINITION
assert result.parts[0].subject == "truth"
assert result.parts[1].tag is IntentTag.CAUSE
assert result.parts[1].subject == "evidence"
def test_decomposition_preserves_source_order(self) -> None:
# Order in compound must match order of fragments in the prompt
# — never re-sorted by tag or alphabet.
result = classify_compound_intent(
"What is wisdom, and why does it matter?"
)
subjects = [p.subject for p in result.parts]
assert subjects == ["wisdom", "wisdom"]
tags = [p.tag for p in result.parts]
assert tags == [IntentTag.DEFINITION, IntentTag.CAUSE]
def test_three_part_compound(self) -> None:
result = classify_compound_intent(
"What is truth, and what is knowledge, and why does it matter?"
)
# Three fragments → three parts. Anaphoric "it" in the trailing
# fragment refers to the immediately prior subject (knowledge).
assert len(result.parts) == 3
assert result.parts[0].subject == "truth"
assert result.parts[1].subject == "knowledge"
assert result.parts[2].tag is IntentTag.CAUSE
assert result.parts[2].subject == "knowledge"
# ---------------------------------------------------------------------------
# Degenerate / fall-through cases
# ---------------------------------------------------------------------------
class TestDegenerateCases:
def test_empty_prompt_returns_single_unknown_part(self) -> None:
result = classify_compound_intent("")
assert len(result.parts) == 1
assert result.parts[0].tag is IntentTag.UNKNOWN
def test_no_connector_returns_single_part(self) -> None:
result = classify_compound_intent("Explain truth.")
assert len(result.parts) == 1
assert result.parts[0].tag is IntentTag.DEFINITION
assert result.parts[0].subject == "truth"
def test_unknown_parts_with_empty_subject_are_dropped(self) -> None:
# The only fragments that get dropped are UNKNOWN with empty
# subject — they carry no useful planning signal. UNKNOWN
# parts that carry a non-empty subject are still preserved
# (the planner will simply not ground them, which is honest).
result = classify_compound_intent("foo, and bar")
# Both classify to UNKNOWN with non-empty subjects ⇒ both kept.
assert len(result.parts) == 2
assert all(p.tag is IntentTag.UNKNOWN for p in result.parts)
def test_pure_whitespace_fragments_fall_back_to_flat(self) -> None:
# Constructed so every split fragment is empty after stripping.
result = classify_compound_intent(", and ,")
# No usable parts ⇒ compound layer falls back to a single-part
# shape so callers always see at least one part.
assert len(result.parts) == 1
# ---------------------------------------------------------------------------
# Determinism
# ---------------------------------------------------------------------------
class TestCompoundDeterminism:
@pytest.mark.parametrize(
"prompt",
[
"What is truth, and why does it matter?",
"Explain memory, but how does it work?",
"What is truth, and what is knowledge?",
"What is truth?",
"",
],
)
def test_byte_stable_across_calls(self, prompt: str) -> None:
results = [classify_compound_intent(prompt) for _ in range(8)]
assert len({r for r in results}) == 1
def test_compound_intent_is_frozen(self) -> None:
result = classify_compound_intent("What is truth?")
with pytest.raises((AttributeError, TypeError)):
result.parts = () # type: ignore[misc]
# ---------------------------------------------------------------------------
# Doctrine: no new IntentTag introduced
# ---------------------------------------------------------------------------
class TestNoNewIntentTagIntroduced:
def test_intent_tag_membership_unchanged(self) -> None:
# The compound layer must not add IMPORTANCE / MATTER /
# any other tag. "Why does it matter?" maps to CAUSE.
names = {tag.name for tag in IntentTag}
assert "IMPORTANCE" not in names
assert "MATTER" not in names
def test_why_does_it_matter_maps_to_cause(self) -> None:
result = classify_compound_intent(
"What is truth, and why does it matter?"
)
assert result.parts[1].tag is IntentTag.CAUSE