Adds compound-intent decomposition for prompts that ask multiple
things in one turn ("What is X, and why does it matter?",
"Explain X, but how does it work?", "What is X, and what is Y?").
Three landings in one PR (rule says additive; the three pieces
are inseparable for the runtime hook to do anything useful):
1. generate/intent.py
* New ``CompoundIntent`` frozen dataclass — ordered tuple of
``DialogueIntent`` parts + raw_text + ``.primary`` back-compat
accessor + ``.is_compound()`` helper.
* New ``classify_compound_intent(prompt)`` sibling to
``classify_intent``. Pure, deterministic, byte-stable. Splits
on closed connector list (``,\s+(and|but|because|while)\s+``);
anaphoric tails ("why does it matter") get the prior part's
subject substituted ("why does truth matter") then are
classified independently.
* ``classify_intent`` return shape is untouched — every existing
caller still receives ``DialogueIntent``.
* No new ``IntentTag`` introduced. v1 semantic approximation:
"why does X matter" routes to ``CAUSE(X)``; "matter" means
causal/relevance support, not metaphysical importance.
2. generate/discourse_planner.py
* New ``plan_compound_discourse(compound, mode, bundles)`` —
concatenates per-part sub-plans in source order with a
``TRANSITION`` bridge (fact=None) between consecutive parts.
No cross-part re-sorting.
* New private kw-only ``_exclude_facts`` parameter on
``plan_discourse`` so subsequent sub-plans can avoid emitting
the same facts the prior sub-plans already used (prevents
"Truth is X. Truth is X." duplicates on shared-subject
compounds). Public signature ``(intent, mode, bundle)`` is
unchanged.
3. chat/runtime.py
* Helper ``_maybe_apply_discourse_planner`` now consults the
compound classifier first. When the prompt is multi-part it
builds per-part bundles and calls ``plan_compound_discourse``;
otherwise it follows the previous single-intent path.
* Compound bypass: when upstream tagged the surface ``oov`` /
``none`` because the flat classifier saw a polluted subject
(e.g. ``"truth, and why does it matter"``), but the compound
decomposition reveals a pack-resident primary subject, the
planner engages on the decomposed parts. This narrowly widens
the gate exclusively for compound prompts with substrate.
* BRIEF mode upgrades to EXPLAIN for compound prompts —
single-anchor sub-plans on shared subjects would emit duplicate
anchor sentences in BRIEF.
* Return shape widened to ``tuple[str, str] | None`` —
``(rendered_surface, new_source_tag)``. ``new_source_tag`` is
``"teaching"`` when the plan uses any teaching fact, else
``"pack"`` — so downstream labels reflect actual provenance
even on the compound bypass. Both cold and warm call sites
updated to apply both fields.
24 new tests pin: compound decomposition correctness, source-order
preservation across sub-plans, anaphoric-followup rewriting,
deterministic byte-stable plans, no new IntentTag introduced,
fact-dedup across sub-plans, compound-bypass engagement, and
source-tag correction on planner-engaged surfaces.
Lane re-measurement after 3 compound cases added to cases.jsonl
(24 total cases):
flag off: articulate=0.0833, disclosure=0.1667, unarticulate=0.7500
flag on : articulate=0.9167, disclosure=0.0000, unarticulate=0.0833
Note: disclosure flag-on dropped to 0.0 because the source-tag
correction now correctly labels compound-bypass surfaces as
``pack/teaching`` instead of letting the upstream ``oov`` label
inflate disclosure. The two remaining unarticulate cases flag-on
are the walkthrough prompts targeted by the next landing.
Critical gates all green:
* flag off cognition byte-identical: public 100/100/91.7/100
* smoke suite 67/67
* 32/32 planner tests pass (helper + render + compound)
* 18/18 compound classifier tests pass
193 lines
7.7 KiB
Python
193 lines
7.7 KiB
Python
"""Tests for ``CompoundIntent`` and ``classify_compound_intent``.
|
|
|
|
Pins:
|
|
|
|
* The compound classifier is purely additive — ``classify_intent``
|
|
return shape is untouched.
|
|
* Decomposition is deterministic, byte-stable, and preserves source
|
|
order (no cross-part re-sorting).
|
|
* Anaphoric follow-ups ("why does it matter") rewrite the pronoun
|
|
with the prior part's subject.
|
|
* Prompts without a recognised connector produce exactly one part
|
|
and are byte-equivalent to ``classify_intent``.
|
|
* The "matter" / "work" / "causes it" follow-ups map to existing
|
|
intent tags (CAUSE in v1) — no new IntentTag is introduced.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import pytest
|
|
|
|
from generate.intent import (
|
|
CompoundIntent,
|
|
DialogueIntent,
|
|
IntentTag,
|
|
classify_compound_intent,
|
|
classify_intent,
|
|
)
|
|
|
|
# Imported for type clarity even when only used in assertion side-effects.
|
|
_ = CompoundIntent
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Back-compat: classify_intent untouched
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestSingleIntentUntouched:
|
|
def test_classify_intent_still_returns_dialogue_intent(self) -> None:
|
|
result = classify_intent("What is truth?")
|
|
assert isinstance(result, DialogueIntent)
|
|
|
|
def test_compound_classifier_does_not_change_single_intent_shape(self) -> None:
|
|
# Identical input through both APIs — single-intent payload
|
|
# remains identical.
|
|
flat = classify_intent("What is truth?")
|
|
compound = classify_compound_intent("What is truth?")
|
|
assert compound.parts == (flat,)
|
|
assert compound.primary == flat
|
|
assert not compound.is_compound()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Decomposition behavior
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestCompoundDecomposition:
|
|
def test_what_is_x_and_why_does_it_matter(self) -> None:
|
|
result = classify_compound_intent("What is truth, and why does it matter?")
|
|
assert result.is_compound()
|
|
assert len(result.parts) == 2
|
|
assert result.parts[0].tag is IntentTag.DEFINITION
|
|
assert result.parts[0].subject == "truth"
|
|
# Anaphoric "it" rewritten to "truth"; CAUSE classification.
|
|
assert result.parts[1].tag is IntentTag.CAUSE
|
|
assert result.parts[1].subject == "truth"
|
|
|
|
def test_explain_x_but_also_how_does_it_work(self) -> None:
|
|
result = classify_compound_intent("Explain truth, but how does it work?")
|
|
assert result.is_compound()
|
|
assert result.parts[0].tag is IntentTag.DEFINITION
|
|
assert result.parts[0].subject == "truth"
|
|
# how-does-X-work routes to CAUSE per the existing _HOW_DOES_X_RE
|
|
# rule once the pronoun is rewritten.
|
|
assert result.parts[1].tag is IntentTag.CAUSE
|
|
assert result.parts[1].subject == "truth"
|
|
|
|
def test_what_is_x_because_y(self) -> None:
|
|
# Non-anaphoric trailing fragment — kept as-is, classified
|
|
# independently.
|
|
result = classify_compound_intent(
|
|
"What is truth, because what causes evidence?"
|
|
)
|
|
assert result.is_compound()
|
|
assert result.parts[0].tag is IntentTag.DEFINITION
|
|
assert result.parts[0].subject == "truth"
|
|
assert result.parts[1].tag is IntentTag.CAUSE
|
|
assert result.parts[1].subject == "evidence"
|
|
|
|
def test_decomposition_preserves_source_order(self) -> None:
|
|
# Order in compound must match order of fragments in the prompt
|
|
# — never re-sorted by tag or alphabet.
|
|
result = classify_compound_intent(
|
|
"What is wisdom, and why does it matter?"
|
|
)
|
|
subjects = [p.subject for p in result.parts]
|
|
assert subjects == ["wisdom", "wisdom"]
|
|
tags = [p.tag for p in result.parts]
|
|
assert tags == [IntentTag.DEFINITION, IntentTag.CAUSE]
|
|
|
|
def test_three_part_compound(self) -> None:
|
|
result = classify_compound_intent(
|
|
"What is truth, and what is knowledge, and why does it matter?"
|
|
)
|
|
# Three fragments → three parts. Anaphoric "it" in the trailing
|
|
# fragment refers to the immediately prior subject (knowledge).
|
|
assert len(result.parts) == 3
|
|
assert result.parts[0].subject == "truth"
|
|
assert result.parts[1].subject == "knowledge"
|
|
assert result.parts[2].tag is IntentTag.CAUSE
|
|
assert result.parts[2].subject == "knowledge"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Degenerate / fall-through cases
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestDegenerateCases:
|
|
def test_empty_prompt_returns_single_unknown_part(self) -> None:
|
|
result = classify_compound_intent("")
|
|
assert len(result.parts) == 1
|
|
assert result.parts[0].tag is IntentTag.UNKNOWN
|
|
|
|
def test_no_connector_returns_single_part(self) -> None:
|
|
result = classify_compound_intent("Explain truth.")
|
|
assert len(result.parts) == 1
|
|
assert result.parts[0].tag is IntentTag.DEFINITION
|
|
assert result.parts[0].subject == "truth"
|
|
|
|
def test_unknown_parts_with_empty_subject_are_dropped(self) -> None:
|
|
# The only fragments that get dropped are UNKNOWN with empty
|
|
# subject — they carry no useful planning signal. UNKNOWN
|
|
# parts that carry a non-empty subject are still preserved
|
|
# (the planner will simply not ground them, which is honest).
|
|
result = classify_compound_intent("foo, and bar")
|
|
# Both classify to UNKNOWN with non-empty subjects ⇒ both kept.
|
|
assert len(result.parts) == 2
|
|
assert all(p.tag is IntentTag.UNKNOWN for p in result.parts)
|
|
|
|
def test_pure_whitespace_fragments_fall_back_to_flat(self) -> None:
|
|
# Constructed so every split fragment is empty after stripping.
|
|
result = classify_compound_intent(", and ,")
|
|
# No usable parts ⇒ compound layer falls back to a single-part
|
|
# shape so callers always see at least one part.
|
|
assert len(result.parts) == 1
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Determinism
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestCompoundDeterminism:
|
|
@pytest.mark.parametrize(
|
|
"prompt",
|
|
[
|
|
"What is truth, and why does it matter?",
|
|
"Explain memory, but how does it work?",
|
|
"What is truth, and what is knowledge?",
|
|
"What is truth?",
|
|
"",
|
|
],
|
|
)
|
|
def test_byte_stable_across_calls(self, prompt: str) -> None:
|
|
results = [classify_compound_intent(prompt) for _ in range(8)]
|
|
assert len({r for r in results}) == 1
|
|
|
|
def test_compound_intent_is_frozen(self) -> None:
|
|
result = classify_compound_intent("What is truth?")
|
|
with pytest.raises((AttributeError, TypeError)):
|
|
result.parts = () # type: ignore[misc]
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Doctrine: no new IntentTag introduced
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestNoNewIntentTagIntroduced:
|
|
def test_intent_tag_membership_unchanged(self) -> None:
|
|
# The compound layer must not add IMPORTANCE / MATTER /
|
|
# any other tag. "Why does it matter?" maps to CAUSE.
|
|
names = {tag.name for tag in IntentTag}
|
|
assert "IMPORTANCE" not in names
|
|
assert "MATTER" not in names
|
|
|
|
def test_why_does_it_matter_maps_to_cause(self) -> None:
|
|
result = classify_compound_intent(
|
|
"What is truth, and why does it matter?"
|
|
)
|
|
assert result.parts[1].tag is IntentTag.CAUSE
|