core/docs/adr/ADR-0049-intent-subject-extraction.md

ADR-0049 — Intent Classifier Head-Noun Subject Extraction

Status: Accepted Date: 2026-05-18 Author: Shay

---

Context

ADR-0048 added a pack-grounded surface for cold-start DEFINITION / RECALL turns where the subject lemma is in en_core_cognition_v1. The eval lift was real but partial. Investigating the misses showed they were not pack gaps — the pack does carry the lemmas — they were subject-extraction gaps in generate/intent.py:

Prompt	Pre-0049 subject	Reason for miss
What is a procedure?	"a procedure"	Article not stripped
What is a relation?	"a relation"	Article not stripped
Why does light exist?	"does light exist"	Aux verb + tail not stripped
Why does knowledge require evidence?	"does knowledge require evidence"	Aux verb + tail not stripped
Does memory require recall?	"Does memory require recall?"	Whole prompt; rule matched full string
Is light a wave?	"Is light a wave?"	Whole prompt; rule matched full string

The _RULES table in generate/intent.py was producing subject spans, not subject lemmas. Downstream consumers (graph_planner.graph_from_intent, ADR-0048 _maybe_pack_grounded_surface, future teaching-store inference) need the lemma — they cannot match a noun phrase like "a procedure" against a lexicon keyed on procedure, and they cannot key a graph node off "does light exist" cleanly.

The cleanest fix is at the classifier boundary: produce a clean lemma in DialogueIntent.subject so every consumer benefits without each implementing its own article-stripping heuristic.

---

Decision

Add a deterministic, pack-agnostic post-processor _normalize_subject(phrase, tag) in generate/intent.py that runs after the rule table fires and rewrites the subject span according to its intent's syntactic shape.

Behaviour by intent

Intent	Transform
DEFINITION / RECALL / PROCEDURE	strip trailing punctuation, strip leading articles; preserve multi-word noun phrases (e.g. "artificial intelligence")
CAUSE / VERIFICATION	strip trailing punctuation, strip leading aux verbs (is, are, does, do, can, could, …), strip leading articles, return the head noun (first remaining token)
CORRECTION	strip trailing punctuation, strip leading articles
UNKNOWN	bypass (preserve raw input for debugging)
COMPARISON / TRANSITIVE_QUERY / FRAME_TRANSFER	already captured by their own named-group regexes; not routed through _RULES

Aux-verb and article sets

Frozen sets in generate/intent.py:

_ARTICLES = frozenset({"a", "an", "the"})
_AUX_VERBS = frozenset({
    "is", "are", "am", "was", "were", "be", "been", "being",
    "does", "do", "did",
    "has", "have", "had",
    "can", "could", "would", "should", "shall", "will",
    "might", "may", "must",
})

These are closed word lists. The normalizer does not depend on the cognition pack, the language pack manifold, or any external state — it is a pure syntactic transform.

Fallback

If stripping aux verbs and articles would empty the subject (e.g. "What is the?"), the normalizer returns the cleaned original phrase rather than producing an empty subject. Downstream consumers (_maybe_pack_grounded_surface) already handle empty subjects correctly (return None), but preserving a non-empty subject keeps debugging output and trace surfaces readable.

---

Why this is doctrine-aligned

CLAUDE.md prohibits opaque LLM fallbacks, stochastic sampling, hidden normalisation. This ADR:

• Is not opaque. Both word sets are static frozen Python sets, visible at module scope. Every transformation rule is explicit.
• Is not stochastic. Identical input produces byte-identical DialogueIntent (test_normalization_is_deterministic).
• Is not hidden normalisation of the algebra. The normalizer rewrites a typed dataclass field, not a versor, not a manifold, not a field state. No hot-path math is touched. No versor_condition invariant is in scope.
• Is not coupled to a specific pack. The aux-verb / article lists are English syntactic categories, not pack vocabulary. The ADR-0048 pack lookup remains the consumer of the cleaner lemma; the classifier itself does not load any pack.

The trust-boundary discipline is preserved: user-controlled text is still escaped at all log/display sites by their respective handlers; this ADR changes only the in-process classification output.

---

Characterisation — core eval cognition

A/B run on the 13-case public cognition split, identical RuntimeConfig except for the merge of this ADR:

Metric	Pre-ADR-0049	Post-ADR-0049	Δ
intent_accuracy	100.0 %	100.0 %	0
surface_groundedness	46.2 %	61.5 %	+15.3 pp
term_capture_rate	33.3 %	50.0 %	+16.7 pp
versor_closure_rate	100.0 %	100.0 %	0
versor_condition < 1e-6	preserved	preserved	invariant

The two cases that lift through the pack-grounded path (definition_procedure_023 and definition_relation_026) carry the article-stripping flow:

"What is a procedure?"  -> intent.subject == "procedure"
"What is a relation?"   -> intent.subject == "relation"

Both then match the cognition pack lexicon and emit a pack-grounded surface via ADR-0048.

The CAUSE / VERIFICATION head-noun extraction ("Why does light exist?" → "light", "Does memory require recall?" → "memory") does not directly move the eval on this split because CAUSE and VERIFICATION intents are scope-excluded from ADR-0048's pack path. That work is foundational for the next ADRs: a future COMPARISON / CAUSE / VERIFICATION pack path or teaching-store inference will inherit clean lemmas without re-implementing the extraction.

---

Consequences

What changes

• generate/intent.py gains the _normalize_subject post-processor and two closed-set frozen sets (_ARTICLES, _AUX_VERBS).
• DialogueIntent.subject is now a clean lemma (or noun phrase) for every intent that routes through _RULES.
• ADR-0048 pack-grounded surface coverage broadens from 4 → 6 of 13 cognition-eval cases.

What does not change

• IntentTag enum is unchanged.
• The rule table (_RULES) is unchanged — the post-processor runs after a rule fires.
• COMPARISON, TRANSITIVE_QUERY, FRAME_TRANSFER, and BELONG_QUERY paths use their own named-group regexes and were already producing clean subjects; they are not routed through _normalize_subject.
• UnknownDomainGate semantics are unchanged.
• versor_condition(F) < 1e-6 invariant — no algebra is touched.

Scope limits

• English only. The aux-verb / article lists are English; a future multilingual cognition pack ADR would extend the sets or move them into the language pack itself.
• The PROCEDURE intent's "How can I VERB ARTICLE NOUN" shape ("How can I correct an error?") is not handled: stripping the verb requires either part-of-speech tagging or a closed list of imperative verbs. Out of scope here. The case procedure_correct_035 has empty expected_surface_contains in the eval anyway, so it does not affect surface_groundedness.
• Multi-word noun phrases for DEFINITION / RECALL (e.g. "artificial intelligence") are preserved as-is. Pack lookup matches on the cleaned phrase; if the pack carries the multi-word lemma, it lifts; if not, it falls through to the universal disclosure. This is the doctrinally correct behaviour.

---

Cross-References

• ADR-0018 — defines DialogueIntent and the _RULES table this ADR post-processes.
• ADR-0048 — the consumer whose pack-lookup gap this ADR closes by producing clean lemmas.
• ADR-0046 / ADR-0047 — the forward-graph-constraint pipeline that consumes intent.subject via graph_planner.graph_from_intent; cleaner subjects make graph nodes single-lemma rather than noun-phrase, increasing the chance the AdmissibilityRegion's CGA neighbourhood intersects the walk's candidate pool.

---

Verification

tests/test_intent_subject_extraction.py           — 30 tests, all green
tests/test_intent_proposition_graph.py            — pre-existing tests still green
tests/test_pack_grounding.py                      — pre-existing tests still green
tests/test_semantic_realizer_integration.py       — pre-existing tests still green

Lanes (all green on this branch):
  core test --suite smoke         67 passed
  core test --suite cognition    121 passed
  core test --suite runtime       19 passed

core eval cognition (pre → post):
  intent_accuracy        100.0% → 100.0%   (=)
  surface_groundedness    46.2% →  61.5%   (+15.3 pp)
  term_capture_rate       33.3% →  50.0%   (+16.7 pp)
  versor_closure_rate    100.0% → 100.0%   (=)

The non-negotiable field invariant (versor_condition(F) < 1e-6) is preserved: this ADR touches a typed dataclass field, no algebra.