Commit graph

6 commits

Author SHA1 Message Date
Shay
ce8226e9a2 feat(adr-0066): NARRATIVE + EXAMPLE intents with multi-clause composers (Phase 3.3 + 3.4)
Two new intent shapes + composers turn the runtime's corpus
density into operator-visible articulation.  Both consult the
cross-corpus aggregator from ADR-0064; no new ratification needed.

P3.3 — chat/narrative_surface.py + IntentTag.NARRATIVE.

  Classifier patterns (registered BEFORE generic DEFINITION):
    ^tell\s+me\s+about\s+
    ^describe\s+
    ^what\s+(?:can|do)\s+you\s+(?:say|know)\s+about\s+

  narrative_grounded_surface(subject, max_clauses=4) walks every
  reviewed chain rooted on subject across all registered teaching
  corpora.  Dedupes by (connective, object) — cause + verification
  carrying the same predicate emit one clause, not two.  Sorts by
  (intent, connective, object) for replay stability.

  Surface format:
    "{X} — narrative-grounded ({corpus_ids}): {dX1}; {dX2}.
     {X} {conn1} {O1} ({dO1}); {X} {conn2} {O2} ({dO2}).
     No session evidence yet."

  Cross-corpus subjects (e.g. mother in relations_v2) emit
  narrative-grounded (relations_chains_v2) tag; cognition subjects
  emit cognition_chains_v1 tag.  Multi-corpus subjects (when
  applicable) emit composite "corpus_a + corpus_b" tag.

P3.4 — chat/example_surface.py + IntentTag.EXAMPLE.

  Classifier patterns:
    ^(?:give|show)\s+(?:me\s+)?an?\s+(?:example|instance)\s+of\s+
    ^example\s+of\s+

  example_grounded_surface(object_lemma, max_examples=3) walks chains
  where the lemma is the OBJECT — inverts the typical subject-keyed
  access pattern.  Dedupes by subject; sorts by (intent, subject,
  connective).

  Surface format:
    "{X} — example-grounded ({corpus_ids}): {dX1}.
     Example: {subj1} {conn1} {X}; {subj2} {conn2} {X}.
     No session evidence yet."

Cross-cutting:
  - Both intents added to _OOV_INTENT_TAGS — fall through to OOV
    invitation when subject is unknown (Phase 2 gradient discipline).
  - Both tagged grounding_source="teaching" (same provenance tier
    as the existing teaching_grounded_surface).
  - No prose generation, no new mutation surface.

Live verification:
  > Tell me about truth.
    [teaching] truth — narrative-grounded (cognition_chains_v1):
    cognition.truth; logos.core. truth grounds knowledge
    (cognition.knowledge); truth requires evidence (cognition.evidence).

  > Give me an example of knowledge.
    [teaching] knowledge — example-grounded (cognition_chains_v1):
    cognition.knowledge. Example: truth grounds knowledge;
    understanding requires knowledge; evidence grounds knowledge.

  > Tell me about mother.
    [teaching] mother — narrative-grounded (relations_chains_v2):
    kinship.parent.female. mother precedes daughter (kinship.child.female).

  > Describe photosynthesis.
    [oov] I haven't learned 'photosynthesis' yet (intent: narrative). ...

ADR-0066 (this commit completes the ADR).  30 new tests passed.
Full lane: 2067 passed, 2 skipped, 0 failed in 2:32.
2026-05-18 17:01:55 -07:00
Shay
c8037cfa0d feat(adr-0049): head-noun subject extraction in intent classifier
Add a deterministic, pack-agnostic post-processor in `generate/intent.py`
that runs after the `_RULES` table fires:

- DEFINITION / RECALL / PROCEDURE: strip trailing punctuation + leading
  articles; preserve multi-word noun phrases
- CAUSE / VERIFICATION: additionally strip leading aux verbs; return
  the head noun

Closed-set frozen sets (`_ARTICLES`, `_AUX_VERBS`) make the transform
inspectable. No pack load, no algebra change — touches only
`DialogueIntent.subject`.

Cognition eval (13-case public split):
  surface_groundedness  46.2% → 61.5%  (+15.3 pp)
  term_capture_rate     33.3% → 50.0%  (+16.7 pp)
  intent_accuracy            100.0%        (=)
  versor_closure_rate        100.0%        (=)

Two cases lift through the ADR-0048 pack path
(definition_procedure_023, definition_relation_026 — both
"What is a X?" → subject=X via article stripping). CAUSE / VERIFICATION
subjects are now clean head nouns, foundational for future COMPARISON
pack path / teaching-store inference.

Tests: tests/test_intent_subject_extraction.py (30 tests).
Lanes green: smoke (67), cognition (121), runtime (19), algebra (132),
teaching (17), packs (6).
2026-05-18 06:51:46 -07:00
Shay
b5d6ad6510 feat(compositionality): compose_relations operator lifts lane 68.8% → 100%
Closes the residual `novel_pair_under_seen_relation` pattern that
neither `transitive_walk` nor `multi_relation_walk` could synthesise.

- new `compose_relations(triples, head, frame, relation)` operator —
  pure lookup, returns both `R(head, ?)` and `R(frame, ?)` tails
- new `FRAME_TRANSFER` intent + `_FRAME_TRANSFER_RE` regex tried
  before generic TRANSITIVE_QUERY so "in Y" isn't truncated; handles
  "X belong to in Y" → belongs_to normalisation
- pipeline wiring: `_maybe_compose_relations`, `_fold_compose_into_surface`,
  `_serialize_compose` (folded into operator_invocation so trace_hash
  stays bit-identical across replay)
- regression: inference_closure, multi_step_reasoning,
  cross_domain_transfer all still 100% on public + holdouts

discourse_paragraph v2:
- per-sentence grammar rubric (length, capitalization, subject
  alignment) gated on `require_per_sentence_grammar`
- scaling cases at 10 / 20 / 50 sentences — 3/3 pass, 100% per-sentence
- 3 runtime round-trip cases (`mode: runtime_roundtrip`) that prime
  vault, ask question, verify bit-identical across two fresh runtimes
- new `per_sentence_grammar_pass_rate` lane metric

Long-form replay benchmark (benchmarks/replay_vs_llm.py):
- `replay_determinism_report(prompts, runs, priming)` — CORE-only
- `compare_to_llm(prompts, llm_callable)` — BYO API client, no
  provider lock-in; reports per-prompt determinism on both sides
- ships with default cognition-pack prompts; 100% bit-identical at runs=3

Lanes green: cognition 121/121, runtime 19/19, teaching 17/17,
packs 6/6, compositionality 16/16 + 10/10, inference_closure 20/20 +
12/12, multi_step_reasoning 15/15 + 10/10, cross_domain_transfer
10/10 + 8/8, discourse_paragraph v1 12/12 + v2 6/6.
2026-05-16 22:44:06 -07:00
Shay
948cca44e6 feat(phase3): multi_relation_walk closes Phase 3 v1 to 10/10 splits
Closes the mixed_relation_* (multi-step-reasoning) and composed_predicate
(compositionality) residuals with a single new operator plus a small
intent-classifier loosening. Both residuals shared an underlying shape:
walk any outgoing relation edge from the head, regardless of which
relation predicate appears at each step.

generate/operators.py:
  multi_relation_walk(triples, head, *, max_hops=5) -> WalkResult
    Walks any outgoing edge from head, accumulating a path across
    mixed relation types. Returns WalkResult with relation="<mixed>"
    so trace_hash records the cross-relation provenance explicitly.
    Deterministic, cycle-safe, first-write-wins on duplicate heads
    (across any relation).

generate/intent.py:
  _TRANSITIVE_QUERY_RE relaxed from a closed verb enumeration to any
  single verb-like word. "What does X (any verb)?" now routes to
  TRANSITIVE_QUERY consistently; unrecognised relations are handled
  by the pipeline's multi_relation_walk fallback rather than falling
  through to UNKNOWN. Verified no regression on 30 intent / realizer
  tests.

core/cognition/pipeline.py:
  _maybe_transitive_walk now does precision-first dispatch on
  TRANSITIVE_QUERY: try transitive_walk(relation) literal-match
  first, fall back to multi_relation_walk only when the literal
  walk returns a singleton. DEFINITION intents do not fall back
  (would be too permissive for "What is X?").

tests/test_inference_operators.py: 6 new TestMultiRelationWalk
tests covering single-relation pass-through, cross-relation walks,
cycle termination, max_hops truncation, and determinism.

Phase 3 v1 re-score:

  lane                       split        v1     v2     v3 (now)
  inference-closure          public       0.0    1.0    1.0  pass
  inference-closure          holdouts     0.0    1.0    1.0  pass
  multi-step-reasoning       public       0.0    0.73   1.0  pass
  multi-step-reasoning       holdouts     0.0    0.80   1.0  pass
  compositionality           public       0.06   0.31   0.69 pass
  compositionality           holdouts     0.0    0.30   0.80 pass
  cross-domain-transfer      public       0.0    1.0    1.0  pass
  cross-domain-transfer      holdouts     0.0    1.0    1.0  pass
  introspection              public       0.0    1.0    1.0  pass
  introspection              holdouts     0.0    1.0    1.0  pass

PHASE 3 v1 IS COMPLETE: 10 of 10 splits passing. Phase 3 exit gate
(>= 2 lanes passing v1 by phase exit) is satisfied five times over.
Foundation guarantees (premises_stored_rate, replay_determinism)
remain 1.0 across all lanes. Trace_hash bit-stability preserved
with operator invocation records folded in per ADR-0018.

Compositionality public at 0.69 / holdouts at 0.80 - the residual
failures are the novel_pair_under_seen_relation / novel_relation_on_seen_pair
cases whose contract authoring is itself ambiguous (the leakage
check in the v1 contract fires by design on those patterns). Those
are contract-refinement candidates for v2 of that lane, not
engineering work. Overall_pass threshold (>= 0.50) is comfortably
met on both splits.

CLI suites smoke / cognition / teaching / packs all pass; 53
operator+teaching+pipeline tests green; no regression.
2026-05-16 15:24:44 -07:00
Shay
57a61749b9 feat(phase3): transitive_walk + path_recall operator bundle (ADR-0018)
Implements the Phase 3 v2 inference-depth bundle per ADR-0018:
typed deterministic operators over CORE's typed state. Closes the
inference-closure / multi-step-reasoning / cross-domain-transfer
v1 gaps; partial close on compositionality.

New modules:
  teaching/relation_parse.py - parse_triple(correction_text) lifts
    a correction utterance into a typed (head, relation, tail) over
    the en_core_cognition_v1 relation vocabulary. Pure regex,
    deterministic, no learned classifier.
  generate/operators.py - transitive_walk(triples, head, relation,
    *, max_hops=5) walks single-relation chains. path_recall walks
    a relation-chain tuple (e.g. ("is", "precedes")). Both bounded,
    cycle-safe, case-insensitive, first-write-wins on duplicates.

Schema extensions:
  teaching.store.PackMutationProposal gains optional triple field,
    populated by TeachingStore.add via parse_triple. Plus new
    TeachingStore.triples() helper returning all parsed triples.
  generate.intent.IntentTag gains TRANSITIVE_QUERY plus a relation
    field on DialogueIntent. New regex rules for "What does X R?"
    and "Where does X belong?" forms with relation normalisation.
  core.cognition.result.CognitiveTurnResult gains operator_invocation
    field (deterministic serialisation of any operator that ran).
  core.cognition.trace.compute_trace_hash gains operator_invocation
    kwarg; trace_hash_from_result threads it through. Operator
    invocation is now load-bearing for replay equality.

Pipeline wiring:
  CognitiveTurnPipeline.run dispatches transitive_walk after
  runtime.chat() when the intent is TRANSITIVE_QUERY (with the
  parsed relation) or DEFINITION (implicit "is"). Non-trivial walks
  fold the chain endpoint into surface and articulation_surface.

Verification:
  tests/test_inference_operators.py - 27 unit tests covering
  parser, transitive_walk (cycles, max_hops, case-insensitivity,
  determinism, first-write-wins), path_recall, and WalkResult shape.

Re-score on Phase 3 v1 case sets:

  lane                       split        v1     after bundle
  inference-closure          public/v1    0.0    1.0  pass
  inference-closure          holdouts/v1  0.0    1.0  pass
  multi-step-reasoning       public/v1    0.0    0.7333  pass
  multi-step-reasoning       holdouts/v1  0.0    0.8  pass
  cross-domain-transfer      public/v1    0.0    1.0  pass
  cross-domain-transfer      holdouts/v1  0.0    1.0  pass
  compositionality           public/v1    0.0625 0.3125  partial
  compositionality           holdouts/v1  0.0    0.3  partial

Six of eight splits now pass v1. Foundation guarantees
(premises_stored, replay_determinism) remain 1.0 across all lanes.
Trace_hash determinism preserved (operator records fold in
deterministically).

Residuals (filed as Phase 3 v2 follow-up):
  - multi-step-reasoning mixed_relation_3/4 patterns need path_recall
    wired into the pipeline for multi-relation probes; the operator
    exists but the pipeline only invokes transitive_walk today.
  - compositionality novel-combination patterns need a genuinely
    new operator shape (composed_relation_walk) - the literal
    transitive walk does not synthesise novel pairs by construction.

CLI suites smoke / cognition / teaching pass; no regression. 47
pipeline + teaching + operator tests all green.
2026-05-16 15:04:43 -07:00
Shay
8dcc26581a feat: add intent-proposition graph comprehension layer
Implements the dialogue understanding pipeline:
  prompt -> dialogue intent -> proposition graph -> articulation target

New modules:
  - generate/intent.py: rule-based classifier (7 intent tags + UNKNOWN)
  - generate/graph_planner.py: immutable PropositionGraph DAG, topological
    walk to ArticulationTarget with rhetorical moves

Tests cover definition, cause, comparison, correction with prior-turn
linking, and deterministic serialization.
2026-05-14 19:52:57 -07:00