core/generate/realizer.py
Shay 257a27c105 feat(benchmarks): discourse_paragraph lane + pipeline profiler + word-selection tracer
Closes the user-flagged scope gap: every previous fluency lane (Phase
5.1 + 5.4-5.7 + grammatical_coverage) operates on 3-word SVO probes.
These three pieces stress paragraph-scale generation, give per-stage
latency visibility, and expose the realizer's word-choice geometry —
all on top of the existing deterministic infrastructure.

# discourse_paragraph lane (paragraph-scale fluency)

Forces the realizer to emit multi-sentence paragraphs from a
multi-step ArticulationTarget with rhetorical moves (ASSERT, SEQUENCE,
ELABORATE, CONTRAST).  Same realizer, much richer input — every case
is 3-5 sentences with deterministic discourse markers.

Public 12 cases / holdouts 5 / dev 1 across 12 + 5 topic chains
(epistemic, scientific method, creation arc, logical dependency,
ethical grounding, linguistic layers, mathematical chain, narrative,
biology, physics, two contrast-shaped, musical, social, computational,
psychological, economic).

Sub-metrics per case:
  - sentence count (within min..max window)
  - subject coverage rate
  - discourse marker presence (next / furthermore / in contrast)
  - sentence-initial capitalization
  - replay determinism (run twice, surfaces match)

Result: 12/12 public + 5/5 holdouts at 100%, replay rate 100%, mean
sentence count 4.

# Realizer capitalization (G4, addresses user-flagged concern)

generate/realizer.py gains `_capitalize_sentence` + `_join_as_paragraph`
helpers.  Sentence-initial alphabetic characters are now uppercased
(skipping leading whitespace/punctuation).  Surfaces went from
"wisdom grounds knowledge. next, knowledge requires evidence."
to
"Wisdom grounds knowledge. Next, knowledge requires evidence."

The discourse_paragraph runner ships a strict per-sentence
capitalization check so future regressions get caught.

# Pipeline-stage profiler (benchmarks/pipeline_profiler.py)

External monkey-patch wrapper around CognitiveTurnPipeline.run() that
records per-stage ns budgets without editing any pipeline source.
Stages: intent, graph_planner, realize_semantic, runtime_chat,
maybe_transitive_walk, fold_walk_into_surface, run_teaching,
trace_hash.

API: `profile_turn(pipeline, text) -> ProfileReport` with
`.stages: dict`, `.total_ns: int`, `.as_dict()`.

Empirical: runtime_chat dominates >99% on the runtime hot path (which
is correct — that's where ingest + propagate + recall + articulate
all happen).  Future optimisation work has a clear per-stage signal.

# Word-selection tracer (benchmarks/word_selection_tracer.py)

External wrapper around generate.articulation._resolve_slot that
records every nearest-neighbor lookup as a WordSelectionStep:
  - slot (subject/predicate/object)
  - input versor (32-d copy)
  - top-K candidate words by CGA inner product
  - chosen word + morphology
  - output language

Top-K scoring uses the diagonal Cl(4,1) metric kernel from
algebra.backend (same vectorised path vault_recall uses), not a
per-word Python loop over cga_inner.  No approximation, exact
deterministic ranking, bit-identical to a scalar scan.

API: `trace_realization(pipeline, text) -> RealizationTrace` with
`.steps`, `.realization_steps`, `.surface`, `.as_dict()`.

# CLI lane registration

Cognition suite now sweeps the benchmark profiler/tracer tests
(test_benchmarks_profiler.py) so any future regression in the
instrumentation surfaces immediately.

# Constraints honoured

- Zero edits to core/, chat/, vault/, teaching/, language_packs/, or
  the algebra hot path.  All instrumentation is external monkey-patch
  with originals restored in finally.
- discourse_paragraph runner bypasses ChatRuntime grounding (named v2
  gap) so paragraph capability is isolated to the realizer.
- No semantic changes; no hidden normalisation; no approximate
  recall.

# Lane health

smoke 55, runtime 19, teaching 17, packs 6, cognition 105 (was 103),
algebra 132.  All Phase 5 fluency lanes still 100% with the
capitalised surfaces (rubric is case-insensitive).  discourse_paragraph
100%.

# What ships next (named v2)

- Round-trip: discourse_paragraph through ChatRuntime end-to-end,
  not just realize_target.
- Per-sentence grammatical_coverage rubric on each emitted sentence.
- Longer chains (10/20/50 sentences) with per-sentence determinism
  scaling curves.
- compose_relations operator to lift compositionality recall from
  68.8% toward 100%.
2026-05-16 21:53:46 -07:00

249 lines
8.3 KiB
Python

"""ArticulationRealizerV2 — deterministic template-based realization.
Converts an ArticulationTarget (ordered rhetorical steps from the graph
planner) into a RealizedPlan: an ordered sequence of surface fragments
joined into a single deterministic surface string.
Design constraints:
- No LLM fallback
- No broad grammar engine
- Deterministic: same ArticulationTarget → same RealizedPlan, always
- Composable: does not replace the existing realize() path yet
"""
from __future__ import annotations
from dataclasses import dataclass
from generate.graph_planner import (
ArticulationStep,
ArticulationTarget,
PropositionGraph,
RhetoricalMove,
)
from generate.intent import IntentTag
from generate.semantic_templates import render_semantic
from generate.templates import render_step
@dataclass(frozen=True, slots=True)
class RealizedFragment:
node_id: str
move: RhetoricalMove
surface: str
def as_dict(self) -> dict[str, str]:
return {
"node_id": self.node_id,
"move": self.move.value,
"surface": self.surface,
}
def _capitalize_sentence(s: str) -> str:
"""Capitalize the first alphabetic character of a sentence.
Skips leading whitespace/punctuation so fragments that start with
discourse markers ("next, knowledge…") still emit a capital first
letter ("Next, knowledge…") at the sentence boundary. Leaves the
rest of the string untouched — proper nouns and embedded all-caps
tokens are preserved.
"""
if not s:
return s
for i, ch in enumerate(s):
if ch.isalpha():
return s[:i] + ch.upper() + s[i + 1:]
return s
def _join_as_paragraph(fragments: list["RealizedFragment"]) -> str:
"""Join fragments into a paragraph with sentence-initial capitalization.
Each fragment becomes one sentence; sentence-initial letters are
capitalized; the paragraph ends with a single terminal period.
"""
if not fragments:
return ""
pieces: list[str] = []
for f in fragments:
s = f.surface.strip()
if not s:
continue
s = _capitalize_sentence(s)
pieces.append(s)
joined = ". ".join(pieces)
if joined and not joined.endswith("."):
joined += "."
return joined
@dataclass(frozen=True, slots=True)
class RealizedPlan:
fragments: tuple[RealizedFragment, ...]
surface: str
def as_dict(self) -> dict[str, object]:
return {
"fragments": tuple(f.as_dict() for f in self.fragments),
"surface": self.surface,
}
def realize_semantic(
target: ArticulationTarget,
graph: PropositionGraph | None = None,
) -> RealizedPlan:
"""Realize using intent-aware semantic templates.
Uses the source intent to select a template that produces structurally
better surfaces (e.g. "X is defined as Y" for definition intents)
rather than the generic rhetorical-move templates.
Returns an empty RealizedPlan for empty/None targets so the caller
can fall back to the older articulation path.
"""
if target is None or not target.steps:
return RealizedPlan(fragments=(), surface="")
intent = target.source_intent
fragments: list[RealizedFragment] = []
if intent is IntentTag.COMPARISON and len(target.steps) >= 2:
step_a = target.steps[0]
step_b = target.steps[1]
obj_a = _resolve_obj(step_a, graph)
secondary = step_b.subject if step_b.subject != step_a.subject else obj_a
surface = render_semantic(
intent=intent,
subject=step_a.subject,
predicate=step_a.predicate,
obj=obj_a,
secondary=secondary,
)
fragments.append(RealizedFragment(
node_id=step_a.node_id,
move=RhetoricalMove.CONTRAST,
surface=surface,
))
else:
for step in target.steps:
obj = _resolve_obj(step, graph)
surface = render_semantic(
intent=intent,
subject=step.subject,
predicate=step.predicate,
obj=obj,
)
move = step.move
if move is RhetoricalMove.ASSERT and intent is IntentTag.CORRECTION:
move = RhetoricalMove.CORRECT
fragments.append(RealizedFragment(
node_id=step.node_id,
move=move,
surface=surface,
))
joined = _join_as_paragraph(fragments)
return RealizedPlan(fragments=tuple(fragments), surface=joined)
def _resolve_obj(step: ArticulationStep, graph: PropositionGraph | None) -> str:
"""Look up the object slot from the graph node matching this step."""
if graph is None:
return "..."
for node in graph.nodes:
if node.node_id == step.node_id:
return node.obj
return "..."
def realize_target(
target: ArticulationTarget,
graph: PropositionGraph | None = None,
) -> RealizedPlan:
"""Realize an ArticulationTarget into a deterministic surface plan.
Handles compound constructions (conjunction, disjunction, complement,
relative clause) by detecting graph edges and joining surfaces with
appropriate connectors rather than sentence-level punctuation.
Returns an empty-but-valid RealizedPlan for empty/None targets.
"""
from generate.graph_planner import Relation
if target is None or not target.steps:
return RealizedPlan(fragments=(), surface="")
edge_map: dict[str, tuple[str, Relation]] = {}
if graph is not None:
for edge in graph.edges:
edge_map[edge.source] = (edge.target, edge.relation)
step_by_id = {step.node_id: step for step in target.steps}
visited: set[str] = set()
fragments: list[RealizedFragment] = []
for step in target.steps:
if step.node_id in visited:
continue
visited.add(step.node_id)
obj = _resolve_obj(step, graph)
move = step.move
if move is RhetoricalMove.ASSERT and target.source_intent is IntentTag.CORRECTION:
move = RhetoricalMove.CORRECT
surface = render_step(
move=move,
subject=step.subject,
predicate=step.predicate,
obj=obj,
negated=step.negated,
quantifier=step.quantifier,
tense=step.tense,
aspect=step.aspect,
)
if step.node_id in edge_map:
target_id, relation = edge_map[step.node_id]
target_step = step_by_id.get(target_id)
if target_step is not None and target_id not in visited:
match relation:
case Relation.CONJUNCTION | Relation.DISJUNCTION | Relation.COMPLEMENT | Relation.RELATIVE:
visited.add(target_id)
target_obj = _resolve_obj(target_step, graph)
target_surface = render_step(
move=RhetoricalMove.ASSERT,
subject=target_step.subject,
predicate=target_step.predicate,
obj=target_obj,
negated=target_step.negated,
quantifier=target_step.quantifier,
tense=target_step.tense,
aspect=target_step.aspect,
)
match relation:
case Relation.CONJUNCTION:
surface = f"{surface} and {target_surface}"
case Relation.DISJUNCTION:
surface = f"{surface} or {target_surface}"
case Relation.COMPLEMENT:
surface = f"{step.subject} {step.predicate} that {target_surface}"
case Relation.RELATIVE:
surface = f"{step.subject}, which {target_step.predicate} {target_obj}, {step.predicate} {obj}"
case _:
pass
fragments.append(
RealizedFragment(
node_id=step.node_id,
move=move,
surface=surface,
)
)
joined = _join_as_paragraph(fragments)
return RealizedPlan(fragments=tuple(fragments), surface=joined)