257a27c105
Closes the user-flagged scope gap: every previous fluency lane (Phase 5.1 + 5.4-5.7 + grammatical_coverage) operates on 3-word SVO probes. These three pieces stress paragraph-scale generation, give per-stage latency visibility, and expose the realizer's word-choice geometry — all on top of the existing deterministic infrastructure. # discourse_paragraph lane (paragraph-scale fluency) Forces the realizer to emit multi-sentence paragraphs from a multi-step ArticulationTarget with rhetorical moves (ASSERT, SEQUENCE, ELABORATE, CONTRAST). Same realizer, much richer input — every case is 3-5 sentences with deterministic discourse markers. Public 12 cases / holdouts 5 / dev 1 across 12 + 5 topic chains (epistemic, scientific method, creation arc, logical dependency, ethical grounding, linguistic layers, mathematical chain, narrative, biology, physics, two contrast-shaped, musical, social, computational, psychological, economic). Sub-metrics per case: - sentence count (within min..max window) - subject coverage rate - discourse marker presence (next / furthermore / in contrast) - sentence-initial capitalization - replay determinism (run twice, surfaces match) Result: 12/12 public + 5/5 holdouts at 100%, replay rate 100%, mean sentence count 4. # Realizer capitalization (G4, addresses user-flagged concern) generate/realizer.py gains `_capitalize_sentence` + `_join_as_paragraph` helpers. Sentence-initial alphabetic characters are now uppercased (skipping leading whitespace/punctuation). Surfaces went from "wisdom grounds knowledge. next, knowledge requires evidence." to "Wisdom grounds knowledge. Next, knowledge requires evidence." The discourse_paragraph runner ships a strict per-sentence capitalization check so future regressions get caught. # Pipeline-stage profiler (benchmarks/pipeline_profiler.py) External monkey-patch wrapper around CognitiveTurnPipeline.run() that records per-stage ns budgets without editing any pipeline source. Stages: intent, graph_planner, realize_semantic, runtime_chat, maybe_transitive_walk, fold_walk_into_surface, run_teaching, trace_hash. API: `profile_turn(pipeline, text) -> ProfileReport` with `.stages: dict`, `.total_ns: int`, `.as_dict()`. Empirical: runtime_chat dominates >99% on the runtime hot path (which is correct — that's where ingest + propagate + recall + articulate all happen). Future optimisation work has a clear per-stage signal. # Word-selection tracer (benchmarks/word_selection_tracer.py) External wrapper around generate.articulation._resolve_slot that records every nearest-neighbor lookup as a WordSelectionStep: - slot (subject/predicate/object) - input versor (32-d copy) - top-K candidate words by CGA inner product - chosen word + morphology - output language Top-K scoring uses the diagonal Cl(4,1) metric kernel from algebra.backend (same vectorised path vault_recall uses), not a per-word Python loop over cga_inner. No approximation, exact deterministic ranking, bit-identical to a scalar scan. API: `trace_realization(pipeline, text) -> RealizationTrace` with `.steps`, `.realization_steps`, `.surface`, `.as_dict()`. # CLI lane registration Cognition suite now sweeps the benchmark profiler/tracer tests (test_benchmarks_profiler.py) so any future regression in the instrumentation surfaces immediately. # Constraints honoured - Zero edits to core/, chat/, vault/, teaching/, language_packs/, or the algebra hot path. All instrumentation is external monkey-patch with originals restored in finally. - discourse_paragraph runner bypasses ChatRuntime grounding (named v2 gap) so paragraph capability is isolated to the realizer. - No semantic changes; no hidden normalisation; no approximate recall. # Lane health smoke 55, runtime 19, teaching 17, packs 6, cognition 105 (was 103), algebra 132. All Phase 5 fluency lanes still 100% with the capitalised surfaces (rubric is case-insensitive). discourse_paragraph 100%. # What ships next (named v2) - Round-trip: discourse_paragraph through ChatRuntime end-to-end, not just realize_target. - Per-sentence grammatical_coverage rubric on each emitted sentence. - Longer chains (10/20/50 sentences) with per-sentence determinism scaling curves. - compose_relations operator to lift compositionality recall from 68.8% toward 100%.
1.6 KiB
1.6 KiB
discourse_paragraph — gaps
v1 (current)
- Realizer-isolation lane: bypasses runtime grounding so the paragraph claim is unconfounded by vault noise.
- Sentence-count window is intentionally generous
(
max_sentences = min + 2) to tolerate small wrapping variance from compound-clause folding inrealize_target(CONJUNCTION / COMPLEMENT / RELATIVE edges merge two steps into one sentence). - Subject coverage threshold is 0.75, not 1.0 — exact-coverage cases pass that bar comfortably but the slack lets a future realizer change ship without rewriting cases.
Known gaps for v2
- No round-trip through the runtime. v1 invokes the realizer
directly with a constructed
ArticulationTarget. v2 should feed the runtime real text inputs that produce the same articulation target throughgraph_from_intent+plan_articulation, end-to-end. - No anaphora / pronoun reduction. Every sentence carries its subject explicitly. Pronominalisation deferred.
- No length scaling above 5 sentences. v2 should push to 10/20/50 sentences and measure per-sentence determinism.
- No grammaticality check per sentence. v1 checks subject coverage + discourse markers; v2 should run each emitted sentence through grammatical_coverage's rubric.
Why this lane exists
First lane that exercises paragraph-scale output. Every previous fluency lane (Phase 5.1 + 5.4–5.7) operates on 3-word SVO probes. The structural capability — folding multiple articulation steps into a coherent paragraph with deterministic discourse markers — was already in the realizer; this lane makes it measurable.