257a27c105
Closes the user-flagged scope gap: every previous fluency lane (Phase 5.1 + 5.4-5.7 + grammatical_coverage) operates on 3-word SVO probes. These three pieces stress paragraph-scale generation, give per-stage latency visibility, and expose the realizer's word-choice geometry — all on top of the existing deterministic infrastructure. # discourse_paragraph lane (paragraph-scale fluency) Forces the realizer to emit multi-sentence paragraphs from a multi-step ArticulationTarget with rhetorical moves (ASSERT, SEQUENCE, ELABORATE, CONTRAST). Same realizer, much richer input — every case is 3-5 sentences with deterministic discourse markers. Public 12 cases / holdouts 5 / dev 1 across 12 + 5 topic chains (epistemic, scientific method, creation arc, logical dependency, ethical grounding, linguistic layers, mathematical chain, narrative, biology, physics, two contrast-shaped, musical, social, computational, psychological, economic). Sub-metrics per case: - sentence count (within min..max window) - subject coverage rate - discourse marker presence (next / furthermore / in contrast) - sentence-initial capitalization - replay determinism (run twice, surfaces match) Result: 12/12 public + 5/5 holdouts at 100%, replay rate 100%, mean sentence count 4. # Realizer capitalization (G4, addresses user-flagged concern) generate/realizer.py gains `_capitalize_sentence` + `_join_as_paragraph` helpers. Sentence-initial alphabetic characters are now uppercased (skipping leading whitespace/punctuation). Surfaces went from "wisdom grounds knowledge. next, knowledge requires evidence." to "Wisdom grounds knowledge. Next, knowledge requires evidence." The discourse_paragraph runner ships a strict per-sentence capitalization check so future regressions get caught. # Pipeline-stage profiler (benchmarks/pipeline_profiler.py) External monkey-patch wrapper around CognitiveTurnPipeline.run() that records per-stage ns budgets without editing any pipeline source. Stages: intent, graph_planner, realize_semantic, runtime_chat, maybe_transitive_walk, fold_walk_into_surface, run_teaching, trace_hash. API: `profile_turn(pipeline, text) -> ProfileReport` with `.stages: dict`, `.total_ns: int`, `.as_dict()`. Empirical: runtime_chat dominates >99% on the runtime hot path (which is correct — that's where ingest + propagate + recall + articulate all happen). Future optimisation work has a clear per-stage signal. # Word-selection tracer (benchmarks/word_selection_tracer.py) External wrapper around generate.articulation._resolve_slot that records every nearest-neighbor lookup as a WordSelectionStep: - slot (subject/predicate/object) - input versor (32-d copy) - top-K candidate words by CGA inner product - chosen word + morphology - output language Top-K scoring uses the diagonal Cl(4,1) metric kernel from algebra.backend (same vectorised path vault_recall uses), not a per-word Python loop over cga_inner. No approximation, exact deterministic ranking, bit-identical to a scalar scan. API: `trace_realization(pipeline, text) -> RealizationTrace` with `.steps`, `.realization_steps`, `.surface`, `.as_dict()`. # CLI lane registration Cognition suite now sweeps the benchmark profiler/tracer tests (test_benchmarks_profiler.py) so any future regression in the instrumentation surfaces immediately. # Constraints honoured - Zero edits to core/, chat/, vault/, teaching/, language_packs/, or the algebra hot path. All instrumentation is external monkey-patch with originals restored in finally. - discourse_paragraph runner bypasses ChatRuntime grounding (named v2 gap) so paragraph capability is isolated to the realizer. - No semantic changes; no hidden normalisation; no approximate recall. # Lane health smoke 55, runtime 19, teaching 17, packs 6, cognition 105 (was 103), algebra 132. All Phase 5 fluency lanes still 100% with the capitalised surfaces (rubric is case-insensitive). discourse_paragraph 100%. # What ships next (named v2) - Round-trip: discourse_paragraph through ChatRuntime end-to-end, not just realize_target. - Per-sentence grammatical_coverage rubric on each emitted sentence. - Longer chains (10/20/50 sentences) with per-sentence determinism scaling curves. - compose_relations operator to lift compositionality recall from 68.8% toward 100%.
1 line
742 B
JSON
1 line
742 B
JSON
{"id": "DP-DEV_001", "topic": "epistemic_chain", "graph": {"nodes": [{"node_id": "n1", "subject": "wisdom", "predicate": "grounds", "obj": "knowledge"}, {"node_id": "n2", "subject": "knowledge", "predicate": "requires", "obj": "evidence"}, {"node_id": "n3", "subject": "evidence", "predicate": "supports", "obj": "truth"}, {"node_id": "n4", "subject": "truth", "predicate": "reveals", "obj": "reality"}], "edges": []}, "steps": [{"node_id": "n1", "move": "ASSERT"}, {"node_id": "n2", "move": "SEQUENCE"}, {"node_id": "n3", "move": "ELABORATE"}, {"node_id": "n4", "move": "SEQUENCE"}], "min_sentences": 4, "must_contain_subjects": ["wisdom", "knowledge", "evidence", "truth"], "discourse_markers": ["furthermore", "next"], "max_sentences": 6}
|