diff --git a/docs/audit/substrate-liveness-registry.md b/docs/audit/substrate-liveness-registry.md index fa756b1b..00c5a411 100644 --- a/docs/audit/substrate-liveness-registry.md +++ b/docs/audit/substrate-liveness-registry.md @@ -13,7 +13,7 @@ | L2 — Vault | ✅ Audited | **PARTIAL** | None — flagged learning as wiring debt | 2026-05-24 | | L3 — Language packs | ✅ Audited | **PARTIAL** | None — flagged readback as wiring debt | 2026-05-24 | | L4 — Recognition | ✅ Audited | **PARTIAL** | None — flagged connector/storage/integration as wiring debt | 2026-05-24 | -| L5 — Cognition pipeline | ⏳ Pending | — | — | — | +| L5 — Cognition pipeline | ✅ Audited | **PARTIAL** | `generate/render.py` deleted; `explain.py` / `provenance.py` flagged as wiring debt | 2026-05-24 | | L6 — Chat runtime + surface composition | ⏳ Pending | — | — | — | | L7 — Teaching loop | ⏳ Pending | — | — | — | | L8 — Inter-session memory + contemplation | ⏳ Pending | — | — | — | @@ -884,3 +884,128 @@ awaiting integration/storage ADRs. reactivates one across sessions. --- + +## L5 — Cognition pipeline + +**Audit date:** 2026-05-24 +**Auditor:** primary agent (Gemini) +**Verdict:** **PARTIAL** + +### Scope-hypothesis correction (per audit step 0) + +The scope's layering table cited `core/cognition/` and `generate/` concerns: intent classification, ratification, articulation target, deterministic realizer, proposition graph planner, cold-start grounding sources. Reality: the concerns are split across two packages: `core/cognition/` (orchestration, result, trace, surface resolution) and `generate/` (intent classification, intent ratification, realizer, templates, graph planner, admissibility, attention, salience, and stream walk). The scope ranges omitted key admissibility and classification ADRs. `generate/render.py` was found to be completely dormant (0 imports, 0 references, 0 tests) and was deleted during cleanup. `core/cognition/explain.py` and `core/cognition/provenance.py` were found to have 0 live production callers, and are flagged as dormant/partially live wiring debt. + +### ADRs in scope for L5 + +| ADR | Title | Status | Belongs at L5? | +|---|---|---|---| +| ADR-0022 | Forward Semantic Control | Accepted | Yes — `classify_intent` and `ratify_intent` at the pipeline step | +| ADR-0023 | Forward Semantic Control Proof | Accepted | Yes — verifies ratification execution and rates | +| ADR-0024 | Inner-Loop Per-Rotor Admissibility | Accepted | Yes — defines admissibility regions, `check_transition`, `InnerLoopExhaustion` | +| ADR-0025 | Rotor / Frame Admissibility | Accepted | Yes — rotor effect on current field against frame cone | +| ADR-0026 | Ranked Admissibility with Margin | Accepted | Yes — margin-mode checks | +| ADR-0046 | PropositionGraph as Forward Admissibility Constraint | Accepted | Yes — proposition graph planning constraints | +| ADR-0047 | Wire the Forward Graph Constraint into the Chat Hot Path | Accepted | Yes — wires forward constraint | +| ADR-0048 | Pack-Grounded Surface for Cold-Start DEFINITION / RECALL | Accepted | Yes | +| ADR-0049 | Intent Classifier Head-Noun Subject Extraction | Accepted | Yes | +| ADR-0050 | Pack-Grounded Surface for Cold-Start COMPARISON | Accepted | Yes | +| ADR-0051 | Trust-Boundary Hardening Pass | Accepted | Yes | +| ADR-0052 | Teaching-Grounded Surface for Cold-Start CAUSE / VERIFICATION | Accepted | Yes | +| ADR-0053 | Cognition Lane Closure | Accepted | Yes | +| ADR-0058 | `forward_graph_constraint`: Engaged but Inert | Accepted | Yes | +| ADR-0061 | PROCEDURE Intent Routes to Pack-Grounded Surface | Accepted | Yes | +| ADR-0069 | Realizer register parameter | Accepted | Yes | +| ADR-0075 | Realizer slot-type guard | Accepted | Yes | +| ADR-0142 | Epistemic State Taxonomy | Accepted | Boundary-only — L5/L9 boundary | +| ADR-0143 | Recognition Spike | Accepted | Boundary-only — L4 output is L5 input | +| ADR-0144 | PropositionGraph: Epistemic Carrier & Gate | Accepted | Yes — carrier integration with L4/L9 | + +### Modules in scope for L5 + +| Module | Lines | Live-import sites (outside own package, outside `tests/`) | Test-import sites | Status | +|---|---|---|---|---| +| `core/cognition/pipeline.py` | 727 | 3 (`chat/runtime.py`, `core/cli.py`, `evals/run_cognition_eval.py`) | 12 | Live | +| `core/cognition/result.py` | 139 | 3 | 11 | Live | +| `core/cognition/trace.py` | 153 | 3 | 11 | Live | +| `core/cognition/surface_resolution.py` | 108 | 1 (`pipeline.py`) | 3 | Live | +| `core/cognition/explain.py` | 124 | 0 (only re-exported in `core/cognition/__init__.py`) | 1 | Dormant (no live production callers) | +| `core/cognition/provenance.py` | 101 | 0 (used by `evals/provenance/runner.py`) | 1 | Partially live (evals/verification only) | +| `generate/intent.py` | 662 | 8 | 15 | Live | +| `generate/intent_bridge.py` | 350 | 2 | 2 | Live | +| `generate/intent_ratifier.py` | 241 | 1 | 4 | Live | +| `generate/realizer.py` | 272 | 3 | 7 | Live | +| `generate/templates.py` | 232 | 3 | 3 | Live | +| `generate/graph_planner.py` | 300 | 4 | 7 | Live | +| `generate/admissibility.py` | 668 | 2 | 11 | Live | +| `generate/graph_constraint.py` | 159 | 2 | 2 | Live | +| `generate/stream.py` | 648 | 2 | 12 | Live | +| `generate/proposition.py` | 417 | 1 | 2 | Live | +| `generate/salience.py` | 62 | 2 | 2 | Live | +| `generate/attention.py` | 43 | 1 | 1 | Live | +| `generate/bridge_trace.py` | 292 | 1 | 1 | Live | +| `generate/render.py` | — | — | — | **DELETED — 0 imports, 0 references, 0 tests** | + +### Caller-trace evidence + +The live chat/cognition trace is: +`ChatRuntime.chat()` → `CognitiveTurnPipeline.run()` → `_ratify_intent()` → `graph_from_intent()` → `plan_articulation()` → `realize_semantic()` → delegated walk → `generate()` → `resolve_surface()` → `compute_trace_hash()`. + +### Exercising suite lane + +- `core test --suite cognition` — **120 passed, 1 skipped**. +- `core test --suite smoke` — **67 passed**. +- `core eval cognition` — **13/13 passed** (100% intent_accuracy, surface_groundedness, versor_closure_rate). +- `scripts/verify_lane_shas.py` — **7/7 lanes match pinned SHAs**. + +### Cross-layer contract check + +**Pass 1 — mechanical (consumer-exists per exposed symbol):** + +| Exposed symbol | Consumer evidence | +|---|---| +| `CognitiveTurnPipeline` | `chat/runtime.py:27`, `core/cli.py:446`, `evals/run_cognition_eval.py` | +| `CognitiveTurnResult` | `chat/runtime.py:27`, `core/cognition/pipeline.py`, tests | +| `resolve_surface` | `core/cognition/pipeline.py:23` | +| `compute_trace_hash` | `core/cognition/pipeline.py:24` | +| `classify_compound_intent` | `core/cognition/pipeline.py:25` | +| `ratify_intent` | `core/cognition/pipeline.py:30` | +| `graph_from_intent` | `core/cognition/pipeline.py:34` | +| `realize_semantic` | `core/cognition/pipeline.py:41` | +| `generate` | `chat/runtime.py:88` | +| `build_graph_constraint` | `chat/runtime.py:88` | + +**Pass 2 — semantic (four invariants checked):** + +1. **Surface contract.** Verified `resolve_surface` in `core/cognition/surface_resolution.py` preserves the user-facing `surface` vs raw walk telemetry `walk_surface` split. User-facing `surface` is the semantic realizer output folded with deterministic walk/compose suffixes; raw manifold-walk telemetry goes strictly to `walk_surface`. +2. **Normalization-site discipline.** `generate/stream.py` is a FORBIDDEN normalization site except for the documented final-state closure: `_close_final_state(state)` calls `unitize_versor(state.F)` when sealing the final state at the generation boundary. No other normalization has crept into `generate/`. `generate/admissibility.py` explicitly documents that its composed blades are *not* unitized to honor the CLAUDE.md discipline. +3. **`refusal_reason` materialization.** Wiring exists: `ChatResponse` has the field, `CognitiveTurnPipeline` reads it, and it is folded into `trace_hash` when non-empty. **However:** `InnerLoopExhaustion` exceptions raised during generation are NOT caught in `ChatRuntime.chat()`, so `refusal_reason` is never populated with the machine-readable `RefusalReason` taxonomy during live runs (it propagates as an unhandled exception). +4. **RatificationOutcome PASSTHROUGH split.** Confirmed sub-values (`PASSTHROUGH_NO_FIELD`, `PASSTHROUGH_NO_VOCAB`, `PASSTHROUGH_NO_VERSOR`) collapse to `"passthrough"` before trace hashing in `core/cognition/pipeline.py`. Byte-identity of existing trace hashes preserved exactly. + +### Semantic mismatches flagged for human review + +- **`InnerLoopExhaustion` propagation gap.** Inner-loop refusal exceptions are never caught in main `ChatRuntime.chat()` execution, meaning refusal reasons for admissibility exhaustions are never materialized to `ChatResponse.refusal_reason` during live execution. They cause unhandled exceptions unless caught externally. +- **`explain.py` and `provenance.py` dormancy.** `core/cognition/explain.py` has 0 live production callers outside its test file. `core/cognition/provenance.py` is only used in tests and `evals/provenance/runner.py`. Review whether these should be fully wired to the live REPL or are reserved for offline audit tools. + +### Closure criteria scorecard + +| Criterion | Status | Evidence | +|---|---|---| +| 1. Design artifact | ✅ | ADR-0022..0026, ADR-0046..0053, ADR-0142..0144 | +| 2. Code artifact | ✅ | `core/cognition/`, `generate/` (after deleting unused `render.py`) | +| 3. Live caller | ⚠️ PARTIAL | Orchestration pipeline, ratification, realizer, stream walk live; `explain.py` dormant; inner-loop refusal exceptions unwired | +| 4. Exercised by suite lane | ✅ | `cognition` + `smoke` pass; `eval cognition` 100% | +| 5. Cross-layer consistency | ⚠️ PARTIAL | Hashing + surface contracts consistent; `refusal_reason` propagation from generator unwired | + +**Verdict:** **PARTIAL** (dormant `explain.py`, partial `provenance.py`, unwired `InnerLoopExhaustion` exception materialization). + +### Cleanup performed + +- **`generate/render.py` deleted** — completely unused module (0 production imports, 0 test imports, 0 references). Tests + SHAs confirmed intact post-deletion. First cleanup-as-found of the audit per [[feedback-cleanup-as-you-find]]. + +### Findings / notes for downstream layers + +- **L6 (Chat runtime) auditor:** Wire the `InnerLoopExhaustion` exceptions caught in the turn loop to populate the `ChatResponse.refusal_reason` field so machine-readable refusal codes are folded into the trace hash and result objects. +- **L7 (Teaching loop) / L8 (Memory) auditor:** `core/cognition/explain.py` is dormant. If teaching correction review requires a natural-language explanation of what was decoded on a turn, wire `explain()` to the REPL or CLI proposal commands. +- **L9 (Epistemic state) auditor:** ADR-0142 implementation debt #3 (wiring refusal reason) is the primary blocker for full epistemic refusal tracking. Fix this boundary in L6/L9. + +--- diff --git a/generate/render.py b/generate/render.py deleted file mode 100644 index aac0e59a..00000000 --- a/generate/render.py +++ /dev/null @@ -1,106 +0,0 @@ -""" -generate/render.py — Default TextRenderer (ADR-0011) - -The renderer is the last thing that happens before output leaves the system. -Nothing after the renderer touches the field. - -Contract (Renderer protocol): - Input: Iterable[str] — the ordered surface-string token stream from stream.py - Output: str — the realized surface text - Stateless: — holds no field state, modifies nothing - Deterministic: — identical token sequences produce identical output - -Modality-specific renderers (markdown, Hebrew RTL, Koine Greek polytonic, -audio phoneme stream) implement the same Renderer protocol and are provided -by the caller. The engine never selects a renderer. - -Design note: - stream.py yields word strings — the .surface form resolved by vocab.nearest(). - The renderer's only job is joining them correctly for the target modality. - Linguistic depth (Hebrew morphology, Greek diacritics) is already encoded - in the surface strings upstream in vocab/. The renderer transcribes; it - does not interpret. -""" - -from __future__ import annotations - -from typing import Iterable, Protocol, runtime_checkable - - -@runtime_checkable -class Renderer(Protocol): - """ - Protocol for all surface renderers. - - The engine yields tokens; the caller provides a Renderer and calls render(). - The engine has no concept of deployment context. - """ - - def render(self, tokens: Iterable[str]) -> str | bytes: - """Realize a token stream as surface output.""" - ... - - -class TextRenderer: - """ - Default renderer: joins tokens into a UTF-8 string. - - Separator logic: - - Tokens that begin with standard punctuation (.,;:!?'\') - are attached to the preceding token without a space. - - All other tokens are separated by a single space. - - Leading and trailing whitespace is stripped. - - This handles English, Koine Greek polytonic, and Hebrew surface strings - correctly provided the VocabEntry.surface fields are well-formed upstream. - Hebrew RTL directionality is a property of the Unicode codepoints themselves - and is respected automatically by any compliant display layer. - """ - - # Punctuation tokens that attach to the left (no preceding space) - _ATTACH_LEFT: frozenset[str] = frozenset(".,;:!?'\")-]}’”;·") - - def render(self, tokens: Iterable[str]) -> str: - parts: list[str] = [] - for token in tokens: - if not token: - continue - if parts and token[0] not in self._ATTACH_LEFT: - parts.append(" ") - parts.append(token) - return "".join(parts).strip() - - -class MarkdownRenderer: - """ - Renderer for markdown surface output. - - Identical join logic to TextRenderer. Extends with optional - code-fence and heading prefix support if the token stream carries - structural markers (tokens that begin with '#' or '```'). - - This is a thin wrapper — it does not parse or validate markdown. - It writes what the field produced. - """ - - _text = TextRenderer() - - def render(self, tokens: Iterable[str]) -> str: - return self._text.render(tokens) - - -# Module-level default — usable directly without instantiation -default_renderer = TextRenderer() - - -def render(tokens: Iterable[str], renderer: Renderer | None = None) -> str: - """ - Convenience function: render a token stream using the provided renderer. - Falls back to the default TextRenderer if none is given. - - Usage: - from generate.render import render - text = render(generate(state, vocab, persona)) - """ - r = renderer if renderer is not None else default_renderer - return r.render(tokens)