4cba6f488c
Lands the Phase C "make cognition legible" slice plus Phase A residue, all backend-reader-first over real engine data (no theater, read-only doctrine intact, zero serving-path imports). C1-a — Cognitive pipeline record (persistence-first, per #729 worthiness edit): - workbench/pipeline_record.py: curated CognitivePipelineRecord over the real CognitiveTurnResult (input → intent → proposition_graph → articulation_target → realizer → walk_telemetry → trace_hash). Raw field multivectors are DELIBERATELY excluded; _assert_no_raw_field_payload recursively rejects raw field keys, and validate_pipeline_record fails closed on missing/duplicate stages, non-recorded status, or dangling edges — the UI can never receive a partial record that claims to be complete. - test_workbench_pipeline_record.py: non-vacuous guards — missing stage, monkeypatched new required stage, and injected raw {"F": [...]} each raise. C2-a — Contemplation as a process: /contemplation route over real persisted contemplation/runs/*.json (glob reader; honest-empty when absent). C4-a — Identity continuity (L10/L11): RunDetail.identity_continuity + Runs Identity tab, sourced from the real core.engine_identity (engine_identity / parent_engine_identity lineage relation, re-derived to verify). Demo Theater: renders backend-owned proof-promotion + entailment DAGs. Phase A residue: density preference wired end-to-end (settings → shell → tokens); cross-route consistency touch-ups. Infra: local API CORS now echoes only validated 127.0.0.1/localhost origins (hostname-checked, not arbitrary reflection) so Vite fallback ports work. Route chunk-split keeps the build warning-free. Cleanup: corrected the stale ADR-0175 practice-lane assertions (build_report is 6 correct / 0 wrong / 44 refused after the current serving lane; wrong=0 held) and the two registry-derived count tests (LeftNav + CommandPalette 12 → 13 for the new Contemplation route). Docs: runtime_contracts.md (pipeline-record contract), UI-UX-GUIDE, api-contract-v1, data-shapes-v1, wave-M-worthiness, phase-a-residue-ledger. Validation: 106 workbench/practice Python tests green (incl. wrong=0 lane + pipeline-record fail-closed guards); 459/459 frontend; pnpm build clean; git diff --check clean. No generate.derivation / reliability_gate / stream / field.propagate / vault.store imports.
110 lines
4 KiB
Python
110 lines
4 KiB
Python
"""Sealed single-turn replay over the turn journal (Wave R3).
|
|
|
|
``GET /replay/{turn_id}`` re-executes a journaled prompt in a sealed fresh
|
|
runtime and compares the resulting envelope leaf-by-leaf against the
|
|
recorded :class:`~workbench.journal.TurnJournalEntry`.
|
|
|
|
The claim demonstrated is exactly the architectural one: same prompt, same
|
|
genesis substrate -> bit-identical envelope. The original turn ran in its
|
|
own fresh ``ChatRuntime()`` that may have loaded an engine-state checkpoint
|
|
present at the time; the journal does not record whether one existed, so
|
|
``origin_state`` is reported as ``"unrecorded"`` and a divergence means
|
|
nondeterminism OR origin-state influence — never claimed to be one or the
|
|
other.
|
|
|
|
This module owns classification and comparison only. Execution is
|
|
injected by the caller (``workbench.api`` passes its own chat-turn
|
|
executor over a sealed runtime), which keeps the comparison pure and the
|
|
no-fabricated-equivalence obligation testable: if the executor raises,
|
|
no comparison object exists.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import time
|
|
from dataclasses import fields, replace
|
|
from typing import Callable
|
|
|
|
from workbench.journal import TurnJournalEntry
|
|
from workbench.schemas import (
|
|
ChatTurnResult,
|
|
TurnReplayComparison,
|
|
TurnReplayDivergence,
|
|
)
|
|
|
|
# Every TurnJournalEntry field must appear in exactly one of these sets —
|
|
# enforced by tests so a future journal field forces an explicit
|
|
# classification decision instead of silently defaulting.
|
|
CRITICAL_FIELDS = frozenset(
|
|
{
|
|
"turn_id",
|
|
"prompt",
|
|
"surface",
|
|
"articulation_surface",
|
|
"walk_surface",
|
|
"trace_hash",
|
|
"grounding_source",
|
|
"epistemic_state",
|
|
"normative_clearance",
|
|
"verdicts",
|
|
"refusal_emitted",
|
|
"hedge_injected",
|
|
"proposal_candidates",
|
|
"leeway_evidence",
|
|
"pipeline_record",
|
|
"checkpoint_emitted",
|
|
"trace_integrity",
|
|
}
|
|
)
|
|
# Wall-clock by nature, or derived over wall-clock bytes (journal_digest
|
|
# hashes the timestamp): expected to differ on every replay and never
|
|
# evidence against equivalence.
|
|
INFORMATIONAL_FIELDS = frozenset({"timestamp", "turn_cost_ms", "journal_digest"})
|
|
|
|
|
|
def replay_turn(
|
|
entry: TurnJournalEntry,
|
|
*,
|
|
execute: Callable[[str], ChatTurnResult],
|
|
) -> TurnReplayComparison:
|
|
"""Re-execute ``entry.prompt`` via ``execute`` and compare envelopes.
|
|
|
|
``execute`` must run the prompt through the same envelope-assembly path
|
|
that produced the recorded entry, over a sealed runtime. This function
|
|
never fabricates a comparison: if ``execute`` raises, the exception
|
|
propagates and no ``TurnReplayComparison`` exists.
|
|
"""
|
|
started = time.perf_counter()
|
|
result = execute(entry.prompt)
|
|
elapsed_ms = max(0, int(round((time.perf_counter() - started) * 1000)))
|
|
result = replace(result, turn_cost_ms=elapsed_ms, turn_id=entry.turn_id)
|
|
replayed = TurnJournalEntry.from_chat_turn(result, turn_id=entry.turn_id)
|
|
|
|
divergences: list[TurnReplayDivergence] = []
|
|
for spec in fields(TurnJournalEntry):
|
|
original_value = getattr(entry, spec.name)
|
|
replay_value = getattr(replayed, spec.name)
|
|
if original_value == replay_value:
|
|
continue
|
|
divergences.append(
|
|
TurnReplayDivergence(
|
|
path=spec.name,
|
|
original=original_value,
|
|
replay=replay_value,
|
|
severity=(
|
|
"critical" if spec.name in CRITICAL_FIELDS else "informational"
|
|
),
|
|
)
|
|
)
|
|
|
|
return TurnReplayComparison(
|
|
turn_id=entry.turn_id,
|
|
comparison_basis="sealed_fresh_runtime_single_turn",
|
|
origin_state="unrecorded",
|
|
original_trace_hash=entry.trace_hash,
|
|
replay_trace_hash=replayed.trace_hash,
|
|
equivalent=not any(d.severity == "critical" for d in divergences),
|
|
replay_turn_cost_ms=elapsed_ms,
|
|
divergences=divergences,
|
|
leeway_evidence=entry.leeway_evidence,
|
|
)
|