core/formation/runner.py
Shay 64c5bc4619 feat(epistemic): truth-seeking schema audit — 3 leaks closed, 4 new lanes, 3 new invariants
Audit of the one-mutation-path invariant (ADR-0021 §3) found three leaks
where pack authority or session-state writes could substitute for coherence
judgment. All three landed fixes or partial closures in this push.

Leaks closed:
- Leak A: pack vocab defaulted to COHERENT — flipped to SPECULATIVE in
  language_packs/{compiler,schema}.py; docstring corrected to align with
  ADR-0021 (it was rationalizing the leak).
- Leak B: vault.recall was epistemic-blind — VaultStore.store() now stamps
  every entry with EpistemicStatus (default SPECULATIVE); recall(min_status=)
  filters to admissible-as-evidence tier. All 4 vault-write sites updated.
- Leak C (write-side): generate/proposition.py:198 stored articulated
  propositions unmarked — now stamps SPECULATIVE, breaking the
  fabrication-feedback loop in principle. Read-side audit of 5 call sites
  is the residual.

New architectural invariants (tests/test_architectural_invariants.py):
- INV-21: one-mutation-path allowlist (caught Leak C on first run)
- INV-22: pack lexicon default is SPECULATIVE (Leak A guard)
- INV-23: vault recall epistemic-aware (Leak B guard)

New eval lanes:
- teaching_injection_resistance — ships GREEN at 1.00/1.00/0 (the
  structural anti-injection claim is real and measurable)
- refusal_calibration — honest gap: 0% refusal, 0% fabrication
- contradiction_detection — honest gap: 50% flag via versor-delta heuristic,
  100% false-positive; motivates the proper coherence-checker
- articulation_of_status — honest gap: 0% speculative articulation, 60%
  false certainty; output-side leak surface

New benchmarks:
- benchmarks/footprint.py — total deployed runtime is 7.06 MiB
  (109,358x smaller than Llama 3.1 405B, runs offline, no GPU)
- benchmarks/learning_curve.py — monotonic + replay-deterministic curve
  per lane

Documentation:
- docs/truth_seeking_schema.md — foundational architectural commitment,
  five rules, mapped to human failure modes, leaks published openly
- evals/CLAIMS.md — five-tier public claims doc; Tier 4.5 publishes
  known gaps with named fixes; verification contract at top
- README.md — new pillar between algebraic substrate and language pillar

Includes in-flight formation pipeline scaffolding (formation/, tests/formation/,
docs/formation_pipeline_plan.md) and minor CLI/contracts/gitignore edits
that were already in the working tree at session start.

Verification: 798 passed, 2 skipped, 1 deselected (pre-existing pack-count
test drift unrelated to schema changes).
2026-05-17 07:27:41 -07:00

111 lines
3.6 KiB
Python

"""Stage 6 — Run. Drive a ``FormationPlan`` through the cognitive pipeline.
The Runner is **a thin shim**. It does not invent operators or pack-mutation
paths; it only invokes a caller-supplied ``pipeline_callable`` per step and
collects the per-step outcomes. This keeps the Runner testable without a
runtime dependency on ``CognitiveTurnPipeline`` (which would pull in the
entire engine).
Hard halt: any turn whose ``versor_condition`` is reported as ``>= 1e-6``
stops the run. The runner never repairs or normalizes the field — per
CLAUDE.md, repair belongs in the algebra/operator layer, not the runtime
shell.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable
from formation.course import FormationPlan, PlanStep
from formation.ratify import StepResult
# Threshold copied from CLAUDE.md "Non-Negotiable Field Invariant" so the
# Runner halt boundary mirrors the project's global invariant rather than
# embedding a divergent value.
VERSOR_HALT_THRESHOLD: float = 1.0e-6
@dataclass(frozen=True, slots=True)
class TurnObservation:
"""Minimum observation surface required by the Runner.
The caller's pipeline_callable returns one of these per step. This
indirection lets the Runner stay decoupled from
``CognitiveTurnResult`` and avoid pulling in the cognitive pipeline at
import time.
"""
trace_hash: str
versor_condition: float
accepted: bool # for adversarial probes: True = runtime accepted
# the probe (= a *failure*); for legit steps: True
# = the runtime accepted the assertion.
has_provenance: bool
class RunnerHalt(Exception):
"""Raised when a step exceeds the versor halt threshold."""
PipelineCallable = Callable[[PlanStep], TurnObservation]
@dataclass(frozen=True, slots=True)
class RunOutput:
results: tuple[StepResult, ...]
halted: bool = False
halt_step_index: int = -1
halt_reason: str = ""
def run_plan(
plan: FormationPlan,
pipeline: PipelineCallable,
) -> RunOutput:
"""Drive every step of ``plan`` through ``pipeline``; collect ``StepResult``s.
Hard-halts on the first step whose ``versor_condition >= VERSOR_HALT_THRESHOLD``.
Returns a ``RunOutput`` describing the partial run; the caller decides
whether to surface the halt as a failure.
"""
out: list[StepResult] = []
for idx, step in enumerate(plan.steps):
obs = pipeline(step)
if obs.versor_condition >= VERSOR_HALT_THRESHOLD:
return RunOutput(
results=tuple(out),
halted=True,
halt_step_index=idx,
halt_reason=(
f"versor_condition {obs.versor_condition!r} >= "
f"{VERSOR_HALT_THRESHOLD!r}"
),
)
out.append(_to_step_result(step, obs))
return RunOutput(results=tuple(out), halted=False)
def _to_step_result(step: PlanStep, obs: TurnObservation) -> StepResult:
return StepResult(
step_type=step.step_type,
payload=dict(step.payload),
trace_hash=obs.trace_hash,
versor_condition_repr=_versor_repr(obs.versor_condition),
accepted=obs.accepted,
has_provenance=obs.has_provenance,
)
def _versor_repr(value: float) -> str:
"""Render a versor condition as a stable string.
Avoid Python's float repr drift across platforms by formatting via
``f"{value:.3e}"`` (three-digit mantissa in scientific notation). This
is enough resolution to read in audit logs without leaking precision
artifacts.
"""
if value == 0.0:
return "0.0e+00"
return f"{value:.3e}"