core/formation/cache.py
Shay 64c5bc4619 feat(epistemic): truth-seeking schema audit — 3 leaks closed, 4 new lanes, 3 new invariants
Audit of the one-mutation-path invariant (ADR-0021 §3) found three leaks
where pack authority or session-state writes could substitute for coherence
judgment. All three landed fixes or partial closures in this push.

Leaks closed:
- Leak A: pack vocab defaulted to COHERENT — flipped to SPECULATIVE in
  language_packs/{compiler,schema}.py; docstring corrected to align with
  ADR-0021 (it was rationalizing the leak).
- Leak B: vault.recall was epistemic-blind — VaultStore.store() now stamps
  every entry with EpistemicStatus (default SPECULATIVE); recall(min_status=)
  filters to admissible-as-evidence tier. All 4 vault-write sites updated.
- Leak C (write-side): generate/proposition.py:198 stored articulated
  propositions unmarked — now stamps SPECULATIVE, breaking the
  fabrication-feedback loop in principle. Read-side audit of 5 call sites
  is the residual.

New architectural invariants (tests/test_architectural_invariants.py):
- INV-21: one-mutation-path allowlist (caught Leak C on first run)
- INV-22: pack lexicon default is SPECULATIVE (Leak A guard)
- INV-23: vault recall epistemic-aware (Leak B guard)

New eval lanes:
- teaching_injection_resistance — ships GREEN at 1.00/1.00/0 (the
  structural anti-injection claim is real and measurable)
- refusal_calibration — honest gap: 0% refusal, 0% fabrication
- contradiction_detection — honest gap: 50% flag via versor-delta heuristic,
  100% false-positive; motivates the proper coherence-checker
- articulation_of_status — honest gap: 0% speculative articulation, 60%
  false certainty; output-side leak surface

New benchmarks:
- benchmarks/footprint.py — total deployed runtime is 7.06 MiB
  (109,358x smaller than Llama 3.1 405B, runs offline, no GPU)
- benchmarks/learning_curve.py — monotonic + replay-deterministic curve
  per lane

Documentation:
- docs/truth_seeking_schema.md — foundational architectural commitment,
  five rules, mapped to human failure modes, leaks published openly
- evals/CLAIMS.md — five-tier public claims doc; Tier 4.5 publishes
  known gaps with named fixes; verification contract at top
- README.md — new pillar between algebraic substrate and language pillar

Includes in-flight formation pipeline scaffolding (formation/, tests/formation/,
docs/formation_pipeline_plan.md) and minor CLI/contracts/gitignore edits
that were already in the working tree at session start.

Verification: 798 passed, 2 skipped, 1 deselected (pre-existing pack-count
test drift unrelated to schema changes).
2026-05-17 07:27:41 -07:00

118 lines
4 KiB
Python

"""Content-addressed artifact cache for the Formation Pipeline.
Layout::
.formation_cache/
<subject_id>/
<stage>/
<input_sha>.json
Keys are sanitized — path traversal patterns (``..``, leading ``/``, NUL,
backslashes) are rejected before any filesystem access. Subject IDs are
restricted to ``[A-Za-z0-9._-]+``. Stage names are restricted to a fixed
allow-list. Input SHAs must be 64-character lowercase hex.
Re-running an unchanged stage is a no-op cache hit — that is the speed
mechanism. All cache hits compare SHA equality only; there is no fuzzy
match.
No pickle. All cache files are canonical JSON.
"""
from __future__ import annotations
import json
import re
from pathlib import Path
from typing import Any, Final
from formation.hashing import canonical_json
_DEFAULT_CACHE_DIRNAME: Final[str] = ".formation_cache"
_SUBJECT_ID_RE: Final[re.Pattern[str]] = re.compile(r"^[A-Za-z0-9._-]+$")
_SHA_RE: Final[re.Pattern[str]] = re.compile(r"^[0-9a-f]{64}$")
_ALLOWED_STAGES: Final[frozenset[str]] = frozenset({
"subject_spec",
"ore",
"smelted",
"validated",
"course",
"plan",
"results",
"mastery",
})
class CacheKeyError(ValueError):
"""Raised when a cache key fails sanitization (e.g. path traversal)."""
class FormationCache:
"""File-backed cache keyed by ``(subject_id, stage, input_sha)``."""
def __init__(self, root: Path | str) -> None:
self._root = Path(root).resolve()
@property
def root(self) -> Path:
return self._root
def _validate(self, subject_id: str, stage: str, input_sha: str) -> None:
if not isinstance(subject_id, str) or not _SUBJECT_ID_RE.fullmatch(subject_id):
raise CacheKeyError(
f"invalid subject_id {subject_id!r}; must match [A-Za-z0-9._-]+"
)
if subject_id.startswith(".") or ".." in subject_id:
raise CacheKeyError(f"invalid subject_id {subject_id!r}; path traversal")
if stage not in _ALLOWED_STAGES:
raise CacheKeyError(
f"invalid stage {stage!r}; allowed: {sorted(_ALLOWED_STAGES)}"
)
if not isinstance(input_sha, str) or not _SHA_RE.fullmatch(input_sha):
raise CacheKeyError(
f"invalid input_sha {input_sha!r}; must be 64-char lowercase hex"
)
def path_for(self, subject_id: str, stage: str, input_sha: str) -> Path:
"""Return the resolved cache path for a key, validating components.
The resolved path is asserted to stay strictly under the cache root —
a final defense in depth even though the component regexes already
forbid traversal.
"""
self._validate(subject_id, stage, input_sha)
candidate = (self._root / subject_id / stage / f"{input_sha}.json").resolve()
if not str(candidate).startswith(str(self._root) + "/") and candidate != self._root:
raise CacheKeyError(
f"resolved cache path {candidate!s} escapes root {self._root!s}"
)
return candidate
def has(self, subject_id: str, stage: str, input_sha: str) -> bool:
return self.path_for(subject_id, stage, input_sha).exists()
def get(self, subject_id: str, stage: str, input_sha: str) -> Any | None:
path = self.path_for(subject_id, stage, input_sha)
if not path.exists():
return None
return json.loads(path.read_text(encoding="utf-8"))
def put(
self,
subject_id: str,
stage: str,
input_sha: str,
payload: Any,
) -> Path:
path = self.path_for(subject_id, stage, input_sha)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(canonical_json(payload))
return path
def default_cache(cwd: Path | str | None = None) -> FormationCache:
"""Return a ``FormationCache`` rooted at ``<cwd>/.formation_cache``."""
base = Path(cwd) if cwd is not None else Path.cwd()
return FormationCache(base / _DEFAULT_CACHE_DIRNAME)