64c5bc4619
Audit of the one-mutation-path invariant (ADR-0021 §3) found three leaks
where pack authority or session-state writes could substitute for coherence
judgment. All three landed fixes or partial closures in this push.
Leaks closed:
- Leak A: pack vocab defaulted to COHERENT — flipped to SPECULATIVE in
language_packs/{compiler,schema}.py; docstring corrected to align with
ADR-0021 (it was rationalizing the leak).
- Leak B: vault.recall was epistemic-blind — VaultStore.store() now stamps
every entry with EpistemicStatus (default SPECULATIVE); recall(min_status=)
filters to admissible-as-evidence tier. All 4 vault-write sites updated.
- Leak C (write-side): generate/proposition.py:198 stored articulated
propositions unmarked — now stamps SPECULATIVE, breaking the
fabrication-feedback loop in principle. Read-side audit of 5 call sites
is the residual.
New architectural invariants (tests/test_architectural_invariants.py):
- INV-21: one-mutation-path allowlist (caught Leak C on first run)
- INV-22: pack lexicon default is SPECULATIVE (Leak A guard)
- INV-23: vault recall epistemic-aware (Leak B guard)
New eval lanes:
- teaching_injection_resistance — ships GREEN at 1.00/1.00/0 (the
structural anti-injection claim is real and measurable)
- refusal_calibration — honest gap: 0% refusal, 0% fabrication
- contradiction_detection — honest gap: 50% flag via versor-delta heuristic,
100% false-positive; motivates the proper coherence-checker
- articulation_of_status — honest gap: 0% speculative articulation, 60%
false certainty; output-side leak surface
New benchmarks:
- benchmarks/footprint.py — total deployed runtime is 7.06 MiB
(109,358x smaller than Llama 3.1 405B, runs offline, no GPU)
- benchmarks/learning_curve.py — monotonic + replay-deterministic curve
per lane
Documentation:
- docs/truth_seeking_schema.md — foundational architectural commitment,
five rules, mapped to human failure modes, leaks published openly
- evals/CLAIMS.md — five-tier public claims doc; Tier 4.5 publishes
known gaps with named fixes; verification contract at top
- README.md — new pillar between algebraic substrate and language pillar
Includes in-flight formation pipeline scaffolding (formation/, tests/formation/,
docs/formation_pipeline_plan.md) and minor CLI/contracts/gitignore edits
that were already in the working tree at session start.
Verification: 798 passed, 2 skipped, 1 deselected (pre-existing pack-count
test drift unrelated to schema changes).
87 lines
3 KiB
Python
87 lines
3 KiB
Python
"""Tests for ``formation.cache`` — path-traversal-safe content-addressed cache."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import pytest
|
|
|
|
from formation.cache import CacheKeyError, FormationCache, default_cache
|
|
|
|
|
|
@pytest.fixture
|
|
def cache(tmp_path):
|
|
return FormationCache(tmp_path / ".formation_cache")
|
|
|
|
|
|
_VALID_SHA = "0" * 64
|
|
_OTHER_SHA = "a" * 64
|
|
|
|
|
|
class TestKeySanitization:
|
|
@pytest.mark.parametrize("bad_subject", [
|
|
"../escape",
|
|
"/abs/path",
|
|
"bad/slash",
|
|
"bad space",
|
|
"bad\\backslash",
|
|
"bad\x00null",
|
|
".dotleading",
|
|
"..",
|
|
"",
|
|
])
|
|
def test_subject_id_rejected(self, cache, bad_subject) -> None:
|
|
with pytest.raises(CacheKeyError):
|
|
cache.path_for(bad_subject, "ore", _VALID_SHA)
|
|
|
|
def test_stage_must_be_allowlisted(self, cache) -> None:
|
|
with pytest.raises(CacheKeyError, match="invalid stage"):
|
|
cache.path_for("subject.x", "arbitrary", _VALID_SHA)
|
|
|
|
@pytest.mark.parametrize("bad_sha", [
|
|
"tooshort",
|
|
"Z" * 64, # uppercase / non-hex
|
|
"0" * 63, # short
|
|
"0" * 65, # long
|
|
"../../bad",
|
|
"",
|
|
])
|
|
def test_input_sha_rejected(self, cache, bad_sha) -> None:
|
|
with pytest.raises(CacheKeyError, match="invalid input_sha"):
|
|
cache.path_for("subject.x", "ore", bad_sha)
|
|
|
|
def test_valid_key_resolves_under_root(self, cache) -> None:
|
|
path = cache.path_for("subject.x", "ore", _VALID_SHA)
|
|
assert str(path).startswith(str(cache.root) + "/")
|
|
assert path.suffix == ".json"
|
|
|
|
|
|
class TestPutGet:
|
|
def test_put_then_get_round_trips(self, cache) -> None:
|
|
payload = {"hello": "world", "n": 42}
|
|
cache.put("subject.x", "ore", _VALID_SHA, payload)
|
|
assert cache.has("subject.x", "ore", _VALID_SHA)
|
|
assert cache.get("subject.x", "ore", _VALID_SHA) == payload
|
|
|
|
def test_miss_returns_none(self, cache) -> None:
|
|
assert cache.get("subject.x", "ore", _VALID_SHA) is None
|
|
assert cache.has("subject.x", "ore", _VALID_SHA) is False
|
|
|
|
def test_different_sha_isolated(self, cache) -> None:
|
|
cache.put("subject.x", "ore", _VALID_SHA, {"v": 1})
|
|
cache.put("subject.x", "ore", _OTHER_SHA, {"v": 2})
|
|
assert cache.get("subject.x", "ore", _VALID_SHA) == {"v": 1}
|
|
assert cache.get("subject.x", "ore", _OTHER_SHA) == {"v": 2}
|
|
|
|
def test_written_bytes_are_canonical(self, cache) -> None:
|
|
cache.put("subject.x", "ore", _VALID_SHA, {"b": 1, "a": 2})
|
|
path = cache.path_for("subject.x", "ore", _VALID_SHA)
|
|
assert path.read_bytes() == b'{"a":2,"b":1}'
|
|
|
|
def test_put_floats_rejected(self, cache) -> None:
|
|
with pytest.raises(TypeError):
|
|
cache.put("subject.x", "ore", _VALID_SHA, {"v": 1.5})
|
|
|
|
|
|
class TestDefaultCache:
|
|
def test_default_cache_relative_to_cwd(self, tmp_path) -> None:
|
|
cache = default_cache(tmp_path)
|
|
assert cache.root == (tmp_path / ".formation_cache").resolve()
|