core/evals/reports/footprint_latest.json
Shay 64c5bc4619 feat(epistemic): truth-seeking schema audit — 3 leaks closed, 4 new lanes, 3 new invariants
Audit of the one-mutation-path invariant (ADR-0021 §3) found three leaks
where pack authority or session-state writes could substitute for coherence
judgment. All three landed fixes or partial closures in this push.

Leaks closed:
- Leak A: pack vocab defaulted to COHERENT — flipped to SPECULATIVE in
  language_packs/{compiler,schema}.py; docstring corrected to align with
  ADR-0021 (it was rationalizing the leak).
- Leak B: vault.recall was epistemic-blind — VaultStore.store() now stamps
  every entry with EpistemicStatus (default SPECULATIVE); recall(min_status=)
  filters to admissible-as-evidence tier. All 4 vault-write sites updated.
- Leak C (write-side): generate/proposition.py:198 stored articulated
  propositions unmarked — now stamps SPECULATIVE, breaking the
  fabrication-feedback loop in principle. Read-side audit of 5 call sites
  is the residual.

New architectural invariants (tests/test_architectural_invariants.py):
- INV-21: one-mutation-path allowlist (caught Leak C on first run)
- INV-22: pack lexicon default is SPECULATIVE (Leak A guard)
- INV-23: vault recall epistemic-aware (Leak B guard)

New eval lanes:
- teaching_injection_resistance — ships GREEN at 1.00/1.00/0 (the
  structural anti-injection claim is real and measurable)
- refusal_calibration — honest gap: 0% refusal, 0% fabrication
- contradiction_detection — honest gap: 50% flag via versor-delta heuristic,
  100% false-positive; motivates the proper coherence-checker
- articulation_of_status — honest gap: 0% speculative articulation, 60%
  false certainty; output-side leak surface

New benchmarks:
- benchmarks/footprint.py — total deployed runtime is 7.06 MiB
  (109,358x smaller than Llama 3.1 405B, runs offline, no GPU)
- benchmarks/learning_curve.py — monotonic + replay-deterministic curve
  per lane

Documentation:
- docs/truth_seeking_schema.md — foundational architectural commitment,
  five rules, mapped to human failure modes, leaks published openly
- evals/CLAIMS.md — five-tier public claims doc; Tier 4.5 publishes
  known gaps with named fixes; verification contract at top
- README.md — new pillar between algebraic substrate and language pillar

Includes in-flight formation pipeline scaffolding (formation/, tests/formation/,
docs/formation_pipeline_plan.md) and minor CLI/contracts/gitignore edits
that were already in the working tree at session start.

Verification: 798 passed, 2 skipped, 1 deselected (pre-existing pack-count
test drift unrelated to schema changes).
2026-05-17 07:27:41 -07:00

79 lines
2.2 KiB
JSON

{
"artifacts": [
{
"bytes_on_disk": 23420,
"human": "22.87 KiB",
"name": "language_pack:en_core_cognition_v1",
"path": "language_packs/data/en_core_cognition_v1",
"present": true
},
{
"bytes_on_disk": 109435,
"human": "106.87 KiB",
"name": "seed_packs",
"path": "packs",
"present": true
},
{
"bytes_on_disk": 44924,
"human": "43.87 KiB",
"name": "vault_module",
"path": "vault",
"present": true
},
{
"bytes_on_disk": 558848,
"human": "545.75 KiB",
"name": "rust_backend",
"path": "core-rs/target/release/libcore_rs.dylib",
"present": true
}
],
"deployment": {
"requires_api_key": false,
"requires_gpu": false,
"requires_network": false,
"runs_offline": true
},
"frontier_context": [
{
"core_is_smaller_by_x": 2160.2,
"name": "Llama 3.1 8B (fp16)",
"parameters_billion": 8.0,
"source_note": "Meta model card, 2024 — 2 bytes/param at fp16.",
"weights_bytes_estimate": 16000000000,
"weights_human": "14.90 GiB"
},
{
"core_is_smaller_by_x": 18901.4,
"name": "Llama 3.1 70B (fp16)",
"parameters_billion": 70.0,
"source_note": "Meta model card, 2024 — 2 bytes/param at fp16.",
"weights_bytes_estimate": 140000000000,
"weights_human": "130.39 GiB"
},
{
"core_is_smaller_by_x": 109358.4,
"name": "Llama 3.1 405B (fp16)",
"parameters_billion": 405.0,
"source_note": "Meta model card, 2024 — 2 bytes/param at fp16.",
"weights_bytes_estimate": 810000000000,
"weights_human": "754.37 GiB"
},
{
"core_is_smaller_by_x": 47253.6,
"name": "GPT-3.5 (175B, fp16 estimate)",
"parameters_billion": 175.0,
"source_note": "Brown et al. 2020, GPT-3 paper; size is public, fp16 storage assumed.",
"weights_bytes_estimate": 350000000000,
"weights_human": "325.96 GiB"
}
],
"python_runtime_bytes": 6670213,
"python_runtime_module_count": 145,
"rss_idle_bytes": 18579456,
"rss_post_pulse_bytes": 34750464,
"rss_pulse_delta_bytes": 16171008,
"total_disk_bytes": 7406840,
"total_disk_human": "7.06 MiB"
}