Removes code_revision from the engine-identity hash: engine_identity is now the
sha256 of the 5 ratified packs ONLY. The build revision is provenance (the
manifest's written_at_revision), not identity — so a behavior-neutral rebuild is
the SAME identity and the always-on daemon no longer flag-day strict-breaks on
every commit (the ADR-0220 defect).
Core:
- core/engine_identity.py: ratified_substrate/compute_engine_identity drop the
git_revision arg (packs-only); add compute_legacy_engine_identity (reproduces
the pre-split packs+rev hash, for migration verification), ENGINE_IDENTITY_SCHEME=2,
IdentityReconciliation enum, and reconcile_loaded_identity — the single source of
truth for the runtime guard AND the workbench reader.
- chat/runtime.py: the load guard reconciles via scheme. Current scheme -> direct
packs-only compare. Legacy (code_revision-folded) stamp -> a VERIFYING migration:
reconstruct the legacy hash from the persisted written_at_revision; a match proves
packs unchanged (warn + re-stamp, resume, no break) — a mismatch means the packs
genuinely changed (DIVERGED -> strict-refuse). Preserves 'distinct packs => refuse'.
- engine_state/save_manifest: stamp identity_scheme alongside engine_identity
(additive-optional; no schema_version bump).
- workbench/readers.py: continuity reader uses the same reconcile (no phantom break
on legacy checkpoints). cli.py break message reworded (packs, not 'build revision').
Callsite cleanup: drop the now-unused git_revision arg + imports across
always_on.py, evals/l10_always_on/runner.py, workbench/readers.py.
Tests:
- test_identity_provenance_split.py (new): 5 reconcile unit proofs + 3 runtime
integration proofs incl. the wrong=identity defense (legacy stamp of DIFFERENT
packs still strict-refuses) and the re-stamp migration.
- test_engine_identity.py: invert the code-revision test (rev no longer changes
identity); assert the substrate is packs-only.
- Restore 3 lineage tests silently red since ADR-0219 (flat-path _manifest helper
now resolves the gen-dir).
- Update remaining callsites/monkeypatches for the new signature.
Hygiene: .gitignore now covers the ADR-0219 gen-dir runtime files
(current, gen-*/, session_state.json, proposals.jsonl).
Verified: 59 identity/migration/lineage/workbench + 32 L10 + 76 invariants/cli +
34 smoke pass; serving lane SHAs unchanged (no derivation/reliability_gate touch).
* feat(l10): autonomous idle frontier-contemplation — the always-on life learns on its own
Frontier survey move #2, path (c): give the always-on idle life a stream of experience via
SYMBOLIC intake (not the afferent→field seam, which is deferred pending a commensurability
ruling — docs/analysis/afferent-field-ingest-scope-2026-06-15.md). The idle heartbeat
converges (idle_tick drains a finite backlog + saturates is-a closure, then does nothing
forever); this makes it autonomously MINE its frontier with no user turn.
Reuse-heavy (the mechanism largely existed): a gated idle pass wires ADR-0080 contemplation
into the always-on loop.
- core/config.py: contemplate_frontier_during_idle (default off → no behavior change).
- chat/runtime.py: idle_tick pass 2b — when on, runs core.contemplation.run_contemplation()
(mines the checked-in frontier-compare reports) and persists a reviewable ContemplationRun
to <engine_state>/contemplation_runs/idle_<substrate_hash>.json. IDEMPOTENT per frontier
(an already-mined frontier is not re-persisted → the life converges, no churn) and MEMOIZED
per session (the static frontier is mined once, not re-read every beat). SPECULATIVE-only
(ADR-0080: ContemplationFinding.__post_init__ RAISES if not SPECULATIVE → wrong=0 by
construction; never ratified here — the HITL path is untouched). IdleTickResult gains
frontier_findings.
- chat/always_on.py: run_continuous did_work + HeartbeatRecord count frontier work, so the
heartbeat/soak convergence (H3) stays honest.
- core/cli.py: `core always-on --contemplate-frontier` (explicit learning-daemon mode) +
honest per-beat log ("+N findings").
Adversarial 3-lens review (wrong=0/convergence, determinism/trust-boundary, test-vacuity)
found 7; fixed the real ones: the MEDIUM torn-write (write_contemplation_run is now atomic
temp+os.replace — the indefinite-uptime daemon expects SIGKILL mid-write, and the skip-guard
would never repair a torn canonical file), the gitignore breadth (contemplation_runs/ at any
engine-state root), the 64-bit filename truncation (full substrate_hash), and the
re-mine-every-beat cost (session memoization). Deferred + documented: the shared
contemplation miner embeds absolute report paths in finding CONTENT — a portability wrinkle
that does NOT affect wrong=0/convergence/soak determinism (those key on the content-only
substrate_hash) and touches shared code + its test surface; a follow-up.
Tests (non-vacuous): 7 — off-by-default (no behavior change), mines-when-enabled,
findings-SPECULATIVE (wrong=0), converges-idempotent, heartbeat-mines-then-converges,
mines-once-per-session (memoization), atomic-no-orphan. 55 idle/contemplation/always-on +
96 invariants/contemplation + 28 smoke green; `core always-on --contemplate-frontier` mines
end-to-end. engine_state contemplation_runs are per-life runtime state (gitignored).
* fix(l10): isolate idle frontier pass + persist frontier_findings (#758)
Three review blockers on the autonomous idle frontier-contemplation pass:
1. Daemon isolation. idle_tick's pass 2b called run_contemplation /
write_contemplation_run bare inside the always-on loop (run_continuous
wraps idle_tick in nothing but an exit-checkpoint finally), so a malformed
frontier report or a transient FS fault would propagate out and KILL the
continuous life. Wrap the optional pass best-effort (the exit-checkpoint
boundary idiom): degrade to 0 findings + RuntimeWarning, leave did_work
untouched, let a later beat retry.
2. Durable did_work explanation. HeartbeatRecord carried frontier_findings
but serialize_report omitted it and AlwaysOnReport had no run total, so
lived_life.json could show did_work=true with facts=0/proposals=0 and no
persisted cause. Persist frontier_findings per-record + total_frontier_findings.
3. Honest HITL claim. The docstring/CLI/config said findings are "reviewable
via the HITL path", but the existing HITL queue + workbench reader scan
<repo>/contemplation/runs while the pass writes <engine_state>/contemplation_runs/.
Narrow the claim to "persisted reviewable artifact, not yet in the HITL
queue" (wiring that discovery is a follow-up).
Tests: 4 new non-vacuous regressions (mining-failure degrade, write-failure
degrade, always-on loop survives frontier failure, lived_life persists
frontier_findings) — all fail against the unpatched pass. 38 L10/always-on/
workbench tests green.
The L10 heartbeat loop (run_continuous) had no process to drive it; `core always-on`
is that process — the T-experience spine made runnable. It ticks idle_tick on a wall-clock
cadence so the engine LIVES and LEARNS with no user turn, persists lived_life.json (the
workbench Lived Life surface) + the checkpoint, and resumes the SAME life on restart.
- chat/always_on.py: run_continuous gains unbounded operation (heartbeats=None, runs until
stop) + an interruptible inter-beat wait (_sleep_until_stop) so shutdown latency is one
slice, not the cadence interval. Persists the run's identity pack ids (resume-verdict
faithfulness). on_heartbeat is now best-effort (a broken log pipe can't kill the life).
- chat/always_on_daemon.py (new): the daemon shell — a single-instance OS lock (fcntl.flock:
kernel-held, atomic, auto-released on death — no stale window, no PID-reuse, no half-written
race), SIGINT/SIGTERM -> graceful stop (handlers saved/restored), the continuous-life config
FORCED on, ephemeral (--no-load-state) writes no durable artifact. Foreground + explicit:
no hidden background execution (CLAUDE.md); only writes the engine-state dir it was given.
- core/cli.py: `core always-on [--interval --max-beats --no-load-state --quiet]` with per-beat
+ summary logging; validates --interval; reports IdentityContinuityError / IncompatibleEngine
StateError (the "different life / newer build" cases) as clean refusals, not tracebacks.
- workbench/readers.py: ENGINE_STATE_ROOT now honors CORE_ENGINE_STATE_DIR (= the daemon's
resolved dir), so the workbench can't be split-brained (reading REPO_ROOT/engine_state while
the daemon writes elsewhere); the Lived Life resume verdict recomputes from the persisted
pack config, not a default config (no false substrate_changed for a non-default-pack life).
- Lived Life absence state now points at the real `core always-on` command (loop closed).
Adversarial 4-lens review (lock/concurrency, signals/shutdown, invariants/trust-boundary,
test-vacuity/CLI) caught 16 findings; this fixes all real ones — the HIGH lock races (two
daemons over one life), the env split-brain, the IO-kill, the uncaught identity/schema errors,
the unvalidated interval, the ephemeral-artifact shadow, and the resume-verdict pack-id bug —
and closes the two test-coverage gaps it flagged (real SIGTERM path + config-forcing-at-the-
runtime boundary).
Tests (non-vacuous): 11 daemon (flock live-holder refusal, leftover-lock reclaim, unbounded+
stop, interruptible sleep, forced-config-at-boundary, no-load-state guard, REAL SIGTERM
subprocess) + the reader pack-id discrimination test (fails under the old default-config bug).
245 workbench+invariants+always-on Python green; frontend tsc + vitest green; `core always-on`
verified end-to-end (bounded, real SIGTERM graceful stop, interval rejection).
engine_state/always_on.lock is runtime state (gitignored, ADR-0146 pattern).
The L10 heartbeat (chat/always_on, #747) holds CORE alive over uptime but had no
surface. This adds the read-only Lived Life route that renders the continuous-life
evidence: the heartbeat over uptime + the resume-as-same-life verdict — both halves
of "one continuous life" on one surface.
Persist-first, never recompute engine-owned values:
- chat/always_on: serialize_report + write_lived_life persist a run deterministically
(sorted keys) to engine_state/lived_life.json; run_continuous gains an opt-in
report_path so a real continuous-life run leaves its evidence where the workbench
reads it. closure_ceiling is persisted so the artifact is self-describing.
- workbench/lived_life.py: projection + a fail-closed validate() gate — the wrong=0
analogue for the continuity surface: closure_held/closure_observed/totals/converged
are each re-checked against the per-beat measurements, and resume_status is re-checked
against identity-vs-current-substrate, so a tampered artifact RAISES rather than
rendering a false claim (a beat lying about a breached ceiling, an inflated closure,
a miscounted total, a falsely-claimed resume).
- workbench/readers.lived_life(): reads the artifact, computes the resume verdict from
the persisted identity vs the canonical engine_identity_for_config (fail-soft ->
"unknown", like IdentityContinuity). Honest absence when no run has been persisted.
- GET /lived-life + LivedLife/LivedLifeHeartbeat schemas (snapshot regenerated; the
schema-drift gate proves every field is mirrored in src/types/api.ts).
The resume verdict made felt: would_resume / substrate_changed / unknown IS the L11
reboot guarantee (a reboot recomputes identity and refuses if it differs) — so the
surface shows this life wakes up as ITSELF, not a copy. The per-reboot lineage chain
stays owned by Runs > Identity (honest cross-link).
Frontend: /lived-life route (Evidence section, route count 15 -> 16). Renders the
headline (heartbeats, closure-held-by-construction, learned-while-idle, at-rest, resume
pill), the summary, the heartbeat timeline (closure flat below the ceiling — read, never
repaired), and the resume verdict. Honest absence + ADR-0162 route conformance
(loading/error/empty), fail-closed like Vault/Calibration.
Tests (non-vacuous): 12 backend (every tamper case raises; resume tracks identity vs
substrate; run_continuous(report_path) round-trips a readable+valid artifact) + 4
frontend render tests (recorded life renders; a breached beat shows the warning, never a
false "held"; a changed substrate shows would-refuse, never a false resume; absent shows
honest absence). All workbench Python (167) + affected vitest (167) + tsc + vite build
green; architectural invariants green.
engine_state/lived_life.json is runtime state (gitignored, ADR-0146 pattern).
Append-only JSONL journal records the exact ChatTurnResult envelope returned by /chat/turn with stable turn_id, trace_hash, all three surfaces, verdicts, and deterministic journal_digest.
GET /trace/turns and GET /trace/{turn_id} serve journal evidence for the Trace route frontend. Read model only; no teaching, pack, or journal-owned engine_state mutation.
Two-pass hardening of the #690 demo so no epistemic state is proposer-mintable:
Pass 1 (sealed corpus, from the reconciliation pass):
- evidence items are references only (evidence_id + content_sha256) resolved
against a committed content-addressed corpus (evidence_corpus.json)
- support derived from committed subject/predicate match; independence from
distinct provenance_root values; proposer support/independence labels are
rejected by the closed schema
- evidence-strength taxonomy (substrate-capability vs interface-contract) in
PUBLIC-DEMO-ROADMAP / DEMO-PACKAGING-CHECKLIST; position-paper refinements
Pass 2 (entailment-decided inference, closes a live false-status hazard):
- the merged inferred leg required only that premise IDs resolve — a claim
with no support citing one unrelated record as a premise was assigned
'inferred' (empirically confirmed). inferred is now decided by
generate.proof_chain.entail (sound+complete ROBDD) over resolved corpus
premises, cross-checked against the independent truth-table oracle
(evals.deductive_logic.oracle, no shared code); proof keys + oracle verdict
in the trace; disagreement => defensive refusal; refutation => contradicted
- committed adversarial scenario unrelated-premise-still-undetermined proves
the attack now lands on undetermined
- corpus seal (corpus_sha256) pinned into every evaluated trace; scope
refusals no longer consult the corpus; corpus cache hands out copies;
claim subject/predicate constrained to atom-compatible snake_case
- .gitignore: private packet dir, agent worktrees, tooling artifacts
Validation: 30 demo tests; 96 across all three demo suites; 7/7 scenarios
double-run byte-identical; 61 architectural invariants green.
Commits the design/analysis artifacts produced while building the lived spine and
planning the next arc (the work itself already merged in #563-#573):
- AGI-candidacy-autonomous-improvement-roadmap-2026-06-05.md — the path from the
lived spine to AGI-candidacy: the comprehend→realize→determine→learn loop, the
cross-domain capability+calibration yardstick, the logical-necessity × technical-
priority execution order, and the corrected epistemic foundation (grounded
honesty designed-in; estimation learned/ratified; confidence always evidence-
grounded; intake first-class — NOT "no ingestion"; calibration+grounding the
measured invariant).
- L10-runtime-scoping + L10-continuity-spike-design — the L10 decision surface and
the falsifiable spike spec (P1-P5) that became evals/l10_continuity/.
- L10-shapeBplus-persistence-scope — the A->E scope that became Shape B+ resume.
.gitignore: ignore the local .system-map/ navigation index (per-developer, never
tracked; regenerated on demand).
C1: delete generate/math_versor_arithmetic.py and its 3 tests (ADR-0139
add-only arithmetic spike; no runtime consumers, no pipeline wiring,
follow-on lift paused per module docstring).
C3: gitignore engine_state runtime artifacts (manifest.json,
recognizers.jsonl, discovery_candidates.jsonl). Module code
(engine_state/__init__.py) remains tracked; generated checkpoint
files should not be.
C5: document reader zero-delta root cause in train_sample/v1/README.md.
Both Phase 2 (whole-problem) and Phase 1 (question-only) reader paths are
called but inert because all 47 refusals are statement-level NO_INJECTOR
gaps, not question-sentence gaps. Reader unblocks when injector coverage
expands (C2 work). report.json use_reader flag corrected to reflect last run.
C6: add deprecation header to generate/math_parser.py pointing at
generate.math_candidate_graph.parse_and_solve as the live path.
C2/C4 briefs: docs/handoff/CLEANUP-C2-run-lane-migration.md and
docs/handoff/CLEANUP-C4-compositions-compile.md added as operator
dispatch docs for the medium-scope wiring tasks.
The 1,319 GSM8K test cases are now sealed at
evals/gsm8k_math/holdouts/v1/cases.jsonl.age, age-encrypted to the
ADR-0119.1 recipient. Plaintext never touched disk in the working
tree; only ciphertext is committed.
First honest CORE-vs-real-GSM8K measurement
cases_total: 1319
correct: 0
wrong: 0 ← ADR-0114a Obligation #4 holds against external corpus
refused: 1319
overall_pass: True
Zero confabulation. Parser refuses what it can't grammar-handle; the
"wrong == 0" discipline survives the move from CORE-original cases
to a real public benchmark. The 0/1319 correct rate is the truthful
gap that ADR-0120's threshold work will quantify.
What landed
scripts/seal_gsm8k_test.py
- Loads GSM8K via datasets.load_dataset("openai/gsm8k", "main")
- Strips worked-solution prose; extracts final-answer integer/float
after "####" (handles "2,125" → 2125 thousands-separator)
- Reads recipient from docs/holdout_recipients.txt (single repo key
per ADR-0119.1)
- Encrypts via pyrage; writes only ciphertext
- Refuses to overwrite test path with train-derived seal
evals/gsm8k_math/runner.py
- Empty expected_unit (sentinel) skips unit-comparison; grades on
answer value alone. Required because GSM8K answers carry no unit
structurally. wrong-zero discipline preserved.
tests/test_adr_0119_7_sealed_gsm8k.py — 6 invariants:
1. sealed file present + age-formatted
2. no plaintext companion files (sibling-leak guard)
3. decrypted JSONL matches documented schema
4. runner against decrypted suite produces wrong==0
5. tests skip (not fail) when CORE_HOLDOUT_KEY unset
6. case ids match "gsm8k-test-NNNN" pattern
Defensive gitignore: plaintext patterns under
evals/gsm8k_math/holdouts/v1/ are explicitly excluded.
ADR-0114a obligation roll-up
10/10 discharged for the gsm8k_math lane:
#1 ✓ sealed-holdout (fab_control + GSM8K test)
#2..#10 ✓ as before
Phase 5 status: 5.1..5.7 done; 5.8 in flight (PR #149). After 5.8
merges, ADR-0120 (first expert promotion contract) becomes
feasible.
Test plan
- pytest tests/test_adr_0119_7_sealed_gsm8k.py with CORE_HOLDOUT_KEY → 6/6
- pytest without CORE_HOLDOUT_KEY → 3 pass + 3 skip
- core test --suite smoke -q → 67/67
- CLAIMS.md regenerated (no diff)
- HF token NEVER in repo (saved at ~/.cache/huggingface/token, mode 600)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The word "expert" in the previous status name implied raw-capability parity
with frontier LLMs on the same benchmark — which the gate does NOT verify.
What the gate actually verifies is CORE *claim-shape compliance*:
* signed digest (replay-reproducible from on-disk lane results)
* replay determinism (same inputs → byte-equal trace_hash)
* typed refusal (fabrication refused, not paraphrased)
* exact recall (no ANN, no cosine, no attention bottleneck)
* grounding-source provenance
These are claim shapes a transformer LLM cannot structurally produce
regardless of raw accuracy. A frontier LLM might score higher on the
same benchmark but cannot pass this contract.
Rename scope (semantics only, per ADR-0113):
status string "expert-demo" → "audit-passed"
predicate key predicates.expert_demo → predicates.audit_passed
reason key expert_demo_reason → audit_passed_reason
YAML key expert_demo_claims → audit_passed_claims
CLI command core demo expert → core demo audit-passed
output dir evals/expert_demos/ → evals/audit_passed/
artifact filenames expert_demo.{json,html} → audit_passed.{json,html}
HTML title CORE Expert-Demo: X → CORE Audit-Passed: X
Internal Python identifiers (module/file/function/class names like
`expert_demo.py`, `evaluate_expert_demo`, `ExpertDemoClaim`,
`expert_demo_claim_for`) are deliberately kept to minimize churn. ADR
file titles (ADR-0106..0112) preserved as historical record.
`expert` namespace reserved for ADR-0114+: an actual capability tier
above `audit-passed` backed by a public benchmark with a stated
threshold. ADR-0114 proposes the first such target — GSM8K-math —
laying out a falsifiable 7-phase arc (parser → solver → verifier →
stepped-realizer → eval lane → first `expert` ledger tier promotion).
Tests: 184 directly-affected tests green (140 capability/expert-demo
suite + 34 demo/audit-tour + 10 correction-cue). Smoke suite 67/67.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the asymmetry between the `expert-demo` ledger status (audit
artifact only) and the actual `core demo` surface (runnable
walkthroughs producing HTML + JSON). Until this commit the word
"demo" in `expert-demo` was aspirational; now it corresponds to
something a reader can open.
What it does
- Reads the signed expert_demo_claims entry from docs/reviewers.yaml
- Loads latest on-disk result files for each attached lane × split
- Re-derives the evidence-bundle digest and asserts byte-for-byte
match against the signed claim_digest — this is the load-bearing
audit step, now exercised at two independent enforcement points
(ledger gate + showcase)
- Runs each lane's metrics through the ADR-0109 lane-shape registry
and surfaces the verdict
- Picks the first three cases from each split verbatim (deterministic
by file order) and renders them as HTML for inspection
- Emits expert_demo.json (canonical bytes, deterministic) + expert_demo.html
Surface
core demo expert --domain mathematics_logic
core demo expert --domain physics
# → evals/expert_demos/<domain>/latest/expert_demo.{json,html}
Read-only by construction: cannot mutate docs/reviewers.yaml or any
lane result file. Tested. Unpromoted domains raise ValueError —
no silent fallback, no "preview" mode that fakes a showcase.
Generated artifacts are gitignored — the inputs they derive from are
already committed, so duplicating the renders would just churn the
tree.
Tests: 16 new cases pinning all five ADR-0112 invariants. Smoke suite
still 67/67 green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
#58 shipped providers.py + model_registry.py for cross-provider
benchmarking but never connected them to runner.py — the adapters
sat unused. This PR wires them through with a clear lane split.
Why a new suite instead of refactoring existing ones
-----------------------------------------------------
The three existing suites (determinism / truth_lock / axis_orthogonality)
pull CORE-only telemetry: trace_hash, versor_condition, register_id,
register_variant_id, anchor_lens_id, register_canonical_surface.
None of those fields can come from OpenAI / Anthropic / Ollama.
Forcing those suites cross-provider would silently produce reports
where the cross-provider rows have empty telemetry — a worse failure
mode than not running them at all. So the routing is explicit:
CORE-only suites → --provider must be 'core'
Cross-provider suites → any provider; CORE is one adapter among many
Operator asks for the wrong combo → loud error with the right alternative.
New module: evals/frontier_compare/cross_provider.py
-----------------------------------------------------
- ProviderObservation dataclass — provider-agnostic observation shape
(prompt, surface, provider, model, elapsed_ms, error fields). No
CORE-internal telemetry expected.
- run_prompt_battery(adapter, *, cfg) → SuiteReport reusing existing
CaseResult / SuiteReport shapes so the report viewer renders both
lanes without schema branching.
- _PROMPT_BATTERY: 7 fixed cases spanning definition / cause /
verification / comparison / procedure / unknown intent shapes.
Stable case_ids so future re-runs against the same provider produce
diffable JSON.
- Per-case 'passed' is loose by design (non-empty surface, no
exception). Cross-provider quality is for human review — not for
the runner to silently score.
Updated CLI: evals/frontier_compare/__main__.py
-----------------------------------------------
- --provider {core, openai, anthropic, ollama} (default: core)
- --model <id> (validated via require_model_card)
- --env-file <path> (default: ./.env)
- Auto-persist non-CORE runs to
evals/frontier_compare/results/<provider>_<model>_<utc>.json
even when --report is omitted. API calls are rate-limited / paid;
losing the artifact is costly.
- Existing CORE-native behavior unchanged when --provider not set.
Results directory: evals/frontier_compare/results/
--------------------------------------------------
Created with .gitkeep — matches the convention used by other lanes
(evals/long_context_cost/results/, evals/koine_greek_fluency/results/,
etc.). Distinct from reports/ which .gitignore excludes for
transient debug output.
Tests: tests/test_frontier_compare_cross_provider.py (9 cases)
--------------------------------------------------------------
- prompt_battery runs with CORE adapter (no API needed)
- adapter exceptions recorded as failed observations, never propagated
- empty surfaces flagged distinctly from adapter errors
- CLI default runs CORE-native (no breaking change)
- CLI prompt_battery with --provider core routes through cross-provider path
- CLI rejects CORE-only suite + non-CORE provider with operator-helpful error
- --help surfaces both suite families
- unregistered model is rejected before any benchmark cycles burn
- ProviderObservation.succeeded handles error / empty / whitespace cases
Live evidence
-------------
$ core test --suite smoke -q
67 passed in 26.55s (no regression)
$ python -m evals.frontier_compare --provider core --suite prompt_battery --json
model=core-native mode=core suite=prompt_battery passed=True score=1.000
[definition_truth ] PASS Truth is a claim or state grounded by evidence...
[definition_knowledge ] PASS Knowledge is justified understanding grounded...
[cause_understanding ] PASS understanding — teaching-grounded (cognition_chains_v1)...
[verification_evidence ] PASS evidence — teaching-grounded (cognition_chains_v1)...
[comparison_knowledge_wisdom ] PASS knowledge contrasts with wisdom...
[procedure_recall ] PASS To recall means to retrieve a stored state from memory...
[unknown_term ] PASS I haven't learned 'xylomorphic' yet...
$ python -m evals.frontier_compare --provider openai --suite determinism
error: suite 'determinism' is CORE-only; pass --suite prompt_battery
(the cross-provider suite) when --provider='openai'.
.gitignore: adds frontier_wave1.json (stray report file repeatedly
written by ad-hoc test invocations).
Audit of the one-mutation-path invariant (ADR-0021 §3) found three leaks
where pack authority or session-state writes could substitute for coherence
judgment. All three landed fixes or partial closures in this push.
Leaks closed:
- Leak A: pack vocab defaulted to COHERENT — flipped to SPECULATIVE in
language_packs/{compiler,schema}.py; docstring corrected to align with
ADR-0021 (it was rationalizing the leak).
- Leak B: vault.recall was epistemic-blind — VaultStore.store() now stamps
every entry with EpistemicStatus (default SPECULATIVE); recall(min_status=)
filters to admissible-as-evidence tier. All 4 vault-write sites updated.
- Leak C (write-side): generate/proposition.py:198 stored articulated
propositions unmarked — now stamps SPECULATIVE, breaking the
fabrication-feedback loop in principle. Read-side audit of 5 call sites
is the residual.
New architectural invariants (tests/test_architectural_invariants.py):
- INV-21: one-mutation-path allowlist (caught Leak C on first run)
- INV-22: pack lexicon default is SPECULATIVE (Leak A guard)
- INV-23: vault recall epistemic-aware (Leak B guard)
New eval lanes:
- teaching_injection_resistance — ships GREEN at 1.00/1.00/0 (the
structural anti-injection claim is real and measurable)
- refusal_calibration — honest gap: 0% refusal, 0% fabrication
- contradiction_detection — honest gap: 50% flag via versor-delta heuristic,
100% false-positive; motivates the proper coherence-checker
- articulation_of_status — honest gap: 0% speculative articulation, 60%
false certainty; output-side leak surface
New benchmarks:
- benchmarks/footprint.py — total deployed runtime is 7.06 MiB
(109,358x smaller than Llama 3.1 405B, runs offline, no GPU)
- benchmarks/learning_curve.py — monotonic + replay-deterministic curve
per lane
Documentation:
- docs/truth_seeking_schema.md — foundational architectural commitment,
five rules, mapped to human failure modes, leaks published openly
- evals/CLAIMS.md — five-tier public claims doc; Tier 4.5 publishes
known gaps with named fixes; verification contract at top
- README.md — new pillar between algebraic substrate and language pillar
Includes in-flight formation pipeline scaffolding (formation/, tests/formation/,
docs/formation_pipeline_plan.md) and minor CLI/contracts/gitignore edits
that were already in the working tree at session start.
Verification: 798 passed, 2 skipped, 1 deselected (pre-existing pack-count
test drift unrelated to schema changes).
