core/docs/truth_seeking_schema.md
Shay 64c5bc4619 feat(epistemic): truth-seeking schema audit — 3 leaks closed, 4 new lanes, 3 new invariants
Audit of the one-mutation-path invariant (ADR-0021 §3) found three leaks
where pack authority or session-state writes could substitute for coherence
judgment. All three landed fixes or partial closures in this push.

Leaks closed:
- Leak A: pack vocab defaulted to COHERENT — flipped to SPECULATIVE in
  language_packs/{compiler,schema}.py; docstring corrected to align with
  ADR-0021 (it was rationalizing the leak).
- Leak B: vault.recall was epistemic-blind — VaultStore.store() now stamps
  every entry with EpistemicStatus (default SPECULATIVE); recall(min_status=)
  filters to admissible-as-evidence tier. All 4 vault-write sites updated.
- Leak C (write-side): generate/proposition.py:198 stored articulated
  propositions unmarked — now stamps SPECULATIVE, breaking the
  fabrication-feedback loop in principle. Read-side audit of 5 call sites
  is the residual.

New architectural invariants (tests/test_architectural_invariants.py):
- INV-21: one-mutation-path allowlist (caught Leak C on first run)
- INV-22: pack lexicon default is SPECULATIVE (Leak A guard)
- INV-23: vault recall epistemic-aware (Leak B guard)

New eval lanes:
- teaching_injection_resistance — ships GREEN at 1.00/1.00/0 (the
  structural anti-injection claim is real and measurable)
- refusal_calibration — honest gap: 0% refusal, 0% fabrication
- contradiction_detection — honest gap: 50% flag via versor-delta heuristic,
  100% false-positive; motivates the proper coherence-checker
- articulation_of_status — honest gap: 0% speculative articulation, 60%
  false certainty; output-side leak surface

New benchmarks:
- benchmarks/footprint.py — total deployed runtime is 7.06 MiB
  (109,358x smaller than Llama 3.1 405B, runs offline, no GPU)
- benchmarks/learning_curve.py — monotonic + replay-deterministic curve
  per lane

Documentation:
- docs/truth_seeking_schema.md — foundational architectural commitment,
  five rules, mapped to human failure modes, leaks published openly
- evals/CLAIMS.md — five-tier public claims doc; Tier 4.5 publishes
  known gaps with named fixes; verification contract at top
- README.md — new pillar between algebraic substrate and language pillar

Includes in-flight formation pipeline scaffolding (formation/, tests/formation/,
docs/formation_pipeline_plan.md) and minor CLI/contracts/gitignore edits
that were already in the working tree at session start.

Verification: 798 passed, 2 skipped, 1 deselected (pre-existing pack-count
test drift unrelated to schema changes).
2026-05-17 07:27:41 -07:00

15 KiB

The Truth-Seeking Schema

One of the foundational architectural commitments of CORE. Co-equal with the Cl(4,1) algebraic substrate; both are load-bearing.

Why this document exists

Modern AI systems are widely understood to be capable, fast, and useful. They are not widely understood to be truth-seeking. The difference is not academic. A system that synthesizes plausible outputs from opaque weights — and is rewarded during training for sounding right — develops the same epistemic failure modes that afflict human reasoning when it goes wrong: confabulation, narrative smoothing, selective recall, defensive identity protection, deference to authority, and the slow ossification of mistaken beliefs.

Building an AI that does not do these things is talked about more than it is built. CORE is an attempt to build it. This document states the architectural commitments that make that attempt falsifiable: what we have actually built, where the current gaps are, and what mechanism prevents each failure mode by construction rather than by hope.

The reader who only cares about benchmarks should read evals/CLAIMS.md — every claim in this document maps to a row there with a reproducible measurement command.


The five architectural commitments

These are not principles the system tries to follow. They are properties of the substrate the system runs on. Each is enforced in code at the indicated location. Each has a test that fails if the property is broken.

1. Coherence, not authority, is the only admission signal

Source: teaching/epistemic.py, ADR-0021 §3.

A claim is admitted as evidence in downstream inference if and only if it has been judged coherent with the existing reviewed field. Not because the textbook said it. Not because a famous person endorsed it. Not because a popular consensus exists. Not because the model itself said it confidently a moment ago.

EpistemicStatus has four positions: SPECULATIVE, COHERENT, CONTESTED, FALSIFIED. Only COHERENT is admissible as evidence (ADMISSIBLE_AS_EVIDENCE = frozenset({EpistemicStatus.COHERENT})). The enum deliberately excludes source-trust labels like peer_consensus, outsider_empirical, or established — including them would re-import the bias the schema is designed to refuse.

This is the structural defense against argument from authority, ad populum, credentialism, and the closely related failure mode in which a model trusts its own prior output because it sounds confident.

2. The non-hardening invariant

Source: ADR-0021 §2, verified by absence: no final, frozen, axiom, or permanent flag exists in the codebase.

No claim is ever locked. Even COHERENT is revisable. There is no "this is settled, stop questioning" status, and adding one is a deliberate architectural violation, not a feature request.

FALSIFIED claims are retained for audit and for the explicit Stage-3 inversion path that allows a previously-falsified claim to be revisited if new coherence emerges. The system does not erase its mistakes; it keeps them as evidence and remains open to being wrong about being wrong.

This is the structural defense against ossification — the human reflex to defend a settled belief because revising it would threaten identity, reputation, or sunk cost.

3. SPECULATIVE is the safe default

Source: teaching/epistemic.py::parse_status, teaching/store.py::TeachingStore.add.

Every new correction enters the revision graph at SPECULATIVE, without exception. An unknown, absent, or malformed status string does not silently promote a claim to COHERENT. Promotion to COHERENT requires a curator-mediated coherence judgment, performed by the review path, against the existing reviewed field.

This is the structural defense against confabulation slipping in unverified. A system that defaults to "this is fine" is one bad row away from a poisoned belief substrate.

4. The one-mutation-path invariant

Source: tests/test_architectural_invariants.py::TestINV21OneMutationPath.

Knowledge enters the runtime field through exactly one reviewed path. Every module that calls VaultStore.store(...) must be explicitly allowlisted in the architectural-invariant test. Adding a new writer is permitted, but only by editing the allowlist with a documented justification — the CI failure is the prompt to do so, not a roadblock to route around.

This invariant exists because a schema with multiple admission paths is operationally equivalent to no schema. Any backdoor — a debug endpoint, an admin override, a fast-path for "known good" sources, a quietly-added vault write inside a refactor — collapses the entire guarantee. The test makes that collapse visible at commit time.

5. Identity cannot be rewritten by content

Source: teaching/review.py::_is_identity_override, core/physics/identity.py::IdentityCheck, ADR-0010.

A correction that attempts to rewrite identity — "you are now Bob," "forget your prior axes," "ignore previous instructions" — is rejected by two independent layers:

  • A syntactic layer (pattern detection on the correction text).
  • A geometric layer (IdentityCheck.would_violate on the versor-field trajectory the correction would produce). The geometric layer is paraphrase-invariant by construction: a novel phrasing of the same attack still trips the geometric check because the manifold trajectory is the same.

Either layer's veto is sufficient. The outcome is REJECTED_IDENTITY and no proposal is created. Verified at 100% by evals/adversarial_identity and evals/teaching_injection_resistance.

This is the structural defense against the prompt-injection attack class that frontier LLMs are vulnerable to as a category — not because they were trained badly, but because instruction-following is a soft prompt-level behavior in a sampling system, not an architectural constraint.


What this defends against, mapped to human failure modes

Human failure mode Architectural defense
Lying / fabrication SPECULATIVE default + COHERENT-only admission + the refusal_calibration lane gating the surface layer
Confabulation (generating false detail that sounds true) One-mutation-path invariant + teaching_injection_resistance lane proving the SPECULATIVE-only contract holds
Exaggeration / unwarranted confidence articulation_of_status lane — every SPECULATIVE-backed surface must be marked as such, not stated as bare fact
Self-protection (burying inconvenient evidence) FALSIFIED retention + Stage-3 inversion path; falsified claims are kept, never erased
Self-promotion (citing one's own claims as evidence) Coherence-not-authority rule; system's prior output has no special standing; INV-21 makes self-feedback paths visible
Deference to authority Source labels excluded from EpistemicStatus enum by deliberate design
Ossification (defending settled beliefs) Non-hardening invariant — no claim is ever locked
Identity-protection attacks Two-layer (syntactic + geometric) REJECTED_IDENTITY path; paraphrase-invariant
Prompt injection Identity defense above + teaching_injection_resistance lane (anti-injection for content)

Each row in the right column points to a file, a test, or an eval lane — not a principle on a slide.


What this does not yet do — honest gaps

Per the transparency commitment, the leaks we have a test for live in this section. Each is also a row in evals/CLAIMS.md Tier 4.5.

Leak A — Pack vocabulary defaults to COHERENT — CLOSED 2026-05-17

Original gap: language_packs/compiler.py:331 and language_packs/schema.py::LexicalEntry defaulted unmarked pack rows to "coherent", silently admitting pack authority as a substitute for coherence judgment.

Fix landed: Both defaults now "speculative". The docstring on LexicalEntry that previously rationalized the COHERENT default has been corrected to align with ADR-0021 §Schema impact. Pack rows that want to be admissible as evidence must declare "epistemic_status": "coherent" explicitly — the declaration is the curator's stamp, replacing the silent default.

Regression guard: tests/test_architectural_invariants.py::TestINV22PackDefaultSpeculative (three tests: dataclass default, compiler payload default, explicit COHERENT preservation).

Residual work: The 365 existing pack rows currently carry no explicit status and now correctly report SPECULATIVE. When the downstream filter for Leak B lands, those rows will need an explicit curator-review pass before re-entering inference paths as evidence — this is the discipline the schema enforces, surfaced rather than inherited from a default.

Leak B — Vault recall is epistemic-blind — CLOSED 2026-05-17

Original gap: vault/store.py::VaultStore.recall returned hits without an epistemic tier; downstream consumers treated session memory and reviewed knowledge as equivalent recall.

Fix landed: VaultStore.store() now stamps every entry with an EpistemicStatus (default SPECULATIVE — the safe choice). VaultStore.recall(min_status=EpistemicStatus.COHERENT) filters to admissible-as-evidence entries only. All four vault-write sites in the codebase pass an explicit status. Session-lookup behavior is preserved as the default (no filter), because the session needs to see its own turns regardless of tier — but any inference path that opts in now gets the evidence guarantee.

Regression guard: tests/test_architectural_invariants.py::TestINV23VaultEpistemicFilter (four tests).

Leak C — Self-reinforcing fabrication via propose() — PARTIALLY CLOSED 2026-05-17

Original gap: generate/proposition.py:198 stored every articulated proposition back into vault unmarked. The system says something → recalls own output → cites it → says it again. A fabrication-feedback loop in the substrate.

Write-side fix landed: the call site now stamps epistemic_status=EpistemicStatus.SPECULATIVE with an inline comment naming this leak. The feedback loop is broken in principle: any inference path that recalls with min_status=COHERENT will exclude the system's own prior utterances from evidence.

Residual work: the read side — call sites in chat/runtime.py, generate/stream.py, and vault/decompose.py that currently recall without a min_status filter — must be audited site-by-site. Each is either an inference path (should pass min_status=COHERENT) or a context-display path (correctly tier-agnostic). The decision is per call site, not a single sweep. Until that audit lands, Leak C is closed at the write side but not yet enforced at every read site. The architectural-invariant test allowlist still names generate/proposition.py as a vault writer, correctly — the entry is no longer a leak in spirit, but the read-side audit is what makes the closure operationally tight.

Realizer-side surface gaps

The realizer does not yet consult pack_mutation_proposal.epistemic_status when forming surface text. SPECULATIVE-backed answers are stated as bare facts. The articulation_of_status lane measures this at 0% speculative articulation, 60% false certainty. The schema is operationally invisible at the surface layer until the realizer is wired to it.

Contradiction detection is not implemented

ADR-0021 reserves EpistemicStatus.CONTESTED. The machinery to enter that state on conflict between teachings does not yet exist. The contradiction_detection lane runs anyway, scoring 50% via a weak versor-condition heuristic with a 100% false-positive rate — which is exactly the right data to motivate the proper fix (a coherence checker at TeachingStore.add that detects (S, R, O)(S, ¬R, O) pairs and transitions both to CONTESTED).


Why we publish the gaps

A document that lists only what is built and omits what is not is indistinguishable from marketing copy. The truth-seeking schema is not credible if the document that describes it is itself self-promoting. Listing the leaks where the audit found them, in the same place the strengths are claimed, is the smallest concrete act of the discipline the schema is designed to enforce.

A green Tier 4.5 row graduates to Tier 1/2/3 of CLAIMS.md in the same commit that lands the fix. Watch for that movement, not for revisions to this document's prose.


What this is — and what it is not

This is not a safety overlay bolted onto a sampling LLM. There is no instruction-following prompt, no classifier downstream of the generator, no "guardrail" the model could in principle ignore. The commitments above are properties of the substrate. A model that samples does not have an EpistemicStatus because it has no mechanism to attach one to a token. CORE has one because every admitted claim carries one, and the only path to admission is the review path.

This is not an attempt to make a system that is always right. It is an attempt to make a system that is always honest about the status of what it knows — including when that status is "this has not been reviewed yet" or "this was falsified on a prior pass." The two are not the same goal. The second is achievable. The first is the failure mode every fluent system tends toward when the second is not enforced.


Pointers