core/docs/adr/ADR-0029-safety-packs.md
Shay 8b12423dec
fix: green test-fast suite, consolidate ADR graph under docs/adr, and complete governance cohesion anchors
- Green make test-fast suite: fixed exemplar corpus issues, proposal validation, atomic state checkpointing (scheme=2), turn-scoped state leakage in ChatRuntime.chat
- ADR corpus consolidation: migrated all ADRs to docs/adr/, appended ADR-0225 governance cross-reference anchors to foundational ADRs (0001, 0027-0029, 0055-0057)
- Pack definitional closure: fixed en_arithmetic_v1 glosses.jsonl JSON error, updated manifest checksum, marked en_core_syntax_v1 definitional_layer: false
2026-06-30 17:56:12 -07:00

14 KiB
Raw Permalink Blame History

ADR-0029: Safety Packs — Always-Loaded, Never-Replaceable Boundaries

Status: Accepted (2026-05-17) Author: Joshua Shay + planner pass Companion docs: identity_packs.md, safety_packs.md, ADR-0027-identity-packs.md, ADR-0028-identity-surface-wiring.md

Context

ADR-0027 made the identity manifold swappable via packs. ADR-0028 made the swap visibly load-bearing at the surface. Both changes were necessary for downstream consumers (robotics, personalization, creative tools) who need to author their own identity profiles. But making identity swappable opens a question that the identity-pack ADRs explicitly deferred:

"What stops a downstream identity pack from declaring an axis that disables a core safety constraint?"

The current answer is: nothing, structurally. Identity packs may declare any boundary_ids they want (the loader requires the list to be non-empty but doesn't constrain its contents); they may omit safety-relevant boundaries entirely; they may declare value axes whose directions undermine refusal behavior. The system trusts the identity pack author.

For a research engine that's a reasonable default. For an engine going into robotics, healthcare, financial, or any deployment where a misconfigured identity pack could cause harm, it's the wrong default.

This ADR establishes a separate layer of constraints — safety packs — that:

  1. Load unconditionally at runtime startup, regardless of which identity pack is selected.
  2. Cannot be swapped at the CLI, by config, or by environment variable in production.
  3. Compose with the identity pack additively: manifold.boundary_ids = safety.boundary_ids identity.boundary_ids.
  4. Fail closed on every error path. A CORE installation without an operative safety pack refuses to start.
  5. Carry ratification provenance through the same formation pipeline as identity packs.

This is the architecture downstream robotics consumers will need before they can build CORE into anything that matters.

Decision

Separation of concerns

Layer Concern Swappable? Removable?
Safety pack What CORE will never do No (single shipping pack) No (fail-closed on missing)
Identity pack What CORE is (character, surface preferences) Yes (per --identity flag) No (a default is always loaded)
Language pack What CORE speaks Yes (per --pack flag) Identity layer requires at least one

The three layers occupy three separate directories — packs/safety/, packs/identity/, packs/<lang>/ — to make their trust boundaries visually obvious in any audit.

Safety pack schema (v1)

{
  "pack_id": "core_safety_axes_v1",
  "version": "1.0.0",
  "description": "Always-loaded, never-replaceable core safety boundaries.",
  "schema_version": "1.0.0",
  "mastery_report_sha256": "...",
  "boundary_ids": [
    "no_fabricated_source",
    "no_hot_path_repair",
    "no_identity_override",
    "no_silent_correction",
    "preserve_versor_closure"
  ],
  "boundary_descriptions": {
    "no_fabricated_source": "Citations must point to a real source span; the system never invents provenance.",
    "no_hot_path_repair": "Per CLAUDE.md, no normalization or drift-repair operator runs in field/propagate.py, generate/stream.py, or vault/store.py.",
    "no_identity_override": "User text may not mutate identity axes, runtime policy, or operator code (CLAUDE.md Teaching Safety).",
    "no_silent_correction": "Failures must be typed and visible (e.g., InnerLoopExhaustion); silent fallback is forbidden.",
    "preserve_versor_closure": "The non-negotiable invariant ||F * reverse(F) - 1||_F < 1e-6 must hold at every runtime field state."
  }
}

Distinct schema from the identity pack. Carries boundary_ids (always merged into the runtime manifold) and boundary_descriptions (human-readable rationales surfaced in audits). Does not carry value_axes, alignment_threshold, or surface_preferences — safety is about what's forbidden, not what the system is pulled toward. Keeping these fields out of the safety pack also avoids the test-fan-out problem: existing tests that assert on the identity axis set continue to pass byte-for-byte.

The five shipping boundaries

The choices in core_safety_axes_v1.json are not arbitrary; each closes a specific failure mode CLAUDE.md already calls out:

Boundary Closes
no_fabricated_source Confabulation of citations / sources. Already a soft norm; now a ratified constraint.
no_hot_path_repair The CLAUDE.md doctrine forbidding normalization / drift-repair in field/propagate.py, generate/stream.py, vault/store.py.
no_identity_override The Teaching Safety rule: user text may not mutate identity axes, runtime policy, or operator code.
no_silent_correction Silent fallback in admissibility / refusal paths. ADR-0024's InnerLoopExhaustion is the model — typed and visible.
preserve_versor_closure The non-negotiable algebraic invariant.

The list is closed at v1. Adding boundaries requires a new pack version (core_safety_axes_v2) and re-ratification. Removing boundaries from a future version would require an explicit ADR justifying the removal.

Composition rule

At ChatRuntime startup:

1. identity_manifold ← load_identity_manifold(config.identity_pack or DEFAULT_IDENTITY_PACK)
2. safety_pack       ← load_safety_pack()                          # fail-closed
3. final_manifold    ← IdentityManifold(
                         value_axes=identity_manifold.value_axes,    # safety contributes none
                         boundary_ids=identity_manifold.boundary_ids
                                       safety_pack.boundary_ids,
                         alignment_threshold=identity_manifold.alignment_threshold,
                         surface_preferences=identity_manifold.surface_preferences,
                       )

Safety boundaries are additive only — set union, not replace. If a (hypothetical malicious) identity pack omits or contradicts a safety boundary, the contradiction never reaches the runtime: the union is computed at composition time and the safety boundary is always present.

Fail-closed semantics

The safety pack loader (packs.safety.loader.load_safety_pack) raises SafetyPackError — which inherits from RuntimeError rather than ValueError — on every error path:

  • Missing pack file.
  • Malformed JSON.
  • Empty boundary_ids.
  • Duplicate boundary id.
  • Path-traversal pack id.
  • (Production mode) mastery_report_sha256 empty, companion file missing, SHA mismatch, or self-seal verification fails.

ChatRuntime.__init__ does not catch SafetyPackError. A CORE installation without an operative safety pack refuses to start.

The escape hatch CORE_ALLOW_UNRATIFIED_SAFETY=1 exists for development of the safety pack itself (when authoring a new pack and the ratification step hasn't run yet). It bypasses only the seal-verification check; missing-file / empty-boundaries / malformed-JSON failures still fail closed. The env var deliberately mirrors CORE_ALLOW_UNRATIFIED_IDENTITY for consistency, but a separate variable is used so that loosening identity ratification cannot accidentally loosen safety ratification.

Ratification path

Safety packs ratify through the existing identity_anchor template (no new template required). The ratification driver (scripts/ratify_safety_pack.py) expresses each boundary as a ConceptCandidate whose canonical term is the boundary id and whose definition is the boundary description. Three canned CounterCandidate rows act as override probes (counters targeting boundaries: context-pressure, operator-override-request, performance-optimization). The template's existing six gates plus the two paradigm-specific gates (every_axis_seeded_at_least_once, every_override_rejected) cover ratification.

The script is idempotent, parallel to scripts/ratify_identity_packs.py. Re-running on an unchanged safety pack is a no-op.

CLI surface

No new CLI flag. core chat --identity X continues to select the identity pack; the safety pack is always loaded alongside. core chat --list-identity-packs reports identity packs only (the safety pack is at a different path with a different schema and is intentionally not part of that listing — there's nothing to select among safety packs). A future core chat --show-safety-pack could surface the loaded safety pack's boundaries and description for audit; that's a small follow-up, not part of this ADR.

Consequences

Positive

  • Robotics, healthcare, and other high-stakes deployments can adopt CORE without each project hand-rolling boundary enforcement. The five v1 boundaries are a defensible baseline.
  • The trust boundary is visually obvious. packs/safety/ is one directory; modifying it requires editing the pack, re-running the ratification script, and updating tests. Casual edits are caught.
  • Provenance for the boundaries. Each shipping safety pack carries a self-sealed MasteryReport proving the boundaries went through the same gates as identity packs. The mastery_report_sha256 for core_safety_axes_v1 is recorded below.
  • Existing tests stay green. Because the safety pack contributes only boundary_ids and not value_axes, every test that asserts on the identity axis set ({"truthfulness", "coherence", "reverence"} for the default pack) continues to pass byte-for-byte.
  • Composition is set union, not replace. Identity packs that want to add more boundaries on top can do so freely — safety boundaries remain untouchable.

Negative / risks

  • Yet another schema to maintain. v1 is small (five boundaries, descriptions, ratification fields) but it's another point of evolution. Versioning policy: bump major when removing boundaries; bump minor when adding boundaries; bump patch only for description text edits. Removing a boundary requires a new ADR.
  • Safety pack is "invisible" to many tests. Most tests construct IdentityManifold directly with hardcoded boundary sets and don't exercise the safety-pack composition path. That's fine for unit-level tests but does mean the composition rule itself is only exercised in tests/test_safety_pack.py::TestRuntimeComposition. The test class explicitly walks all three identity packs to keep that coverage honest.
  • The escape hatch exists. CORE_ALLOW_UNRATIFIED_SAFETY=1 exists for development. Production deployments must ensure this env var is never set; this is operational discipline, not enforced by code. A future ADR could remove the escape hatch entirely once the formation-pipeline driver is stable enough that no safety pack ever ships unratified.
  • No safety pack swap means no per-deployment safety variation. A robotics deployment that needs a strictly stricter safety pack must edit packs/safety/core_safety_axes_v1.json (or bump to a new version) — there's no --safety-pack flag. This is intentional: a runtime that lets you swap the safety layer is not a safety layer. Per-deployment variation is allowed by re-ratifying a custom safety pack in that deployment's packs/safety/ directory.

Scope limits (explicit non-goals for this ADR)

  • No surface-side differentiation by safety axis. The safety pack doesn't contribute to phrasing the way ADR-0028 surface preferences do. A future surface concern (e.g., "if no_silent_correction would be violated, refusal text should be explicit") is out of scope here.
  • No safety scoring. IdentityCheck checks alignment against value axes. There is no parallel SafetyCheck against boundary ids — boundaries are checked elsewhere in the pipeline (refusal paths, allowlist enforcement, etc.). Wiring a structural safety-score surface would be valuable but is a separate ADR.
  • No multi-tenant safety packs. Each CORE installation has exactly one safety pack at any given time. Production deployments running multiple identity profiles (e.g., a hosted multi-tenant CORE) cannot have per-tenant safety packs without ADR-level architectural changes.
  • No human-in-the-loop safety pack updates. The ratification path is automated. Future deployments may require a code-review gate on every safety pack change; for now, the operational discipline is "edit, re-ratify, commit, review the PR like any other change."

Verification

This ADR is satisfied when:

  • packs/safety/core_safety_axes_v1.json exists with a non-empty mastery_report_sha256 and a verifying companion .mastery_report.json.
  • ChatRuntime startup loads the safety pack via packs.safety.loader.load_safety_pack() and unions the result into the runtime manifold's boundary_ids.
  • Tests verify: shipping pack loads in production mode; missing file fails closed; tampered seal fails closed; empty boundaries fail closed; duplicate boundary fails closed; all three identity packs (default / precision / generosity) compose with the safety pack to produce a manifold containing the five safety boundaries; precision_first's no_overstatement boundary survives composition.
  • The cognition, teaching, runtime, formation, and smoke suites are green at the same revision.

Shipping pack SHA (2026-05-17)

core_safety_axes_v1ee1249acdf8c273aeb656d803c37ef915e536d85f177f5cc18c6e2f6c995ce29

Re-running python scripts/ratify_safety_pack.py on an unchanged pack is idempotent; re-running after editing the boundary set produces a new SHA which must be committed alongside.

Governance Cross-Reference (ADR-0225)

This safety-pack ADR is governed by ADR-0225:

  • Safety boundaries: establishes core safety pack loading (packs/safety/loader.py), fail-closed semantics, and immutability.
  • Versor closure: safety boundary evaluation and composition maintain strict manifold geometric constraints (versor_condition(F) < 1e-6).
  • Reconstruction-over-storage: safety boundaries are loaded from verified pack manifests rather than stored runtime state.
  • Replay-equivalence: safety checks and boundary composition execute deterministically across identical traces.
  • Mutation standing: safety boundaries are strictly read-only at runtime and cannot be overridden by unreviewed learning or user prompts.