core/docs/analysis/L10-shapeBplus-persistence-scope-2026-06-05.md

Shape B+ Persistence — Scope

Date: 2026-06-05 · Status: SCOPE (await go-ahead before code) · Parent: milestone-l10-continuity-spike (the acceptance oracle) · Telos: [project-core-is-one-continuous-life] — this is the T-resume half (make a reboot transparent), not the always-on process.

Goal (one sentence)

Make the lived session state — field, vault, anchor, graph, referents, dialogue blade — durable across reboot, bit-exactly, so that resuming from a checkpoint continues the same life instead of starting a fresh one sharing only recognizers/candidates.

The acceptance oracle is already built

The L10 continuity spike (evals/l10_continuity/, #567) gives us a precise, falsifiable target — we do not invent acceptance criteria:

• The flip: today evaluate_p2b_reboot_transparency(...).post_reboot_transparent == False and test_p2b_documents_current_resume_gap pins that. Shape B+ makes it True. The build is done when that test flips and we invert its assertion to assert transparency (P2b becomes a green guard: a reboot is byte-identical to no-reboot).
• The guards that must stay green prove the round-trip is honest:
  • P1 closure — versor_condition < 1e-6 must survive serialize→deserialize (the #1 risk: float round-trip must be bit-exact or closure breaks).
  • P2a determinism — restored state must yield identical trace_hash.
  • P4 crash recovery — recovery from a session-state checkpoint stays deterministic + atomic (ADR-0156).
  • P5b/P5c — anchor/coherence behavior unchanged across reboot.
• The TDD loop writes itself: restore SessionContext fields until P2b is transparent. If a field is load-bearing and we miss it, P2b stays False and tells us exactly that. The spike drives the scope.

Lived state to persist (verified inventory)

ChatRuntime._context is a SessionContext assigned once and never restored (chat/runtime.py:608). Everything it holds is pure data (no locks / files / RNG / threads — verified), so the whole object is serializable:

Component	Fields	Serialization note
state: FieldState	F (32-vec), node, step, holonomy (opt array), energy (EnergyProfile), valence (ValenceBundle)	arrays bit-exact; energy/valence are frozen dataclasses of primitives
vault: VaultStore	_versors (deque of arrays), _metadata (deque of dicts), _store_count, _reproject_interval, _max_entries	the bulk; _exact_index + _matrix_cache are DERIVED → rebuild on load, do not persist
_anchor_field	32-vec array	bit-exact; load-bearing for _session_anchor_pull
graph: SessionGraph	_nodes: list[TurnNode] (turn_idx, input/output versors, tokens, role, slots, edges)	per-node arrays bit-exact
referents: ReferentRegistry	_slots, _history (ReferentEntry: surface/versor/turn/slot), _last_resolved_*	per-entry versor bit-exact
running_dialogue_blade	array | None	bit-exact
scalars/caches	turn, _last_input_tokens, _last_resolved_input_tokens, _last_input_versor, _last_response_tokens, _dialogue_history_compat	restore for safety; P2b reveals which are load-bearing

Architecture

• Per-component to_dict/from_dict mirroring the existing Shape-B convention (DerivedRecognizer.to_json, DiscoveryCandidate.as_dict/from_dict): FieldState, VaultStore, SessionGraph, ReferentRegistry (+ the small nested types). Compose into SessionContext.snapshot() -> dict / SessionContext.restore(dict).
• Exact array codec — arrays serialize as {dtype, shape, b64(raw bytes)} (NOT decimal JSON — decimal truncates and breaks both closure and determinism; this is the Cl(4,1) float-truncation pitfall). Preserve float32 vs float64 exactly.
• Storage — extend engine_state/ with a session_state.json written via the existing _atomic_write_text (ADR-0156), and bump _SCHEMA_VERSION 1→2. The #563 migration discipline already makes this backward-compatible: a v1 checkpoint (no session_state) loads gracefully into a fresh session — i.e. exactly today's behavior — so old checkpoints don't break.
• Wiring — ChatRuntime.checkpoint_engine_state() saves the snapshot; _load_engine_state() restores it into self._context.

Phases (entry → exit gates)

Phase	Build	Exit gate (CLI/test)
A. Exact array codec + FieldState	b64(dtype,shape,bytes) codec; FieldState.to_dict/from_dict	bit-exact round-trip test + closure preserved (versor_condition identical post-restore)
B. VaultStore (de)serialize	snapshot versors+metadata+store_count; rebuild index/cache on load; NO reproject/normalize on load	round-trip test: recall(q) returns identical results post-restore (exact CGA recall preserved); bright-line review that load calls no normalization
C. Graph + referents + anchor + blade → SessionContext.snapshot/restore	compose all per-component codecs	SessionContext snapshot→restore round-trip is object-equal
D. Wire into engine_state (schema v2) + ChatRuntime	save in checkpoint, restore in load; v1 back-compat	a v1 checkpoint still loads (fresh session); a v2 checkpoint restores the context
E. Flip the oracle	invert test_p2b_documents_current_resume_gap to assert transparency	P2b transparent + P1/P2a/P4/P5 still green + the spike deterministic_digest stable; python -m evals.l10_continuity all gates pass incl. reboot transparency

Risks (surfaced up front)

1. Bit-exact float round-trip (closure + determinism). The whole flip depends on it. De-risk in Phase A with a base64 raw-bytes codec; never decimal.
2. vault/store.py is a CLAUDE.md forbidden normalization site. Adding pure (de)serialization is allowed, but the load path must not reproject, re-normalize, or repair — it restores raw persisted versors and rebuilds only the derived index/cache. Needs an explicit bright-line review and ideally an architectural-invariant assertion that from_dict invokes no normalizer.
3. Reproject interaction. reproject() rewrites all versors every N stores, so the vault content changes at reproject boundaries. A full snapshot captures post-reproject state correctly; an append-only log would diverge. Start with full snapshot (correct); optimize to incremental only if Phase E / P3 shows the I/O cost matters.
4. Persistence cost vs P3. Full-snapshot each turn is O(n) I/O → O(n²) over a session. Measure with the spike's P3 + a longer soak; if it breaches, move to snapshot-at-reproject + append-between. Correctness first.
5. trace_hash depends on vault_hits + recall surface. This is why the flip works (restored vault → identical recall → identical hash) and why the round-trip must be exact. Phase B's exact-recall test de-risks it.
6. Teaching-safety boundary. This persists session memory (immediate per CLAUDE.md), NOT reviewed/ratified memory — it is continuity, not a learning path. Do not let it become a parallel correction/teaching channel.

Scope boundary (what this is NOT)

• NOT the always-on process / idle-tick runtime (the other L10 half) — this is resume-transparency only.
• NOT cross-reboot identity-lineage work (EngineIdentity / lineage chain) — that builds on top of a transparent resume.
• NOT persisting reviewed/ratified memory (already the teaching path).

Proposed PR shape

One stacked sequence, each PR green on its own exit gate: A (codec+FieldState) → B (vault) → C (context compose) → D (engine_state wiring) → E (flip the oracle). A–C are additive + unit-tested in isolation (fast); D wires; E is the soak-lane flip. E is the load-bearing PR — it's the one that proves resume-as-same-life.