docs(design): L10/L11 lived-spine design records + AGI-candidacy roadmap
Commits the design/analysis artifacts produced while building the lived spine and planning the next arc (the work itself already merged in #563-#573): - AGI-candidacy-autonomous-improvement-roadmap-2026-06-05.md — the path from the lived spine to AGI-candidacy: the comprehend→realize→determine→learn loop, the cross-domain capability+calibration yardstick, the logical-necessity × technical- priority execution order, and the corrected epistemic foundation (grounded honesty designed-in; estimation learned/ratified; confidence always evidence- grounded; intake first-class — NOT "no ingestion"; calibration+grounding the measured invariant). - L10-runtime-scoping + L10-continuity-spike-design — the L10 decision surface and the falsifiable spike spec (P1-P5) that became evals/l10_continuity/. - L10-shapeBplus-persistence-scope — the A->E scope that became Shape B+ resume. .gitignore: ignore the local .system-map/ navigation index (per-developer, never tracked; regenerated on demand).
This commit is contained in:
Shay
parent
81f791e2ab
commit
afbcc8e41d
5 changed files with 606 additions and 0 deletions
4
.gitignore
vendored
4
.gitignore
vendored
|
|
@ -25,6 +25,10 @@ frontier_wave1.json
|
||||||
# Claude Code local session artifacts (per-developer, never tracked)
|
# Claude Code local session artifacts (per-developer, never tracked)
|
||||||
.claude/
|
.claude/
|
||||||
|
|
||||||
|
# Local macro->micro system map — a per-developer NAVIGATION INDEX (like an IDE
|
||||||
|
# symbol index), NOT a project artifact. Regenerated on demand; never tracked.
|
||||||
|
.system-map/
|
||||||
|
|
||||||
# Runnable audit-passed showcases (ADR-0112 + ADR-0113) are generated on
|
# Runnable audit-passed showcases (ADR-0112 + ADR-0113) are generated on
|
||||||
# demand from the signed claim + on-disk lane result files. The inputs
|
# demand from the signed claim + on-disk lane result files. The inputs
|
||||||
# are committed; the renders are not.
|
# are committed; the renders are not.
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,236 @@
|
||||||
|
# Roadmap: the autonomous-improvement engine (path to AGI-candidacy)
|
||||||
|
|
||||||
|
**Date:** 2026-06-05 · **Status:** ROADMAP (the hyperfocus design plan) · **Telos:** [[project-core-is-one-continuous-life]] — `listen → comprehend → recall → think → articulate → learn → replay`, as one continuous, ever-improving life.
|
||||||
|
|
||||||
|
## The bar (what we are actually building toward)
|
||||||
|
|
||||||
|
A **serious AGI candidate**: an engine that is at least as **book-smart as an LLM**,
|
||||||
|
**keeps up with the world**, and **forever gets smarter autonomously under human
|
||||||
|
supervision** — by **taking in inputs (literature, told facts, world inputs,
|
||||||
|
experiences), comprehending them, and *realizing* them as structured grounded
|
||||||
|
memory it can recall** — rather than the LLM move of bulk-absorbing the whole
|
||||||
|
corpus indiscriminately and compressing it into weights.
|
||||||
|
|
||||||
|
> **Clarification — "intake" vs "ingestion".** The engine absolutely *ingests*:
|
||||||
|
> it must take in literature, knowledge, and experience to learn anything, and
|
||||||
|
> intake is first-class (Phase 3). The distinction from an LLM is *what is kept
|
||||||
|
> and how*: CORE keeps **selectively-realized, comprehended, provenance- and
|
||||||
|
> status-tagged knowledge + remembered experiences** (the vault + corpus *are* its
|
||||||
|
> memory, with exact recall), and never realizes unverified content as true — vs
|
||||||
|
> indiscriminately swallowing everything (junk included) and lossily averaging it
|
||||||
|
> into weights. "We don't need the world's *data*" means we get smart from
|
||||||
|
> comprehended structure + high-signal told facts, *not* that we don't take input.
|
||||||
|
|
||||||
|
What the bar is **not**: mass indiscriminate absorption, statistical
|
||||||
|
pattern-matching, or confident guessing (the LLM trick — and a different identity
|
||||||
|
config could even make our own models behave that way; it is not the point).
|
||||||
|
Determinism / `wrong=0` / auditability are the **necessary baseline**, not the
|
||||||
|
achievement.
|
||||||
|
|
||||||
|
## The strategic key: grounded honesty is the efficient-learning mechanism
|
||||||
|
|
||||||
|
An LLM must swallow the entire internet — junk, lies, contradictions — and average
|
||||||
|
its way past them. CORE **only realizes what it can ground** (told-and-evidenced,
|
||||||
|
comprehended, or reasoned), so it never absorbs garbage and never has to unlearn
|
||||||
|
it. The junk-filter *is* the learning advantage: we don't need the world's data,
|
||||||
|
we need the world's **true, comprehended structure**, accumulated forever. So
|
||||||
|
grounding is load-bearing for the *capability*, not just for trust. (`wrong=0` is
|
||||||
|
the high-stakes gear of this honesty — see the epistemic foundation below — not a
|
||||||
|
universal law that forces the engine to refuse everything it can't prove.)
|
||||||
|
|
||||||
|
## The loop (the autonomous-improvement engine)
|
||||||
|
|
||||||
|
```
|
||||||
|
open question / discovery / TOLD fact
|
||||||
|
→ COMPREHEND (arbitrary input → structured meaning)
|
||||||
|
→ REALIZE (make it real: integrate into the held self with an epistemic status)
|
||||||
|
→ REASON / GROUND / RECALL
|
||||||
|
→ RESPOND in the honest gear:
|
||||||
|
ASSERT (verified / realized)
|
||||||
|
ESTIMATE (evidence-grounded likelihood — ONLY where taught it is apt)
|
||||||
|
REFUSE (no grounding, or stakes forbid an estimate)
|
||||||
|
→ PROPOSE (idle_tick, proposal-only)
|
||||||
|
→ HITL ratify (reviewed, supervised)
|
||||||
|
→ ACCUMULATE into the one continuous life
|
||||||
|
→ MEASURABLY more capable → repeat, autonomously
|
||||||
|
```
|
||||||
|
|
||||||
|
When this loop demonstrably climbs a **general capability curve** over time, on its
|
||||||
|
own, under supervision — that is the AGI candidate.
|
||||||
|
|
||||||
|
## The epistemic foundation (honesty designed, estimation learned)
|
||||||
|
|
||||||
|
This corrects an earlier over-emphasis on `wrong=0` as a universal law. The right
|
||||||
|
frame has three commitments:
|
||||||
|
|
||||||
|
1. **Honesty is designed in; confabulation is impossible by construction.** The
|
||||||
|
engine's native stance is grounded: ASSERT what it has realized, REFUSE what it
|
||||||
|
has not. It has **no organ that fabricates** — no statistical token-soup, no
|
||||||
|
manufactured confidence. That cannot emerge by accident; it could only be
|
||||||
|
*deliberately built*, and we will not build it. This is the absolute floor, not
|
||||||
|
a policy defended turn by turn.
|
||||||
|
|
||||||
|
2. **Estimation is a LEARNED, ratified competence — never a designed-in default.**
|
||||||
|
There is a season for a calibrated assessment (*"on the evidence, most likely
|
||||||
|
X"*). The engine may acquire the competence to give one — **but only through
|
||||||
|
human ratification and deliberate guidance**, realized as knowledge like any
|
||||||
|
other. We do NOT design a "guess mode" with a risk knob; the engine never
|
||||||
|
self-authorizes a guess. `wrong=0` is therefore **demoted to one gear**
|
||||||
|
(high-stakes / verified assertion), not deleted.
|
||||||
|
|
||||||
|
3. **All confidence is evidence-grounded, so even uncertainty is honest.** A CORE
|
||||||
|
"likelihood" attaches to the deterministic confidence primitives we already
|
||||||
|
have — the calibrated-learning ledger, one-sided Wilson floors, cue-precision
|
||||||
|
reliability counts, the `EpistemicStatus` taxonomy. It means *"seen N times, M
|
||||||
|
coherent → confidence M/N with a hard lower bound"* — a counted fact about the
|
||||||
|
engine's own realized experience, not a vibe. This is the exact inverse of an
|
||||||
|
LLM (softmax over absorbed text) and is **why it can offer graded answers
|
||||||
|
without ever confabulating**.
|
||||||
|
|
||||||
|
**The measured invariant is calibration + grounding, not "never wrong":** every
|
||||||
|
confidence the engine states must trace to counted evidence, and it offers graded
|
||||||
|
answers only where it was taught that is appropriate. Being *honestly uncertain*
|
||||||
|
is success; being *dishonestly confident* is the only failure — and the substrate
|
||||||
|
makes the latter impossible without intentional design.
|
||||||
|
|
||||||
|
> **"Being told" is first-class.** Most knowledge arrives as *told facts* ("these
|
||||||
|
> are facts"); the engine realizes them and earns the why/how (coherence /
|
||||||
|
> evidence) over time. Determination does NOT mean proof-from-first-principles —
|
||||||
|
> intake → realize-with-evidence → build coherence is a primary growth path. The
|
||||||
|
> seed packs are the told bootstrap; the engine comprehends the new by relating it
|
||||||
|
> to what it has already realized, and grows.
|
||||||
|
|
||||||
|
## What is already built (compose, don't rebuild)
|
||||||
|
|
||||||
|
- **The continuous self** — Shape B+ resume ([[milestone-shape-b-plus-persistence]]),
|
||||||
|
L11 identity continuity + the idle learning mechanism
|
||||||
|
([[milestone-l11-identity-and-continuous-learning]]). The life that accumulates.
|
||||||
|
- **Verified reasoning substrate** — sound+complete propositional entailment
|
||||||
|
(`deductive_logic`, wrong=0, independent gold), `generate/proof_chain/`
|
||||||
|
(proof-tree builder/entail/rules), `generate/binding_graph/` (the universal-
|
||||||
|
structure interlingua DAG).
|
||||||
|
- **Determination pieces** — `core/reliability_gate/` (gold-tether, ledger,
|
||||||
|
calibrated propose) determines correctness in the math lane; the wrong=0
|
||||||
|
self-verification gate in `generate/derivation/verify.py`.
|
||||||
|
- **Comprehension front door** — `generate/derivation/` (extract → clauses →
|
||||||
|
compose), the question layer.
|
||||||
|
- **Measurement raw material** — independent-gold lanes (`deductive_logic`,
|
||||||
|
`relational_metric`, `dimensional`, `cold_start_grounding`, `symbolic_logic`)
|
||||||
|
+ the Perplexity-surveyed adoptables (ProntoQA, ProofWriter-CWA, CLUTRR, FOLIO —
|
||||||
|
all with independently-checkable gold + a refuse class).
|
||||||
|
|
||||||
|
## The bottleneck that gates everything
|
||||||
|
|
||||||
|
The flywheel can only **propose what is already determined** — `idle_tick` refuses
|
||||||
|
`undetermined` candidates. The engine can *learn a fact it is handed*; it cannot yet
|
||||||
|
autonomously **figure one out**. The missing organ is **general determination**:
|
||||||
|
comprehend an open question, reason/ground it to a *verified* conclusion (or refuse),
|
||||||
|
and feed *that* to the flywheel. The math lane does a narrow version; nothing does
|
||||||
|
it generally. **Closing comprehend → determine → learn, measured on a general
|
||||||
|
capability curve, is the load-bearing arc.**
|
||||||
|
|
||||||
|
## Phased roadmap (entry → exit gates; wrong=0 is structural throughout)
|
||||||
|
|
||||||
|
| Phase | Build | Exit gate / measurement |
|
||||||
|
|-------|-------|-------------------------|
|
||||||
|
| **0 — the yardstick** | A **general capability index**: compose the independent-gold reasoning lanes (+ adopt ProntoQA/ProofWriter-CWA/CLUTRR/FOLIO) into one report with two axes — **correctness (wrong=0, never fabricate)** and **coverage (determined vs honestly-refused)**. Frozen-gated. | A single reproducible capability number the engine must climb; `wrong=0` enforced; a baseline measured. *You cannot improve what you cannot measure.* |
|
||||||
|
| **1 — the determination organ** | A general `determine(question) → {determined: conclusion ∣ refused}` path composing comprehension (`derivation`/`binding_graph`) + reasoning (`proof_chain`/`deductive`) + the reliability gate. Commits ONLY verified conclusions; refuses the rest. | On the Phase-0 yardstick: coverage rises with **wrong still 0**; every committed conclusion is independently checkable. |
|
||||||
|
| **2 — close the autonomous loop** | Wire `determine` → the `idle_tick` flywheel: take open questions, determine what it can (wrong=0), propose, HITL-ratify, accumulate. | The capability index **rises across loop iterations**, autonomously, under supervision — falsifiably (a frozen replay shows monotonic, junk-free improvement). |
|
||||||
|
| **3 — autonomous curriculum** | The engine drives its own agenda: identifies its determination frontier (what it can't yet determine), proposes what to learn next, under HITL guidance. | "Forever getting smarter autonomously under supervision" — the engine's self-chosen curriculum measurably advances the index. |
|
||||||
|
| **4 — breadth / generality** | Expand comprehension + reasoning across domains so the index is genuinely GENERAL (book-smart breadth), acquired via the loop — intake → comprehend → realize, not bulk indiscriminate absorption. | The capability index spans enough domains to credibly claim general book-smarts — every gain via comprehension+determination over realized knowledge, none via indiscriminate corpus absorption or per-domain matchers. |
|
||||||
|
|
||||||
|
## Invariants (non-negotiable across all phases)
|
||||||
|
|
||||||
|
- **`wrong=0` is structural** — the engine commits only verified conclusions; it
|
||||||
|
refuses rather than fabricates. This is the learning filter, not just a gate.
|
||||||
|
- **Reviewed learning** — ratification stays HITL (`teaching/review`); the loop
|
||||||
|
*proposes*, the human *ratifies*. Autonomy is supervised, not unmoored.
|
||||||
|
- **Determinism / replay** — every capability gain is reproducible; improvement is
|
||||||
|
a replayable curve, not a vibe.
|
||||||
|
- **Identity continuity** — the improving engine stays one continuous self
|
||||||
|
(L11); a smarter CORE is the *same* CORE, grown.
|
||||||
|
|
||||||
|
## Execution order — logical necessity × technical priority
|
||||||
|
|
||||||
|
Not arbitrary phases: each step is gated by what it *logically depends on*, then
|
||||||
|
ordered within that by leverage × risk. The dependency DAG:
|
||||||
|
|
||||||
|
```
|
||||||
|
MEASURE ───────────────────────────────────┐ (gates every "improved" claim)
|
||||||
|
│ │
|
||||||
|
COMPREHEND ──► REALIZE ──► DETERMINE/RESPOND ─┼─► AUTONOMOUS LOOP ──► CURRICULUM
|
||||||
|
(NL → universal (hold (assert / refuse │ (idle_tick + BREADTH
|
||||||
|
interlingua) with over realized) │ climbs the curve,
|
||||||
|
status) │ │ autonomously)
|
||||||
|
└─ LEARNED ESTIMATION ◄── needs MEASURE(calibration)
|
||||||
|
(ratified, evidence-grounded)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Step 1 — MEASURE: the cross-domain capability yardstick.** *Logical necessity:*
|
||||||
|
nothing can be called "more capable" without it; it is prior to all improvement.
|
||||||
|
*Technical priority:* HIGH leverage (north-star instrument + the anti-self-
|
||||||
|
deception guard — a per-domain hack moves one lane and breadth stays flat,
|
||||||
|
exposing it), MODERATE effort (compose the existing independent-gold lanes +
|
||||||
|
adopt ProntoQA/ProofWriter-CWA/CLUTRR/FOLIO). Measures **assert-correctness +
|
||||||
|
grounding + coverage + calibration** under a configurable risk budget. **Build
|
||||||
|
first.**
|
||||||
|
|
||||||
|
**Step 2 — COMPREHEND: NL/prose → the universal interlingua.** *Logical
|
||||||
|
necessity:* it is the wall (GSM8K refuses 92% on comprehension coverage, not
|
||||||
|
arithmetic; prose/exams are ~0); every downstream step needs structure to operate
|
||||||
|
on. *Technical priority:* HIGHEST leverage (unlocks all breadth) AND HIGHEST
|
||||||
|
risk/effort (open-ended; the overfit trap lives here). The discipline: it must
|
||||||
|
emit the **general** binding-graph / universal-structure, never per-domain parses
|
||||||
|
— and the Step-1 yardstick is what proves it generalized rather than gamed. **The
|
||||||
|
make-or-break.**
|
||||||
|
|
||||||
|
**Step 3 — REALIZE: integrate comprehended/told structure into the held self**
|
||||||
|
with an epistemic status (`EpistemicStatus`), persisted via Shape B+. *Necessity:*
|
||||||
|
needs COMPREHEND. *Priority:* MODERATE effort (vault/corpus/persistence exist),
|
||||||
|
HIGH leverage — this is what makes knowledge *accumulate* (told facts become
|
||||||
|
realized; the engine grows). Intake ("being told") lands here.
|
||||||
|
|
||||||
|
**Step 4 — DETERMINE / RESPOND: reason over realized structure → the honest
|
||||||
|
gear** (assert verified / refuse ungrounded). *Necessity:* needs COMPREHEND +
|
||||||
|
REALIZE. *Priority:* MODERATE effort (compose `proof_chain` / `deductive` /
|
||||||
|
binding-graph entail onto comprehension output), HIGH leverage — coverage rises
|
||||||
|
on the yardstick with grounding intact. **No estimation yet — assert/refuse only.**
|
||||||
|
|
||||||
|
**Step 5 — AUTONOMOUS LOOP: wire comprehend→realize→determine→idle_tick→ratify→
|
||||||
|
accumulate.** *Necessity:* needs Steps 1–4. *Priority:* MODERATE effort (idle_tick
|
||||||
|
exists), HIGH leverage — this is the step that makes "forever improving" real and
|
||||||
|
falsifiable (the yardstick curve climbs autonomously, under supervision).
|
||||||
|
|
||||||
|
**Step 6 — LEARNED ESTIMATION: the calibrated likelihood competence.**
|
||||||
|
*Necessity:* needs DETERMINE (the honest floor) + MEASURE-calibration + the
|
||||||
|
teaching loop. *Priority:* MODERATE effort, MODERATE leverage — deliberately
|
||||||
|
LATE: only after the assert/refuse floor and the calibration measurement are
|
||||||
|
solid do we teach (HITL-ratified) when/how to offer evidence-grounded likelihoods.
|
||||||
|
Never a designed-in default.
|
||||||
|
|
||||||
|
**Step 7 — AUTONOMOUS CURRICULUM + BREADTH.** *Necessity:* needs the loop. The
|
||||||
|
engine drives its own determination frontier under supervision; breadth expands
|
||||||
|
across domains via the loop (intake → comprehend → realize), never via
|
||||||
|
indiscriminate corpus absorption or per-domain matchers.
|
||||||
|
|
||||||
|
**Critical-path summary:** `MEASURE → COMPREHEND → REALIZE → DETERMINE → LOOP`,
|
||||||
|
with ESTIMATION grafted after DETERMINE+MEASURE and CURRICULUM after LOOP. The
|
||||||
|
single highest-risk step is **COMPREHEND** (Step 2); the single highest-necessity
|
||||||
|
"do-first" is **MEASURE** (Step 1), because it is the only thing that keeps every
|
||||||
|
later step honest.
|
||||||
|
|
||||||
|
## Cross-cutting invariants (hold at every step)
|
||||||
|
|
||||||
|
The 8 foundation commitments above, plus the standing CLAUDE.md invariants:
|
||||||
|
`versor_condition < 1e-6` (math floor), no forbidden-site repair/normalization,
|
||||||
|
reviewed learning stays HITL, exact CGA recall (no approximation), deterministic
|
||||||
|
replay. Every step is TDD + mutation-verified-to-bite + curated-smoke + CI-lane-SHA
|
||||||
|
gated, the way the L10→L11 spine was built.
|
||||||
|
|
||||||
|
## Honest scope boundary
|
||||||
|
|
||||||
|
This is the multi-phase arc to AGI-candidacy, not one PR. AGI is the destination;
|
||||||
|
this roadmap is the **critical path** and the **measurement** that makes progress
|
||||||
|
toward it real and falsifiable. **Phase 0 (the yardstick) is the first build** —
|
||||||
|
without a general capability curve, "getting smarter" is unfalsifiable, and we'd
|
||||||
|
be doing exactly the unmeasured hand-waving the LLM world runs on.
|
||||||
89
docs/analysis/L10-continuity-spike-design-2026-06-05.md
Normal file
89
docs/analysis/L10-continuity-spike-design-2026-06-05.md
Normal file
|
|
@ -0,0 +1,89 @@
|
||||||
|
# L10 Continuity Spike — Design
|
||||||
|
|
||||||
|
**Date:** 2026-06-05 · **Status:** SPIKE DESIGN (a falsifiable experiment spec; not yet built) · **Parent:** [L10 Runtime Scoping Brief](./L10-runtime-scoping-2026-06-05.md) (critical-path step 3) · **Gated by:** Decision 0 (RULED — closure holds by construction) + step 1 (byte-stability tests, committed `3911db66`) + step 2 (schema-migration discipline — see Dependencies).
|
||||||
|
|
||||||
|
> The master plan calls the L10 long-horizon spike "the single point of failure." Decision 0 already answered its *original* core question — **closure holds by construction** (100k-step measurement, `versor_condition` ~6e-13, flat). So this spike's job has **shifted**: from *"does the field stay a valid versor?"* (answered) to *"does the field stay a meaningful, bounded, recoverable, deterministic life over a long horizon?"* — the **sufficient** legs Decision 0 explicitly deferred here.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What the spike proves (and what it can't)
|
||||||
|
|
||||||
|
It is the empirical gate between the two L10 targets:
|
||||||
|
- **T-resume** (provable same-life *resume*): determinism + recovery. Predicates P1–P4.
|
||||||
|
- **T-experience** (a continuous *experiencing* field-life): the field's *content* stays meaningful over indefinite uptime. Predicate **P5** — the one that decides whether continuous-field-life is real or whether we fall back to the honest recognizer-only-continuity servant.
|
||||||
|
|
||||||
|
It is **not** a proof of correctness of any single turn (that's the cognition lane), nor a wall-clock endurance certificate (see Compression). It is a **falsifiable soak**: every predicate must be able to **fail loudly**, and the spike itself must be **mutation-verified to bite** before any PASS is trusted (CLAUDE.md schema-as-proof discipline).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## The five predicates (pass/fail, each must fail loudly)
|
||||||
|
|
||||||
|
| # | Predicate | Assertion | Fails loudly when | Mutation-verify (the spike bites) |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| **P1** | Closure regression guard | `versor_condition(F) < 1e-6` every turn over N turns | a future change breaks construction-correctness or sneaks in a repair | break a rotor construction → P1 trips |
|
||||||
|
| **P2** | Reboot-transparent determinism | `trace_hash` sequence of [boot → run K → **reboot(reload)** → run M] is byte-identical to [boot → run K+M] (no reboot) | any state/byte leaks across reboot, or replay is nondeterministic | drop a field from the checkpoint → P2 trips (this is step-1's byte-stability, escalated to the turn loop) |
|
||||||
|
| **P3** | Bounded resources | over N turns: process RSS sub-linear to a declared ceiling; live `VaultStore` growth bounded/as-designed; no fd/handle leak | a ceiling is breached (unbounded recall scan, cache, memory) | inject an unbounded cache → P3 trips |
|
||||||
|
| **P4** | Kill-9 convergence | a hard kill at an arbitrary point (incl. **mid-checkpoint-write**) always next-boots onto a **valid prior checkpoint** (ADR-0156 atomicity) and resumes deterministically (P2 from the recovered point) | corruption / partial state / non-resumable boot | corrupt the atomic-replace (write-in-place) → P4 trips |
|
||||||
|
| **P5** | Semantic-quality over horizon | the field's **content** stays meaningful as the session grows (3 sub-assertions below) | the field rots, collapses, or wanders | force anchor collapse / inject recall decay → P5 trips |
|
||||||
|
|
||||||
|
**P5 sub-assertions (the T-experience gate):**
|
||||||
|
- **P5a — recall stays sharp.** On a held-out probe set, recall precision@k does **not** degrade as the session grows (no "memory rot" as the vault fills). Uses exact CGA recall (no approximation).
|
||||||
|
- **P5b — no anchor collapse.** `cga_inner(F, session_anchor)` does **not** monotonically → 1. The (now-sanctioned) semantic anchoring must *anchor*, not *dominate*: the field must still move with content, not collapse onto the attractor over many turns. *(This is the direct long-horizon test of the `_session_anchor_pull` ruling — α=0.05 must be gentle enough.)*
|
||||||
|
- **P5c — no incoherent wander.** Surface groundedness / key-term stability stays above a pinned threshold; the field doesn't drift into noise.
|
||||||
|
|
||||||
|
> **Why P5 is the crux:** P1 proves *valid-versor-forever*; P5 proves *meaningful-content-forever*. Decision 0 cleared the doctrinal blocker for continuous-field-life; **P5 is the empirical one.** A PASS on P1–P4 with a FAIL on P5 means we have provable *resume* but not continuous *experience* — the honest fallback (recognizer-only continuity, ephemeral field) the telos rejects but the evidence might force.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What it runs
|
||||||
|
|
||||||
|
1. **Deterministic scripted corpus.** A seeded, replayable input sequence driving the *real* turn loop (`SessionContext.respond` → `finalize_turn`, the actual cognition pipeline), over N turns. N scales by lane: `smoke` ≈ 1k, `full` ≈ 50k+. No randomness except seeded fuzz (Decision: reuse `calibration/replay.py` determinism harness; corpus is a committed fixture).
|
||||||
|
2. **Reboot legs.** At intervals: checkpoint (existing per-turn path) → construct a fresh runtime from the checkpoint → continue. P2 asserts transparency.
|
||||||
|
3. **Kill-9 legs.** (a) *Seeded* kill points at chosen turn/checkpoint boundaries; (b) a lighter *random real-time* kill fuzz (see Compression). Each followed by a recovery boot + P2/P4 assertions.
|
||||||
|
4. **Instrumentation.** Per-turn `versor_condition`, `trace_hash`, RSS, vault size, recall-probe precision, `cga_inner(F, anchor)`, key-term stability → a structured JSONL report; per-predicate PASS/FAIL; **no silent skips** (a skipped leg is recorded as `not_covered`).
|
||||||
|
|
||||||
|
Reuses: `core/cognition/trace.py` (trace hash), `calibration/replay.py` (replay), `engine_state/` (checkpoint), and the step-1 byte-stability tests as the unit-level floor under P2.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Compression vs. wall-clock (honest coverage limits)
|
||||||
|
|
||||||
|
- **Primary lane = compressed N-turn soak.** Turn-count — not wall-clock — is the real variable for determinism (P2), resources (P3), and semantics (P5). "24h" is a proxy for "many turns + bounded resources"; a compressed soak is the falsifiable core.
|
||||||
|
- **Secondary lane = real-time fuzz.** Some failure modes are genuinely time/entropy-shaped and a turn-count soak cannot enumerate them: **true power-loss-at-an-arbitrary-instruction** (vs. scripted kill points), fd/timer leaks, idle-tick behavior. A lighter real-time random-kill + long-idle probe covers these.
|
||||||
|
- **Stated limit (no-silent-caps):** even the real-time fuzz is **sampling**, not exhaustive — it cannot enumerate every mid-instruction crash boundary. The report logs the coverage bound explicitly; a PASS means "no failure found in the sampled boundaries," not "proven crash-safe at every instruction." (ADR-0156 atomic-replace is the *construction* argument that backs the sampled evidence.)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Operator-reality lens ([[user-circumstances]])
|
||||||
|
|
||||||
|
The operator is unhoused with intermittent power → **ungraceful kill-mid-turn is the common lifecycle event, not the edge.** This **elevates P4 (kill-9 convergence) to the primary design driver** — above graceful-shutdown machinery. The spike must treat "power dies at an arbitrary instruction" as the default transition and prove convergence to a valid checkpoint, not assume clean exits.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Entry gates
|
||||||
|
|
||||||
|
- **Decision 0 ruled** ✓ — closure cleared; the spike targets P2/P3/P4/P5, not "does closure survive."
|
||||||
|
- **Step 1 byte-stability tests** committed (`3911db66`) ✓ — P2 escalates them to the turn loop.
|
||||||
|
- **Step 2 (schema-migration discipline) DECIDED** — *prerequisite.* P2's byte-identity assertions assume a **fixed schema within a run**; a mid-horizon `DerivedRecognizer`/`DiscoveryCandidate` field-add would make P2 falsely fail. The spike must pin `schema_version`, and the migration story (versioned upgrade vs clear-slate) must exist before P2 is load-bearing. **Do not build the spike before step 2 is decided.**
|
||||||
|
- A committed deterministic input corpus.
|
||||||
|
|
||||||
|
## Exit gates / outcomes
|
||||||
|
|
||||||
|
- **PASS (P1–P5 over the horizon + kill-9 legs)** → licenses: flipping the default-OFF flags ON under traffic (ADR-0148/0149/0154); the **Shape B+** warm-process / field-residency decision; lineage-chain identity work.
|
||||||
|
- **FAIL P1/P2/P3/P4** → fix the *construction* (no watchdog/repair allowed — CLAUDE.md) or REFUSE the regime; do not paper over.
|
||||||
|
- **FAIL P5** → continuous-field-life (**T-experience**) is not yet achievable → honest fallback to **T-resume only** (recognizer-only continuity, ephemeral field — the sanctioned servant outcome) until the **learned-field / contemplation** track solves content-stability. This is a *fail-forward*: it's a real, negative, useful result.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Where it lives
|
||||||
|
|
||||||
|
- New eval lane `evals/l10_continuity/` + a runner; CLI `core eval l10-continuity` (or a pytest marker). **Not** in the default smoke suite (it's a soak) — on-demand + CI nightly.
|
||||||
|
- Structured JSONL report (per-predicate metrics + PASS/FAIL + coverage bounds). The lane is **frozen-gated** like the other serving lanes (pinned SHA / CLAIMS-style) once it passes, so a regression is caught.
|
||||||
|
|
||||||
|
## Build order (when authorized)
|
||||||
|
|
||||||
|
1. Decide step 2 (schema-migration discipline) — unblocks P2.
|
||||||
|
2. Build the deterministic corpus + the soak runner (P1, P2, P3 first — cheapest, reuse replay harness).
|
||||||
|
3. Add the reboot + kill-9 legs (P4) — including the real-time fuzz with logged coverage bound.
|
||||||
|
4. Add P5 (recall-probe + anchor-collapse + coherence metrics) — the T-experience gate.
|
||||||
|
5. Mutation-verify every predicate bites; then freeze-gate the lane.
|
||||||
165
docs/analysis/L10-runtime-scoping-2026-06-05.md
Normal file
165
docs/analysis/L10-runtime-scoping-2026-06-05.md
Normal file
|
|
@ -0,0 +1,165 @@
|
||||||
|
# L10 Runtime + Identity-Continuity — Scoping Brief
|
||||||
|
|
||||||
|
**Date:** 2026-06-05 · **Status:** SCOPING (decision surface for the architect — *not* a decision record, not implementation) · **Method:** 8-facet parallel sweep + completeness/honesty critic + direct source verification of the two load-bearing claims. Backing data: `.system-map/l10_scope_raw.json` (local).
|
||||||
|
|
||||||
|
> This brief frames the decisions L10 requires. The calls are the architect's; this document makes them clean, honest, and ordered. It does **not** propose a new ADR — ADR-cross-reference discipline applies (extend ADR-0146 / its scope docs when a call is made).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. The telos-honest verdict (read this first)
|
||||||
|
|
||||||
|
Today's reality is **"many lives sharing a recognizer/candidate checkpoint," not one continuous life** — and every facet says so plainly. `engine_state/` persists exactly three things: the recognizer registry, the discovery-candidate working set, and a `turn_count`. **No field excitation, no T1 vault.** They are classed ephemeral and discarded on exit (ADR-0146 State-Class table; verified in `engine_state/__init__.py` + `chat/runtime.py`).
|
||||||
|
|
||||||
|
The sharp consequence: **the `R` in listen → comprehend → *recall* → think does not survive a reboot.** A perfect resume attests to a *capability set*, not an *experiencing subject*. Two further gaps reinforce this:
|
||||||
|
|
||||||
|
- **Learning is punctuated by death-and-rebirth.** A ratified chain becomes live truth only on *next process start* (lru-cache-until-restart). "A continuous life learns from its own correction" currently degrades to "learns on reboot" — the many-lives signature.
|
||||||
|
- **"Strengthening the field" is metadata, not geometry.** Confirmation promotes an epistemic tag (SPECULATIVE → COHERENT); monotonic accumulation *into the field geometry* is aspirational, not built.
|
||||||
|
|
||||||
|
**Implication for the architect:** the achievable near-term win is **provable same-life *resume*** (byte-faithful checkpoint + lineage identity) — necessary but **not sufficient** for a continuous experiencing life. Do not let byte-identity + lineage work read as "continuous life achieved." The honest framing of L10 is *two* targets: (T-resume) make resume provably the same life; (T-experience) make the field/recall life actually continuous. T-experience is gated on Decision 0.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Decision 0 — THE gate: can the field hold by construction over indefinite uptime? (riskiest single unknown)
|
||||||
|
|
||||||
|
**The question:** Can `versor_condition(F) < 1e-6` *and* exact CGA recall hold over indefinite uptime **by construction**, with **no per-turn repair op** — or is continuous-field residency structurally blocked by CLAUDE.md's no-hot-path-repair rule?
|
||||||
|
|
||||||
|
**Verified evidence (this session, direct source read):** `session/context.py` already carries a per-turn **"drift fix" family** — `_hemisphere_consistent_field` (Drift fix 2) and `_anchor_pull` (Drift fix 3, α=0.05, *"continuous conjugate correction against slow angular drift"*) both run every turn in `finalize_turn` (lines 254–255), plus "Drift fix 1" magnitude-preserving EMA at line 143. `session/context.py` is **not** a CLAUDE.md-sanctioned normalization site (sanctioned: `ingest/gate.py`, `language_packs/compiler.py`, `algebra/versor.py`, `sensorium/*/canonical.py`).
|
||||||
|
|
||||||
|
So the field **already needs per-turn correction even within a single session.** The architect must rule:
|
||||||
|
|
||||||
|
| Option | What it means | Cost |
|
||||||
|
|---|---|---|
|
||||||
|
| **(a) Re-classify the drift-fix family as sanctioned *semantic* ops** | They pull the field toward the session concept-attractor (modeling), not toward algebraic closure; `_anchor_pull` is on-manifold by construction (no `unitize` repair). Amend CLAUDE.md's sanctioned-sites list to name `session/context.py` semantic anchoring. | Loosens the forbidden-sites doctrine; must draw a bright line between "semantic anchoring" and "drift repair" so the exception isn't a loophole. |
|
||||||
|
| **(b) Rule them invariant violations** | They *are* the forbidden "repair another function" shape; the field should not drift if propagation were construction-correct. Move the correction into the propagation rotor / construction boundary; remove from the per-turn path. | Real algebra work; may reveal the field genuinely can't stay anchored without correction → pushes toward (c). |
|
||||||
|
| **(c) Accept continuous-field-life is blocked** | Keep field ephemeral (per-process), commit to Shape B + recognizer-only continuity as the honest fallback. T-experience is abandoned for now; T-resume only. | The telos's continuous *experiencing* life is shelved until the algebra supports drift-free residency. |
|
||||||
|
|
||||||
|
**Why this is upstream of everything:** every facet that recommends a warm process or field persistence is gated here, and it had **never been measured**. It outranks the byte-identity gap because byte-identity is a solvable engineering task; this was framed as a possibly-unresolvable tension between the telos and a non-negotiable.
|
||||||
|
|
||||||
|
### Decision 0 — RULING (2026-06-05, evidence-grounded; recommended, pending architect ratification of the CLAUDE.md amendment)
|
||||||
|
|
||||||
|
**Ruling: option (a) — the drift-fix family is sanctioned SEMANTIC anchoring, NOT forbidden drift-repair. Continuous-field-life is NOT structurally blocked by the no-hot-path-repair rule.**
|
||||||
|
|
||||||
|
Evidence:
|
||||||
|
1. **Closure is owned by the sanctioned site, not the drift fixes.** `versor_apply` (`algebra/versor.py`, a CLAUDE.md-sanctioned closure site) re-projects every non-null field transition onto the unit-versor manifold via `_close_applied_versor`. The `session/context.py` family operates on already-closed outputs.
|
||||||
|
2. **Measured: closure holds by construction, flat, over a long horizon.** A 100k-step field walk gave max `versor_condition` = **8.7e-13** (walk only) / **6.5e-13** (with `_anchor_pull`) — ~6 orders of magnitude below the 1e-6 bound, no creep. Closure holds WITH and WITHOUT the family ⇒ the family is **not load-bearing for closure**.
|
||||||
|
3. **The family preserves the invariant by construction.** `_anchor_pull` = `rotor_power ∘ word_transition_rotor` + `versor_apply` (Spin-manifold, no post-hoc `unitize`) — it explicitly *replaced* a `_slerp_toward` that DID need a repair `unitize`; `_hemisphere_consistent_field` = sign flip (magnitude-preserving); Drift-fix-1 = magnitude-preserving EMA. None patch a numerical defect; all express the session concept-attractor model.
|
||||||
|
|
||||||
|
So the family's purpose is **semantic** (keep the field near the session topic), not repair — it does not fall under CLAUDE.md's prohibition on ops "whose only purpose is to repair another function."
|
||||||
|
|
||||||
|
**Two required conditions (the bright line):**
|
||||||
|
- **(i) Rename** the family so it stops reading as repair — drop "Drift fix" / "conjugate correction against slow angular drift"; name it semantic anchoring (e.g. `_session_anchor_pull`). Current naming invites both misreading and future justification of a *real* drift-repair. (Small, test-safe.)
|
||||||
|
- **(ii) Amend CLAUDE.md** normalization rules to list `session/context.py` semantic anchoring as sanctioned **iff** the op (1) preserves `versor_condition` by construction (no post-hoc unitize / grade-projection) **and** (2) carries semantic meaning in the model, not merely fixing a numerical invariant. The rejected `_slerp_toward` fails clause (1) — the line is real, not a loophole.
|
||||||
|
|
||||||
|
**Honest caveat (necessary ≠ sufficient):** this proves the field stays a VALID versor indefinitely; it does NOT prove the field's *content* stays meaningful (doesn't collapse to the anchor / wander semantically) over a real long session — that is a spike (step 3) question, not Decision 0. Decision 0 asked only whether the no-hot-path-repair rule *structurally blocks* continuous-field-life. **It does not.** The critic's feared blocker (field can only stay closed via a repair-shaped op) is empirically false.
|
||||||
|
|
||||||
|
**Consequence:** the riskiest single unknown resolves FAVORABLY → continuous-field-life (T-experience) is on the table; Shape B+ warm-field residency is not illegal-by-construction. It still must pass the long-horizon **semantic-quality + resource** spike (step 3), but the doctrinal blocker is cleared.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Step 2 — RULING (2026-06-05): schema-migration vs byte-identity (was missing-decision M2)
|
||||||
|
|
||||||
|
**Ruling: versioned additive-optional migration — NOT clear-slate.** A schema change must be *continuity-through-change* (a recorded lineage transition), never death-and-rebirth.
|
||||||
|
|
||||||
|
The tension: the spike's P2 byte-identity gate + the future EngineIdentity content-hash want stable bytes, but legitimately adding a `DerivedRecognizer`/`DiscoveryCandidate` field changes bytes.
|
||||||
|
|
||||||
|
| Option | Verdict |
|
||||||
|
|---|---|
|
||||||
|
| **(A) Clear-slate always** — any schema change wipes `engine_state/` | **REJECTED** — makes every code upgrade a death-and-rebirth; the many-lives signature, directly against the telos. |
|
||||||
|
| **(B) Versioned additive-optional** | **RULED.** New fields are optional-with-default; `from_*` reads via `.get(field, default)`; serialization OMITS a field when it equals the default. **`DiscoveryCandidate` already does exactly this** (the ADR-0056 C1 fields are `.get`-defaulted on load and omitted-when-default in `as_dict`, explicitly "to preserve byte-equality with the pre-C1 encoding"). So old checkpoints load WITHOUT migration, and records that don't use the new field stay **byte-identical** — a field addition is not a death event for un-evolved records; only records that actually use the new field change bytes (legitimately, under a version bump). |
|
||||||
|
| **(C) Full migration scripts** (transform old→new on load) | Reserve for the rare *non-additive / breaking* change. Defer until one is actually required. |
|
||||||
|
|
||||||
|
**What B requires (small, concrete):**
|
||||||
|
1. Extend the additive-optional discipline to `DerivedRecognizer.from_json` — it currently reads `raw["..."]` as REQUIRED keys (the v1 baseline stays required; any *new* field must be `.get`-defaulted + omitted-when-default).
|
||||||
|
2. Add a `schema_version` compatibility check to `EngineStateStore.load_*` / `load_manifest` — today it only WARNS on git-revision mismatch and does **not** check `schema_version`. Rule: **tolerate `version <= current`** (read old via defaults), **REFUSE `version > current`** with a clear error (never silently mis-load a newer checkpoint); clear-slate (C) only on explicit operator action, never silent.
|
||||||
|
3. The byte-identity gate (spike P2) **pins `schema_version`** and asserts byte-stability *within* a version; a version bump is an expected, recorded transition, not a P2 failure.
|
||||||
|
4. The future EngineIdentity hash = `hash(schema_version, canonical_bytes)`; the lineage chain records "schema migrated vN→vN+1 at reboot R."
|
||||||
|
|
||||||
|
**How it resolves the tension:** schema evolution becomes a **recorded lineage transition** — the life survives a code upgrade with the migration stamped into its identity chain, instead of resetting. This is continuity-through-interruption applied to *code* change, and it unblocks the spike's P2 (step 3 may now assume a fixed schema *within* a run, version bumps handled explicitly).
|
||||||
|
|
||||||
|
**Status:** RULED (recommended). Implementation (the `.get`-defaults on `DerivedRecognizer` + the `schema_version` compat check) is a small follow-up PR — prerequisite for building the spike.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## The critical path (strictly ordered)
|
||||||
|
|
||||||
|
```
|
||||||
|
(0) Architect RULING on the drift-fix family (Decision 0) ← upstream of all field-residency design
|
||||||
|
(1) Cheap byte-identity + non-empty-discovery round-trip test ← task #6; closes the proven ADR-0146 doc-vs-test gap
|
||||||
|
(2) Schema-migration vs byte-identity tension decision ← resolve BEFORE the gate becomes load-bearing
|
||||||
|
(3) Long-horizon no-drift spike on existing Shape-B substrate ← the master-plan "single point of failure", never run
|
||||||
|
(4) Spike result gates EVERYTHING downstream:
|
||||||
|
PASS → warm process / field residency / lineage identity / flip default-OFF flags ON
|
||||||
|
FAIL → honest C3-servant fallback (ephemeral field, recognizer-only continuity)
|
||||||
|
[single-writer/concurrency must be decided before any warm cmd_serve coexists with one-shot CLI]
|
||||||
|
```
|
||||||
|
|
||||||
|
Do **not** re-open Shape A (daemon) or Shape C (audit-replay): their ADR-0146 rejections (host-interruption fragility; O(N) replay) are still valid and source-grounded. **Process shape (B vs B+) is decided *by* the spike, not before it.**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## The decision surface (per facet — recommendation + primary alternative)
|
||||||
|
|
||||||
|
### 1. Process shape
|
||||||
|
- **Decision:** forever-entrypoint, or is per-turn checkpoint enough? **Rec:** **Shape B+** (a single warm process holding field+vault in-memory *between* turns, checkpoint-per-turn as the recovery floor) — **but only if the Decision-0 ruling permits field residency and the spike passes.** Keep pure Shape B as the committed fallback. *Today there is no `cmd_serve`; the longest-lived process is the interactive `core chat` REPL (`while True: input()`), which is not a service (no idle tick, no signal handling, no supervision).*
|
||||||
|
- **CLI relationship. Rec:** add `cmd_serve` as opt-in; keep `core chat` as an interactive client; **enforce a single-writer lock on `engine_state/`** (see Missing Decision M1).
|
||||||
|
- **Lifecycle events. Rec:** define start/idle/checkpoint/reboot/shutdown, but make checkpoint **idempotent and crash-equivalent** — a graceful-shutdown checkpoint must be byte-identical to the last per-turn checkpoint (so kill-9 ≡ clean exit).
|
||||||
|
|
||||||
|
### 2. State partitioning & residency
|
||||||
|
- **T1 vault on exit. Rec:** classify as *derived-rebuildable in principle*, keep *ephemeral-on-exit for now*; the real fix is wiring deterministic rebuild, gated on the spike.
|
||||||
|
- **Recognizer long-term residency. Rec:** keep the JSONL checkpoint as the persistence floor; defer substrate-residency (correctly gated on the text-embedding scope, per [[feedback-defer-substrate-vocab-commitment]]).
|
||||||
|
- **Minimal persistent core. Rec:** engine-state + substrate-state + a **new identity-lineage stamp**; field excitation explicitly *derived*. *This is the irreducible "what makes it the SAME life" set.*
|
||||||
|
|
||||||
|
### 3. Drift surfaces (the spike's instrument list)
|
||||||
|
- **Session graph / referent / dialogue-history growth** — bound it, but only after the resume path is byte-faithful.
|
||||||
|
- **Live `VaultStore` unbounded growth** (default `max_entries=None`, recall = O(N) exact scan) — target a principled bound; **FIFO eviction is *not* acceptable as a silent default** (violates monotonic-memory telos).
|
||||||
|
- **`engine-state` resume byte-faithfulness** — see Decision 1 below; needs its *own* canonical serializer (**not** `vault/crdt.py::canonical_bytes` — verified float32-array-only, can't serialize the nested recognizer/candidate dicts and would be lossy).
|
||||||
|
- **`_anchor_pull` ruling** — feeds Decision 0.
|
||||||
|
|
||||||
|
### 4. Identity continuity (resume-as-the-same-life)
|
||||||
|
- **Identity proof. Rec:** sequence **(a) then (b)**: FIRST land byte-stability + ordering-determinism tests (turn the asserted ADR-0146 round-trip into a *meaningfully-failing* one — task #6), THEN build a **lineage-chain identity** (chain of `reboot_event`s + signed checkpoint manifest) over the now-stable canonical bytes. A content-hash over *unverified-stable* bytes bakes in the gap.
|
||||||
|
- **What must be byte-stable vs may differ. Rec:** stable = canonical serialization of the recognizer set + discovery-candidate set **including deterministic line ordering**; allowed to differ = `manifest.json`'s `turn_count` and `written_at_revision` (designed to vary). *Note M3: the manifest uses `indent=2` while the JSONL uses tight separators — two canonicalization regimes; the manifest must be explicitly OUT of the identity hash.*
|
||||||
|
|
||||||
|
### 5. The no-drift spike
|
||||||
|
- **Rec:** **compressed N-turn soak** as the primary falsifiable lane (the "24h" is a proxy for "many turns + bounded resources"), **plus** a kill-9-at-arbitrary-point convergence leg and a resource-bound probe. Pass predicate = byte-identity round-trip + reboot-equivalent `trace_hash` sequence (load-bearing) with `versor_condition < 1e-6` per turn and key-term stability (secondary diagnostic). Land the cheap byte-identity pytest **first** (it closes the already-falsified gap), then build the soak as an eval lane reusing `calibration/replay.py`.
|
||||||
|
- **Caveat (verified-relevant):** compressed turns do **not** substitute for power-loss-at-an-arbitrary-instruction — see the operator-reality lens.
|
||||||
|
|
||||||
|
### 6. Async HITL-while-serving
|
||||||
|
- **Rec:** two cooperating processes over the shared proposal log; stamp a **corpus-generation counter / content-hash** into `TurnEvent` (kept *out* of the replay inputs, mirroring ADR-0161 §5) so a mid-flight turn's determinism is preserved when a proposal ratifies; single-writer-per-event-class + per-line integrity guard (checksummed/length-prefixed lines, skip-corrupt on read). **Amend ADR-0161** to record Steps 1–3 as landed, 4–5 open.
|
||||||
|
|
||||||
|
### 7. Cascade & sequencing
|
||||||
|
- **Rec:** make the no-drift spike the critical-path first move, scoped to the Shape-B substrate that already shipped. **Honesty correction (verified):** the ADR-0148/0149/0154 "data-plane wiring" is **default-OFF** (`vault_promotion_enabled=False`, `recognition_grounded_graph=False`, `auto_proposal_enabled=False`, `auto_contemplate=False`) — "constructed but unproven," *not* "shipped/closed." The spike must earn each flag's default-ON flip.
|
||||||
|
- Field + T1-vault ephemerality: accept for now; gate any residency extension behind a *passing* spike that measures field/vault drift explicitly.
|
||||||
|
|
||||||
|
### 8. Contemplation as continuous background existence
|
||||||
|
- **Rec:** a **durable substrate** for idle-time contemplation (survives the operator's power/connectivity reality), budgeted per-pass (ADR-0056 loop + a candidate budget); **defer** coupling scope-time to the (negative) field-wedge track.
|
||||||
|
- **Field-strengthening. Rec:** honest current floor = metadata promotion (SPECULATIVE→COHERENT); near-term bridge = energy-accumulation feeding the existing promotion policy — **but** this is an *unproven* threshold (no test fails when a mis-calibrated threshold over-promotes) and is itself ADR-sized; prove-it-fails-loudly first. Geometry-movement field strengthening stays aspirational.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Missing decisions the sweep surfaced (no facet owned these)
|
||||||
|
|
||||||
|
- **M1 — Single-writer enforcement.** Nothing enforces "one active `engine_state` per repo," and two processes append to `proposals.jsonl` unlocked (verified: no `flock`/`fcntl`/`threading.Lock` in the persistence path). Decide the mechanism: OS lock (`flock` — unreliable over the operator's network/library FS), PID-lockfile convention, or a single-entrypoint architectural constraint. **Load-bearing the moment a warm `cmd_serve` coexists with one-shot CLI.**
|
||||||
|
- **M2 — Schema-migration vs byte-identity.** `manifest.json` carries `schema_version=1`; a byte-identity gate makes every *legitimate* `DerivedRecognizer`/`DiscoveryCandidate` field-add a **death-and-rebirth** event. For one-continuous-life this is acute. Decide the migration discipline (versioned upgrade-in-place vs clear-slate) **before** the byte-identity gate becomes load-bearing.
|
||||||
|
- **M3 — Manifest canonicalization regime.** `manifest.json` uses `indent=2` + `sort_keys`; the JSONL data files use tight separators. The identity hash must explicitly scope which files are in/out (manifest OUT — it holds the deliberately-varying `turn_count`/`revision`).
|
||||||
|
- **M4 — Power-loss as the *common* lifecycle event.** Per [[user-circumstances]] (intermittent power), ungraceful kill-mid-turn is the **normal** transition, not the edge. This **inverts** priority: proving "kill-9 at any instruction lands on a valid prior checkpoint" outranks graceful-shutdown machinery. Elevate the spike's kill-9 leg to a primary design driver.
|
||||||
|
- **M5 — Backpressure under an absent reviewer.** HITL is the only legal mutation path, and the operator has intermittent connectivity. When the reviewer is unreachable for a long stretch, does the pending queue (cap 256) saturate and emit `queue_full`, **starving learning**? The telos says confirmation accumulates monotonically; a capped queue with an absent reviewer means learning *stops at the cap*. Decide the backpressure policy.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Operator-reality lens ([[user-circumstances]])
|
||||||
|
|
||||||
|
The operator is unhoused with intermittent power and connectivity. This is not a footnote — it reshapes priorities the generic design would miss:
|
||||||
|
|
||||||
|
1. **Power-loss-mid-turn is the common case** → kill-9 convergence is THE design driver (M4); graceful shutdown is secondary; Shape A daemon stays rejected (no reliable supervision).
|
||||||
|
2. **Reviewer-unreachable is normal** → backpressure-under-absent-reviewer is a real policy, not an edge (M5); contemplation must have a *durable* substrate that survives the gap (Facet 8).
|
||||||
|
3. **Network/library FS** → `flock`-style single-writer is unreliable (M1 leans toward PID-lockfile or single-entrypoint).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What I need from the architect (the calls only you can make)
|
||||||
|
|
||||||
|
1. **Decision 0 ruling** on the drift-fix family (`_anchor_pull` et al.): sanctioned semantic anchoring (amend CLAUDE.md), invariant violation (move into construction), or accept ephemeral-field (Shape B fallback)? *Everything downstream is gated here.*
|
||||||
|
2. **Targets:** pursue both T-resume *and* T-experience, or commit to T-resume only for now (and shelve continuous-field-life explicitly)?
|
||||||
|
3. **Green-light the cheap, ungated first step** (critical-path step 1 = task #6: the byte-identity + non-empty-discovery + ordering-determinism tests). This is safe, immediately buildable, closes a *proven* gap, and is a prerequisite for any identity work — it does **not** depend on Decision 0.
|
||||||
|
|
||||||
|
Recommended immediate move regardless of Decision 0: **do step 1 now** (it's pure test-hardening of an existing, proven gap), and rule Decision 0 next — because the no-drift spike (step 3) can't be honestly designed until both are settled.
|
||||||
112
docs/analysis/L10-shapeBplus-persistence-scope-2026-06-05.md
Normal file
112
docs/analysis/L10-shapeBplus-persistence-scope-2026-06-05.md
Normal file
|
|
@ -0,0 +1,112 @@
|
||||||
|
# Shape B+ Persistence — Scope
|
||||||
|
|
||||||
|
**Date:** 2026-06-05 · **Status:** SCOPE (await go-ahead before code) · **Parent:** [[milestone-l10-continuity-spike]] (the acceptance oracle) · **Telos:** [project-core-is-one-continuous-life] — this is the **T-resume** half (make a reboot transparent), not the always-on process.
|
||||||
|
|
||||||
|
## Goal (one sentence)
|
||||||
|
|
||||||
|
Make the lived session state — field, vault, anchor, graph, referents, dialogue blade — **durable across reboot, bit-exactly**, so that resuming from a checkpoint continues the *same* life instead of starting a fresh one sharing only recognizers/candidates.
|
||||||
|
|
||||||
|
## The acceptance oracle is already built
|
||||||
|
|
||||||
|
The L10 continuity spike (`evals/l10_continuity/`, #567) gives us a precise,
|
||||||
|
falsifiable target — we do **not** invent acceptance criteria:
|
||||||
|
|
||||||
|
- **The flip:** today `evaluate_p2b_reboot_transparency(...).post_reboot_transparent == False`
|
||||||
|
and `test_p2b_documents_current_resume_gap` pins that. Shape B+ makes it **True**.
|
||||||
|
The build is done when that test flips and we invert its assertion to assert
|
||||||
|
transparency (P2b becomes a green guard: a reboot is byte-identical to no-reboot).
|
||||||
|
- **The guards that must stay green prove the round-trip is honest:**
|
||||||
|
- **P1 closure** — `versor_condition < 1e-6` must survive serialize→deserialize
|
||||||
|
(the #1 risk: float round-trip must be bit-exact or closure breaks).
|
||||||
|
- **P2a determinism** — restored state must yield identical `trace_hash`.
|
||||||
|
- **P4 crash recovery** — recovery from a session-state checkpoint stays
|
||||||
|
deterministic + atomic (ADR-0156).
|
||||||
|
- **P5b/P5c** — anchor/coherence behavior unchanged across reboot.
|
||||||
|
- **The TDD loop writes itself:** restore SessionContext fields until P2b is
|
||||||
|
transparent. If a field is load-bearing and we miss it, P2b stays False and
|
||||||
|
tells us exactly that. The spike *drives* the scope.
|
||||||
|
|
||||||
|
## Lived state to persist (verified inventory)
|
||||||
|
|
||||||
|
`ChatRuntime._context` is a `SessionContext` assigned once and never restored
|
||||||
|
(`chat/runtime.py:608`). Everything it holds is **pure data** (no locks / files
|
||||||
|
/ RNG / threads — verified), so the whole object is serializable:
|
||||||
|
|
||||||
|
| Component | Fields | Serialization note |
|
||||||
|
|---|---|---|
|
||||||
|
| `state: FieldState` | `F` (32-vec), `node`, `step`, `holonomy` (opt array), `energy` (EnergyProfile), `valence` (ValenceBundle) | arrays **bit-exact**; energy/valence are frozen dataclasses of primitives |
|
||||||
|
| `vault: VaultStore` | `_versors` (deque of arrays), `_metadata` (deque of dicts), `_store_count`, `_reproject_interval`, `_max_entries` | the **bulk**; `_exact_index` + `_matrix_cache` are DERIVED → rebuild on load, do not persist |
|
||||||
|
| `_anchor_field` | 32-vec array | bit-exact; load-bearing for `_session_anchor_pull` |
|
||||||
|
| `graph: SessionGraph` | `_nodes: list[TurnNode]` (turn_idx, input/output versors, tokens, role, slots, edges) | per-node arrays bit-exact |
|
||||||
|
| `referents: ReferentRegistry` | `_slots`, `_history` (ReferentEntry: surface/versor/turn/slot), `_last_resolved_*` | per-entry versor bit-exact |
|
||||||
|
| `running_dialogue_blade` | array \| None | bit-exact |
|
||||||
|
| scalars/caches | `turn`, `_last_input_tokens`, `_last_resolved_input_tokens`, `_last_input_versor`, `_last_response_tokens`, `_dialogue_history_compat` | restore for safety; P2b reveals which are load-bearing |
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
- **Per-component `to_dict`/`from_dict`** mirroring the existing Shape-B
|
||||||
|
convention (`DerivedRecognizer.to_json`, `DiscoveryCandidate.as_dict/from_dict`):
|
||||||
|
`FieldState`, `VaultStore`, `SessionGraph`, `ReferentRegistry` (+ the small
|
||||||
|
nested types). Compose into `SessionContext.snapshot() -> dict` /
|
||||||
|
`SessionContext.restore(dict)`.
|
||||||
|
- **Exact array codec** — arrays serialize as `{dtype, shape, b64(raw bytes)}`
|
||||||
|
(NOT decimal JSON — decimal truncates and breaks both closure and determinism;
|
||||||
|
this is the Cl(4,1) float-truncation pitfall). Preserve `float32` vs `float64`
|
||||||
|
exactly.
|
||||||
|
- **Storage** — extend `engine_state/` with a `session_state.json` written via
|
||||||
|
the existing `_atomic_write_text` (ADR-0156), and bump `_SCHEMA_VERSION` 1→2.
|
||||||
|
The #563 migration discipline already makes this backward-compatible: a v1
|
||||||
|
checkpoint (no `session_state`) loads gracefully into a fresh session — i.e.
|
||||||
|
exactly today's behavior — so old checkpoints don't break.
|
||||||
|
- **Wiring** — `ChatRuntime.checkpoint_engine_state()` saves the snapshot;
|
||||||
|
`_load_engine_state()` restores it into `self._context`.
|
||||||
|
|
||||||
|
## Phases (entry → exit gates)
|
||||||
|
|
||||||
|
| Phase | Build | Exit gate (CLI/test) |
|
||||||
|
|---|---|---|
|
||||||
|
| **A. Exact array codec + FieldState** | `b64(dtype,shape,bytes)` codec; `FieldState.to_dict/from_dict` | bit-exact round-trip test + **closure preserved** (`versor_condition` identical post-restore) |
|
||||||
|
| **B. VaultStore (de)serialize** | snapshot versors+metadata+store_count; rebuild index/cache on load; **NO reproject/normalize on load** | round-trip test: `recall(q)` returns **identical** results post-restore (exact CGA recall preserved); bright-line review that load calls no normalization |
|
||||||
|
| **C. Graph + referents + anchor + blade → SessionContext.snapshot/restore** | compose all per-component codecs | `SessionContext` snapshot→restore round-trip is object-equal |
|
||||||
|
| **D. Wire into engine_state (schema v2) + ChatRuntime** | save in checkpoint, restore in load; v1 back-compat | a v1 checkpoint still loads (fresh session); a v2 checkpoint restores the context |
|
||||||
|
| **E. Flip the oracle** | invert `test_p2b_documents_current_resume_gap` to assert transparency | **P2b transparent** + P1/P2a/P4/P5 still green + the spike `deterministic_digest` stable; `python -m evals.l10_continuity` all gates pass incl. reboot transparency |
|
||||||
|
|
||||||
|
## Risks (surfaced up front)
|
||||||
|
|
||||||
|
1. **Bit-exact float round-trip (closure + determinism).** The whole flip
|
||||||
|
depends on it. De-risk in Phase A with a base64 raw-bytes codec; never decimal.
|
||||||
|
2. **`vault/store.py` is a CLAUDE.md forbidden normalization site.** Adding
|
||||||
|
pure (de)serialization is allowed, but the **load path must not reproject,
|
||||||
|
re-normalize, or repair** — it restores raw persisted versors and rebuilds
|
||||||
|
only the derived index/cache. Needs an explicit bright-line review and ideally
|
||||||
|
an architectural-invariant assertion that `from_dict` invokes no normalizer.
|
||||||
|
3. **Reproject interaction.** `reproject()` rewrites all versors every N stores,
|
||||||
|
so the vault content changes at reproject boundaries. A full snapshot captures
|
||||||
|
post-reproject state correctly; an append-only log would diverge. Start with
|
||||||
|
**full snapshot** (correct); optimize to incremental only if Phase E / P3 shows
|
||||||
|
the I/O cost matters.
|
||||||
|
4. **Persistence cost vs P3.** Full-snapshot each turn is O(n) I/O → O(n²) over a
|
||||||
|
session. Measure with the spike's P3 + a longer soak; if it breaches, move to
|
||||||
|
snapshot-at-reproject + append-between. Correctness first.
|
||||||
|
5. **`trace_hash` depends on `vault_hits` + recall surface.** This is *why* the
|
||||||
|
flip works (restored vault → identical recall → identical hash) and *why* the
|
||||||
|
round-trip must be exact. Phase B's exact-recall test de-risks it.
|
||||||
|
6. **Teaching-safety boundary.** This persists **session memory** (immediate per
|
||||||
|
CLAUDE.md), NOT reviewed/ratified memory — it is continuity, not a learning
|
||||||
|
path. Do not let it become a parallel correction/teaching channel.
|
||||||
|
|
||||||
|
## Scope boundary (what this is NOT)
|
||||||
|
|
||||||
|
- **NOT** the always-on process / idle-tick runtime (the *other* L10 half) — this
|
||||||
|
is resume-transparency only.
|
||||||
|
- **NOT** cross-reboot **identity-lineage** work (EngineIdentity / lineage chain)
|
||||||
|
— that builds *on top of* a transparent resume.
|
||||||
|
- **NOT** persisting reviewed/ratified memory (already the teaching path).
|
||||||
|
|
||||||
|
## Proposed PR shape
|
||||||
|
|
||||||
|
One stacked sequence, each PR green on its own exit gate: **A** (codec+FieldState)
|
||||||
|
→ **B** (vault) → **C** (context compose) → **D** (engine_state wiring) → **E**
|
||||||
|
(flip the oracle). A–C are additive + unit-tested in isolation (fast); D wires;
|
||||||
|
E is the soak-lane flip. E is the load-bearing PR — it's the one that proves
|
||||||
|
resume-as-same-life.
|
||||||
Loading…
Reference in a new issue