core/docs/decisions/L10-runtime-model-scope.md

# Scope: L10 — Runtime Model (Process Lifecycle for Forever-Running CORE)

**Status:** Draft v1 / scope-only (not a decision yet — prerequisite for one)
**Date:** 2026-05-24
**Author:** CORE agents
**Anchor:** [thesis-decoding-not-generating](../../../.claude/projects/-Users-kaizenpro-Projects-core/memory/thesis-decoding-not-generating.md) (memory)
**Discipline:** [feedback-adr-cross-reference-discipline](../../../.claude/projects/-Users-kaizenpro-Projects-core/memory/feedback-adr-cross-reference-discipline.md) (memory)
**Companions:** [substrate-liveness-audit-scope](./substrate-liveness-audit-scope.md), [recognizer-storage-scope](./recognizer-storage-scope.md), [teaching-derived-recognition-scope](./teaching-derived-recognition-scope.md)
**Shelved-but-relevant:** [project-engine-identity-candidate](../../../.claude/projects/-Users-kaizenpro-Projects-core/memory/project-engine-identity-candidate.md) (memory) — DNA-analog `EngineIdentity` concept; may become load-bearing under this scope's decisions

---

## Why this document exists

CORE's recent scope work (recognizer-storage v2; substrate-liveness-audit
v2) repeatedly named a load-bearing prerequisite that has no scope or
ADR of its own: **the runtime model**. Several decisions are gated on
it:

- The recognizer-storage scope explicitly states: "this scope is gated
  on the runtime-model scope existing and committing to forever-running."
- The substrate-liveness-audit scope names L10 (runtime model) and L11
  (forever-running engine) as not-audit-targets because they have no
  design yet — the audit's job is to surface need for them.
- The forever-running engine vision ("listen, comprehend, recall, think,
  articulate, learn from reviewed correction, replay deterministically
  — with capability compounding across turns and surviving reboot as
  recovery, not as control flow") is the destination CORE is being built
  toward, but the *process shape* in which that vision is realized is
  unspecified.

CORE today is session-bounded: every `core` CLI invocation builds a
fresh `ChatRuntime`, packs are loaded fresh, the engine has no
long-lived process. The thesis demands the engine *accumulate
capability over its lifetime*; the current process shape doesn't have
a lifetime in any meaningful sense.

This scope defines the question. The answer belongs to the ADR that
follows, and to the substrate-liveness-audit findings that will
inform what's actually load-bearing.

---

## Cross-reference audit (applying the discipline up front)

Per [[feedback-adr-cross-reference-discipline]], the runtime-model
question must be drafted *against* the existing ADRs and code that
already touch lifecycle, persistence, and process shape — not in a
vacuum.

### Existing ADRs touching lifecycle / persistence / process shape

| ADR | Subject | Relevance |
|---|---|---|
| ADR-0040 | Telemetry sink (JSONL) | Persistent turn-event audit trail; survives across sessions; load-bearing for any audit-based "what happened" recovery. |
| ADR-0041 | CLI verdicts + fan-out | Operator readout surface; relevant to HITL entry shape. |
| ADR-0042 | Audit-tour demo | Four-scene reproducibility tour; demonstrates that telemetry IS the cross-session record. |
| ADR-0055 | Inter-session memory | **Most directly relevant.** Defines four-tier memory: T1 session vault (ephemeral-ish), T2 turn-event JSONL (audit trail, persistent), T3 reviewed teaching corpus (persistent, append-only), T4 ratified packs (long-term substrate). Explicitly names "what survives across all sessions and reboots." |
| ADR-0056 / ADR-0080 | Contemplation loop | Async-feeling work (enriching discovery candidates) — needs a process model to know when/where it runs. |
| ADR-0057 | Teaching-chain proposal review | Append-only proposal log; HITL review surface; the existing HITL machinery this scope must build on. |
| ADR-0014 | Train/learning loop (`VaultPromotionPolicy`) | Promotion gate, currently dormant; L2 audit will verify. Live promotion is part of "live mode." |
| ADR-0027 | Identity packs | Loaded at startup; mutation = restart today. Persistent identity continuity is open. |
| ADR-0029 | Safety packs | Same — startup-loaded, fail-closed; mutation = restart. |
| ADR-0033 | Ethics packs | Same shape. |

### Existing code shape (session-bounded reality today)

- **All entrypoints are `argparse`-based CLI commands in `core/cli.py`.**
  `cmd_chat`, `cmd_test`, `cmd_check`, `cmd_trace`, `cmd_oov`,
  `cmd_capability_*` (many), `cmd_pack_*`, `cmd_teaching_*`. Every
  command is one-shot; the process exits when the command returns.
- **No `cmd_serve` / `cmd_daemon` / long-lived process exists.** There
  is no current "forever" entrypoint.
- **`ChatRuntime` (`chat/runtime.py:418`)** is per-invocation. State
  on it (manifold, session thread, contemplation state, recognizer
  registry) does not survive process exit.
- **Persistent state today:** vault store (on-disk, reloaded each
  invocation), teaching corpus (append-only JSONL on disk), telemetry
  sink (JSONL on disk, ADR-0040), packs (on-disk with manifests),
  proposal log (when written, append-only, ADR-0057).
- **Ephemeral state today:** ChatRuntime instance, field manifold,
  session_thread context, contemplation working state, recognizer
  registry (no persistence layer yet), HITL queue (no in-memory
  representation yet).
- **HITL today** is operator-runs-CLI-commands. There is no async
  queue the operator reviews while the engine continues running —
  because the engine doesn't continue running.

### Existing HITL machinery this scope must build on

Per [[feedback-adr-cross-reference-discipline]] and the recognizer-
storage v2 self-review (which previously missed this): the HITL path
is *not* greenfield. ADR-0057 establishes the shape:

- **`teaching/review.py`** — `ReviewOutcome` enum, `review_correction()`
- **`teaching/store.py`** — `PackMutationProposal`, `TeachingStore`,
  append-only proposal log discipline
- **Automated gate:** replay-equivalence (ADR-0057's load-bearing
  innovation)
- **Operator review surface:** today, CLI commands (`cmd_teaching_*`)
- **Append-only proposal log:** persistent across sessions

L10 must reuse this machinery; the *new* concern is making it
asynchronous (operator reviews while engine runs) rather than
synchronous (operator review is a CLI command between engine
invocations).

---

## The runtime-model question

> **What process shape allows CORE to accumulate capability over its
> lifetime, survive reboot as recovery rather than as control flow,
> and present a narrow async HITL ratification entrypoint that is
> never bypassed and never required for runtime continuation?**

Four sub-questions, each load-bearing:

### Sub-question 1 — Process shape

Three candidates, evaluated against current state:

- **A. Long-lived daemon (`cmd_serve`).** One process; CLI commands
  become clients of the daemon via local IPC. Most thesis-aligned
  (engine has a lifetime). Largest delta from today's code shape;
  requires concurrency model decisions, signal handling, supervision.
- **B. Hybrid (engine state externalized; CLI commands restore it).**
  Engine state serialized to disk on every "logical action" boundary;
  any CLI invocation can restore the latest state and continue. No
  long-lived process needed; lifetime is the lifetime of the
  serialized state on disk. Smaller code delta; serialization
  discipline becomes load-bearing.
- **C. Continue with one-shot CLI; teach the audit/recovery layer
  to be the lifetime.** Every invocation is a fresh process, but
  every turn appends to a deterministic audit trail (ADR-0040 +
  ADR-0055 T2) from which the *next* invocation reconstructs
  capability. The audit trail IS the engine's lifetime. Smallest
  delta; pushes the cost of "always on" into "always rebuild from
  audit."

Honest assessment of trade-offs is the spike/ADR's job, not the
scope's. Scope names the three candidates.

### Sub-question 2 — State partitioning

Three state classes the runtime-model must distinguish:

- **Session-state (ephemeral, per-turn-window):** anaphora context,
  current intent, immediate field excitation. May be lost on reboot
  without architectural concern.
- **Engine-state (live, persistent across reboot):** the recognizer
  registry, the contemplation working set, the HITL ratification
  queue. *Expensive to rebuild from primitives;* MUST persist; reboot
  reloads, does not re-derive.
- **Substrate-state (cold, persistent across reboot):** ratified packs,
  ratified teaching corpus, vault store, telemetry JSONL, proposal
  log. Already on disk today; the discipline question is when each
  is updated and how reboot validates consistency.

The scope's commitment is to **naming the three classes**; the ADR
decides what concrete state objects fall in which class.

### Sub-question 3 — Reboot recovery

Three questions reboot recovery must answer:

- **What does reboot verify?** Pack manifest checksums (already do);
  vault integrity (does); reviewed corpus consistency (does); engine-
  state integrity (does NOT today — engine state doesn't survive). If
  any verification fails, what happens? (Refuse to start, fall back,
  surface to HITL?)
- **What does reboot reload vs. rederive?** Substrate-state reloads;
  engine-state reloads if Process Shape A or B, or rederives from
  audit if C; session-state is always rederived (it's per-turn).
- **What does reboot record?** A `reboot_event` analog of `TurnEvent`,
  written to the audit trail, that lets future audit reconstruct the
  fact that this engine instance lost and regained its lifetime here.

The shelved [[project-engine-identity-candidate]] (`EngineIdentity`
content-derived hash) is one candidate mechanism for verifying engine
identity continuity across reboot — explicitly NOT committed by this
scope; flagged here so the ADR knows the candidate exists.

### Sub-question 4 — HITL async entrypoint

Today: operator runs CLI commands. There is no async queue. ADR-0057
establishes the proposal-log shape but the operator interacts with it
through `cmd_teaching_*` commands.

For forever-running, the HITL queue is async by definition:

- **What is the queue's persistent representation?** Likely an
  extension of the ADR-0057 append-only proposal log, possibly with a
  "review state" axis (pending / under-review / accepted / rejected /
  expired).
- **How does the operator interact with it?** Continued CLI commands
  (Process Shape A/B/C compatible) vs. a TUI / web surface (larger
  delta; out of scope for this ADR most likely).
- **What does the engine do while a proposal is pending HITL?**
  Continue serving turns normally; the proposal is not blocking. This
  matches the [[feedback-adr-cross-reference-discipline]] commitment
  ("HITL is the narrow entrypoint, never bypassed, never required for
  runtime continuation").
- **How are proposal queues bounded?** Already named in recognizer-
  storage v2 as a load-bearing constraint: backpressure (queue cap?
  rate limit? operator alert?). The drop-off sibling ADR will specify
  for deprecation; this scope must specify the *generic* shape that
  drop-off and discovery-promotion and pack-mutation all share.

---

## Constraints (non-negotiable)

From CLAUDE.md and the existing thesis:

1. **Deterministic replay.** Whatever the runtime model, given the
   same teaching corpus + same input stream + same ratified substrate,
   the engine must produce the same turn outputs and the same
   trace_hashes. Process shape MUST NOT introduce non-determinism
   (no wall-clock timestamps in deterministic payloads, no parallelism
   without explicit ordering, no race-condition surface in the turn
   loop).
2. **No hidden state.** Engine-state that persists across reboot MUST
   be auditable: serialized in a human-and-machine-readable form,
   checksummable, reproducible from the audit trail when possible.
3. **HITL is the narrow entrypoint.** No autonomous mutation of
   ratified state. No path that lets the engine modify packs,
   teaching corpus, or recognizer registry without operator
   ratification. The forever-running engine's autonomy ends at
   "propose"; ratification is always operator.
4. **Reboot is recovery, not control flow.** No CLI command, no test,
   no operator action should require a reboot. Reboot is a hardware-
   event analog; the engine survives it but does not depend on it.
5. **Existing append-only artifacts stay append-only.** Telemetry
   JSONL, teaching corpus, proposal log — these are audit substrate
   and must remain append-only across this scope's decisions.
6. **No drift repair, no hot-path normalization in the new code.**
   Per CLAUDE.md normalization rules — runtime-model code is a
   forbidden site for these patterns.

---

## What this scope explicitly rejects

- **A runtime-model that requires re-architecting `ChatRuntime`.**
  The session-bounded `ChatRuntime` is the unit of work; whatever
  shape the runtime model takes, `ChatRuntime` should remain
  recognizable.
- **Database persistence for engine state.** Per ADR-0055 north-star:
  "not in a database/embedding store." Engine state persists as files
  (JSONL, pack manifest, vault store), not in a DBMS.
- **Network surface as a primary entrypoint.** This scope assumes
  local-only operation (the user's circumstances make
  always-on-internet unsafe to assume; per [[user-circumstances]]).
  A network surface may come later but is not in scope.
- **Multi-tenant or multi-instance concerns.** Single engine instance
  on a single machine. Sharing substrate across instances is a
  separate scope (or a feature of [[project-engine-identity-candidate]]
  if it ever un-shelves).

---

## What this scope does NOT commit

- **Process Shape A vs. B vs. C.** Spike/ADR decides. The audit (L4-L9
  findings, especially L7/L8) will inform which is least disruptive.
- **HITL surface beyond CLI.** A TUI or web surface may eventually
  exist; this scope doesn't decide.
- **EngineIdentity adoption.** Shelved candidate; ADR may un-shelve
  it if reboot-recovery sub-question demands it.
- **Concurrency model.** If Shape A wins, the daemon's concurrency
  model (threads? async? per-turn process?) is an implementation
  detail of the ADR, not the scope.
- **Specific persistence format for engine-state.** JSONL extension?
  Custom binary? Pack-style? Implementation detail.
- **Migration path from current session-bounded shape.** ADR
  specifies; scope notes that migration MUST be incremental (no big-
  bang switchover that breaks `core chat`).

---

## Determinism requirements (non-negotiable)

The runtime model must preserve:

1. **Byte-identical replay.** Same substrate + same input stream ⇒
   same turn outputs. Process shape MUST NOT introduce wall-clock,
   PID, or other process-bound entropy into deterministic payloads.
2. **Reboot-equivalent state.** State after `(boot, run N turns,
   reboot, reload)` is byte-identical to state after `(boot, run N
   turns)` minus process-memory artifacts. (Reboot loses
   session-state; engine-state and substrate-state are restored.)
3. **HITL ratification trace.** Every ratification produces an
   append-only log entry with deterministic content; the log is the
   audit trail for engine-state mutations.

---

## Risks the spike / first ADR must surface

- **Shape-A concurrency complexity.** A long-lived daemon introduces
  signal handling, supervision, possibly inter-process concurrency.
  CLAUDE.md's "Do not add hidden background execution" is at risk;
  daemon design must make all concurrency explicit and auditable.
- **Shape-B serialization correctness.** Every engine-state object
  must round-trip serialization byte-identically. One drifted
  serializer breaks reboot recovery silently.
- **Shape-C rebuild cost.** If the audit trail grows to N turns,
  rebuilding capability on every CLI invocation is O(N). May be fine
  for small N, untenable for large.
- **HITL queue starvation.** If the operator goes offline for an
  extended period, proposals accumulate. The recognizer-storage v2
  scope flagged this as load-bearing for drop-off; it's load-bearing
  for every proposal kind. The ADR must specify backpressure
  generically.
- **Pack-mutation during running engine.** If Shape A or B is chosen,
  a ratified pack mutation while the engine is running raises
  questions: hot-reload? Refuse-and-restart? Queue-until-quiescent?
  Each option has different operator-experience implications.
- **Audit-trail compaction.** If telemetry JSONL grows unbounded,
  reboot recovery (under Shape C) becomes slower over time. Compaction
  / snapshotting is the natural answer but introduces a new mutation
  axis on append-only state. ADR specifies or defers.
- **State-class boundary mistakes.** Putting engine-state in
  session-state's bucket loses learning on reboot; putting session-
  state in engine-state's bucket bloats persistence and breaks
  determinism. The state-partitioning sub-question is high-leverage.

---

## Open questions for the audit and follow-up scopes to inform

- **What does the substrate-liveness audit (L4-L9) find about state
  that wants to be persistent but currently isn't?** Each closure gap
  the audit surfaces is potential engine-state for this scope to
  classify. Specifically:
  - L4 (Recognition) — recognizer registry persistence (already
    scoped in recognizer-storage; cross-references here).
  - L7 (Teaching loop) — proposal log liveness; HITL queue persistence.
  - L8 (Inter-session memory + contemplation) — Tier 1/2/3 transitions;
    contemplation working set.
- **Should the runtime-model ADR un-shelve `EngineIdentity`?** If
  Sub-question 3 (reboot recovery) commits to verifying engine
  identity continuity across reboot, the shelved candidate likely
  becomes the right primitive. Trigger: sub-question 3 commits to
  cross-reboot identity verification.
- **Does Process Shape A require a new top-level package
  (`core/server/` or `core/daemon/`)?** Implementation-detail
  question, but if yes, the substrate-liveness audit will need to
  add a layer (L10b? a daemon layer?) to its map.
- **How does the runtime model interact with the audit's "live mode"
  framing?** The substrate-liveness-audit scope frames "live mode" as
  the destination state; the runtime model is the mechanism. They
  must be drafted together as the ADR cluster lands.

---

## Summary

L10 — the runtime model — is the missing prerequisite for forever-
running CORE. Several recent scopes (recognizer-storage,
substrate-liveness-audit) flagged it as gated work without it
existing.

This scope frames the question against four sub-questions: process
shape (daemon / hybrid / one-shot-with-replay), state partitioning
(session / engine / substrate), reboot recovery (verify what, reload
vs. rederive, record what), and HITL async entrypoint (queue shape
+ backpressure). It explicitly cross-references the ADR-0040/0041/
0042/0055/0056/0057 cluster that already implements parts of the
answer, and the recognizer-storage scope that depends on this scope's
decisions.

It rejects database persistence, network primary entrypoints,
multi-tenant concerns, and re-architecting `ChatRuntime`. It commits
no specific shape; the spike/ADR decides, informed by audit findings.

The scope's commitment is to **the question framed against existing
machinery**. Answers belong to the spike and the ADR that follows.
Audit findings (L4-L9) refine the question over time.