# Master plan — post-substrate-audit **Authored:** 2026-05-24 (end-of-day, PST) — call it a night, fresh AM. **Author intent:** carry the strategic plan + active-PR snapshot across the session boundary so morning-self picks up cleanly. --- ## Three guiding rules 1. **"Fully wired"** = every `W-*` entry in [substrate-liveness-ratchet](audit/substrate-liveness-ratchet.md) CLOSED. When the registry is empty, the design executes as written. 2. **Thesis check on every move**: per [[thesis-decoding-not-generating]] — does this teach the engine to find better, or just store another found thing? Only the former clears the bar. 3. **L10 is the load-bearing decision.** It gates W-003, W-007, W-009, W-017, W-018, and the recognizer-storage ADR. Wrong shape locks in wrong architecture for the rest of the cascade. --- ## End-of-day snapshot (2026-05-24) ### Active in flight | PR | Branch | State | Blocker | |----|--------|-------|---------| | **#261** (B speedup) | `chore/test-speed-b-extract-lane-runners` | CI rerunning after timeout bump + test_lane_sha_verifier fix | Wait for `verify pinned lane SHAs` to go green | | **(unopened)** Gate | `ci/full-pytest-gate-quarantine-markers` (pushed, no PR) | Locally verified: 6657 pass / 0 unexpected fail / 76 skip / 49 quarantine | Open PR **after #261 merges** (otherwise gate's CI hits pre-fix lane-shas state) | ### Closed today (5 W-* + 5 PRs) - **W-004** — vault E2 re-thaw (PR #251) - **W-015** — `_slerp_toward` → rotor-geodesic anchor pull (PR #255) - **W-016** — vault probe in discovery loop (PR #257) - **W-011** — recognition refusal propagation (PR #258, paired) - **W-012** — `InnerLoopExhaustion` caught in ChatRuntime (PR #258, paired) - Ratchet v5 (PR #260) — registry updated ### Quarantine registry (49 tests) Cluster A (4) — ADR ledger drift, one-token extensions, shape of W-002 Cluster B (15) — surface decoration drift (`pack-grounded ()` suffix) Cluster C (27) — lane/runner metric drift (some are real regressions) Cluster D (2) — CLI/internal API drift Cluster E (1) — pytest-xdist parallel-execution incompatibility (`test_articulation_bench::test_footprint_emits_samples_and_bounds`) All confirmed pre-existing via bisect against `c1a1b7a` (pre-W-*-work commit). Today's 5 closures introduced **zero** new failures. Cluster E is the lone xdist-induced regression — caused by gate's `-n 4` choice, not by today's substantive work. ### Resumption order in AM 1. **First**: check PR #261's CI re-run status. If green, merge. If red, debug from CI logs. 2. **Second**: rebase `ci/full-pytest-gate-quarantine-markers` onto post-#261 main; open as PR. 3. **Third**: read this plan top-to-bottom to re-load context. 4. **Fourth**: Phase 2 work — five operator decisions queued (W-006, W-010, W-013, W-014, W-019). Each ~30 min. --- ## Phase 1 — Hygiene & honesty (today, in flight) - **B PR #261** — extract `math_teaching_corpus` lane from pytest into CI lane SHAs (−9 min suite time, ~30% reduction) - **Gate PR** (pending #261) — `conftest.py` QUARANTINE registry of 49 known-failing tests + new `full-pytest.yml` workflow running `pytest -m "not quarantine" -n 4` - **5 W-* closures** — W-004, W-011, W-012, W-015, W-016 **Outcome:** test debt is **visible and ratcheted**; speed recovered; the audit-derived registry is operational. We can ship the next dozen PRs without flying blind. **Follow-up queued (task #7):** small PR migrating `lane-shas.yml` from plain `pip` to `uv` to match the gate's precedent. Per [[feedback-use-uv-consistently]]. --- ## Phase 2 — Operator decisions (small, you-decide, fast) Five short calls clear five `W-*` entries with minimal-to-no engineering. | Item | Decision needed | Resolution paths | |------|-----------------|------------------| | **W-006** | Pack readback rules: wire or delete? | (a) wire `packs//readback_rules.py` into surface generation per original intent; (b) accept that `generate/realizer.py` superseded the design and DELETE the dormant modules | | **W-010** | L4 recognition vocab: intentional token-level or wire L3? | (a) document as intentional in ADR-0143 amendment (no code change); (b) wire `VocabManifold` consumption into `derive_recognizer()` via new ADR (larger) | | **W-013** | `core/cognition/explain.py`: wire / relocate / delete | (a) wire into `core chat` for "explain this turn" REPL; (b) move to `evals/` if offline audit tool; (c) delete | | **W-014** | `core/cognition/provenance.py`: wire / relocate / delete | (a) wire into live turn result for per-turn provenance; (b) relocate to `evals/`; (c) leave as-is, document as evals-only | | **W-019** | `teaching/from_miner.py` + `from_curriculum.py`: CLI / runtime / library? | (a) wire CLI: `core teaching propose --from-miner ` (smallest fix); (b) wire into W-017 promotion pipeline (later, larger); (c) leave as test-live library | Each ~30 min thought + small PR. Mostly cleanup-as-you-find calls per [[feedback-cleanup-as-you-find]]. --- ## Phase 3 — Test debt paydown Goal: shrink `QUARANTINE` from 49 → 0. One PR per cluster. ### Sequence (easiest → hardest) 1. **Cluster A** (4 tests, ADR ledger drift) — one-token extensions, same shape as W-002 (#240). **~1 hour.** 2. **Cluster D** (2 tests, CLI/API drift) — quick reads, quick fixes. **~30 min.** 3. **Cluster E** (1 test, xdist incompatibility) — rewrite to measure only self-allocations OR mark `@pytest.mark.xdist_group("serial")`. **~30 min.** 4. **Cluster B** (15 tests, surface decoration drift) — mechanical: update assertions to use `in` containment or accept the `pack-grounded ()` suffix. **~3-4 hours.** 5. **Cluster C** (27 tests, lane metric drift) — **the dangerous one.** Each test needs investigation: stale threshold (re-pin) vs real quality regression (fix the regression). Some will surface honest engine-quality problems we've been blind to. Expect 1-3 PRs worth of work to surface what's stale vs what's broken. By end of Phase 3, the gate enforces zero quarantine and every contract pinned in tests is a contract the engine actually holds. --- ## Phase 4 — THE load-bearing decision: L10 runtime model This is the architectural commitment that gates W-003, W-007, W-009, W-017, W-018. **Until L10 commits, the rest of the audit registry can't sequence.** Scope already landed (#236). What's left: - **Spike**: prove a long-lived process can hold field state + vault + session continuity without leak/drift across 24+ hours of turns - **ADR (or cluster)**: pick process shape A/B/C, state partitioning, reboot recovery model, HITL async substrate - **First implementation**: minimal long-lived runtime that survives the spike's stress test This is the **riskiest single decision in the whole plan**. Wrong shape locks in wrong architecture for everything downstream. **Discipline note** (per [[feedback-scope-time-is-cheap]]): treat L10's spike with the same rigor as the substrate-liveness audit — multi-resolution scope, agent-cross-checked, before writing any committing ADR. The audit just demonstrated what "do it right" looks like; L10 deserves the same. **Open offer**: draft a v1 of the L10 *scope-of-scope* (what questions the spike must answer) once Phase 1 lands. Operator's call to take. --- ## Phase 5 — The L10 cascade (after Phase 4) Each is small **once L10 commits process shape**: | Item | What lands | Depends on | |------|-----------|------------| | **W-003** | `VaultPromotionPolicy` wired into runtime promotion path | L10 process shape | | **Recognizer-storage ADR** | Where recognizers live across turns/sessions/reboots | L10 + W-003 | | **W-007** | `DerivedRecognizer` integration into `CognitiveTurnPipeline` | Recognizer-storage ADR | | **W-009** | HITL async queue (operator reviews while engine serves turns) | L10 + drop-off sibling ADR | | **W-017** | Automated T1/T2 → T3 promotion | W-009 (W-016 portion already satisfied by #257) | | **W-018** | Autonomous contemplation loop | L10 | When all six land + recognizer-storage answers, **the audit registry is all CLOSED.** The design executes. --- ## Phase 6 — The "decodes, not generates" milestone This is where smashing expectations actually lives. With Phase 5 done, CORE demonstrably: - **Runs forever** (Phase 4) — capability accumulates across invocations, not per-turn - **Recognizes deterministically** (W-007) — turn loop dispatches through anti-unifier - **Promotes evidence automatically** (W-017) — T1 vault → T3 reviewed corpus on HITL ratification - **Learns from refusals** (W-011 + recognizer wiring) — typed refusals become curriculum candidates - **Contemplates autonomously** (W-018) — between-turn slack becomes hypothesis exploration That set of behaviors is **not what a transformer-based chatbot can do**. That's the architectural distinctiveness moment. Patent territory. --- ## Phase 7 — Validation & projection After Phase 6: - **Patent filings** — operator deferred until cleanup complete; Phase 6 is the cue - **Rust backend parity** — CLAUDE.md work-sequencing #5, was waiting on Python semantics being locked by tests; Phase 3 locks them - **Curriculum expansion** — CLAUDE.md #6; now safe because eval/replay/calibration are deterministic - **Long-context benchmarks** vs transformer baselines — NIAH probe exists at `evals/long_context_cost/`; concrete "we recall at N=100k where they hallucinate" demonstrations --- ## What "smashing expectations" honestly means Two flavors, worth distinguishing: 1. **Architectural distinctiveness** (Phase 6) — CORE does things transformers structurally can't. Replay-equivalent, identity-preserving, audit-trailed, exact-recall. **This is achievable on the path above.** 2. **Benchmark wins** (GSM8k, etc.) — possible but requires Phase 7's curriculum expansion AND a real claim on what we're testing. Cluster C in Phase 3 will tell us whether current grounding accuracy is actually competitive or whether we've been measuring with stale thresholds. **The thesis (decodes, not generates) makes (1) the legitimate target. (2) is downstream validation, not the goal. Aim at (1); (2) follows.** --- ## The one critical risk **L10 is the single point of failure.** If we commit to the wrong process shape, every Phase 5 item gets re-done. Treat L10's spike with the same rigor as the substrate-liveness audit — multi-resolution scope, agent-cross-checked, before writing any committing ADR. --- ## Cross-references - [substrate-liveness-ratchet](audit/substrate-liveness-ratchet.md) — wiring debt registry (v5) - [substrate-liveness-registry](audit/substrate-liveness-registry.md) — per-layer audit evidence - [test-debt-quarantine](test-debt-quarantine.md) — pytest QUARANTINE registry (49 tests, 5 clusters) - [L10-runtime-model-scope](decisions/L10-runtime-model-scope.md) — gates W-003, W-007, W-009 - [recognizer-storage-scope](decisions/recognizer-storage-scope.md) — gates W-007 - [substrate-liveness-audit-scope](decisions/substrate-liveness-audit-scope.md) — defines audit shape ## Memory cross-references - [[thesis-decoding-not-generating]] — load-bearing project thesis - [[feedback-adr-cross-reference-discipline]] — grep ALL existing ADRs before any new scope - [[feedback-cleanup-as-you-find]] — applies to Phase 2 W-006/W-013/W-014 - [[feedback-scope-time-is-cheap]] — applies to Phase 4 L10 - [[feedback-address-critiques-dont-waive]] — if a critique surfaces in any phase, fix don't waive - [[feedback-parallel-agent-worktrees]] — for any parallel-agent dispatch - [[feedback-use-uv-consistently]] — `uv venv` / `uv pip install` / `uv run`, never `python3 -m venv` / `--break-system-packages` - [[project-engine-identity-candidate]] — DNA-analog concept SHELVED; un-shelve trigger = L10 sub-question 3 (cross-reboot identity verification) ## Tonight's lessons (in case they surface tomorrow) 1. **Lane subprocess timeout (300s) was too tight** for any lane that takes more than ~5 min on CI. Bumped to 900s in #261's third commit. Future lane additions should consider their CI-runtime cost. 2. **pytest-xdist `-n 4` surfaces parallel-execution incompatibilities.** First example was `test_articulation_bench` (memory-RSS measurement). Cluster E is the place to track these; consider whether the gate workflow should drop to `-n 2` if more parallel-flake surfaces. 3. **Wrong-branch PR pattern** continues to hit subagents. Seen earlier with #256 (Sonnet's W-016 on wrong branch) and earlier with Gemini's L2/L3/L5/L7. Every parallel-agent brief should emphasize the rebase-onto-current-main step before PR creation. 4. **`/tmp/`** was a workaround for PEP 668 — sourced [[feedback-use-uv-consistently]]. The CI uses uv (the gate workflow already sets the precedent); the existing lane-shas workflow still uses pip and gets migrated in task #7.