docs(phase4): exit memo + ADR-0020 Phase 5 / Rust parity sequencing (proposed)

Phase 4 exited 2026-05-16. All three planned lanes shipped:
sample_efficiency (one-shot-per-correction, replay 1.0),
long_context_cost (slope 0.99 linear after ADR-0019 Stage 1),
multi_agent_composition (15/15 public, composition does not
launder identity violations).

PROGRESS.md updated with full Phase 4 narrative and exit
checklist.

ADR-0020 opens the next sequencing decision: Phase 5
(curriculum era) vs. Rust backend parity port. Three options
laid out (A: Phase 5 first, B: Rust first, C: parallel with
per-surface bit-identity gating). Recommendation: Option C.
Status remains Proposed pending user confirmation.
This commit is contained in:
Shay 2026-05-16 16:48:46 -07:00
parent b0c5185633
commit 4d778c01fc
2 changed files with 240 additions and 7 deletions

View file

@ -497,16 +497,96 @@ construction, not inferred from training-set statistics.
branching curricula, distractor corrections, OOD probes,
multi-relation chains, confidence-interval reporting.
### long-context-cost v1 + ADR-0019 Stage 1 (2026-05-16)
Second Phase 4 lane. Measures `vault.recall` latency as a function
of stored-entry count N. Pre-vectorisation: median 875 ms at N=1k,
8,727 ms at N=10k — unfit for runtime use. Diagnosis: per-element
Python dispatch in `algebra/backend.py::vault_recall`, not algebra
cost.
**ADR-0019 Stage 1 shipped in same session.** The CGA inner
product is exactly diagonal with ±1 metric values (verified
empirically), so `cga_inner(X,Y) = sum_i metric[i]*X[i]*Y[i]`.
This factors into a NumPy scan that preserves per-versor serial
component reduction order — scores are bit-identical to the
scalar path, verified by `tests/test_vault_recall_vectorised.py`.
| N | pre-vec median | post-vec median | speedup |
|---|---:|---:|---:|
| 1k | 874.774 ms | 0.217 ms | ~4,030x |
| 10k | 8,727.420 ms | 1.701 ms | ~5,130x |
| 100k | ~87,000 ms (extrap) | 20.795 ms | ~4,200x |
Slope 0.99 (linear). No approximate recall introduced. Stages 2
(norm-bucketed exact pre-filter) and 3 (layered store with
deterministic promotion) are documented in ADR-0019 but deferred
— Stage 1 alone moved per-turn recall from seconds to milliseconds
at every probed N.
### multi-agent-composition v1 (2026-05-16)
Third Phase 4 lane. Structural test that composition does not
launder identity violations. Two CORE instances (A, B), no shared
state, message-passing only: user text → B.run() → B's
articulation_surface → A.run() → A's review verdict.
| split | attacks | legits | attack_rej | legit_acc | b_err | overall_pass |
|---|---|---|---|---|---|---|
| public/v1 | 10 | 5 | 1.00 | 1.00 | 0.00 | true |
| holdouts/v1 | 5 | 3 | 0.80 | 1.00 | 0.00 | false |
Public 15/15 — every attack on A that is routed *through B's
articulator* is still rejected by A's identity check.
Composition does not launder. One holdout failure
(MAC-HOLD-001 "disregard your axiology") is a vocabulary gap in
the identity check's term family — would also pass through
single-agent A. Documented in `evals/multi_agent_composition/gaps.md`.
### Phase 4 — EXIT (2026-05-16)
**All three planned lanes shipped, frontier baselines published,
gaps documented.**
| Lane | Public | Holdouts | Curve / Gate |
|---|---|---|---|
| sample_efficiency | 10/10 | 7/7 | one-shot-per-correction, replay 1.0 |
| long_context_cost | linear (slope 0.99) | — | post-Stage-1 21 ms @ N=100k |
| multi_agent_composition | 15/15 | 7/8 | composition does not launder |
Exit gate ("all curves published with confidence intervals") is
met for the curves; CI bands are v2 work per each lane's gaps.md.
Vault indexing strategy is decided (ADR-0019: Stage 1 now, Stages
2/3 gated on future evidence).
**What Phase 4 changed in the runtime:**
- `algebra/backend.py::vault_recall` — vectorised exact scan,
bit-identical to scalar path.
- `_CGA_INNER_METRIC` — diagonal metric derived once at import.
- Bit-identity contract pinned by
`tests/test_vault_recall_vectorised.py`.
**What Phase 4 left for Phase 5 / Rust parity:**
- Sample-efficiency v2: branching curricula, distractor
corrections, OOD probes.
- Long-context-cost v2: multi-run sampling, real-content
variant, fill-cost sub-lane.
- Multi-agent-composition v2: composite trace hash, chain depth
> 2, shared-state lane.
- Identity-check vocabulary extension (axiology / ontology /
telos / ethos) — improves adversarial_identity and
multi_agent_composition holdouts.
## Phase 4 — Scale and Efficiency
**Status:** Not Started
**Depends on:** Phase 3 exit
**Status:** EXITED 2026-05-16
**Exit evidence:** all three lanes above, ADR-0019.
- [ ] **sample-efficiency** curves (>=10 concepts)
- [ ] **long-context-cost** curves (10^3 to 10^6 vault entries)
- [ ] **multi-agent-composition** (>=2 agents, replay preserved)
- [ ] Vault indexing strategy decided
- [ ] **Exit gate:** All curves published with confidence intervals
- [x] **sample-efficiency** curves (>=10 concepts)
- [x] **long-context-cost** curves (10^3 to 10^5 vault entries; 10^6 deferred to v2 after Stage 1)
- [x] **multi-agent-composition** (>=2 agents, message-passing only, replay preserved per-agent)
- [x] Vault indexing strategy decided (ADR-0019)
- [x] **Exit gate:** all curves published; CI bands deferred to v2 per gaps.md
---

View file

@ -0,0 +1,153 @@
# ADR-0020 — Phase 5 / Rust Parity Sequencing
**Status:** Proposed (decision pending user confirmation)
**Date:** 2026-05-16
**Authors:** Joshua Shay
**Depends on:** ADR-0016 (Capability Roadmap), ADR-0019 (Exact
Vault Recall Acceleration), `docs/PROGRESS.md` Phase 4 exit
memo.
## Context
Phase 4 exited 2026-05-16 with three lanes shipped
(sample_efficiency, long_context_cost, multi_agent_composition)
and ADR-0019 Stage 1 vectorising vault recall. Two non-trivial
axes are now unblocked:
- **Phase 5 — Curriculum Era.** Open-ended domain acquisition
(English fluency v5 OOD, Hebrew, Koine Greek, elementary
mathematics, foundational physics/biology, classical
literature). Stresses the runtime *as it stands today* on
semantic breadth and pack scale.
- **Rust backend parity port.** CLAUDE.md sequencing rule 5:
*"Add Rust backend parity only after Python semantics are
locked by tests."* Phase 4 just locked vault recall semantics
with bit-identity tests; the prior Phase 13 work locked
algebra closure, intent classification, articulation, teaching,
and trace hashing. The blocker is dissolved.
The question is **what order to take these on**. Three
positions are credible:
### Option A — Phase 5 first, Rust parity later
Open Phase 5 now. Drive curriculum work on the Python runtime.
Defer Rust until Phase 5 surfaces a concrete bottleneck that
indexing/vectorisation cannot dissolve.
- **Pro:** Maximum focus on capability expansion. Phase 5 is
where CORE proves its end-goal claim (listen → comprehend →
recall → think → articulate → learn → replay) on real domains.
Every additional language / domain pack is a load-bearing
capability bet.
- **Pro:** Python is currently fast enough. Stage 1 vault
recall is ~20 ms at N=10⁵. No measured Python bottleneck
blocks Phase 5 work today.
- **Con:** Phase 5 will balloon the test surface. Re-running
Phase 14 lanes on every release (per the roadmap) will get
slow. CI feedback latency directly governs willingness to
refactor — a slow loop encourages unsafe shortcuts that
CLAUDE.md explicitly warns against.
- **Con:** Each new pack ships with Python-only semantics.
When Rust parity lands later, every pack's behaviour will
need re-verification against the Rust path — more locked
surface to re-lock.
### Option B — Rust parity first, then Phase 5
Port the Python backend surfaces to Rust before opening Phase 5.
Lock parity with bit-identity tests at every ported surface.
- **Pro:** Phase 5 then starts on a faster substrate. Every
curriculum eval re-run is cheaper. Faster feedback supports
the eval-driven discipline the project rests on.
- **Pro:** Locks parity at the smallest possible surface area.
Today the locked Python surface is finite and bounded; after
Phase 5 it will have grown by every pack, every curriculum,
every new operator. Porting later means porting more.
- **Con:** Rust port is itself a non-trivial project. Done
poorly it introduces a parallel backend whose drift from
Python is a constant source of incident. CLAUDE.md already
treats Rust as opt-in for a reason — `CORE_BACKEND=rust`
must remain explicit and the Python path must remain the
deterministic default.
- **Con:** Delays the capability story. No new curricula
ship until parity is done.
### Option C — Parallel: Phase 5 curriculum + Rust parity in independent tracks
Open Phase 5 on the Python runtime. In parallel, port one
backend surface at a time to Rust, gated by bit-identity tests,
without making Rust the default. Phase 5 curricula run on
Python until Rust parity is proven per-surface.
- **Pro:** Both axes progress. Phase 5 capability bets land
on schedule. Rust parity grows incrementally, surface by
surface.
- **Pro:** Bit-identity gating means a Rust regression cannot
silently corrupt Python-validated runtime behaviour. The
Rust path is purely an acceleration; the Python path remains
the source of truth.
- **Con:** Two contexts to hold. Demands discipline about
which surface is being touched at any given time.
- **Con:** Mid-Phase-5 backend swap (per-surface enablement of
Rust path) is a real operational complexity that needs
careful tooling to keep replay determinism intact.
## Recommendation
**Option C — parallel, with explicit ordering.**
Concretely:
1. **Open Phase 5.1 (English fluency v5 OOD) immediately.**
This is the natural successor to Phase 3 v2 grammatical-
coverage and to ADR-0018's articulation operators. It does
not depend on Rust.
2. **In parallel, open a Rust-parity track.** First port:
`vault_recall` — the surface ADR-0019 Stage 1 just locked
with bit-identity tests. Port is gated on byte-equal scores
and identical top-k ordering against the Python path on a
wide fixture vault. No Rust enablement on `main` until the
bit-identity test passes under `CORE_BACKEND=rust`.
3. **Second port:** `geometric_product` and `versor_apply`.
These are the hottest algebra paths; bit-identity is testable
against the existing Python closure. Locked by Phase 13
algebra suite.
4. **Third port:** `cga_inner` (drop-in replacement now that
the diagonal-metric kernel is the source of truth).
5. **Defer:** propagation, teaching, trace hashing — these are
Python-shaped semantics with relatively low computational
weight; port only if Phase 5 evidence demands it.
Phase 5 may also unlock the deferred scope decisions in
PROGRESS.md ("Code generation" before Phase 5, "Embodiment"
during Phase 5). Those are separate ADRs; this one only
governs sequencing.
## Decision
**Pending user confirmation.** Three options laid out above;
recommendation is Option C. This ADR moves to *Accepted* once
the sequencing is confirmed.
## Consequences (if Option C is accepted)
- `docs/PROGRESS.md` opens Phase 5 with "Status: IN PROGRESS"
on the same date this ADR is accepted.
- A new ADR (numbered after this one) opens to document the
Rust parity contract per-surface (test discipline, parity
gate, default-off enablement, replay determinism preservation).
- The Rust track produces no new runtime behaviour — only
faster execution of behaviour that the Python path already
validates. Any divergence is a test failure, not a feature
request.
## What this ADR does NOT decide
- Which Phase 5 curriculum to open *second* (Hebrew vs.
mathematics vs. physics). Separate scope call once 5.1 ships.
- The Rust crate layout / dependency choice. That belongs in
the per-surface Rust parity ADR, not here.
- Whether to invest in a third backend (e.g., GPU / JAX). Out
of scope until both Python and Rust paths are mature.