70e58ce446
ADR-0020 next-level: close the parity-gate hole on the four remaining
ungated Rust surfaces.
Gates landed (subprocess-based, raw f32/f64 byte equality):
cga_inner — 14/14 bit-identical (random + basis blades + self-norm)
geometric_product — 15/15 bit-identical (random + basis blades + scalar identity)
versor_condition — 9/9 bit-identical AFTER kernel fix
versor_apply — 8/8 intentionally skipped (see below)
Kernel fix: versor_condition_raw
The Python source-of-truth (algebra.versor.versor_unit_residual) folds
the geometric product + identity subtraction + Frobenius norm in f64.
The Rust kernel was folding in f32, drifting by 1 ULP on out-of-shell
inputs. Rewrote versor_condition_raw to promote inputs to f64, use the
existing geometric_product_f64/reverse_f64 building blocks, and cast
only the final scalar back to f32. Python is canonical per CLAUDE.md
sequencing rule 5.
Honest disable: versor_apply
The Rust versor_apply_closed diverges structurally:
(1) precision — f32 sandwich vs Python's f64 throughout
(2) closure form — Rust has a null-vector early branch + no
post-unitize condition recheck; Python is the
inverse (no null branch; recheck + seed-rotor
fallback)
Per ADR-0020 "default-off until parity passes", the Rust dispatch for
versor_apply is disabled in algebra/backend.py with a pointer to the
gate. The parity tests are skipped with explicit reason. The follow-up
f64 port is documented in the ADR's new Parity status table.
Lane registration: all four parity files added to --suite algebra.
After: algebra 124 passed, 8 skipped (was 86). All other lanes green:
smoke 54, runtime 19, cognition 57, teaching 17, packs 6. Cognition
eval 100%.
186 lines
8.4 KiB
Markdown
186 lines
8.4 KiB
Markdown
# ADR-0020 — Phase 5 / Rust Parity Sequencing
|
||
|
||
**Status:** Accepted (2026-05-16)
|
||
**Date:** 2026-05-16
|
||
**Authors:** Joshua Shay
|
||
**Depends on:** ADR-0016 (Capability Roadmap), ADR-0019 (Exact
|
||
Vault Recall Acceleration), `docs/PROGRESS.md` Phase 4 exit
|
||
memo.
|
||
|
||
## Context
|
||
|
||
Phase 4 exited 2026-05-16 with three lanes shipped
|
||
(sample_efficiency, long_context_cost, multi_agent_composition)
|
||
and ADR-0019 Stage 1 vectorising vault recall. Two non-trivial
|
||
axes are now unblocked:
|
||
|
||
- **Phase 5 — Curriculum Era.** Open-ended domain acquisition
|
||
(English fluency v5 OOD, Hebrew, Koine Greek, elementary
|
||
mathematics, foundational physics/biology, classical
|
||
literature). Stresses the runtime *as it stands today* on
|
||
semantic breadth and pack scale.
|
||
- **Rust backend parity port.** CLAUDE.md sequencing rule 5:
|
||
*"Add Rust backend parity only after Python semantics are
|
||
locked by tests."* Phase 4 just locked vault recall semantics
|
||
with bit-identity tests; the prior Phase 1–3 work locked
|
||
algebra closure, intent classification, articulation, teaching,
|
||
and trace hashing. The blocker is dissolved.
|
||
|
||
The question is **what order to take these on**. Three
|
||
positions are credible:
|
||
|
||
### Option A — Phase 5 first, Rust parity later
|
||
|
||
Open Phase 5 now. Drive curriculum work on the Python runtime.
|
||
Defer Rust until Phase 5 surfaces a concrete bottleneck that
|
||
indexing/vectorisation cannot dissolve.
|
||
|
||
- **Pro:** Maximum focus on capability expansion. Phase 5 is
|
||
where CORE proves its end-goal claim (listen → comprehend →
|
||
recall → think → articulate → learn → replay) on real domains.
|
||
Every additional language / domain pack is a load-bearing
|
||
capability bet.
|
||
- **Pro:** Python is currently fast enough. Stage 1 vault
|
||
recall is ~20 ms at N=10⁵. No measured Python bottleneck
|
||
blocks Phase 5 work today.
|
||
- **Con:** Phase 5 will balloon the test surface. Re-running
|
||
Phase 1–4 lanes on every release (per the roadmap) will get
|
||
slow. CI feedback latency directly governs willingness to
|
||
refactor — a slow loop encourages unsafe shortcuts that
|
||
CLAUDE.md explicitly warns against.
|
||
- **Con:** Each new pack ships with Python-only semantics.
|
||
When Rust parity lands later, every pack's behaviour will
|
||
need re-verification against the Rust path — more locked
|
||
surface to re-lock.
|
||
|
||
### Option B — Rust parity first, then Phase 5
|
||
|
||
Port the Python backend surfaces to Rust before opening Phase 5.
|
||
Lock parity with bit-identity tests at every ported surface.
|
||
|
||
- **Pro:** Phase 5 then starts on a faster substrate. Every
|
||
curriculum eval re-run is cheaper. Faster feedback supports
|
||
the eval-driven discipline the project rests on.
|
||
- **Pro:** Locks parity at the smallest possible surface area.
|
||
Today the locked Python surface is finite and bounded; after
|
||
Phase 5 it will have grown by every pack, every curriculum,
|
||
every new operator. Porting later means porting more.
|
||
- **Con:** Rust port is itself a non-trivial project. Done
|
||
poorly it introduces a parallel backend whose drift from
|
||
Python is a constant source of incident. CLAUDE.md already
|
||
treats Rust as opt-in for a reason — `CORE_BACKEND=rust`
|
||
must remain explicit and the Python path must remain the
|
||
deterministic default.
|
||
- **Con:** Delays the capability story. No new curricula
|
||
ship until parity is done.
|
||
|
||
### Option C — Parallel: Phase 5 curriculum + Rust parity in independent tracks
|
||
|
||
Open Phase 5 on the Python runtime. In parallel, port one
|
||
backend surface at a time to Rust, gated by bit-identity tests,
|
||
without making Rust the default. Phase 5 curricula run on
|
||
Python until Rust parity is proven per-surface.
|
||
|
||
- **Pro:** Both axes progress. Phase 5 capability bets land
|
||
on schedule. Rust parity grows incrementally, surface by
|
||
surface.
|
||
- **Pro:** Bit-identity gating means a Rust regression cannot
|
||
silently corrupt Python-validated runtime behaviour. The
|
||
Rust path is purely an acceleration; the Python path remains
|
||
the source of truth.
|
||
- **Con:** Two contexts to hold. Demands discipline about
|
||
which surface is being touched at any given time.
|
||
- **Con:** Mid-Phase-5 backend swap (per-surface enablement of
|
||
Rust path) is a real operational complexity that needs
|
||
careful tooling to keep replay determinism intact.
|
||
|
||
## Recommendation
|
||
|
||
**Option C — parallel, with explicit ordering.**
|
||
|
||
Concretely:
|
||
|
||
1. **Open Phase 5.1 (English fluency v5 OOD) immediately.**
|
||
This is the natural successor to Phase 3 v2 grammatical-
|
||
coverage and to ADR-0018's articulation operators. It does
|
||
not depend on Rust.
|
||
2. **In parallel, open a Rust-parity track.** First port:
|
||
`vault_recall` — the surface ADR-0019 Stage 1 just locked
|
||
with bit-identity tests. Port is gated on byte-equal scores
|
||
and identical top-k ordering against the Python path on a
|
||
wide fixture vault. No Rust enablement on `main` until the
|
||
bit-identity test passes under `CORE_BACKEND=rust`.
|
||
3. **Second port:** `geometric_product` and `versor_apply`.
|
||
These are the hottest algebra paths; bit-identity is testable
|
||
against the existing Python closure. Locked by Phase 1–3
|
||
algebra suite.
|
||
4. **Third port:** `cga_inner` (drop-in replacement now that
|
||
the diagonal-metric kernel is the source of truth).
|
||
5. **Defer:** propagation, teaching, trace hashing — these are
|
||
Python-shaped semantics with relatively low computational
|
||
weight; port only if Phase 5 evidence demands it.
|
||
|
||
Phase 5 may also unlock the deferred scope decisions in
|
||
PROGRESS.md ("Code generation" before Phase 5, "Embodiment"
|
||
during Phase 5). Those are separate ADRs; this one only
|
||
governs sequencing.
|
||
|
||
## Decision
|
||
|
||
**Option C — parallel, with explicit ordering.** Confirmed
|
||
2026-05-16.
|
||
|
||
## Consequences
|
||
|
||
- `docs/PROGRESS.md` opens Phase 5 with "Status: IN PROGRESS"
|
||
on the same date this ADR is accepted.
|
||
- A new ADR (numbered after this one) opens to document the
|
||
Rust parity contract per-surface (test discipline, parity
|
||
gate, default-off enablement, replay determinism preservation).
|
||
- The Rust track produces no new runtime behaviour — only
|
||
faster execution of behaviour that the Python path already
|
||
validates. Any divergence is a test failure, not a feature
|
||
request.
|
||
|
||
## Parity status (live)
|
||
|
||
| Surface | Gate file | Status | Dispatch |
|
||
|----------------------|---------------------------------------------------------|-------------------|-------------------------|
|
||
| `vault_recall` | `tests/test_vault_recall_rust_parity.py` | passing | enabled |
|
||
| `cga_inner` | `tests/test_cga_inner_rust_parity.py` | passing | enabled |
|
||
| `geometric_product` | `tests/test_geometric_product_rust_parity.py` | passing | enabled |
|
||
| `versor_condition` | `tests/test_versor_condition_rust_parity.py` | passing (after f64 fold fix) | enabled |
|
||
| `versor_apply` | `tests/test_versor_apply_rust_parity.py` | **skipped** | **disabled pending f64 port** |
|
||
|
||
### Outstanding work: `versor_apply` f64 parity port
|
||
|
||
The current Rust `versor_apply_closed` diverges from Python on two
|
||
axes that the bit-identity gate exposes:
|
||
|
||
1. **Precision** — Rust folds the sandwich (V·F·rev(V)) in f32; Python
|
||
in f64.
|
||
2. **Closure structure** — Rust has a null-vector early branch and no
|
||
post-unitize `versor_condition < 1e-6` recheck; Python is the
|
||
inverse (no null branch; recheck + seed-rotor fallback).
|
||
|
||
The f64 building blocks already exist in the Rust crate
|
||
(`geometric_product_f64`, `reverse_f64`, `unitize_closed` in f64).
|
||
A focused parity port replaces `versor_apply_closed` with an f64-
|
||
throughout sandwich + a closure path mirroring Python's
|
||
`_close_applied_versor` exactly, then un-skips
|
||
`test_versor_apply_rust_parity.py` and removes the dispatch disable
|
||
in `algebra/backend.py::versor_apply`.
|
||
|
||
Until that port lands, `versor_apply` runs Python under any
|
||
`CORE_BACKEND` setting. This is the ADR's "default-off until parity
|
||
passes" discipline applied: the surface is honestly disabled, the
|
||
gate is honestly skipped, and the follow-up is documented here.
|
||
|
||
## What this ADR does NOT decide
|
||
|
||
- Which Phase 5 curriculum to open *second* (Hebrew vs.
|
||
mathematics vs. physics). Separate scope call once 5.1 ships.
|
||
- The Rust crate layout / dependency choice. That belongs in
|
||
the per-surface Rust parity ADR, not here.
|
||
- Whether to invest in a third backend (e.g., GPU / JAX). Out
|
||
of scope until both Python and Rust paths are mature.
|