docs(adr-0244): D4 implementation plan + progress tracker (resume anchor)

Comprehensive phased plan for full ADR-0244 realization (D4) plus the
ADR-0245 companion: locked design decisions (Q_top dropped from egress,
grade-1 axis lift, signed overlap, Euclidean leakage norm, frozen
identity manifold, bounded/abstaining C_id), code-verified ground truth
(Fact A dim-3 axes, Fact B two field surfaces, boundary_ids dormant),
folded-in external-review considerations, and a per-phase progress
dashboard for seamless cross-session resume after rate-limit breaks.

[Verification]: docs-only (docs/handoff/), no executable surface; smoke
deferred to Phase 0 (first phase touching governance/ADR tests).
This commit is contained in:
Shay 2026-07-17 15:17:05 -07:00
parent ee38c97617
commit 0f2b8d5e4f

View file

@ -0,0 +1,236 @@
# ADR-0244 D4 — Wave-Field Identity Manifold: Implementation Plan & Progress Tracker
**Owner:** Shay (sole worker on this arc) · **Authored:** 2026-07-17
**Arc base:** `main @ ee38c976` (== `forgejo/main`) · **Worktree:** `/Users/kaizenpro/Projects/core-adr0244d4` · **Branch:** `feat/adr-0244-d4`
**Companion ADRs:** [ADR-0244](../adr/ADR-0244-wave-field-identity-manifold-and-inalienable-geometric-alignment.md) (identity consumer) · **ADR-0245** (mechanical-sympathy + semantic-rigor foundation — *to be committed in Phase 0*)
**Prior audit:** [adr-0244-cohesion-directive-audit-2026-07-17.md](../analysis/adr-0244-cohesion-directive-audit-2026-07-17.md) · **Q_top proof:** `evals/adr_0244_qtop_vacuity/`
> **This document is the resume anchor.** It is committed to `main` and updated at every phase boundary. On a cold start: read §0 → §10 status dashboard → the first phase whose status is not ✅, then its Resume notes. Everything needed to continue is here; do not re-derive from the relays or Google-Drive docs (see §3 for why).
---
## 0. Purpose & how to resume
This is a **multi-phase, multi-commit arc** expected to span rate-limit interruptions. Progress is tracked two ways:
1. **This committed doc** (durable; survives new sessions) — the source of truth. §10 dashboard + per-phase Status/Resume notes.
2. The in-session harness task list (convenience; lost across sessions).
**Resume procedure:**
- `cd /Users/kaizenpro/Projects/core-adr0244d4` (create the worktree from `forgejo/main` if gone: see §8).
- Read §10 dashboard. Find the first phase not ✅ DONE. Read its **Resume notes**.
- Continue from there. Update Status + append to §9 Progress log at each phase boundary.
---
## 1. Scope: the two ADRs and what's already done
ADR-0244 is the **identity-layer consumer**; ADR-0245 is the **mechanical-sympathy + semantic-rigor foundation** it sits on. Measured against `main @ ee38c976`:
| ADR-0244 § | Mechanism | Status entering D4 |
|---|---|---|
| 2.1 | Gram-subspace identity projection | ❌ unbuilt (identity.py is legacy ADR-0010 heuristic) |
| 2.2 | Spectral-leakage **fail-closed** gate + `C_id` + `IdentityGateRefusal` | ❌ unbuilt as gate (call site exists at `chat/runtime.py:2679`, but **advisory** only) |
| 2.3 | `Q_top` topological charge | ✅ resolved → **retire from egress** (proven vacuous, `evals/adr_0244_qtop_vacuity`) |
| 2.4 | Bracketed-local Fibonacci search + `γ_id` calibration | ◐ search-honesty done (D3); calibration half unbuilt |
| 2.5 | Serving-boundary f64→f32 cast contract | ❌ unbuilt (== ADR-0245 §2.2; the one genuinely-absent seam) |
| 2.6 | Rust GP fast-path + parity gate | ✅ done (f32 pre-existing + f64 D2, bit-identical) |
| 2.7 | Full digests / no `default=str` / byte-order guard | ◐ hot-path done (D1); contemplation content-ids residual |
| 2.8 | `eigh` memoization | ✅ done (`_cached_eigh`, D2) |
| 2.9 | Low-discrepancy mode-centroid allocator | ✅ primitive built (`atlas_packing.py`); **adoption unwired** — needs routing |
| 2.10 | Fibonacci-word background scheduler | ✅ built + spec-satisfying (`fibonacci_word_schedule.py`); audit-confirm only |
| ADR-0245 § | Mechanism | Status entering D4 |
|---|---|---|
| 2.1 | PyO3 Rust `geometric_product` f32 fast-path | ✅ done (`algebra/backend.py`) |
| 2.2 | Gated f64→f32 serving boundary | ❌ unbuilt (== 0244 §2.5) |
| 2.3 | Semantic rigor in content addressing | ◐ hot-path done (D1); residual |
| 2.4 | `_cached_eigh` memoization | ✅ done |
| 3 | Acceptance gate (parity, ≥10× f32 speedup, 0-LAPACK-on-repeat, collision-resistance) | ◐ parity ✓, 0-LAPACK partial (`test_adr_0244_mechanical_sympathy.py`); f32-speedup + collision proofs **missing** |
**Net remaining build = ADR-0244 §2.1 + §2.2 (the identity gate rebuild), §2.4 calibration, §2.5/0245§2.2 cast, §2.7 residual, §2.9 wiring, + the ADR-0245 §3 gaps.**
---
## 2. Ratified decisions (LOCKED — Shay signed off 2026-07-17)
1. **Drop `∧ ΔQ_top = 0` from the §2.2 egress condition.** Proven vacuous (`Q_top ≡ 0` on every versor; hollow gate). Egress becomes `score ≥ threshold` (leakage-based). Scope the §2.3 conservation claim to versors only; keep `Q_top` as a diagnostic, never an admit condition.
2. **§2 governs §4.** §4's illustrative per-axis-resonance code is rewritten to the §2.1/2.2 Gram/leakage formulation. Remove the (formerly phantom) "ADR-0245" reference by pointing it at the **now-real** ADR-0245. Replace the bare `assert` byte-order guard with a typed guard.
3. **Commit ADR-0245 as a real companion ADR** (currently untracked in `~/Downloads`). Cross-link 0244 ↔ 0245.
4. **Grade-1 axis lift.** Pack `direction` vectors are **dim-3** (verified); lift them to Cl(4,1) 32-vectors by placing the 3 components at the grade-1 `e1/e2/e3` blade slots (NOT `embed_point`, which would make Gram a distance table). Orthonormal axes ⇒ `G = I`.
5. **Signed overlap** — never `abs()` the per-axis inner product; a large negative overlap is anti-alignment (opposition), tracked distinctly from orthogonality.
6. **Euclidean leakage norm**`‖S_id‖` is the positive-definite coefficient-Euclidean norm `sqrt(Σ S_id[k]²)`, explicitly **not** the indefinite `⟨S_id, reverse(S_id)⟩₀` (which can be 0 for nonzero leakage under signature (+,+,+,+,)). Projection geometry uses the indefinite metric Gram; leakage *magnitude* uses the Euclidean norm.
7. **Activate `boundary_ids`** — hard-boundary evaluation, currently dormant (`check()` iterates `value_axes` only).
8. **Frozen identity manifold** — axis eigenmodes are fixed at pack load and never mutated within a session. Biography holonomy (`H_bio ← H_bio·R`) accumulates **separately** and does not rewrite the identity subspace. This preserves inalienability-by-construction.
9. **Bounded, abstaining `C_id`** — the corrector may make a bounded corrective displacement toward the manifold; if it cannot recover alignment within the bound it **abstains** (emits `IdentityGateRefusal`, live params unchanged). It must NOT arbitrarily rewrite reasoning to force a low leakage score ("good-metric / bad-cognition" guard).
10. **Paraphrase-invariance is empirical, not automatic.** ADR wording reworded to: *"invariant under transformations that preserve the represented trajectory's identity-relevant field geometry; paraphrase robustness is measured across the encoder+propagation pipeline."* The eval must measure it, not assert it.
---
## 3. Ground-truth facts (code-verified against `main @ ee38c976`)
Verified by reading the tree, **not** the relays (which contain at least one hallucination — LanceDB — and a false "identity.py already refactored on the VM" claim; disk shows identity.py **unchanged legacy**).
- **Fact A — axes are R³ 3-vectors.** `packs/identity/default_general_v1.json` ships `direction` = dim-3 unit vectors; loader enforces `_DIRECTION_LEN`. §2.1 assumes 32-vector eigenmodes → requires the grade-1 lift (decision #4).
- **Fact B — two field surfaces, not one.** §2.2 gate runs on `result.final_state.F` (cognition-pipeline field, `chat/runtime.py:2679`). §2.5 `ψ_steady` cast lives in `core/physics/cognitive_lifecycle.py` (ADR-0243 lifecycle), which `chat/` + `core/cognition/` **do not import** (A-04 off-serve). The ADR conflates them; they are separate contracts in separate files.
- **Advisory, not a gate.** Current identity_score feeds `_build_surface_context` (hedge/claim-strength, `runtime.py:2705`) + telemetry. `would_violate` is never called in runtime. §2.2 converts advisory → fail-closed = a **live-serving behavior change** (highest risk).
- **`psi_traj` is never populated.** `_make_trajectory_from_result` (`runtime.py:385`) builds from `result.trajectory or (result.final_state,)`; no `psi_traj` attr. The wave-field 32-vector *is* available as `final_state.F` (used by `versor_condition(result.final_state.F)`). Phase 2 must thread it.
- **`boundary_ids` dormant** — stored on the manifold, never evaluated (decision #7).
- **Filename drift** — ADR-0245 + relays reference `multimodal_lifecycle.py`; it does **not** exist. The real file is `cognitive_lifecycle.py`.
- **Surrounding substrate already present**`wave_manifold.py`, `chiral_gate.py`, `trajectory_invariants.py`, `fibonacci_search.py`, `atlas_packing.py`, `fibonacci_word_schedule.py`. identity.py is the one unpromoted file.
---
## 4. Folded-in review considerations (from the external grounded critique)
All code-verified or sound; woven into the phases below.
1. Signed overlap (decision #5) → Phase 1/2.
2. Euclidean leakage norm (decision #6) → Phase 1.
3. Paraphrase-invariance empirical (decision #10) → Phase 0 wording + Phase 2 eval.
4. Bounded/abstaining `C_id` (decision #9) → Phase 2.
5. `boundary_ids` activation (decision #7) → Phase 2.
6. 5-layer inalienability framing (algebraic / runtime / pipeline / operational / semantic) → Phase 0 ADR rigor.
7. Richer identity verification suite: **ablation** (legacy-only vs wave-only vs dual — prove incremental detection value), fail-closed malformed-ψ (NaN / wrong-dim / nonfinite / missing-cert / stale-manifold), near-singular-Gram mode-basis, metamorphic, conservation → Phase 2/3.
8. `TurnEvent` telemetry: `psi_leakage_norm: float = 0.0`, `wave_mode_active: bool = False` → Phase 2.
---
## 5. Phase plan
Dependencies: `0 → {1, 4}` · `1 → 2 → 3` · `5 after 0` · `6 last`. Each phase = its own gated commit (in-worktree smoke → fast lane for code phases → push `feat/adr-0244-d4:main`, per Shay's direct-to-main authorization for this arc). No auto-merge.
### Phase 0 — Governance reconciliation (ADR edits, NO code)
**Objective:** make the spec self-consistent so nothing downstream is built on a contradiction.
**Files:** `docs/adr/ADR-0244-...md`, new `docs/adr/ADR-0245-cga-unification-mechanical-sympathy-and-semantic-rigor.md`, `docs/analysis/adr-0244-cohesion-directive-audit-2026-07-17.md`.
**Steps:**
- Commit ADR-0245 (from `~/Downloads/ADR-0245-*.md`) as **Proposed**; cross-link 0244↔0245; add a status map (0245 §2.1/2.4 done, §2.3 hot-path done, §2.2 open).
- ADR-0244 §2.2: drop `∧ ΔQ_top = 0`; scope §2.3 conservation to versors + cite the proof.
- ADR-0244 §2.1/§2.2: record signed overlap (#5), Euclidean leakage norm (#6), the leakage-ratio score `1 ‖S_id‖/‖ψ‖`, grade-1 lift (#4); rewrite §4 to Gram/leakage; typed guards + `isfinite` (no bare `assert`); point the `ADR-0245` reference at the real ADR.
- Reword paraphrase-invariance (#10); add 5-layer inalienability framing (#6); label scripture by translation + math-as-analogy disclaimer; note `boundary_ids` activation + frozen-manifold decision + Fact A/B + filename drift.
- Update audit doc: mark Q4 (Q_top) + the §4-vs-§2 blocker RESOLVED; map 0245 status.
**Acceptance:** both ADRs internally consistent; no `ΔQ_top` egress conjunct; no phantom refs; no bare `assert` in normative code blocks; provenance-guard test still green (docs-only, status stays **Proposed** — flips happen in Phase 6).
**Gate:** smoke 176 (docs-only, but run it) + provenance/ADR pins.
**Status:** ⬜ NOT STARTED
**Resume notes:** —
### Phase 1 — §2.1 Gram identity manifold (pure primitive, off-path) — TDD
**Objective:** the metric-exact projection primitive, no runtime wiring.
**Files:** new `core/physics/identity_manifold.py` (keep `identity.py` as compat shell + dual-mode host); `tests/test_adr_0244_identity_manifold.py`.
**Steps:**
- `lift_axis(direction3) → ψ_axis(32)`: grade-1 embedding at `e1/e2/e3` blade indices (VERIFY indices against `algebra.cl41` basis ordering first).
- `gram(axes) → G` (`G_ij = scalar_part(gp(ψ_i, reverse(ψ_j)))`), symmetric; `cond(G) > 1e5 → ManifoldConditioningError` (typed).
- `project(ψ, axes, Ginv) → P_id(ψ) = Σ ψ_i (G⁻¹)_ij c_j`, `c_j = ⟨ψ_j, ψ⟩₀` (**signed**).
- `leakage(ψ) = ψ P_id(ψ)`; `leakage_norm = ‖·‖₂` (Euclidean coeff norm).
**Acceptance (falsifiable):** orthonormal default pack ⇒ `G=I`; projection idempotent (`P_id∘P_id = P_id` to 1e-12); in-subspace ψ ⇒ leakage_norm ≈ 0; orthogonal ψ ⇒ leakage_norm ≈ ‖ψ‖; anti-aligned ψ ⇒ negative signed overlap detected; near-degenerate synthetic axes ⇒ `ManifoldConditioningError`. Deterministic.
**Gate:** smoke + fast lane + new tests.
**Status:** ⬜ NOT STARTED
**Resume notes:** —
### Phase 2 — §2.2 fail-closed gate + `C_id` + `boundary_ids` + telemetry + eval — TDD
**Objective:** convert advisory → fail-closed egress gate on `final_state.F`; wire in-path behind a flag.
**Files:** `core/physics/identity.py` (dual-mode `_axis_projection`/`check`, `IdentityGateRefusal`, `C_id`), `chat/runtime.py` (thread `final_state.F` as `psi_traj`; act on verdict behind config flag; `TurnEvent` fields), evals (`evals/adversarial_identity`, `teaching_injection_resistance`, `identity_divergence`), tests.
**Steps:**
- Dual-mode: `psi_traj` present → wave-field projection (Phase 1 primitive); absent → legacy ADR-0010 fallback (back-compat). `psi_traj` present but **malformed** (NaN/nonfinite/wrong-shape) → typed error, **never** silent legacy.
- Score = `1 leakage_norm/‖ψ‖`; flagged if `< manifold.alignment_threshold`; deviation_axes from signed per-axis contributions (opposition ⇒ deviation).
- `boundary_ids` hard evaluation (design the violation predicate; likely ties to safety-pack boundaries — scope in-phase).
- `C_id`: bounded corrective displacement; abstain (`IdentityGateRefusal`, params unchanged) if unrecoverable.
- Wire `runtime.py:2679` as fail-closed **behind a config flag defaulting to current advisory behavior** until Phase 3 calibrates. Add `TurnEvent.psi_leakage_norm`, `wave_mode_active`.
**Acceptance:** all three identity eval suites still pass; the gate demonstrably catches the injections; ablation shows wave path adds detection value over legacy; fail-closed tests (NaN/dim/nonfinite/missing-cert/stale-manifold) never fall through permissively; `C_id` bounded + abstains on unrecoverable; byte-identity of non-identity turns preserved (flag default off).
**Gate:** smoke + fast lane + eval suites + new tests.
**Status:** ⬜ NOT STARTED
**Resume notes:** —
### Phase 3 — §2.4 `γ_id` calibration
**Objective:** replace the hardcoded threshold with a certifiable, calibrated bound.
**Files:** calibration eval under `evals/`, uses `core/physics/fibonacci_search.py` (bracketed-local, already built); tests.
**Steps:** define a `BoundedUnimodalObjective` over reference traces (ID traces pass, adversarial fail); run the Fibonacci search; emit an audit-logged tuning certificate; pin the calibrated threshold; flip the Phase-2 flag on once evidenced.
**Acceptance:** calibrated threshold separates ID vs adversarial reference sets; certificate reproducible + deterministic; flag-on run keeps eval suites green.
**Gate:** smoke + fast lane + calibration tests.
**Status:** ⬜ NOT STARTED
**Resume notes:** —
### Phase 4 — §2.5 / ADR-0245 §2.2 serving-boundary cast (lifecycle-internal, independent)
**Objective:** governed explicit f64→f32 cast at the certified lifecycle egress.
**Files:** `core/physics/cognitive_lifecycle.py` (cast `psi_steady`→f32 at the certified boundary), tests.
**Steps:** add the typed cast contract at the egress hand-off; precision-sufficiency + parity test; keep f64 inside relaxation/eigendecomp.
**Acceptance:** f32 cast only after certification; parity within documented f32 tolerance; f64 preserved upstream; determinism intact. Satisfies **both** 0244 §2.5 and 0245 §2.2.
**Gate:** smoke + fast lane + cast tests.
**Status:** ⬜ NOT STARTED (may run parallel to 13; sequence after 0)
**Resume notes:** —
### Phase 5 — §2.7 residual + §2.9 adoption + §2.10 audit + ADR-0245 §3 gate
**Objective:** finish semantic-rigor residuals, wire the allocator, close the 0245 acceptance gate.
**Files:** `core/contemplation/schema.py`, `plan_preflight.py`, `miners/articulation_quality.py` (full 256-bit digests + typed coercion); `core/physics/holographic_vault.py` (route standing-wave-mode centroid via `atlas_packing.golden_angle_pack`, or prove existing registration insertion-order-independent); new benchmark + collision tests.
**Steps:**
- §2.7: full-digest + typed-coercion the three contemplation content-id sites.
- §2.9: wire/verify the allocator into standing-wave-mode registration.
- §2.10: audit-confirm `fibonacci_word_schedule.py` satisfies the spec (no build).
- ADR-0245 §3: add f32 ≥10× speedup benchmark + `_content_id` collision-resistance proof (parity + 0-LAPACK already covered).
**Acceptance:** no truncated machine merge keys in contemplation content-ids; allocator insertion-order-independent + wired; 0245 §3 four assertions all green.
**Gate:** smoke + fast lane + new tests.
**Status:** ⬜ NOT STARTED
**Resume notes:** —
### Phase 6 — Close-out
**Objective:** acceptance packets + ratified status flips + memory + cleanup.
**Files:** `docs/audit/adr-0244-acceptance-packet-*.md`, `docs/audit/adr-0245-acceptance-packet-*.md`; ADR status flips; memory.
**Steps:** D10-pattern acceptance packets for **both** ADRs; **user-ratified** Proposed→Accepted flips (provenance guard: inline `**Accepted** — ratified by Joshua Shay <date>` + packet `## 8. RULING RECORD`); update memory; delete worktree + branch (merge-then-cleanup).
**Acceptance:** both packets complete; provenance-guard green; status flips only on explicit user ratification; worktree/branch swept.
**Gate:** smoke + provenance/ADR pins.
**Status:** ⬜ NOT STARTED
**Resume notes:** —
---
## 6. Out-of-scope backlog (do NOT build in D4)
Forward-looking mechanical-sympathy items from the critique — a separate optimization backlog, not ADR-0244/0245:
- SIMD 64-byte alignment of the Cl(4,1) array.
- Relaxation-propagator precompute cache.
- Columnar `TurnEvent` audit storage.
- `biography.py` holonomy versioned storage.
- Parallel Fibonacci branch exploration.
(Gap-3 psi_digest byte-order is **already** handled by D1's LE guard.)
---
## 7. Open questions
- ✅ **Q_top vacuity** — RESOLVED (proven, `evals/adr_0244_qtop_vacuity`).
- ✅ **§4-vs-§2 reconciliation** — RESOLVED (decision #2; §2 governs).
- ✅ **Identity-continuity (mutate vs frozen)** — RESOLVED (decision #8; frozen manifold).
- ⚠️ **`boundary_ids` violation predicate** — how a trajectory "violates" a stored boundary id is undefined; design in Phase 2 (likely ties to safety-pack boundaries).
---
## 8. Workflow & env notes
- **Worktree:** `/Users/kaizenpro/Projects/core-adr0244d4`, branch `feat/adr-0244-d4`, based on `ee38c976` (== forgejo/main). Recreate if gone: `git worktree add -b feat/adr-0244-d4 /Users/kaizenpro/Projects/core-adr0244d4 forgejo/main` (fetch forgejo first).
- **Push target:** `forgejo` remote; land phases via `git push forgejo feat/adr-0244-d4:main` (Shay authorized direct-to-main for this sole-worker arc), then ff local `main`. Forgejo→GitHub `origin` auto-mirrors.
- **Local-first CI gates (per phase, before push):** in-worktree smoke `uv run core test --suite smoke -q` (176, doctrinal merge gate) → for code phases, fast lane `uv run python -m pytest -n auto -m "not quarantine and not slow"` (~910 min). `[Verification]:` line in every commit body.
- **Worktree venv:** real (not symlinked). First code phase: `uv sync` in the worktree. `core_rs` build (if algebra touched): `uv run --project /Users/kaizenpro/Projects/core-adr0244d4 maturin develop --release` (plain `VIRTUAL_ENV`+`PYO3_PYTHON` trips maturin cross-compile detection).
- **A-04 serve quarantine:** new capability code goes in `evals/`; in-place core refactors keep the off-serve quarantine (no imports into `chat/runtime.py` from off-serve modules).
- **No auto-merge / no self-Accept.** Status flips only on explicit Shay ratification.
---
## 9. Progress log (append-only)
- **2026-07-17** — Arc kicked off. Plan doc authored + committed to `main`. Worktree `core-adr0244d4` created off `ee38c976`. Both sign-offs recorded (§2). Next: Phase 0.
---
## 10. Status dashboard
| Phase | Objective | Status | Landed at |
|---|---|---|---|
| 0 | Governance reconciliation (ADR-0244 edits + commit ADR-0245) | ⬜ NOT STARTED | — |
| 1 | §2.1 Gram identity manifold primitive | ⬜ NOT STARTED | — |
| 2 | §2.2 fail-closed gate + boundary_ids + C_id + telemetry + eval | ⬜ NOT STARTED | — |
| 3 | §2.4 γ_id calibration | ⬜ NOT STARTED | — |
| 4 | §2.5 / 0245 §2.2 serving-boundary cast | ⬜ NOT STARTED | — |
| 5 | §2.7 residual + §2.9 wiring + §2.10 audit + 0245 §3 gate | ⬜ NOT STARTED | — |
| 6 | Close-out (2 acceptance packets + 2 ratified flips) | ⬜ NOT STARTED | — |
**▶ NEXT: Phase 0 — Governance reconciliation.**