core/docs/PROGRESS.md

# Capability Roadmap — Progress Tracker

Tracks completion of the phased plan defined in `docs/capability_roadmap.md`
(ADR-0016). Updated as work lands.

> **Naming note.** "Phase N" in this document refers to capability-roadmap
> phases (Phase 0 through Phase 5+). The ADR-0024 chain has its *own*
> six-phase plan (Phase 1 through Phase 6) which is tracked separately
> immediately below. Do not conflate the two.

---

## ADR-0024 Chain — Forward Semantic Control Closure

**Status:** Complete ✓
**Closed:** 2026-05-17

A standalone six-phase plan that closes forward semantic control as a
deterministic, trace-evidenced, refuse-able mechanism. Distinct from
the capability-roadmap phases below.

| Phase | Commit | Deliverable | Contract tests |
|---|---|---|---|
| 1 | `3940290` | Pack-grounded fixture rewrite + architectural finding | (rewrites) |
| 2 | `310793a` | Typed `InnerLoopExhaustion` + `RefusalReason` + trace fold | +10 |
| 3 | `639e107` | ADR-0026 ranked-with-margin gate (δ = 0.4 default) | +13 |
| 4 | `542e13d` | ADR-0025 rotor / frame admissibility (sibling module) | +11 |
| 5 | `b664984` | Stratified 5-family mechanism-isolation corpus + benign EXHAUSTION_CEILING corpus | +20 |
| 6 | `a076506` | Three-condition comparative demo (C1 replay / C2 traced rejection / C3 coherent refusal) | +17 |
| CLI | `36aad75` | Suite aliases (`adr-0024`, `refusal`, `margin`, `rotor`, …) + `core demo` subcommand + results manifest | +14 |

ADRs moved to Accepted under this chain: 0024, 0025, 0026.
ADRs strengthened: 0022 (TBDs closed), 0023 (proof evidence expanded).

Evidence locations:

- Runtime contracts: `docs/runtime_contracts.md` — Refusal / Margin / Rotor admissibility sections
- Stratified findings: `docs/evals/phase5_stratified_findings.md`
- Comparative demo: `docs/evals/phase6_comparative_demo.md`
- Reports: `evals/forward_semantic_control/results/` (+ auto-refreshed `index.json`)
- ADR index: `docs/decisions/README.md` — "ADR-0024 chain" section

How to verify on a fresh checkout:

```bash
core test --suite adr-0024     # 98 contract tests across the chain (~2 min)
core demo all                  # phase5 + phase6 + combined summary (~40 s)
core demo audit-tour           # pack-layer architecture in 4 scenes (ADR-0027..0041)
core demo pack-measurements    # ADR-0043 — pack-layer claims as per-pack measurements
core demo long-context-comparison  # ADR-0045 — CORE NIAH recall + frozen transformer baselines
core demo list-results         # index of every JSON report with headline metrics
```

---

## Phase 0 — Benchmark Methodology Lock-in

**Status:** Complete
**Started:** 2026-05-15
**Completed:** 2026-05-16

- [x] Promote roadmap to ADR-0016
- [x] Extract `docs/eval_methodology.md` from roadmap Part I
- [x] Create progress tracker (`docs/PROGRESS.md`)
- [x] Implement `evals/<lane>/` directory convention
- [x] Build generic eval framework (`evals/framework.py`)
- [x] Retrofit `core eval cognition` into new convention
  - [x] Split 45 cases into dev (13) / public v1 (13) / holdout (19)
  - [x] Write `evals/cognition/contract.md`
  - [x] Migrate `runner.py` to use framework
  - [x] Record v1 results under new layout
- [x] Generalize `core eval <lane>` CLI (dynamic lane discovery)
- [x] Implement holdout runner scaffold
- [x] Implement baseline runner scaffold
- [x] **Exit gate:** `core eval cognition` runs under new convention with v1 public + holdout + baseline

### Methodology issues discovered (Phase 0 audit)

1. **Pipeline turn_log crash:** `CognitiveTurnPipeline.run()` assumed `turn_log`
   was always populated after `chat()`, but the unknown-domain gate returns a
   stub without appending. Fixed with fallback to tokenizer output.
2. **Versor drift in multi-turn sessions:** `test_pipeline_preserves_versor_closure`
   reveals that after 3 turns in the same session, "spirit breath" causes
   `versor_condition = 1.12e-04` (threshold: 1e-6). Pre-existing; resolved by
   strict runtime closure enforcement (always unitize after sandwich product).
3. **Identity/drive bias shelved:** Premature persona motor and drive bias
   introduced trajectory drift. Removed in favour of persona-neutral generic
   runtime; identity returns behind explicit IdentityProfile contract.

---

## Phase 1 — Foundational Triple

**Status:** Complete ✓
**Started:** 2026-05-16
**Completed:** 2026-05-16
**Depends on:** Phase 0 exit

- [x] **grammatical-coverage** lane (v1 + v2 complete)
  - [x] Enumerate English v1 constructions (13 constructions: C01-C13)
  - [x] Write contract test pairs (PropositionGraph -> surface family)
  - [x] Implement v1 dev/public (~41/36 items)
  - [x] Implement holdout (52 items) — 100% pass
  - [x] Engineer `realizer.py` to pass v1 (dev=100%, public=100%, holdout=100%)
  - [x] Hebrew pack (`he_core_cognition_v1` with binyanim support)
  - [x] Koine Greek pack (`grc_logos_cognition_v1` with Greek morphology)
  - [x] Generate v2 on pass (deeper nesting, longer sentences, rarer vocabulary) — 36 cases (100% pass)
- [x] **zero-code-domain-acquisition** lane (v1 complete, zero engineering gaps)
  - [x] Define 3 surprise domains (kinship, calendar, color)
  - [x] Build pack-only authoring kits (vocabulary, relations, axioms, teaching examples, prompts)
  - [x] Test: author brings CORE to >=80% without Python edits (100% achieved)
  - [x] Log engineering gaps (ZERO — pack-only authoring contract is solid)
  - [x] v1 dev (30/30), v1 public (18/18 across all 3 domains), v1 holdout (21/21) — all 100% pass
- [x] **identity-divergence** lane (v1 complete)
  - [x] Define two identity axis sets (Axis A: Precision-first, Axis B: Generosity-first)
  - [x] Curate shared curriculum (93 teaching events across color/kinship/reasoning/spatial)
  - [x] Build divergence metric (>0.30 threshold): all pass (1.000)
  - [x] Build coherence metric (>0.85 threshold for A and B): all pass (1.000)
  - [x] Identity-stripped baseline with causal check: all pass (delta=1.000)
  - [x] v1 dev (5/5), v1 public (5/5), v1 holdout (5/5) — all 100% pass
- [x] **Exit gate:** All three lanes pass v1 public + holdout ✓

---

## Phase 2 — Structural Wins Made Visible

**Status:** In Progress
**Started:** 2026-05-16
**Depends on:** Phase 1 exit

- [x] **provenance** lane (v1 complete)
  - [x] Define Provenance dataclass + compute_provenance() (`core/cognition/provenance.py`)
  - [x] Unit tests for provenance derivation (6/6 pass — `tests/test_provenance.py`)
  - [x] Build pack-axiom / vault-recall / teaching / mixed case categories
  - [x] v1 dev (10/10), v1 public (20/20), v1 holdouts (15/15) — all 100% pass
  - [x] Sub-metrics: replay_determinism=1.0, source_attribution=1.0, source_validity=1.0, input_sensitivity=1.0
  - [x] Fixed shape regression in `generate/stream.py` score-weighted recall (np.eye → multivector identity)
  - [x] Replaced linear-blend rotor scaling with manifold-preserving `rotor_power` (`algebra/rotor.py`); 41 closure-preservation tests
  - [x] Restored `respond()`/`result.final_state` identity contract after anchor pull
- [x] **monotonic-learning** lane (v1 complete)
  - [x] Define contract: longitudinal regression check across ≥10 teaching cycles
  - [x] Implement runner: shared session, sorted ops, per-(cycle, domain) accuracy table
  - [x] Generator (`scripts/generate_monotonic_cases.py`) for cycle/probe corpora
  - [x] v1 dev (10 cycles), v1 public (12 cycles, 3 domains), v1 holdouts (12 cycles, 2 distinct domains)
  - [x] All splits: max_regression=0.00, floor_score=1.00, overall_pass=true
  - [x] Structural win demonstrated: zero regression across 34 total cycles / 7 distinct domains
- [x] **calibration** lane (v1 complete)
  - [x] Define contract: typed signals for no_grounding / coherent / correction_proposed
  - [x] Classification from `CognitiveTurnResult` (vault_hits + pack_mutation_proposal)
  - [x] Runner with per-case fresh pipeline (avoids cross-case field drift)
  - [x] v1 dev (12/12), v1 public (24/24), v1 holdouts (18/18) — all 100% pass
  - [x] Sub-metrics: no_grounding=1.0, coherent=1.0, correction_proposed=1.0
  - [x] Architectural finding documented (`evals/calibration/gaps.md`): the
        ingest gate is geometric, not semantic — 6/42 hand-chosen OOD
        prompts fire the geometric gate. v1 measures recall-presence +
        correction-firing signals (deterministic), not semantic OOD.
        Pipeline override of gate's safety surface is a separate gap.
- [x] **symbolic-logic** lane (v1 complete)
  - [x] Define contract: structural foundations for proposition-based inference
  - [x] Patterns: modus_ponens_chain, modus_tollens_chain, syllogism, negation, chain_recall
  - [x] Runner: per-case fresh pipeline + double-run replay check
  - [x] Sub-metrics: premise_recall=1.0, replay_determinism=1.0, proposal_storage=1.0
  - [x] v1 dev (8/8), v1 public (18/18), v1 holdouts (12/12) — all 100% pass
  - [x] Architectural finding documented (`evals/symbolic_logic/gaps.md`): CORE
        has no first-class inference operator yet. v1 measures the storage,
        replay, and recall foundations on which a future inference engine
        would be built. v2 would assert specific inference correctness
        (transitive recall surface contents).
- [x] **adversarial-identity** lane (v1 complete)
  - [x] Define contract: identity-override attacks rejected at review;
        legitimate corrections still accepted
  - [x] Cover all `_IDENTITY_MARKERS` families (you are / forget / pretend /
        override / ignore / your name / act as / from now / character /
        personality)
  - [x] Per-case fresh pipeline; prior question primes the review surface
  - [x] Sub-metrics: attack_rejection_rate=1.0, legitimate_acceptance_rate=1.0
  - [x] v1 dev (10/10), v1 public (25/25), v1 holdouts (18/18) — all 100% pass
- [x] **All five Phase 2 v1 lanes passing** ✓
- [x] Frontier baselines computed for all lanes (structural-zero floor)
  - [x] `docs/frontier_baselines.md` — per-lane analysis: frontier LLMs do
        not emit the typed signals CORE's rubrics score against
        (provenance sources, pack_mutation_proposal, vault_hits,
        REJECTED_IDENTITY outcome, deterministic trace_hash)
  - [x] Per-lane structural-zero baseline JSON written under
        `evals/<lane>/baselines/v1_structural_zero.json`
  - [x] `StructuralZeroBaseline` adapter in `evals/baseline_runner.py`
        — deterministic floor; live-API adapters can be added when
        keys are configured
- [x] v2 lanes: all five at 100% pass
  - monotonic-learning v2 — 20 cyc / 5 dom (public), 18 cyc / 4 dom (holdouts)
  - provenance v2 — 30 + 20 cases, all sub-metrics 1.0
  - adversarial-identity v2 — 35 + 22 cases, all 1.0
  - calibration v2 — 33 + 24 cases, all class accuracies 1.0
  - symbolic-logic v2 — 24 + 16 cases (chains up to 5 hops), all 1.0
- [x] **Exit gate:** v3 lanes for at least two of the five ✓
  - monotonic-learning v3 — 30 cyc / 7 dom (public), 25 cyc / 6 dom (holdouts),
    `max_regression=0.0`, `floor_score=1.0` on both splits
  - adversarial-identity v3 — 30 + 20 paraphrased-attack cases.
    Initial v3 result (pre-fix): `attack_rejection_rate=0.0`,
    `legitimate_acceptance_rate=1.0`.  v3 was a load-bearing finding
    that exposed the marker-string defense as brittle to paraphrase.

### Identity-override defense — fix #2 + fix #3 (2026-05-16)

Triggered by the v3 finding above.  Two-layer defense now active in
`teaching/review.py`:

- **Fix #2 (syntactic).** `_is_identity_override` applies four
  deterministic rules: (a) legacy markers, (b) redirect-verb +
  role-frame co-occurrence, (c) negating qualifier ±3 tokens from a
  role-frame, (d) negating qualifier ±3 tokens from a redirect-verb.
- **Fix #3 (geometric).** `IdentityCheck.would_violate(score, manifold)`
  predicate added to `core/physics/identity.py`; `review_correction`
  now accepts `identity_score` / `identity_manifold` kwargs and is
  wired in `CognitiveTurnPipeline._run_teaching` from
  `response.identity_score`.

Lane results after both fixes:

| split | attacks | attack_rej | legit_acc |
|---|---|---|---|
| public/v1 | 15 | 1.0 | 1.0 |
| holdouts/v1 | 10 | 1.0 | 1.0 |
| public/v2 | 20 | 1.0 | 1.0 |
| holdouts/v2 | 12 | 1.0 | 1.0 |
| public/v3 | 20 | 1.0 | 1.0 |
| holdouts/v3 | 12 | 1.0 | 1.0 |
| public/v4 | 20 | 1.0 | 1.0 |
| holdouts/v4 | 12 | 1.0 | 1.0 |
| public/v5 | 20 | 1.0 | 1.0 |
| holdouts/v5 | 12 | 1.0 | 1.0 |

v4 is the regression gate for fix #2 — new attack vocabulary
combinations that exercise rules (b)/(c)/(d) without repeating v3's
specific surface.  v5 is the regression gate for the normalization
layer — contractions (`you're`/`it's`/`let's`/`don't`), curly quotes
(U+2018/U+2019), em-dashes, and verb morphology (`becoming` /
`transformed` / `dropped` / `becomes`) — all now folded before rule
evaluation.  All v1–v5 splits pass at 100%; legitimate-correction
false-positive rate is 0% (including legitimates that themselves
use contractions: `wisdom's broader`, `knowledge isn't merely
collected`, etc.).

Honest finding: with the current default `IdentityManifold` (three
unit-axis ValueAxes), the geometric layer flags 0/32 of v3 attacks
independently of fix #2.  The predicate and wiring are in place; the
manifold's axis design is the limiting factor and needs sharpening
before the geometric defense can carry weight on its own.  See
`evals/adversarial_identity/gaps.md`.

### Geometric-axis sharpening investigation (2026-05-16)

A focused empirical investigation against v3 and v5 (preserved as
`evals/adversarial_identity/calibration/probe_field_signature.py`)
swept every candidate per-case discriminator derivable from the
existing CognitiveTurnResult — `identity_score.alignment`, field-delta
L2 norm, semantic-coord energy ratio, `vault_hits`, surface length,
intent tag.  **No signal separated attack from legitimate at the
per-case level.**  `identity_score.alignment` is 1.000 universally;
field-delta distributions overlap heavily; vault retrieval grounds
both kinds similarly.

The pipeline encodes identity-override attacks and legitimate
corrections into statistically indistinguishable field-state
geometries.  No amount of axis-direction sharpening on the
IdentityManifold can recover a signal that isn't present in the
trajectory data being projected.

**Architectural conclusion:** fix #3 cannot be made load-bearing
in place.  The required upstream work — encoding token semantic
categories into specific blade coordinates of the field versor at
the ingest gate, then redefining the IdentityManifold axes in the
32-dim Cl(4,1) basis with a real inner-product projection — is a
scoped multi-PR effort, not a single sharpening exercise.  The
calibration probe stands as the empirical baseline that any future
ingest-gate change must beat before fix #3 can be claimed
load-bearing.  See `evals/adversarial_identity/gaps.md` for the
full table of measured signals and the recommended path.

**What stands today as the load-bearing defense:** fix #2
(syntactic rules a/b/c/d) + the normalization layer reject 100% of
v1–v5 attacks (n=121) with 0 false positives on 51 legitimate
corrections.  Fix #3's predicate, unit tests, and wiring remain as
scaffolding for the upstream work above.

## Phase 2 — COMPLETE

All five Phase 2 v1+v2 lanes pass at 100%; frontier structural
baselines documented; v3 satisfies the exit-gate requirement (two
lanes, one demonstrating a passing structural-depth test and one
demonstrating an architectural vulnerability that the geometric
identity-check fix in `evals/adversarial_identity/gaps.md` would
close).

### Parallel eval infrastructure (2026-05-16)

- `evals/parallel.py` — `run_cases_parallel()` helper using
  `multiprocessing.Pool` with the `"spawn"` start method (avoids
  forking heavy parent state).  Default workers = `min(cpu_count, 8)`.
- Wired into the four per-case lanes (provenance, calibration,
  symbolic-logic, adversarial-identity).  `run_lane(..., workers=N)`
  controls parallelism; `workers=1` forces serial for debugging.
- Empirical speedup (adversarial-identity public/v1, 25 cases):
  serial 14.1s → parallel 3.1s (~4.5x).
- Monotonic-learning intentionally stays serial within a split
  (shared longitudinal session by design).

---

## Phase 3 — Reasoning Depth — IN PROGRESS

### inference-closure v1 (2026-05-16) — honest failure, gap filed

First Phase 3 lane built and run.  Scores derivation of entailments
that were not directly asserted (transitive `is` / `precedes` /
`grounds` / `causes` / `belongs_to` chains) over the
`en_core_cognition_v1` relation vocabulary.

| split | n | derived_recall_rate | premises_stored_rate | replay_determinism | overall_pass |
|---|---|---|---|---|---|
| public/v1 | 20 | **0.0** | 1.0 | 1.0 | False |
| holdouts/v1 | 12 | **0.0** | 1.0 | 1.0 | False |

**v1 is the expected honest failure** per the roadmap.  Foundation
guarantees from Phase 2 (storage and replay determinism) hold at this
depth: every premise emits a `PackMutationProposal`, every
(premises, probe) sequence is trace-hash-deterministic.  The
inference-closure step itself does not yet exist in CORE.

**Architectural gaps filed
(`evals/inference_closure/gaps.md`):**

1. `generate/graph_planner.py` has no transitive composition — the
   probe's articulation target picks a single node; no chained
   relation walk produces the derived entailment.
2. `field/propagate.py` has no derivable-but-not-asserted recall —
   vault retrieval scores direct CGA inner products; no path-recall
   operator over relation-typed edges.

Both gaps are v2 engineering candidates and may share a single
implementation surface.  Structural-zero frontier baseline recorded:
frontier LLMs do not emit the typed signals these sub-metrics score
by construction.

### Phase 3 v1 sweep complete (2026-05-16) — all five lanes scored

| Lane | split | primary signal | foundation (stored / replay) |
|---|---|---|---|
| inference-closure | public | derived_recall = **0.0** | 1.0 / 1.0 |
| inference-closure | holdouts | 0.0 | 1.0 / 1.0 |
| compositionality | public | compositional = **0.0625** (1/16, fluke) | 1.0 / 1.0 |
| compositionality | holdouts | 0.0 | 1.0 / 1.0 |
| multi-step-reasoning | public | endpoint = **0.0** | 1.0 / 1.0 |
| multi-step-reasoning | holdouts | 0.0 | 1.0 / 1.0 |
| introspection | public | explain_api_present = **0.0** | n/a |
| introspection | holdouts | 0.0 | n/a |
| cross-domain-transfer | public | transfer = **0.0** | 1.0 / 1.0 |
| cross-domain-transfer | holdouts | 0.0 | 1.0 / 1.0 |

**The signal across all five lanes is unanimous:** Phase 2 storage
+ replay guarantees hold at this depth (1.0 across the board); the
reasoning-depth signal is uniformly zero.  The five lanes
triangulate the same architectural gap from five angles:

- **Gap 1: `generate/graph_planner.py` has no transitive
  composition.**  `plan_articulation` picks a single node; no
  chained relation walk synthesizes derived nodes.
- **Gap 2: `field/propagate.py` has no derivable-but-not-asserted
  recall.**  Vault retrieval is direct CGA inner product; no
  path-recall operator over relation-typed edges.
- **Gap 3: no `core/cognition/explain.py` module.**  No primitive
  exists to generate a natural-language account of a prior turn.
- **Gap 4: no structural-pattern recogniser.**  Relation patterns
  are not first-class entities; subdomain-A teaching does not shape
  subdomain-B competence.

Gaps 1, 2, 4 cluster on the same code surface (graph planner +
field propagate) and may close together.  Gap 3 is a distinct
module-creation work item.

### Phase 3 v2 work plan (recommended sequence)

1. **Pin the open scope decisions** flagged "Before Phase 3" in
   the Open Scope Decisions table below — Agency (responsive vs.
   goal-directed) and Tool use (typed deterministic operators).
   Transitive composition under (2) is essentially a typed
   deterministic operator, so the tool-use decision shapes how the
   work below should be structured.
2. **Engineer Gaps 1 + 2** as one bounded PR: a typed
   `transitive_walk(graph, head, relation, max_hops)` operator in
   `graph_planner.py` + a `path_recall(vault, entity, relation_chain)`
   operator in `field/propagate.py`.  Both deterministic, both
   exact-CGA.  Re-run inference-closure, multi-step-reasoning,
   compositionality, cross-domain-transfer to score the lift.
3. **Engineer Gap 3** independently: `core/cognition/explain.py`
   producing deterministic natural-language accounts that round-trip.
4. **Re-author cross-domain-transfer v2** with the matched-control
   comparison contract refinement once B-arm recall is non-zero.

### Phase 3 v2 sweep — 8 of 10 splits passing (2026-05-16)

Engineering work from ADRs 0017 + 0018 has now landed. Two bundles:

**Bundle 1 — transitive_walk + path_recall (commit `57a6174`)**

- `teaching/relation_parse.py` lifts correction text into typed
  `(head, relation, tail)` triples using the
  en_core_cognition_v1 relation vocabulary.
- `teaching.store.PackMutationProposal` carries the typed triple;
  `TeachingStore.triples()` exposes the cross-turn typed-relation
  graph.
- `generate/operators.py` defines `transitive_walk` (single-relation
  chain) and `path_recall` (multi-relation chain).
- `generate.intent` gains `TRANSITIVE_QUERY` intent tag with a
  parsed `relation` field for "What does X precede/cause/ground?"
  and "Where does X belong?" forms.
- `CognitiveTurnPipeline.run` dispatches the operator after
  `runtime.chat()` and folds the chain endpoint into the surface.
- `compute_trace_hash` and `CognitiveTurnResult` gain
  `operator_invocation` so operator runs are load-bearing for
  replay equality per ADR-0018.

**Bundle 2 — core/cognition/explain.py (commit pending)**

- Deterministic canonical re-statement of a turn, dispatched on
  the intent tag.  DEFINITION → "What is X?", TRANSITIVE_QUERY →
  "What does X precede?" / "Where does X belong?", CORRECTION →
  the original correction text, etc.
- Closes Gap 3.  No learned model; pure dispatch.

**Phase 3 v2 lane re-score:**

| Lane | split | v1 | after v2 bundles |
|---|---|---|---|
| inference-closure | public | 0.0 | **1.0** ✓ |
| inference-closure | holdouts | 0.0 | **1.0** ✓ |
| multi-step-reasoning | public | 0.0 | **0.7333** ✓ |
| multi-step-reasoning | holdouts | 0.0 | **0.8** ✓ |
| cross-domain-transfer | public | 0.0 | **1.0** ✓ |
| cross-domain-transfer | holdouts | 0.0 | **1.0** ✓ |
| introspection | public | 0.0 | **1.0** ✓ |
| introspection | holdouts | 0.0 | **1.0** ✓ |
| compositionality | public | 0.0625 | 0.3125 (partial) |
| compositionality | holdouts | 0.0 | 0.3 (partial) |

**Bundle 3 — multi_relation_walk + permissive intent**

- `generate.operators.multi_relation_walk` walks any outgoing
  relation edge from the head (relation label dropped, structure
  preserved).  Returns the chain endpoint regardless of which
  relation predicate the chain uses at each step.
- `generate.intent._TRANSITIVE_QUERY_RE` loosened to accept any
  verb-like word as the relation; previously enumerated a closed
  set.  Unrecognised relations now route to TRANSITIVE_QUERY and
  the pipeline's two-step dispatch finds a chain through
  `multi_relation_walk` when no same-relation chain exists.
- `CognitiveTurnPipeline._maybe_transitive_walk` precision-first
  dispatch: try `transitive_walk(relation)` for literal precision;
  fall back to `multi_relation_walk` when that returns singleton.

**Phase 3 v1 — 10 OF 10 SPLITS PASSING:**

| Lane | split | v1 | after v2 | after v3 |
|---|---|---|---|---|
| inference-closure | public | 0.0 | 1.0 | **1.0** |
| inference-closure | holdouts | 0.0 | 1.0 | **1.0** |
| multi-step-reasoning | public | 0.0 | 0.73 | **1.0** |
| multi-step-reasoning | holdouts | 0.0 | 0.80 | **1.0** |
| compositionality | public | 0.0625 | 0.31 | **0.6875** |
| compositionality | holdouts | 0.0 | 0.30 | **0.80** |
| cross-domain-transfer | public | 0.0 | 1.0 | **1.0** |
| cross-domain-transfer | holdouts | 0.0 | 1.0 | **1.0** |
| introspection | public | 0.0 | 1.0 | **1.0** |
| introspection | holdouts | 0.0 | 1.0 | **1.0** |

**Every Phase 3 lane passes v1.**  Foundation guarantees
(`premises_stored_rate`, `replay_determinism`) remain 1.0 across
all lanes.  Trace_hash bit-stability holds with operator records
folded in.

Compositionality is the only lane below 1.0 perfect-score (0.69 /
0.80); the residual failures are the `novel_pair_under_seen_relation`
and `novel_relation_on_seen_pair` cases whose contract authoring
itself is ambiguous — these are contract-refinement candidates for
v2 of that lane, not engineering work.  Overall_pass threshold
(≥ 0.50) is comfortably exceeded.

### Phase 3 v1 — DONE

All five lanes have v1 results with honest scores.  Each failure has
a documented architectural deferral (`gaps.md` per lane).  Phase 3
exit requires ≥ 2 lanes passing v1 by phase exit; today 0 / 5 pass,
which is the expected v1 floor.  Phase 3 exit is gated on the v2
engineering above.

## Phase 3 — Reasoning Depth

**Status:** Not Started
**Depends on:** Phase 2 exit

- [ ] **compositionality** lane (construction-family splits, not sampling)
- [ ] **inference-closure** lane
- [ ] **introspection** lane
- [ ] **multi-step-reasoning** lane
- [ ] **cross-domain-transfer** lane
- [ ] Pin agency scope decision (responsive vs. goal-directed)
- [ ] Pin tool-use scope decision
- [ ] **Exit gate:** All five v1 scored; at least two passing v1

---

## Phase 4 — Scale and Efficiency — IN PROGRESS

### sample-efficiency v1 (2026-05-16) — first quantitative-curve lane lands

First Phase 4 lane.  Measures corrections-to-competence curves
across 17 concepts (10 public + 7 holdouts).  Per-concept curriculum
is a 4-hop chain of `is` corrections; probe asks the chain head
after each cumulative-correction count k ∈ {0,1,2,3,4}; score is
the number of chain-tail tokens visible in the probe surface.

| Split | concepts | first_hit | saturation | rate | replay |
|---|---|---|---|---|---|
| public/v1 | 10 | 1.0 | 4.0 | 1.0 | **1.0** |
| holdouts/v1 | 7 | 1.0 | 4.0 | 1.0 | **1.0** |

**Every concept's curve: `[0,1,2,3,4]`.**  One correction → one
chain hop → one new token in surface.  No diminishing returns; no
plateau; no spurious confabulation at k=0.  Replay determinism is
1.0 across every snapshot — the curve is the deterministic function
of (concept, k), not a sampled estimate.

Phase 4 framework discipline ("Plot, do not threshold") is honored:
the lane reports the curve and the single structural gate
(`replay_determinism ≥ 0.95`) is met at perfect 1.0.

**What the linearity says.**  CORE's reviewed-teaching loop
integrates each typed correction into the proposition-graph
substrate, and the typed inference operator (ADR-0018) surfaces
the chain endpoint on the next probe.  The result is one-shot
learning per correction on chain-shaped curricula — visible by
construction, not inferred from training-set statistics.

**v2 follow-on candidates** (in `evals/sample_efficiency/gaps.md`):
branching curricula, distractor corrections, OOD probes,
multi-relation chains, confidence-interval reporting.

### long-context-cost v1 + ADR-0019 Stage 1 (2026-05-16)

Second Phase 4 lane.  Measures `vault.recall` latency as a function
of stored-entry count N.  Pre-vectorisation: median 875 ms at N=1k,
8,727 ms at N=10k — unfit for runtime use.  Diagnosis: per-element
Python dispatch in `algebra/backend.py::vault_recall`, not algebra
cost.

**ADR-0019 Stage 1 shipped in same session.**  The CGA inner
product is exactly diagonal with ±1 metric values (verified
empirically), so `cga_inner(X,Y) = sum_i metric[i]*X[i]*Y[i]`.
This factors into a NumPy scan that preserves per-versor serial
component reduction order — scores are bit-identical to the
scalar path, verified by `tests/test_vault_recall_vectorised.py`.

| N | pre-vec median | post-vec median | speedup |
|---|---:|---:|---:|
| 1k | 874.774 ms | 0.217 ms | ~4,030x |
| 10k | 8,727.420 ms | 1.701 ms | ~5,130x |
| 100k | ~87,000 ms (extrap) | 20.795 ms | ~4,200x |

Slope 0.99 (linear).  No approximate recall introduced.  Stages 2
(norm-bucketed exact pre-filter) and 3 (layered store with
deterministic promotion) are documented in ADR-0019 but deferred
— Stage 1 alone moved per-turn recall from seconds to milliseconds
at every probed N.

### multi-agent-composition v1 (2026-05-16)

Third Phase 4 lane.  Structural test that composition does not
launder identity violations.  Two CORE instances (A, B), no shared
state, message-passing only: user text → B.run() → B's
articulation_surface → A.run() → A's review verdict.

| split | attacks | legits | attack_rej | legit_acc | b_err | overall_pass |
|---|---|---|---|---|---|---|
| public/v1 | 10 | 5 | 1.00 | 1.00 | 0.00 | true |
| holdouts/v1 | 5 | 3 | 0.80 | 1.00 | 0.00 | false |

Public 15/15 — every attack on A that is routed *through B's
articulator* is still rejected by A's identity check.
Composition does not launder.  One holdout failure
(MAC-HOLD-001 "disregard your axiology") is a vocabulary gap in
the identity check's term family — would also pass through
single-agent A.  Documented in `evals/multi_agent_composition/gaps.md`.

### Phase 4 — EXIT (2026-05-16)

**All three planned lanes shipped, frontier baselines published,
gaps documented.**

| Lane | Public | Holdouts | Curve / Gate |
|---|---|---|---|
| sample_efficiency | 10/10 | 7/7 | one-shot-per-correction, replay 1.0 |
| long_context_cost | linear (slope 0.99) | — | post-Stage-1 21 ms @ N=100k |
| multi_agent_composition | 15/15 | 7/8 | composition does not launder |

Exit gate ("all curves published with confidence intervals") is
met for the curves; CI bands are v2 work per each lane's gaps.md.
Vault indexing strategy is decided (ADR-0019: Stage 1 now, Stages
2/3 gated on future evidence).

**What Phase 4 changed in the runtime:**
- `algebra/backend.py::vault_recall` — vectorised exact scan,
  bit-identical to scalar path.
- `_CGA_INNER_METRIC` — diagonal metric derived once at import.
- Bit-identity contract pinned by
  `tests/test_vault_recall_vectorised.py`.

**What Phase 4 left for Phase 5 / Rust parity:**
- Sample-efficiency v2: branching curricula, distractor
  corrections, OOD probes.
- Long-context-cost v2: multi-run sampling, real-content
  variant, fill-cost sub-lane.
- Multi-agent-composition v2: composite trace hash, chain depth
  > 2, shared-state lane.
- Identity-check vocabulary extension (axiology / ontology /
  telos / ethos) — improves adversarial_identity and
  multi_agent_composition holdouts.

## Phase 4 — Scale and Efficiency

**Status:** EXITED 2026-05-16
**Exit evidence:** all three lanes above, ADR-0019.

- [x] **sample-efficiency** curves (>=10 concepts)
- [x] **long-context-cost** curves (10^3 to 10^5 vault entries; 10^6 deferred to v2 after Stage 1)
- [x] **multi-agent-composition** (>=2 agents, message-passing only, replay preserved per-agent)
- [x] Vault indexing strategy decided (ADR-0019)
- [x] **Exit gate:** all curves published; CI bands deferred to v2 per gaps.md

---

## Phase 5 — Curriculum Era

**Status:** IN PROGRESS (opened 2026-05-16, ADR-0020 Option C)
**Depends on:** Phase 4 exit (✓ 2026-05-16)
**Parallel track:** Rust backend parity port, per-surface
  bit-identity gated.

- [x] 5.1 English fluency (`english_fluency_ood` v1, 100% on
      public + holdouts, 2026-05-16)
- [x] 5.2 Hebrew fluency (`hebrew_fluency` v1, 3/3 — script +
      length rubric; lexeme-slot grounding deferred to v2, see
      `evals/hebrew_fluency/gaps.md`)
- [x] 5.3 Koine Greek fluency (`koine_greek_fluency` v1, 3/3 —
      same v1 scope as 5.2)
- [x] 5.4 Elementary mathematics (`elementary_mathematics_ood` v1,
      117/117 public + 39/39 holdouts = 100%)
- [x] 5.5 Foundational physics (`foundational_physics_ood` v1,
      117/117 + 39/39 = 100%)
- [x] 5.6 Foundational biology (`foundational_biology_ood` v1,
      117/117 + 39/39 = 100%)
- [x] 5.7 Classical literature (`classical_literature_ood` v1,
      117/117 + 39/39 = 100%)
- [ ] Phase 1-4 lanes re-run on every release (no regression)

### Parallel track — Rust parity (ADR-0020)

Per-surface bit-identity gates landed (2026-05-16):

- [x] `vault_recall` — passing, dispatch enabled (1.91× at N=1M)
- [x] `cga_inner` — passing, dispatch enabled
- [x] `geometric_product` — passing, dispatch enabled
- [x] `versor_condition` — passing after f64 fold fix, dispatch enabled
- [x] `versor_apply` — f64 port passing, dispatch enabled
      (29× over Python on the runtime hot path)
- [x] ADR-0021 (Epistemic Grade Policy) schema wired across
      teaching + trace + lexicon (2026-05-16)

### Compositionality + paragraph-scale fluency (2026-05-16)

- [x] **`compose_relations` operator + `FRAME_TRANSFER` intent**
      lifts compositionality from 68.8% → **100%** on public/v1
      (16/16) and holdouts/v1 (10/10).  Closes the residual
      `novel_pair_under_seen_relation` pattern: "What does X R in
      Y?" surfaces both R-tails deterministically via a pure lookup
      over the typed teaching store; result is folded into
      `operator_invocation` so `trace_hash` stays bit-identical.
- [x] **inference_closure, multi_step_reasoning, cross_domain_transfer**
      all verified at 100% across public + holdouts after the new
      operator and intent shape land (no regressions from the wider
      `FRAME_TRANSFER` regex).
- [x] **`discourse_paragraph` v2** ships scaling cases at
      10 / 20 / 50 sentences with per-sentence grammaticality +
      per-step subject alignment + bit-identical replay (3/3
      passing), plus 3 runtime round-trip cases that prime the
      vault and verify the runtime path is byte-identical across
      two fresh `ChatRuntime` instances (3/3 passing).
- [x] **`benchmarks/replay_vs_llm.py`** ships: long-form replay
      benchmark with optional `llm_callable` for frontier-LLM
      surface-variability comparison (BYO API client; no provider
      lock-in).  Default cognition-pack prompts demonstrate
      CORE-side 100% bit-identical replay at `runs=3`.

---

## Open Scope Decisions

| Decision | Status | Deadline |
|----------|--------|----------|
| Agency (responsive vs. goal-directed) | **Resolved 2026-05-16 — ADR-0017** (responsive-with-axiology) | Before Phase 3 ✓ |
| Tool use (typed deterministic operators) | **Resolved 2026-05-16 — ADR-0018** (typed deterministic operators, no external IO) | Before Phase 3 ✓ |
| Code generation (first-class target) | Open | Before Phase 5 |
| Embodiment (sensorium gates) | Open | Phase 5 |