Shay 819c8b81ac feat(phase3): compositionality, multi-step-reasoning, introspection, cross-domain-transfer v1

Spreads the four remaining Phase 3 lanes to map the full reasoning-
depth surface alongside inference-closure (already landed at e509e0d).
Each lane is a v1 honest probe per the roadmap; engineering work
follows once the full surface is visible.

Results across all five Phase 3 lanes:

  lane                      split        primary signal  foundation
  inference-closure         public/v1    0.0             1.0 / 1.0
  inference-closure         holdouts/v1  0.0             1.0 / 1.0
  compositionality          public/v1   0.0625 (1/16)   1.0 / 1.0
  compositionality          holdouts/v1  0.0             1.0 / 1.0
  multi-step-reasoning      public/v1    0.0             1.0 / 1.0
  multi-step-reasoning      holdouts/v1  0.0             1.0 / 1.0
  introspection             public/v1    0.0 (no api)    n/a
  introspection             holdouts/v1  0.0             n/a
  cross-domain-transfer     public/v1    0.0             1.0 / 1.0
  cross-domain-transfer     holdouts/v1  0.0             1.0 / 1.0

Foundation guarantees (storage + replay) intact across every lane
that has them. The reasoning-depth signal is uniformly zero. The
five lanes triangulate four architectural gaps:

  Gap 1. generate/graph_planner.py has no transitive composition.
  Gap 2. field/propagate.py has no derivable-but-not-asserted recall.
  Gap 3. core/cognition/explain.py module does not exist.
  Gap 4. no structural-pattern recogniser (cross-subdomain transfer).

Gaps 1, 2, 4 cluster on the same code surface and may close together
as a single bounded PR. Gap 3 is independent module-creation work.

Lane scaffolding mirrors inference-closure (contract.md, runner.py,
dev + public/v1 + holdouts/v1 cases.jsonl, baselines/v1_structural_zero.json,
gaps.md). All runners are parallel-safe and use the standard
run_lane(cases, *, config, workers) interface.

Per-lane gaps.md records the engineering shape for v2 plus future
directions worth not forgetting:
  - compositionality/gaps.md: metaphor is compositionality with
    selective property transfer; building it is correctly downstream
    of closing this lane.
  - cross-domain-transfer/gaps.md: metaphor + narrative as
    cross-domain operators; narrative requires the Agency open-scope
    decision to pin first.
  - introspection/gaps.md: explain API is also the substrate for
    first-person narrative self-account.

Recommended v2 sequence in docs/PROGRESS.md:
  1. Pin Agency + Tool-use open-scope decisions (deadline: before
     Phase 3 engineering).
  2. Engineer Gaps 1 + 2 as one bounded PR.
  3. Engineer Gap 3 independently.
  4. Re-author cross-domain-transfer v2 with matched-control
     contract refinement.

Phase 3 v1 exit: 0/5 lanes passing, which is the expected v1 floor.
CLI suites smoke / cognition / teaching pass; no regression on
Phase 2.

2026-05-16 14:48:36 -07:00

3.2 KiB

Raw Blame History

cross-domain-transfer lane — architectural findings (v1)

v1 result

Split	n	transfer_endpoint_recall	A_stored	B_stored	replay
public/v1	10	0.0	1.0	1.0	1.0
holdouts/v1	8	0.0	1.0	1.0	1.0

No transfer. Both A-domain and B-domain premises are independently stored (storage rate 1.0 on each side); replay is deterministic; the B-domain endpoint never appears in the probe surface.

What this confirms (vs. inference-closure)

This lane is inference-closure plus a prior teaching pass in a disjoint semantic subdomain. v1's result establishes that:

The A-domain teaching has no carry-over effect on B-domain competence. This is consistent with CORE having no structural- pattern recogniser — the A-domain chain doesn't shape how the B-domain chain is articulated or recalled.
Whatever fix closes inference-closure's Gap 1 / Gap 2 may close this lane's failure too, since B-domain alone is a literal inference-closure case. But it will not demonstrate transfer — that requires a different signal, captured in v2.

v2 contract refinement

To actually score transfer (rather than just "B-domain inference works after A-domain teaching"), v2 of this lane should include a matched control: same B-domain probe without prior A-domain teaching. Pass criterion becomes:

transfer_endpoint_recall_rate(with_A_teaching) > transfer_endpoint_recall_rate(without_A_teaching)

That delta is the genuine transfer signal. v1 leaves this on the table because the floor is currently zero on both arms — a v1 "transfer = 0 − 0 = 0" result would be uninformative. When the inference-closure engineering lands and the B-arm starts producing non-zero recall, v2's matched-control comparison becomes the load-bearing measurement.

Architectural gaps

No structural-pattern recogniser. CORE's proposition graph has no concept of "the relation pattern R(x1,x2)→R(x2,x3) was seen N times across these subdomains" — patterns are not first-class entities.
No cross-subdomain transfer operator. Vault retrieval and field propagation are entity-local; nothing maps "structural competence in subdomain A" to "expected competence in subdomain B."
Both gaps are downstream of (and overlap with) inference-closure Gap 1 + Gap 2.

Future directions (recorded here so they're not forgotten)

Metaphor as cross-domain transfer with selectivity. A metaphor is the same shape as this lane's probe with an added filter: which relations transfer across the analogy and which do not. Once literal cross-domain transfer works, building metaphor-comprehension on top is a natural Phase 3 v2 lane rather than a separate operator.
Narrative as multi-step cross-domain transfer. A story is a multi-step inference chain bound to a point-of-view (agent / intention). Both substrates (multi-step chaining and POV) need to land before a narrative lane is meaningful.

Status

v1 stands as honest-failure baseline. v2 contract refinement (matched-control comparison) is the next authoring step once inference-closure engineering lifts B-arm recall off the floor.

3.2 KiB Raw Blame History Unescape Escape