819c8b81ac
Spreads the four remaining Phase 3 lanes to map the full reasoning-
depth surface alongside inference-closure (already landed at e509e0d).
Each lane is a v1 honest probe per the roadmap; engineering work
follows once the full surface is visible.
Results across all five Phase 3 lanes:
lane split primary signal foundation
inference-closure public/v1 0.0 1.0 / 1.0
inference-closure holdouts/v1 0.0 1.0 / 1.0
compositionality public/v1 0.0625 (1/16) 1.0 / 1.0
compositionality holdouts/v1 0.0 1.0 / 1.0
multi-step-reasoning public/v1 0.0 1.0 / 1.0
multi-step-reasoning holdouts/v1 0.0 1.0 / 1.0
introspection public/v1 0.0 (no api) n/a
introspection holdouts/v1 0.0 n/a
cross-domain-transfer public/v1 0.0 1.0 / 1.0
cross-domain-transfer holdouts/v1 0.0 1.0 / 1.0
Foundation guarantees (storage + replay) intact across every lane
that has them. The reasoning-depth signal is uniformly zero. The
five lanes triangulate four architectural gaps:
Gap 1. generate/graph_planner.py has no transitive composition.
Gap 2. field/propagate.py has no derivable-but-not-asserted recall.
Gap 3. core/cognition/explain.py module does not exist.
Gap 4. no structural-pattern recogniser (cross-subdomain transfer).
Gaps 1, 2, 4 cluster on the same code surface and may close together
as a single bounded PR. Gap 3 is independent module-creation work.
Lane scaffolding mirrors inference-closure (contract.md, runner.py,
dev + public/v1 + holdouts/v1 cases.jsonl, baselines/v1_structural_zero.json,
gaps.md). All runners are parallel-safe and use the standard
run_lane(cases, *, config, workers) interface.
Per-lane gaps.md records the engineering shape for v2 plus future
directions worth not forgetting:
- compositionality/gaps.md: metaphor is compositionality with
selective property transfer; building it is correctly downstream
of closing this lane.
- cross-domain-transfer/gaps.md: metaphor + narrative as
cross-domain operators; narrative requires the Agency open-scope
decision to pin first.
- introspection/gaps.md: explain API is also the substrate for
first-person narrative self-account.
Recommended v2 sequence in docs/PROGRESS.md:
1. Pin Agency + Tool-use open-scope decisions (deadline: before
Phase 3 engineering).
2. Engineer Gaps 1 + 2 as one bounded PR.
3. Engineer Gap 3 independently.
4. Re-author cross-domain-transfer v2 with matched-control
contract refinement.
Phase 3 v1 exit: 0/5 lanes passing, which is the expected v1 floor.
CLI suites smoke / cognition / teaching pass; no regression on
Phase 2.
16 lines
4.5 KiB
JSON
16 lines
4.5 KiB
JSON
{"id":"CMP-V1-001","pattern":"composed_predicate","premises":["What is wisdom?","Actually wisdom is judgment.","What is judgment?","Actually judgment precedes decision."],"probe":"What does wisdom precede?","expected_entailment_tokens":["decision"],"expected_proposals":2}
|
|
{"id":"CMP-V1-002","pattern":"composed_predicate","premises":["What is light?","Actually light is clarity.","What is clarity?","Actually clarity causes recognition."],"probe":"What does light cause?","expected_entailment_tokens":["recognition"],"expected_proposals":2}
|
|
{"id":"CMP-V1-003","pattern":"composed_predicate","premises":["What is principle?","Actually principle is order.","What is order?","Actually order grounds coherence."],"probe":"What does principle ground?","expected_entailment_tokens":["coherence"],"expected_proposals":2}
|
|
{"id":"CMP-V1-004","pattern":"composed_predicate","premises":["What is creation?","Actually creation is movement.","What is movement?","Actually movement precedes change."],"probe":"What does creation precede?","expected_entailment_tokens":["change"],"expected_proposals":2}
|
|
{"id":"CMP-V1-005","pattern":"composed_predicate","premises":["What is reason?","Actually reason is inference.","What is inference?","Actually inference produces conclusion."],"probe":"What does reason produce?","expected_entailment_tokens":["conclusion"],"expected_proposals":2}
|
|
{"id":"CMP-V1-006","pattern":"novel_pair_under_seen_relation","premises":["What is truth?","Actually truth grounds judgment.","What is knowledge?","Actually knowledge grounds inference."],"probe":"What does truth ground in knowledge?","expected_entailment_tokens":["inference","judgment"],"expected_proposals":2}
|
|
{"id":"CMP-V1-007","pattern":"novel_pair_under_seen_relation","premises":["What is order?","Actually order precedes meaning.","What is structure?","Actually structure precedes coherence."],"probe":"What does order precede in structure?","expected_entailment_tokens":["coherence","meaning"],"expected_proposals":2}
|
|
{"id":"CMP-V1-008","pattern":"novel_pair_under_seen_relation","premises":["What is question?","Actually question causes inquiry.","What is answer?","Actually answer causes recall."],"probe":"What does question cause in answer?","expected_entailment_tokens":["recall","inquiry"],"expected_proposals":2}
|
|
{"id":"CMP-V1-009","pattern":"novel_pair_under_seen_relation","premises":["What is recognition?","Actually recognition belongs_to memory.","What is naming?","Actually naming belongs_to language."],"probe":"What does recognition belong to in naming?","expected_entailment_tokens":["language","memory"],"expected_proposals":2}
|
|
{"id":"CMP-V1-010","pattern":"novel_pair_under_seen_relation","premises":["What is wisdom?","Actually wisdom reveals truth.","What is light?","Actually light reveals clarity."],"probe":"What does wisdom reveal in light?","expected_entailment_tokens":["clarity","truth"],"expected_proposals":2}
|
|
{"id":"CMP-V1-011","pattern":"novel_relation_on_seen_pair","premises":["What is judgment?","Actually judgment is decision.","What is wisdom?","Actually wisdom precedes judgment."],"probe":"What does wisdom decide?","expected_entailment_tokens":["decision","judgment"],"expected_proposals":2}
|
|
{"id":"CMP-V1-012","pattern":"novel_relation_on_seen_pair","premises":["What is inquiry?","Actually inquiry is thought.","What is question?","Actually question precedes inquiry."],"probe":"What does question think?","expected_entailment_tokens":["thought","inquiry"],"expected_proposals":2}
|
|
{"id":"CMP-V1-013","pattern":"novel_relation_on_seen_pair","premises":["What is clarity?","Actually clarity is recognition.","What is light?","Actually light precedes clarity."],"probe":"What does light recognise?","expected_entailment_tokens":["recognition","clarity"],"expected_proposals":2}
|
|
{"id":"CMP-V1-014","pattern":"novel_relation_on_seen_pair","premises":["What is knowledge?","Actually knowledge is judgment.","What is truth?","Actually truth grounds knowledge."],"probe":"What does truth judge?","expected_entailment_tokens":["judgment","knowledge"],"expected_proposals":2}
|
|
{"id":"CMP-V1-015","pattern":"composed_predicate","premises":["What is meaning?","Actually meaning is relation.","What is relation?","Actually relation grounds coherence."],"probe":"What does meaning ground?","expected_entailment_tokens":["coherence"],"expected_proposals":2}
|
|
{"id":"CMP-V1-016","pattern":"composed_predicate","premises":["What is correction?","Actually correction is adjustment.","What is adjustment?","Actually adjustment precedes learning."],"probe":"What does correction precede?","expected_entailment_tokens":["learning"],"expected_proposals":2}
|