First Phase 3 lane. Scores whether CORE can derive entailments that
were not directly asserted, given a chain of premises taught through
the correction loop. Five transitive relation patterns drawn from
en_core_cognition_v1:
transitive_is A is B; B is C -> What is A?
transitive_precedes A precedes B; B precedes C -> What does A precede?
transitive_grounds A grounds B; B grounds C -> What does A ground?
transitive_causes A causes B; B causes C -> What does A cause?
transitive_belongs_to A belongs_to B; B belongs_to C -> Where does A belong?
Pass = expected entailment token appears in probe response surface
or walk surface (M1 or M2) AND every premise stored (M3) AND
trace_hash deterministic across two fresh runs (M4).
Results:
split n derived stored replay overall_pass
public/v1 20 0.0 1.0 1.0 False
holdouts/v1 12 0.0 1.0 1.0 False
This is the expected honest failure per docs/capability_roadmap.md
Phase 3. Foundation guarantees from Phase 2 (storage + replay) hold
at this depth; the inference-closure step itself does not yet exist
in CORE. The lane scores exactly the gap.
Concrete trace recorded in gaps.md: for premises 'wisdom is light',
'light is truth', probe 'What is wisdom?' returns the template
'wisdom is defined as ...' — vault retrieves 9 entries including
both premises, but the realizer emits a definition stub instead of
a derivation.
Architectural gaps filed (evals/inference_closure/gaps.md):
Gap 1. generate/graph_planner.py has no transitive composition —
plan_articulation picks a single node; there is no chained
relation walk that produces a derived node from premises.
Gap 2. field/propagate.py has no derivable-but-not-asserted recall
path — vault retrieval is direct CGA inner product; no
path-recall operator over relation-typed edges.
Both gaps are v2 engineering candidates and may share an
implementation surface. The lane is permanent regression evidence
of what specifically is missing.
Includes:
- contract.md: pass criteria, anti-overfitting note, sub-metric
definitions, calibration approach.
- runner.py: parallel, fresh-pipeline-per-case, M1-M4 scoring,
two-run replay-determinism check.
- dev/cases.jsonl (5), public/v1 (20), holdouts/v1 (12) — disjoint
entity sets, all five patterns covered.
- baselines/v1_structural_zero.json: frontier LLMs do not emit
the typed signals by construction.
- gaps.md: full architectural finding, engineering shapes for v2.
CLI suites smoke / cognition / teaching pass; no regression on
Phase 2 work.
