core-labs/core
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
core/evals/symbolic_logic/gaps.md
Shay 0053648efd feat(evals): symbolic-logic lane v1 — premise-chain foundations 
Adds the fourth Phase 2 lane. v1 measures the structural foundations
on which a future inference engine would be built:

  M1. premise_recall    — probe vault_hits >= min after chain teaching
  M2. replay_determinism — same chain + probe → same trace_hash
  M3. proposal_storage  — correction premises store as PackMutationProposals

Patterns covered: modus_ponens_chain, modus_tollens_chain, syllogism,
negation, chain_recall (up to 4-hop chains).

v1 results across 38 cases (8 dev + 18 public + 12 holdouts):
  premise_recall=1.0, replay_determinism=1.0, proposal_storage=1.0.

Each case runs twice on fresh CognitiveTurnPipelines to verify the
trace_hash matches — confirming deterministic replay over premise chains.

Architectural finding logged in evals/symbolic_logic/gaps.md:

  CORE has no first-class inference operator. Chain "inference" today is
  emergent from teaching-store commits + cumulative vault recall, not a
  named-rule symbolic engine. v1 honestly tests what CORE deterministically
  *does* (store, replay, recall chains) without overclaiming that CORE
  reasons symbolically. v2 would assert specific transitive recall
  contents in the probe surface, which requires either a
  PropositionGraph traversal operator or pack-axiom rules — both filed
  as suggested follow-up work.
2026-05-16 12:34:55 -07:00

2.1 KiB
Raw Permalink Blame History

symbolic-logic lane — architectural findings

Finding: No first-class inference operator

CORE has no operator that takes premises A→B, B→C and returns A→C. Inference, when it happens, is emergent from:

  • The teaching loop committing each correction premise to the vault.
  • The probe's CGA recall surfacing entries that were geometrically linked by the cumulative field state.
  • The realizer composing a surface from whatever the recall returned.

This is not the same as named-rule symbolic inference (modus ponens, modus tollens, syllogism). The v1 lane therefore measures the foundations that any future inference operator would require:

  • Premise chains store deterministically (M3).
  • Premise chains replay deterministically (M2).
  • Premise chains are recallable from probe state (M1).

A v2 lane would assert specific inference correctness — e.g., after teaching A is B and B is C, the probe What is A? produces a surface mentioning C (transitive recall through the relation graph). That requires either:

  • A first-class proposition-graph traversal operator on top of the vault, or
  • A pack-axiom layer where pack-declared A→B rules participate in recall.

Neither exists in the current runtime. The v1 lane is honest about this; it tests what CORE does deterministically (chain storage and replay) without overclaiming that CORE reasons symbolically.

Suggested follow-up work

  1. PropositionGraph + reasoning operator: Add an explicit module that consumes the cumulative teaching store, builds a relation graph, and applies named inference rules. Output: an inference_trace field on CognitiveTurnResult carrying the rule chain that derived a recalled conclusion.

  2. Pack-axiom rules: Extend pack manifests to declare rules (X is_a Y, Y is_a Z → X is_a Z). Compile rules into versor space so recall can traverse them deterministically.

  3. v2 symbolic-logic lane: Score correctness of specific inference outputs (e.g., probe surface contains the transitive target), not just chain storage.