core/tests/test_determine_relational_inference.py
Shay 6f70e834ff
feat: one-hop sound relational entailment (inverse/symmetric) + capability-index lane (#775)
* feat(determine): one-hop sound relational entailment — inverse/converse + symmetric

Mastery-v2 Step 3 lead capability (core). DETERMINE could perceive relational
structure (16 predicates, breadth-complete) but could not derive the simplest
entailed relational facts — it read only the stored direction. This adds two
SOUND one-hop rules that read a stored edge in its other lawful direction:

  INVERSE/converse   greater_than(a,b)  <=  told less_than(b,a)
  SYMMETRIC          sibling_of(b,a)    <=  told sibling_of(a,b)

Strictly scoped (per review): OPEN-WORLD (asserts only True, never False — the
answer=False path stays unbuilt, INV-30's firewall holds), ONE hop (NO transitive
chaining), DECLARED rules only. Ontology/metadata-driven, not prose intuition:

- generate/meaning_graph/relational.py: declarative algebra tables — _INVERSE_PAIRS
  (the converse edges; the pack carries no inverse metadata) + INVERSE_OF (derived,
  an involution) + SYMMETRIC_PREDICATES (mirrors the pack's graph.edge.symmetric tag).
  load_relational_pack_symmetric() reads the pack ontology; a test pins the constant
  equal to it (no silent divergence).
- generate/determine/determine.py: _relational_one_hop() between direct entailment
  and transitive subsumption; new Determined.rule provenance field (direct/inverse/
  symmetric/subsumption) — replay-safe (render reads only basis; trace hashes surface).

wrong=0 confuser block proves the rule cannot over-fire: less_than is not
self-inverse, sibling_of does not imply parent_of, greater_than does not imply
equal_to, NO transitive chain (direct or through-inverse), and no answer=False is
ever emitted. An obsolete pin (test_symmetric_converse_is_not_faked, which asserted
the pre-rule direct-only floor) is updated to the new sound behavior and a preserved
asymmetric-converse bite.

Tests: test_determine_relational_inference.py (13, the capability contract) +
updated test_relational_reader.py. Broad relational/determine/grounding +
capability_index baseline sweep: 114 green; baseline digest UNCHANGED (this is a
determination capability; the comprehension lanes are untouched).

REMAINING for the full lead PR (next): the measurement lane — an independent-oracle
evals/relational_inference lane + a capability_index adapter + baseline re-freeze
(breadth 9->10, wrong_total still 0), so the capability registers on the yardstick.

INTEGRATION NOTE (cross-PR): this adds 2 Determined() construction sites (now 4:
direct/inverse/symmetric/subsumption, ALL answer=True). When rebased onto main with
INV-30 (PR #770), update INV-30's test_determine_construction_sites_are_visible count
2 -> 4 and confirm all four assert True. Merge order: #770 -> this.

* feat(capability-index): relational-inference lane — breadth 9->10, wrong=0

Mastery-v2 Step 3 lead, measurement half. Puts the one-hop relational-inference
capability ON the capability index with an independent-oracle lane:

- evals/relational_inference/v1/: cases.jsonl (13 positive: 8 inverse + 5 symmetric,
  authored from the relation algebra INDEPENDENTLY of determine/relational per
  INV-25/27), refusals.jsonl (8 confusers that MUST refuse), provenance.md (incl. the
  honest overlaps_event reader-surface coverage gap).
- evals/comprehension/relational_inference_runner.py: told fact(s) -> determine(query)
  vs gold; refusal = coverage miss, disagreement = wrong (structurally 0 on positives).
- evals/capability_index/adapters.py: comprehension_relational_inference_result added
  to ADAPTERS.
- baseline.json re-frozen: breadth 9 -> 10, capability_score 0.94403 -> 0.949483,
  wrong_total still 0, digest deliberately changed (35dea2b2...).
- tests/test_relational_inference_lane.py: SHA-pinned gold, positive wrong=0 +
  coverage>0, and the confuser wrong=0 BITE (every confuser must refuse).

Also reconciles INV-30 (now in main via #770): determine.py grew from 2 to 3
Determined() construction sites (direct, the shared relational one-hop constructor,
transitive subsumption) — ALL answer=True. test_determine_construction_sites_are_visible
updated 2 -> 3 (correcting the prior commit note's overcount of 4; inverse and symmetric
share the single _relational_determined constructor, so it is ONE new site).

Also fixes a .gitignore gap: ADR-0219 gen-dir checkpoints (engine_state/current,
engine_state/gen-*/) were not ignored (only the old flat engine_state/*.json patterns
were), so runtime checkpoints could be committed by accident.

Verification: new lane 13/0/0 coverage 1.0; capability_baseline digest matches the
re-freeze; INV-30 + architectural invariants green; smoke 99/99.

* docs+test: reconcile stale relational-inference docstrings; assert rule provenance (review #775)

Addresses two review-requested cleanups on #775:

1. Stale docstrings. generate/meaning_graph/relational.py said the symmetric converse
   is "a sound-but-incomplete refusal at DETERMINE" and there is "NO transitive/
   symmetric/rule inference"; generate/determine/determine.py said "symmetric-converse
   questions are Undetermined", "no transitive/symmetric/rule inference", and "asserts
   only answer=True on a direct hit". All false after the one-hop relational algebra
   landed. Updated: the reader does direct-reading only; DETERMINE applies declared
   one-hop inverse/converse + pack-declared symmetric (plus the existing member/subset
   subsumption), still open-world / never answer=False; transitive relational closure,
   negation, and closed-world falsehood remain out of scope.

2. Rule-provenance gold now meaningful. The relational_inference runner compared only
   (answer, predicate, subject, object), ignoring res.rule though the gold carries
   "rule": "inverse"/"symmetric". Added res.rule to the got/gold tuple so a "right
   answer via the wrong rule path" can no longer pass silently. Lane still 13/0/0;
   baseline counts (and digest) unchanged.

Verification: runner 13/0/0; lane + capability_baseline + unit + INV-30 green (22).
2026-06-15 11:39:41 -07:00

183 lines
7.2 KiB
Python

"""DETERMINE — one-hop sound relational entailment (mastery-v2 Step 3 lead).
A relational query may hold by ONE sound predicate-algebra rule that reads a stored
edge in its OTHER lawful direction:
INVERSE/converse greater_than(a, b) <= told less_than(b, a)
SYMMETRIC sibling_of(b, a) <= told sibling_of(a, b)
Scope is deliberately narrow: OPEN-WORLD (asserts only True, never False), ONE hop
(NO transitive chaining), DECLARED rules only (less_than is NOT self-inverse;
parent_of is NOT symmetric). Closed-world / assert-False / FrameVerdict are out of
scope (a later slice). The wrong=0 discipline lives in the confuser block below.
"""
from __future__ import annotations
import pytest
from chat.runtime import ChatRuntime
from generate.determine import Determined, Undetermined, determine
from generate.meaning_graph.relational import (
INVERSE_OF,
RELATIONAL_PREDICATES,
SYMMETRIC_PREDICATES,
comprehend_relational,
load_relational_pack_lemmas,
load_relational_pack_symmetric,
)
from generate.realize import realize_comprehension
from session.context import SessionContext
_HIGH = 10**9
@pytest.fixture(scope="module")
def vocab_persona():
rt = ChatRuntime(no_load_state=True)
return rt._context.vocab, rt._context.persona
@pytest.fixture(scope="module")
def pack():
return load_relational_pack_lemmas()
def _ctx(vocab_persona) -> SessionContext:
vocab, persona = vocab_persona
return SessionContext(vocab=vocab, persona=persona, vault_reproject_interval=_HIGH)
def _tell(text: str, ctx: SessionContext, pack):
return realize_comprehension(comprehend_relational(text, pack), ctx)
def _ask(text: str, ctx: SessionContext, pack):
return determine(comprehend_relational(text, pack), ctx)
# --------------------------------------------------------------------------- #
# Positive — the engine now derives the simplest entailed relational facts
# --------------------------------------------------------------------------- #
def test_inverse_converse_admits_true(vocab_persona, pack) -> None:
ctx = _ctx(vocab_persona)
_tell("Bob is less than Alice.", ctx, pack) # less_than(bob, alice)
res = _ask("Is Alice greater than Bob?", ctx, pack) # greater_than(alice, bob)
assert isinstance(res, Determined)
assert res.answer is True and res.basis == "as_told" and res.rule == "inverse"
assert res.predicate == "greater_than"
assert res.subject == "alice" and res.object == "bob"
# grounds = the single stored converse edge
assert len(res.grounds) == 1 and res.grounds[0].relation_predicate == "less_than"
def test_symmetric_admits_true(vocab_persona, pack) -> None:
ctx = _ctx(vocab_persona)
_tell("Alice is the sibling of Bob.", ctx, pack) # sibling_of(alice, bob)
res = _ask("Is Bob the sibling of Alice?", ctx, pack) # sibling_of(bob, alice)
assert isinstance(res, Determined) and res.answer is True
assert res.rule == "symmetric" and res.predicate == "sibling_of"
assert res.subject == "bob" and res.object == "alice"
def test_direct_stored_direction_still_determines(vocab_persona, pack) -> None:
ctx = _ctx(vocab_persona)
_tell("Alice is the sibling of Bob.", ctx, pack)
res = _ask("Is Alice the sibling of Bob?", ctx, pack) # the stored direction
assert isinstance(res, Determined) and res.answer is True and res.rule == "direct"
# --------------------------------------------------------------------------- #
# wrong=0 confuser block — the rule must not over-fire
# --------------------------------------------------------------------------- #
def test_less_than_is_not_self_inverse(vocab_persona, pack) -> None:
"""less_than is asymmetric: a<b does NOT entail b<a (its converse is greater_than)."""
ctx = _ctx(vocab_persona)
_tell("Alice is less than Bob.", ctx, pack) # less_than(alice, bob)
res = _ask("Is Bob less than Alice?", ctx, pack) # less_than(bob, alice)?
assert isinstance(res, Undetermined)
def test_sibling_does_not_imply_parent(vocab_persona, pack) -> None:
ctx = _ctx(vocab_persona)
_tell("Alice is the sibling of Bob.", ctx, pack)
res = _ask("Is Alice the parent of Bob?", ctx, pack)
assert isinstance(res, Undetermined)
def test_greater_than_does_not_imply_equal(vocab_persona, pack) -> None:
ctx = _ctx(vocab_persona)
_tell("Alice is greater than Bob.", ctx, pack)
res = _ask("Is Alice equal to Bob?", ctx, pack)
assert isinstance(res, Undetermined)
def test_no_transitive_chain_direct(vocab_persona, pack) -> None:
"""One hop only: a<b and b<c must NOT entail a<c (transitive is a later slice)."""
ctx = _ctx(vocab_persona)
_tell("Alice is less than Bob.", ctx, pack)
_tell("Bob is less than Carol.", ctx, pack)
res = _ask("Is Alice less than Carol?", ctx, pack)
assert isinstance(res, Undetermined)
def test_no_transitive_chain_through_inverse(vocab_persona, pack) -> None:
"""The inverse rule is also one hop: it must not become a transitive bridge."""
ctx = _ctx(vocab_persona)
_tell("Bob is less than Alice.", ctx, pack) # => greater_than(alice, bob) by inverse
_tell("Carol is less than Bob.", ctx, pack) # => greater_than(bob, carol) by inverse
res = _ask("Is Alice greater than Carol?", ctx, pack) # needs transitive — refuse
assert isinstance(res, Undetermined)
def test_unsupported_predicate_refuses(vocab_persona, pack) -> None:
ctx = _ctx(vocab_persona)
# No told fact at all → ungrounded refusal, never a guess.
res = _ask("Is Alice the sibling of Bob?", ctx, pack)
assert isinstance(res, Undetermined)
def test_never_asserts_false(vocab_persona, pack) -> None:
"""Open-world: every determination is True-or-refuse; answer=False is unreachable
(INV-30 is the structural guarantee — this is the behavioral echo on this path)."""
ctx = _ctx(vocab_persona)
_tell("Alice is less than Bob.", ctx, pack)
for q in (
"Is Bob less than Alice?",
"Is Alice greater than Bob?",
"Is Alice equal to Bob?",
"Is Alice the parent of Bob?",
"Is Alice less than Bob?",
):
res = _ask(q, ctx, pack)
if isinstance(res, Determined):
assert res.answer is True
# --------------------------------------------------------------------------- #
# Ontology pins — the algebra cannot silently diverge from the pack / vocab
# --------------------------------------------------------------------------- #
def test_symmetric_table_matches_pack_ontology(pack) -> None:
"""SYMMETRIC_PREDICATES MUST equal the pack's graph.edge.symmetric declarations —
the pack is the source of truth; the constant is the runtime-cheap mirror."""
assert SYMMETRIC_PREDICATES == load_relational_pack_symmetric()
def test_algebra_members_are_relational_predicates() -> None:
"""Every inverse/symmetric lemma is a real reader predicate — a typo cannot mint
an unknown predicate into the determination path."""
for lemma in set(INVERSE_OF) | SYMMETRIC_PREDICATES:
assert lemma in RELATIONAL_PREDICATES
def test_inverse_is_an_involution() -> None:
"""inverse(inverse(p)) == p, and no predicate is its own inverse (asymmetry)."""
for lemma, other in INVERSE_OF.items():
assert lemma != other
assert INVERSE_OF[other] == lemma