* feat(determine): one-hop sound relational entailment — inverse/converse + symmetric Mastery-v2 Step 3 lead capability (core). DETERMINE could perceive relational structure (16 predicates, breadth-complete) but could not derive the simplest entailed relational facts — it read only the stored direction. This adds two SOUND one-hop rules that read a stored edge in its other lawful direction: INVERSE/converse greater_than(a,b) <= told less_than(b,a) SYMMETRIC sibling_of(b,a) <= told sibling_of(a,b) Strictly scoped (per review): OPEN-WORLD (asserts only True, never False — the answer=False path stays unbuilt, INV-30's firewall holds), ONE hop (NO transitive chaining), DECLARED rules only. Ontology/metadata-driven, not prose intuition: - generate/meaning_graph/relational.py: declarative algebra tables — _INVERSE_PAIRS (the converse edges; the pack carries no inverse metadata) + INVERSE_OF (derived, an involution) + SYMMETRIC_PREDICATES (mirrors the pack's graph.edge.symmetric tag). load_relational_pack_symmetric() reads the pack ontology; a test pins the constant equal to it (no silent divergence). - generate/determine/determine.py: _relational_one_hop() between direct entailment and transitive subsumption; new Determined.rule provenance field (direct/inverse/ symmetric/subsumption) — replay-safe (render reads only basis; trace hashes surface). wrong=0 confuser block proves the rule cannot over-fire: less_than is not self-inverse, sibling_of does not imply parent_of, greater_than does not imply equal_to, NO transitive chain (direct or through-inverse), and no answer=False is ever emitted. An obsolete pin (test_symmetric_converse_is_not_faked, which asserted the pre-rule direct-only floor) is updated to the new sound behavior and a preserved asymmetric-converse bite. Tests: test_determine_relational_inference.py (13, the capability contract) + updated test_relational_reader.py. Broad relational/determine/grounding + capability_index baseline sweep: 114 green; baseline digest UNCHANGED (this is a determination capability; the comprehension lanes are untouched). REMAINING for the full lead PR (next): the measurement lane — an independent-oracle evals/relational_inference lane + a capability_index adapter + baseline re-freeze (breadth 9->10, wrong_total still 0), so the capability registers on the yardstick. INTEGRATION NOTE (cross-PR): this adds 2 Determined() construction sites (now 4: direct/inverse/symmetric/subsumption, ALL answer=True). When rebased onto main with INV-30 (PR #770), update INV-30's test_determine_construction_sites_are_visible count 2 -> 4 and confirm all four assert True. Merge order: #770 -> this. * feat(capability-index): relational-inference lane — breadth 9->10, wrong=0 Mastery-v2 Step 3 lead, measurement half. Puts the one-hop relational-inference capability ON the capability index with an independent-oracle lane: - evals/relational_inference/v1/: cases.jsonl (13 positive: 8 inverse + 5 symmetric, authored from the relation algebra INDEPENDENTLY of determine/relational per INV-25/27), refusals.jsonl (8 confusers that MUST refuse), provenance.md (incl. the honest overlaps_event reader-surface coverage gap). - evals/comprehension/relational_inference_runner.py: told fact(s) -> determine(query) vs gold; refusal = coverage miss, disagreement = wrong (structurally 0 on positives). - evals/capability_index/adapters.py: comprehension_relational_inference_result added to ADAPTERS. - baseline.json re-frozen: breadth 9 -> 10, capability_score 0.94403 -> 0.949483, wrong_total still 0, digest deliberately changed (35dea2b2...). - tests/test_relational_inference_lane.py: SHA-pinned gold, positive wrong=0 + coverage>0, and the confuser wrong=0 BITE (every confuser must refuse). Also reconciles INV-30 (now in main via #770): determine.py grew from 2 to 3 Determined() construction sites (direct, the shared relational one-hop constructor, transitive subsumption) — ALL answer=True. test_determine_construction_sites_are_visible updated 2 -> 3 (correcting the prior commit note's overcount of 4; inverse and symmetric share the single _relational_determined constructor, so it is ONE new site). Also fixes a .gitignore gap: ADR-0219 gen-dir checkpoints (engine_state/current, engine_state/gen-*/) were not ignored (only the old flat engine_state/*.json patterns were), so runtime checkpoints could be committed by accident. Verification: new lane 13/0/0 coverage 1.0; capability_baseline digest matches the re-freeze; INV-30 + architectural invariants green; smoke 99/99. * docs+test: reconcile stale relational-inference docstrings; assert rule provenance (review #775) Addresses two review-requested cleanups on #775: 1. Stale docstrings. generate/meaning_graph/relational.py said the symmetric converse is "a sound-but-incomplete refusal at DETERMINE" and there is "NO transitive/ symmetric/rule inference"; generate/determine/determine.py said "symmetric-converse questions are Undetermined", "no transitive/symmetric/rule inference", and "asserts only answer=True on a direct hit". All false after the one-hop relational algebra landed. Updated: the reader does direct-reading only; DETERMINE applies declared one-hop inverse/converse + pack-declared symmetric (plus the existing member/subset subsumption), still open-world / never answer=False; transitive relational closure, negation, and closed-world falsehood remain out of scope. 2. Rule-provenance gold now meaningful. The relational_inference runner compared only (answer, predicate, subject, object), ignoring res.rule though the gold carries "rule": "inverse"/"symmetric". Added res.rule to the got/gold tuple so a "right answer via the wrong rule path" can no longer pass silently. Lane still 13/0/0; baseline counts (and digest) unchanged. Verification: runner 13/0/0; lane + capability_baseline + unit + INV-30 green (22).
183 lines
7.2 KiB
Python
183 lines
7.2 KiB
Python
"""DETERMINE — one-hop sound relational entailment (mastery-v2 Step 3 lead).
|
|
|
|
A relational query may hold by ONE sound predicate-algebra rule that reads a stored
|
|
edge in its OTHER lawful direction:
|
|
|
|
INVERSE/converse greater_than(a, b) <= told less_than(b, a)
|
|
SYMMETRIC sibling_of(b, a) <= told sibling_of(a, b)
|
|
|
|
Scope is deliberately narrow: OPEN-WORLD (asserts only True, never False), ONE hop
|
|
(NO transitive chaining), DECLARED rules only (less_than is NOT self-inverse;
|
|
parent_of is NOT symmetric). Closed-world / assert-False / FrameVerdict are out of
|
|
scope (a later slice). The wrong=0 discipline lives in the confuser block below.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import pytest
|
|
|
|
from chat.runtime import ChatRuntime
|
|
from generate.determine import Determined, Undetermined, determine
|
|
from generate.meaning_graph.relational import (
|
|
INVERSE_OF,
|
|
RELATIONAL_PREDICATES,
|
|
SYMMETRIC_PREDICATES,
|
|
comprehend_relational,
|
|
load_relational_pack_lemmas,
|
|
load_relational_pack_symmetric,
|
|
)
|
|
from generate.realize import realize_comprehension
|
|
from session.context import SessionContext
|
|
|
|
_HIGH = 10**9
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def vocab_persona():
|
|
rt = ChatRuntime(no_load_state=True)
|
|
return rt._context.vocab, rt._context.persona
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def pack():
|
|
return load_relational_pack_lemmas()
|
|
|
|
|
|
def _ctx(vocab_persona) -> SessionContext:
|
|
vocab, persona = vocab_persona
|
|
return SessionContext(vocab=vocab, persona=persona, vault_reproject_interval=_HIGH)
|
|
|
|
|
|
def _tell(text: str, ctx: SessionContext, pack):
|
|
return realize_comprehension(comprehend_relational(text, pack), ctx)
|
|
|
|
|
|
def _ask(text: str, ctx: SessionContext, pack):
|
|
return determine(comprehend_relational(text, pack), ctx)
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# Positive — the engine now derives the simplest entailed relational facts
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_inverse_converse_admits_true(vocab_persona, pack) -> None:
|
|
ctx = _ctx(vocab_persona)
|
|
_tell("Bob is less than Alice.", ctx, pack) # less_than(bob, alice)
|
|
res = _ask("Is Alice greater than Bob?", ctx, pack) # greater_than(alice, bob)
|
|
assert isinstance(res, Determined)
|
|
assert res.answer is True and res.basis == "as_told" and res.rule == "inverse"
|
|
assert res.predicate == "greater_than"
|
|
assert res.subject == "alice" and res.object == "bob"
|
|
# grounds = the single stored converse edge
|
|
assert len(res.grounds) == 1 and res.grounds[0].relation_predicate == "less_than"
|
|
|
|
|
|
def test_symmetric_admits_true(vocab_persona, pack) -> None:
|
|
ctx = _ctx(vocab_persona)
|
|
_tell("Alice is the sibling of Bob.", ctx, pack) # sibling_of(alice, bob)
|
|
res = _ask("Is Bob the sibling of Alice?", ctx, pack) # sibling_of(bob, alice)
|
|
assert isinstance(res, Determined) and res.answer is True
|
|
assert res.rule == "symmetric" and res.predicate == "sibling_of"
|
|
assert res.subject == "bob" and res.object == "alice"
|
|
|
|
|
|
def test_direct_stored_direction_still_determines(vocab_persona, pack) -> None:
|
|
ctx = _ctx(vocab_persona)
|
|
_tell("Alice is the sibling of Bob.", ctx, pack)
|
|
res = _ask("Is Alice the sibling of Bob?", ctx, pack) # the stored direction
|
|
assert isinstance(res, Determined) and res.answer is True and res.rule == "direct"
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# wrong=0 confuser block — the rule must not over-fire
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_less_than_is_not_self_inverse(vocab_persona, pack) -> None:
|
|
"""less_than is asymmetric: a<b does NOT entail b<a (its converse is greater_than)."""
|
|
ctx = _ctx(vocab_persona)
|
|
_tell("Alice is less than Bob.", ctx, pack) # less_than(alice, bob)
|
|
res = _ask("Is Bob less than Alice?", ctx, pack) # less_than(bob, alice)?
|
|
assert isinstance(res, Undetermined)
|
|
|
|
|
|
def test_sibling_does_not_imply_parent(vocab_persona, pack) -> None:
|
|
ctx = _ctx(vocab_persona)
|
|
_tell("Alice is the sibling of Bob.", ctx, pack)
|
|
res = _ask("Is Alice the parent of Bob?", ctx, pack)
|
|
assert isinstance(res, Undetermined)
|
|
|
|
|
|
def test_greater_than_does_not_imply_equal(vocab_persona, pack) -> None:
|
|
ctx = _ctx(vocab_persona)
|
|
_tell("Alice is greater than Bob.", ctx, pack)
|
|
res = _ask("Is Alice equal to Bob?", ctx, pack)
|
|
assert isinstance(res, Undetermined)
|
|
|
|
|
|
def test_no_transitive_chain_direct(vocab_persona, pack) -> None:
|
|
"""One hop only: a<b and b<c must NOT entail a<c (transitive is a later slice)."""
|
|
ctx = _ctx(vocab_persona)
|
|
_tell("Alice is less than Bob.", ctx, pack)
|
|
_tell("Bob is less than Carol.", ctx, pack)
|
|
res = _ask("Is Alice less than Carol?", ctx, pack)
|
|
assert isinstance(res, Undetermined)
|
|
|
|
|
|
def test_no_transitive_chain_through_inverse(vocab_persona, pack) -> None:
|
|
"""The inverse rule is also one hop: it must not become a transitive bridge."""
|
|
ctx = _ctx(vocab_persona)
|
|
_tell("Bob is less than Alice.", ctx, pack) # => greater_than(alice, bob) by inverse
|
|
_tell("Carol is less than Bob.", ctx, pack) # => greater_than(bob, carol) by inverse
|
|
res = _ask("Is Alice greater than Carol?", ctx, pack) # needs transitive — refuse
|
|
assert isinstance(res, Undetermined)
|
|
|
|
|
|
def test_unsupported_predicate_refuses(vocab_persona, pack) -> None:
|
|
ctx = _ctx(vocab_persona)
|
|
# No told fact at all → ungrounded refusal, never a guess.
|
|
res = _ask("Is Alice the sibling of Bob?", ctx, pack)
|
|
assert isinstance(res, Undetermined)
|
|
|
|
|
|
def test_never_asserts_false(vocab_persona, pack) -> None:
|
|
"""Open-world: every determination is True-or-refuse; answer=False is unreachable
|
|
(INV-30 is the structural guarantee — this is the behavioral echo on this path)."""
|
|
ctx = _ctx(vocab_persona)
|
|
_tell("Alice is less than Bob.", ctx, pack)
|
|
for q in (
|
|
"Is Bob less than Alice?",
|
|
"Is Alice greater than Bob?",
|
|
"Is Alice equal to Bob?",
|
|
"Is Alice the parent of Bob?",
|
|
"Is Alice less than Bob?",
|
|
):
|
|
res = _ask(q, ctx, pack)
|
|
if isinstance(res, Determined):
|
|
assert res.answer is True
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# Ontology pins — the algebra cannot silently diverge from the pack / vocab
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_symmetric_table_matches_pack_ontology(pack) -> None:
|
|
"""SYMMETRIC_PREDICATES MUST equal the pack's graph.edge.symmetric declarations —
|
|
the pack is the source of truth; the constant is the runtime-cheap mirror."""
|
|
assert SYMMETRIC_PREDICATES == load_relational_pack_symmetric()
|
|
|
|
|
|
def test_algebra_members_are_relational_predicates() -> None:
|
|
"""Every inverse/symmetric lemma is a real reader predicate — a typo cannot mint
|
|
an unknown predicate into the determination path."""
|
|
for lemma in set(INVERSE_OF) | SYMMETRIC_PREDICATES:
|
|
assert lemma in RELATIONAL_PREDICATES
|
|
|
|
|
|
def test_inverse_is_an_involution() -> None:
|
|
"""inverse(inverse(p)) == p, and no predicate is its own inverse (asymmetry)."""
|
|
for lemma, other in INVERSE_OF.items():
|
|
assert lemma != other
|
|
assert INVERSE_OF[other] == lemma
|