core/evals/relational_transitive/v1/cases.jsonl
Shay 1ff06726a6
feat(determine): add transitive strict-order relational inference (#781)
* feat(determine): add transitive strict-order relational inference

Add sound transitive closure for declared strict-order relational predicates —
less_than, greater_than, before_event, after_event, and ONLY these. A query
p(a, c) determines True when a same-predicate chain p(a, b), p(b, c), ... over the
predicate's OWN realized edges is found (BFS reachability) and the proof_chain ROBDD
verifies the transitive entailment (search-then-verify, through the UNCHANGED
evaluate_entailment via a new lower_transitive_chain). Mirrors _determine_subsumption.

Scope / firewall:
- TRANSITIVE_PREDICATES is closed and default-off; sibling_of, parent_of,
  left_of/right_of, inside_of, during_event, overlaps_event are EXCLUDED.
- Same-predicate edges only — no transitive-through-inverse, no cross-predicate
  (triple firewall: recall filter + lowering re-reject + ROBDD re-verify).
- Open-world: asserts only answer=True via the shared _relational_determined
  surface (rule="transitive"), so INV-30 stays at exactly 3 Determined sites and
  no answer=False is added. One-hop inverse/symmetric (#775) is unchanged.
  FrameVerdict / closed-world (ADR-0222) is untouched.

wrong=0 bite (unit + lane + independent oracle): non-transitive-predicate chains,
non-admitted spatial chains, mixed-predicate chains, disjoint chains, reflexive
cycles, and inverse+transitive composition all REFUSE.

Measurement: new evals/relational_transitive lane with an INDEPENDENT BFS
transitive-closure oracle (imports no engine module; INV-25/27), cross-checked both
directions (positives oracle-True, confusers oracle-False). Capability-index breadth
10 -> 11, wrong_total 0, deterministic digest re-frozen.

Cross-PR reconcile: migrated the one-hop confuser rinf-ref-001 (a less_than chain,
which now correctly determines transitively) to a B2 positive; repurposed its unit
test to a non-transitive parent_of chain; corrected a pre-existing stale breadth==9
assertion (#775 added a 10th domain but never updated test_capability_index).

Verified in the worktree: determine + transitive/one-hop lanes + capability
baseline/index + ProofWriter-OWA floor + INV-30/29/21/02 + full smoke — all green.
A 3-skeptic read-only adversarial review found no wrong=0 leak.

* test(determine): B2 hardening — TRANSITIVE_PREDICATES pin + lower_transitive defensive + budget

Add the required B2 hardening tests before merge:

- test_transitive_predicates_closed_and_excludes: the table is EXACTLY
  {less_than, greater_than, before_event, after_event}, every member is in
  RELATIONAL_PREDICATES, and every deliberately-excluded predicate (sibling_of,
  spouse_of, parent_of, child_of, left_of, right_of, inside_of, during_event,
  overlaps_event) is explicitly absent. (Closes the gap where relational.py's
  comment claimed this test existed but it did not.)

- tests/test_composition_lower_transitive.py: direct defensive tests on
  lower_transitive_chain — empty path, mislabeled (cross-predicate) edge, wrong
  arity, non-contiguous path, and path-not-reaching-target all lower to None
  (refuse); plus exact 2-hop and 3-hop theory pins.

- test_edge_budget_exhaustion_refuses: above _TRANSITIVE_EDGE_BUDGET the transitive
  search declines (a safe coverage refusal), never proves.
2026-06-15 14:20:30 -07:00

7 lines
1.9 KiB
JSON

{"id": "rtrans-001", "facts": ["Alice is less than Bob.", "Bob is less than Carol."], "query": "Is Alice less than Carol?", "edges": [["less_than", "alice", "bob"], ["less_than", "bob", "carol"]], "query_edge": ["less_than", "alice", "carol"]}
{"id": "rtrans-002", "facts": ["Amy is less than Bea.", "Bea is less than Cleo.", "Cleo is less than Dana."], "query": "Is Amy less than Dana?", "edges": [["less_than", "amy", "bea"], ["less_than", "bea", "cleo"], ["less_than", "cleo", "dana"]], "query_edge": ["less_than", "amy", "dana"]}
{"id": "rtrans-003", "facts": ["Carol is greater than Bob.", "Bob is greater than Alice."], "query": "Is Carol greater than Alice?", "edges": [["greater_than", "carol", "bob"], ["greater_than", "bob", "alice"]], "query_edge": ["greater_than", "carol", "alice"]}
{"id": "rtrans-004", "facts": ["Delta is greater than Charlie.", "Charlie is greater than Bravo.", "Bravo is greater than Alpha."], "query": "Is Delta greater than Alpha?", "edges": [["greater_than", "delta", "charlie"], ["greater_than", "charlie", "bravo"], ["greater_than", "bravo", "alpha"]], "query_edge": ["greater_than", "delta", "alpha"]}
{"id": "rtrans-005", "facts": ["Dawn is before noon.", "Noon is before dusk."], "query": "Is dawn before dusk?", "edges": [["before_event", "dawn", "noon"], ["before_event", "noon", "dusk"]], "query_edge": ["before_event", "dawn", "dusk"]}
{"id": "rtrans-006", "facts": ["Dawn is before noon.", "Noon is before dusk.", "Dusk is before midnight."], "query": "Is dawn before midnight?", "edges": [["before_event", "dawn", "noon"], ["before_event", "noon", "dusk"], ["before_event", "dusk", "midnight"]], "query_edge": ["before_event", "dawn", "midnight"]}
{"id": "rtrans-007", "facts": ["Dusk is after noon.", "Noon is after dawn."], "query": "Is dusk after dawn?", "edges": [["after_event", "dusk", "noon"], ["after_event", "noon", "dawn"]], "query_edge": ["after_event", "dusk", "dawn"]}