core/evals/comprehension
Shay 6f70e834ff
feat: one-hop sound relational entailment (inverse/symmetric) + capability-index lane (#775)
* feat(determine): one-hop sound relational entailment — inverse/converse + symmetric

Mastery-v2 Step 3 lead capability (core). DETERMINE could perceive relational
structure (16 predicates, breadth-complete) but could not derive the simplest
entailed relational facts — it read only the stored direction. This adds two
SOUND one-hop rules that read a stored edge in its other lawful direction:

  INVERSE/converse   greater_than(a,b)  <=  told less_than(b,a)
  SYMMETRIC          sibling_of(b,a)    <=  told sibling_of(a,b)

Strictly scoped (per review): OPEN-WORLD (asserts only True, never False — the
answer=False path stays unbuilt, INV-30's firewall holds), ONE hop (NO transitive
chaining), DECLARED rules only. Ontology/metadata-driven, not prose intuition:

- generate/meaning_graph/relational.py: declarative algebra tables — _INVERSE_PAIRS
  (the converse edges; the pack carries no inverse metadata) + INVERSE_OF (derived,
  an involution) + SYMMETRIC_PREDICATES (mirrors the pack's graph.edge.symmetric tag).
  load_relational_pack_symmetric() reads the pack ontology; a test pins the constant
  equal to it (no silent divergence).
- generate/determine/determine.py: _relational_one_hop() between direct entailment
  and transitive subsumption; new Determined.rule provenance field (direct/inverse/
  symmetric/subsumption) — replay-safe (render reads only basis; trace hashes surface).

wrong=0 confuser block proves the rule cannot over-fire: less_than is not
self-inverse, sibling_of does not imply parent_of, greater_than does not imply
equal_to, NO transitive chain (direct or through-inverse), and no answer=False is
ever emitted. An obsolete pin (test_symmetric_converse_is_not_faked, which asserted
the pre-rule direct-only floor) is updated to the new sound behavior and a preserved
asymmetric-converse bite.

Tests: test_determine_relational_inference.py (13, the capability contract) +
updated test_relational_reader.py. Broad relational/determine/grounding +
capability_index baseline sweep: 114 green; baseline digest UNCHANGED (this is a
determination capability; the comprehension lanes are untouched).

REMAINING for the full lead PR (next): the measurement lane — an independent-oracle
evals/relational_inference lane + a capability_index adapter + baseline re-freeze
(breadth 9->10, wrong_total still 0), so the capability registers on the yardstick.

INTEGRATION NOTE (cross-PR): this adds 2 Determined() construction sites (now 4:
direct/inverse/symmetric/subsumption, ALL answer=True). When rebased onto main with
INV-30 (PR #770), update INV-30's test_determine_construction_sites_are_visible count
2 -> 4 and confirm all four assert True. Merge order: #770 -> this.

* feat(capability-index): relational-inference lane — breadth 9->10, wrong=0

Mastery-v2 Step 3 lead, measurement half. Puts the one-hop relational-inference
capability ON the capability index with an independent-oracle lane:

- evals/relational_inference/v1/: cases.jsonl (13 positive: 8 inverse + 5 symmetric,
  authored from the relation algebra INDEPENDENTLY of determine/relational per
  INV-25/27), refusals.jsonl (8 confusers that MUST refuse), provenance.md (incl. the
  honest overlaps_event reader-surface coverage gap).
- evals/comprehension/relational_inference_runner.py: told fact(s) -> determine(query)
  vs gold; refusal = coverage miss, disagreement = wrong (structurally 0 on positives).
- evals/capability_index/adapters.py: comprehension_relational_inference_result added
  to ADAPTERS.
- baseline.json re-frozen: breadth 9 -> 10, capability_score 0.94403 -> 0.949483,
  wrong_total still 0, digest deliberately changed (35dea2b2...).
- tests/test_relational_inference_lane.py: SHA-pinned gold, positive wrong=0 +
  coverage>0, and the confuser wrong=0 BITE (every confuser must refuse).

Also reconciles INV-30 (now in main via #770): determine.py grew from 2 to 3
Determined() construction sites (direct, the shared relational one-hop constructor,
transitive subsumption) — ALL answer=True. test_determine_construction_sites_are_visible
updated 2 -> 3 (correcting the prior commit note's overcount of 4; inverse and symmetric
share the single _relational_determined constructor, so it is ONE new site).

Also fixes a .gitignore gap: ADR-0219 gen-dir checkpoints (engine_state/current,
engine_state/gen-*/) were not ignored (only the old flat engine_state/*.json patterns
were), so runtime checkpoints could be committed by accident.

Verification: new lane 13/0/0 coverage 1.0; capability_baseline digest matches the
re-freeze; INV-30 + architectural invariants green; smoke 99/99.

* docs+test: reconcile stale relational-inference docstrings; assert rule provenance (review #775)

Addresses two review-requested cleanups on #775:

1. Stale docstrings. generate/meaning_graph/relational.py said the symmetric converse
   is "a sound-but-incomplete refusal at DETERMINE" and there is "NO transitive/
   symmetric/rule inference"; generate/determine/determine.py said "symmetric-converse
   questions are Undetermined", "no transitive/symmetric/rule inference", and "asserts
   only answer=True on a direct hit". All false after the one-hop relational algebra
   landed. Updated: the reader does direct-reading only; DETERMINE applies declared
   one-hop inverse/converse + pack-declared symmetric (plus the existing member/subset
   subsumption), still open-world / never answer=False; transitive relational closure,
   negation, and closed-world falsehood remain out of scope.

2. Rule-provenance gold now meaningful. The relational_inference runner compared only
   (answer, predicate, subject, object), ignoring res.rule though the gold carries
   "rule": "inverse"/"symmetric". Added res.rule to the got/gold tuple so a "right
   answer via the wrong rule path" can no longer pass silently. Lane still 13/0/0;
   baseline counts (and digest) unchanged.

Verification: runner 13/0/0; lane + capability_baseline + unit + INV-30 green (22).
2026-06-15 11:39:41 -07:00
..
__init__.py
CANONICALIZATION.md feat(comprehend): multi-word NP chunking under a canonicalization contract 2026-06-05 22:47:34 -07:00
propositional_runner.py feat(comprehend): propositional-logic comprehension (4th domain, flagship oracle) 2026-06-05 23:24:54 -07:00
relational_inference_runner.py feat: one-hop sound relational entailment (inverse/symmetric) + capability-index lane (#775) 2026-06-15 11:39:41 -07:00
relational_metric_runner.py feat(comprehend): arithmetic word-problems via binding_graph (5th domain, real admissibility) 2026-06-06 00:43:16 -07:00
relational_predicate_runner.py feat(measure): put the relational reader (#596) on the capability index 2026-06-06 10:09:15 -07:00
set_membership_runner.py
syllogism_runner.py
total_ordering_runner.py