Commit graph

2 commits

Author SHA1 Message Date
Shay
a005a92fed feat(comprehend): arithmetic word-problems via binding_graph (5th domain, real admissibility)
The binding-graph's FIRST comprehension consumer (doctrine-aligned: quantities live
in binding_graph, NOT the MeaningGraph). generate/quantitative_comprehension.py
reads arithmetic prose into SymbolBinding/BoundFact/BoundEquation and runs the REAL
check_admissibility (shell -> verify -> rebuild with the actual UnitProof) — there
is NO stamped "admitted": an equation is admitted only if its operand units verify.
Then to_relational_metric projects the binding-graph to the independent
relational_metric oracle for the verdict.

Templates (digits only; non-digit quantity REFUSES):
  "<X> has <N> <unit>"                 -> BoundFact(X = N)
  "<Y> has <N> more <unit> than <X>"   -> BoundEquation(Y = X + N)  op=add
  "<Y> has <N> fewer <unit> than <X>"  -> BoundEquation(Y = X - N)  op=subtract
  "How many <unit> does <Y> have"      -> ask Y
  "How many <unit> do <X> and <Y> have"-> total = X + Y; ask total

Unit modelling (honest, not faked): a noun the closed en_units_v1 pack knows is
used verbatim (dollars -> dollar/money); an UNKNOWN sortal noun (stickers, coins)
is a count of discrete objects -> the existing 'item' lemma (dimension count). So
admissibility stays a REAL check: count+count admits, count+money (a mixed-unit
sum) REFUSES with unit_mismatch — verified to bite.

comprehension_relational_metric: 15/15 wrong=0 (full coverage). Located OUTSIDE
generate/meaning_graph (it targets binding_graph, not the MeaningGraph) so INV-28
neutrality stays intact; oracle imports none of the SUT (new INV-25 lane).
Capability index breadth 7->8, score 0.928622 -> 0.937258, wrong_total 0, digest
50e0675b…

Tests: reader templates + count/known-unit modelling + admissibility-bite (mixed
unit refuses) + non-digit refusal; end-to-end full-coverage wrong=0; arithmetic
added to the structure-preservation generative panel (projected relations+query ==
ground truth); capability breadth 7->8; INV-25 arithmetic lane. 93 targeted + 90
smoke green; lane SHAs 8/9 (sole miss = public_demo env flake; deductive_logic +
math_teaching unchanged -> no GSM8K coupling).
2026-06-06 00:43:16 -07:00
Shay
f63e790ab0 test(comprehend): structure-preservation + perturbation invariance (close coincidental-correctness)
Addresses the central review finding: the generative wrong=0 tests compared oracle
VERDICTS, so a misread graph that coincidentally yields the same verdict passed
silently (coincidental correctness in cleaner clothes). This adds the conjugate
check — the reader must recover the EXACT structure the prose encodes, not merely a
verdict-equivalent one.

tests/test_comprehension_structure_preserving.py:
  - Structure preservation (all 4 domains): over randomly generated structures
    rendered to prose that FULLY determines them, assert projected structure AND
    query == ground truth exactly (order-insensitive canonicalization), or refuse.
    Empirically this is strictly stronger: under a subject<->predicate premise swap,
    361/400 reads are structurally wrong and 307 of those (85%) coincide in verdict
    — the answer test misses all 307; the structure test catches all 361.
  - Perturbation invariance: meaning-preserving surface changes (premise/clause
    reordering, capitalization, extra whitespace) yield the SAME structure.

The existing answer-preservation property tests stay (verdict agreement is still a
valid, separate check — exactly the "assert structure, then separately assert
oracle agreement" the review recommends). Tests-only; no source change; capability
index unchanged. 7 new + 86 comprehension/capability targeted green.
2026-06-05 23:59:20 -07:00