Strengthen the ruler before it judges real GSM8K frames — NOT a capability change, no
new parser behavior, no serving path. The setup-oracle now grades UNITS, read from the
binding-graph itself, against an independent expected-unit gold.
- New per-symbol unit axis (symbol_unit_signature) — read from the GRAPH's symbols
(reader_symbol_units), not the answer projection; covers fact + relation-operand units
(operands are symbols). A None unit canonicalizes to "unset" so an unmodelled symbol
never silently matches a declared gold unit.
- The question-target signature gains the target's expected_unit (reader from
BoundUnknown.expected_unit; gold from the fixture).
- expected_units.json — independent, hand-authored MODELING unit per entity for the 15
cases (a discrete sortal count -> the generic count unit "item"; money -> "dollars").
Authored from the PROBLEM, not copied from the reader.
- The reader's per-symbol units match the independent gold on all 15 (setup_wrong=0) —
agreement, not circularity (both apply the same correct count->item / money modelling).
- Meaningful-fail: a reading whose STRUCTURE matches but whose UNIT diverges now FAILS
(test_unit_mismatch_is_caught, test_target_unit_mismatch_is_caught). The ruler is not
decoration on the unit axis either.
Gate held: setup-oracle 15/15 setup_wrong=0 (now structure + units + target). The
relational_metric answer lane is untouched (oracle-only change). No generate/ change.
This is the unit-aware ruler PR-5b (independent R1 gold) and PR-5c (the first R1
comparative/derived-symbol frame) must pass before serving.
Two coupled, additive, off-serving changes toward the typed math-comprehension organ.
No serving path touched; the relational_metric answer lane stays 15/15 wrong=0.
PR-1 — QuantQuery → BoundUnknown. comprehend_quantitative now emits the question
target as a BoundUnknown INSIDE the binding-graph (symbol_id, state_index="terminal",
question_form "count"|"total", expected_unit), so the graph is a real question-bearing
mathematical object and its canonical serialization carries the target. The external
QuantQuery is RETAINED, consistent-by-construction, so the two consumers
(to_relational_metric, realize/quantitative) are byte-identical; a follow-up rewires
them onto graph.unknowns and drops the duplicate field.
Setup-oracle lane (evals/setup_oracle) — grade the READING, not the answer. The
relational_metric lane scores answers, which can bless a semantically-wrong derivation
that coincidentally lands on the right number (the exact hazard the held-out
measurements + the 2/87 resolve_pooled probe exposed). The setup-oracle compares the
reader's comprehended STRUCTURE — a span-free signature of facts + typed equations +
the BoundUnknown target — against the INDEPENDENT gold structure (the relational_metric
cases' own relations/query, authored separately from the binding-graph reader). A
structural mismatch is setup_wrong, the wrong=0-critical count, even when the answer
would be right. v1 grades structure (units deferred — covered by admissibility). The
reader reads all 15 cases with the gold structure (setup_wrong=0); a meaningful-fail
test proves the oracle catches a right-answer/wrong-structure reading (it is not
decoration). `python -m evals.setup_oracle` exits nonzero iff setup_wrong > 0.
This is the measurement rig BEFORE investing in frame families: setup_wrong=0 is the
gate; serving must not move while setup_wrong > 0. It is the first milestone of the
math-comprehension organ, not a path to "solve GSM8K".
Verified: setup-oracle 15/15 setup_correct wrong=0; quantitative + setup-oracle unit
tests (17); realize-binding-graph + binding-graph + architectural invariants (183).
