Commit graph

4 commits

Author SHA1 Message Date
Shay
0951d80e04 feat(comprehension): the divisive comparative frame — "half as many" as exact integer division (PR-6c)
PR-6c adds the divisive comparative frame: "half as many" read as EXACT INTEGER
DIVISION. It is the divisor twin of PR-5c's multiplicative frame, and moves the
independent R1 gold's r1-02-half from refused → correct.

No serving path touched. No rational/fractional answer support added. Non-exact
division refuses.

Design (ADR-0134 amended — divide made symmetric with multiply):
- `_check_divide` now admits a SINGLE-DEP divide-by-dimensionless-literal
  (item / dimensionless = item), the exact twin of single-dep multiply. The
  2-dep rate-divide path is untouched. This keeps the IR's "literal operands
  are not deps" invariant (proven in PR-6a) uniform across Mul AND Div, so the
  reader builds both without a per-op special case and WITHOUT synthesizing a
  divisor symbol that would pollute the setup-oracle's unit signature.
- `Div(Symbol, Literal)` IR node: "ref / divisor", operation_kind "divide",
  projects to `divide_by`. Divisor-only contract mirrors the scalar-only one.
- Reader: `_DIVISOR_WORDS={half:2}` slots into the same 8-token "<WORD> as many"
  template as the factor words; graph carries only the two entities.
- Gold reconciliation: r1-02 placeholder `times_as_many factor 0.5` → exact
  `divide_by divisor 2` (gold 4). Makes the INDEPENDENT gold integer-faithful.

The wrong=0 boundary — exact divisibility:
  the oracle admits `divide_by` only when `base % divisor == 0`. An odd base
  halved REFUSES (gold_error), never floors to a wrong integer. Divisor must be
  a nonzero int (0, 0.5, 1.5, bool all refuse); divisor=1 is intentionally the
  identity (pinned). admissibility proves DIMENSION; the oracle proves EXACT VALUE.

Meaningful-fail (CLAUDE.md Schema-Defined Proof Obligations), both verified red:
- drop the `% divisor` guard → test_oracle_refuses_non_exact_division fails (returns 3).
- disable the single-dep divide branch → the admissibility test AND the reader's
  `half` test fail (admissibility refuses → reader refuses → half stays refused).

Gates:
  R1 setup:   3 correct / 0 wrong / 7 refused
  R1 answers: 3 correct / 0 wrong / 7 refused / setup_wrong 0 / gold_error 0
  15-case setup: 15 / 0 / 0
  91 PR-6c tests + 60 relational lanes + 56 architectural invariants + 502
  binding-graph/proof-chain/adapter tests green. All 8 SHA-content lanes match
  (serving unmoved; admissibility has no generate.derivation/reliability_gate consumer).
2026-06-06 20:18:39 -07:00
Shay
e9cbe65d77 feat(comprehension): the multiplicative comparative frame — first R1 capability (PR-5c)
The first capability slice on the R1 arc, gated by the setup-oracle: turn the
"twice / N times as many" reading from REFUSED into a correct setup, without a single
misread. Builds on the typed IR (PR-4) and the R1 gold (PR-5b).

- IR: a Mul(symbol, literal-factor) node — to_canonical_string "ref * factor",
  operation_kind "multiply", dependencies {ref}, to_relation -> times_as_many. The
  product keeps the symbol's unit (count * scalar = count), admitted by the REAL
  check_admissibility multiply path (the literal factor is dimensionless, not a dep).
- Reader: a multiplicative template "Y has <factor> as many <unit> as X" (factor word:
  twice/double/triple/quadruple) and "Y has <N> times as many <unit> as X", checked
  BEFORE the digit gate (the factor may be a word). 'half' (a /2) is deliberately
  deferred — divide-by-literal is a separate admissibility path.
- setup-oracle: relation_signature now canonicalizes times_as_many.

Setup-oracle R1 result: 2 setup_correct (r1-01 twice; r1-05 the multi-step chain
ivy/jon=3*ivy/kim=jon+2), 0 setup_WRONG, 8 setup_refused. Every hard negative stays a
safe refusal: missing-base (Rosa ungrounded), ambiguous referent, distractor, inverse,
partition, 'altogether'/'in total' phrasings, and 'half' (divide). wrong=0 held through
the first capability addition.

Gates green: setup-oracle R1 setup_wrong=0; 15-case setup gate 15/15 setup_wrong=0;
relational_metric answer lane 15/15 wrong=0; binding-graph admissibility + realize +
architectural invariants + chat-runtime + pipeline (122+). No serving path touched (this
reader feeds the relational_metric / setup-oracle lanes, not the candidate-graph serving).
2026-06-06 17:29:23 -07:00
Shay
4056069152 feat(setup-oracle): make the ruler UNIT-AWARE (setup-oracle v2, PR-5a)
Strengthen the ruler before it judges real GSM8K frames — NOT a capability change, no
new parser behavior, no serving path. The setup-oracle now grades UNITS, read from the
binding-graph itself, against an independent expected-unit gold.

- New per-symbol unit axis (symbol_unit_signature) — read from the GRAPH's symbols
  (reader_symbol_units), not the answer projection; covers fact + relation-operand units
  (operands are symbols). A None unit canonicalizes to "unset" so an unmodelled symbol
  never silently matches a declared gold unit.
- The question-target signature gains the target's expected_unit (reader from
  BoundUnknown.expected_unit; gold from the fixture).
- expected_units.json — independent, hand-authored MODELING unit per entity for the 15
  cases (a discrete sortal count -> the generic count unit "item"; money -> "dollars").
  Authored from the PROBLEM, not copied from the reader.
- The reader's per-symbol units match the independent gold on all 15 (setup_wrong=0) —
  agreement, not circularity (both apply the same correct count->item / money modelling).
- Meaningful-fail: a reading whose STRUCTURE matches but whose UNIT diverges now FAILS
  (test_unit_mismatch_is_caught, test_target_unit_mismatch_is_caught). The ruler is not
  decoration on the unit axis either.

Gate held: setup-oracle 15/15 setup_wrong=0 (now structure + units + target). The
relational_metric answer lane is untouched (oracle-only change). No generate/ change.

This is the unit-aware ruler PR-5b (independent R1 gold) and PR-5c (the first R1
comparative/derived-symbol frame) must pass before serving.
2026-06-06 17:09:45 -07:00
Shay
59974865ef feat(comprehension): question target in the graph (PR-1) + setup-oracle lane (grade the reading)
Two coupled, additive, off-serving changes toward the typed math-comprehension organ.
No serving path touched; the relational_metric answer lane stays 15/15 wrong=0.

PR-1 — QuantQuery → BoundUnknown. comprehend_quantitative now emits the question
target as a BoundUnknown INSIDE the binding-graph (symbol_id, state_index="terminal",
question_form "count"|"total", expected_unit), so the graph is a real question-bearing
mathematical object and its canonical serialization carries the target. The external
QuantQuery is RETAINED, consistent-by-construction, so the two consumers
(to_relational_metric, realize/quantitative) are byte-identical; a follow-up rewires
them onto graph.unknowns and drops the duplicate field.

Setup-oracle lane (evals/setup_oracle) — grade the READING, not the answer. The
relational_metric lane scores answers, which can bless a semantically-wrong derivation
that coincidentally lands on the right number (the exact hazard the held-out
measurements + the 2/87 resolve_pooled probe exposed). The setup-oracle compares the
reader's comprehended STRUCTURE — a span-free signature of facts + typed equations +
the BoundUnknown target — against the INDEPENDENT gold structure (the relational_metric
cases' own relations/query, authored separately from the binding-graph reader). A
structural mismatch is setup_wrong, the wrong=0-critical count, even when the answer
would be right. v1 grades structure (units deferred — covered by admissibility). The
reader reads all 15 cases with the gold structure (setup_wrong=0); a meaningful-fail
test proves the oracle catches a right-answer/wrong-structure reading (it is not
decoration). `python -m evals.setup_oracle` exits nonzero iff setup_wrong > 0.

This is the measurement rig BEFORE investing in frame families: setup_wrong=0 is the
gate; serving must not move while setup_wrong > 0. It is the first milestone of the
math-comprehension organ, not a path to "solve GSM8K".

Verified: setup-oracle 15/15 setup_correct wrong=0; quantitative + setup-oracle unit
tests (17); realize-binding-graph + binding-graph + architectural invariants (183).
2026-06-06 16:40:15 -07:00