core/evals/setup_oracle
Shay e9cbe65d77 feat(comprehension): the multiplicative comparative frame — first R1 capability (PR-5c)
The first capability slice on the R1 arc, gated by the setup-oracle: turn the
"twice / N times as many" reading from REFUSED into a correct setup, without a single
misread. Builds on the typed IR (PR-4) and the R1 gold (PR-5b).

- IR: a Mul(symbol, literal-factor) node — to_canonical_string "ref * factor",
  operation_kind "multiply", dependencies {ref}, to_relation -> times_as_many. The
  product keeps the symbol's unit (count * scalar = count), admitted by the REAL
  check_admissibility multiply path (the literal factor is dimensionless, not a dep).
- Reader: a multiplicative template "Y has <factor> as many <unit> as X" (factor word:
  twice/double/triple/quadruple) and "Y has <N> times as many <unit> as X", checked
  BEFORE the digit gate (the factor may be a word). 'half' (a /2) is deliberately
  deferred — divide-by-literal is a separate admissibility path.
- setup-oracle: relation_signature now canonicalizes times_as_many.

Setup-oracle R1 result: 2 setup_correct (r1-01 twice; r1-05 the multi-step chain
ivy/jon=3*ivy/kim=jon+2), 0 setup_WRONG, 8 setup_refused. Every hard negative stays a
safe refusal: missing-base (Rosa ungrounded), ambiguous referent, distractor, inverse,
partition, 'altogether'/'in total' phrasings, and 'half' (divide). wrong=0 held through
the first capability addition.

Gates green: setup-oracle R1 setup_wrong=0; 15-case setup gate 15/15 setup_wrong=0;
relational_metric answer lane 15/15 wrong=0; binding-graph admissibility + realize +
architectural invariants + chat-runtime + pipeline (122+). No serving path touched (this
reader feeds the relational_metric / setup-oracle lanes, not the candidate-graph serving).
2026-06-06 17:29:23 -07:00
..
__init__.py feat(setup-oracle): independent R1 gold + the reader refuses, never misreads (PR-5b) 2026-06-06 17:18:45 -07:00
__main__.py feat(setup-oracle): independent R1 gold + the reader refuses, never misreads (PR-5b) 2026-06-06 17:18:45 -07:00
expected_units.json feat(setup-oracle): make the ruler UNIT-AWARE (setup-oracle v2, PR-5a) 2026-06-06 17:09:45 -07:00
r1_gold.jsonl feat(setup-oracle): independent R1 gold + the reader refuses, never misreads (PR-5b) 2026-06-06 17:18:45 -07:00
runner.py feat(setup-oracle): independent R1 gold + the reader refuses, never misreads (PR-5b) 2026-06-06 17:18:45 -07:00
signature.py feat(comprehension): the multiplicative comparative frame — first R1 capability (PR-5c) 2026-06-06 17:29:23 -07:00