core/evals/rate_oracle
Shay e77f93c485 test(rate-oracle): port _canonical_outcome non-vacuous validation to R3 (R3-vac)
Closes the sibling vacuity CMB-a surfaced: R3's evals/rate_oracle solver_refuses branch
accepted any fixture labelled non_integer_solution without checking the arithmetic, and the
solved branch never recomputed the gold. A curator could certify an arithmetically-impossible
refusal or a wrong gold (with a self-consistent answer key) as valid — the 'obligation that
can't meaningfully fail = decoration' case CLAUDE.md forbids.

Adds _canonical_outcome (independent reimplementation of the single-rate algebra WITH R3.2
conversion — NOT a call to generate.rate_comprehension.solver, which would be circular). The
validator now cross-checks every solved gold (== computed) and every solver_refuses reason
(genuine) against it, with dedicated meaningful-fail tests anchored on hand-written literals.
Also removes the dead determinism self-comparison.

R3 gold stays 13/13 valid (7/2/4); reader 9/0/4 unchanged; 26 oracle/reader tests pass.
Off-serving; serving + other organs untouched.
2026-06-08 07:38:20 -07:00
..
__init__.py feat(rate): R3 RateProblem model + rate gold + setup oracle (R3b) 2026-06-07 23:01:25 -07:00
__main__.py feat(rate): single-rate reader + run_reader lane + answer-choice reuse (R3d) 2026-06-07 23:07:51 -07:00
rate_gold.jsonl feat(rate): exact time-unit conversion for single-rate (R3.2b-e) 2026-06-08 05:43:51 -07:00
runner.py test(rate-oracle): port _canonical_outcome non-vacuous validation to R3 (R3-vac) 2026-06-08 07:38:20 -07:00
signature.py feat(rate): exact time-unit conversion for single-rate (R3.2b-e) 2026-06-08 05:43:51 -07:00