core/evals/setup_oracle
Shay ef06923866 feat(comprehension): additive aggregate query variants — 'altogether' / 'in total'
Widen the aggregate-query recognizer so a multi-part total may be asked with a
trailing qualifier after 'have': 'How many X do A and B have altogether?' and
'... in total?'. The qualifier is stripped and honored ONLY for the multi-part
(sumquery) form — a single-entity query carrying it refuses, guarded by
'not aggregate'.

Phrasing-only: no new arithmetic, no new relation kind, no inverse solving, no
distractor/pronoun handling. The parts still flow through the existing sum_of;
an ungrounded part (unit unbound) or a unit-incompatible part (unit mismatch) is
refused downstream by the REAL admissibility check, so the recognizer cannot
over-read. Off-serving organ only (no generate.derivation / reliability_gate).

Flips r1-03 (more+altogether -> 25) and r1-04 (fewer+in total -> 34):
  R1 setup   6 / 0 / 4    R1 answers 6 / 0 / 4  (setup_wrong 0, gold_error 0)
  15-case 15 / 0 / 0      29 quant tests, 102 affected-file tests green

Tests are meaningful-fail: the single-entity-qualifier, ungrounded-part, and
unit-incompatible-part refusals each fail loudly if their guard is removed.
2026-06-07 05:49:28 -07:00
..
__init__.py test(comprehension): expose R1 answer lane 2026-06-06 18:28:49 -07:00
__main__.py test(comprehension): expose R1 answer lane CLI 2026-06-06 18:31:41 -07:00
expected_units.json
r1_gold.jsonl feat(comprehension): additive aggregate query variants — 'altogether' / 'in total' 2026-06-07 05:49:28 -07:00
runner.py feat(comprehension): add R1 answer-oracle lane 2026-06-06 18:26:30 -07:00
signature.py feat(comprehension): the divisive comparative frame — "half as many" as exact integer division (PR-6c) 2026-06-06 20:18:39 -07:00