core/evals/gsm8k_math/train_sample
Shay da9f89ae16 chore(eval): regenerate train_sample/v1 report.json after 86d4e98 multi-word unit fix
86d4e98 (multi-word unit grounding fix) changed refusal reasons for two
cases without changing the 3/47/0 count:
- case 0019 (currency_amount): refusal moves from 'requires 3 vet
  appointments which cost $400 each' to 'After the first appointment,
  John paid $100 for pet insurance...' — first sentence now passes,
  refusal moves to subsequent sentence
- case 0023 (Nicole/Pokemon cards): refusal moves from 'Nicole collected
  400 Pokemon cards' (now passes via multi-word unit grounding) to
  'Cindy collected twice as many, and Rex collected half of...'

Counts unchanged: correct=3 refused=47 wrong=0. Updates report.json to
match current behavior so subsequent eval runs are byte-deterministic
from the committed snapshot.
2026-05-28 08:09:51 -07:00
..
v1 chore(eval): regenerate train_sample/v1 report.json after 86d4e98 multi-word unit fix 2026-05-28 08:09:51 -07:00
__init__.py feat(gsm8k): ADR-0126 P6 — train-sample runner + exit-criterion gate 2026-05-23 06:33:06 -07:00