core/evals/math_capability_axes
Shay 52f2bf6f4c
feat(ADR-0136.S.1): rate/event statement parsing — capacity + earnings shapes, axis lane 20/20, wrong==0, gsm8k-0014 admits (#201)
* docs(ADR-0136.S.0): refusal taxonomy + S.1 brief for rate/event statement corridor

Taxonomy: deterministic classification of all 50 GSM8K train-sample refused cases
into primary + secondary barriers. Key findings:

  context_filler (primary): 23/50 — legitimately refuses; not parser gaps
  compound_statement:         5/50 — two ops in one sentence
  rate/capacity class:        4/50 — direct S.1 targets
  distributive_multiply:      1/50 primary, 5/50 secondary
  long-tail (diverse):       17/50

Honest S.1 ceiling: 0/50 → ≤4/50 admission. gsm8k-0014 ('Bob can shuck 10
oysters in 5 minutes') is the only case with capacity_rate as sole barrier.

Ships:
- evals/gsm8k_math/train_sample/v1/refusal_taxonomy.json (schema v1, 50 records)
- docs/briefs/parallel-2026-05-23/L17-ADR-0136-S1-rate-event-statements.md
- full briefs archive (parallel-2026-05-23)

No implementation changes. Taxonomy and brief only.

* feat(ADR-0136.S.1): rate/event statement parsing — capacity + earnings shapes, axis lane 20/20, wrong==0, gsm8k-0014 admits

Two closed statement shapes added to candidate parser and graph:

Shape A (capacity-rate): "<Actor> can <verb> N <unit> in M <time-unit>"
  - 13 closed verbs (shuck/pick/pack/make/produce/type/read/write/paint/run/score/answer/complete)
  - Pronoun question form (he/she/they/it) accepted
  - Time-unit conversion (second/minute/hour/day)

Shape B (earnings-rate): "<Actor> <verb> $N per/an/a <time-unit>"
  - 5 closed verbs (make/earn/receive/get/charge)
  - Currency: $ only, 0-2 decimal places
  - Per-token alternation: per/a/an/for each/every

Short-circuit paths in parse_and_solve run before the Cartesian product,
computing rate_per_sec × T_seconds directly. Actor mismatch → refusal
(not wrong). Answer ≤ 0 → fall through to refusal.

GSM8K honest delta: 0/50 → 1/50 (gsm8k-0014: answer=240.0, correct).
23 context-filler cases correctly remain refused.
Axis lane: 20/20 pass, wrong=0.
B3 bounded-grammar lane: unchanged (wrong=0).
35 new tests including B3 regression guard and GSM8K admitted_wrong=0 rail.
2026-05-23 20:36:01 -07:00
..
G1_verb_classes/v1 feat(G1): verb-classes capability axis (ADR-0131.G.1) 2026-05-23 15:39:14 -07:00
G2_comparatives fix(ADR-0131.G.2): rebase + mastery hardening — quarter/third fraction anchors, gate regex, boundary refusals (#196) 2026-05-23 19:28:09 -07:00
G3_numerics feat(ADR-0131.G.3.1): numerics extensions — fractions + multi-currency + multi-token cardinals + word-num-adjective 2026-05-23 15:16:46 -07:00
G4_multi_clause feat(ADR-0131.G.4): multi-clause composition (conj subjects + conj objects + embedded quantifiers + conj embedded) — admission 0/50 (Δ0), multi-clause refusals 2→1 2026-05-23 14:43:16 -07:00
G5_aggregate feat(ADR-0131.G.5): aggregate answer composition — combined/together cues wired, axis lane 20/20, wrong==0 (#197) 2026-05-23 19:42:55 -07:00
S1_rate_events feat(ADR-0136.S.1): rate/event statement parsing — capacity + earnings shapes, axis lane 20/20, wrong==0, gsm8k-0014 admits (#201) 2026-05-23 20:36:01 -07:00
__init__.py feat(ADR-0131.G.4): multi-clause composition (conj subjects + conj objects + embedded quantifiers + conj embedded) — admission 0/50 (Δ0), multi-clause refusals 2→1 2026-05-23 14:43:16 -07:00