feat(ADR-0136.S.1): rate/event statement parsing — capacity + earnings shapes, axis lane 20/20, wrong==0, gsm8k-0014 admits (#201 )

* docs(ADR-0136.S.0): refusal taxonomy + S.1 brief for rate/event statement corridor

Taxonomy: deterministic classification of all 50 GSM8K train-sample refused cases
into primary + secondary barriers. Key findings:

  context_filler (primary): 23/50 — legitimately refuses; not parser gaps
  compound_statement:         5/50 — two ops in one sentence
  rate/capacity class:        4/50 — direct S.1 targets
  distributive_multiply:      1/50 primary, 5/50 secondary
  long-tail (diverse):       17/50

Honest S.1 ceiling: 0/50 → ≤4/50 admission. gsm8k-0014 ('Bob can shuck 10
oysters in 5 minutes') is the only case with capacity_rate as sole barrier.

Ships:
- evals/gsm8k_math/train_sample/v1/refusal_taxonomy.json (schema v1, 50 records)
- docs/briefs/parallel-2026-05-23/L17-ADR-0136-S1-rate-event-statements.md
- full briefs archive (parallel-2026-05-23)

No implementation changes. Taxonomy and brief only.

* feat(ADR-0136.S.1): rate/event statement parsing — capacity + earnings shapes, axis lane 20/20, wrong==0, gsm8k-0014 admits

Two closed statement shapes added to candidate parser and graph:

Shape A (capacity-rate): "<Actor> can <verb> N <unit> in M <time-unit>"
  - 13 closed verbs (shuck/pick/pack/make/produce/type/read/write/paint/run/score/answer/complete)
  - Pronoun question form (he/she/they/it) accepted
  - Time-unit conversion (second/minute/hour/day)

Shape B (earnings-rate): "<Actor> <verb> $N per/an/a <time-unit>"
  - 5 closed verbs (make/earn/receive/get/charge)
  - Currency: $ only, 0-2 decimal places
  - Per-token alternation: per/a/an/for each/every

Short-circuit paths in parse_and_solve run before the Cartesian product,
computing rate_per_sec × T_seconds directly. Actor mismatch → refusal
(not wrong). Answer ≤ 0 → fall through to refusal.

GSM8K honest delta: 0/50 → 1/50 (gsm8k-0014: answer=240.0, correct).
23 context-filler cases correctly remain refused.
Axis lane: 20/20 pass, wrong=0.
B3 bounded-grammar lane: unchanged (wrong=0).
35 new tests including B3 regression guard and GSM8K admitted_wrong=0 rail.

2026-05-23 20:36:01 -07:00

3 KiB

Raw Blame History

ADR-0136.S.1 — Rate/Event Statement Parsing

Status: Accepted
Parent: ADR-0136 (Statement Layer Corridor)
Date: 2026-05-23

Context

The GSM8K refusal taxonomy (evals/gsm8k_math/train_sample/v1/refusal_taxonomy.json) reveals that 23/50 cases are blocked by context-filler sentences (correctly refused — no parseable numeric state), while 4/50 have rate/capacity/price as their primary barrier. The remaining cases are compound-statement, distributive-multiply, and diverse long-tail shapes.

This ADR targets the 4 rate-class cases with two closed statement shapes.

Taxonomy Finding

Primary barrier	Cases	S.1 scope?
`context_filler`	23	No — correctly refused
rate/capacity/price	4	Yes
`compound_statement`	5	No
`distributive_multiply`	1 (+5 secondary)	No
diverse long-tail	17	No

Closed Verb Sets

Capacity verbs: shuck, pick, pack, make, produce, type, read, write, paint, run, score, answer, complete (+ third-person -s forms).

Earnings verbs: make, earn, receive, get, charge (+ third-person -s forms).

No regex wildcards for verbs — every admitted verb is explicitly listed in a frozen set. Sentences with verbs outside the closed set are refused (not wrong).

Short-Circuit Rationale

Both rate shapes bypass the Cartesian-product candidate graph because the rate computation is a direct rate × time multiplication with unit conversion, not a graph of initial-possessions and operations. The short-circuit runs before _filtered_statement_choices so that rate-shaped sentences don't trigger the "no admissible candidate" refusal.

Actor matching is required: capacity questions with pronouns (he/she) accept any actor; named-actor questions require case-insensitive match. Mismatched actors produce refusal, not wrong answers.

Honest GSM8K Claim

Pre-S.1: 0/50 admitted (all refused).
Post-S.1: 1/50 admitted — gsm8k-0014 (Bob shucks oysters) with answer 240.0 (correct).
admitted_wrong = 0 (safety rail preserved).

The other 3 rate-class cases remain blocked by context-filler sentences in their opening statements; the rate parsing behind them is irrelevant until those sentences parse.

Deferred

Context-filler gated problems (23 cases — needs semantic classification of narrative scene-setter sentences).
Conditional branching (overtime rules, e.g. "if she works more than 8 hours").
Percentage/interest rates (10% simple interest).
Multi-statement earnings (duration asserted in a separate sentence from the rate — needs general duration-statement parser).

Evidence

Axis lane: evals/math_capability_axes/S1_rate_events/v1/ — 20/20 pass, wrong=0.
B3 bounded-grammar lane: unchanged (wrong=0).
GSM8K candidate-graph probe: wrong=0, admitted=1/50.
Tests: tests/test_adr_0136_S1_rate_events.py — ≥15 tests including B3 regression guard and GSM8K admitted_wrong=0 rail.

3 KiB Raw Blame History Unescape Escape