54e6bfc0d0
Implements the 4-phase documentation reorganization master plan. - Consolidation: Merged brief/, handoff/, planning/, and decisions/ into briefs/, handoffs/, plans/, and adr/ respectively (101 ADRs relocated) - Root Cleanup: Relocated HANDOFF-gpt55-*.md and key top-level docs (runtime_contracts.md, etc.) to canonical folders. Added superseded alerts. - Indices & Navigation: Created docs/README.md navigation document, docs/sessions/README.md index, docs/adr/README.md index - Note: Also includes prior commit adding ADR-0200+ corpus hygiene governance (ADR-0225, dependency map, backfilled cross-references)
3.2 KiB
ADR-0136.S.1 — Rate/Event Statement Parsing
Status: Accepted — regex patterns scheduled for removal under ADR-0164 Phase 3; closed-set vocabulary preserved as lexicon seed Parent: ADR-0136 (Statement Layer Corridor) — see ADR-0136 §Amendment 2026-05-26 Date: 2026-05-23
Context
The GSM8K refusal taxonomy (evals/gsm8k_math/train_sample/v1/refusal_taxonomy.json)
reveals that 23/50 cases are blocked by context-filler sentences (correctly
refused — no parseable numeric state), while 4/50 have rate/capacity/price as
their primary barrier. The remaining cases are compound-statement,
distributive-multiply, and diverse long-tail shapes.
This ADR targets the 4 rate-class cases with two closed statement shapes.
Taxonomy Finding
| Primary barrier | Cases | S.1 scope? |
|---|---|---|
context_filler |
23 | No — correctly refused |
| rate/capacity/price | 4 | Yes |
compound_statement |
5 | No |
distributive_multiply |
1 (+5 secondary) | No |
| diverse long-tail | 17 | No |
Closed Verb Sets
Capacity verbs: shuck, pick, pack, make, produce, type, read, write, paint, run, score, answer, complete (+ third-person -s forms).
Earnings verbs: make, earn, receive, get, charge (+ third-person -s forms).
No regex wildcards for verbs — every admitted verb is explicitly listed in a frozen set. Sentences with verbs outside the closed set are refused (not wrong).
Short-Circuit Rationale
Both rate shapes bypass the Cartesian-product candidate graph because the
rate computation is a direct rate × time multiplication with unit conversion,
not a graph of initial-possessions and operations. The short-circuit runs
before _filtered_statement_choices so that rate-shaped sentences don't
trigger the "no admissible candidate" refusal.
Actor matching is required: capacity questions with pronouns (he/she)
accept any actor; named-actor questions require case-insensitive match.
Mismatched actors produce refusal, not wrong answers.
Honest GSM8K Claim
- Pre-S.1: 0/50 admitted (all refused).
- Post-S.1: 1/50 admitted —
gsm8k-0014(Bob shucks oysters) with answer 240.0 (correct). - admitted_wrong = 0 (safety rail preserved).
The other 3 rate-class cases remain blocked by context-filler sentences in their opening statements; the rate parsing behind them is irrelevant until those sentences parse.
Deferred
- Context-filler gated problems (23 cases — needs semantic classification of narrative scene-setter sentences).
- Conditional branching (overtime rules, e.g. "if she works more than 8 hours").
- Percentage/interest rates (10% simple interest).
- Multi-statement earnings (duration asserted in a separate sentence from the rate — needs general duration-statement parser).
Evidence
- Axis lane:
evals/math_capability_axes/S1_rate_events/v1/— 20/20 pass, wrong=0. - B3 bounded-grammar lane: unchanged (wrong=0).
- GSM8K candidate-graph probe: wrong=0, admitted=1/50.
- Tests:
tests/test_adr_0136_S1_rate_events.py— ≥15 tests including B3 regression guard and GSM8K admitted_wrong=0 rail.